ServerKarmicCloudPowerManagement

Summary

In Jaunty, we proved that at least some Ubuntu servers can successfully, suspend and hibernate, as well as resume from remote triggers such as wake-on-lan.

In Karmic, we plan to use this technology to make Ubuntu Enterprise Clouds more energy efficient than other cloud solutions, by ensuring that a minimum number of cloud nodes are powered up at any given time.

This specification divides Cloud Power Management into 3 distinct parts:

  1. Putting nodes to sleep (powernap)
  2. Waking nodes up (powerwake)
  3. Compressing the cloud (eucalyptus)

Release Note

Ubuntu 9.10 Enterprise Cloud deployments can leverage dynamic power management facilities. Unused nodes are suspended/hibernated/powered-down, and are resumed only when necessary, yielding a more energy efficient data center.

Rationale

Cloud Computing offers versatile, scalable resources in the form of virtual machines scattered across server hardware. In the interest of energy efficiency, a minimum amount of server hardware should be powered on and running in an Ubuntu Enterprise Cloud.

User stories

  1. Kim is the administrator of a cloud which is often under utilized. She's also tasked with making her data center as energy efficient as possible. Ideally, each server in her entire cloud would run in a low-power state (suspended, hibernated, or powered off), and only wake when necessary.

Assumptions

  1. The suspend/hibernate/resume functionality should only be enabled on server hardware that supports such power management features. Power-on/power-off will need to be used otherwise. This can be determined by examining /proc/acpi/sleep and querying pm-is-supported.

  2. A mechanism for remotely waking systems is necessary, and may require support in the BIOS or auxiliary hardware. Wake-on-lan is the simplest remote-wake mechanism. We will focus on this in the short term.

Design

In Karmic, we plan to integrate into existing cloud management frameworks:

  1. A configurable utility installed on cloud nodes (or any Ubuntu server, really) that takes a prescribed action (such as lowering the power state) when certain conditions are met (such as not running some specific processes for some number of seconds)

    • powernap - in Karmic universe as of 2009-06-15

    • Promoted to Karmic main in 2009-07
  2. A utility installed on a cloud controller (or any router/server, really) that can wake sleeping systems according to a method configured for the target system (wakeonlan, ipmi, nut, etc)
    • should be another binary package (powerwake?) under the powernap source package
    • Eucalyptus (and others) will need to be modified to use powerwake when a target system appears offline
    • In Karmic main in 2009-07
  3. A cloud controller with a node selection algorithm ensuring that new virtual machines are deployed onto running systems with spare capacity first, and new systems are only powered-up when need arises
    • Eucalyptus already has an algorithm that sort of does this, "greedy scheduling"; it simply needs to be enhanced to give priority to running systems

Implementation

PowerNap

  • Initial code released, in Karmic
  • MIR (Done)
  • Additional features would be nice
    • powernap now (Done)
    • send warning message to logged in users (Done)
    • email authorities (To Do)
    • logging to /var/log/powernap.log (To Do)

PowerWake

  • Separate binary package under powernap sources (Done)
  • Depend on waking technologies (wakeonlan (Done), openipmi, nut)
  • Define a configuration file/database where target host information is stored (Done)
  • Use the appropriate waking mechanism per host, log appropriately, and notify caller as to success/failure of waking

Eucalyptus

There are three pieces to the Eucalyptus integration:

  1. Integration of powerwake, hooking eucalyptus' cloud controller code where unresponsive-host exception handling takes place -- should be very doable once powerwake is implemented.
    • Redesigned -- Eucalyptus itself maintains state in its database and directly calls "powernap now" when it deems a system ready to sleep, or "powerwake" when it needs to wake a system. (Done)
  2. Tweaking Eucalyptus' "greedy" node scheduling algorithm to give preference to powered-on machines first -- should be doable (Done)
  3. Implementing a compression/refactoring/defragmentation algorithm. Eucalyptus team says that this is not a 1.6 milestone goal at this point. This is a complex problem that they have begun thinking about, however, they have concerns about being able to handle this in a hypervisor-agnostic and a persistent-storage-agnostic manner. More discussion needed. (This is deferred from Karmic due to missing dependencies and low cost/value ratio)

Test/Demo Plan

Attach a watt-meter in series with the power drawn for some set of cloud hardware. Ensure that no virtual machines are running in the cloud. Observe the unused nodes enter lower power states, as configured. Observe a minimum value of wattage drawn by the cloud. Request new VM's from the cloud controller. Observe nodes waking and serving the newly requested virtual machines, and the increase in power drawn. Destroy virtual machines, and observe nodes powering down and power consumption decreasing.

Unresolved issues

  • Persistent shared storage for live migration (Deferred)

UDS Raw Notes

  • Machines migrate when there is lite load.
  • For Eucalyptus
    • Boot nodes into suspended state.
    • Wake up as needed.
  • Use live migration to compress a cloud into the minimum amount of hardware.
    • Suspend unneeded hardware.
  • pm-suspend and pm-hibernate are now in server seed.
  • Re-establishing network connectivity after resume may be an issue.
  • Manual process of compressing the cloud hardware.
    • SSH to send and wakeonlan to resume.
  • With new improvements to boot speed actually powering off may be useful.
  • Hibernate may be preferable when reloading a cache.
  • Support poweroff, hibernate and suspend depending on the admin preference.
  • Support IPMI as well as wakeonlan.
  • Make framework configurable as to what tools to use.
  • Do a wakeonlan testing day to generate a list of working hardware.
  • Wakeonlan
    • Simple
    • Doesn't scale
  • IPMI
    • Server hardware required.
    • Authentication supported
  • NUT
    • Network UPS Tool
    • Can also wake systems.
  • Use Cases:
    • Manually compress cloud hardware.
    • Process to actively monitor cloud load and adjust hardware according to load.
  • Use libvirt for VM migration.
  • Eucalyptus integration
    • Power Management needs to live in the same space as the service level agreement.
  • Integrate power management into Landscape.
    • Set SLAs in Landscape which then uses the plugins for power management.
  • Use application to monitor power consumption.
  • May have to target specific hardware.
  • Strongly consider adding power management hooks to libvirt.
    • Could automatically wake up a machine when connecting.
  • What is the appropriate amount of idle time before suspending a system.
  • Allow configuration of the amount of hot nodes.
  • Adjust cloud load based on node load.

CategorySpec

ServerKarmicCloudPowerManagement (last edited 2009-07-28 22:24:12 by cpe-66-69-254-183)