Energy Efficient Operations: Some Challenges and Opportunities

Yet more notes from Velocity.

After the break, the next session is Energy Efficient Operations: Some Challenges and Opportunities. Luiz Barroso from Google is the presenter. I got a couple minutes late as I had to pick the charger.



Server electricity usage in perspective:

• worldwide electricity usage of servers is around 1% of total electricity consumption.
• usage doubled between 2000 and 2005
• could increase by 40%-76% by 2010.
  

PC enery consumption likely higher:

• installed base for servers in 2005 - 27M
• installed base for PCs in 2005: 870M
  

Measuring computing energy efficiency

• harder for computers than for refrigerators
• efficiency = work done / energy used = computing speed / power
• biggest thing you can do for energy efficiency is write fast code. it can have really big impact.
• from measurement standpoint, it is useful to break down the energy efficiency/budget equation
• breaking it down:
• efficiency = (work done / energy used in chips) * (energy used in chips / energy provided to servers) * (energy provided to servers / energy entering the building)
• first: computing efficiency
• second: server efficiency
• third: datacenter efficiency or 1/PUE (power usage efficiency)
  

Energy efficiency opportunities:

• datacenter energy efficiency
• LBNL survey of 24 facilities shows avg PUE of 1.83
• underutilized data centers
• wasted power provisioning investment
• makes cooling and power distribution less efficient
• server energy efficiency
• typical server power supplies dissipate 25% of total energy
• DC-to-DC voltage regulatorscan lose another 25%
• computing efficiency
• servers have poor energy efficiency in their most common usage range
  

Plan for today:

• datacenter efficiency
• the power provisioning efficiency: What can you achieve if you utilize all energy in your data center.
• two key energy related costs:
• 10 year energy costs ($9/watt)
• cost of building a datacenter ($10-22/watt)
• Facility costs are as important as energy consumption costs
  

TCO components: Rough cost breakdown: datacenter (28%) hardware (50%) energy (22%)

Datacenter buildout can be larger than energy itself.

Efficiency provisioning playbook:

• consolidate workloads into the minimum number of machines needed for peak usage requirements
• smart scheduling or virtualization help here
• measure actual power usage of devices
• nameplates lie!
• study activity trends and investigate the oversubscription potential
• the subject of our ISCA 07 article
  

Six month power monitoring study at Google (ISCA 07)

• Basic setup
• model based power monitoring scheme
• measure usage statistics at rack, PDU and cluster levels
• 4 diferent workloads over 5k servers
  

More servers leads to higher oversubscription potential.

Safely oversubscribing power

• oversubscribe at the datacenter level, not of at server or rack levels
• profile power usage of applications: learn what to expect
• mix workloads
• manage overload
• provision a sizeable 'best effort' workload; victimize it first
• use applications with QoS stack
• good news: time constants to react are long
  

Energy-proportional computing: (An article was published in december of last year)

• look at datacenter as a device you have to lower power for
• he calls the datacenter: a land-held
• CPU activity distribution over six months (graph)
• real production systems don't run full blast all the time.
• systems run 10% to 50% of their full capacity most of the time.
• fraction of time these servers are doing nothing is very small.
• A datacenter and a laptop are indeed different
  

Characteristics of well designed internet services:

• high performance and high availability requires
• load balancing and wide data distribution -> no useful idle intervals, lots of low activity intervals
• example: Google file system:
• replicas distributed across multiple machines
• reads load balancing across replicas, writes need to reach all.
  

Key implications:

• sleep or power-down strategies are much less useful in servers
• focus on energy efficiency at peak performance is misguided
  

Power varies with amount of activity in servers. When a machine is completely idle, it still pretty much uses half of peak power it consumes. At 1/3 of peak, power efficiency is halved.

Energy-proportional computing: (the idea)

• no work, no power consumed
• some work, some power consumed
• lots of work, lots of power consumed
  

That would be the end of power management software.

What if we could build machines with a wide activity range? He shows a graph.

Estimated impact of energy proportionality is quite huge based on another graph.

Conclusion:

• write fast code!
• the software engineer's biggest contribution to energy efficiency
• consider reduction of all energy-related costs
• electricity, and datacenter provisioning
  

Some Google initiatives

• carbon neutrality
• 1.6MW solar panel installation in Mtn. View
• plugin-in hybrids (http://rechargeit.org)
  

One of the best presentations at Velocity.

More publications by Luiz:

• The Case for Energy-Proportional Computing
• Power Provisioning for a Warehouse-sized Computer
• Failure Trends in a Large Disk Drive Population
• The Price of Performance: An Economic Case for Chip Multiprocessing
• Web Search for A Planet: The Architecture of the Google Cluster