Matthew Garrett <mjg@redhat.com> Going green with Linux Matthew Garrett <mjg@redhat.com>

Data centres are expensive ~3% of total US electrical consumption in 2010 ~15GW ~15 nuclear power stations

Fixing infrastructure On average, 1W of overhead for 1W of machine More efficient cooling systems More efficient power supplies

Things are getting better! 0.2W of overhead per W of machine

Flying car future Use waste heat to generate power

Linux doesn't solve this (Sorry)

Things need to get a lot better... High end hardware takes a lot of power 48 core system = ~350W at idle (with power management...)

What can we do in software? CPU power management dominates right now Modern x86 idles at ~0W Full power = ~20W per core (48 core system = 960W from CPUs alone!)

Gettings CPUs to do less is key An idle CPU is a cheap CPU Full package sleep means idle memory ~5W per stick of DDR3 at load, ~0.1W in self refresh Optimising CPU and memory use patterns makes a huge difference

Idle CPU cores = idle CPU package RAM

One active core = active package RAM

Keeping idle is key

Keeping idle is key

Keeping idle is key

Huge efforts to reduce wakeups IPMI reworked Automounter moved to event driven model Application-level tuning Debug tools (powertop, systemtap)

What are the numbers? At fully idle, Linux ~3% better than Windows Things start getting interesting under load...

(Illustrative results)

(Illustrative results)

Take home message From 10%-50% load, Linux significantly more power efficient Margin decreases up to 90%, equality reached at 100%

Tweaking Linux behaviour /sys/devices/system/cpu/sched_mc_power_savings

RAM RAM

One task RAM RAM

Two tasks (sched_mc_power_savings = 0) RAM RAM

Two tasks (sched_mc_power_savings = 1) RAM RAM

Two tasks (sched_mc_power_savings = 2) Process migrated RAM RAM

Future trends Smarter scheduling More runtime memory power management Increasing virtualisation as a power management strategy Power management of smaller system components

Scheduling NUMA-aware migration Consolidate active pages in smaller nodes Thermal-aware scheduling If a CPU is limited because of heat, don't use it

Virtualisation Run virtualised rather than on real hardware Consolidate guests on a smaller number of hosts Bring up hosts and migrate guests as demand increases (requires rapid guest migration)

Load balancing Put backend machines to sleep Wake when demand increases Reduced resume latency

What are the tradeoffs? Focus has been on “free” power management Is it really true?

Well... Enterprise customers have enterprise demands Sometimes that performance hit really does matter

Latency can be a pain C-states inherently create latency Some people really don't want latency There's an obvious conflict here

pm_qos Simple solution is to let applications indicate their requirement pm_qos interface provides that

Performance/power tradeoff Not always the direction you'd think Intel's turbo mode depends on good idle PM (Benchmarks in the snow)

Turning more off Marginal but measurable gains from caring about unused hardware Lots of machines have USB ports Many don't have anything in them...

Genuinely user transparent USB SD Firewire Audio?

Focus on power management Reducing power consumption reduces costs

Focus on power management Reducing power consumption reduces costs Reducing power consumption is good PR

Focus on power management Reducing power consumption reduces costs Reducing power consumption is good PR Reducing power consumption is common sense

Questions?