Frequency Governors for Cloud Database OLTP Workloads Rathijit Sen and Alan Halverson Gray Systems Lab Microsoft Corporation

Overview Goal: Maximize power savings while serving the offered load Reduction in COGS Meet customer SLOs Mechanism: Per-core frequency control Decision metric: Effective Utilization Allows more aggressive operations than traditional utilization New reactive governor leverages this metric Study Microsoft SQL Server on Linux Transactional cloud workload

Outline Cloud database workload overview Existing frequency governors Reactive governor motivation and design Experiments Conclusion

Cloud Workload: Schema Six tables Fixed size: 2 Scaling: 3 Growing: 1 Data types: integer, numeric, character, date/time Constraints: primary & secondary keys, no foreign key Synthetic data: distributions, permutations of sets of values, weighted lists of words, etc. Designed to support a broad range of operations

Cloud Workload: Transaction Mix Read Lite SELECT; in-memory; read-only Read Medium SELECT; mostly in-memory; read-only Read Heavy SELECT; mostly not in-memory; read-only Update Lite UPDATE; in-memory; read-write Update Heavy UPDATE; mostly not in-memory; read-write Insert Lite INSERT; in-memory; read-write Insert Heavy INSERT; mostly not in-memory; read-write Delete DELETE; mix of in-memory and not in-memory; read-write CPU Heavy SELECT; in-memory; relatively heavy CPU load; read-only

Cloud Workload: Pacing Delay (D) “think time” between transactions Negative exponential distribution Capped at 10x the mean Mean: Pacing Delay (D) TPS increases as D decreases We consider D=1, 0.5, 0.2, 0.1, 0.05, 0.04, 0 Other values could be chosen

Linux Frequency Governors Intel P-state Performance: static, highest available frequency Powersave: dynamic, load-dependent Cpufreq Ondemand: dynamic , load-dependent Conservative: dynamic , load-dependent Powersave: static, lowest frequency Userspace: up to the administrator Reactive: dynamic, load-dependent operates with Cpufreq Userspace enabled

Design Constraint: No Application Knowledge Host Server VM Docker App What application is running? Is it transactional? When do transactions start/end? How many queries/responses per transaction? Which guest thread does this request correspond to? …

Reactive Governor Design Goal: react to load, but use minimum power Constraint: only host server (and host OS) counters available %C0 is not enough! Other idle states such as C1, C3 may be occupied even under full load Write-ahead logging Network processing Scheduling delays … %C0 (and performance) can decrease if frequency is erroneously lowered

Example execution at max. frequency

Example execution at max. frequency

Example execution at max. frequency

Example execution at max. frequency

Reactive Governor Heuristics/Assumptions %C0 scales inversely with frequency Deviations in practice due to memory stalls %C6 decreases as load increases or as frequency decreases %C0 has opposite behavior %C6 may be fully eliminated %C1 + … +%C5 may not change with frequency at a fixed load As long as system is not over-committed Deviations in practice, e.g., due to kernel processing

Effective Utilization

Reactive Governor Operations eff > Threshold? New frequency = eff x Current frequency Increase current frequency by , subject to some cap N Y repeat Parameter choices can tradeoff power savings for performance Targeted for 10 ms intervals, invoked about 70 times per second

Experimental Setup Client-Server setup, transactional cloud workload E5-2620 v4 (Broadwell Xeon) Dual socket 8 cores/socket x 2 (hyperthreading enabled) 1.2 GHz – 2.1 GHz Turbo: upto 3.0 GHz, all cores: 2.3 GHz Per-core frequency control 64GB DDR4 2 x 512 GB SSD BIOS setting favors energy efficient operations Microsoft SQL Server on Linux (Ubuntu 16.04.1 LTS), CTP 1.1

Power-Performance: Intel P-state & Reactive Approx. 30W savings at D=0.1 over P-state Performance Approx. 20W savings at D=0.1 over P-state Powersave

Power-Performance: Cpufreq & Reactive Reactive most power-efficient among governors that serve full load Approx. 7% performance loss at D=0.05, 0.04

Service Level Objectives (SLOs) Dependent on Class of Service Resource allocations also vary Premium 95th percentile ≤ 0.5 second Standard 90th percentile ≤ 1.0 second Basic 80th percentile ≤ 2.0 second

Conclusion Hosting database workloads at the scale of the cloud brings need to manage both COGS and customer SLOs We propose a reactive per-core frequency governor to reduce power and power-related costs while meeting SLOs Key concept is effective utilization eff, with frequency changes based on this metric