Dynamic Classification of Repetitive Jobs in Linux For Energy-Aware Scheduling: A Feasibility Study Shane Case and Kanad Ghose Dept. of Computer Science State University of New York (SUNY) at Binghamton

The Nature of the Problem: Raw Numbers for Data Centers Lifecycle: 90% of energy expended during the operations Breakdown of operating costs: Equipment energy consumption: 45% HVAC: 50% (about 25% to 30% in water chilling facilities) Remainder: 5% Breakdown inside a modern server: CPU energy: 20% to 25% Memory energy: 30% to 45% Interconnection energy: 10% to 15% Total energy consumptions: 2.8% of total national electricity consumption in US 1.6% for Western Europe

The Nature of the Problem (contd.) Server overprovisioning is a common practice Situation becoming worse with smaller form factors 100% Performance is not proportional to power Poor energy efficiency (performance per Watt) on average Power Energy Efficiency Typical Operating Region 0% Individual Server Utilization 100%

The Nature of the Problem (contd.) HVAC equipment typically use water chillers to supply cold air: takes a long time to adjust cooling system output (water has a very high thermal capacity) Overprovisioning is common here as well to permit handling of sudden load surges Using sensed temperature to adjust cooling system performance is a viable first step, but not enough Solutions for improving the energy efficiency of data centers have been fairly isolated Examples: Improving CPU energy efficiency in isolation DDR-3 energy management, green drives Improved packaging, cooling system design System-level, holistic solutions are a MUST

Key Enablers of Our Technique Many server jobs are repetitive in nature Similar CPU usages Similar I/O activities  In a busy server, similar requests arrive in quick succession For many server requests, a small increase in the service latency is tolerable  Energy usage of servers and response times improve with reduction in numbers of changes made to the DVFS setting (energy-performance gear) 12/8/2018 E3S BRIEFING

The Proposed Technique Reduce server energy dissipation by exploiting repetitive nature of requests Fix frequency-voltage gear setting of cores to match job characteristics instead of discovering matching gear settings on-the-fly Schedule previously classified similar jobs with common gear setting Sources of energy savings: Frequency-voltage setting matched to job from the beginning No need for changing gear settings as jobs execute 12/8/2018 E3S BRIEFING

Background: DVFS and Governors Modern CPUs allow the OS to adjust the DVFS (“speedstep”) settings of sockets Clock frequency and voltage dropped simultaneously to save energy when CPU utilization is low Clock frequency and voltage raised simultaneously to provide performance when CPU utilization is high Software module that performs this functions is called a governor Most current CPU chips have a common DVFS setting for all of its cores, but chips in different socket can have different DVFS settings Emerging CPU chips: per-core DVFS 12/8/2018 E3S BRIEFING

Overview of Technique: A Few Details Observe CPU and I/O usages of jobs Classify jobs as CPU-bound or IO bound  Delay services slightly to bunch together jobs of same class  Run queued-up jobs of same class back-to-back (“batch mode”) with common DVFS setting CPU-bound jobs on socket with high frequency setting IO bound jobs on sockets with low frequency setting Unclassified jobs run on socket with high-frequency setting to avoid performance loss Also works with emerging CPUs that support per-core DVFS 12/8/2018 E3S BRIEFING

Implementation Overview Extensions to Apache server using worker for Linux platforms Uses Apache’s forensic log Forensic log stored on RAM file system to avoid any slowdown Job classifications, indexed by name of executable file also stored on RAM file system CPU usage and IO stats for job obtained from OS maintained stats Hash table used for fast lookup of classifications by requested binary module name ACPI interface used for controlling DVFS setting 12/8/2018 E3S BRIEFING

Main Modules Used in the Prototype Scheduling Daemon Classify transactions Signal children in batches Manage CPU frequencies Set process CPU affinity Core Kernel Scheduler Change the frequency/voltage settings of socket/core Change process affinity in scheduler Apache Receive HTTP requests Child process read stats CPU Affinity & Frequency Changes Stats “Continue” Signal Child Process Statistics 12/8/2018 E3S BRIEFING

Apache Server: Pre-Forking Model This approach will focus on the pre-forking model of the web server: N (child) processes are forked and wait for tasks (=requests) to be serviced N-1 workers wait in an idle queue, while one “listens” for request When a request arrives, the listener becomes a worker After servicing the request, the worker rejoins the idle queue 12/8/2018 E3S BRIEFING

Transactions and the Forensic Log A request for service is viewed as a transaction When forensic logging is enabled, the transaction’s start and stop time are recorded Each worker that serves a transaction is a process and the operating system kernel records it resource usage in the file /proc/$PID/schedstat: CPU usage IO wait time, IO Bytes etc. Modified forensic log: uses Signals as IPC to communicate with scheduling daemon Must have knowledge of daemon PID Effective UID Maintain counters to send to daemon 12/8/2018 E3S BRIEFING

The Scheduling Daemon Signal sent to daemon upon transaction start Apache server sleeps awaiting signal to proceed Daemon performs table lookup Daemon looks up classification database If request has been classified, schedule task to socket that has the appropriate DVFS setting If job is not classified, job is scheduled on the core with the faster DVFS setting and its recorded statistics will be added to the classification database when the transaction completes Daemon signals the server to continue 12/8/2018 E3S BRIEFING

Queuing Transactions for Back-to-back Execution Transactions are forced to sleep – that is, wait in the queue to permit other transactions of similar classes to arrive Queue size is reconfigurable Wait time on queue is limited: timeout on waiting is enforced to avoid performance losses Time-critical transactions do not wait 12/8/2018 E3S BRIEFING

Other scheduler Daemon Functions Maintain resource usage function Maintain small classification table Age entries out Revise classifications if need be Implement hash-based table lookup Interact with core kernel scheduler: Dictate socket/core pinning Adjust DVFS settings for sockets/cores 12/8/2018 E3S BRIEFING

Experimental Assessments Apache 2.2.11 servers on Linux, Dell 2U servers Three DVFS control mechanisms evaluated: On-demand governor (described earlier) “Performance” governor (highest DVFS setting when any core in the socket is used) The proposed scheme SPECweb2005 benchmark used to generate load: Banking, e-commerce and support workloads 100, 150 and 200 concurrent connections 12/8/2018 E3S BRIEFING

Measurement Setup Measured DC power going to CPU Data-logging power meter Data from power log synched with CPU activity to validate implementation CPU level power savings reported System level power also recorded 12/8/2018 E3S BRIEFING

DC Power Consumption: Banking 12/8/2018 E3S BRIEFING

CPU Power Savings 12/8/2018 E3S BRIEFING

Energy and Performance 12/8/2018 E3S BRIEFING

Conclusions Classifying repetitive jobs can result in energy and power savings for servers Proposed technique achieves 10% power savings compared to other governors for the Linux kernel Performance impact is < 1% Impact on total server level power savings lower at this time (about 5%) but emerging DDR-3 adaptive power management support, working in conjunction with the proposed scheme can reduce server-level energy consumption by an estimated 8% to 12% 5% energy savings at the server level translates to 7.5% to 10% total data center level power savings considering savings in cooling cost Kernel-level classification has the potential for increasing the energy savings further. 12/8/2018 E3S BRIEFING