1. Dynamic Classification of Repetitive Jobs in Linux For Energy-Aware Scheduling: A Feasibility Study
Shane Case and Kanad Ghose
Dept. of Computer Science
State University of New York (SUNY) at Binghamton

2. The Nature of the Problem: Raw Numbers for Data Centers
Lifecycle: 90% of energy expended during the operations
Breakdown of operating costs:
Equipment energy consumption: 45%
HVAC: 50% (about 25% to 30% in water chilling facilities)
Remainder: 5%
Breakdown inside a modern server:
CPU energy: 20% to 25%
Memory energy: 30% to 45%
Interconnection energy: 10% to 15%
Total energy consumptions:
2.8% of total national electricity consumption in US
1.6% for Western Europe

3. The Nature of the Problem (contd.)
Server overprovisioning is a common practice
Situation becoming worse with smaller form factors
100%
Performance is not proportional to power
Poor energy efficiency (performance per Watt) on average
Power
Energy
Efficiency
Typical Operating Region 0%
Individual Server Utilization
100%

4. The Nature of the Problem (contd.)
HVAC equipment typically use water chillers to supply cold air: takes a long time to adjust cooling system output (water has a very high thermal capacity)
Overprovisioning is common here as well to permit handling of sudden load surges
Using sensed temperature to adjust cooling system performance is a viable first step, but not enough
Solutions for improving the energy efficiency of data centers have been fairly isolated
Examples:
Improving CPU energy efficiency in isolation
DDR-3 energy management, green drives
Improved packaging, cooling system design
System-level, holistic solutions are a MUST

5. Key Enablers of Our Technique
Many server jobs are repetitive in nature
Similar CPU usages
Similar I/O activities 
In a busy server, similar requests arrive in quick succession
For many server requests, a small increase in the service latency is tolerable 
Energy usage of servers and response times improve with reduction in numbers of changes made to the DVFS setting (energy-performance gear)
12/8/2018
E3S BRIEFING

6. The Proposed Technique
Reduce server energy dissipation by exploiting repetitive nature of requests
Fix frequency-voltage gear setting of cores to match job characteristics instead of discovering matching gear settings on-the-fly
Schedule previously classified similar jobs with common gear setting
Sources of energy savings:
Frequency-voltage setting matched to job from the beginning
No need for changing gear settings as jobs execute
12/8/2018
E3S BRIEFING

7. Background: DVFS and Governors
Modern CPUs allow the OS to adjust the DVFS (“speedstep”) settings of sockets
Clock frequency and voltage dropped simultaneously to save energy when CPU utilization is low
Clock frequency and voltage raised simultaneously to provide performance when CPU utilization is high
Software module that performs this functions is called a governor
Most current CPU chips have a common DVFS setting for all of its cores, but chips in different socket can have different DVFS settings
Emerging CPU chips: per-core DVFS
12/8/2018
E3S BRIEFING

8. Overview of Technique: A Few Details
Observe CPU and I/O usages of jobs
Classify jobs as CPU-bound or IO bound 
Delay services slightly to bunch together jobs of same class 
Run queued-up jobs of same class back-to-back (“batch mode”) with common DVFS setting
CPU-bound jobs on socket with high frequency setting
IO bound jobs on sockets with low frequency setting
Unclassified jobs run on socket with high-frequency setting to avoid performance loss
Also works with emerging CPUs that support per-core DVFS
12/8/2018
E3S BRIEFING

9. Implementation Overview
Extensions to Apache server using worker for Linux platforms
Uses Apache’s forensic log
Forensic log stored on RAM file system to avoid any slowdown
Job classifications, indexed by name of executable file also stored on RAM file system
CPU usage and IO stats for job obtained from OS maintained stats
Hash table used for fast lookup of classifications by requested binary module name
ACPI interface used for controlling DVFS setting
12/8/2018
E3S BRIEFING

10. Main Modules Used in the Prototype
Scheduling Daemon
Classify transactions
Signal children in batches
Manage CPU frequencies
Set process CPU affinity
Core Kernel Scheduler
Change the frequency/voltage settings of socket/core
Change process affinity in scheduler
Apache
Receive HTTP requests
Child process read stats
CPU Affinity & Frequency Changes
Stats
“Continue” Signal
Child Process Statistics
12/8/2018
E3S BRIEFING

11. Apache Server: Pre-Forking Model
This approach will focus on the pre-forking model of the web server: N (child) processes are forked and wait for tasks (=requests) to be serviced
N-1 workers wait in an idle queue, while one “listens” for request
When a request arrives, the listener becomes a worker
After servicing the request, the worker rejoins the idle queue
12/8/2018
E3S BRIEFING

12. Transactions and the Forensic Log
A request for service is viewed as a transaction
When forensic logging is enabled, the transaction’s start and stop time are recorded
Each worker that serves a transaction is a process and the operating system kernel records it resource usage in the file /proc/$PID/schedstat:
CPU usage
IO wait time, IO Bytes etc.
Modified forensic log: uses Signals as IPC to communicate with scheduling daemon
Must have knowledge of daemon PID
Effective UID
Maintain counters to send to daemon
12/8/2018
E3S BRIEFING

13. The Scheduling Daemon Signal sent to daemon upon transaction start
Apache server sleeps awaiting signal to proceed
Daemon performs table lookup
Daemon looks up classification database
If request has been classified, schedule task to socket that has the appropriate DVFS setting
If job is not classified, job is scheduled on the core with the faster DVFS setting and its recorded statistics will be added to the classification database when the transaction completes
Daemon signals the server to continue
12/8/2018
E3S BRIEFING

14. Queuing Transactions for Back-to-back Execution
Transactions are forced to sleep – that is, wait in the queue to permit other transactions of similar classes to arrive
Queue size is reconfigurable
Wait time on queue is limited: timeout on waiting is enforced to avoid performance losses
Time-critical transactions do not wait
12/8/2018
E3S BRIEFING

15. Other scheduler Daemon Functions
Maintain resource usage function
Maintain small classification table
Age entries out
Revise classifications if need be
Implement hash-based table lookup
Interact with core kernel scheduler:
Dictate socket/core pinning
Adjust DVFS settings for sockets/cores
12/8/2018
E3S BRIEFING

16. Experimental Assessments
Apache servers on Linux, Dell 2U servers
Three DVFS control mechanisms evaluated:
On-demand governor (described earlier)
“Performance” governor (highest DVFS setting when any core in the socket is used)
The proposed scheme
SPECweb2005 benchmark used to generate load:
Banking, e-commerce and support workloads
100, 150 and 200 concurrent connections
12/8/2018
E3S BRIEFING

17. Measurement Setup Measured DC power going to CPU
Data-logging power meter
Data from power log synched with CPU activity to validate implementation
CPU level power savings reported
System level power also recorded
12/8/2018
E3S BRIEFING

18. DC Power Consumption: Banking
12/8/2018
E3S BRIEFING

19. CPU Power Savings
12/8/2018
E3S BRIEFING

20. Energy and Performance
12/8/2018
E3S BRIEFING

21. Conclusions
Classifying repetitive jobs can result in energy and power savings for servers
Proposed technique achieves 10% power savings compared to other governors for the Linux kernel
Performance impact is < 1%
Impact on total server level power savings lower at this time (about 5%) but emerging DDR-3 adaptive power management support, working in conjunction with the proposed scheme can reduce server-level energy consumption by an estimated 8% to 12%
5% energy savings at the server level translates to 7.5% to 10% total data center level power savings considering savings in cooling cost
Kernel-level classification has the potential for increasing the energy savings further.
12/8/2018
E3S BRIEFING