1. Application Heartbeats Henry Hoffmann, Jonathan Eastep, Marco Santambrogio, Jason Miller, Anant Agarwal CSAIL Massachusetts Institute of Technology Cambridge, MA 02139

2. Outline Introduction/Motivation –Problem –Related Work –Solution: Standard interface for expresses performance/goals Application Heartbeats –Idea –Interface Experiments –Heartbeat use within an application –Heartbeat use by an external system –Other systems using Heartbeats Conclusion –Request for feedback/usage –Summary

3. As System Complexity Increases, Self-Tuning Systems Emerge System Complexity is Skyrocketing –Multicore processors –Heterogeneous architectures –Distributed, deep memory hierarchies –Special-purpose functional units –Unreliable components –New constraints: power, energy, wire delay Solution: Self-Tuning Systems Systems observe their runtime behavior, learn, and take actions to meet desired goals

4. Related Work Software techniques –Tracing kernel performance: K42, KernInst –Profiling frameworks: Oprofile, gprof Mostly focused on off-line collection of performance data Not able to support self-* frameworks to take online decisions –Assertion based mechanisms Assertion used to verify if runtime meets expected performance Extensive code annotation and only internal updates to itself Hardware techniques –Hardware-based metrics: cache miss rate –Hardware assistance: counters (PAPI) Largely ad-hoc approaches

5. Self-tuning Systems Must Monitor The Apps They Support Disk I/O Devices DRAM App 2 App 1 App 3 miss rate voltage, freq, precision cache size, associativity power IPC, power, temp App 1 Core Cache App 2App 3 Core Cache speed Application Layer Self-Tuning Services Layer We propose Application Heartbeats as a standard API for applications to specify their goals and performance to self-tuning system services Scheduler, Memory manager, file system Operating System Currently, applications run as performance black-boxes:

6. Outline Introduction/Motivation –Problem –Related Work –Solution: Standard interface for expresses performance/goals Application Heartbeats –Idea –Interface Experiments –Heartbeat use within an application –Heartbeat use by an external system –Other systems using Heartbeats Conclusion –Request for feedback/usage –Summary

7. Application Heartbeats Provide Standard API Applications Use to Communicate With Self-tuning Services Application Heartbeats allow apps to express goals and current performance System software can use Heartbeats to directly measure performance Application Heartbeats Disk I/O Devices DRAM App 2 App 1 App 3 miss rate voltage, freq, precision cache size, associativity power activity, power, temp App 1 Core Cache App 2 App 3 Core Cache speed heartbeat, goals Heartbeat App 1 Min heart rate = 10 Max heart rate = 100 Current heart rate = 75 App 2 Min heart rate = 29.5 Max heart rate = 30 Current heart rate = 29.8 App 3 Min heart rate = 0.5 Max heart rate = 1.5 Current heart rate =.2 Scheduler, Memory manager, file system Operating System Apps no longer performance black-boxes

8. Heartbeat API Functions FunctionParametersDescription heartbeat_initialize[int] window_size Initialize the heartbeat object to collect heartbeats. Uses a sliding window of window_size to calculate current hear trate heartbeat[int] tag Records a heartbeat with a given tag hb_get_current_rate Returns the current heart rate averaged over the last window_size heartbeats hb_set_target_rate[float] min, [float] max Sets the desired min and max heart rates for this app hb_get_target_min_rate Returns the minimum desired heart rate hb_get_target_max_rate Returns the maximum desired heart rate hb_set_target_latency[float] min, [float] max, [int] tag1, [int] tag2 Sets the desired latency between heartbeats with tags tag1 and tag2 hb_get_min_latency[int] tag1, [int] tag2 Returns the minimum desired latency between two tags hb_get_max_latency[int] tag1, [int] tag2 Returns the maximum desired latency between two tags hb_get_history[int] n Returns all heartbeat information for the last n heartbeats

9. Outline Introduction/Motivation –Problem –Related Work –Solution: Standard interface for expresses performance/goals Application Heartbeats –Idea –Interface Experiments –Heartbeat use within an application –Heartbeat use by an external system –Other systems using Heartbeats Conclusion –Request for feedback/usage –Summary

10. Experiment 1: Internal Heartbeat Usage Experiment 1: Adaptive H.264 Encoder Goal: produce the highest quality video in real-time –Highest quality parameters = slow encoder, even on an 8-core Intel x86slow –Tuning the parameters is tedious - have the encoder do it for us Method: –A heartbeat is registered for each frame (frame rate = heart rate) Target heart rate is 30 frames/second –Encoder reads heartbeat and changes algorithm to reach target Start the encoder with most aggressive parameters Let the encoder find the parameters that meet performance needs Results: –Now the encoder is fast and still high qualityfast –Achieve target performance with barely visible quality loss –You can see the effect in the videos, but we also measure performance and quality

11. Example 1: Performance 0 5 10 15 20 25 30 35 0100200300400500600 Time (Frame Number) Heart Rate (Frames/s) Adaptive Encoder Target Heart Rate esa search umh search dia search eliminated I4x4 mode in I-frames eliminated I4x4 mode in P-frames eliminated P8x8 mode in P-frames eliminated sub-16x16 modes in P-frames eliminated rate-distortion optimizations reduced sub-pixel search

12. Example 1: Image Quality

13. Example 2: External Heartbeat Usage Experiment 2: External System Reads Heart Rate and Assigns Cores Goal: Assign cores to keep performance within target range –Assigning too few cores leads to poor performance –Assigning too many cores wastes resources and power Method: Use PARSEC benchmarks –Application registers heartbeat at regular intervals –Target heart rates set to be achievable using less than full number of cores –External scheduler assigns: More cores if heart rate is below target Less cores if heart rate is above target Does nothing if heart rate is in right zone Results: –The scheduler keeps the applications running at the target speed –Scheduler can adapt to changes in the difficulty of the inputs Harder sections of input assigned more cores, easier assigned less

14. Example 2: bodytrack

15. Example 2: streammcluster

16. Example 2: x264

17. Other Heartbeat Uses SpeedPress compiler and SpeedGuard runtime system –The SpeedGuard compiler discovers possible quality-of-service/ performance tradeoffs –The SpeedPress runtime makes these tradeoffs dynamically in response to maintain a given heart rate in the face of environmental changes SmartLocks –Subject of an upcoming SMART talk

18. Outline Introduction/Motivation –Problem –Related Work –Solution: Standard interface for expresses performance/goals Application Heartbeats –Idea –Interface Experiments –Heartbeat use within an application –Heartbeat use by an external system –Other systems using Heartbeats Conclusion –Request for feedback/usage –Summary

19. Request for Feedback Thanks to the reviewers for their feedback, but we need more… Heartbeat code is available online The idea is to provide a simple and standard interface This will only succeed if people use the interface and find it helpful We need your feedback! –If you have an self-tuning service that could benefit from being able to directly measure an application’s performance try the interface –Let us know if functionality is missing or incomplete

20. Summary Presented the Application Heartbeat interface –API provides a standard means for an application to make its performance and goals known Presented several experiments showing basic usage –Several other systems at MIT are using Heartbeats in more advanced applications Requested feedback from the community