1. 1 A Self-Tuning Configurable Cache Ann Gordon-Ross and Frank Vahid* Department of Computer Science and Engineering University of California, Riverside *Also with the Center for Embedded Computer Systems at UC Irvine This work was supported by the U.S. National Science Foundation, and by the Semiconductor Research Corporation

2. of 11 2 Cache Power Consumption Memory access: 50% of embedded processor’s system power Caches are power hungry ARM920T (Segars 01) M*CORE (Lee/Moyer/Arends 99) Thus, caches are a good candidate for optimizations Different applications have vastly different cache requirements Total size, line size, associativity 4KB 16 byte, 2-way 2KB 32 byte direct-mapped 8KB 64 byte, 4-way

3. of 11 3 Cache Tuning Cache tuning is the process of determining the appropriate cache parameters for an application - can be done during runtime Requires a tunable cache Cache parameter values can be varied during runtime Requires tuning hardware Orchestrates cache tuning Energy Executing in base configuration Tunable cache Tuning hw TC Cache Tuning TC Download application Microprocessor

4. of 11 4 Online Cache Tuning Reconfigure the cache dynamically to adapt to different phases of program execution or different applications in a multi-application environment In this talk, I describe research that addresses when to reconfigure the cache for a periodic system Feedback-control system for online cache tuning Base cache energy Time Energy Consumption Phase-tuned Change cache

5. of 11 5 Online Cache Tuning Challenges Base cache energy Time Energy Consumption Phase Interval Base cache energy Time Energy Consumption Runtime energy Tuning interval Excess tuning energy Tuning interval too short Tuning interval too long Base cache energy Time Energy Consumption Runtime energy Tuning interval Wasted energy in suboptimal configuration Need a good tuning interval Tuning interval is the time between invocations of the tuning hardware Should closely match phase interval - length of time the system executes between phase changes Problem: How does the tuning hardware determine when to invoke cache tuning - must have knowledge of the future to obtain optimal results Previous methods use a fixed tuning interval and do not analyze the value chosen

6. of 11 6 Periodic System - Fixed Phase Interval Phase interval fixed at 10 million cycles Tuning interval too short Tuning interval too long Energy savings = 32% (includes 7% overhead due to tuning) Base Line Negative savings if tuning interval is greater than phase interval!

7. of 11 7 Online Algorithms Need to determine tuning interval while system is executing Online algorithms process data piecemeal - unable to view entire dataset Online tuner must be able to determine the tuning interval based on current and past events with no knowledge of future Goal: Adjust tuning interval (TI) to match phase interval Observe change in energy due to tuning Cache tuning No change in energy Time Energy Consumption TI Change in energy Phase change occurred TI No change in energy

8. of 11 8 Online Cache Tuner – Feedback Control System We model our online cache tuner as a feedback control system Controller Logic is based on attack/decay online algorithm Draw on fuzzy logic to stabilize tuning interval Change tuning interval based on how close or far the system is to being “stable” Use a 2 part equation

9. of 11 9 Controller Logic %∆E 0 100% Multiplicative change to tuning interval (∆TI) Stable System PoS 1.0 Large energy change, tunes too infrequently, decrease tuning interval (tuning interval too long) Small energy change, tunes too frequently, increase tuning interval (tuning interval too short) U D If %∆E < PoS, If %∆E >= PoS, %∆E averaged over last W measurements to eliminate erratic behavior Determine U, D, PoS and W through experimentation %∆ E ∆TI

10. of 11 10 Online Cache Tuner Energy Savings Base line Observed similar results for less periodic systems and systems with more applications - see paper for details. 29% energy savings - within 8% of optimal Normalized Energy

11. of 11 11 Conclusions Observed that fixed tuning intervals may not reveal energy savings Important to vary tuning interval to match system needs Developed a feedback control system for online cache tuning 29% energy savings on average - 8% from optimal Continuing work for more random systems