Power to the People: Leveraging Human Physiological Traits for Microprocessor Frequency Control Alex Shye, Yan Pan, Ben Scholbrock, J. Scott Miller, Gokhan Memik, Peter A. Dinda, Robert P. Dick Northwestern University, EECS ESP Project: http://www.empathicsystems.org International Symposium on Microarchitecture, November 11, 2008. Lake Como, Italy.

Summary Claim: Any optimization ultimately exists to satisfy the user Observation: Architectures largely ignore the individual user Summary of Findings/Contributions Make a case for adding biometric input devices to future architectures Show that biometric devices can be used to indicate changes in user satisfaction as performance is altered Demonstrate that these devices can be leveraged for user-aware optimization Our goal is to attack this problem 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Why care about the user? 1 2 3 User-centric applications Architectural trade-offs exposed to the user 1) Iphone inherently interactive Nintendo Wii : user experience over pure computational power 3) Mention ISCA paper We are proposing a change in how humans and computers interact 3 Optimization opportunity User variation = optimization potential 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Typical User Interaction User Direction (from keyboard, mouse,etc.) Output (from display,speakers,etc.) 1-sided interaction. Note that computer has no information about the human. Decisions to make affect perceived performance 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

From the computer’s perspective Performance Level ? Problem: We can’t get user-related information that we’d like. Explicit input possible, but not practical and may be annoying. Without the appropriate information, it is difficult (if not impossible) for the computer to take the user into account 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Our Goal 1 2 Provide computer user-related Physiological traits (biometric inputs) 1 Provide computer user-related information with biometric inputs Performance Level Informed 2 Leverage human physiological traits for user-aware optimization 1) Input devices with sole purpose of getting user-related information 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Biometric Input Devices Hypothesis: A change in human state due to changes in performance should be reflected by a change in physiological traits We explore using three biometric devices: Eye tracker Galvanic skin response (GSR) sensor Force sensors 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Eye Tracker Process video feed for: 2 measurements: PupilRadius X-Y Coordinates of pupil on video 2 measurements: PupilRadius Mental workload [Iqbal CHI2005] Perceptual changes [Einhauser NAS 2008] Emotion processing [Partala JHCS 2003] PupilMovement Event Perception [Smith ETRA 2006] 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Galvanic Skin Response (GSR) Conductance of skin Reflects “fight-or-flight” response Increases with engagement Decreases with relaxation 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

GSR Behavior GSR spikes with interest DeltaGSR metric measures only the increases in GSR If we want to measure GSR, we can’t just use the absolute value 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Force Sensors Piezoresistive Force Sensors Conductance α Force MaxArrow = Max(4 sensors) We are not aware of previous work but the intuition behind this is that perhaps there will also be measureable behavioral differences as performance is changed. 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Sensor Selection They do not impede with the computer use Require little effort to activate/mount Can be easily integrated Laptops contain integrated camera for eye tracking Mouse/keyboard can be enhanced with GSR and force sensors Power consumption negligible “Cheap” extensions We didn’t just select them randomly. There are several reasons we have selected these sensors. 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Sensor Metrics Four measurements: Sample at 30 Hz PupilRadius, PupilMovement, DeltaGSR, MaxArrow Sample at 30 Hz Each second, compute three statistics: Max, Mean, and Variance Sensor metric = Statistic_Measurement E.g., Max_MaxArrow and Mean_PupilRadius 30 Hz, each second we compute a sensor metric Now that we have our sensors, we are ready to do user studies to understand the sensors 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

User Study Setup IBM Thinkpad T61 Three user studies: Intel Core 2 Duo CPU supporting Intel Speedstep (DVFS) 5 Frequencies (2.2Ghz -- 600Mhz) Windows XP Three user studies: First two show that physiological traits change with performance Third evaluates a system leveraging this information Compare to an Adaptive DVFS scheme modeled after the Linux ondemand governor Three interactive applications: Need for Speed Tetris Arena (third user study) Microsoft Word (third user study) Word is a non-CPU intensive application different than the games 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

First User Study Goal: How: Do human physiological traits change with changes in performance? How: 14 users Play Need for Speed Drop performance to 600Mhz for 20 seconds At same point in game every time Better wording to explain “same point every time” 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Mean_PupilMovement Decrease of pupil movement across most users Explain why we normalized; only change is important Decrease of pupil movement across most users 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Max_MaxArrow Decrease in arrow pressure across most users Surprising, and unexpected that ALL users decreased key force Decrease in arrow pressure across most users 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Max_DeltaGSR Change varies among users We attribute this change to the fact that users react differently to changes in performance. Nevertheless, we notice a definite change in DeltaGSR Change varies among users Some get more aroused (irritated) Some get less aroused (bored) 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Second User Study Goal: How: Can changes in physiological traits be distinguished during game play? Are the changes correlated to user satisfaction? How: 20 users Play Need for Speed Randomly change to each of four other frequencies twice First time, just collect sensor metrics Second time, ask for user satisfaction rating: 1 (bad) – 5 (good) With this information, we need a way to compare or interpret different sets of sensor readings 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Detecting/Interpreting Changes in Sensor Readings “Good” sensor metric behavior If user satisfaction same, sensor metrics should remain same If user satisfaction different, sensor metrics should reflect this We develop a T-test-based Similarity Metric T-test distribution of sensor metric samples from different frequencies High confidence indicates difference in user satisfaction Low confidence indicates no change in user satisfaction We now compute the t-test across the users and sensor metrics 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Using the T-test Similarity Metric We adopt an 85% confidence threshold Always compare to the highest frequency; it is known to be good performance. 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Sensor Metrics vs. User Satisfaction Success: T-test prediction matches change in user satisfaction False Positive: T-test prediction falsely predicts change False Negative: T-test prediction falsely predicts no change Sensor Metric Success Rate False Positive False Negative Max_PupilRadius 70.2% 14.3% 15.5% Max_MaxArrow 69.0% 13.1% 17.9% Mean_MaxArrow Mean_PupilRadius 67.9% 11.9% 20.2% Mean_PupilMovement 57.1% 29.8% Max_DeltaGSR 58.3% 9.5% 32.1% Now that we have a metric, we can evaluate prediction via biometric inputs. False positives are lost opportunity; false negatives are more problematic False negatives higher than we would have liked… 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Using Biometrics for Optimization We have shown that: Human physiological traits do change with performance We can use biometric readings to distinguish these changes We construct PTP to leverage biometric readings Physiological Traits-based Power-management Power To the People Built on top of Adaptive DVFS Tests physiological traits to find a performance level comfortable for the user (settled frequency) Uses settled frequency to set a ceiling for Adaptive DVF Overview: we have shown that… 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

PTP Learning Algorithm Start at highest frequency Successively test lower frequencies one by one Each frequency test consists of three trials One trial consists of: 20 seconds at highest frequency, 20 seconds at test frequency Compute T-test for sensor metrics Majority vote across sensors Majority vote across trials If a majority vote says OK, try next frequency If majority vote predicts difference, go up one frequency and settle there More details in the paper. “test” frequency and NOT lowest frequency cPTP described in paper 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

PTP Learning Algorithm Finish with “we settle at 1.6 Ghz”. 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Third User Study: PTP Evaluation Goal: Does PTP work? How Run the learning algorithm to find the settled frequency for the individual user Run once with PTP at the settled frequency and once with the Adaptive scheme Order is randomized 2.5 minutes each Ask for user satisfaction rating from 1 (bad) – 5 (good) Measure total system power for comparison 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Need for Speed Slightly decrease user satisfaction Decrease, but the user satisfaction is still high cPTP: ~6% total system power savings Slightly decrease user satisfaction 18% total system power savings 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Tetris No change to user satisfaction 33% total system power savings 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Microsoft Word No change to user satisfaction Not same interactiveness, and low CPU-utilization Were able to settle at a considerably lower frequency so still able to understand user satisfaction No change to user satisfaction 2% total system power savings 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Conclusion Motivate new biometric input devices for future architectures Eye tracker, GSR, and force sensors Human physiological traits change with performance Show biometric inputs can be used to indicate user satisfaction Demonstrate PTP for user-aware power management 18% total system power savings across three applications Little to no change in user satisfaction 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Thank you! Questions? Alex Shye http://www.ece.northwestern.edu/~ash451 shye@northwestern.edu ESP: Empathic Systems Project http://www.empathicsystems.org This concludes my talk. My contact information and the project website are below for reference. Thank you for listening and I would be happy to answer any questions. 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy

Sensors and User Satisfaction Work on transition into this slide Note that this is the average trends; for many, not linear, just average is 11/11/08 International Symposium on Microarchitecture (MICRO-41), Lake Como, Italy