1. Power-Saving Techniques with High Visual-Quality for Mobile Displays Dep. of Computer Science & Engineering Yuan Ze University Speaker: Chun-Han Lin National Taiwan Normal University

2. Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

3. Motivation Mobile applications and services are having a profound effect on people's lifestyles The energy consumption of mobile devices is a major challenge in sustaining the applications and services Chun-Han Lin, NTNU

4. Possible Solution Battery Extenders Power-Saving Techniques Chun-Han Lin, NTNU

5. Power Consumption The display subsystem stays in active mode for various applications –Liquid Crystal Displays (LCDs) –Organic Light-Emitting Diode (OLED) Displays Chun-Han Lin, NTNU

6. Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

7. Thin-Film Transistor LCDs Chun-Han Lin, NTNU

8. Mobile LCDs What hardware to target? Chun-Han Lin, NTNU Power distribution on HTC Desire when browsing videos on YouTube Power distribution on Apple iPad when browsing videos YouTube

9. LCD Power-Saving Techniques Dim the backlight –Image distortion Challenge –Limit the distortion Image compensation techniques Chun-Han Lin, NTNU

10. Video A video stream comprises a series of image frames Challenge –Flickering effects –Interframe brightness distortion –Hardware requires time to react and adjust the backlight Previous work –Groups the image frames of a video –Quantizes the number of backlight levels Adjacent frames, instead of having an overall consideration based on all the frames Chun-Han Lin, NTNU

11. Backlight scaling – Dynamically adjust backlight levels for video frames Video distortion Hardware/software limitation User perception Etc. Backlight Scaling Technique Chun-Han Lin, NTNU

12. Input and Output Input data –Video –Constraint Video distortion Hardware/software limitation User perception –Power model of mobile device Output data –Backlight file Chun-Han Lin, NTNU

13. Flowchart Image frames in video Backlight assignment Chun-Han Lin, NTNU

14. Algorithm 1 Challenge –Video distortion –User perception Solution –Avoid abrupt changes in backlight levels Chun-Han Lin, NTNU

15. Principle of A1 Chun-Han Lin, NTNU

16. Algorithm 2 Challenge –Video distortion –HW/SW limitation Solution –Avoid frequent changes in the backlight level Chun-Han Lin, NTNU

17. Dynamic-Programming in A2 Min. Energy Chun-Han Lin, NTNU

18. Principle of A2 Chun-Han Lin, NTNU

19. Algorithm 3 Challenge –Video distortion –User perception –HW/SW limitation Solution –Avoid abrupt changes –Avoid frequent changes Chun-Han Lin, NTNU

20. Dynamic-Programming in A3 Chun-Han Lin, NTNU

21. Principle of A3 Chun-Han Lin, NTNU

22. Cloud-Based Power-Saving Services With the service is applied, the service provider help reduce the energy consumption of mobile devices when they access Internet applications Chun-Han Lin, NTNU

23. System Architecture Video Stream Backlight File Backlight Server Streaming Server Mobile Device Chun-Han Lin, NTNU

24. Responsible for generating backlight files The Cloud Side Phase 1 Analyze the video to decide the critical backlight levels (i.e., the dimmest backlight level with respect to the tolerable video distortion) Phase 2 Determine an optimal backlight assignment for the video based on the device’s power model and capability Backlight File Download the requested video from YouTube Critical backlight levels of the video Chun-Han Lin, NTNU

25. The Device Side Measure power models Develop mobile application programs iPad’s display subsystem Power Monitors Chun-Han Lin, NTNU

26. Demonstration Approach validation Performance evaluation Chun-Han Lin, NTNU

27. System Deployment Case studies System architecture Chun-Han Lin, NTNU

28. Backlight File Process Time & Transmission Delay Approach Validation Video DownloadPhase IPhase II 124 seconds1020 seconds2.1 seconds Transmission Delay 335 milliseconds Cloud Side Device Side Chun-Han Lin, NTNU

29. Performance Evaluation Experimental Results Case Studies Chun-Han Lin, NTNU

30. Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

31. Organic Light-Emitting Diode (OLED) Displays Chun-Han Lin, NTNU

32. Mobile OLED Displays OLED is deemed promising technology to replace LCD for mobile displays –Brighter colors, wider viewing angles, faster response times, etc. –Power consumption increases dramatically with the pixel values of the displayed image Chun-Han Lin, NTNU

33. Low-Power Techniques for OLED Displays Partial display disabling or dimming –Darken the contents that are not of interest –Impact user perception Color remapping –Change colors into colors that consume less power –Suit for GUI but not natural images OLED dynamic voltage scaling –Decrease the supply voltage of each pixel’s circuit –Require hardware support and partition the display into rectangular regions Chun-Han Lin, NTNU

34. Inspired by Human Visual Attention Different regions in an image –Receive varying degrees of visual attention –Can tolerate different degrees of image distortion Chun-Han Lin, NTNU

35. Quality-Retaining Power Saving Technique Image pixel scaling Segmentation Scaling Combination Chun-Han Lin, NTNU

36. Fast and optimal without accurate OLED power models Distortion (SSIM) Analysis Attention (Itti) Perception (JND) Conversion Optimal Algorithm Chun-Han Lin, NTNU

37. Visual Attention Not every region in an image receives the same attention level Image can be segmented based on its saliency map into a set of attention regions The saliency map indicates a saliency value for each pixel in an image Chun-Han Lin, NTNU

38. Image Distortion Different regions in an image receive varying degrees of attention Different regions can tolerate different degrees of image distortion. Attention regions should be given tolerable distortion in inverse proportion to their attention levels Chun-Han Lin, NTNU

39. Perception Lowering the pixel values by applying the critical scaling ratio to each region may result in sharp edges between adjacent regions These sharp edges will severely interfere with visual experience The difference between the scaling ratios applied to two adjacent regions should be limited Chun-Han Lin, NTNU

40. Optimal Algorithm Chun-Han Lin, NTNU

41. Conversion Software Image converter Image editing software Power-Saving Mode OLED mobile device Chun-Han Lin, NTNU

42. Experiment Setup 4 images on Samsung Galaxy Tab 7.7 –Different characteristics in terms of luminance and saliency –Performance Metrics Execution time (second) and power consumption (watt) –Comparison A grid-based approach revised based on that in a DAC’12 paper. Chun-Han Lin, NTNU

43. GRID vs. CURA Execution time (seconds) Power consumption (watts) Visual quality –See a video demo GRIDCURA Image Converter27~2197.6~8.8 Power-Saving Mode0.97~4.770.72~0.811 GRIDCURA Image Converter237~648284~572 Power-Saving Mode362~797305~595 *PSM uses Lanczos resampling to scale down the resolution for speedup at a cost of less power saving. Chun-Han Lin, NTNU

44. Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

45. Conclusion We raise the concept of cloud-based energy-saving services and have developed the dynamic backlight scaling service for mobile LCDs –With the service is applied, an HTC Desire mobile phone can save 18-31% backlight energy when browsing videos on YouTube We introduce visual attention into the quality-retaining power-saving design on mobile OLED displays –We present CURA to realize the notion. Samsung Galaxy Tab 7.7 can save 38-42% OLED power while retaining visual quality Chun-Han Lin, NTNU