Power-Saving Techniques with High Visual-Quality for Mobile Displays Dep. of Computer Science & Engineering Yuan Ze University Speaker: Chun-Han Lin National Taiwan Normal University

Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

Motivation Mobile applications and services are having a profound effect on people's lifestyles The energy consumption of mobile devices is a major challenge in sustaining the applications and services Chun-Han Lin, NTNU

Possible Solution Battery Extenders Power-Saving Techniques Chun-Han Lin, NTNU

Power Consumption The display subsystem stays in active mode for various applications –Liquid Crystal Displays (LCDs) –Organic Light-Emitting Diode (OLED) Displays Chun-Han Lin, NTNU

Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

Thin-Film Transistor LCDs Chun-Han Lin, NTNU

Mobile LCDs What hardware to target? Chun-Han Lin, NTNU Power distribution on HTC Desire when browsing videos on YouTube Power distribution on Apple iPad when browsing videos YouTube

LCD Power-Saving Techniques Dim the backlight –Image distortion Challenge –Limit the distortion Image compensation techniques Chun-Han Lin, NTNU

Video A video stream comprises a series of image frames Challenge –Flickering effects –Interframe brightness distortion –Hardware requires time to react and adjust the backlight Previous work –Groups the image frames of a video –Quantizes the number of backlight levels Adjacent frames, instead of having an overall consideration based on all the frames Chun-Han Lin, NTNU

Backlight scaling – Dynamically adjust backlight levels for video frames Video distortion Hardware/software limitation User perception Etc. Backlight Scaling Technique Chun-Han Lin, NTNU

Input and Output Input data –Video –Constraint Video distortion Hardware/software limitation User perception –Power model of mobile device Output data –Backlight file Chun-Han Lin, NTNU

Flowchart Image frames in video Backlight assignment Chun-Han Lin, NTNU

Algorithm 1 Challenge –Video distortion –User perception Solution –Avoid abrupt changes in backlight levels Chun-Han Lin, NTNU

Principle of A1 Chun-Han Lin, NTNU

Algorithm 2 Challenge –Video distortion –HW/SW limitation Solution –Avoid frequent changes in the backlight level Chun-Han Lin, NTNU

Dynamic-Programming in A2 Min. Energy Chun-Han Lin, NTNU

Principle of A2 Chun-Han Lin, NTNU

Algorithm 3 Challenge –Video distortion –User perception –HW/SW limitation Solution –Avoid abrupt changes –Avoid frequent changes Chun-Han Lin, NTNU

Dynamic-Programming in A3 Chun-Han Lin, NTNU

Principle of A3 Chun-Han Lin, NTNU

Cloud-Based Power-Saving Services With the service is applied, the service provider help reduce the energy consumption of mobile devices when they access Internet applications Chun-Han Lin, NTNU

System Architecture Video Stream Backlight File Backlight Server Streaming Server Mobile Device Chun-Han Lin, NTNU

Responsible for generating backlight files The Cloud Side Phase 1 Analyze the video to decide the critical backlight levels (i.e., the dimmest backlight level with respect to the tolerable video distortion) Phase 2 Determine an optimal backlight assignment for the video based on the device’s power model and capability Backlight File Download the requested video from YouTube Critical backlight levels of the video Chun-Han Lin, NTNU

The Device Side Measure power models Develop mobile application programs iPad’s display subsystem Power Monitors Chun-Han Lin, NTNU

Demonstration Approach validation Performance evaluation Chun-Han Lin, NTNU

System Deployment Case studies System architecture Chun-Han Lin, NTNU

Backlight File Process Time & Transmission Delay Approach Validation Video DownloadPhase IPhase II 124 seconds1020 seconds2.1 seconds Transmission Delay 335 milliseconds Cloud Side Device Side Chun-Han Lin, NTNU

Performance Evaluation Experimental Results Case Studies Chun-Han Lin, NTNU

Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

Organic Light-Emitting Diode (OLED) Displays Chun-Han Lin, NTNU

Mobile OLED Displays OLED is deemed promising technology to replace LCD for mobile displays –Brighter colors, wider viewing angles, faster response times, etc. –Power consumption increases dramatically with the pixel values of the displayed image Chun-Han Lin, NTNU

Low-Power Techniques for OLED Displays Partial display disabling or dimming –Darken the contents that are not of interest –Impact user perception Color remapping –Change colors into colors that consume less power –Suit for GUI but not natural images OLED dynamic voltage scaling –Decrease the supply voltage of each pixel’s circuit –Require hardware support and partition the display into rectangular regions Chun-Han Lin, NTNU

Inspired by Human Visual Attention Different regions in an image –Receive varying degrees of visual attention –Can tolerate different degrees of image distortion Chun-Han Lin, NTNU

Quality-Retaining Power Saving Technique Image pixel scaling Segmentation Scaling Combination Chun-Han Lin, NTNU

Fast and optimal without accurate OLED power models Distortion (SSIM) Analysis Attention (Itti) Perception (JND) Conversion Optimal Algorithm Chun-Han Lin, NTNU

Visual Attention Not every region in an image receives the same attention level Image can be segmented based on its saliency map into a set of attention regions The saliency map indicates a saliency value for each pixel in an image Chun-Han Lin, NTNU

Image Distortion Different regions in an image receive varying degrees of attention Different regions can tolerate different degrees of image distortion. Attention regions should be given tolerable distortion in inverse proportion to their attention levels Chun-Han Lin, NTNU

Perception Lowering the pixel values by applying the critical scaling ratio to each region may result in sharp edges between adjacent regions These sharp edges will severely interfere with visual experience The difference between the scaling ratios applied to two adjacent regions should be limited Chun-Han Lin, NTNU

Optimal Algorithm Chun-Han Lin, NTNU

Conversion Software Image converter Image editing software Power-Saving Mode OLED mobile device Chun-Han Lin, NTNU

Experiment Setup 4 images on Samsung Galaxy Tab 7.7 –Different characteristics in terms of luminance and saliency –Performance Metrics Execution time (second) and power consumption (watt) –Comparison A grid-based approach revised based on that in a DAC’12 paper. Chun-Han Lin, NTNU

GRID vs. CURA Execution time (seconds) Power consumption (watts) Visual quality –See a video demo GRIDCURA Image Converter27~2197.6~8.8 Power-Saving Mode0.97~ ~0.811 GRIDCURA Image Converter237~648284~572 Power-Saving Mode362~797305~595 *PSM uses Lanczos resampling to scale down the resolution for speedup at a cost of less power saving. Chun-Han Lin, NTNU

Outline Introduction Liquid Crystal Displays Organic Light-Emitting Diode Displays Conclusion Chun-Han Lin, NTNU

Conclusion We raise the concept of cloud-based energy-saving services and have developed the dynamic backlight scaling service for mobile LCDs –With the service is applied, an HTC Desire mobile phone can save 18-31% backlight energy when browsing videos on YouTube We introduce visual attention into the quality-retaining power-saving design on mobile OLED displays –We present CURA to realize the notion. Samsung Galaxy Tab 7.7 can save 38-42% OLED power while retaining visual quality Chun-Han Lin, NTNU