Copyright © 2008 UCI ACES Laboratory Kyoungwoo Lee, Minyoung Kim, Nikil Dutt, and Nalini Venkatasubramanian Error-Exploiting Video Encoder to Extend Energy/QoS Tradeoffs for Mobile Embedded Systems Department of Computer Science University of California at Irvine
Copyright © 2008 UCI ACES Laboratory DIPES '08 #2 Outline Motivation Our Solution Experiments Conclusion
Copyright © 2008 UCI ACES Laboratory Energy Reduction is Essential DIPES '08 #3 Mobile computing is popular Wellness Science Communication, Entertainment, & Education Business Battlefield Resource-limited mobile devices! Fundamental problem is to achieve low power with high performance
Copyright © 2008 UCI ACES Laboratory Mobile Video Applications Mobile video applications demand high energy consumption Complex video encoding/decoding algorithms Transmitting huge volume of video data DIPES '08 #4 Network Mobile Video Applications ME DCTQ EE f4 f3 f2 f1 Essential to extend the tradeoff space of energy consumption and QoS
Copyright © 2008 UCI ACES Laboratory Energy/QoS Video Encodings Energy/QoS-aware video encoding Video encoding parameters [Mopatra, IPDPS05] Motion estimation algorithm [Tourapis, VCIP00] Integrated power management [Mohapatra, ACM MM03] Global cross-layer adaption [Yuan, MMCN04] Transmission power and QoS [Eisenberg, IEEE Trans. on CSVT 02] No Error Resilience PBPAIR (Probability-Based Power Aware Intra Refresh) – error-resilient and energy-efficient [Kim, MCCR06] Passive Error Exploitation DIPES 08 #5 Our solution – Active Error Exploitation to reduce the energy consumption for video encoding
Copyright © 2008 UCI ACES Laboratory Active Error Exploitation Active Error Exploitation – Intentional Frame Dropping Skip the expensive video encoding algorithm Energy saving Degrade the video quality Inherent error-tolerance mitigates the impact of frame drops on video quality DIPES 08 #6 f1 f4 f3 f2 Network Mobile Video Applications
Copyright © 2008 UCI ACES Laboratory Frame Drop Types FDT-1 affects the following components with respect to power, performance, and QoS in mobile video applications This work studies FDT-1, and future work includes FDT-2 and FDT-3 DIPES 08 #7 Error-prone Networks Mobile Video Application Enc CPU Tx WNI Dec CPU Rx WNI FDT-1 FDT-2 FDT-3 FDT: Frame Drop Type Enc: Encoding, Dec: Decoding WNI: Wireless Network Interface Packet Loss
Copyright © 2008 UCI ACES Laboratory Frame Losses due to Packet Losses DIPES '08 #8 Error-Induced Video Data f1 f4 f3 f2 f4 is lost Error-Prone Network f1 f3 f2 Inherent Error-Tolerance of Video Data Error-Resilient/Error-Concealment Techniques Mobile Video Applications
Copyright © 2008 UCI ACES Laboratory Inherent Error-Tolerance of Video Data Error-Tolerance of Video Data Spatial and temporal correlation among consecutive video frames Lossy video encoding (e.g.) High Quantization Scale Energy Reduction and Error- Tolerance Error-tolerance can be used for energy reduction (e.g.) partial ME vs. Full ME One frame loss may not be noticed by users (e.g.) One frame loss out of 30 frames per second DIPES '08 #9 f1 f4 f3 f2 Mobile Video Encoding ME DCT Q EE ME – Motion Estimation DCT – Discrete Cosine Transform Q – Quantization EE – Entropy Encoding
Copyright © 2008 UCI ACES Laboratory Error-Resilient Techniques Insert Intra-frames (I-frames) periodically DIPES '08 #10 Error-Induced Video Data f1 f4 f3 f2 Error-Prone Network f4 is lost f1 f3 f2 GOP-K (Group-Of-Picture with K) (e.g.) GOP-3 inserts I-frame every 3 P-frames Error-resilient GOP [Yang, JVCIP07] Intra refresh video encoding techniques AIR (Adaptive Intra Refreshing) [Worral, ICASSP01] PGOP (Progressive GOP) [Cheng, PCS04] PBPAIR (Probability-Based Power Aware Intra Refresh) [Kim, MCCR06] (e.g.) PBPAIR encodes video data resilient against 25% frame loss rate (1 frame out of 4 frames)
Copyright © 2008 UCI ACES Laboratory Energy Efficiency Energy-efficient error-resilient video encodings (e.g.) PBPAIR or Probability-Based Power Aware Intra Refresh [Kim, MCCR06] It may improve not only the video quality but also energy consumption DIPES 08 #11 Error-Induced Video Data f1 f4 f3 f2 Error-Prone Network f4 is lost f1 f3 f2
Copyright © 2008 UCI ACES Laboratory DIPES '08 #12 Outline Motivation Our Solution Error-Exploiting Video Encoding EE-PBPAIR Experiments Conclusion
Copyright © 2008 UCI ACES Laboratory Our Proposal Error-exploiting video encoder Intentional frame dropping + error-resilient video encoding Extends the tradeoff space for energy consumption / QoS DIPES 08 #13
Copyright © 2008 UCI ACES Laboratory Error-Resilient Video Encoder DIPES 08 #14 Error-Resilient Encoder Error-Resilient Video Encoder Original Video Data Error- Resilient Video Data Parameters
Copyright © 2008 UCI ACES Laboratory Error-Exploiting Video Encoder DIPES 08 #15 Error Controller Error-Resilient Encoder Error-Exploiting Video Encoder Original Video Data Error- Injected Video Data Error- Aware Video Data Constraints Parameters & Feedback Error-Injecting UnitError-Canceling Unit Reduce Energy Consumption Incur Energy Overhead Combined approach may consume less energy
Copyright © 2008 UCI ACES Laboratory Intentional Frame Dropping and PBPAIR Energy Efficiency Frame Dropping (e.g.) f2 is dropped PBPAIR DIPES 08 #16 Quality Management Error-Resilience (e.g.) EE-PBPAIR encodes video data resilient against f2 and f4 Error-Tolerance Error-Induced Video Data f1 f4 f3 f4 is lost Error-Prone Network f1 f3 Intentional Frame Dropping PBPAIR EE-PBPAIR Error-Injecting UnitError-Canceling Unit f2 is dropped
Copyright © 2008 UCI ACES Laboratory EIR – Error Injection Rate EIR adjusts the rate of intentional frame dropping EIR is Frame Drop Rate at Error-Injecting Unit EIR is translated for PBPAIR (considering it as PLR) Feedback-based quality adjustment High EIR increases energy saving but degrades video quality DIPES 08 #17 Frame Dropping PBPAIR EE-PBPAIR Original Video Data Error- Aware Video Data Parameters Quality Constraint and Quality Feedback Error-Injecting UnitError-Canceling Unit
Copyright © 2008 UCI ACES Laboratory DIPES '08 #18 Outline Motivation Our Solution Experiments Conclusion
Copyright © 2008 UCI ACES Laboratory End-to-End Experimental Framework End-to-End Experimental Framework Mobile video applications such as video conferencing consist of mobile encoding, wireless(and wired) networks, and mobile decoding They affect each other in terms of energy consumption and QoS System Prototype and NS2 Simulator System Prototype Runs video encoding and decoding on system prototype emulating mobile devices, and returns video quality in PSNR Estimates the energy consumption of a processor (CPU power) NS2 [ Network Simulator Estimates the energy consumption of WNI (transmission power) DIPES 08 #19
Copyright © 2008 UCI ACES Laboratory Mobile Video Decoding Experimental Setup DIPES 08 #20 Encoder Transmit Decoder Receive Network NS2 System Prototype System Prototype Mobile Video Encoding CPU energy for encoding video quality (frame drop) WNI energy for transmit WNI energy for receive CPU energy for decoding video quality (packet loss)
Copyright © 2008 UCI ACES Laboratory Evaluation Video Encoding GOP-K (Group-Of-Picture with K) PBPAIR (Probability-Based Power Aware Intra Refresh) EE-PBPAIR (Error-Exploiting PBPAIR) Energy Consumption Enc EC (Energy Consumption for Encoding) + Tx EC (Energy Consumption for Transmission) Rx EC (Energy Consumption for Receiving) + Dec EC (Energy Consumption for Decoding) Video Quality Video Quality at encoder after intentional frame dropping Video Quality at decoder after packet losses in networks DIPES 08 #21
Copyright © 2008 UCI ACES Laboratory Experimental Results Energy Reduction from Active Error Exploitation Extended Energy/QoS Tradeoff DIPES 08 #22
Copyright © 2008 UCI ACES Laboratory Energy Saving DIPES '08 #23 EC = Energy Consumption Enc EC = EC for Encoding Tx EC = EC for Transmission Dec EC = EC for Decoding Rx EC = EC for Receiving PSNR: Peak Signal to Noise Ratio PLR = 10% and EIR = 10% Energy saving occurs at every component in a path from encoding to decoding in mobile video applications Mobile Video Decoding Encoder Transmit Decoder Receive Network Mobile Video Encoding
Copyright © 2008 UCI ACES Laboratory Experimental Results Energy Reduction from Active Error Exploitation Extended Energy/QoS Tradeoff DIPES 08 #24
Copyright © 2008 UCI ACES Laboratory Extended Tradeoff Space DIPES '08 #25 PLR = 5% and EIR = 0% to 50% EE-PBPAIR extends interesting tradeoff spaces
Copyright © 2008 UCI ACES Laboratory Energy Reduction at QoS Cost DIPES '08 #26 At 10% cost of video quality, EE-PBPAIR can save the energy consumption of Enc and Tx by up to 49%
Copyright © 2008 UCI ACES Laboratory DIPES '08 #27 Outline Motivation Our Solution Experiments Conclusion
Copyright © 2008 UCI ACES Laboratory Conclusion Intentional Frame Drop is one way to exploit errors actively Propose an error-aware video encoding (EE-PBPAIR) Intentional frame dropping and the nature of energy-efficiency of PBPAIR reduces the energy consumption for video encoding Present a knob (EIR) to adjust the amount of errors considering the QoS feedback Maintain the video quality using error-resilience of PBPAIR Future Work Intelligent Frame Dropping Techniques Extend Active Error Exploitation to the system level with error- aware architecture and network protocols in distributed embedded systems DIPES 08 #28
Copyright © 2008 UCI ACES Laboratory Thanks! Any Questions?
Copyright © 2008 UCI ACES Laboratory Backup Slides
Copyright © 2008 UCI ACES Laboratory Intentional Frame Drop and Packet Loss DIPES '08 #31 Error-prone Networks Packet LossIntentional frame drop
Copyright © 2008 UCI ACES Laboratory EE-PBPAIR DIPES '08 #32 Error-prone Networks Packet Loss Intentional frame drop Error-Exploiting Video Encoder Error-Resilient Encoder (e.g., PBPAIR) Error-Controller (e.g., frame dropping) Error-Controller (e.g., frame dropping) Original Video Error- Resilient Video Error- Aware Video
Copyright © 2008 UCI ACES Laboratory Error Controller DIPES '08 #33
Copyright © 2008 UCI ACES Laboratory Error-Concealment Error-Concealment Techniques Interpolate the lost frame using near frames Substitute the near frame for the lost one (e.g.) f2 is copied for f3 (the lost one) in displaying frames DIPES '08 #34 Error-Induced Video Data f1 f4 f3 f2 Error-Prone Network f4 is lost f1 f3 f2
Copyright © 2008 UCI ACES Laboratory 35 GOP (Group of Picture) Standard H.263 Video Encoder with varying IP- ratio Higher IP-ratio generates more compressed video output, which consumes more energy Encoder Static Constraint of Compression Rate GOP Standard video encoding, which is unaware of energy consumption and error-resiliency IP-ratio (KNOB) Intra Frame
Copyright © 2008 UCI ACES Laboratory 36 PBPAIR Proactively estimate the probability of the correctness, and adapt the intra_th (KNOB) based on the current network PLR (Packet Loss Rate) Encoder PLR from Network Channel PBPAIR Error-Resilient Encoding, which can satisfy a given PSNR, and reduce the energy consumption for encoding Intra_Th (KNOB) Intra MB
Copyright © 2008 UCI ACES Laboratory 37 EE-PBPAIR EE-PBPAIR introduces another KNOB (intentional EIR) other than Intra_Th, and can further save the energy consumption Encoder PLR from Network Channel EE-PBPAIR Error-Introduced Video Encoding, which can still satisfy a given PSNR, and further maximize the energy saving compared to PBPAIR Intra_Th (KNOB) Intra MB Intentional EIR (KNOB) I-FS
Copyright © 2008 UCI ACES Laboratory System Prototype + NS2 DIPES '08 #38
Copyright © 2008 UCI ACES Laboratory Adaptive EE-PBPAIR DIPES '08 #39
Copyright © 2008 UCI ACES Laboratory Adaptive EE-PBPAIR DIPES '08 #40