1. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Kyoungwoo Lee, Minyoung Kim, Nikil Dutt, and Nalini Venkatasubramanian Error-Exploiting Video Encoder to Extend Energy/QoS Tradeoffs for Mobile Embedded Systems Department of Computer Science University of California at Irvine

2. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces DIPES '08 #2 Outline  Motivation  Our Solution  Experiments  Conclusion

3. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Energy Reduction is Essential DIPES '08 #3  Mobile computing is popular Wellness Science Communication, Entertainment, & Education Business Battlefield Resource-limited mobile devices! Fundamental problem is to achieve low power with high performance

4. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Mobile Video Applications  Mobile video applications demand high energy consumption  Complex video encoding/decoding algorithms  Transmitting huge volume of video data DIPES '08 #4 Network Mobile Video Applications ME DCTQ EE f4 f3 f2 f1 Essential to extend the tradeoff space of energy consumption and QoS

5. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Energy/QoS Video Encodings  Energy/QoS-aware video encoding  Video encoding parameters [Mopatra, IPDPS05]  Motion estimation algorithm [Tourapis, VCIP00]  Integrated power management [Mohapatra, ACM MM03]  Global cross-layer adaption [Yuan, MMCN04]  Transmission power and QoS [Eisenberg, IEEE Trans. on CSVT 02]  No Error Resilience  PBPAIR (Probability-Based Power Aware Intra Refresh) – error-resilient and energy-efficient [Kim, MCCR06]  Passive Error Exploitation DIPES 08 #5 Our solution – Active Error Exploitation to reduce the energy consumption for video encoding

6. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Active Error Exploitation  Active Error Exploitation – Intentional Frame Dropping  Skip the expensive video encoding algorithm  Energy saving  Degrade the video quality  Inherent error-tolerance mitigates the impact of frame drops on video quality DIPES 08 #6 f1 f4 f3 f2 Network Mobile Video Applications

7. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Frame Drop Types  FDT-1 affects the following components with respect to power, performance, and QoS in mobile video applications  This work studies FDT-1, and future work includes FDT-2 and FDT-3 DIPES 08 #7 Error-prone Networks Mobile Video Application Enc CPU Tx WNI Dec CPU Rx WNI FDT-1 FDT-2 FDT-3 FDT: Frame Drop Type Enc: Encoding, Dec: Decoding WNI: Wireless Network Interface Packet Loss

8. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Frame Losses due to Packet Losses DIPES '08 #8 Error-Induced Video Data f1 f4 f3 f2 f4 is lost Error-Prone Network f1 f3 f2  Inherent Error-Tolerance of Video Data  Error-Resilient/Error-Concealment Techniques Mobile Video Applications

9. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Inherent Error-Tolerance of Video Data  Error-Tolerance of Video Data  Spatial and temporal correlation among consecutive video frames  Lossy video encoding  (e.g.) High Quantization Scale  Energy Reduction and Error- Tolerance  Error-tolerance can be used for energy reduction  (e.g.) partial ME vs. Full ME  One frame loss may not be noticed by users  (e.g.) One frame loss out of 30 frames per second DIPES '08 #9 f1 f4 f3 f2 Mobile Video Encoding ME DCT Q EE ME – Motion Estimation DCT – Discrete Cosine Transform Q – Quantization EE – Entropy Encoding

10. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Error-Resilient Techniques  Insert Intra-frames (I-frames) periodically DIPES '08 #10 Error-Induced Video Data f1 f4 f3 f2 Error-Prone Network f4 is lost f1 f3 f2  GOP-K (Group-Of-Picture with K)  (e.g.) GOP-3 inserts I-frame every 3 P-frames  Error-resilient GOP [Yang, JVCIP07]  Intra refresh video encoding techniques  AIR (Adaptive Intra Refreshing) [Worral, ICASSP01]  PGOP (Progressive GOP) [Cheng, PCS04]  PBPAIR (Probability-Based Power Aware Intra Refresh) [Kim, MCCR06]  (e.g.) PBPAIR encodes video data resilient against 25% frame loss rate (1 frame out of 4 frames)

11. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Energy Efficiency  Energy-efficient error-resilient video encodings  (e.g.) PBPAIR or Probability-Based Power Aware Intra Refresh [Kim, MCCR06]  It may improve not only the video quality but also energy consumption DIPES 08 #11 Error-Induced Video Data f1 f4 f3 f2 Error-Prone Network f4 is lost f1 f3 f2

12. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces DIPES '08 #12 Outline  Motivation  Our Solution  Error-Exploiting Video Encoding  EE-PBPAIR  Experiments  Conclusion

13. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Our Proposal  Error-exploiting video encoder  Intentional frame dropping + error-resilient video encoding  Extends the tradeoff space for energy consumption / QoS DIPES 08 #13

14. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Error-Resilient Video Encoder DIPES 08 #14 Error-Resilient Encoder Error-Resilient Video Encoder Original Video Data Error- Resilient Video Data Parameters

15. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Error-Exploiting Video Encoder DIPES 08 #15 Error Controller Error-Resilient Encoder Error-Exploiting Video Encoder Original Video Data Error- Injected Video Data Error- Aware Video Data Constraints Parameters & Feedback Error-Injecting UnitError-Canceling Unit Reduce Energy Consumption Incur Energy Overhead Combined approach may consume less energy

16. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Intentional Frame Dropping and PBPAIR  Energy Efficiency  Frame Dropping  (e.g.) f2 is dropped  PBPAIR DIPES 08 #16  Quality Management  Error-Resilience  (e.g.) EE-PBPAIR encodes video data resilient against f2 and f4  Error-Tolerance Error-Induced Video Data f1 f4 f3 f4 is lost Error-Prone Network f1 f3 Intentional Frame Dropping PBPAIR EE-PBPAIR Error-Injecting UnitError-Canceling Unit f2 is dropped

17. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces EIR – Error Injection Rate  EIR adjusts the rate of intentional frame dropping  EIR is Frame Drop Rate at Error-Injecting Unit  EIR is translated for PBPAIR (considering it as PLR)  Feedback-based quality adjustment  High EIR increases energy saving but degrades video quality DIPES 08 #17 Frame Dropping PBPAIR EE-PBPAIR Original Video Data Error- Aware Video Data Parameters Quality Constraint and Quality Feedback Error-Injecting UnitError-Canceling Unit

18. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces DIPES '08 #18 Outline  Motivation  Our Solution  Experiments  Conclusion

19. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces End-to-End Experimental Framework  End-to-End Experimental Framework  Mobile video applications such as video conferencing consist of mobile encoding, wireless(and wired) networks, and mobile decoding  They affect each other in terms of energy consumption and QoS  System Prototype and NS2 Simulator  System Prototype  Runs video encoding and decoding on system prototype emulating mobile devices, and returns video quality in PSNR  Estimates the energy consumption of a processor (CPU power)  NS2 [http://www.isi.edu/nsnam/ns/]  Network Simulator  Estimates the energy consumption of WNI (transmission power) DIPES 08 #19

20. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Mobile Video Decoding Experimental Setup DIPES 08 #20 Encoder Transmit Decoder Receive Network NS2 System Prototype System Prototype Mobile Video Encoding CPU energy for encoding video quality (frame drop) WNI energy for transmit WNI energy for receive CPU energy for decoding video quality (packet loss)

21. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Evaluation  Video Encoding  GOP-K (Group-Of-Picture with K)  PBPAIR (Probability-Based Power Aware Intra Refresh)  EE-PBPAIR (Error-Exploiting PBPAIR)  Energy Consumption  Enc EC (Energy Consumption for Encoding) + Tx EC (Energy Consumption for Transmission)  Rx EC (Energy Consumption for Receiving) + Dec EC (Energy Consumption for Decoding)  Video Quality  Video Quality at encoder after intentional frame dropping  Video Quality at decoder after packet losses in networks DIPES 08 #21

22. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Experimental Results  Energy Reduction from Active Error Exploitation  Extended Energy/QoS Tradeoff DIPES 08 #22

23. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Energy Saving DIPES '08 #23 EC = Energy Consumption Enc EC = EC for Encoding Tx EC = EC for Transmission Dec EC = EC for Decoding Rx EC = EC for Receiving PSNR: Peak Signal to Noise Ratio  PLR = 10% and EIR = 10% Energy saving occurs at every component in a path from encoding to decoding in mobile video applications Mobile Video Decoding Encoder Transmit Decoder Receive Network Mobile Video Encoding

24. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Experimental Results  Energy Reduction from Active Error Exploitation  Extended Energy/QoS Tradeoff DIPES 08 #24

25. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Extended Tradeoff Space DIPES '08 #25  PLR = 5% and EIR = 0% to 50% EE-PBPAIR extends interesting tradeoff spaces

26. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Energy Reduction at QoS Cost DIPES '08 #26 At 10% cost of video quality, EE-PBPAIR can save the energy consumption of Enc and Tx by up to 49%

27. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces DIPES '08 #27 Outline  Motivation  Our Solution  Experiments  Conclusion

28. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Conclusion  Intentional Frame Drop is one way to exploit errors actively  Propose an error-aware video encoding (EE-PBPAIR)  Intentional frame dropping and the nature of energy-efficiency of PBPAIR reduces the energy consumption for video encoding  Present a knob (EIR) to adjust the amount of errors considering the QoS feedback  Maintain the video quality using error-resilience of PBPAIR  Future Work  Intelligent Frame Dropping Techniques  Extend Active Error Exploitation to the system level with error- aware architecture and network protocols in distributed embedded systems DIPES 08 #28

29. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Thanks! Any Questions? kyoungwl@ics.uci.edu

30. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Backup Slides

31. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Intentional Frame Drop and Packet Loss DIPES '08 #31 Error-prone Networks Packet LossIntentional frame drop

32. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces EE-PBPAIR DIPES '08 #32 Error-prone Networks Packet Loss Intentional frame drop Error-Exploiting Video Encoder Error-Resilient Encoder (e.g., PBPAIR) Error-Controller (e.g., frame dropping) Error-Controller (e.g., frame dropping) Original Video Error- Resilient Video Error- Aware Video

33. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Error Controller DIPES '08 #33

34. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Error-Concealment  Error-Concealment Techniques  Interpolate the lost frame using near frames  Substitute the near frame for the lost one  (e.g.) f2 is copied for f3 (the lost one) in displaying frames DIPES '08 #34 Error-Induced Video Data f1 f4 f3 f2 Error-Prone Network f4 is lost f1 f3 f2

35. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces 35 GOP (Group of Picture)  Standard H.263 Video Encoder with varying IP- ratio  Higher IP-ratio generates more compressed video output, which consumes more energy Encoder Static Constraint of Compression Rate GOP Standard video encoding, which is unaware of energy consumption and error-resiliency IP-ratio (KNOB) Intra Frame

36. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces 36 PBPAIR  Proactively estimate the probability of the correctness, and adapt the intra_th (KNOB) based on the current network PLR (Packet Loss Rate) Encoder PLR from Network Channel PBPAIR Error-Resilient Encoding, which can satisfy a given PSNR, and reduce the energy consumption for encoding Intra_Th (KNOB) Intra MB

37. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces 37 EE-PBPAIR  EE-PBPAIR introduces another KNOB (intentional EIR) other than Intra_Th, and can further save the energy consumption Encoder PLR from Network Channel EE-PBPAIR Error-Introduced Video Encoding, which can still satisfy a given PSNR, and further maximize the energy saving compared to PBPAIR Intra_Th (KNOB) Intra MB Intentional EIR (KNOB) I-FS

38. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces System Prototype + NS2 DIPES '08 #38

39. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Adaptive EE-PBPAIR DIPES '08 #39

40. Copyright © 2008 UCI ACES Laboratory http://www.cecs.uci.edu/~aces Adaptive EE-PBPAIR DIPES '08 #40