En-hui Yang Multimedia Communications Laboratory Dept. of Electrical & Computer Engineering

About the Speaker Professor and Canada Research Chairs Fellow of the Royal Society of Canada Fellow of the Canadian Academy of Engineering Co-founder of SlipStream Data Inc. (now a subsidiary of Research In Motion) Areas of expertise: Multimedia compression; information theory; digital communications; image and video coding

Convergence of Mobile Communications and Computing (3COM) Ten years ago 3C 3CInfo 3CInfoTech SlipStream 3C (Convergence of communication and computing) did not happen. Today we have witnessed not only 3C, but also 3COM Global mobile data traffic: 237 petabytes per month in 2010, over three times the entire global Internet traffic in 2000 (Cisco VNI Mobile 2011) Smart devices  Smart phones: 297 million units sold in 2010 (Gartner); represent about 13 percent of total global handsets in use  Tablets: 20 million units sold in 2010 (iHS iSuppli) Contents and carriers:  CTV acquisition by BCE  Canwest Global acquisition by Shaw  NBC acquisition by Comcast

3COM: Opportunities The convergence is happening at a speed much faster than most of us have anticipated. According to the forecast from Cisco VNI Mobile 2011, By 2015, there will be over 7.1 billion mobile-connected devices (including machine-to-machine modules), roughly one mobile device per capita. The global mobile data traffic will grow at a compound annual growth rate (CAGR) of 92% from 2010 to 2015, reaching about 6.3 exabytes per month by The amount of mobile data traffic generated by tablets will reach 248 petabytes per month in 2015, more than the entire global mobile data traffic in By 2015, more than 60% of the global mobile data traffic will be video.

Mobile Video: Traffic Forecast

Mobile Video: Typical Applications  Local video playback (including downloading, streaming, etc.)  Video chat  Video telephony  Video conferencing  Video broadcast/multicast  Video uploading Behind all these applications is video encoding/decoding

Mobile Video: Bottlenecks With so much data processed and moved around, two related issues stand out: Bandwidth Power: radio power needed for communicating, and computing power needed for processing For real-time communications such as video chat, telephony, and conferencing, the interplay among bandwidth, radio power, and computing power becomes very intriguing.

Mobile Video: Shannon Capacity Formula For AWGN channels, the best trade-off between bandwidth and radio power for real-time communications is described as Although the above capacity formula is established for AGWN channels, it is also a basis for the capacity of many other practical channels. Take-away message in our context: Given W, the radio power needed for real-time communications increases exponentially with the date rate.

Existing Video Coding Standards: Evolution Path ITU-T ISO/IEC H.120 ( ) H.261 (1990+) MPEG-1 (1993) H.262/MPEG-2 (1994/ ) H.263 ( ) MPEG-4 Visual ( ) H.264/MPEG-4 AVC ( ) H.265/ HEVC

Existing Video Coding Standards: Predictive Video Coding (PVC) Paradigm Original Frames: Reconstructed Frames: IPPP All video coding standards proposed so far fall into PVC

Existing Video Coding Standards: Frame Level Structure Prediction/motion compensation

Existing Video Coding Standards: Good Solutions for Mobile Video? Each standard improves the compression performance of its predecessors at the cost of significant increase in encoding and decoding complexity. Compression gains of these standards over their predecessors come, to a large extent, from the improvements in their signal processing components---prediction/motion compensation. Further gains from this direction, if any, are diminishing for given frame sizes and would come at extreme high computation complexity. In the pursuit of improving compression performance, computing power for real-time mobile applications is almost forgotten in the development of these standards. Answer: No

IT Video Coding: Green Solutions for Mobile Video For mobile video, we need green solutions which can drive up the compression performance while reducing the computing power needed for encoding and decoding. Such solutions could be offered by what I call information theoretic (IT) video coding. Instead of driven by signal processing components, IT video coding includes new ways of quantization and coding, tight integration of algorithm innovation and software/hardware implementation, and possibly paradigm shifts such as from PVC to causal video coding. Within IT video coding, IT methods and signal processing methods (such as prediction/motion compensation) are complementary to each other.

IT Video Coding: Future Standards

IT Video Coding: Early Successes Soft decision quantization (SDQ) and optimal algorithms for SDQ developed by Yang and his former PhD students/postdocs in recent years and now widely used. Iterative video coding developed by Yang and his former PhD student. Causal video coding paradigm: More and Less Coding Theorem: Under some conditions on source frames and distortion, the more frames need to be encoded and transmitted, the less amount of data after encoding has to be actually sent!

Thank you