1. Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission Mathias Johanson Swedish Research Institute for Information Technology mathias@siti.se

2. Scalability in Videoconferencing Large number of video receivers (and senders) Multiple quality levels in a single multipoint conference session Differentiated host and network requirements Realizable over public internetworks

3. CODEC operates at fixed bandwidth Multipoint operation involves gateways Differentiated quality levels in a multipoint session require transcoders that are expensive and introduce latency Often dependent on level 2 network protocols (e.g. ISDN systems) Limitations of Traditional Videoconferencing Systems

4. Approach... Scalable codec based on subband transform coding Receiver-driven layered IP-multicast transmission Software implementation + DSP-based implementation

5. Layered Video Coding Temporal layering –Increased number of refinement layers correspond to higher framerate Spatial layering –Increased number of refinement layers correspond to higher image resolution Layered quantization –Increased number of refinement layers correspond to finer quantization

6. Temporal Layering Channel 1 Channel 2 Channel 3 Channel 4 Transmission channels that can be received independently Images of a video sequence

7. Spatial Layering Channel 1 Transform Channel 3 Channel 2 Original image Base signal + refinement signals

8. Layered image and video encoding/compression formats Hierarchical JPEG MPEG-2 scalable mode –temporal, spatial, SNR scalability H.263 scalable mode Wavelets Block-based DCT Subband transform

9. Base layer Refinement layer Down- sample x(t) Encode Decode Upsample Spatial scalability in block based image and video encodings

10. Wavelet-based approach to spatial scalability G low x(t) (t)y 0 y 1 2 2 G high base layer refinement layer Quadrature mirror filters implementing the wavelet transform Encode

11. Wavelet transform Iterate…. horizontal transformvertical transform Original image Transformed image

12. Wavelet compression Colorspace conversion and subsampling –RGB -> YCrCb 4:2:2 Wavelet transform (separately on Y, Cr, Cb) –Subband decomposition Quantization of each subband/component –Lossy compression step Huffman encoding –entropy coding

13. Communication Architecture Transmit the subbands of the transformed images on separate channels that can be received independently Multicasting Leaf-initiated JOIN-mechanism RLM Receiver-driven Layered (IP) Multicast

14. 224.3.4.5 224.3.4.6 224.3.4.7 224.3.4.8 Refinement layers Base layer R Internet Sender Receiver (4 layers) Receiver (1 layer) High bandwidth Low bandwidth Multicast router

15. Wavelet RTP header

16. Prototype implementation Based on Smile! Software wavelet codec Receiver-driven layered IP multicast network module RTP/RTCP Spatial and temporal scalability SGI O2, MIPS R5000 processor

17. Usage Scenario highly heterogeneous environment R High-speed LAN Internet Dial-up access Medium quality Low quality High quality Leased Line Transmitter

24. Performance Tests Image quality scalabilityBandwidth scalability

25. Future work... Temporal compression DSP implementation (TMS320C80 or similar) Automatic selective refinement based on ”bandwidth discovery” Subband audio coding