Scalable Video Conferencing Using Subband Transform Coding and Layered Multicast Transmission Mathias Johanson Swedish Research Institute for Information Technology

Scalability in Videoconferencing Large number of video receivers (and senders) Multiple quality levels in a single multipoint conference session Differentiated host and network requirements Realizable over public internetworks

CODEC operates at fixed bandwidth Multipoint operation involves gateways Differentiated quality levels in a multipoint session require transcoders that are expensive and introduce latency Often dependent on level 2 network protocols (e.g. ISDN systems) Limitations of Traditional Videoconferencing Systems

Approach... Scalable codec based on subband transform coding Receiver-driven layered IP-multicast transmission Software implementation + DSP-based implementation

Layered Video Coding Temporal layering –Increased number of refinement layers correspond to higher framerate Spatial layering –Increased number of refinement layers correspond to higher image resolution Layered quantization –Increased number of refinement layers correspond to finer quantization

Temporal Layering Channel 1 Channel 2 Channel 3 Channel 4 Transmission channels that can be received independently Images of a video sequence

Spatial Layering Channel 1 Transform Channel 3 Channel 2 Original image Base signal + refinement signals

Layered image and video encoding/compression formats Hierarchical JPEG MPEG-2 scalable mode –temporal, spatial, SNR scalability H.263 scalable mode Wavelets Block-based DCT Subband transform

Base layer Refinement layer Down- sample x(t) Encode Decode Upsample Spatial scalability in block based image and video encodings

Wavelet-based approach to spatial scalability G low x(t) (t)y 0 y G high base layer refinement layer Quadrature mirror filters implementing the wavelet transform Encode

Wavelet transform Iterate…. horizontal transformvertical transform Original image Transformed image

Wavelet compression Colorspace conversion and subsampling –RGB -> YCrCb 4:2:2 Wavelet transform (separately on Y, Cr, Cb) –Subband decomposition Quantization of each subband/component –Lossy compression step Huffman encoding –entropy coding

Communication Architecture Transmit the subbands of the transformed images on separate channels that can be received independently Multicasting Leaf-initiated JOIN-mechanism RLM Receiver-driven Layered (IP) Multicast

Refinement layers Base layer R Internet Sender Receiver (4 layers) Receiver (1 layer) High bandwidth Low bandwidth Multicast router

Wavelet RTP header

Prototype implementation Based on Smile! Software wavelet codec Receiver-driven layered IP multicast network module RTP/RTCP Spatial and temporal scalability SGI O2, MIPS R5000 processor

Usage Scenario highly heterogeneous environment R High-speed LAN Internet Dial-up access Medium quality Low quality High quality Leased Line Transmitter

Performance Tests Image quality scalabilityBandwidth scalability

Future work... Temporal compression DSP implementation (TMS320C80 or similar) Automatic selective refinement based on ”bandwidth discovery” Subband audio coding