Technion - IIT Dept. of Electrical Engineering Signal and Image Processing lab Transrating and Transcoding of Coded Video Signals David Malah Ran Bar-Sella Michael Lavrentiev Research Proposal to STRIMM Meeting June 25, 2003

Transcoder Coded Bit Stream Transcoded Bit Stream Converting a coded bit stream to another bitstream to match a destination profile in terms of: – Media Formats {e.g., MPEG-2 H.264} – Resolutions (Spatial, Temporal) – Bit Rates (Transrating) – Color depths / formats {e.g., 4:4:4  4:2:0} Transcoding

Cell-phone

Video Coding Standards

Generic Hybrid Video Encoder.. Image sequence Bit stream + DCT Q VLC Q -1 DCT -1 + MEM MC - + Intra / Inter switch + + ME MC-Motion Compensation ME-Motion Estimation MEM-Frame store DCT-Discrete Cosine Transform Q-Quantization VLC-Variable Length Code MPEG Frame Structure H.264 Intra Spatial Prediction

Frame dropping (B, P) Color suppression Discarding high-frequency DCT coefficients Reducing spatial resolution (size reduction) DCT coefficients Re-quantization Transrating Methods (Bit-Rate Reduction - BRR)

Cascading Decoder and Encoder Full Decoder Full Encoder Coded Bit Stream Transcoded Bit Stream Transcoder Architectures Related Issues Open Loop Closed Loop - Pixel-Domain - DCT-Domain Partial Encoder Complexity-Reduced Transcoding (Compressed Domain) Partial Decoder Processing Motion vectors / coding modes Coded Bit Stream Transcoded Bit Stream

Closed Loop Drift Compensation (Werner 1997) Encoder Transcoder Open-Loop

Re-quantization Error Q1 Q2 Original pixel value Quantized values with Q1 Quantized values with Q2

Re-quantization Step-size Selection (Wang and Woods, 1998)

Preferred Step-size Ratios (Wang and Woods, 1998)

MSE Re-quantization

MAP Re-quantization p(x) xp 1 p 2 d 1 d 2 p 2 < p 1 p 2 > p 1 Finer quantizer R 1 R 2 D Coarser quantizer Q1 Q2

Constrained Minimization problem: Lagrangian cost function becomes sum of independent MB level calculated parts: Lagrangian parameter, λ≥0, is iteratively updated to achieve desired bit-rate R T Re-quantization by Lagrangian Optimization (Assuncao and Ghanbari, 1997) Recent result (Laventer and Porat – ICIP-03): Optimal quantizer steps are obtained at values close to specific multiples of the input quant. step.

Variable Length Coding

Macro-Block Trellis Re-quantization q*k max q*k 2 block_1 q*k 1 block_2block_N

HVS-based Bit Allocation Different regions have different perceptual importance and may be encoded at different rates for similar subjective quality. Video frames can be partitioned (segmented) into regions having different characteristics: - Textured regions - Smooth regions - Edges - Moving objects For efficiency, segmentation should be based on compressed- domain data: Block DCT Coeff., MB type, MB q-step, MB rate, and Motion Vectors (MV). Possible approach: Block classification, Region-growing, and Segment tracking. Bit allocation to each region should reflect perceptual importance.

MPEG-2  H.264 Transcoding MEC - 7 modes (block sizes 16x16 – 4x4) - ¼ Pel resolution - Improved MV coding Transform - Integer transform (4x4) Intra Prediction - spatial prediction (9 modes) - 16x16 or 4x4 blocks Inter Prediction - Multiple reference frames (1 – 5) - SP frame for stream transition H.264 – Main Differences vis-à-vis MPEG-2 Quantization and Coeff. Scanning - 32 logarithmic steps - no ‘dead-zone’ - 2 scan modes De-blocking Filter (4x4) Entropy Coding - Universal VLC (UVLC) - Arithmetic Coding (CABAC) Network Friendly Representation - Video Coding Layer - Network Adaptation Layer (packetization, priority control)

Proposed Activity Re-quantization Restricted Lagrangian optimization with MSE / MAP Trellis re-quantization (MSE optional) Bit-rate Control Integrating other rate-reduction techniques (temporal and spatial resolution reduction – BGU) I. Transrating (MPEG2  MPEG2) HVS-based bit-rate reduction Frame segmentation and tracking HVS-based bit allocation to segments Restricted Lagrangian optimization in each segment Trellis re-quantization in each segment

Proposed Activity (Cont’d) II. Transcoding (MPEG-2  H.264) Base-line Transcoder Maximum matching to MPEG-2 - MEC – MB based; ½ pixel res. - B&P frames, Single ref. frame - No Intra spatial prediction - Same MB types decisions Adapted H.264 features: - Syntax - UVLC - Transform - Quantization Reduced rate - Adapting developed re-quantization techniques from MPEG-2 - Integrating other rate- reduction techniques (temporal and spatial resolution reduction – BGU) Full Transcoder Similar rate - Intra prediction - All motion modes - Multiple ref. Frames - Coding control decisions

END