G. Valenzise *, M. Tagliasacchi *, S. Tubaro *, L. Piccarreta Picture Coding Symposium 2007 November 7-9, 2007 – Lisboa, Portugal * Dipartimento di Elettronica e Informazione, Politecnico di Milano

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Transmit multiple video streams over a shared channel ◦ Broadcast television ◦ Video-surveillance ◦ etc…  The channel bandwidth is limited  Equal bandwidth partitioning is not optimal

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Find an optimal way to allocate channel bandwidth among sequences  Example: two video sequences time quality time quality time quality

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Formulate the MINAVE and MINVAR problems in the  -domain  Assumptions: ◦ Constant bit rate (CBR) channel ◦ Frame-by-frame optimization  Find a closed form solution for the MINVAR  Compare the MINAVE and MINVAR criteria ◦ Check the coding efficiency loss for the average distortion  Relax the CBR assumption ◦ Temporal smoothing

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Rate-distortion operational curve of each frame can be described in the ρ-domain (He and Mitra, 2002):  ρ-domain parameters can be estimated from decoded sequences

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007 (He and Mitra, 2002)

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Which is equivalent to solving  When

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007 MINAVE MINVAR

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Average distortion of MINAVE  Average distortion of MINVAR  We know (by definition!)  QUESTION: ◦ What is the coding efficiency loss?

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Example: ◦ 3 video sequences to be multiplexed on a shared channel ◦ Total rate: R = 1.2 bpp ◦ For each frame we allocate the available bandwidth according to either the MINAVE or the MINVAR criteria ◦ We can evaluate then the resulting average distortion using the exponential  -domain model

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Average distortion of MINAVE  Average distortion of MINVAR  The coding efficiency loss is

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Simulations: ◦ Extract the  -domain parameters from 3 real sequences ◦ Compute the MINAVE and MINVAR rates for each sequences, for each frame ◦ Compute the coding efficiency loss

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  From Information Theory we know that Bad news....  But we can find a more stringent bound We need to find this term

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Relax the CBR assumption ◦ Introduce a shared encoder buffer to perform VBR encoding  We apply the MINVAR rate allocation while at the same time achieving temporal smoothing  For each time instant: 1.Compute the CBR distortion profile 2.Smooth it with the low-pass filter (He, Zen and Chen, 2005) 3.Set D smooth as the target distortion and compute rates R i  Relax or tighten the rate constraint according to the current buffer level

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Temporal smoothness is achieved by means of a shared encoder buffer  Target distortion level ◦ obtained solving MINVAR and CBR constraint C (He et al., 2005) Bits addedBits drained

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Let  If B res < 0 ◦ solve unconstrained minimization problem  Else, if B res > 0 ◦ Solve constrained minimization problem with new rate constraint

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007 H.264 decoder H.264 decoder H.264 decoder... H.264 bitstream Rate controller H.264 encoder H.264 encoder H.264 encoder BUFFER

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007 MINAVE MINVAR smoothed MINVAR

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007 MINAVE MINVAR smoothed MINVAR

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007 F = Foreman H = Hall Monitor S = Soccer C = Coastguard

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Summary: ◦ Proposed a MINVAR bit allocation for multiplexed video sequences ◦ The MINVAR allocation leverages on the  -domain model (but works on any exponential model, i.e. at high rates) ◦ The coding efficiency loss w.r.t. MINAVE is, on average, of 0.5 dB  Future work: ◦ Apply the MINVAR approach to Scalable Video Coding  solve a discrete optimization problem…

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007  Z. He and S. K. Mitra, “Optimum bit allocation and accurate rate control for video coding via  -domain source modeling,” IEEE Trans. Circuits Syst. Video Technol., vol. 12, pp. 840– 849, November  Z. He, W. Zeng, and C.W. Chen, “Low-pass filtering of rate-distortion functions for quality smoothing in real-time video communication,” IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 8, pp. 973–981, 2005.

A  -domain Rate Controller for Multiplexed Video Sequences PCS 2007, November 7, 2007 Questions?