1. 7. Design and Performance

2. 7.1 INTRODUCTION Good Performance = Careful Design x Careful Choice of Coding Parameters [ Issues on this Chapter ] The design of software or hardware implementations Comparing the performance of video coding algorithms Practical bit-rate control and network transport issues

3. 7.2 FUNCTIONAL DESIGN Good Compression & Computational Performance 를 유지하면서 MPEG4/H.264 standard 를 만족시키려면 ? –Requires careful design of the CODEC functional blocks – 좋은 rate/distortion performance 을 얻을 수 있음 (7.4.3 절 ) Functional Block Design 시 염두에 두어야 할 사항 –Functions such as motion estimation, transforms, entropy coding can be highly computaionally intensive. –Many practical platforms for video compression are power-limited or computation-limited. In this section we discuss practical approaches and tradeoffs in the design of the main functional blocks of a video CODEC.

4. 7.2 FUNCTIONAL DESIGN 7.2.1 Segmentation The object-based functionalities of MPEG-4 require a video scene to be segmented into objects. Manual Segmentation –A human operator identifies manually the borders of each object in each source video frame –a very time-consuming process / suitable for ‘offline’ video content –Appropriate for segmentation of an important visual object ; be viewed by many users, re-used many times in different composed video sequences Semi-automatic Segmentation –A human operator identifies objects and object boundaries in one frame –A segmentation algorithm refines the object boundaries and tracks the video objects through successive frames of the sequence Fully-automatic Segmentation –An algorithm attempts to carry out a complete segmentation of a visual scene without any user input –Based on spatial characteristics(edges) and temporal characteristics(object motion between frames)

5. 7.2 FUNCTIONAL DESIGN 7.2.2 Motion Estimation Motion estimation is the process of selecting an offset to a suitable reference area in a previously coded frame. Motion estimation is carried out in a video encoder and has a significant effect on CODEC performance.

6. 7.2 FUNCTIONAL DESIGN 7.2.2.1 Block-Based Motion Estimation- Energy Measures Motion estimation aims to find a ‘match’ to the current block(or region) that minimizes the energy in the motion compensated residual. Evaluating the residual energy at a number of different offsets. >> The choice of measure for energy affects computational complexity and the accuracy.

7. 7.2 FUNCTIONAL DESIGN 7.2.2.1 Block Based Motion Estimation- Energy Measures

8. 7.2 FUNCTIONAL DESIGN 7.2.2.1 Block Based Motion Estimation- Full Search Full Search Motion Estimation involves evaluating equation 7.3 (SAE) at each point in the search window. It is computationally intensive since the energy measure must be calculated at every locations.

9. 7.2 FUNCTIONAL DESIGN 7.2.2.1 Block Based Motion Estimation - ‘Fast’ Search Algorithms Preferable for computation-limited or power-limited applications Calculating the energy measure(SAE) at a subset of locations Three Step Search (TSS or N-step Search)

10. 7.2 FUNCTIONAL DESIGN 7.2.2.1 Block Based Motion Estimation - ‘Fast’ Search Algorithms Nearest Neighbors Search Algorithm has low computational complexity but closely approaches the performance of Fully Search Algorithm within the framework of MPEG-4 Simple Profile

11. 7.2 FUNCTIONAL DESIGN 7.2.2.1 Block Based Motion Estimation - Sub-pixel Motion Estimation Increased fractional accuracy can provide a better match and reduce the energy in the motion-compensated residual. Ex) ½ pixel vectors in MPEG-4 Simple Profile ¼ pixel vectors in Advanced Simple Profile and H.264 Sub-pixel Motion Estimation requires the encoder to interpolate between integer sample positions in the reference.

12. 7.2 FUNCTIONAL DESIGN 7.2.3 DCT/IDCT - 8 × 8 DCT 8x8 FDCT 를 위한 식 (3.4) 는 64 x 64 = 4096 개의 곱셈과 누적연산이 필요 행렬 형식 Y = AXA T 에서 2D 변환은 두 단계로 나눌 수 있음 : AX 를 계산한 다음, 행렬 A T 를 곱함 1D FDCT 는 식 (7.4) 와 같은데 여기서 f i 는 N 개의 입력 샘플이고 F X 는 N 개의 출력 계수

13. 7.2 FUNCTIONAL DESIGN 7.2.3 DCT/IDCT - 8 × 8 DCT (cont.) 2D FDCT 식 (3.4) 를 다시 정리해보면, 두 개의 1D 변환 ( 식 7.5) 으로 구성 2D FDCT 는 입력 행렬의 각 열에 대한 1D FDCT 를 계산한 다음, 변환 결 과의 각 행에 대해 1D FDCT 를 계산하여 수행 2D IDCT 는 유사한 방법으로 변경될 수 있다.( 식 7.6) 코싸인 함수의 대칭성을 이용하여 다음과 같이 간략화

14. 7.2 FUNCTIONAL DESIGN 7.2.3 DCT/IDCT - 8 × 8 DCT

15. 7.2 FUNCTIONAL DESIGN 7.2.3 DCT/IDCT - H.264 4 × 4 Transform The integer IDCT approximations specified in the H.264 standard have been designed to be suitable for fast, efficient software and hardware implementation The original proposal for the forward and inverse transforms describes alternative implementation using –(i) a series of shifts and additions –(ii) a flowgraph algorithm –(iii) matrix multiplications

16. 7.2 FUNCTIONAL DESIGN 7.2.3 DCT/IDCT - Object Boundaries In a MPEG-4 CODEC, residual coefficients in a boundary MB are coded using the 8 x 8 DCT

17. 7.2 FUNCTIONAL DESIGN 7.2.4 Wavelet Transform The DWT(Discrete Wavelet Transform) was chosen for MPEG-4 still texture coding because it can out-perform block-based transforms for still image coding

18. 7.2 FUNCTIONAL DESIGN 7.2.5 Quantise/Rescale 스칼라 Quantize 와 Rescale 은 constant parameters 의 곱셈, 나눗셈 으로 구현 가능하다. 일반적으로, 곱셈은 복잡한 연산이므로 quantization 와 rescaling 의 곱셈을 각각 forward transform 과 inverse transform 에 통합시키면 performance 상 약간의 이득을 얻을 수 있다. H.264 에서는 quantization 의 과정이 transform 과정에 통합되었다.

19. 7.2 Functional Design 7.2.6 Entropy Coding – Variable Length Encoding MPEG-4 Visual/H.264 에서는 각 데이터 심볼을 인코딩하는 데 필요한 VLC 가 표준안에 정의되어 있음 실제 전송환경에서는 인코더에 의해 생성된 일련의 VLC 를 Byte 또는 word 의 스트림으로 전환할 필요가 있음

20. 7.2 Functional Design 7.2.6 Entropy Coding – Variable Length Decoding VLC 를 Decoding 하기 위해서는 유효한 코드워드를 찾기 위해 전송 받은 비트 스트림을 분석 및 추출하여 적절하게 Decoding 하는 과정이 필요 올바른 코드워드 테이블을 선택하기 위해 Decoder 가 현재의 Context 를 알고 있어야 함