MPEG-4 Video Coding System 學生 : 鄭馥銘 教師 : 楊士萱

Outline Introduction Decoding process overview Motion Compensation Code review Future work

Introduction Purpose ISO/IEC was developed in response to the growing need for coding method that can facilitate access to visual object in natural and synthetic moving picture.

Introduction (Profiles and levels)

Introduction – Structure of coded video data Macroblock Motion_vector VOP Y CbCr

Outline Introduction Decoding process overview Motion Compensation Code review Future work

A high level view of visual decoding Entropy Decoding FBA Decoding Still Texture Decoding Mesh Decoding Motion compensation Decoding Shape Decoding Texture Decoding To composition

A high level view of visual decoding Entropy Decoding FBA Decoding Still Texture Decoding Mesh Decoding Motion compensation Decoding Shape Decoding Texture Decoding To composition

Simplified video decoding process Motion Decoding Motion Compensation Previous Reconstructed VOP Variable Length Decoding Inverse Scan Inverse DC & AC Prediction Inverse Quantization IDCT VOP Reconstruction Coded bit stream (motion) Coded bit stream (texture)

Visual information – logical structure Visual Object Sequence Header VO 1 Header VO 2 header VO 1 VOL 1 Header VO 1 VOL 2 Header VO 2 VOL 1 Header Elementary Stream Visual Object 1 Layer1 Elementary Stream Visual Object 1 Layer2 Elementary Stream Visual Object 1 Layer1

Visual information – logical structure Visual Object Sequence Header VO 1 Header VO 2 header VO 1 VOL 1 Header VO 1 VOL 2 Header VO 2 VOL 1 Header Elementary Stream Visual Object 1 Layer1 Elementary Stream Visual Object 1 Layer2 Elementary Stream Visual Object 1 Layer1

Visual information – logical structure Visual Object Sequence Header VO 1 Header VO 2 header VO 1 VOL 1 Header VO 1 VOL 2 Header VO 2 VOL 1 Header Elementary Stream Visual Object 1 Layer1 Elementary Stream Visual Object 1 Layer2 Elementary Stream Visual Object 1 Layer1

Visual Object Sequence Header VisualObjectSequence(){ do{ visual_object_sequence _start_code32 profile_and_level_indication8 visualObject() }while(next_bits()!=visual_object_sequence_end_code) visual_object_sequence_end_code32 }

Visual information – logical structure Visual Object Sequence Header VO 1 Header VO 2 header VO 1 VOL 1 Header VO 1 VOL 2 Header VO 2 VOL 1 Header Elementary Stream Visual Object 1 Layer1 Elementary Stream Visual Object 1 Layer2 Elementary Stream Visual Object 1 Layer1

VisualObject Header VisualObject(){ visual_object_start_code32 visual_object_type4 if(visual_object_type== ” Video ID ” ){ video_object_start_code32 videoObjectLayer(); } if((visual_object_type== “ mesh ID ” ) MeshObject(); …. shape,FBA,3D mesh,still texture }

Visual information – logical structure Visual Object Sequence Header VO 1 Header VO 2 header VO 1 VOL 1 Header VO 1 VOL 2 Header VO 2 VOL 1 Header Elementary Stream Visual Object 1 Layer1 Elementary Stream Visual Object 1 Layer2 Elementary Stream Visual Object 1 Layer1

VideoObjectLayer() VideoObjectLayer(){ if(next_bits()==video_object_layer_start_code){ short_video_header=0 Width,Height,Interlaced ……… do{ VideoObjectPlane(); }while(next_bits()==vop_start_code) } else{ short_video_header=1 do{ video_plane_with_short_header() }while(next_bits()==short_video_start_marker) }

Visual information – logical structure Visual Object Sequence Header VO 1 Header VO 2 header VO 1 VOL 1 Header VO 1 VOL 2 Header VO 2 VOL 1 Header Elementary Stream Visual Object 1 Layer1 Elementary Stream Visual Object 1 Layer2 Elementary Stream Visual Object 1 Layer1

Outline Introduction Decoding process overview Motion Compensation Code review Future work

General motion vector decoding process r_size=vop_fcode-1 f=1<<r_size gight=(32*f)-1 low=(-32*f) range=(64*f) MVDx=horizontal_mv_data MVDy=vertical_mv_data MVx=Px+MVDx Mvy=Py+MVDy Range control

General motion vector decoding process Px=Median(MV1x,MV2x,MV3x) Py=Median(MV1y,MV2y,MV3y)

Overlapped motion compensation H0 H1H2 Obmc_disable=0

Outline Introduction Decoding process overview Motion Compensation Code review Future work

Code review void main(){ CVideoObjectDecoder* pvodec [2];.. while (iEof != EOF){ if (main_short_video_header) iEof = pvodec [BASE_LAYER] -> h263_decode (); else iEof = pvodec [BASE_LAYER] -> decode ();.. dumpFrame(); } visualObject() videoObjectLayer()

Code review CVideoOjectDecoder :: decode (){ decodeVOPHead ().. updateAllRefVOPs ();.. decodeVOP (); } VideoObjectPlane()

Code review CVideoOjectDecoder :: decodeVOP (){ if(bDataPartitioning){ if (vopPredType == PVOP) decodePVOP_DataPartitioning (); else if(vopPredType == IVOP) decodeIVOP_DataPartitioning (); else decodeBVOP (); } else{ if (vopPredType == PVOP) decodePVOP(); else if(vopPredType == IVOP) decodeIVOP(); else decodeBVOP(); }

Code review Void CVideoObjectDecoder::decodePVOP (){ while (uiGobNumber < uiNumberOfGobs){ for (iMBY = iMBYstart; iMBY < iMBYstop; iMBY++){ decodeMBTextureHeadOfPVOP(); if(!pmbmd -> m_bMCSEL) decodeMV(); for (iMBX = iMBXstart; iMBX < iMBXstop; iMBX++){ if (!pmbmd->m_bSkip) decodeTextureInterMB (pmbmd); if (m_uiSprite == 0 || m_uiSprite == 2) decodeMBTextureHeadOfPVOP(); if(!pmbmdRight -> m_bMCSEL) decodeMV(); ….. motionCompMB(); motionCompUV(); }

Future work Write a general and integrity-ME program Collate papers about DIS and analysis what methods may good for my application.