1. SVC(Scalable Video Coding) & JSVM(Joint Scalable Video Model)
Kyumin Jeong
Computer Communications Lab.
October 1, 2007

2. What is SVC? Scalable Video Coding (SVC)
Project of the Joint Video Team (JVT)
ISO/IEC Moving Pictures Experts Group (MPEG)
ITU-T Video Coding Experts Group (VCEG)
As an amendment of the H.264/MPEG-4 AVC standard
One coding for multiple usage
Universal Multimedia Access paradigm
하나의 비디오 스트림으로 다양한 전송 네트워크와 다양한 수신 단말에 적응적 서비스가 가능한 비디오 부호화 방법
One source, use Anytime, Anywhere, Anydevice

3. What is SVC?
Scalable Video Coding (SVC)

4. Scalability

5. JSVM JSVM (Joint Scalable Video Model) software
Reference software for the Scalable Video Coding (SVC) project
Written in C++
Provided as source code
Can be obtained via a CVS server
Preparations for using JSVM (윈도우 환경)
Python
WinCVS를 사용하기 위해서 필요함, tcl/tk는 Python에 포함
WinCVS
WinCVS 실행시 발생하는 문제 해결 (아래 링크)
WinCvs could not find Python 2.1 installed

6. Accessing the latest JSVM Software
CVS (Concurrent Versions System)
최신 JSVM 소프트웨어를 사용하기 위해 필요
SVC 프로젝트 진행중 => JSVM 업데이트
CVS 서버 접속을 위한 세팅
authentication
pserver
host address
garcon.ient.rwth-aachen.de
path
/cvs/jvt
user name
jvtuser
password
jvt.Amd.2
module name
jsvm or jsvm_red

7. Accessing the latest JSVM Software
Using a command line CVS client
Admin - Command Line 선택 후, 아래 명령어를 차례대로 입력
1. cvs –d login
2. cvs –d checkout jsvm
실행 후 결과 :
C:\JSVM 생성

8. Structure of the CVS repository for the JSVM Software
Folder
Content
bin
location of binaries after building the software
lib
location of libraries after building the software
JSVM
source code and project files for the JSVM software
JSVM/H264Extension/build
workspaces and makesfiles
JSVM/H264Extension/data
basic examples of encoder configuration files
JSVM/H264Extension/include
includes files that are required by other libraries inside the JSVM project
JSVM/H264Extension/src
include and source files for all libraries and test projects
JSVM0-config-samples
usage examples
MVC-Configs
configuration examples for Multiple View Coding
Validation
validation scripts for checking the JSVM software
C:\jsvm\
Software Manual.doc <사용법 등 참고>

9. Building the JSVM software
Windows32 platform with Microsoft Visual Studio
C:\jsvm\JSVM\H264Extension\build\windows
Microsoft Visual Studio .NET 2003 (VC7) workspace
H264AVCVideoEncDec.sln
Microsoft Visual Studio .NET 2005 and 2006 (VC8)
H264AVCVideoEncDec_vc8.sln
Microsoft Visual Studio 6 workspace
H264AVCVideoEncDec.dsw
Build all project files by selecting Build→Batch Build

10. Building the JSVM software
Result
The versions with a “d” before the dot
Debug mode
Without a “d” before the dot
Release mode

11. Information on binaries and libraries

12. Information on binaries and libraries

13. Information on binaries and libraries

14. Usage and Configuration of the JSVM Software

15. 테스트 파일 URL Contents 내용 Contents 종류
ftp://ftp.tnt.uni-hannover.de/pub/svc/testsequences/
Contents 내용
City, Crew, Football, Foreman, Bus, Ice, Harbour, Mobile, Soccer
Contents 종류
4CIF, CIF, QCIF에 대해 60, 30, 15, 7.5 Frame/Sec

16. 테스트 인코딩 및 디코딩 테스트 과정 JSVM SW 및 Test Contents 구하기 Build a JSVM S/W
Manual 참조하여 Configure file을 수정
이 때 가급적 test directory를 만들어 이 곳에 executables, test contents, cfg 파일들을 모아놓고 처리하는 것이 편함 (단, versioning에 주의)
DownConvertStatic.exe 수행
필요한 경우 Test Contents의 변형을 위해 down convert 시킴
H264AVCEncoderLibTestStatic.exe 수행
Multi-Layer encoding인 경우 (e.g) test.264 파일생성 됨
BitStreamExtractorStatic.exe 수행
원하는 layer의 bitstream을 추출
H264AVCDecoderLibTestStatic.exe 수행
각 layer별 인코딩 된 미디어 데이터를 decoding하여 YUV 포맷 화
PSNRStatic.exe
PSNR 측정

17. Resampler – DownConvertStatic.exe
Spatial/temporal resampling
Usage
DownConvertStatic <win> <hin> <in> <wout> <hout> <out> [<method> [<t> [<skip> [<frms> ]]]] [[-crop <args>] [-phase <args>]]
Examples
#Down-sampling a 4CIF 60Hz sequence to a QCIF 15Hz sequence using the dyadic method
DownConvertStatic CIF60.yuv QCIF15.yuv 1 2
#Resampling of a CIF 30Hz sequence to a 528x432 15Hz sequence using the normative
upsampler.
DownConvertStatic CIF30.yuv 528  432  528x432_15.yuv 0 1
method : rescaling methods (default: 0)
0: normative upsampling
non-normative downsampling (JVT-R006)
1: dyadic upsampling (AVC 6-tap (1/2 pel) on odd luma samples
dyadic downsampling (MPEG-4 downsampling filter)
Different resampling modes, methods
Parameters that are specific to Extended Spatial Scalability (ESS), Cropping Window, Chroma Phase Shift features
-Method : resampling mode, t : the number of temporal down-sampling stages, skip : how many frames skipped
-Frms : maximum number of output frames
2: crop only
3: upsampling (Three-lobed Lanczos-windowed sinc)
4: upsampling (JVT-O041: AVC 6-tap (1/2 pel) + bilinear 1/4 pel)

18. Resampler – DownConvertStatic.exe
Cropping window
Generating the output sequence as a cropped area of the input
Cropping parameters
crop_type
0 : for the entire sequence
1 : for each picture of the sequence
crop_file
Filename of the file containing the cropping parameters
-crop crop_type crop_file
<x_orig> <y_orig> <crop_width> <crop_height> // parameters for frame 0
<x_orig> <y_orig> <crop_width> <crop_height> // parameters for frame 1
<x_orig> <y_orig> <crop_width> <crop_height> // parameters for frame 2
<x_orig> <y_orig> <crop_width> <crop_height> // parameters for frame 3
...
Different resampling modes, methods
Parameters that are specific to Extended Spatial Scalability (ESS), Cropping Window, Chroma Phase Shift features
-Method : resampling mode, t : the number of temporal down-sampling stages, skip : how many frames skipped
-Frms : maximum number of output frames
DownConvertStatic.exe p50.yuv SD25.yuv 0 –crop 0 crop.txt 1
content of crop.txt:

19. Resampler – DownConvertStatic.exe
Chroma phase shift management
-phase option
in_uv_ph_x, in_uv_ph_y, out_uv_ph_x and out_uv_ph_y
The horizontal and vertical phase shift, the chroma components in comparison to the luma component
Input : Luma samples for the input sequence
Output : Chroma phase shift of the output sequence
Value range of -1 to 1
Resampling mode
-resample_mode option
Allow the resampling of both progressive or interlace material
Value range of 0 to 5

20. Encoder - H264AVCEncoderLibTestStatic
Generating AVC or SVC bit-streams
Using the encoder
No guarantee rate-distortion efficient coding
For obtaining optimized encoding results, the encoder configuration has to be carefully specified
Not provide a rate-control
The bit-rate needs to be controlled by selecting appropriate quantization parameters
Single-layer coding mode and Scalable coding mode
Single-layer bit-stream can also be generated in the scalable coding mode
Encoding mode is specified by the parameter AVCMode
Usage
H264AVCEncoderLibTestStatic.exe –pf <mcfg> [command line options]

21. Encoder - H264AVCEncoderLibTestStatic
Single-layer coding mode
AVCMode = 1
AVC compatible bit-stream is generated
Encoder configuration file in single-layer coding mode

22. Encoder - H264AVCEncoderLibTestStatic
Command line options
Scalable coding mode
AVCMode = 1
One or more layer configuration files have to be specified inside the main configuration files
-bf (bitstream)
The parameter bitstream specifies the filename for the bit-stream to be generated.
-frms (frames)
The parameter frames specifies the number of frames of the input sequence to be encoded.
-pf (config)
The parameter config specifies the name of the config file to be used. -h
Prints out a brief help on using the encoder.
C:\jsvm\JSVM0-config-samples\Contrib-examples\JVT-O018\Munich-Test-Points\cfg_ags

23. Other JSVM Softwares Decoder - H264AVCDecoderLibTestStatic.exe
SVC to AVC Bit-stream Rewriter - AvcRewriterStatic.exe
Converts an SVC bitstream to an AVC bitsream
Usage
H264AVCDecoderLibTestStatic <str> <rec> [-ec <ec>] [<maxPOCDiff>]
str: bit-stream file (input)
rec: reconstructed video sequence (output)
ec: error concealment method (1-3), 0 means no error concealment
maxPocDiff: maximum difference of POC values of successive output frames
Usage
AvcRewriterStatic <svcstr> <avcstr>
svcstr: bit-stream file (input)
avcstr: rewritten avc bit-stream file (output)

24. Other JSVM Softwares
Bit-stream extractor - BitStreamExtractorStatic.exe
Extract sub-streams of an AVC or SVC stream
Streams with a reduced spatial and/or temporal resolution and/or a reduced bit-rate
Usage
BitStreamExtractorStatic [-pt trace] <in>
[<out> [[-e] [-ql | -qlord]] | [-sl] | [[-l] [-t] [-f]] | [-b] | [-et] ]
Extraction of a scalable layer
BitStreamExtractorStatic input.svc output.svc –sl 7
Extraction of a scalable sub-stream using the general option –e
BitStreamExtractorStatic input.svc output.svc –e

25. Quality level assigner – QualityLevelAssignerStatic.exe
Other JSVM Softwares
Quality level assigner – QualityLevelAssignerStatic.exe
Embed information about quality layers inside a bit-stream
Quality layer information can be employed by the extraction process
To optimize the rate-distortion efficiency of the extracted sub-stream
Usage
QualityLevelAssignerStatic -in Input -org L Original [-org L Original]
[-out Output [-sei] | -wp DatFile] [-dep | -ind] [-mlql] or
QualityLevelAssignerStatic -in Input -out Output -rp DatFile [-sei]
-in Input input bit-stream
-out Output output bit-stream with determined quality layer id's
-org L Original - original image sequence for layer L
-wp DatFile - data file for storing rate and distortion values
-rp DatFile - data file with previously computed rate and
distortion values
-sei provide quality layer info using SEI mesages
-dep determine only dependent distortions
(speed-up by factor of 2, slight coding eff. losses)
-ind determine only independent distortions
-mlql determine Multi Layer Quality Layer Ids

26. MCTF pre-processing tool – MCTFPreProcessorStatic.exe
Other JSVM Softwares
MCTF pre-processing tool – MCTFPreProcessorStatic.exe
Pre-filtering image sequences
PSNR tool – PSNRStatic.exe
Measuring the Peak-Signal-To-Noise-Ratio (PSNR) between two sequences
Calculating the bit-rate
Usage
MCTFPreProcessor -w Width -h Height -f frms -i Input -o Output
[-gop GOPSize] [-qp QP]
-w Width - frame width in luma samples (multiple of 16)
-h Height - frame height in luma samples (multiple of 16)
-gop GOPSize - GOP size for MCTF (2,4,8,16,32,64, default: 16)
-qp QP QP for motion estimation and mode decision (>0, default: 26)
PSNRStatic org.yuv rec.yuv 0 0 str.svc 15 2>PSNR.txt
type PSNR.txt
128,00 32,23 38,
Usage
PSNRStatic <w> <h> <org> <rec> [<t> [<skip> [<strm> <fps>]]]
- rec: reconstructed file
- t: number of temporal downsampling stages (default: 0)
- strm: coded stream
- frms: frames per second
(1) bit-rate in kbit/s, (2) Y-PSNR in dB – luminance component, (3) U-PSNR in dB – chrominance component U or Cb,
(4) V-PSNR in dB – chrominance component V or Cr

27. Other JSVM Softwares Fixed QP encoder - FixedQPEncoderStatic.exe
Find the basis quantization parameter to meet
A rate or a quality constraint for single layer coding
The rate or quality constraints for the layers of a scalable stream
SIP Analyser tool – SIPAnalyser.exe
Make the selective inter-layer prediction decision
Usage
FixedQPEncoderStatic <rc_cfg>
rc_cfg : the rate control parameter file configuring the fixed QP encoder
Usage
SIPAnalyser.exe <sip_cfg> [FileLabel]
sip_cfg : the SIP parameter file
FileLabel : A suffix to the output filename

28. Use Examples as a Brief Tutorial - Single Layer Coding

29. Original Sequences Generation
Goal
YUV 시퀀스를 시공간적으로 다운샘플된 4CIF, CIF로 생성
The original CIF resolution sequences
BUS_352x288_30_orig_01.yuv,
FOREMAN_352x288_30_orig_01.yuv
FOOTBALL_352x288_30_orig_01.yuv
MOBILE_352x288_30.yuv
URL : ftp.tnt.uni-hannover.de/pub/svc/testsequences/

30. Original Sequences Generation
How to generate dowsampled versions
Test file : SOCCER_352x288_30_orig_02.yuv
Change : SOCCER_352x288_30.yuv
Down-sampling original CIF 30Hz sequence to QCIF 15Hz sequence for the CIF scenario

31. Single Layer Coding Single layer coding
Encoded bit-stream does not provide several spatial resolutions or several bit-rates for a specific spatio-temporal resoultion
Generated bit-stream contains spatial or CGS/MGS enhancement layer
Preparation for encoding “SOCCER_176x144_15.yuv”
아래 폴더에서 encoder.cfg와 layer0.cfg를 C:\jsvm\bin에 복사
C:\jsvm\JSVM\H264Extension\data
encoder.cfg와 layer0.cfg를 수정

32. Single Layer Coding encoder.cfg 수정 layer0.cfg 수정 아래 나와 있는 옵션들을 각각 수정
# JSVM Main Configuration File
OutputFile test # Bitstream file
FrameRate # Maximum frame rate [Hz]
FramesToBeEncoded # Number of frames (at input frame rate)
GOPSize # GOP Size (at maximum frame rate)
BaseLayerMode # Base layer mode (0: AVC w larger DPB,
# 1:AVC compatible, 2:AVC w subseq SEI)
SearchMode # Search mode (0:BlockSearch, 4:FastSearch)
SearchRange # Search range (Full Pel)
NumLayers # Number of layers
LayerCfg layer0.cfg # Layer configuration file
# JSVM Layer Configuration File
InputFile SOCCER_176x144_15.yuv # Input file
SourceWidth # Input frame width
SourceHeight # Input frame height
FrameRateIn # Input frame rate [Hz]
FrameRateOut # Output frame rate [Hz]

33. Single Layer Coding Encoder call for single-layer coding
Result : test.264 and rec_layer0.yuv

34. Single Layer Coding
Extracting a temporal sub-sequence

35. Single Layer Coding
Decoding a temporal sub-sequence

36. Single Layer Coding
Use of PSNR tool