How to use JSVM toolkit orig. Feb – updated July Eun-Seok Ryu

Access the latest JSVM software Option 1. Use CVS –Use WinCVS or any other CVS client User ID : "jvtuser" Password : "jvt.Amd.2". –The path is "/cvs/jvt", and the module name is "jsvm“. For example, with a command line client, you can check out the software using the following commands. cvs –d aachen.de:/cvs/jvt login cvs –d checkout jsvm (for further information, you can see the JSVM manual) Option 2. Use web browser ( currently, it is not working ) –Follow the link given below: User ID : jvtuser Password : jvt.Amd.2

Download test sequences URL : ftp://ftp.tnt.uni- hannover.de/pub/svc/testsequences/ftp://ftp.tnt.uni- hannover.de/pub/svc/testsequences/ Contents : –City, Crew, Football, Foreman, Bus, Ice, Harbour, Mobile, Soccer Resolution and frame rate : –4CIF, CIF, QCIF : 60, 30, 15, 7.5 fps

Understand JSVM versioning In this slide, I used JSVM 8.1 (Feb. 07, 2007, CVS tag: JSVM_8_1) Current version (July 2011) : JSVM You can download any version using CVS tool. JSVM 9.8 was last version supporting error concealment (EC) options –As a decoding option, you could use “–ec 2 (frame copy)”. JSVM has three contents –changes : bug fix histories –SW manual –source codes

Structure of the CVS repository for the JSVM (1) foldercontent binlocation of binaries after building the software More information about the binaries are given in section 1.4. liblocation of libraries after building the software More information regarding the libraries are given in section 1.4. JSVMsource code and project files for the JSVM software All files that are required for building and using the JSVM software are contained in this folder. JSVM/H264Extension/buildworkspaces and makesfiles Workspaces are provided for Microsoft Visual Studio 6 and Microsoft Visual Studio.NET. Makefiles are provided for Linux. JSVM/H264Extension/databasic examples of encoder configuration files These examples are not guaranteed to be complete or to contain all available configuration parameters. The examples only serve as a basis for writing encoder configuration files.

Structure of the CVS repository for the JSVM (2) JSVM/H264Extension/includeincludes files that are required by other libraries inside the JSVM project Only include files for classes that are required by other libraries or test projects should be placed into this folder. This folder contains a subfolder for the frequently used library H264AVCCommonLib. Include files of classes that are only required inside a library or test project are contained in the library or test project folder in JSVM/H264Extension/src. JSVM/H264Extension/srcinclude and source files for all libraries and test projects This folder contains all source files and the include files that are only required inside a library or test project. This folder is organized in appropriate sub-folders. JSVM0-config-samplesusage examples This folder contains usage examples with configuration files, scripts as well as some text descriptions. These usage examples are not updated with each change of the JSVM software, and hence they are not guaranteed to be up-to-date.

Structure of the CVS repository for the JSVM (3) MVC-Configsconfiguration examples for Muliple View Coding This folder contains examples including configuration files and scripts for using the JSVM software for Multi View Coding (MVC). Validationvalidation scripts for checking the JSVM software Perl scripts for validating the JSVM software are provided in this folder. These scripts shall be run after each modification of the JSVM software. They have been designed to check the functionality of integrated tools. More information on how to run these scripts are given in 4.3. changes.txtchanges log file This file described the (main) changes from one CVS version to the next. It starts with JSVM version 4.0 (CVS tag: JSVM_4_0).

Build JSVM Software (1) Build environment –Visual C Workspace file –H264AVCVideoEncDec.ds w Build → Batch Build : Rebuild All –Linux (gcc ver. 4) > cd JSVM/H264AVCExtension/ build/linux > make –.net Workspace file : –H264AVCVideoEncDec.sln (VC 7) H264AVCVideoEncDec_vc 8.sln (VC 8) Build → Batch Build : Rebuild All The executable which has ‘d’ in their file name is for debugging (compiled with debug option). ===== binaries ===== bin/BitStreamExtractorStatic.exe bin/BitStreamExtractorStaticd.exe bin/DownConvertStatic.exe bin/DownConvertStaticd.exe bin/FixedQPEncoderStatic.exe bin/FixedQPEncoderStaticd.exe bin/H264AVCDecoderLibTestStatic.e xe bin/H264AVCDecoderLibTestStaticd. exe bin/H264AVCEncoderLibTestStatic.ex e bin/H264AVCEncoderLibTestStaticd.e xe bin/MCTFPreProcessorStatic.exe bin/MCTFPreProcessorStaticd.exe bin/PSNRStatic.exe bin/PSNRStaticd.exe bin/QualityLevelAssignerStatic.exe bin/QualityLevelAssignerStaticd.exe bin/SIPAnalyser.exe bin/SIPAnalyserd.exe ===== libraries === lib/H264AVCCommonLibStatic.lib lib/H264AVCCommonLibStaticd.li b lib/H264AVCDecoderLibStatic.lib lib/H264AVCDecoderLibStaticd.li b lib/H264AVCEncoderLibStatic.lib lib/H264AVCEncoderLibStaticd.li b lib/H264AVCVideoIoLibStatic.lib lib/H264AVCVideoIoLibStaticd.lib

Build JSVM Software (2) Recent version supports visual studio 2010 environment with “H264AVCVideoEncDec_vc10.sln”

JSVM Libraries (1) librarydescription H264AVCCommonLibStaticcommon lib This library provides classes that are used by both the encoder and decoder, as for example macroblock data structures, buffers for storing and accessing image data, or algorithms for deblocking. H264AVCEncoderLibStaticencoder lib This library provides classes that are only used by the encoder. For example, it includes classes for motion estimation, mode decision, and entropy encoding. H264AVCDecoderLibStaticdecoder lib This library provides classes that are only used by the decoder. For example, it includes classes for entropy decoding and bit- stream parsing. H264AVCVideoIoLibStaticio lib This library provides classes for reading and writing NAL units in the byte-stream format as well as classes for reading and writing raw video data.

JSVM Libraries (2) executabledescription DownConvertStaticresampler The resampler can be used for spatial/temporal resampling (up- sampling or down-sampling) of video sequences. More information on using the resampler are provided in section 2.1. H264AVCEncoderLibTestStaticAVC/SVC encoder The encoder can be used for generating single-layer (AVC) or scalable (SVC) bit-streams. More information on using the encoder are provided in section 2.2. H264AVCDecoderLibTestStaticSVC decoder The decoder can be used for decoding AVC or SVC bit-streams and reconstructing raw video sequences. More information on using the decoder are provided in section 2.3 BitStreamExtractorStaticbit-stream extractor The bit-stream extractor can be used for extracting sub-bitstreams with a lower spatio-temporal resolution and/or bit-rate from a global scalable (SVC) bit-stream. More information on using the bit-stream extractor are provided in section 2.4.

JSVM Libraries (3) QualityLevelAssignerStaticquality level assigner The quality level assigner can be used for generating a bit-stream with additional quality layer information given a scalable bit-stream. Beside the additional quality layer information input and output bit- stream are identical. More information on using the quality layer assigner are provided in section 2.5. MCTFPreProcessorMCTF pre-processor The MCTF pre-processing tool can be used for pre-filtering image sequences. More information on using the MCTF pre-processor are provided in section 2.6. PSNRStaticPSNR tool The PSNR tool can be used for measuring the PSNR between two raw video sequences. In addition it can be used for measuring the bit-rate of a given bit-stream. More information on using the PSNR tool are provided in section 2.6. FixedQPEncoderStaticfixed QP encoder This tool can be used for controlling the encoder and adjusting the bit-rate of the generated bit-stream. More information on using this tool are provided in section 2.8.

JSVM Libraries (4) SIPAnalyser The SIPAnalyser tool can be used to make the selective inter-layer prediction decision. More information on using the PSNR tool are provided in section 2.9.

Test encoding and decoding Steps : –Download JSVM and test sequences –Build the JSVM –Modifying config files (*.cfg) based on JSVM manual You can use original config files in JSVM SW –(option) DownConvertStatic.exe If you want to do (1)downsampling, (2)upsampling, or (3) cropping, use this tool. –H264AVCEncoderLibTestStatic.exe There are many encoding options This makes encoded SVC file such as ‘test.264’ –(option) packet loss testing with additional tools –BitStreamExtractorStatic.exe Extract specific layers what I want from ‘test.264’ –H264AVCDecoderLibTestStatic.exe This makes reconstructed YUV file such as ‘test.yuv’ –PSNRStatic.exe Measuring PSNR

Test en/decoding steps (Simple) H264AVCEncoderLibTestStatic.exe BitStreamExtractorStatic.exe H264AVCDecoderLibTestStatic.exe Test Sequence (.YUV) Config file SVC-encoded file (.svc) Extracted SVC-encoded file (e.g. Base layer only or Base+Enh1) Reconstructed Sequence (.YUV) or layers Reconstructed Sequence (.YUV) Reconstructed Sequence (.YUV) or

Test en/decoding steps for packet loss simulation (e.g. UEP) With JSVM ver. 9.8 error conce alment (EC) option (recent vers ion does not support EC) or Open-source SVC decoder (this supports frame copy-based EC) You can use packet loss simula tor, but it drops NAL-based pa cket. You need to adjust each NAL size or use your own tool for accurate testing. (jvt-q069 : available: /jvt-site/ Nice/JVT-Q069.zip)

CVS Usage ( free tortoise CVS app ) (1) (1) input (3) click “Fetch list” (2) set “JSVM” (3) click “OK”

Building Executable Files with VS 6.0 H264AVCVideoEncDec.dsw

Example of Test Contents Example of the Test Clip ‘Foreman’ –FOREMAN_176x144_7.5_orig_01_yuv –FOREMAN_176x144_15_orig_01_yuv –FOREMAN_352x288_15_orig_01_yuv –FOREMAN_352x288_30_orig_01_yuv

Main Configure File – encoder.cfg

Main Configure File – encoder.cfg (2)

Layer Configure File - Layer 0.cfg

Layer Configure File – Layer1.cfg

Test Encoding >H264AVCencoderLibTestStatic.exe –pf encoder.cfg

Bitstream Extract (1)

Bitstream Extract (2) BitstreamExtractorStatic.exe –pt trace3_sl0.txt test.264 test3_sl0.out –sl 0 BitstreamExtractorStatic.exe –pt trace3_sl2.txt test.264 test3_sl2.out –sl 2

Example of Trace File 1.you can test packet dropping(loss) simulation by deleting some lines of text file. Bitstream extractor’s option ‘-et trace’ uses a (modified) packet trace file for bit-strea m extraction. 2.‘Packet loss simulator(jvt-q069 : available: 10 Nice /JVT-Q069.zip)’ omits those NAL data when it makes (received) bitstream with error pattern file (S. Wenger, “Error patterns for internet experiments,” ITU-T SG16 Doc. Q, 2002.). They are quite simple, but I don’t recommend those tools.

Test Decoding H264AVCDecoderLibTestStatic.exe test3_sl0.out test3_sl0.yuv  45 Frames Decoded H264AVCDecoderLibTestStatic.exe test3_sl2.out test3_sl2.yuv  90 Frames Decoded

Using a YUV Viewer

Checking PSNR Value PSNRStatic.exe foreman_352x288_30_orig_01.yuv test3_sl2.yuv

Personal comments (1) My blog has more information on this : Multimedia Communications using Scalable Video Coding – Read JSVM software manual carefully, it explains all you want. AVC/SVC encoding time is quite long, and you may need to reduce encoding frame number. Bitstream extractor cannot extract enhan. layer only but extract base+enh layers. Thus, to extract a specific layer separately you need to write your own code (you can borrow my code from the blog mentioned above). For academic research you can use more options such as FMO and MCTF. You can also use ‘X264’, ‘ffmpeg’, ‘Yarm’, and ‘mp 4box’ tools for your own testing.

Personal comments (2) Recent JSVM (after ver. 9.8) does not support EC(error concealment option). Even the old tool cannot decode damaged bitstream well (mostly died). You can use open SVC decoder from source forge website ( ts/opensvcdecoder/) ; personally thanks Mederic Blestehttp://sourceforge.net/projec ts/opensvcdecoder/ and Mickael Raulet). you can use packet loss simulator because this tool just omits some NALs according to error pattern files. Even you can apply different packet loss rate to each layers with this tool. These slides were originally prepared for internal lab seminar of operating system La b in computer science and engineering department of Korea university, and I upd ated some of them for general student and researcher. July Eun-Seok Ryu