VIC Development National Center of High-performance Computing Barz Hsu 15 January 2007

Start Point Intend to develop high visual quality video tool Based on AccessGrid vic-2.8ucl Initial release of NCHC-VIC with  MPEG4 codec  Up to NTSC/PAL resolution  High-efficient render  Fullscreen mode  Linux version only

Timeline Develop NCHC-VIC from January 2004 to August 2005 Develop GStreamer-based media tools from June to August 2006 Merge NCHC-VIC to UCL code base from September to December 2006

Outlines Review NCHC-VIC Porting Issues Demos GStreamer based new media tools

NCHC-VIC Features Codec  MPEG4  H.264 Render  Xvideo  Swscale  Fullscreen Postprocess  Deinterlace  Deblocking  Colorspace conversion

MPEG4 FFmpeg encoder/decoder ME (motion estimation)  EPZS algorithm  QPEL CBR (constant bit rate) GOP (group of picture): 15 Low latency  No B-frame  rtp_callback  Old_latency = encoding_latency + tramsmission_latency  New_latency =~ encoding_latency/5 + tramsmission_latency Low CPU usage

MPEG4 (cont ’ d) IETF RFC 3640 RTP payload type 45 Rearrangement RTP packet buffer for only one frame Error resilience/concealment Compatible with VP Handy to use other codec such as  MPEG2, H.261, H.263, H.263+, DV, MJPEG, Theora

H.264 X264 encoder  Turn off CABAC save computation up to 30%, increasing coding bit rate around 15%  Inter-ME X264_ANALYSE_PSUB16x16  RC_METHOD X264_RC_ABR  i_direct_mv_pred X264_DIRECT_PRED_NONE  Deblocking filter  No B-frame  GOP is 20~50 frames

H.264 (cont ’ d) FFmpeg encoder FFmpeg decoder Each RTP packet is a collection of NAL units IETF RFC 3894? Compatibility?  Quicktime 、 VLC 、 inSORS Contributed by Champ Yeh and Barz Hsu from NCHC

Advanced UI Contributed by Morris Lin from NCHC

Accelerated Render XVideo+Shared memory Software scaling CPU capability runtime detection Win32/Win64, Linux, MacOSX Colorspace Conversion

SWSCALE: Software Scaling Libswscale  Shipping with FFmpeg and MPlayer Arbitrary resizing Bilinear interpolation Colorspace conversion  YUV420 to RGB32/RGB24/RGB16 for render  YUYV to YUV420 for grabber CPU multimedia instructions  Intel: MMX/MMX2/SSE/SSE2  AMD: 3D Now/3D Now2  PowerPC: Altivec Runtime CPU capability detection Speed up to 10X than pure C code

Xvideo Extension: Graphic Accelerated Render XFree86 X11 Extension Supported by Nvidia and ATI graphic card Direct YUV420 render  No colorspace conversion needed Perform image scaling and colorspace conversion by graphics Most efficient way Up to 40~100 videos at same time Automatically use swscale if xvideo no longer available

Fullscreen Boardless Light weight If graphics accelerated Dual head is fine Only for Windows and Linux

Fullscreen

Postprocess Provide better visual quality Remove coding artifact by specific encoders Deinterlace Deblocking Dering Automatic level  Use higher level of postprocessing if CPU usage is not full

Linux Port Nothing special! File size of statically linking binary is around 10MB Most robust and efficient version V4l grabber bugfix

Windows Port Visual Studio 2003 Inline assembly syntax different from GCC  AT&T v.s. INTEL Create DLLs and LIBs (import library) under Mingw/Cygwin  ffmpeg, x264, libswscale, libpostproc  Is it possbile to use sttatical linking? S-video, Composite input support correct framerate for win32 DS grabber

MacOSX Port Merge AG Aqua branch  Aqua and Quicktime framework Without X11 Use Aqua Tcl/Tk 8.4 shipped with MacOSX Native look Grabber-macosx  iSight, QuickCam support More robust than AG Aqua branch

MacOSX Port (cont ’ d)

Compilation Steps (Linux) Check out source code  svn co Configuration ./configure Check out FFmpeg with revision 7110 (Dec, 2007)  svn co -r 7110 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg ./configure --enable-gpl --enable-swscaler --enable-pp Check out X264 with revision 600 (Dec, 2007)  svn co -r 600 svn://svn.videolan.org/x264/trunk x264  Options --with-ffmpeginc --with-ffmpeglib --with-x264inc --with-x264lib Build  make

Compilation Steps (MacOSX) Install TclTkAquaBI for Tcl/Tk  Configuration and Building Steps are same as Linux platform

Compilation Steps (Windows) Configuration ./configure Check out FFmpeg with revision 7110 (Dec, 2007)  svn co -r 7110 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg ./configure --enable-mingw32 --enable-memalign-hack --enable-swscaler - -enable-gpl --enable-shared --disable-static Check out X264 with revision 600 (Dec, 2007)  svn co -r 600 svn://svn.videolan.org/x264/trunk x264 ./configure --enable-shared  Patch Makefile for generating DLL  add the 3-6 lines in SONAME targe  $(SONAME):.depend $(OBJS) $(OBJASM)  $(CC) -shared -o $(OBJS) $(OBJASM) -Wl,-soname,$(SONAME) $(LDFLAGS)  ifeq ($(SYS),MINGW)  $(CC) -shared -Wl,--output-def,libx264.def -o libx264.dll $(OBJS) $(OBJASM) lib /machine:i386 /def:libx264.def  endif

Compilation Steps (Windows) (cont ’ d)  Copy LIBs for VS2003 avcodec-51.lib avutil-49.lib libx264.lib postproc-51.lib swscale-0.lib  Copy DLLs to the path of excuteable avcodec-51.dll avutil-49.dll libx264.dll postproc-51.dll swscale-0.dll

Tcl/Tk 8.4 Multi-threading Tcl_init.tcl and tk_init.tcl files required Further work needed to embed the init scripts to binary It’s critical to call Tk_FindExecuteable() to locate init scripts for Windows platform

Open Issues?