codecs (H.261 and H.263) performance new features and GUI interoperability with the core system future developments

Video tool [ introduction ] default video client of VRVS/EVO systems is based on VIC project of OpenMash consortium – last milestone release 5.2 from February 2002 the application is written in C/C++ and OTcl programming languages (Tcl/Tk v8.3) development has started in July 2003 (2 developers + 2 developers in 2004) the source codes were significantly reduced with the aim to remove all sources which don’t belong to video tool itself (like VAT, MB, archive tools, etc.) compatibility with 3 main platforms supported by VRVS/EVO systems (Windows, MAC OS, Linux)

Video tool [ H encoder ] new encoder is fully compliant with the H.261 standard old encoder used only INTRA-mode – compression was based on reducing of intra frame spatial redundancy added was INTER-mode with inter-frame motion estimation and compensation - difference between two subsequent video frames is encoded - compression is based on reducing of inter frame temporal redundancy reduces bitstream (higher compression) - allows to use higher framerate and quality with the same bitstream frame n frame n+1 difference

block scheme of H.261 encoder (highlighted are newly developed parts) Diamond searching algorithm Searching area +/- 7 SAD sum of absolute differences Video tool [ H encoder ] Median filter defined in H.261 standard

Video tool [ H encoder ] BETTER COMPATIBILITY WITH H.323 DEVICES – solutions on encoder side: equal RTP packet size (via MBA stuffing) – avoids packet reordering and “freezing” of decoded video fixed H.261 payload RTP headers – GOB header cannot be at the end of the packet summation of EBIT and SBIT values of two subsequent packets must be 8 – leads to correctly decoded video on Polycom FX, ViaVideo … INTER mode, thanks to higher compression, avoids buffer overflowing on HW H.323 devices and therefore brings possibility to send video with better quality encoded RTP packets are sent in time intervals depending on current bps value, what avoids buffer overflowing on HW H.323 devices

Video tool [ H decoder ] BETTER COMPATIBILITY WITH H.323 DEVICES – solutions on decoder side: some H.323 clients (Codian MCU, Polycom VSX 7000 ) don’t send correct information in RTP H.261 payload headers (MBA, GOB, MVDH, MVDV), what causes artifacts in decoded video - VIC ignores this information in RTP payload headers and restores it from payload data bug which causes skipping of the last macroblock in RTP packet was fixed (avoids artifacts in decoded video) buffering of packets on decoder side – allows to prevent corrupted video due to packet reordering

Video tool [ H decoder ] MCU Codian - before MCU Codian - now Polycom VSX before Polycom VSX now

Video tool [ H.263 codec ] H.263 is more computationally intensive than H.261 due to new enhanced features and larger video resolution PRESENT STATUS current implementation is based on Baseline Mode utilizing only compression based on reducing INTRA frame spatial redundancy encoder and decoder support only A and B types of RTP packets decoder is able to decode video stream containing empty GOB headers development is focused on compatibility with H.323 devices, improvement of codec performance, capturing video with larger resolutions – currently QCIF, CIF, VGA, 4CIF, XGA

Video tool [ The Performance ] IPP (Intel Performance Primitives) libraries were used for better performance of video codecs IPP allows to replace a part of code, which is heavy on computation – DCT, Motion compensation,… automatic processor type detection available for Linux and Windows (for MAC with the new family of INTEL processors for this platform ) Intel C++ compiler optimizes compilation for target processor implementation of IPP functions in H.26x encoder and decoder leads to significant decrease of CPU usage

Video tool [ new features and GUI ] the same version for Windows, Linux and Mac OS with the same features new GUI – more user friendly, intuitive and easier to customize, complex technical settings have been removed from the main settings interface window Speaker Mode - video tool is able to identify and highlight video of current speaker - information about who is the current speaker is generated by reflector and carried by specific RTCP packet (SDES) new video display modes which automatically arrange video windows on the desktop - all videos in CIF size - all videos in QCIF size - only the video of current speaker in CIF size. - current speaker in CIF size plus all the other videos in QCIF size

Video tool [ new features and GUI ] possibility to save all settings from GUI including position on the desktop multiple display support for Windows OS - user can choose display, which the videos from participants will be displayed on Still Image Mode – allows transmitting of static picture instead of video from camera Minimize Mode – hides small video “stamps” in the main application window, saves space on the desktop and many others improvements like using FireWire cameras on MAC without the IOXperts driver and freezing of video, handling “blue faces” phenomena

Video tool [ new features and GUI ] DESKTOP SHARING MODE - allows to capture PC desktop and send it like H.263 video one application as video client and desktop sharing client advantage of existing infrastructure - signal will pass through network limitations like NAT higher compression of the signal using H.263 capabilities possibility to capture whole desktop or part of the desktop surrounded by red transparent window (WINDOWS, MAC OS X) or top-left part of the desktop (LINUX) supported resolutions are CIF [352 x 288], 4CIF [704 x 576], XGA [1024 x 768] Windows user have possibility to choose screen, which is to be shared in case of multiple display usage

Video tool [ new features and GUI ] H.261 CIF (352 x 288) H.261 CIF (352 x 288) H.263 XGA (1024 x 768) H.263 VGA (640 x 480)

Video tool [ new features and GUI ]

Overview of videoconference clients in EVO Video and audio tools on WINDOWS (17 participants)

Video tool on LINUX (17 participants)

On Mac OS X Video and audio tools on MAC OS X (9 participants)

Interoperability with the core system - communication interface interface allowing to receive commands from external applications - like VRVS/EVO client - for interaction with video and audio tools during runtime communication interface uses existing RTCP channel of the applications – doesn’t need additional channel remote commands are carried by specific Application Defined (APP) RTCP packets that are understood and subsequently processed only by VRVS/EVO video and audio tools and ignored by the others useful in conference scenarios, when privileged user can control audio and video clients of other participants (e-Learning: teacher vs. students) VRVS/EVO client can take appropriate action as reaction on alarm notifications when monitored parameters (system or network) will go beyond a preset threshold

Communication Interface [ e-Learning scenario ] Teacher Student

Communication Interface [ localhost monitoring scenario ] EVO Client Localhost monitoring module LISA audio client video client ALARM: CPU overloaded Response: decrease framerate, quality, resolution, change video codec, turn off display of big video windows, …

Video tool [ Future development ] video codecs – continue in work on H.263, later on H.264 handle different video sources (video, desktop, still images) and display them in different way development of new display scenarios (utilization of OpenGL library, concentrating all videos in one window, etc.) new GUI implementation, more user friendly, intuitive, reflecting all recent improvements keep application compatible with all 3 supported platforms, replacement of Tcl/Tk part by another language (Java?, C++?) new integrated audio and video application

Video tool [ Future development ] all videos displayed in one window new GUI

codecs (G.722) new features and GUI interoperability with the core system future development

Audio tool default audio client of VRVS/EVO systems is based on RAT project of University College London (C and Tcl/Tk programming languages) new audio codec G.722 was added for compatibility with H.323 devices (based on IPP) priority of development was to unify source code for all supported platforms (Windows, Linux, Macintosh) – required new audio interface for MAC OS 3 processes – rat, ratmedia, ratui have been merged into one binary (MBUS removed) and audio processing and windows management have been split into two separated threads, what increase performance and stability of the application support of multiple NIC, native ALSA support (Linux) future: echo canceller, integrated audio and video application

Thank you for your attention

Additional slides

PocketVRVS – the mobile solution PocketVRVS is new VRVS videoconferencing client developed for Pocket PC platform application is written in C++ and Windows CE API it supports H.261 video standard and G.711 (µ-Law) audio standard user can choose CIF sized still image to be transmitted (JPEG format) the logic of video encoding was adapted to encoding only still images - the process was optimized to reach the compromise between framerate, CPU usage and fast update of video on remote clients PocketVRVS – the mobile solution

utilization of INTEL Performance Primitives and Intel compiler for optimization of application performance (Discrete Cosine Transform, color model conversions RGB to YUV) received and decoded is only video of current speaker – this functionality is implemented on reflector’s side PocketVRVS works inside networks with network address translation (NAT/NAPT) new web based interface for Pocket PC clients (designed for smaller screens) includes full functionality available for desktop clients (meeting scheduler, booking, …) and also possibility to connect H.323 device installation package of PocketVRVS application with multi- language OS support is intended directly for Pocket PC platform, desktop computer is not needed H.323 connection interface PocketVRVS – the mobile solution

PocketVRVS web interface PocketVRVS application GUI Documentation Control Panel Talk, Listen, Send and Receive controls Download PocketVRVS package Booking Connect to any ongoing meeting Video Screen Remote Video, Own Video in PIP window Session Info Virtual Room, Participant’s name, … Program Menu Connection and Session Settings