1. Aug 25, 2005 page1 Aug 25, 2005 Integration of Advanced Video/Speech Codecs into AccessGrid National Center for High Performance Computing Speaker: Barz Hsu Email: barz@nchc.org.tw

2. Aug 25, 2005 page2 Outline Overview NCHC-VIC: –Introduction to MPEG4/H.264 codec –Codec Optimization NCHC-RAT: –Introduction to Speex codec –Echo Cancellation and Noise Control Future Work

3. Aug 25, 2005 page3 Overview AccessGrid is scalable collaborative environment. AccessGrid is complicated and not easy to use. Major complexity comes from the two core media – vic (video) and rat (speech) Most of video/speech codecs inside AccessGrid are out-of- date. Integrate state-of-art MPEG4/H.264 codecs for video coding, and Speex codec for speech coding. High quality and low latency

4. Aug 25, 2005 page4 MPEG4 Codec Related Open Source Project –FFmpeg, xvid, and sklmp4 FFmpeg/libavcodec –High performance on CPU usage –Support various codec MPEG4, MPEG2, MPEG1 H.263++, H.263, H.261 –Provide error resilience tools slices, data partition –Cross Platform Linux, Windows, and Mac OS

5. Aug 25, 2005 page5 H.264 Codec NCHC-VIC based on X264 project of VideoLAN. X264 is widely used by other OpenSource projects, ex., FFMpeg, Mplayer, VideoLan. X264 is the only free software among current H.264 codecs. The project is still under aggressive development. –http://developers.videolan.org/x264.html

6. Aug 25, 2005 page6 H.264 Codec (cont’d) Adopt new techniques to effectively reduce bandwidth requirement and provide better visual quality –Smaller block division –Multiple prediction modes –Multiple reference frames –Finer motion estimation (Quarter-Pel) –Deblocking filter More computation intensive (5 to 10 times of MPEG4 ASP) Visual Comparison between H.264 and MPEG4 (static video with 128kbps)

7. Aug 25, 2005 page7 Codec Optimization Multithread Encoding –Take advantage of symmetric multiple processor architecture Pipeline the latency of transmission and encoding SIMD Instruction Sets –Optimized by MMX, SSE/SSE2/SSE3

8. Aug 25, 2005 page8 H.264 Optimization Due to the need of huge computation for H.264 encoding, we close some advanced features to meet the requirement of the real-time video conference –Using CAVLC instead of CABAC –Larger block division –Simpler block mode prediction

9. Aug 25, 2005 page9 FFMpeg MPEG4 vs x264 (RD Curve) x264: cavlc, only 16x16 block

10. Aug 25, 2005 page10 FFMpeg MPEG4 vs x264 (Speed) x264: cavlc, only 16x16 block CPU: P4 2.8GHz RAM: 512MB

11. Aug 25, 2005 page11 Visual Comparison between H.264 and MPEG4 (static video with 128kbps) H.264 MPEG4

12. Aug 25, 2005 page12 Visual Comparison between H.264 and MPEG4 (dynamic video with 128kbps) H.264 MPEG4

13. Aug 25, 2005 page13 Full Screen Snapshot

14. Aug 25, 2005 page14 Experimental Video

15. Aug 25, 2005 page15 NCHC-VIC for Win32 Port Based on Linux Version Update IPv6 code Compiled by Visual Studio 6 Some parts are compiled by gcc and then link.

16. Aug 25, 2005 page16 NCHC VIC for Win32 Screenshot

17. Aug 25, 2005 page17 Speex Codec for Speech NCHC-RAT adopts Speex open source project Paten-free audio compression format Narrowband (8 kHz), wideband (16 kHz), and ultra- wideband (32 kHz) Subband coding to reduce computation overhead Packet loss concealment Intensity stereo encoding Project website, –http://www.speex.org/

18. Aug 25, 2005 page18 Echo Cancellation and Noise Control Adaptive filter to model acoustic echo path –Multidelay Block Frequency Domain Adaptive Filter Double Talk Detector –Multidelay Double-Talk Detector Combined with the MDF Adaptive Filter Noise Control –Mexico-hat low pass filter

19. Aug 25, 2005 page19 Echo Model

20. Aug 25, 2005 page20 NLMS Algorithm Normalize Least Mean Square Algorithm

21. Aug 25, 2005 page21 Future Work Session Description Protocol Session Announcement Protocol Advanced User Interface Zero-Configure Cluster Decoding

22. Aug 25, 2005 page22 Thanks for your attendance