1. Jongchurl Park (jcpark@nm.gist.ac.kr) Networked Media Laboratory Dept. of Information and Communications School of Information & Mechatronics Gwangju Institute of Science & Technology (GIST) System Enhancements and Extension of Uncompressed HD Media Transport System Lab. Seminar May 30 th, 2009

2. School OF I NFO. & Mecha., G IST  Introduction ◦ Uncompressed HD media transport system ◦ Networked tiled display & 3DTV  Enhanced uncompressed HD media transport system ◦ Preview video & bi-directional transport system ◦ Networked display interface  Extension of uncompressed HD media transport system ◦ Multiple streams for free-viewpoint ◦ Views in 3D effects  Conclusion & Future works

3. School OF I NFO. & Mecha., G IST

4.  No need encoding and decoding procedures ◦ Highly interactive (i.e., low-delay) media services  Video format ◦ 720p: progressive, 60 fps, 1280x720, 20 bits/sample ◦ 1080i: interlaced, 30 fps, 1920x1080, 20 bits/sample  Requied bandwidth: 1.2~1.5 Gbps  Delay: ~200msec.  Interactive application ◦ Video conference ◦ Remote lecture (i.e., e-learning) ◦ Interactive remote performance (e.g., DancingQ)

5. School OF I NFO. & Mecha., G IST  Standard Definition Television (SDTV) ◦ Resolution - 480i: 720 x 480, 60 interlaced fields per second ◦ Uncompressed  Bandwidth: ~200Mbps  Delay: ~100msec. ◦ MPEG2  Bandwidth: 2~6Mbps  Delay: 1~2 sec.  High Definition Television (HDTV) ◦ Resolution  1080i: 1920 x 1080, 60 interlaced fields per sec.  720p: 1280 x 720, 60 frames per sec. (progressive scan) ◦ Uncompressed  Bandwidth: 1.2 ~ 1.5 Gbps  Delay: ~100msec. ◦ MPEG2  Bandwidth: 20~80Mbps  Delay: 1~2 sec.  Limitation ◦Uncompressed HD video transport system on raw is not enough for displaying correct HD videos ◦More realistic presence is required like three dimension

6. School OF I NFO. & Mecha., G IST  Networked tiled display ◦ Features  Monitors of Lattice shape, users can use as if tiled display is single monitor  Cost-effective ultra-high resolution device (more than 4K)  Simultaneously run various applications on local or remote clusters, and share them by streaming the pixels of each application  Users freely move and resize each application's imagery in run-time  Color-space  Compatible with RGB16, RGB24 and RGB32 ◦ Applications  Scalable Adaptive Graphics Environment (SAGE)  The role of display middleware to visualize any kind of pixel-stream to a tiled display color-space  SMeet One Display (SMOD)  Realized after SAGE, Integrating collaboration services that include media delivery scheme to offer A/V communication among participants, high-resolution dis play service  3DTV ◦ “Realistic” ghost-like illusions has always been a goal ◦ A new generation of broadcast  Immersive 3D video display in real-time

7. School OF I NFO. & Mecha., G IST

8.  The criterion of uncompressed HD media transport system ◦ Hardware capture and software display  Specification ◦ Ultragrid (USC/ISI) extension ◦ XENA HS (AJA Video systems) – Rx & Tx ◦ 1080i (25,50,29.97,59.94,30,60 fps) ◦ 6-channel audio (24-bit/48KHz) ◦ Video out  XENA HS, SDL, xVideo ◦ Audio out  XENA HS, ALSA ◦ Platform  Redhat 9, Mandrake 10.1, and Fedora Core 5

9. School OF I NFO. & Mecha., G IST  Preview ◦ users can preview and see the current stage of the process before producing into a final form. ◦ lets users to visualize current/final product and correct possible errors easily before finalizing the product. ◦ Interactive video conference and online forums allow users to preview their place environment before transport.  Particularly useful on sites with complex markup, where it serves as an opportunity to identify and correct video quality, light and so on.

10. School OF I NFO. & Mecha., G IST HW-based Bi-directional Transport  Alternating system configuration ◦ Supporting multiple network interfaces  Verified for bi-directional streams ◦ Multiple HD-SDI interfaces support  Separating functions to capture and display  Simplifying system handling with bi-directional transport  Hardware performance analysis ◦ Requires both HD-SDIs and Network Interface Cards support 1.3Gbps bandwidth  To exploit hardware overload from bulky traffic ◦ Hardware bandwidth requires more than PCI-X 100Mhz extended slot  133Mhz extended slot can support maximum 1 GByte/s

11. School OF I NFO. & Mecha., G IST Occurrence rate Frame per second  Comparison of frame rate ◦ Outgoing/incoming video stream (single load)  61.5% of frame rate maintain at 29.97 fps ◦ Outgoing/incoming video stream (dual load)  99.3% of frame rate maintain at 29.97 fps Dual load paths improved and stabilized performance than single load network

12. School OF I NFO. & Mecha., G IST

13.  Interfacing uncompressed HD media transport system and networked tiled display ◦ A scalable hundreds of megapixels of contiguous display with cost-effective prices as possible

14. School OF I NFO. & Mecha., G IST  Video source: Uncompressed HD ◦ Captured 1920x1080i format HD signal by using HD-SDI card ◦ Covert analog signal to digital signal, due to SMPTE-292M by using A/D converter  Interfacing Module ◦ Put video data to buffer ◦ Color-space conversion  Transport ◦ Put frames to buffer and receive frames by using SAIL or STP library  Display modules ◦ Display video on all display nodes

15. School OF I NFO. & Mecha., G IST  Performance enhancement for networked display interface ◦ Using binary increase congestion control (BIC) is not good for heavy traffic  Long delay over high-speed networks  Improving TCP-friendliness and RTT-fairness ◦ Using extended version of BIC (CUBIC) – good for heavy traffic (around 1Gbps bandwidth usage)  Increasing TCP transport window size for no acknowledge (NACK)  Allocating large enough socket buffers  Reconfiguring TCP segment sizes  Maximum MTU packet by using a path MTU discovery function ◦ System tuning (After receiving 1000 packets, it cause freezing HD display)  All nodes as well as a controller should be set around MTU 9000 byte  Increasing buffer size can solve freezing problem  Hard sync stream type makes it more stable

16. School OF I NFO. & Mecha., G IST  Cost-effective interfacing system ◦ Interfacing system is required to set up both sender and receiver of uncompressed HD media transport system  Directly transmit frames sender to networked tiled display (without receiver)  Using sail library instead of RTP in uncompressed HD media transport system

17. School OF I NFO. & Mecha., G IST  Comparison of Bandwidth (BIC and CUBIC) ◦ BIC Input: 921Mbps (29.97 fps) Output: 239.4Mbps (7 fps) ◦ CUBIC Input: 921Mbps (29.97 fps) Output: 682.1Mbps (23 fps) CUBIC improved and stabilized performance than BIC Approximately 16fps difference Time (sec.) Bandwidth (Mbps)

18. School OF I NFO. & Mecha., G IST  Environment ◦ 8bit uncompressed HD video is transmitted over 1Gbps (MTU: 9180 Bytes)  Top: Artistic dance performance is delivered in live to international destinations by HW- based playout mode  Middle: Interactive video conferencing scenario with preview HD videos by SW- based playout  Bottom: Interfacing with networked tiled display  Ultra-high-resolution support  Appropriate control of frame rate

19. School OF I NFO. & Mecha., G IST

20. ◦ Interactive clustered broadcasting with free-viewpoint display  Multiple streams for free-viewpoint  Views in 3D effects (Interactive views)  Ultra-high-resolution support

21. School OF I NFO. & Mecha., G IST  Multiple video streams in single machine ◦ Required to handle more TWO uncompressed HD video data (over 2Gbps)  Simplifying system handling multiple ways for transport ◦ Improved configuration of network interface card and HD-SDI interface  Up to 4 media interfaces ◦ Verified for bi-directional streams and two ways streams  Free-viewpoint video ◦ A multi-view display emits more than two views, a viewer can be positioned anywhere ◦ One category of 3D video - the viewer shall be given the flexibility to interactively position itself  Focus on representations and methodologies to capture and process dynamic scenes

22. School OF I NFO. & Mecha., G IST  Generalizing 2D video pixels towards 3D ◦ Interfacing imCast to visualization toolkit for 3D effects  Instant virtual freeze-and-rotate effects  Put interaction mechanism for mouse/key/time events

23. School OF I NFO. & Mecha., G IST  Interactive video service for 3D effects ◦ Real-time video display with video rotating and rendering ◦ Basic procedure  Source -> mapper -> actor -> renderer -> rendering window ◦ Applied mapping and rendering process

24. School OF I NFO. & Mecha., G IST  Environment ◦ 8bit uncompressed HD video is transmitted  Over 1Gbps (MTU: 9180 Bytes)  Resolution: 1920 x 960  Uncompressed HD video display in 3D effects ◦ Real-time video display with interaction ◦ Performance problem  While image actor is working, CPU overhead and user memory increase  CPU overhead: 20% to 70~90%  User memory: 12.5% to 36.7%

25. School OF I NFO. & Mecha., G IST

27. Conclusion & Future works  Conclusion ◦ Enhanced uncompressed HD media transport system  Preview video support  Networked display interface  Bi-directional transport system ◦ Extension of uncompressed HD media transport system (ongoing)  Multiple streams for free-view-point  Views in 3D effects  Future works ◦ Interfacing views in 3D effects to networked tiled display (SAGE) ◦ Performance problem (CPU overhead, user memory)

28. School OF I NFO. & Mecha., G IST