A Web Services Based Streaming Gateway for Heterogeneous A/V Collaboration Hasan Bulut Computer Science Department Indiana University

Motivation Videoconferencing systems provide a framework to send/receive audio and video streams. Streaming is a widely used media delivery technology across Internet to receive Media-On- Demand. It enables users to receive multimedia content and provides some control functionality over the media. To enable streaming clients to join the real-time videoconferencing sessions and receive multimedia content generated by other clients in the system.

Videoconferencing Systems Major videoconferencing systems are: H323 H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment. SIP The Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences Access Grid Enhanced Mbone A/V tools ( VIC, RAT ). Internet 2 network ( Multicast support )

Videoconferencing vs Streaming A streaming client connects to a streaming server, primarily using RTSP, to establish a session and receive the stream. Streaming is primarily used for Media-On-Demand. It is also possible to make a live streaming broadcast. Streaming media aims to improve the quality of service of the stream, by using various streaming codec to improve stream quality and a buffering mechanism to reduce the jitter. While the delay in conferencing is around 200 msec, the delay in streaming buffer is around 2 – 15 sec.

In this paper we will explain the components of XGSP Streaming Gateway, their functionality and messages added to the framework to achieve integration. Other components will be briefly explained. GlobalMMCS, which implements XGSP, uses Web Services technology to integrate heterogeneous conferencing systems. Streaming Gateway has been recently added to this system and it is used internally by other components in the system.

XGSP Collaboration Framework To integrate heterogeneous systems into one collaboration system, we need to reach the following goals: Different kinds of application endpoints should join leave in the same collaboration session. A common A/V signaling protocol has to be designed to support interactions between different A/V collaboration endpoints. A common user interface should be present for all the collaboration participants using different A/V and data application endpoints. Different A/V endpoints should collaborate in the same collaboration session.

Global Multimedia Conferencing Systems (GlobalMMCS) The first prototype of this system includes: A XGSP media server provides the services of bridging multicast and unicast, video-switching, video-mixing and audio- mixing to H.323, SIP as well as AG endpoints. H.323, SIP gateways and Real Servers for A/V clients XGSP A/V Session Server manages real-time A/V sessions, receives messages from gateways and the web server, and performs appropriate actions on the media server. The web server provides an easy-to-use web interface for users to join multimedia sessions and for administrators to perform administrative tasks.

GlobalMMCS Architecture RealStreaming Clients SIP Gateway Media Servers; Audio Mixers, Video Mixers, Image Graber SIP Clients XGSP Session Server XGSP Web Server NaradaBrokering Publish/Subscribe Messaging H.323 Gateway Access Grid Multicast Streaming Gateway Helix Server H.323 Clients Two-way RTP-link/links One-way RTP-link / links Pushed streams Streaming Communication link / links

XGSP Streaming Gateway (XGSP–SG) Due to the requirements and underlying technology of streaming clients, XGSP–SG demonstrates different functionalities compared to other gateways: Only uses XGSP message formats, while other gateways only transforms XGSP signal to another form and vice versa. Performs stream format conversion so that streaming clients can play. Provides a mechanism to synchronize streams Streaming gateway allows streaming clients only to receive audio and/or video streams.

XGSP–SG Components and Design Stream Engine Stream Conversion Handler SMIL File Generator Helix Streaming Server Stream Engine Communication link Pushed streams Process generation Streaming Gateway components

XGSP–SG Components I Stream Conversion Handler performs the communication between Session Server and Streaming Gateway. maintains an internal database for the streams. Database is updated when the streaming jobs are started or deleted. initiates/stops Stream Engines for the requested stream

XGSP–SG Components II Stream Engine converts received audio or video streams into a specified RealStreaming format and pushes the output stream to Helix Streaming Server composed of two parts, JMF RTP Handler and HXTA Wrapper JMF RTP Handler: transform the received packets into raw audio or video format HXTA Wrapper: makes use of HXTA engine to encode streams into RealMedia format

XGSP–SG Components III SMIL File Generator dynamically constructs a SMIL file using the received audio and video stream IDs deletes SMIL file if the stream is no longer in the session SMIL files enables clients to receive multiple streams and synchronize them Each session in a XGSP session contains only one audio stream, which is the mixed of all audio streams received from clients.

XGSP–SG Messages I InitializeRealGateway - deletes all current jobs in the session specified JoinStream - starts a job for the stream specified. JoinStreamReply - reply message for JoinStream message. OK indicates the job is successfully started, otherwise returns FAIL. LeaveStream - stops the stream specified. LeaveStreamReply - reply message for LeaveStream. FAIL indicates that an error occurred during the stop operation, otherwise returns OK.

XGSP–SG Messages II RealStreamEvent - It has two modes, NewRealStream to indicate a new stream in the session and ByeRealStream to indicate the stream is no longer in the session RealStreams - requests the list of the RealStreams available in the session RealStreamsReply - reply message for RealStreams. Includes a list of the RealStreams in the session.

Services added to Session Server RS Join Service - handles JoinStream messages. If stream job is successfully started, sends RealStreamEvent message (mode = NewRealStream). RS Leave Service - handles LeaveStream messages. Sends RealStreamEvent (mode = ByeRealStream). RS List Service - handles RealStreams messages. Replies back with RealStreamsReply listing all the available RealStreams in the session. RS Gateway Service - handles InitializeRealGateway messages. Sends RealStreamEvent with ByeRealStream mode for each RealStream in that session.

Streaming Interface I Two types of interfaces are developed for Streaming Gateway. Streaming Admin Interface: provides administrative features such as starting and stopping streaming jobs. Streaming Client Interfaces: enables clients to plays streams from an interface Streaming Admin interface also includes Streaming Client interface functionality to enable administrator with client capabilities.

Streaming Interface II A snapshot from Streaming Admin interface

Streaming Interface III A snapshot from Streaming Client interface

Streaming Interface IV A snapshot of the stream played in RealOne Player window

Performance Discussion for Streaming Gateway I Number of Stream Frame rates of streams involved CPU Usage Range Memory Usage 123 fps%10 - %2529 MB 223 fps, 25 fps%22- %4034 MB 323 fps, 25 fps, 26 fps %50 - %7543 MB 423 fps, 25 fps, 26 fps, 16 fps %65- %9553 MB OSWindows XP CPUIntel® Pentium® 4 CPU 2.26 GHz Memory512 MB JVM version1.4.2_02 Approximate CPU and memory usage with respect to number of streams Streaming Gateway machine specifications

Performance Discussion for Streaming Gateway II In a session (or classroom) usually there is one presenter. Sometimes other clients in the session may be active too. In a session most of the clients are only listeners. In such a session providing 3 video streams and a mixed audio stream ( mixed of all audio streams received from clients in the session) is sufficient.

Conclusion and Future Work XGSP framework and, Global-MMCS enable multiple communities to collaborate with each other. Streaming Gateway enriches this heterogeneous system by integrating streaming media world into Global-MMCS. Streaming Gateway is part of Global-MMCS and uses XGSP in order to achieve this integration. In order to increase the number of streams converted, we also would like to build a streaming job scheduler, which will schedule and coordinate streaming jobs in a distributed environment.

Thank you!