1. Design and Implementation of Audio/Video Collaboration System Based on Publish/subscribe Event Middleware
CTS04 San Diego 19 January 2004
PTLIU Laboratory for Community Grids
Geoffrey Fox, Wenjun Wu, Ahmet Uyar, Hasan Bulut
Computer Science Department
Indiana University, Bloomington IN 47404
9/20/2018
uri="

2. A/V Collaboration Systems
H323
H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment.
SIP
The Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences
Access Grid
enhanced Mbone A/V tools ( VIC, RAT )
Internet 2 network ( Multicast support )
9/20/2018
uri="

3. XGSP Collaboration Framework
To integrate heterogeneous systems into one collaboration system
A unified, scalable, robust “overlay” network is needed to support A/V and data group communication over heterogeneous networking environments.
A common A/V signaling protocol has to be designed to support interactions between different A/V collaboration endpoints.
Different A/V endpoints should collaborate in the same collaboration session.
9/20/2018
uri="

4. Group Communication in Collaboration
Collaboration applications usually need a group communication service for both multipoint data and control information delivery
Centralized conferencing systems usually depend upon a single conference server
Distributed conferencing systems use IP multicast
Access Grid uses Internet2 multicast for audio/video transmission.
Problems:
Centralized conferencing systems don’t have good scalability
IP multicast has not become ubiquitously available
9/20/2018
uri="

5. uri="http://www.globalmmcs.org" email="gcf@indiana.edu"
Narada Broker Network
(P2P) Community
For message/events service
Broker
Broker
(P2P) Community
Resource
Broker
Broker
Broker
Data
base
(P2P) Community
Software multicast
Broker
(P2P) Community
9/20/2018
uri="

6. uri="http://www.globalmmcs.org" email="gcf@indiana.edu"
Narada Brokering
Narada Brokering from the Community Grid Labs is adopted as a general event brokering middleware
supports publish-subscribe messaging model with a dynamic collection of brokers
provides services for TCP, UDP, Multicast, SSL and raw RTP clients
Supports the communication through firewalls and proxies
operate either in a client-server mode like JMS or in a completely distributed JXTA-like peer-to-peer mode.
9/20/2018
uri="

7. uri="http://www.globalmmcs.org" email="gcf@indiana.edu"
Narada Brokering
Covers the heterogeneity of network transportation and provides unified multipoint transportation API
Software multicast
Communication over firewalls and proxy boundaries
Communication over multiple transports
Application level Quality of Service
Filter messages to slow (collaborative/real-time) clients
Provides robust, scalable and high efficient multipoint transportation services
Availability and scalability
Efficient routing and bandwidth utilizations
9/20/2018
uri="

8. A/V Collaboration over publish/subscribe Middleware
video streams (VS) VS {v1, v2, … vm }
audio streams (AS) AS { a1, a2, … an }
A/V endpoints: E1, E2, … En
Each endpoint may send a single or multiple video streams, but only send an audio stream
Different types of A/V endpoints have different collaboration capabilities.
Multicast endpoints are able to receive multiple video and audio streams, display all the video streams in their screens, and mix all the audio streams by themselves
Unicast endpoints like Polycom Via Video can only receive and play a single video and audio stream
9/20/2018
uri="

9. Publish/subscribe of A/V
A stream in VS and AS is regarded as a “topic”
Each RTP packet from this stream is regarded as an “event” for this topic
Only the sender of this stream can “publish” A/V events to this topic
Other endpoints need to subscribe to this topic in order to receive the stream data
Create mixed video and audio stream topics for unicast endpoints
9/20/2018
uri="

10. Major Events for each stream topic
Event Name
Change in the stream
NewStreamEvent
This stream has been created
ByeEvent
The stream gives a BYE RTCP packet, indicating it has left the session
TimeOutEvent
This stream has not send any RTCP packet for a long time, indicating it may have left the session
Active-To-Passive
The stream has stopped sending RTP packet
Passive-To-Active
The stream resumed sending RTP packet
9/20/2018
uri="

11. A/V Collaboration Architecture
9/20/2018
uri="

12. RTP packets encapsulation
RTPLink: RTP Packets ~ Narada Brokering Event
Every legacy A/V client needs one corresponding RTPLink to be set up at a broker within broker network.
Unicast RTPLink:
Integer Topic Numbers for RTP and RTCP
Multicast RTPLink:
A reflector between NaradaBrokers and multicast groups, encapsulating raw RTP packets from a multicast IP address to RTP events, publishing these events to NaradaBrokers, and forwarding the data it receives from broker network on the same IP address
9/20/2018
uri="

13. Audio Mixer, Video Mixer, Image Grabber
Audio Mixing
The audio mixer creates a mixed audio stream from all the audio streams in the session
Video Mixing
Video mixing makes the unicast users watch the pictures of multiple participants in a meeting through one video stream
Video Thumbnail
visualize the VS set in the session, embedded into the control panel of each endpoint, which
Image grabbers capture video streams and save them as static JPEG files.
All the media processing components can be distributed among the pool of the media servers connected to NaradaBrokering infrastructures.
9/20/2018
uri="

14. H.323, SIP Gateway Servers, A/V Session Server
H.323 and SIP gateway transform these protocol specific messages into XGSP signaling messages so that H.323 and SIP A/V endpoints could communicate with the XGSP A/V session server
The session server implements session management logics
creating/destroying A/V sessions
allowing endpoints to join/leave session
Allowing users to make audio/video selection, managing A/V application components
9/20/2018
uri="

15. uri="http://www.globalmmcs.org" email="gcf@indiana.edu"
Implementation
Java Media Framework (JMF) :
Openh323 is basis of H323 Gateway
NIST SIP stack is basis of SIP Gateway
NaradaBrokering is open source messaging from Indiana
9/20/2018
uri="

16. uri="http://www.globalmmcs.org" email="gcf@indiana.edu"
Demo Pics ( I )
9/20/2018
uri="

17. uri="http://www.globalmmcs.org" email="gcf@indiana.edu"
Demo Pics ( II )
9/20/2018
uri="

18. uri="http://www.globalmmcs.org" email="gcf@indiana.edu"
Demo Pics ( III )
9/20/2018
uri="

19. Please watch our demo at 5:30 PM !
Thank you!
Please watch our demo at 5:30 PM !
9/20/2018
uri="