1 Using Quality Objects (QuO) Middleware for QoS Control of Video Streams BBN Technologies Cambridge, MA Craig Rodrigues OMG’s Third Workshop on Real-Time and Embedded Distributed Object Computing January 7-10, 2002 Burlingame, California, USA

2 US Navy UAV Characteristics *From the World Wide Web site for official information about Unmanned Aerial Vehicles, QoS Limited Off- Board Links Varying Missions Contention with other Subsystems Constrained Shipboard Resources Streaming Video COTS

3 Video feed from off-board source (unmanned aerial vehicle) Users interact with UAV in real time Users’ hosts receive video and display it Video Distributor sends video to hosts via ship’s network UAV Basic Scenario

4 Highest priority frames must reach the receiver Importance Receiving or collecting sufficient imagery (e.g., surveillance) Importance Invariant { f | f  Frames(U)  Importance(f) > I  f  Frames(R) } Example Dynamic Variations in UAV Mission Requirements Trading off these properties, while maintaining invariants, under dynamic conditions is crucial to mission success End-to-end latency + processing time at distributor and display, must be below threshold Timeliness An out-of-the-window view of UAV imagery (e.g., piloting, targeting) Timeliness Invariant L(U  D) + P(D) + L(D  R) + P(R) < T Image precision at display must approximate that at the UAV source Fidelity Determining relative importance of time-critical targets (e.g., commander) Fidelity Invariant Fidelity(R) / Fidelity(U) > Q Different mission requirements require optimizing an aspect or aspects, possibly trading off others

5 Example Variations & Adaptations in UAV Scenario I P B I I I I I I I I P P P P P P P P P P B B B B B B B BB B B B B B B B BB B B B B B B B B B B B B B B B B B B B B B BBBBBBBBBBBBBB B BBB BBBBBBBBBBBBBBBBBBBBBBBBBBBB P P P P P P P P P P I BB P BB P BB P BB P BB P BB I BB P B NETWORK RESERVATION Condition: excessive Network load Action: Use IntServ to reserve bandwidth... P BB P BB P BB II I I DATA FILTERING Condition: When excessive network or CPU load Action: Drop selective frames Dynamic Variations in Operating Conditions Dynamic Variations in Operating Conditions Mission requirements of UAV scenario Fidelity Image fidelity must be maintained LOAD BALANCING Condition: Excessive CPU load Action: Migrate distributor to a lightly loaded host Timeliness Maintain an out- of-the-window view of UAV imagery Importance Important imagery must be received

6 UAV Application: A View of the Software Components Video Source Process UAV SIMULATION HOST UAV Video File Video DIstributor VIDEO DISTRIBUTION HOST QuO Contract Video Display VIDEO DISPLAY HOST 1 Video Display Proxy CORBA A/V Streaming Service VIDEO DISPLAY HOST 2 VIDEO DISPLAY HOST N Reused Off the Shelf Software DVDView video player Common middleware services TAO CORBA A/V Service QuO AQoSA TAO Naming Service Frame filtering commands Observed throughput Reservation requests QuO Contract

7 Functional description of UAV architecture Video Source Process UAV SIMULATION HOST Video Distributor Process VIDEO DISTRIBUTION HOST Video Display Process VIDEO DISPLAY HOST 1 Display Proxy Process VIDEO DISPLAY HOST N Functionality Read bytes from a file Convert into frames Send out pipe Timing and sequencing of frames sent out Base Functionality Receive frames Send frames to registered receivers QuO Functionality QuO Delegate: Frame filtering (i.e., frame dropping) QuO Sysconds and Contract: Measuring frame rate Measuring resource usage and availability (CPU and network) Adapting to performance degradation (e.g., frame dropping, load balancing) Base Functionality Receive frames Display frames on the screen QuO Functionality QuO Sysconds and Contract: Measuring frame rate QuO Delegate: Removing time stamp Removing sequencing Discarding frame if late or out of sequence VIDEO DISPLAY HOST 2

8 COTS Middleware Components used in the UAV Software Quality Objects (QuO) –provides higher level programming model for specifying application-level QoS and adaptation CORBA ORB technology and services – TAO RT-CORBA ORB – CORBA Audio/Video Service

9 Quality Objects (QuO) Provides a higher level programming model for specifying application QoS –operating regions specified in QuO Contracts –transitions between regions trigger adaptive behaviors Support for different middleware architectures –CORBA (Java and C++) –Java RMI –local method call

10 QuO Adds Specification, Measurement, and Adaptation to the Distributed Object Model Application Developer Mechanism Developer CLIENT Network operation() in args out args + return value IDL STUBS IDL SKELETON OBJECT ADAPTER ORB IIOP ORB IIOP CLIENT OBJECT (SERVANT) OBJECT (SERVANT) OBJ REF CLIENT Delegate Contract SysCond Contract Network MECHANISM/PROPERTY MANAGER operation() in args out args + return value IDL STUBS Delegate SysCond IDL SKELETON OBJECT ADAPTER ORB IIOP ORB IIOP CLIENT OBJECT (SERVANT) OBJECT (SERVANT) OBJ REF Application Developer QuO Developer Mechanism Developer CORBA DOC MODEL QUO/CORBA DOC MODEL

11 CORBA Audio/Video Streaming Service Goals of OMG CORBA A/V Service Specification –Define standard mechanisms for: Stream Establishment Stream Control Multiple Flows Multiple Protocols QoS Goals of TAO A/V Streaming Service Project –Implement OMG CORBA A/V Service Specification using TAO

12 UAV Application High Level Design Source Stream Endpoint (distributor) Stream Adaptor Sink Stream EndPoint (Display) Stream Adaptor Video Stream TAO A/V Streaming Service ACE AQoSA API RSVP-enabled routers reservation request reservation accept/ reject event notifications.. reservationsflows QoS updates QuO measurement and control reservation notification Video Source Process UAV Video File Video Distributor VIDEO DISTRIBUTION HOST QuO Video Stream Video Display VIDEO DISPLAY HOST Video Display Proxy QuO

13 UAV Application: A View of the Software Components Video Source Process UAV SIMULATION HOST UAV Video File Video DIstributor VIDEO DISTRIBUTION HOST QuO Contract Video Display VIDEO DISPLAY HOST 1 Video Display Proxy CORBA A/V Streaming Service VIDEO DISPLAY HOST 2 VIDEO DISPLAY HOST N Reused Off the Shelf Software DVDView video player Common middleware services TAO CORBA A/V Service QuO AQoSA TAO Naming Service Frame filtering commands Observed throughput Reservation requests QuO Contract

14 UAV Application: Managed Behavior for Computational Resources In order to preserve end-to-end QoS requirements, different technologies must be used to manage the behavior of computational resources Processor Resources –RT-CORBA Network Resources –RSVP –DiffServ

15 RT-CORBA and UAV application Distributor will need to perform real-time processing of video frames and/or sensor data which may accompany the video, or video frames Distributor invokes a method on an RT-CORBA servant which performs data processing Thread priority of task is requested in the CORBA invocation Frame Filter Delegate TAO_AV_Protocol_ Object TAO_AV_ UDP_Object transport->send() distributor.{h,cpp} Distributor Code Receive Frame Event Callback wrapper-> send_frame() RT-CORBA Thread pool RT-CORBA servant RT-CORBA request Video frames

16 What is RSVP? Resource Reservation Protocol, specified in IETF RFC 2205 QoS properties are requested by an “out-of-band” signalling protocol Can be used to request a reserved bandwidth between a sender and receiver host Information about the reservation is stored in each intermediate router: “soft-state”

17 RSVP fragmentation problem MPEG in UDP packets were being fragmented by IP. Routers did not know that fragmented IP packets were part of a RSVP reservation of UDP packets. Solution: QuO delegates for fragmentation/reassembly, so UDP packets are not fragmented at IP level. Frame Filter Delegate TAO_AV_Protocol_ Object TAO_AV_ UDP_Object transport->send() distributor.{h,cpp} Distributor Code Receive Frame Event Callback wrapper-> send_frame() Host B (Distributor) receiver.{h,cpp} Receiver Code Receive Frame Event Callback Fragment Frame Delegate Reassemble Frame Delegate Host C (Receiver)

18 What is Diffserv? Diffserv Field (8-bits) Header Length (4-bits) Total Length (16-bits) Source Address (32-bits) Destination Address (32-bits) TTL (8-bits)Protocol (8-bits)Checksum (16-bits) Flags (3-bits) Version (4-bits) Identification (16-bits)Fragment offset (13-bits) Diffserv field: 6 bits of Diffserv Codepoint, 2 bits ECN Each DSCP (0-63) specifies a Per-Hop-Behavior (PHB), which is a type of router-level QoS (RFC 2475) IP Datagram Header

19 Specifying Diffserv QoS properties using the TAO AV Service CORBA IDL for AVStreams contains a method for changng the QoS characteristics of a stream: We have modified the TAO AV service, and use modify_QoS() to specify Diffserv codepoint values in video streams. AV service contributions are integrated in TAO 1.2. typedef sequence flowSpec; struct QoS { string QoSType CosPropertyService::Properties QoSParams; }; typedef sequence streamQoS; interface Basic_StreamCtrl { boolean modify_QoS(inout streamQoS, in flowSpec) raises (noSuchFlow, QoSRequestFailed) }

20 Integration of QuO QoS Components in UAV Application POA::Sender_ Control_ Callback Sender_ Control_ Callback Object receiver.{h,cpp} Receiver Code Receive Frame Event Callback Contract Normal High Excess Contract SendAllFrames SendTenFrames SendTwoFrames Frame Filter Delegate Expected Frame Rate Host A (Sender)Host C (Receiver) Actual Frame Rate QuO provides higher level feedback and control as video frames are transmitted from sender to receiver TAO_AV_Protocol_ Object TAO_AV_ UDP_Object transport->send() distributor.{h,cpp} Distributor Code Receive Frame Event Callback wrapper-> send_frame() Sender Control sender.{h,cpp} Sender Code wrapper-> send_frame() Header Delegate Host B (Distributor)

21 Creating RSVP and Diffserv Reusable Qoskets Desirable to create reusable network-centric QoS components (“Qoskets”), to make distributed, QoS enabled applications easier to develop. Code for RSVP qosket is currently being refactored out of existing application. Diffserv qosket needs to be rewritten. Perhaps a good use for the CORBA Component Model? New Qoskets paper available: RSVP Qosket Diffserv Qosket

22 Some Future Directions and Questions CORBA seems to work well as a signalling platform for QoS. Can we use it to request higher level QoS strategies instead of protocols like RSVP? –Create reusable Qosket components for RSVP, Diffserv, and other QoS properties How can we integrate with Resource Managers for requesting end-to-end QoS: network, CPU resources,….? How can we use more RT-CORBA features to deliver end-to- end QoS? – integration of behaviors like adaptive filtering How can we scale the application to send multiple simultaneous data streams (video, sensor readings)? –use more advanced features of CORBA A/V service

23 Distribution information QuO software available from: / UAV software available from: Help available from: Requires latest version of TAO ORB, version 1.2, available from: