Copyright © MilSOFT,Turkey UNCLASSIFIED1 Ertan DENIZ MilSOFT A.S, Teknokent ODTU,Ankara/Turkey Huseyin Kutluca, Video Over DDS DDS Information Day Brussels, Belgium-June-26,2007

Copyright © MilSOFT,Turkey UNCLASSIFIED2 Introduction Image Exploitation System for UAVs Why Use DDS for Video Streaming? MilSOFT DDS DDS Topics QoS choices Future Work Conclusion Outline

Copyright © MilSOFT,Turkey UNCLASSIFIED3 Introduction DDS middleware Location transparency with pub/sub mechanism Redundancy and fault tolerance mechanisms Multiple readers/writers Transient and persistent services Asynchronous, time-independent data distribution Rich set of QoS, suitable for real-time systems Wide range of Application Areas C4I Industrial automation Distributed control and simulation Telecom equipment control Sensor networks Use Case: Image Exploitation System for UAVs

Copyright © MilSOFT,Turkey UNCLASSIFIED4 Data Receiver Pre-processor DB Server Screening Exploitation 1 Exploitation 2 Sensor Data (EO/IR, SAR, GMTI) Telemetry High Capacity Storage Unit Raw Data Recording DDS based Data Streamin g Pre-processed & Georecorded Data Artifacts All Data in DB Raw Data DDS on UAV Image Exploitation System

Copyright © MilSOFT,Turkey UNCLASSIFIED5 DDS on UAV Image Exploitation System Data Events User information Resource Management Radar data streaming SAR streaming GMTI streaming Telemetry streaming Video streaming Live data streaming Recorded data streaming Data Publisher: DataReceiver Data Subscribers: Consoles and Pre-Processor Content of data MPEG-2 TS

Copyright © MilSOFT,Turkey UNCLASSIFIED6 MPEG-2 TS (Transport Stream) Elementary Stream Video, audio, telemetry data, etc... From each elementary stream Packetized Elementary Stream (PES) is formed And each PES is broken into fixed-size Transport Packets TS packets are 188 bytes of which 4 bytes are for the header and 184 bytes are for the payload Transport Stream can multiplex multiple elementary streams The TS packet header contains a Packet Identifier (PID) to identify different elementary streams 8 Mbit data rate for UAV Image Exploitation System

Copyright © MilSOFT,Turkey UNCLASSIFIED7 Why use DDS for video streaming? Publish-Subscribe simplifies the development process The clients do not need to know where the server is, and the server does not need to know the clients DDS specification implies mutlicast communication and the interoperability wire protocol RTPS supports and recommends multicasting Dramatically decreased bandwith usage with DDS implementations which support multicast Quality Of Service attributes like reliability, durability, transport priority directly can be used for video streaming channels Generally DDS implementations have very low latencies in data transmission compared to other middlewares, which makes it a good choice

Copyright © MilSOFT,Turkey UNCLASSIFIED8 Why MilSOFT DDS MilSOFT DDS is compliant with current DDS specification, version 1.2 Provides Minimum Profile QoS attributes, which are neccessary for video streaming Completely decentralized design No dynamic memory allocation after initialization Zero-copy access to data Implements multicast communications

Copyright © MilSOFT,Turkey UNCLASSIFIED9 More information on MilSOFT DDS Middleware

Copyright © MilSOFT,Turkey UNCLASSIFIED10 Video Over DDS Architecture

Copyright © MilSOFT,Turkey UNCLASSIFIED11 DDS Types and Topics Used For Video Streaming So how do we encapsulate MPEG2-TS packets into DDS types? There are two DDS types and two corresponding Topics. VideoHeader  VideoHeaderTopic VideoPacket  VideoPacketType VideoHeader Contains the header of the video, which is needed to initialize the decoder. (FPS, height, width, etc... ) VideoPacket Contains MPEG2-TS packets

Copyright © MilSOFT,Turkey UNCLASSIFIED12 VideoHeader Topic struct VideoHeader { long headerType; // key sequence headerData; }; Published once for each Transport Stream Contains the necessary information initialize the video decoder information about the other multiplexed elementary streams Key to distinguish the video header from other header data Late joiners also get this topic

Copyright © MilSOFT,Turkey UNCLASSIFIED13 QOS for VideoHeader Topic VideoHeaderTopic is necessary to initialize decoding of the video Header information is critical for clients Use Reliability QoS (RELIABLE) Both publishers and subscribers are Reliable Late joining viewers need this information Use Durability QoS (TRANSIENT_LOCAL) There are different instances of the header according to it’s type. We need only the last update of each instance! History QoS is default, kind = KEEP_LAST, depth = 1

Copyright © MilSOFT,Turkey UNCLASSIFIED14 VideoPacket Topic struct VideoPacketType { long frameNumber; sequence packetData; }; MPEG2-TS packets are too small in size (188 bytes). VideoPacket type sends multiple TS packets of a video frame at each VideoPacket sample Decreases the overhead and increases the bandwidth utilization A frame can span multiple VideoPacket samples (fragmented) Each sample contains TS packets of only one frame

Copyright © MilSOFT,Turkey UNCLASSIFIED15 QOS for VideoPacket Topic For VideoPacketTopic we may need both Reliable and Best Effort delivery, according to client needs Use Offered >= Requested semantics Server publishes with a Reliable writer ( Offer Reliable communication ) Clients subscribe with either Reliable or Best Effort readers Data is not needed by late joiners Durability is Volatile VideoPacket Type has no key, but we can use History QoS for buffering History QoS, kind = KEEP_LAST, depth value can be used as a buffer (set it to more than 1)

Copyright © MilSOFT,Turkey UNCLASSIFIED16 Other Concerns And Solutions We don’t want out of sequence frames DDS already satisfies this, no out of order packets! We don’t want frames which are too late Use Lifespan QoS Decoding takes time Either read data from a DataReaderListener and immediately put into the queue of another thread Use WaitSets

Copyright © MilSOFT,Turkey UNCLASSIFIED17 Future Work Implement advanced services like rewind, replay, fast-forward etc... Network reservations Use RSVP (IntServ) to reserve bandwith Analyze different methods to give DDS video streams a priority Use Transport_Priority QoS (not completely standard) More standardization is needed in this area Standardize Flow Control mechanisms in DDS for better utilization of network bandwith

Copyright © MilSOFT,Turkey UNCLASSIFIED18 Conclusion Video over DDS is possible! DDS provides a good set of QoS attributes for video streaming Performance requirements Image Exploitation system for UAVs uses 8 Mbit MPEG-2 TS 20 Mbit video streaming is possible with DDS

Copyright © MilSOFT,Turkey UNCLASSIFIED19 Thank You!