1. 백석대학교 궁상환 The OMG Data Distribution Service (Object Management Group)

2. Historically, mission-critical apps were built directly atop hardware & OS Tedious, error-prone, & costly over lifecycles Standards-based COTS middleware helps: Control end-to-end resources & QoS Leverage hardware & software technology advances Evolve to new environments & requirements Provide a wide array of reusable, off- the-shelf developer-oriented services There are layers of middleware, just like there are layers of networking protocols Hardware Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware Applications The Evolution of Middleware Operating Systems & Protocol

3. Distribution Middleware Distribution middleware defines higher-level distributed programming models whose reusable APIs & components automate & extend native OS capabilities Domain-Specific Services Common Middleware Services Distribution Middleware Host Infrastructure Middleware Distribution middleware avoids hard-coding client & server application dependencies on object location, language, OS, protocols, & hardware Examples OMG Real-time CORBA & DDS, Sun RMI, Microsoft DCOM, W3C SOAP Data Distribution Service CORBA CORBA : Common Object Request Broker Architecture

4. Overview of the Data Distribution Service (DDS) Application ‘Global’ Data Store read write DDS supports continuous data distribution among distributed applications DDS provides flexibility, power, & modular structure by decoupling: Location – anonymous pub/sub Redundancy – any number of readers & writers QoS – async, disconnected, time-sensitive, scalable, & reliable Platform & protocols – portable & interoperable Network latency & bandwidth Workload & Replicas CPU & memory Connections & priority bands

5. DDS Application Architecture DCPS Application DLRL Application DLRL Application DLRL Application DLRL Communication The Application { DCPS : Data-Centric Publish-Subscribe DLRL : Data Local Reconstruction Layer

6. DDS Architectural Elements Data-Centric Publish-Subscribe (DCPS)  Targets towards efficient delivery of the proper information to proper recipient  The lower layer APIs apps can use to exchan ge topic data with other DDS-enabled apps a ccording to designated QoS policies Data Local Reconstruction Layer (DLRL)  Allows for a simple integration of the Service into the App layer  The upper layer APIs that define how to build a lo cal object cache so apps can access topic data as if it were local DDS spec only defines policies & interfaces between application & service Doesn’t address protocols & techniques for different actors implementing the service Doesn’t address management of internal DDS resources

7. DDS Domains & Domain Participants 1 2 3 1 2 3 1 1 DomainParticipant Node Domain 1 Domain 2 Domain 3 Node The domain is the basic construct used to bind individual applications together for communication : like a VPN

8. DCPS Entities DCPS Entities include  Topics  Typed data  Publishers  Contain DataWriters  Subscribers  Contain DataReaders  DomainParticipants  Entry points Topic Data Reader Data Writer Data Reader Data Writer Subscriber Publisher Subscriber Data Domain Domain Participant

9. Data Reader R Data Writer R PublisherSubscriber Topic R Benefits of DDS Tactical Network & RTOS DDS Pub/Sub Using Proposed RT Info to Cockpit & Track Processing A highly efficient OMG pub/sub standard Fewer layers, less overhead RTPS over UDP will recognize QoS Real-Time Publish Subscribe (RTPS) Protocol

10. Data Reader R Data Writer R PublisherSubscriber Topic R NEW TOPIC NEW SUBSCRIBER DDS provides meta-events for detecting dynamic changes NEW PUBLISHER Benefits of DDS

11. DDS provides policies for specifying many QoS requirements of tactical information management systems, e.g., Establish contracts that precisely specify a wide variety of QoS policies at multiple system layers Data Reader R Data Writer R PublisherSubscriber S1 S2 S3 S4 S5 S6 S7 S6S5S4S3S2S1 Topic R S7 X HISTORY RELIABILITY COHERENCY RESOURCE LIMITS LATENCY Benefits of DDS

12. Data Reader R Data Writer R PublisherSubscriber S1 S2 S3 S4 S5 S6 S7 Topic R SOURCE FILTER DESTINATION FILTER TIME-BASED FILTER Move processing closer to data : filtering Benefits of DDS

13. Overview of DDS Implementation Architectures  Decentralized Architecture  embedded threads to handle communication, reliability, QoS etc  Federated Architecture  a separate daemon process to handle communication, reliability, QoS, etc.  Centralized Architecture  one single daemon process for domain node Network daemon node Network node Network daemon node daemon

14. NDDS (Decentralized Architecture) Pros : Self-contained communication end-points, needs no extra daemons Cons : User process more complex, e.g., must handle config details (efficient discovery, multicast) Participant comm/ aux threads Network User process Node (computer) Participant comm/ aux threads User process Node (computer)

15. OpenSplice (Federated Architecture) Pros : Less complexity in user process & potentially more scalable to large # of subscribers Cons : Additional configuration / failure point; overhead of inter-process communication Participant aux threads Networ k User process Node (computer) comm threads Daemon process Participant aux threads User process Node (computer) comm threads Daemon process

16. TAO DDS (Centralized Architecture) Pros: Easy daemon setup Cons: Single point of failure, scalability problems (centralized) Participant comm threads User process Node (computer) Aux + comm threads Daemon process Participant comm threads User process Node (computer) Networ k Node (computer) data control

17. QoS Policies Supported by DDS(1)  DCPS entities (i.e., topics, data readers/writers) configurable via QoS policie s  Types of QoS  DEADLINE  Establishes contract regarding rate at which periodic data is refreshed  Reader expects a new sample at least once every deadline period  LATENCY_BUDGET  Establishes guidelines for acceptable end-to-end delays  TIME_BASED_FILTER  Mediates exchanges between slow consumers & fast producers  Ex. At most one for minimum separation  RESOURCE_LIMITS  Controls resources utilized by service  Ex. Number of data samples to be managed

18. QoS Policies Supported by DDS(2)  RELIABILITY (BEST_EFFORT, RELIABLE)  Enables use of real-time transports for data  HISTORY (KEEP_LAST, KEEP_ALL)  Controls which data values are delivered for the case of data changes before complete delivery  DURABILITY (VOLATILE, TRANSIENT, PERSISTENT)  Determines if data outlives time when they are written  … and many more …  Transport priority  Ownership  Presentation  Lifespan  Liveness  Partition …

19. Best Effort Reliability QoS in DDS Publisher Subscriber QoS Reliability = BEST_EFFORT Data is sent without reply Best Effort with the policies Publishers and subscribers match and obey QoS Deadline policy settings Time-based Filter QoS policy gives bandwidth control

20. Reliable QoS in DDS QoS Reliability = RELIABLE Topic R Data Reader R Data Writer R PublisherSubscriber history S1 S2 S3 S4 S5 S6 S7 S6S5S4S3S2S1 Ordered instance delivery is guaranteed

21. Tradeoff Between Reliability & Determinism X X X QoS Reliability = RELIABLE QoS Reliability = BEST_EFFORT Can’t have reliability and determinism Best effort mode for streaming data No retries of dropped packets Minimizes latency Reliable mode for commands & events Retry dropped packets until timeout Every packet received in order Speculative cache mechanism DDS reliability is tunable : RESOURCE_LIMIT Can be adjusted per subscription.

22. Single vs. Multiple Domain Systems

23. Data Writers & Publishers  Data Writers are the primary access point for an application to publish data into a DDS data domain  The Publisher entity is a container to group together individual Data Writers  User applications  Associate Data Writers with Topics  Provide data to Data Writers  Data is typically defined using OMG IDL  It can therefore be as strongly or weakly typed as you desire

24. Data Readers & Subscribers  A Data Reader is the primary access point for an application to access data that has b een received by a Subscriber  Subscriber is used to group together Data Readers  Subscribers & Data Readers can have their own QoS policies  User applications  Associate Data Readers with Topics  Receive data from Data Readers using Listeners (async) or Wait-Sets (sync)  Take & Read  take() : have ownership  read() : not remove from buffer

25. Types & Keys  A DDS Type is represented by a collection of data items  e.g. “IDL struct” struct AnalogSensor { string sensor_id; // key float value; // other sensor data };  A subset of the collection is designated as the Key.  The Key can be indicated by IDL annotation #pragma DDS_KEY AnalogSensor::sensor_id  A Type is associated with generated code: AnalogSensorDataWriter // write values AnalogSensorDataReader // read values AnalogSensorType // can register itself with Domain

26. Topics  A DDS Topic is the connection between publishers & subscribers  A Topic is comprised of a Name and a Type  Name must be unique in the Domain  Many Topics can have the same Type  Provision is made for content-based subscriptions  MultiTopics correspond to SQL join  Content-Filtered Topics correspond to SQL select Domain ID 35 Topic name “MySensor” Type string sensor_id[ key ] float value “AnalogSensor” name

27. Topic-Based Publish/Subscribe  DataWriter is bound to a single Topic  DataReader is bound with one or more topics (Topic, ContentFilteredTopic, or MultiTo pic) Pressure Temperature

28. Subscription = Topic + DataReader Topic 1 application Topic 2Topic n Data Reader Data Reader Data Reader subscriber QoS

29. Content-based Subscriptions  ContentFilteredTopic (like a DB-selection)  Enables subscriber to only receive data-updates whose value verifies a condition  e.g. subscribe to “Pressure” of type AnalogData where “value > 200”  MultiTopic (like a DB-join operation)  Enables subscription to multiple topics & re-arrangement of the data-format  e.g. combine subscription to “Pressure” & “Temperature” & re-arrange the data into a new type: struct { float pres; float temp; };

30. DDS Content-Filtered Topics Content-Filtered Topic Topic Filter Expression: Value > 260 Expression Params Topic Instances in Domain Instance 1 Instance 2 Instance 3 Instance 4 Instance 5 Instance 6 Instance 7 Value = 249 Value = 230 Value = 275 Value = 262 Value = 258 Value = 261 Value = 259 Filter Expression and Expression Params determine which Topic instances the Subscriber receives.

31. DDS MultiTopic Subscriptions Topic MultiTopic Data Reader Subscriber MultiTopics can combine, filter, and rearrange data from multiple Topics. Domain Participant

32. Example: Create Domain Participant // used to identify the participant DomainId_t domain_id =...; // get the singleton factory instance DomainParticipantFactory_var dpf = DomainParticipantFactory::get_instance (); // create domain participant from factory DomainParticipant_var dp = dpf->create_participant (domain_id, PARTICIPANT_QOS_DEFAULT, NULL); DomainParticipant object acts as factory for Publisher, Subscriber, Topic and MultiTopic entity objects

33. Example: Create Topic …… // register the data type associated with the topic FooDataType foo_dt; foo_dt.register_type (dp, “Foo”); // create a topic Topic_var foo_topic = dp->create_topic (“Foo_topic”, // topic name “Foo”, // type name TOPIC_QOS_DEFAULT, // Qos policy NULL); // topic listener

34. Example: Create Subscriber and DataReader …… // create a subscriber from the domain participant SubscriberQos sQos; dp->get_default_subscriber_qos (sQos); Subscriber_var s = dp->create_subscriber (sQos, NULL); // create a data reader from the subscriber // and associate it with the created topic DataReader_var reader = s->create_datareader (foo_topic.in (), DATAREADER_QOS_DEFAULT, NULL); FooDataReader_var foo_reader = FooDataReader::_narrow (reader.in ()); downcast

35. Example: Create Publisher and DataWriter …… // create a publisher from the domain participant PublisherQos pQos; dp->get_default_publisher_qos (pQos); Publisher_var p = dp->create_publisher (pQos, NULL); // create a data writer from the publisher // and associate it with the created topic DataWriter_var writer = p->create_datawriter (foo_topic.in (), DATAWRITER_QOS_DEFAULT, NULL); // narrow down to specific data writer FooDataWriter_var foo_writer = FooDataWriter::_narrow (writer); // publish user-defined data Foo foo_data; foo_writer->write (foo_data);

36. How to Get Data (Async Listener-based) Listener_var subscriber_listener = new MyListener(); foo_reader->set_listener(subscriber_listener); MyListener::on_data_available(DataReader reader) { FooSeq_var received_data; SampleInfoSeq_var sample_info; reader->take(received_data.out (), sample_info.out (), ANY_SAMPLE_STATE, ANY_LIFECYCLE_STATE); // Use received_data …… }

37. How to Get Data (Sync Wait-based) Condition_var foo_condition = reader->create_readcondition(ANY_SAMPLE_STATE, ANY_LIFECYCLE_STATE); WaitSet waitset; waitset->attach_condition(foo_condition); ConditionSeq_var active_conditions; Duration_t timeout = {3,0}; // like select call waitset->wait(active_conditions.out (), timeout);... FooSeq_var received_data; SampleInfoSeq_var sample_info; reader->take_w_condition(received_data.out (), sample_info.out (), foo_condition); // Use received_data

38. subscribers publishers Summary of DCPS features  Efficient Publish/Subscribe interfaces  QoS suitable for real-time systems  deadlines, levels of reliability, latency, resource usage, time-based filter  Listener & wait-based data access suits different application styles  Support for content-based subscriptions  Support for data-ownership  Support for history & persistence of data-modifications DDS Information consumer subscribe to information of interest Information producer publish information DDS matches & routes relevant info to interested subscribers

39. Data Local Reconstruction Layer (DLRL) Track 3D_Track Track_Topic3D -Track DLRL DCPS Cache

40. Goals of DLRL  Provides more direct access to exchanged data  Integrated with native-language constructs in Object-oriented way  Application developers can  describe classes with their methods, data fields, & relations  attach some of those data fields to DCPS entities  manipulate these objects (i.e., create, read, write, delete) using native la nguage constructs  activate attached DCPS entities to update objects  have those objects managed in a cache

41. Levels of mapping between two Layers  Structural mapping  Relations between DLRL objects and DCPS data  Operational mapping  Mapping of DLRL objects to the DCPS Entities (Publisher, DataWriter, etc)  Functional Mapping  Relations between the DLRL functions (mainly access to DLRL objects) and the DCPS functions (write/publish/etc)

42. Oid 2 z 300 T3D_TOPIC Oid 11 RADAR_TOPIC Oid 1 2 x 100 102 y 200 201 TRACK_TOPIC Class Track Track3D radar 11 COMMENTS_TOPIC Oid 1 1 comments a comment another comment Class Track index 0 1 R_Oid 11 RADAR_TRACKS_TOPIC T_Oid 1 2 T_Class Track Track3D index 0 1 DLRL Radar Classes & Topics Track x : real y : real comments [*] : string w : integer Track3D z : real Radar x : real y : real comments [*] : string z : real tracksa_radar *0..1 w : integer

43. Evaluating DDS Pros  Low overhead & efficient use of transport bandwidth  e.g., less features/overhead than CORBA in the main request path  Dynamically scalable & highly flexible  e.g., can receive notification about all sorts of meta-events, such as new topics, publishers, s ubscribers, etc.  Supports one-to-one, one-to-many, many-to-one, & many-to-many communication  Large number of configuration parameters & QoS policies that give developers extensive cont rol of message transmission & reception Cons  DDS is not well suited to request-reply services, file transfer, & transaction processing  The data-distribution paradigm is not optimized for sending a request & waiting for a reply  Implementations don’t yet cover the entire spec  Lack of interoperability between different vendor’s implementations

44. Comparing CORBA with DDS Distributed object Client/server Remote method calls Reliable transport Best for Remote command processing File transfer Synchronous transactions Distributed data Publish/subscribe Multicast data Configurable QoS Best for Quick dissemination to many nodes Dynamic nets Flexible delivery requirements DDS & CORBA address different needs Complex systems often need both…

45. Comparing DDS with JMS DDS Topic(Many-toMany) QoS Configured Real-time, scalable dissemination Configuration Date Writer/Reader Publisher/Subscriber DomainParticipant Domain MOM Service Provider Messaging Servers Messaging API Client Client Client IBMMQSeries OpenUMSObjectWebJoram FalconMSMQ SUN Java MessageQueue ScriptWAVEJMS SofwiredMQ FiranoMQ Java Message Service(JMS) API Messages JMS Topic(One-to-Many), Queue(One-toOne) Limited QoS : reliability Transaction-Oriented Service Configuration Publisher/Subscriber, Sender/Receiver Session Connection