An Overview of the OMG Data Distribution Service (DDS) Dr. Douglas C. Schmidt Vanderbilt University Nashville, Tennessee Institute for Software Integrated Systems

Overview of OMG Data Distribution Service (DDS) High Performance real-time data-centric publish/subscribe middleware ‣ The right data, at the right place, at the right time—all the time ‣ Fully distributed, multicast-enabled, high performance, highly scalable, & high availability, hot-swap/hot-hot architecture Blends data-centric & real-time publish/subscribe technologies ‣ Content-based subscriptions, (continuous) queries, & filters ‣ Fine-grained, policy-based tuning of resource usage, data delivery, availability QoS ‣ Optimal networking & computing resources usage Global Data Space Loosely coupled ‣ Plug & play architecture with dynamic discovery ‣ Time & space decoupling Open standard ‣ OMG DDS v1.2 latest version

DDS Architectural Elements Data-Centric Publish-Subscribe (DCPS) – The lower layer APIs apps can use to exchange topic data with other DDS- enabled apps according to designated QoS policies Data Local Reconstruction Layer (DLRL) – The upper layer APIs that define how to build a local object cache so apps can access topic data as if it were local DDS spec only defines policies & interfaces between application & service Doesn’t address protocols & techniques for different actors implementing the service Doesn’t address management of internal DDS resources

DDS Application Architecture DCPS Application DLRL Application DLRL Application DLRL Application DLRL Communication The Application

DDS Domains & Domain Participants The domain is the basic construct used to bind individual applications together for communication Like a VPN

Key DCPS Entities DCPS Entities include ‣ Topics ‣ Typed data ‣ Publishers ‣ Contain DataWriters ‣ Subscribers ‣ Contain DataReaders ‣ DomainParticipants ‣ Entry points Data can be accessed in two ways – Wait-based (synchronous calls) – Listener-based (asynchronous callbacks) Sophisticated support for filtering – e.g., Topic, Content-FilteredTopic, or MultiTopic Configurable via (many) QoS policies

DDS: Foundational Abstractions ‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object-oriented modeling ‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time ‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space ‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc. Global Data Space

Flat Modeling ‣ Modeling. Items distributed within the system are modeled as Topics ‣ Topics. Defined as associations between a data type & a set of QoS & identified by a unique name ‣ Data Types. The data type associated to a Topic is specified by means of IDL ‣ Subscriptions. Topics, Content- Filtered Topics, or Multi-Topics (join between topics) can be used to specify subscriptions. SQL expression are used to specify filters & joins struct StockQuote { string symbol; string name; sting exchange; float quointe; string xml_extra; }; How do I define the data & subscribe to it?

Relational Modeling ‣ Modeling. As in a Relational database, a DDS information model can be represented by means of Entity Relationship (ER) diagrams ‣ Topics. The entities, represented by means of Topics, are in turns an association between a data type & a set of QoS & identified by a unique name (like tables in an RDBMS) ‣ Data Types. The data type associated to a Topic must be a structured type expressed in IDL ‣ Instances. Key values in a datatype uniquely identify an instance (like rows in table) ‣ Correlation. SQL Expressions can be used to correlate information by means of key values struct StockQuote { string symbol; string name; sting exchange; float quote; string xml_extra; }; #pragma keylist StockQuote symbol How do I define the data & subscribe to it?

Object-Oriented Modeling DDS supports object-oriented distributed information modeling providing: ‣ Reduced Complexity & Improved Productivity ‣ Focus on the architecture & business logic, while hiding away the details involved with the diffusion of shared objects state ‣ Encapsulation ‣ Attributes are only accessible through dedicated getter/setter operations, i.e., don’t need to the messaging middleware or the application to have privileged access to business objects representation ‣ Local Operations ‣ Besides getters/setters, all other kind of manipulations can be done using custom operations ‣ Inheritance ‣ Single inheritance supported for Data Local Reconstruction Layer (DLRL) Objects ‣ Navigable Relationships ‣ Single Relationships ‣ Multi Relationships (Set, Map, List) Unleashing the power of Objects...

Relational Object Oriented Relational ↔ OO Mapping OO → Relational Mapping ‣ Middleware can automatically manage the generation & association between the DDS object-oriented model & the relational model Relational → OO Mapping ‣ The Relational Model can be mapped to an object-oriented model ‣ The mapping is under control of the architect Unleashing the power of Objects...

DDS: Foundational Abstractions ‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object-oriented modeling ‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time ‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space ‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc. Global Data Space

‣ Global Data Space can organized into domains, which in turn can have partitions ‣ Each partition is mapped to an IP Multicast Address ‣ QoS policies control the availability & consistency of data Partition IP Multicast per Partition Hardware Filtering Traffic Confinement Information Segregation Traffic Confinement How can I organize my data?

DDS: Foundational Abstractions ‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object-oriented modeling ‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time ‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space ‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc. Global Data Space

Publisher/DataWriter & Subscriber/DataReader Publisher/DataWriter ‣ Publishers manage dissemination of data samples ‣ Dissemination is driven by QoS policies associated with DataWriter, Publisher, & Topic ‣ A DataWriter is associated with one Publisher & one Topic; it embeds knowledge of dealing with Topic’s Data Type` Subscriber/DataReader ‣ Subscribers manage reception of data samples ‣ Presentation of data is driven by QoS policies associated with DataReader, Subscriber, & Topic ‣ A DataReader is associated with one Subscriber & one Topic; it embeds knowledge of dealing with Topic’s Data TypeNetwork

DDS: Foundational Abstractions ‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object oriented modeling ‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time ‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space ‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc. Global Data Space

DDS QoS Model ‣ QoS policies can be associated with all relevant DDS entities ‣ Some QoS policies are matched based on a Request vs. Offered Model ‣ Publications & Subscriptions match only if the declared & requested QoS are compatible ‣ e.g., it is not possible to match a publisher that delivers data unreliably with a subscriber that requires reliability How is QoS matched in the System?

QoS Policies ‣ Rich set of QoS policies allows applications to configure several different aspects of data availability, delivery, & timeliness ‣ Applications can use QoS policies to control/optimize network & computing resources QoS PolicyApplicabilityRxOModifiable DURABILITYT, DR, DWYNData Availability DURABILITY SERVICE T, DWNN LIFESPANT, DW-Y HISTORYT, DR, DWNN PRESENTATIONP, SYNData Delivery RELIABILITYT, DR, DWYN PARTITIONP, SNY DESTINATION ORDER T, DR, DWYN OWNERSHIPT, DR, DWYN OWNERSHIP STRENGTH DW-Y DEADLINET, DR, DWYYData Timeliness LATENCY BUDGET T, DR, DWYY TRANSPORT PRIORITY T, DW-Y TIME BASED FILTER DR-YResources RESOURCE LIMITS T, DR, DWNN USER_DATADP, DR, DWNYConfiguration TOPIC_DATATNY GROUP_DATAP, SNY What are the key QoS policies supported?

Data Timeliness

Deadline QoS Policy DEADLINE QoS policy defines the maximum inter-arrival time between data samples You can’t be later than... ‣ DataWriter indicates that the application commits to write a new value at least once every deadline period ‣ DataReaders are notified by the DDS when the DEADLINE QoS contract is violated ‣ DataWriter & DataReader policies must “match”

Latency Budget QoS Policy LATENCY_BUDGET QoS policy specifies maximum acceptable delay from time data is written until data is inserted in receiver's application-cache ‣ The default value of the duration is zero indicating that the delay should be minimized ‣ This policy is a hint to DDS, not something that must be monitored or enforced ‣ This QoS policy can be used to implement “batching” I need to get there in at most...

Transport Priority QoS Policy TRANSPORT_PRIORITY QoS policy is a hint to DDS infrastructure how to set priority of underlying transport used to send the data VIP Data, stay clear! ‣ TRANSPORT_PRIORITY can be used to implement priority bands that ensure high priority data preempts lower priority data ‣ With priority bands, priority inversion can be controlled by developers who design/deploys the system

Data Delivery

Reliability QoS Policy RELIABILITY QoS policy indicates the level of assurance offered by the DDS in delivering data to subscribers via the following values: ‣ Reliable. In steady-state the middleware ensures that all samples in DataWriter history will eventually be delivered to all DataReaders ‣ Best Effort. Indicates that it is acceptable to not retry propagation of any samples How much effort should be taken to deliver data?

Ownership QoS Policy OWNERSHIP QoS policy specifies whether multiple DataWriters can write same instance of data & if so how these modifications should be arbitrated, via following values ‣ Shared. Multiple writers are allowed to update the same instance & all the updates are made available to the reader ‣ Exclusive. Indicates that each instance can only be owned by one DataWriter, but owner of an instance can change dynamically due to liveliness changes ‣ Owner selection for exclusive controlled by OWNERSHIP_ STRENGTH QoS policy Who owns the data?

Ownership Strength QoS Policy OWNERSHIP_STRENGTH QoS policy specifies the value of “strength” used to arbitrate among DataWriters that attempt to modify same data instance ‣ Data instance are identified by tuple ‣ QoS policy applies only if OWNERSHIP is EXCLUSIVE How strong are you?

Data Availability

Durability QoS Policy ‣ Volatile. No need to keep data instances for late joining data readers ‣ Transient Local. Data instance availability for late joining data reader is tied to the data writer availability ‣ Transient. Data instance availability outlives the data writer ‣ Persistent. Data instance availability outlives system restarts DURABILITY QoS policy controls data availability wrt late joiners via the following variants DURABILITY_SERVICE QoS policy provide control over configuration of service that implements transient & persistent durability features For whom will this data be available?

Lifespan QoS Policy For how long will this data be available? LIFESPAN QoS policy allows applications to control what happens to stale data ‣ It specifies the validity (time) interval for data written by the DataWriter ‣ The default validity interval is infinite ‣ Different values regulate how long samples reside in caches with a DDS implementation

History QoS Policy HISTORY QoS policy controls whether DDS delivers only most recent value, attempt to deliver all intermediate values, or do something in between, via the following semantics: ‣ Keep Last. The DDS will only attempt to keep the most recent “depth” samples of each instance of data identified by its key ‣ Keep All. The DDS will attempt to keep all the samples of each instance of data identified by its key. ‣ Samples on DataWriter side are kept until delivered to all known subscribers ‣ Samples on DataReader side samples are kept until the application “takes” them How many data samples should I keep?

Concluding Remarks ‣ Next-generation QoS-enabled information management for tactical applications requires innovations & advances in tools & platforms ‣ Conventional SOA middleware technologies are poorly suited for these types of systems due to insufficient QoS support Benefits of DDS Standard-based pub/sub Very low latency & predictable data dissemination Support for >>100K messages/sec Stability under overload condition Scalability Fairness Control over latency/throughput tradeoffs Traffic Engineering Hardware Filtering Traffic Shaping Priority driven delivery High Performance Persistence Event Processing