1. 1 5/4/99ISORC ‘99 BBN Technologies An Object-level Gateway Supporting Integrated Property Quality of Service Rick Schantz John Zinky, David Karr, Dave Bakken, James Megquier, Joe Loyall, Partha Pal BBN Technologies ISORC ‘99 Saint Malo, France May 4, 1999 “Adapt or perish, now as ever, is nature's inexorable imperative.” – H. G. Wells QuO DARPA Air Force Research Lab/Rome Laboratory

2. 2 5/4/99ISORC ‘99 BBN Technologies Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status QuO

3. 3 5/4/99ISORC ‘99 BBN Technologies Many Distributed Systems have Critical QoS Requirements, e.g., Performance, Dependability, and Survivability

4. 4 5/4/99ISORC ‘99 BBN Technologies Chronology 197519801985199019951970 ARPANETARPANET EthernetEthernet IP/TCPIP/TCP OSF-DCE OMG/CORBAOMG/CORBA PC JAVA OLE/DCOM SBS RSEXEC National Software Works (NSW) CRONUS/CORBUS QualityObjects (QuO) NGI? WWW [Simnet] C++ TENEX/ Multics UNIX MS-DOS 200? Distrib Objects MessagePassing RPC Distrib SQL

5. 5 5/4/99ISORC ‘99 BBN Technologies State of the Art Today No effective way to control behavior of critical applications in today’s highly networked environment Gap between –Emerging low-level mechanisms to control some types of resources –High-level strategies appropriate for adaptable, dependable systems Researchers generally working on one piece of the solution e.g. –Focus on one critical QoS property: real time behavior, fault tolerance... –Focus on implications of one time epoch; (boundaries are fuzzy and getting fuzzier with improved dynamic binding) –Functional integration has taken precedence over system properties End goal is support building integrated QoS, adaptive applications (with varying requirements on granularity of changing behavior): –adaptable: change at runtime while application/service running reconfigurable: change at runtime while application/service halted evolvable: change at development time

6. 6 5/4/99ISORC ‘99 BBN Technologies The Quality Objects (QuO) framework supports development of distributed applications with QoS QuO The QuO framework provides Separation of concerns between software functional properties and QoS needs Consistent interfaces for QoS measurement and resource management control Standard middleware interfaces between application and QoS-provider layers Facilities to enable application- and system-level adaptation QuO is being developed as a common approach to adaptable QoS for a number of projects focusing on different QoS dimensions: –Dependability, replication, group communication (U.Illinois, Cornell, UCSB) –Managed bandwidth, resource reservation (CMU, Columbia Univ.) –Real-time behavior (Washington Univ in St. Louis) –Security, access control, survivability, intrusion detection (TIS) –Integrating different dimensions in an application, e.g., real-time and dependability –Suitability for supporting different approaches to a dimension

7. 7 5/4/99ISORC ‘99 BBN Technologies Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status QuO

8. 8 5/4/99ISORC ‘99 BBN Technologies QuO’s emphasis is on specification, measurement, control, integration, and adaptation, not enforcement QuO is based upon distributed object computing –Prototype system is based on CORBA, but the concepts are equally applicable to DCOM, RMI QuO provides a combination of high-level languages for specifying aspects of handling QoS –Aspect Oriented Programming –Code generators generate executable code from the high-level descriptions QuO enables the application to control and measure QoS and adapt to changes in the level of QoS provided If a manager or mechanism(s) exists that enforces QoS, QuO provides the means to interface to it

9. 9 5/4/99ISORC ‘99 BBN Technologies System condition objects monitor QoS in the system system condition objects recognize changes in the system and notify the contracts that observe them QuO contracts notify client programs, users, managers, and other system condition objects through transition behavior System Condition Objects QuO applications specify, control, monitor, and adapt to QoS in the system Application Alternate Implementations Contract (operating regions) Servers Network ORB Replication Mgr Resource Reservation Manager IDS Specification of operating regions, alternate implementations, and adaptation strategies using QuO’s QDL Multiple layers of adaptation managers and mechanisms can adapt to changes in the system QuO contracts provide another layer of adaptation Client and user can also adapt Mechanisms and managers control QoS in the system a layer below QuO that provides ORB-level services, such as managed communi- cation, replication, or security contracts and delegates interface to these services through system condition objects

10. 10 5/4/99ISORC ‘99 BBN Technologies Simplified DOC (CORBA) Runtime Components ClientNetwork Server Application Developer Middleware Developer Logical Method Call Client ORB Proxy COTS ORB Object ORB Proxy COTS ORB Network

11. 11 5/4/99ISORC ‘99 BBN Technologies A QuO application contains additional components (from traditional DOC applications) Contracts summarize the possible states of QoS in the system and behavior to trigger when QoS changes –Regions can be nested, representing different epochs at which QoS information becomes available, e.g., negotiated regions represent the levels of service a client expects to receive and a server expects to provide, while reality regions represent observed levels of service –Regions are defined by predicates over system condition objects –Transitions specify behavior to trigger when the active regions change System condition objects are used to measure and control QoS –Provide interfaces to system resources, client and object expectations, mechanisms, managers, and specialized ORB functions –Changes in system condition objects observed by contracts can cause region transitions –Methods on system condition objects can be used to access QoS controls provided by resources, mechanisms, managers, and ORBs Delegates implement the QoS specific adaptivity behavior –Upon method call/return, delegate can check the current contract state and choose behavior based upon the current state of QoS –For example, delegate can choose between alternate methods, alternate remote object bindings, perform local processing of data, or simply pass the method call or return through

12. 12 5/4/99ISORC ‘99 BBN Technologies Measured capacity >= 10 As_expected: Insufficient_resources: Measured capacity < 10 Contracts summarize system conditions into regions and define transitions between them The contract defines nested regions representing possible states based upon measured system conditions Predicates using system condition objects determine which regions are valid Transitions occur when a region becomes invalid and another becomes valid Transitions might trigger adaptation by the client, object, ORB, or system Normal: Expected capacity >= 10 Degraded: Expected capacity < 10 Expected capacity >= 2 As_expected: Extra_resources: Measured capacity < 10 Measured capacity >= 2 Measured capacity < 2 Insufficient_resources: Measured capacity >= 10 Unusable: Expected capacity < 2 As_expected: Extra_resources: Measured capacity < 2 Measured capacity >= 2 = Expected Region = Reality Region

13. 13 5/4/99ISORC ‘99 BBN Technologies System Conditions Project a Value to the Application, But also Must Maintain the Value Simple Value Measured Value (Sensor) Composed Value Application Developer QoS Developer Mechanism Developer QuO Kernel Control Value Specialized ORBs or Services RSVP Controller Control Value Status Value CORBA Object Device Status Service

14. 14 5/4/99ISORC ‘99 BBN Technologies Delegates change their behavior based on their contract’s current regions Dispatch Contract Pass ThroughBlockShape PreMethod Measurements Method Call Lower Method Call Current Regions Result Lower Result SysCond Current Regions PostMethod Measurements Delegate Alternative Behaviors

15. 15 5/4/99ISORC ‘99 BBN Technologies The QuO Toolkit provides tools for building QuO applications CORBA IDL Code Generators Code Generators Contract Description Language (CDL) QuO Runtime Structure Description Language (SDL) Delegates Contracts Quality Description Languages (QDL) –Support the specification of QoS contracts (CDL), delegates and their adaptive behaviors (SDL), connection, creation, and initialization of QuO application components (TBD) –QuO includes code generators that parse QDL descriptions and generates Java and C++ code for contracts, delegates, creation, and initialization System Condition Objects, implemented as CORBA objects QuO Runtime Kernel –Contract evaluator –Factory object which instantiates contract and system condition objects

16. 16 5/4/99ISORC ‘99 BBN Technologies QuO adds QoS control and measurement into the DOC remote method call ClientNetworkServer Application Developer Qosketeer Mechanism Developer Logical Method Call Client Delegate ORB Proxy Specialized ORB Contract SysCond Object Delegate ORB Proxy Specialized ORB Contract Network Mechanism/Property Manager SysCond

17. 17 5/4/99ISORC ‘99 BBN Technologies Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status QuO

18. 18 5/4/99ISORC ‘99 BBN Technologies QuO Gateway IIOPGlue Control QuO Gateways Support Specialized Communication Protocols and Below the ORB Mechanisms Client-Side ORB IIOP Group Replication (AQuA) WAN Bandwidth Reservation (DIRM) IIOP over TCP/IP (default) IIOPGlue Control IIOP Server-Side ORB The QuO gateway enables insertion of below-the-ORB mechanisms and specialized network controls The gateway translates IIOP messages into specialized communication protocols or network level controls To the client-side, the QuO gateway looks like the remote ORB To the object-side, the QuO gateway looks like the client’s ORB The two ends of the gate- way are on the same LAN as the client/object Currently, we have gate- ways that support Ensemble group communication, RSVP resource reservation, and IIOP over TCP/IP

19. 19 5/4/99ISORC ‘99 BBN Technologies QuO Based Bandwidth Management Measures and Controls Network Resources ORB Proxy Commercial ORB contract Measured Performance Skeleton Contract Bandwidth Control Bandwidth Control User Expectation Commercial ORB Logical Method Call Expected Performance Rights Policies Measured Performance Application Manager Status Collection Network Management Rights Policies Server Host Skeleton Client Object Delegate Sensors QuO Gateway SNMP custom Corba CORBA ORB Proxy CORBA With Trace Record Functional Signal Client Host Network IIOP Glue IIOP Glue IIOP Resource Aware Transport Resource Aware Transport IIOP RSVP bandwidth reservation Darwin bandwidth reservation Host Status Sensor Host Status Sensor Functional Sensors Functional

20. 20 5/4/99ISORC ‘99 BBN Technologies An RSVP Aware Gateway can Set up and Tear Down RSVP Sessions Between Hosts. QoSME from Columbia University will be used to control RSVP for the -ORB QuO-GTWY QoSME RT. RSVP QoSME QuO-GTWY Client Bandwidth Reservation API RT.RSVP RSVP Reservation Object ORB Proxy Smarts InCharge CORBA QoS API QoS Socket API IIOP QuO D & C QuO Delegate and Contract ORB Proxy SysCond QoSMib Application Management API SNMP

21. 21 5/4/99ISORC ‘99 BBN Technologies A simplified example: Improved Dependability through Replication Contract Low Cost: Requested = 1 Too Low: Measured < 1 Normal: Measured > 1 Available: Requested >= 2 Too Low: Measured < 2 Normal: Measured > 2 Object Replica Group Client application Code Remote method call Object Delegate Region? NormalMulticast

22. 22 5/4/99ISORC ‘99 BBN Technologies Replication Groups: composed of one or more identical objects Connection Groups: composed of the members of two replication groups that wish to communicate Connection group 1 Connection group 3 Connection group 2 Replication group 1 Replication group 4 Replication group 2 Replication group 3 Proteus manager 1 Proteus manager 2 Proteus manager 3 PCS group Object Factory 1 Object Factory 2 Object Factory 3 Object Factory 4 Object Factory 5 Point-to-Point groups PCS Group: composed of the Proteus replicas and objects that desire to communicate with the Proteus manager Point-to-point Groups: composed of one dependability manager replica and one object factory Groups in AQuA

23. 23 5/4/99ISORC ‘99 BBN Technologies CORBA ORB Client IIOP Glue Dispatcher Handler #1 Voter Group Member Buffers Maestro/Ensemble Object Gateway-Half Process Application Process

24. 24 5/4/99ISORC ‘99 BBN Technologies Network DTEL++ and OO DTE add access control into the DOC remote method call NetworkServer Application Developer DTEL++ Security Developer Mechanism Developer Logical Method Call Client ORB Proxy ORB Object ORB Proxy ORB DTE metadata OO domain definitions type bindings for interfaces, and operations; policy distribution manager ORB Gateway OO DTE Client

25. 25 5/4/99ISORC ‘99 BBN Technologies The QuO Gateway Manages IIOP Connections and Interfaces to Protocols which Manage QoS Shell is a Base to build Specialized QoS Aware Gateways To the “Client” ORB, the QuO Gateway looks like the object To the “Server” ORB, the QuO Gateway looks like a client The ends of the gateway are on the same LAN as the Client/Object and may be on the same host IIOP Glue will multiplex and demultiplex GIOP messages IIOP Glue Specialized Protocol IIOP Glue Control ClientServer Control WAN IIOP Client-Side Gateway half Server-Side Gateway half

26. 26 5/4/99ISORC ‘99 BBN Technologies Client-Side Interface Presents a Partially Parsed GIOP Request and Call Block for the Reply The DSI interface returns a Server_Request structure. Additional information is derived from call context The suspended thread is remembered in Call block This info is bundled into a Call block and puts it on the Request_Queue for the specialized transport When the specialized transport attaches the reply and puts it on the Reply_Queue for the DSI wrapper. Mechanism Specific Handler IIOP Glue Client IIOP DSI Control Server Request Request Queue Reply Queue Context Call Block Thread Suspend Call Block

27. 27 5/4/99ISORC ‘99 BBN Technologies Server-Side Interface Accepts a Call Block and Creates a DII Request. The Call Block is removed from the Request_Queue The DII interface sends a Request structure to an object reference The return values is bundled into the Call block and put on the Reply_Queue for the specialized transport IIOP Glue Server IIOP DII Request Mechanism Specific Handler Control Request Queue Reply Queue Obj Ref Call Block Thread Suspend Call Block

28. 28 5/4/99ISORC ‘99 BBN Technologies IIOP Glue IIOP Glue RSVP Network RSVPd Bandwidth Aware Transport Instrumentation Bandw - Control SNMP CONVERTCONVERT QoSME Bandwidth Reservations RSVPd Bandwidth Aware Transport Instrumentation Bandw - Control SNMP QoSME CONVERTCONVERT Specialized Embedded Protocols: Network Resource Reservations

29. 29 5/4/99ISORC ‘99 BBN Technologies (Leader) C-Rep1 ORB GW Group Comm GW ORB S-Rep1 C-Rep2 ORB GW ORB S-Rep2 C-RepN ORB GW ORB S-RepM... (Leader) 1 2 4 5 6 7 1 2 5 6 7 1 2 5 7 5 33 3 Specialized Embedded Protocols: Group Communications

30. 30 5/4/99ISORC ‘99 BBN Technologies Outline Motivation & Context Background & Overview of QuO technology The Object Gateway Component Examples of Use Design Considerations Experience to Date & Current Status QuO

31. 31 5/4/99ISORC ‘99 BBN Technologies Status QuO version 1.0 released September 1998, including the QuO runtime kernel, CDL and SDL and their code generators, system condition object library, concept demonstration gateway, example applications, and documentation Version 2.0 scheduled for end of May 1999 –improved language and contract support –a revised general gateway mechanism supporting RSVP –instrumentation support –Quorum Distributed Object individual QoS property incremental results: DIRM bandwidth management using RSVP & CMU Beagle AQuA dependability using Proteus and Ensemble TAO real time ORB and schedulableevent channels TIS ORB based DTE access control with dynamic security policy (tentative)

32. 32 5/4/99ISORC ‘99 BBN Technologies QuO Architecture Delegate ORB Proxy Specialized ORB SysCond Delegate ORB Proxy Network Mechanism/Property Manager SysCond Specialized ORB WAN CORBA IDL Code Generators Code Generators Contract Description Language (CDL) QuO Runtime Structure Description Language (SDL) Delegates Client Object QuO Gateway Control Client-Side ORB Group Replication (AQuA) Bandwidth Reserve (DIRM) IIOP over TCP/IP (default) Control Object-Side ORB IIOP Glue IIOP Logical Method Call Contracts IIOP Glue IIOP Quality description languages –currently CDL (QoS contracts), SDL (adaptation strategies), simple connection language –soon to come: capabilities to describe real- time behavior (TAO’s RIDL) and security (TIS’s DTEL++), plus improved SDL and connection QuO runtime kernel System condition libraries, example applications, documentation QuO gateway and instantiations for group communication, resource reservation, TCP/IP, and (soon to come) security QuO instrumentation

33. 33 5/4/99ISORC ‘99 BBN Technologies QuOIN - Distributed Object Integration DARPA SM OO domain definitions type bindings for modules, interfaces, and operations ORB Gateway OO DTE SIGMA Trusted Information Systems  Darwin Control Obj Remos Collector Network Management SNMP Beagle Network Darwin Bandwidth Reservation Remos HostLoad RSVP Network RSVPd Bandwidth Aware Transport Instrumentation Bandw - Control SNMP RSVP Bandwidth Reservations RSVPd Bandwidth Aware Transport Instrumentation Bandw - Control SNMP QoSME Remos HostLoad QuO Based Bandwidth Mgmt QuO Based Dependability Mgmt Caching and Consistency Georgia Tech Common IDL Compiler Specialized Realtime ORB OO Security Realtime Control Language Cached Data & Impl Consistenc y Protocols Expectations Callbacks Contract Object Delegate IIOP / Proteus / Ensemble Interface Application Logic System Conditions Proteus Dependability Manager Application QuO AQuA Gateway Managed Network with Ensemble Object Replica Object Replica... Object Replica Object Replica Object Replica Object Factory Object Factory Object Factory Protocol Coordinator Advisor AQuA Gateway