1. IST 37652 Hard Real-time CORBA HRTC 1 WP2: Hard Real-Time Protocols Thomas Losert, Miguel Segarra, Karl-Erik Årzén

2. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 2 Content  Introduction – Thomas Losert  CORBA & OCI – Miguel Segarra  Protocol Issues – Thomas Losert TTPIOP – Thomas Losert TTPIOP – Thomas Losert RTEIOP – Karl-Erik Årzén RTEIOP – Karl-Erik Årzén

3. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 3 Introduction  Decision on Testbeds is based on D2.1 „Analysis of Protocols for Real-Time Control“  Chosen TTP/C and RT-Ethernet as Protocols for the Testbeds  100% reliability usually is not necessary in Control Systems (thus no retransmissions)

4. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 4 Main Activities and Achievements  D2.2 HRT Protocol Specification  D2.3 HRT Protocol  Extra Documents: TTP Transport Definition RTE Transport Definition  2 Prototype Implementations based on OCI exploring different variants (e.g., connection-less and connection-oriented)

5. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 5 Recommendations from Previous Review regarding D2.2 We have implemented the following:  The selected pluggable framework (OCI) has been moved to the annex  A clarification for synchronization has been added  We have issued a revised version of the document (e.g., added timestamps, (a)periodic/sporadic tasks, next transmission time)

6. IST 37652 Hard Real-time CORBA HRTC 6 RT-CORBA & OCI Miguel Segarra

7. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 7 Contents  How does CORBA work?  Specifications vs Reality  Sources of nondeterminism  Real-Time CORBA Transport Selection  Transport Plugin Framework

8. IST 37652 Hard Real-time CORBA HRTC 8 How does CORBA work?

9. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 9 How does CORBA work? Server Side Client Side Object Request Broker IDLIDL IDLIDL POA CLIENTCLIENT SERVANTSERVANT IDL

10. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 10 How does CORBA work? Object Request Broker IDLIDL IDLIDL POA CLIENTCLIENT SERVANTSERVANT Client.GetValue(); Servant.GetValue();

11. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 11 How does CORBA work? Object Request Broker IDLIDL IDLIDL POA CLIENTCLIENT SERVANTSERVANT Object Request Broker IDLIDL IDLIDL POA CLIENTCLIENT SERVANTSERVANT GIOP Servant.GetValue(); Client.GetValue();

12. IST 37652 Hard Real-time CORBA HRTC 12 Specification vs Reality

13. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 13 Specification vs Reality Real-Time CORBA Messaging CORBA Minimum CORBA Time Service But close to the process resources are very limited!

14. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 14 Specification vs Reality CORBA Minimum CORBA Real-Time CORBA Messaging Time Service Dynamic part of CORBA In embedded/real-time systems a lot of decisions are made at design time!

15. IST 37652 Hard Real-time CORBA HRTC 15 Sources of non- determinism

16. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 16 Sources of non-determinism Stub Client thread priority Provide predictability by controlling ORB behavior ORB connection ORB Invoke priority protocol Skeleton POA thread priority concurrency Servant

17. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 17 Sources of non-determinism Stub Client Impact on ORB architecture of predictability issues ORB buffering ORB delay Skeleton POA thread dispatching Servant buffering request dispatching marhalling Memory mgmt

18. IST 37652 Hard Real-time CORBA HRTC 18 Real-Time CORBA Transport Selection

19. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 19 Real-Time CORBA Transport Selection A real-time CORBA broker

20. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 20 Real-Time CORBA Transport Selection RTOS A Client ORB AORB B Skeleton RTOS B Servant Stub TCP/IPTTP OtherTTPTCP/IP Object Reference Invocation

21. IST 37652 Hard Real-time CORBA HRTC 21 Transport Plugin Framework

22. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 22 Transport Plugin Framework Pluggable transport protocol Pluggable message support Pluggable protocol framework Acceptor Connector POA threads ORB Service handler

23. IST 37652 Hard Real-time CORBA HRTC 23 OCI: Open Communications Interface

24. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 24 Open Communications Interface  It is an interface (abstract) for a transport plugin framework It supports connection-oriented, reliable “byte-stream” transports. That is, transports which allow the transmission of a continuous stream of bytes (octets) from the sender to the receiver. It supports connection-oriented, reliable “byte-stream” transports. That is, transports which allow the transmission of a continuous stream of bytes (octets) from the sender to the receiver. Non-reliable or non-connection-oriented protocols can also be used if the transport plug-in itself takes care of reliability and connection management. Non-reliable or non-connection-oriented protocols can also be used if the transport plug-in itself takes care of reliability and connection management.

25. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 25 Open Communications Interface ORB Conn. Fact. Reg. Acc. Fact. Reg. CLIENT SIDESERVER SIDE Conn. Fact.Acc. Fact. Acceptor Connector Transport TCP/IP Conn. Fact. TCP/IP Acc. Fact. TCP/IP Acceptor TCP/IP Connector TCP/IP Transport 1 1 n n creates ABSTRACT IMPLEMENTATION

26. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 26 Open Communications Interface OCI OCI_TCPIP OCI+_RTE OCI+_TTP GIOP IIOP TTPIOP RTEIOP

27. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 27 Open Communications Interface  Hardware and software used Sun UltraSparc Sun UltraSparc Power PC for TTP nodes Power PC for TTP nodes Power PC for VME boards Power PC for VME boards Axis developer board Axis developer board  Operating systems Sun Solaris 8.0 Sun Solaris 8.0 RTAI Linux RTAI Linux

28. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 28 Open Communications Interface  We did not foresee to make a lot of modifications to the ORB core but at the end we needed to make modifications in order to have pure oneway requests and to provide special transport handles for RTEthernet

29. IST 37652 Hard Real-time CORBA HRTC 29 Protocol Issues Thomas Losert

30. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 30 General Issues  A Distributed Control System must be aware of the Progression of Time  The ORB must be aware of the Progression of Time

31. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 31 Adding Time Awareness  Proposed Extension of the Transport: protocol_time, next_transmission_time, delivery_time, and precision  Additional Interfaces in RTCORBA -module allow the Application to Read these Values via the RTObject which is an additional interface we have implemented  Time-Format should be Simple: Chosen Time Format from Smart Transducers Interface Specification (formal/2003-01-01)

32. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 32 Deadlines  In RTCORBA just Timeouts are supported (when set at client side)  Defined policies in the ORB but not implemented in the testbeds

33. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 33 Changes/Extensions  An RTORB contains a policy that decides between 100 ns and 60 ns  Timestamping (not implemented)  Proposed extension of IDL allows classification of tasks: periodic (RT), sporadic (RT), and aperiodic (no RT)

34. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 34 State vs. Event Values

35. IST 37652 Hard Real-time CORBA HRTC TTPIOP

36. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 36 Composability  We call an architecture composable with respect to a specified property, if the system integration will not invalidate this property provided it has been established at the subsystem level, e.g.: Timeliness Timeliness Testability Testability  System properties should follow from subsystem properties. Otherwise the system integrator is left with the challenging task to find out why the system does not work, although all subsystems work according to their specifications.

37. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 37 TDMA Media Access Real Time  TT communication system  Periodic transmission of state messages  Two redundant channels with TDMA Sending slots Sending slots TDMA rounds TDMA rounds

38. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 38 Flow of State and Event Information in TTPIOP Event Information State Information

39. IST 37652 Hard Real-time CORBA HRTC RTEIOP   Real-Time Communication over Switched Ethernet   ThrottleNet: The link layer in the RT Ethernet approach

40. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 40 Isolated collision domains Full duplex communication Inexpensive hardware (COTS) and high performance Traffic control (throttling) guarantees end-to-end latency Buffer delay causes jitter Switched Ethernet Switch

41. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 41 Periodic Real-Time Channel  One-way communication channel between sender and receiver  Parameters: Frequency (maximum send rate) Frequency (maximum send rate) Maximum transmit time (maximum message size) Maximum transmit time (maximum message size) Maximum allowed latency Maximum allowed latency  Schedulability analysis decides if latency constraint holds

42. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 42 Non Real-Time Traffic  Bandwidth is allocated to non-real time traffic  TCP, UDP, ….

43. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 43 Throttling  Traffic control through buffering  Ensure that sending nodes do not violate their periodicity  Bandwidth limitation  Applies to real-time traffic and non real- time traffic

44. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 44 Schedulability Analysis  Worst-case based  Calculate the latencies for the worst-case buffering scenario  Take the buffer overflow into account  May result in considerable jitter  Implemented by a special node (the “GlobeThrottle”)  Dynamic admission of new channels

45. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 45 GlobeThrottle GlobeThrottle: contains the global traffic information schedules the RT traffic requests updates the nodes when the schedule is changed router for non-RT traffic gateway to Internet converts incoming broadcasts to scheduled unicasts GlobeThrottle

46. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 46 RT-Layer RT layer:  Traffic control for worst- case scheduling  Distinguish RT and non- RT traffic (RT header)  Fragmentation Non RT traffic Non RT traffic RT traffic (may improve schedulability) RT traffic (may improve schedulability)

47. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 47 Related Work  K.G Shin et al (U of Michigan) Throttled shared Ethernet Throttled shared Ethernet Statistical latency bounds Statistical latency bounds  RTnet Time-triggered Real-Time Ethernet Time-triggered Real-Time Ethernet RTAI RTAI Univ of Hannover Univ of Hannover

48. IST 37652 Hard Real-time CORBA HRTC 48 RTEIOP: ThrottleNet CORBA

49. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 49 ThrottleNet and CORBA  ThrottleNet as the data link layer  Real-time CORBA requests on the same network as ordinary CORBA requests

50. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 50 Ordinary CORBA traffic  IIOP messages tunneled over ThrottleNet  Fragmented and buffered to not disturb the real-time traffic  ThrottleNet transparent to IIOP  Buffering the only effect on IIOP

51. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 51 Real-Time Traffic  GIOP requests from client to server mapped to ThrottleNet real-time channels  True one-way requests Support for this added to the ICa ORB Support for this added to the ICa ORB  GIOP messages tunneled over ThrottleNet

52. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 52 Real-Time Traffic  Explicit connection bindings -- validate_connection avoid binding related overhead avoid binding related overhead the communication between the client and server in connection with the binding is from a scheduling point of view be considered as being part of the real-time channel communication the communication between the client and server in connection with the binding is from a scheduling point of view be considered as being part of the real-time channel communication

53. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 53 ThrottleNet CORBA  OCI with two parallel transport plug-ins  The application decides which transport to use Ethernet Interface IP TCP IIOP RT-ORB RTE OCI+ CORBA client & server application code ThrottleNet RT

54. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 54 ThrottleNet R-T Client 1.Register the communication channel with the GlobeThrottle server. 2.If schedulability test in GlobeThrottle returns OK  Setup explicit connection  Communicate over the connection

55. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 55 GlobeThrottle  Could be implemented in different ways 1.As an ordinary CORBA object residing in one the nodes. Communication using IIOP Communication using IIOP 2.As a new CORBA “real-time scheduling” service

56. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 56 Main Characteristics  Close in spirit to ordinary CORBA Unicast Unicast Invocation-oriented event-triggered communication Invocation-oriented event-triggered communication  RT traffic and CORBA IIOP traffic coexist on the network  One-way communication  Hard upper bounds on latency, but considerable jitter  Open Issues: Is the GIOP message format efficient enough? Is the GIOP message format efficient enough?

57. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 57 Possible Future Extension  Global Clock Improves schedulability and reduces jitter Improves schedulability and reduces jitter Compare RTnet Compare RTnet  Schedulability analysis for two-way invocations

58. Copyright © 2002 The HRTC Consortium. www.hardrealtimecorba.org. All rights reserved.www.hardrealtimecorba.org IST 37652 / Hard Real-time CORBA 58 Outlook  Moving ORB from Usermode to Kernelmode is outside the scope of this project  Tool for offline-scheduling of IDL (for TTPIOP)  Clock Synchronisation (for RTEIOP)  Running the ORB in RTAI context or running the ORB in user context (e.g., with lxrt)