Global Time in Distributed Real-Time Systems Dr. Konstantinos Tatas

OUTLINE Revision of real-time system Distributed real-time system requirements Global time Clock synchronization

What is real-time? Is there any other kind? A real-time computer system is a computer system where the correctness of the system behavior depends not only on the logical results of the computations, but also on the physical time when these results are produced. By system behavior we mean the sequence of outputs in time of a system.

Real-time means reactive A real-time computer system must react to stimuli from its environment The instant when a result must be produced is called a deadline. If a result has utility even after the deadline has passed, the deadline is classified as soft, otherwise it is firm. If severe consequences could result if a firm deadline is missed, the deadline is called hard. Example: Consider a traffic signal at a road before a railway crossing. If the traffic signal does not change to red before the train arrives, an accident could result.

Distributed RT system model From the POV of an outside observer, a real-time (RT) system can be decomposed into three communicating subsystems: –a controlled object (the physical subsystem, the behavior of which is governed by the laws of physics), –a “distributed” computer subsystem (the cyber system, the behavior of which is governed by the programs that are executed on digital computers) –a human user or operator The distributed computer system consists of computational nodes that interact by the exchange of messages. A computational node can host one or more computational components.

Design Challenges in Distributed Systems Theoretically, a distributed system features the same design challenges as a centralized embedded system in terms of performance, power consumption, battery life, etc. However, an additional challenge exists: synchronization between nodes

Definition of time Newtonian physics time model is adequate for most temporal phenomena and is much simpler than relativistic time Time is modeled as an infinite set [T] with the following properties: –{T} is an ordered set, that is, if p and q are any two instants, then either p is simultaneous with q, or p precedes q, or q precedes p, where these relations are mutually exclusive. We call the order of instants on the timeline the temporal order. –{T} is a dense set. This means that there is at least one q between p and r iff p is not the same instant as r, where p, q, and r are instants.

Events in the time model A section of the time line between two different instants is called a duration. An event takes place at an instant of time and does not have a duration. If two events occur at the same instant, then the two events are said to occur simultaneously. Instants are totally ordered; however, events are only partially ordered, Events can be totally ordered if another criterion is introduced to order events that occur simultaneously

Causal order In many real time systems determining cause and effect relations between events is of interest, especially determining the primary event Temporal order is necessary but not sufficient to establish causal order

Global time An important yet challenging task is maintaining a consistent global time in a distributed real-time system There is no global clock, only local clocks Local clocks drift arbitrarily No local clock is always correct A global time is an abstract notion that is approximated by properly selected microticks from the synchronized local physical clocks of an ensemble.

Global time Assume a set of nodes, each one with its own local physical clock that ticks with granularity. Assume that all of the clocks are internally synchronized with a precision Π, i.e., for any two clocks j, k, and all microticks i It is then possible to select a subset of the microticks of each local clock k for the generation of the local implementation of a global notion of time. We call such a selected local microtick i a macrotick (or a tick) of the global time. For example, every tenth microtick of a local clock k may be interpreted as the global tick, the macrotick, of this clock (see Fig. 3.2). If it does not matter at which clock k the (macro) tick occurs, we denote the tick ti without a superscript. A global time is thus an abstract notion that is approximated by properly selected microticks from the synchronized local physical clocks of an ensemble.

Reasonable global time The global time t is called reasonable, if all local implementations of the global time satisfy the condition g>Π Then for a single event e, that is observed by any two different clocks of the ensemble, their global time-stamps can differ by at most one tick. This is the best we can achieve.

Temporal order When to events differ by less than two ticks temporal order cannot be maintained

Errors in duration measurement

Internal clock synchronization The purpose of internal clock synchronization is to ensure that the real-time clocks of each correct node are within precision Π, independently of their drift rates. The global time ticks of each node must be periodically resynchronized within the ensemble of nodes to establish a global time base with specified precision. The period of resynchronization is called resynchronization interval. After that the clocks are left to drift again until they are resynchronized.

Internal clock synchronization The synchronization algorithm must bring the clocks so close together that the amount of divergence during the next free-running resynchronization interval will not cause a clock to leave the precision interval. Φ+Γ≤Π Where Φ is the convergence function and Γ is the drift offset Φ=2ρRint Where Rint is the length of the resynchronization interval and ρ is the maximum specified drift rate

Malicious clock clock synchronization can only be guaranteed in the presence of Byzantine errors if the total number of clocks N (3k + 1), where k is the number of Byzantine faulty clocks.

Central Master Synchronization the central master, periodically sends the value of its time counter in synchronization messages to all other nodes the slave records the time-stamp of message arrival. The difference between the master’s time, contained in the synchronization message, and the recorded slave’s time-stamp of message arrival, corrected by the known latency of the message transport, is a measure of the deviation of the clock of the master from the clock of the slave. The slave then corrects its clock by this deviation to bring it into agreement with the master’s clock. Used at system startup Not fault-tolerant

External clock synchronization External synchronization links the global time of a cluster to an external standard of time. For this purpose it is necessary to access a timeserver, i.e., an external time source that periodically broadcasts the current reference time in the form of a time message. GPS (Global Positioning System). –The accuracy of a GPS receiver is better than 100 ns and it has an authoritative long-term stability – in some sense, GPS is the worldwide measurement standard for measuring the progression of time. Alternatively, the external time source can temperature compensated crystal oscillators (TCXO) –Typical drift rate of better than 1 ppm, causing a drift offset of better 1 μs/s atomic clocks – Rubidum clock typical drift rate in the order of 10^-12 causing a drift offset of about 1 μs in 10 days.

Example 1 Given a clock synchronization system that achieves a precision of 90 μs, what is a reasonable granularity for the global time? What are the limits for the observed values for a time interval of 1.1 ms?

Example 2 Given a –latency jitter of 20 μs, –a clock drift rate of 10^-5 s/s, and –a resynchronization period of 1 s what precision can be achieved by the central master algorithm?

Example 3 A distributed system uses GPS for clock synchronization: –What is the reasonable granularity for the global time? –What are the limits for the observed values for a time interval of 200 ms?

References H. Kopetz, “Real time systems: Design principles for distributed systems” Springer Kopetz, H. & W. Ochsenreiter. (1987). Clock Synchronization in Distributed Real-Time Systems. IEEE Trans. Computers. Vol. 36(8). (pp ). Kopetz, H. (1992). Sparse Time versus Dense Time in Distributed Real-Time Systems. Proc. 14th Int. Conf. on Distributed Computing Systems. IEEE Press. (pp ).