1. Dr. Hugh Melvin, Dept. of IT, NUI,G1 Clock Synchronisation for RTS

2. Dr. Hugh Melvin, Dept. of IT, NUI,G2 Importance of RTS Clocks RealTime implies need for accurate timekeeping Examples –Hard RTS Distributed Control Systems Power System / Fly-by-wire –Soft/Firm RTS TDM within GSM/POTS –POTS : SONET/SDH »Synchronous Opt. Network /Synch. Digital Hierarchy MM applications

3. Dr. Hugh Melvin, Dept. of IT, NUI,G3 Power System Control AS station –Token Bus Synchronisation via Master Clock Critical for chronological data logging / fault diagnosis –Timeslicing for token management –Synchronising 2v3 voter systems Need to deliver verdicts simultaneously Fault Diagnosis –Impossible without Chronological Data Generator Earth Fault / Overcurrent.. –Which came first.. msec level data required Power Line Fault Monitoring –Noise burst travels in both directions.. usec level synch

4. Dr. Hugh Melvin, Dept. of IT, NUI,G4 Token Bus : Master Clock U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B 101 N8 AS220E 102 N8 AS220E 103 N8 AS220E 104 N8 AS220E 105 N8 AS220E 106 N8 AS220E 107 N8 AS220E 108 N8 AS220E U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B 126 N-BK Bus 0 123 N-UHR M-Clock 121 N16 OS254 112 N8 AS220E 111 N8 AS220E 110 N8 AS220E 109 N8 AS220E U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B U/I A U/I B 160 NS5NAT PG750 133 N8 AS EHF 132 N8 AS EHF 131 N8 AS EHF 128 N8 AS231 125 N16 R30 141 NAT-24 Synogate U/I A U/I B 127 N-BK Bus 1 Master Clock

5. Dr. Hugh Melvin, Dept. of IT, NUI,G5 Soft-Firm RTS POTS operation based on TDM –PCM  E1  E2..E4  SDH/SONET –Precise synchronisation reqd throughout the network for correct system operation GSM : FDM + TDM –Each FDM channel divided out to 8 users via TDM Multimedia Applications –Delay / Jitter Measurement increasingly imp in packet (IP) networks –More advanced QoS through synchronised time Recall G.1010 –Basis of SLA  measurement important –Skew Issues between various system/media clocks

6. Dr. Hugh Melvin, Dept. of IT, NUI,G6

7. 7

8. 8 Audio-System Clock Skew

9. Dr. Hugh Melvin, Dept. of IT, NUI,G9 Computer Clocks Most commonly consist of quartz crystal and a counter Crystal oscillates at defined rate (Hz) which generates a consistent tick and increments a software counter Counter value translated to time standard –UTC (Univ. Coord. Time).. Based on GMT Primary Source: Atomic Clocks TAI (International Atomic Time) –But requires leap seconds every few years! –UTC = TAI + Leap_Seconds Crystal Quality described by Accuracy & Stability

10. Dr. Hugh Melvin, Dept. of IT, NUI,G10 Computer Clocks Accuracy relates to how close the crystal freq is to rated value –Determined by manufacturing process Get what you pay for! Stability relates to how frequency varies –Influenced by parameters such as: Temperature.. Eg. 2ppm /C Ageing –Eg. Cesium Beam: 3 x 10 -12 / year Noise

11. Dr. Hugh Melvin, Dept. of IT, NUI,G11 Computer Clocks Improved Quality Timekeeping ? –Option A: Stick with crystals Precision manufacturing  costly Temperature Compensated Crystal Osc.(TCXO) Oven Controlled Crystal Osc.(OCXO) –Option B: Buy an Atomic Clock –.. or GPS Receiver (based on atomic clock) Most popular approach to providing accurate/stable time –Option C : Cheaper Approach Software based approach to discipline cheap crystal clocks

12. Dr. Hugh Melvin, Dept. of IT, NUI,G12 Clock Terminology Confusion with terms in literature –Paxson/Mills terminology used here –Offset Difference between time reported by clock C, C(t) and true clock (UTC) at true time t. Also relative offset between clocks C 1 and C 2 –C 1 (t) - C 2 (t) –Skew Difference in frequency between clock C and a true clock (UTC), C ’ (t) Defined in ppm (usec per sec) +/-12 ppm approx = +/- 1 sec/day Also relative skew between clocks C 1 and C 2 –C 1 ’ (t) - C 2 ’ (t)

13. Dr. Hugh Melvin, Dept. of IT, NUI,G13

14. Dr. Hugh Melvin, Dept. of IT, NUI,G14 Clock Terminology Skew –A large skew rate  rapidly increasing offset  frequent resynchronisation –If specify max abs skew rate for clock C of –Clock should operate within cone of acceptability Drift –Rate of change of frequency C ’’ (t) Eg. Ageing influence or change in temperature –Not usually that significant except over long timescales –Note linear relationship in previous slide

15. Dr. Hugh Melvin, Dept. of IT, NUI,G15 Cone of Acceptability Real Time Clock Time Slope = 1 = True Clock Slope = 1 - Slope = 1 +

16. Dr. Hugh Melvin, Dept. of IT, NUI,G16 Clock Synchronisation Perfect clocks do not exist Eg. PC System Clock  NTP Server  GPS Receiver  GPS Atomic Clock  GPS Master Atomic Clock  ?? Examine two separate scenarios Localised Cluster of Clocks –Eg. Power System Control / Fly-by-wire Systems –Also widely distributed clocks over deterministic network »Propagation time known (can be compensated for) »Eg. POTS Widely distributed clocks over non-deterministic network –More difficult scenario –Eg. Internet Synchronisation

17. Dr. Hugh Melvin, Dept. of IT, NUI,G17 Clock Synchronisation Some General Principles –Fault Tolerance critical Identify and isolate faulty clocks Note: A faulty clock is one that does not operate within cone of acceptability –Cf Clock Quality: May be stable but inaccurate –Avoid setting clocks backward –Event processing nightmare –OS problems eg. Timers / timeslicing –Avoid large step changes Amortize the required change (+/-) over a series of short intervals (eg. over multiple ticks)

18. Dr. Hugh Melvin, Dept. of IT, NUI,G18 Localised Cluster of Clocks Hardware-based Phase Locked Loops (PLL) –Oscillator output is aligned to the input signal. –Input signal can come from a Master Clock Combination of outputs from all other clocks –Input signal used to drive its PLL –Can also compensate for Propagation Delay variations –Expensive but precise approach Similar approach used in widely distributed scenario –GPS / POTS / GSM all use variants of this approach

19. Dr. Hugh Melvin, Dept. of IT, NUI,G19 PLL VCO ComparatorInput Signal VCO = Voltage Controlled Oscillator Freq controlled by applied input voltage

20. Dr. Hugh Melvin, Dept. of IT, NUI,G20 Widely Distributed Clocks More difficult environment if underlying network non deterministic Expense of hardware based approach cannot be justified for many Soft-Firm RTS Cheap software based approach –Network Time Protocol (NTP) –RFC 1305 (www.ietf.org)www.ietf.org

21. Dr. Hugh Melvin, Dept. of IT, NUI,G21 Clock Synchronisation : NTP Network Time Protocol (NTP) synchronises clocks of hosts and routers in the Internet Increasingly deployed in the Internet –Increased need for time synchronisation –Facilitated via always-on Internet connection Provides nominal accuracies of low milliseconds on WANs, submilliseconds on LANs, and submicroseconds on workstations using a precision time source such as a cesium oscillator or GPS receiver Unix-based NTP daemon now ported to most OS

22. Dr. Hugh Melvin, Dept. of IT, NUI,G22 NTP The NTP architecture, protocol and algorithms have evolved over the last twenty years to the latest NTP Version 4 Internet standard protocol for time synchronisation and coordinated time distribution using UTC Fault tolerant protocol – automatically selects the best of several available time sources to synchronise with Highly scalable – nodes form a hierarchical structure with reference clock(s) at the top –Stratum 0: Time Reference Source GPS / GOES (GeoSat) / LORC (LoranC) / ATOM / DTS –Stratum 1: Primary Time Server

23. Dr. Hugh Melvin, Dept. of IT, NUI,G23

24. Dr. Hugh Melvin, Dept. of IT, NUI,G24 NTP Operation Complex Software comprising various algorithms Filtering Alg. Clustering and Intersection Alg. Combining Alg. Clock Discipline NTP Messages Peer 1 Peer 2 Filter 1 Peer 3 Filter 2 Filter 3 Intersection and Clustering Algorithms Combining Algorithm Loop Filter VFO P/F-Lock Loop

25. Dr. Hugh Melvin, Dept. of IT, NUI,G25 NTP Operation NTP Algorithms act upon a set of variables –Offset / Delay / Dispersion –Dispersion w = 0.75 These are relative to both peer and root Offset θ Θ DelayδΔ Dispersionε Ε

26. Dr. Hugh Melvin, Dept. of IT, NUI,G26 Client Server Mode UDP/IP packets for data transfer –Several packet exchanges between client/server –Client originate timestamp A within packet being sent. –Server receives such a packet: receive timestamp B transmit timestamp C –Client Processes A,B,C as well as final packet arrival D Determine offset and Round Trip Delay (RTD) Note: RTD != RTT

27. Dr. Hugh Melvin, Dept. of IT, NUI,G27 NTP Operation C 3.59.022 D 3.59.032 B 3.59.020 A 3.59.000 15 ms Symmetric Network : 15 ms each way (actual delay) RTD = (D - A) – (C – B) = 32 – 2 = 30 msec (RTT =?) Offset = ½[(B-A) - (D-C)] = (20 – 10)/2 = 5 ms

28. Dr. Hugh Melvin, Dept. of IT, NUI,G28 Filtering Algorithm Filtering algorithm looks at last 8 samples Chooses sample with min RTD Reduces offset errors by a factor of about ten Effective at removing spikes

29. Dr. Hugh Melvin, Dept. of IT, NUI,G29 Intersection Algorithm Clocks 1, 2,3 are truechimers 4 is a falseticker 3 2 1 4 Selects a subset of peers Based on intersection of confidence intervals Identifies truechimers & falsetickers eg. From 1,2,3,4 above X1 X2

30. Dr. Hugh Melvin, Dept. of IT, NUI,G30 Intersection Algorithm Estimated offset to each clock is mid pt But: Any point in each confidence interval may represent actual time as seen by that peer If clocks 1  4 are correct, there must exist a common intersection  Clock 4 most likely incorrect.. disregard Interval X1 = smallest intersection containing points from 1,2,3 But also include the max no of interval midpoints – Select X2 interval – Could select mid pt of X2.. or refine further

31. Dr. Hugh Melvin, Dept. of IT, NUI,G31 Clustering (Clock Selection) Sort surviving clocks by stratum and incr synch distance (RTD/2 + disp), S 1 S 2 S 3 Remove outliers that have significant dispersion relative to other survisors –Compute Select Dispersion of each clock Weighted sum of differences to other clocks –Compute Sample Dispersion of each clock Weighted sum of diff relative to past samples of same clocks –If Max SelDisp > Min SamDisp Remove this survivor and repeat Favours candidates at start of sorted list  Favours lowest stratum / delay

32. Dr. Hugh Melvin, Dept. of IT, NUI,G32 Clustering algorithm no yes For each survivor s i, compute the select dispersion (weighted sum of clock difference) between s i and all others. Let s max be the survivor with max select dispersion (relative to all other survivors) and s min the survivor with min sample dispersion (clock differences relative to past samples of the same survivor). s max  s min or n  n min  Delete the survivor s max ; reduce n by one The resulting survivors are processed by the combining algorithm to produce a weighted average used as the final offset adjustment Sort survivors of intersection algorithm by increasing synchronization distance(RTD/2 + dispersion). Let n = no of survivors and n min a lower limit (eg.3).

33. Dr. Hugh Melvin, Dept. of IT, NUI,G33 Combining Algorithm Combine result from survivors of selection algorithm Weighted offset determined based on –Offset of survivors Θ –Synchronisation distance Λ –Eg. 2 survivors (S1,S2) with parameters –Final Offset =

34. Dr. Hugh Melvin, Dept. of IT, NUI,G34 Combining Algorithm Example S1,S2 where S1 = (2 ms, 30) and S2 = (3 ms, 10) Final Adjustment = –(2(10) + 3(30)) / (30 + 10) = 110 / 40 = 2.75 msec Implemented via the Clock Discipline

35. Dr. Hugh Melvin, Dept. of IT, NUI,G35 Clock Discipline Recall –No time reversal! –Avoid step changes Hybrid phase/frequency-lock (PLL/FLL) feedback loop PLL/FLL Mode: Depends on polling interval

36. Dr. Hugh Melvin, Dept. of IT, NUI,G36 PLL and FLL weight factors Weight factors (not to scale) PLL predict (red) most important at shorter poll intervals to 2 4 s FLL predict (blue) most important at longer poll intervals to 2 17 s

37. Dr. Hugh Melvin, Dept. of IT, NUI,G37 Clock Models Unix Clock Model settimeofday( ), adjtime( ) Kernel variables tick, tickadj adjtime adjusts clock every tick –Can amortise reqd change gradually by making adjustment every tick eg. every 10 msec –Note: Newer Unix/Linux kernels 1000Hz  1msec 3 clock rates –Normal rate.. Add 10 msec every tick (100 Hz) –Normal Rate +/- tickadj –Eg. If tickadj = 5us  Normal Rate +/- 500 ppm

38. Dr. Hugh Melvin, Dept. of IT, NUI,G38 NTP Operation NTP adjusts every sec via adjtime –Eg. If clock skew is +100 ppm & tickadj=5us NTP will operate to keep clock effectively running at correct rate –Normal Rate - 500 ppm over 0.2 sec –Normal Rate for 0.8 sec –  Effective skew = 0 ppm –Results in sawtooth – pattern Newer Unix Kernels have advanced NTP features –ntp_adjtime( ), ntp_gettime() –Eliminates the sawtooth pattern

39. Dr. Hugh Melvin, Dept. of IT, NUI,G39 NTP Implementation Install NTP Set up ntp.conf file –List of servers that you wish to connect to –Redundancy & Path Diversity & Low RTD Start up NTP daemon ntpd File ntp.drift records clock skew Other utilities –ntpq, ntpdate –See www.ntp.org

40. Dr. Hugh Melvin, Dept. of IT, NUI,G40 Refid: DCF: 77.5 KHz Radio Signal PTB: German time signal

41. Dr. Hugh Melvin, Dept. of IT, NUI,G41

42. Dr. Hugh Melvin, Dept. of IT, NUI,G42 Time difference

43. Dr. Hugh Melvin, Dept. of IT, NUI,G43 Server Details when: no of sec since last response poll : interval between queries reach : Reachability in octal –11111111 = 377 8 = max –11101110 = 356 8  last + 5 th probe lost Symbol to LHS of server –* : Synch Source – survivor with smallest dispersion –+ :other candidates included in final combination alg – - : Discarded by clustering alg –x : Falseticker acc to intersection alg

44. Dr. Hugh Melvin, Dept. of IT, NUI,G44

45. Dr. Hugh Melvin, Dept. of IT, NUI,G45 NTP Robustness Issues Redundancy Path Diversity Symmetric Networks Proximity to Primary Reference Sources –See results OS & Network Load –Platform Dependencies

46. Dr. Hugh Melvin, Dept. of IT, NUI,G46 NTP Operation : Asymmetry C 3.59.017 D 3.59.032 B 3.59.015 A 3.59.000 10 ms20 ms Offset still 5 ms but Asymmetric Network RTD = (D - A) – (C – B) = 32 – 2 = 30 msec Offset = ½[(B-A) - (D-C)] = (15 – 15)/2 = 0 ms.. Error

47. Dr. Hugh Melvin, Dept. of IT, NUI,G47 NTP Operation : Asymmetry C 3.59.017 D 3.59.032 B 3.59.015 A 3.59.000 15 ms NTP’s Symmetric view of Asymmetric Network RTD = (D - A) – (C – B) = 32 – 2 = 30 msec Offset = ½[(B-A) - (D-C)] = (15 – 15)/2 = 0 ms ! Exercise: What is the maximum error in this calculation?

48. Dr. Hugh Melvin, Dept. of IT, NUI,G48

49. Dr. Hugh Melvin, Dept. of IT, NUI,G49

50. Dr. Hugh Melvin, Dept. of IT, NUI,G50 Server Offsets: Problem?