Sir John Tenniel; Alice’s Adventures in Wonderland,Lewis Carroll 7-Aug-151 NTP Architecture, Protocol and Algorithms David L. Mills University of Delaware

7-Aug-152 Process decomposition o Peer process runs when a packet is received. o Poll process sends packets at intervals determined by the clock discipline process and remote server. o System process runs when a new peer process update is received. o Clock discipline process runs at intervals determined by the measured network phase jitter and clock oscillator (VFO) frequency wander. o Clock adjust process runs at intervals of one second. Remote Servers Server 1 Server 2 Peer/Poll 1 Server 3 Peer/Poll 2 Peer/Poll 3 Selection and Clustering Algorithms Combining Algorithm Loop Filter VFO Clock Discipline Process System Process Peer/Poll Processes Clock Adjust Process

7-Aug-153 NTP Protocol Header Format (32 bits) NTP protocol header and timestamp formats StratPollLIModeVN NTP v3 and v4 Root Delay Root Dispersion Reference Identifier Reference Timestamp (64) Originate Timestamp (64) Receive Timestamp (64) Transmit Timestamp (64) Message Hash (64 or 128) LIleap warning indicator VNversion number (4) Stratstratum (0-15) Pollpoll interval (log2) Precprecision (log2) Seconds (32)Fraction (32) NTP Timestamp Format (64 bits) Value is in seconds and fraction since 0 h 1 January 1900 Authenticator uses DES-CBC or MD5 cryptosum of NTP header plus extension fields (NTPv4) Key/Algorithm Identifier Cryptosum Authenticator (Optional) Extension Field 1 (optional) Extension Field 2… (optional) NTP v4 only Prec Extension Field (padded to 32-bit boundary) Field LengthField Type NTPv4 Extension Field Last field padded to 64-bit boundary authentication only

7-Aug-154 NTP packet header format StratPollLIModeVN Root Delay Root Dispersion Reference Identifier Reference Timestamp (64) Originate Timestamp (64) Receive Timestamp (64) Transmit Timestamp (64) Prec leapleap indicator (LI) versionversion number (VN) modeprotocol mode stratumstratum  poll interval (log 2 s)  clock reading precision (log 2 s)  root delay  root dispersion refidreference ID reftimereference timestamp T 1 originate timestamp T 2 receive timestamp T 3 transmit timestamp T 4 destination timestamp* MACMD5 message hash (optional) MAC (optional 160) Packet header VariablesDescription * Strictly speaking, T 4 is not a packet variable; it is the value of the system clock upon arrival.

7-Aug-155 NTP date and timestamp formats and important dates Era (32)Seconds (32)Fraction (32) Seconds (64)Fraction (32 or 64) NTP Date (signed, twos-complement, 128-bit integer) NTP Timestamp (unsigned 64-bit integer)Era Number

7-Aug-156 Process decomposition o Peer process runs when a packet is received. o Poll process sends packets at intervals determined by the clock discipline process and remote server. o System process runs when a new peer process update is received. o Clock discipline process runs at intervals determined by the measured network phase jitter and clock oscillator (VFO) frequency wander. o Clock adjust process runs at intervals of one second. Remote Servers Server 1 Server 2 Peer/Poll 1 Server 3 Peer/Poll 2 Peer/Poll 3 Selection and Clustering Algorithms Mitigation Algorithms Loop Filter VFO Clock Discipline Process System Process Peer/Poll Processes Clock Adjust Process

7-Aug-157 NTP on-wire protocol t 2 = clock t3t3 t4t4 t5t5 t1t1 t5t5 t6t6 t7t7 org rec T 3 = t 3 T4T4 t 7 <> T 3 ? T 8 = t 8 T 5 = t 5 T6T6 t 5 <>T 1 ? T 6 = t 6 org rec T 1 = t 1 T2T2 t 1 <>0? T 2 = t 2 t 4 = rec t 3 = org t 2 = rect 6 = rec t 5 = org 0 0t1t1 t2t2 t3t3 t 3 <> 0? T 4 = t 4 t4t4 t3t3 t2t2 t1t1 t5t5 t6t6 t8t8 t7t7 t 6 = clock T 1 = clockT 5 = clock t 4 = clockt 8 = clock T 3 = clockT 7 = clock t 1 = org orgoriginate timestamp recreceive timestamp xmttransmit timestamp Packet Variables Peer B State Variables Packet Variables Peer A State Variables State Variables NameDescription t 7 = xmit t 3 ==T 3 ? xmt t 3 = xmit 0 t 1 = xmit xmt t 5 = xmit t 5 == T 5 ?t 1 == T 1 ? t n originate timestamp t n+1 receive timestamp t n+2 transmit timestamp t n+3 destination timestamp Packet Header Variables NameDescription t 7 <> T 3 ? t 5 == T 5 ? org Duplicate Test xmt Bogus Test t1t1 t2t2 t3t3 t4t4 t5t5 t6t6 t7t7 t8t8

7-Aug-158 Transition matrix ModeACTIVEPASSIVECLIENTSERVERBCAST NO_PEERNEWPSFXMITNEWMCNEWBC ACTIVEPROC PASSIVEPROCERROR CLIENTPROC SERVER BCAST BCLIENTERRORPROC Packet Mode Association Mode The default (empty box) behavior is to discard the packet without comment.

7-Aug-159 Packet sanity tests

7-Aug-1510 T1T1 T3T3 T2T2 T4T4 o The most accurate offset   is measured at the lowest delay   (apex of the wedge scattergram). o The correct time  must lie within the wedge      o The   is estimated as the minimum of the last eight delay measurements and (  ,    becomes the peer update. o Each peer update can be used only once and must be more recent than the previous update. Server Client Clock filter algorithm x 00

7-Aug-1511 Clock filter performance o Left figure shows raw time offsets measured for a typical path over a 24-hour period (mean error 724  s, median error 192  s) o Right graph shows filtered time offsets over the same period (mean error 192  s, median error 112  s). o The mean error has been reduced by 11.5 dB; the median error by 18.3 dB. This is impressive performance.

7-Aug-1512 Correct DTSS Clock select principles o The correctness interval for any candidate is the set of points in the interval of length twice the synchronization distance centered at the computed offset. o The DTSS interval contains points from the largest number of correctness intervals, i.e., the intersection of correctness intervals. o A truechimer has a correctness interval that includes points in the intersection interval.. o Formal correctness assertions require at least half the candidates be be truechimers. If not, no candidate can be considered a truechimer. C A B D correctness interval = q - l £ q 0 £ q + l m = number of clocks f = number of presumed falsetickers A, B, C are truechimers D is falseticker

7-Aug-1513 system process: select algorithm no Set n = 0. Scan from lowest endpoint to highest. Add one to n for every lowpoint, subtract one for every highpoint. If n ≥ m  f, stop; set l = current lowpoint Set n = 0. Scan from highest endpoint to lowest. Add one to n for every highpoint, subtract one for every lowpoint. If n ≥ m  f, stop; set u = current highpoint. Add one to f. Is f < m / 2? Consider the lowpoint, midpoint and highpoint of these intervals. Sort these values in a list from lowest to highest. Set the number of falsetickers f = 0. Failure; a majority clique could not be found.. Success; the intersection interval is [l, u]. yes For each of m associations construct a correctness interval [  –,  + ] If l < u? no yes

7-Aug-1514 Cluster principles o Candidate 1 is further from the others, so its select jitter  S1 is highest. o (a)  max =  S1 and  min =  R2. Since  max >  min, the algorithm prunes candidate 1 to reduce select jitter and continues. o (b)  max =  S3 and  min =  R2. Since  max <  min, pruning additional candidates will not reduce select jitter. So, the algorithm ends with  R2,  R3 and  R4 as survivors.  S (1)  R (1)  R (2)  R (3 )  R (4)  R (2)  R (3)  R (4)  S (3) peer jitter select jitter ab

7-Aug-1515 system process: cluster algorithm For each candidate compute the selection jitter  S (RMS peer offset differences between this and all other candidates). Select  max as the candidate with maximum  S. Delete the outlyer candidate with  max ; reduce n by one. Done. The remaining cluster survivors are the pick of the litter. no yes Let ( ,  R,  ) represent a candidate with peer offset  jitter  and a weight factor  equal to stratum as the high order field and root distance as the low order field. Select  min as the candidate with minimum . Sort the candidates by increasing . Let n be the number of candidates and n min ≤ n the minimum number of survivors.  max <  min or n ≤ n min or  max is prefer peer?

7-Aug-1516 NTP dataflow analysis o Each server provides delay  and dispersion  relative to the root of the synchronization subtree. o As each NTP message arrives, the peer process updates peer offset  delay  dispersion  and jitter . o At system poll intervals, the clock selection and combining algorithms updates system offset , delay  dispersion  and jitter  o Dispersions  and  increase with time at a rate depending on specified frequency tolerance  Server 1  Server 2  Peer 1  Server 3  Peer 2  Peer 3  Selection and Combining Algorithms System 

7-Aug-1517 Error budget - notation o System variables  clock offset  root delay  root dispersion  s selection jitter  jitter  interval since last update mnumber of peers o Peer variables  clock offset  roundtrip delay  dispersion  r filter jitter nnumber of filter stages  interval since last update o Constants (peers A and B)  maximum reading error  maximum frequency error w dispersion normalize: 0.5 o Packet variables  B peer root delay  B peer root dispersion o Sample variables T 1, T 2, T 3, T 4 protocol timestamps xclock offset yroundtrip delay zdispersion  interval since last update

7-Aug-1518 Definitions o Precision: elapsed time to read the system clock from userland. o Resolution: significant bits of the timestamp fraction. o Maximum error: maximum error due all causes (see error budget). o Offset: estimated time offset relative to the server time. o Jitter: exponential average of first-order time differences o Frequency: estimated frequency offset relative to UTC. o Wander: exponential average of first-order frequency differences. o Dispersion: maximum error due oscillator frequency tolerance. o Root delay: accumulated roundtrip delay via primary server. o Root dispersion: accumulated total dispersion from primary server. o Estimated error: RMS accumulation from all causes (see error budget).

7-Aug-1519 Time values and computations   Packet VariablesPeer Variables Client System Variables  Server o Packet variables are computed directly from the packet header. o Peer variables are groomed by the clock filter. o System variables are groomed from the available peers.

7-Aug-1520 Clock discipline algorithm VdVd VcVc Phase/Freq Prediction Clock Filter Clock Adjust Phase Detector VFO VsVs r+r+ cc NTP Loop Filter o V d is a function of the phase difference between NTP and the VFO. o V s depends on the stage chosen on the clock filter shift register. o x and y are the phase update and frequency update, respectively, computed by the prediction functions. o Clock adjust process runs once per second to compute V c, which controls the frequency of the local clock oscillator. o VFO phase is compared to NTP phase to close the feedback loop. x y

7-Aug-1521 NTP r+r+ oo VsVs VdVd VcVc y NTP clock discipline with PPS steering o NTP daemon disciplines variable frequency oscillator (VFO) phase V c relative to accurate and reliable network sources. o Kernel disciplines VFO frequency y to pulse-per-second (PPS) signal. o Clock accuracy continues to be disciplined even if NTP daemon or sources fail. o In general, the accuracy is only slightly degraded relative to a local reference source. Loop Filter Clock Filter Frequency Estimator Phase Detector VFO PPS

7-Aug-1522 Measured PPS time error for Alpha 433 Standard error 51.3 ns

7-Aug-1523 Further information o NTP home page Current NTP Version 3 and 4 software and documentation FAQ and links to other sources and interesting places o David L. Mills home page Papers, reports and memoranda in PostScript and PDF formats Briefings in HTML, PostScript, PowerPoint and PDF formats Collaboration resources hardware, software and documentation Songs, photo galleries and after-dinner speech scripts o Udel FTP server: ftp://ftp.udel.edu/pub/ntp Current NTP Version software, documentation and support Collaboration resources and junkbox o Related projects Current research project descriptions and briefings