22-Sept-2005 Google Summer of Code Projects: Lightweight Precision Timestamps Jeff Boote

2005-Sept-22 Summer of Code 2005 Program funded by Google ($2M) Supported students to work on open- source projects ($4500 per student) 400 students were selected Each student was required to have a mentoring organization 40 mentoring organizations Internet2 mentored 10 students

2005-Sept-22 Internet2 mentored projects Mostly transport-related things ( Two projects to characterize internal delays Three projects to work on a transport tool Two projects to work on thrulay One project to build a Gaim plug-in 8 success, 2 failures

2005-Sept-22 Lightweight Precision Timestamps Implement a lightweight timestamp API Basically gettimeofday() without a system call 2 projects successful

2005-Sept-22 Uses For Internet2 mostly network delay measurements Owamp Thrulay Bulk-transport protocol (delay based congestion control) Wide applicability (code profiling, etc…)

2005-Sept-22 OWAMP (the problem) One-way network latency measurements Control protocol Test protocol Sample implementation Find problems in the network Congestion usually happens in one direction first… Routing (asymmetric, or just changes) SNMP polling intervals mask high queue levels that active probes can show

2005-Sept-22 Sample Implementation Applications owampd daemon owping client Built upon protocol abstraction library Supports one-off applications Allows authentication/policy hooks to be incorporated

2005-Sept-22 Functionality (owping client) owping client requests OWD tests from an OWAMP server Client can be sender or reciever Communication can be “open”, “authenticated”, or “encrypted” Supports the setup of many tests concurrently Supports the buffering of results on the server for later retrieval

2005-Sept-22 Functionality (owampd) owampd Accepts requests for OWD tests Responds with accepted/denied Tests are formally started with a StartSessions message from the client. Runs tests Sessions with packets received at the server are buffered for later retrieval

2005-Sept-22 Architecture

2005-Sept-22 Systemic Application Errors Our tests indicate a systemic error of 73 usec * Experiments with two systems connected via cross-over cable Two concurrent sessions (send,recv) 10 packets/second Intel SCB2 motherboard 2x512 MB ECC registered RAM Intel PRO/100+ integrated NIC FreeBSD 4.6 * 95% confidence level (RFC 2679) * Error is specific to this OS/hardware/intensity level * Tests with older version of owamp, should be a little better now.

2005-Sept-22 Calibration Distribution

2005-Sept-22 Characterization of Error Owamp (Min delay was 56 usec) Timestamp fetching: usec Serialization: usec Unaccounted: usec Ping (RTT 57 usec) Timestamp Fetching: 5.07 usec Serialization: usec Unaccounted: usec

2005-Sept-22 Precision Related Context Switches

2005-Sept-22 Problems with “system” time for latency measurement Poor clock resolution (typically 1-10 msec) Large amount of time to actually fetch timestamp Some systems better than others Large jitter System-call - context switch threat No error estimates Drift estimate only happens at boot time for some systems Cold hardware

2005-Sept-22 Project Definition Use clock register (TSC) along with system-wide state to incorporate drift adjustment and conversion to UTC without system calls.

2005-Sept-22 Implementation Daemon Treat “system” clock as an upstream NTP server Synchronize TSC values to “system” clock –Saves transformation information in shared memory so all local process have access to identical transformation –Does not use locks - implemented with a rotating buffer with an index to the “current” set of coefficients (index updated with “atomic” operation)

2005-Sept-22 Implementation Client Read TSC values directly Read transformation information from shared memory

2005-Sept-22 Validation of Implementation See websites

2005-Sept-22 Features No system calls Clock resolution related to processor speed Potential to provide error estimates Quick convergence Spike supressor to deal with system time jitter PLL/FLL state machine based on NTP algorithms

2005-Sept-22 Future Enhancements to support OWAMP Add interaction with NTP to determine error of system clock Expose error of TSC synchronization as well as NTP

2005-Sept-22 Project Pages Fasttime (Alex Holkner) TSC-xluo (Xun Luo) (Both sites site references for related work)