OpenNTPD with adjtime/adjfreq tuning

This article briefly describes the improvements in the timekeeping capabilities of OpenBSD when an ntpd is able to discipline the clock with adjtime(2) and adjfreq(2). The word "NTP" refers to various versions of the Network Time Protocol; "ntpd" refers to the NTP daemon shipping with OpenBSD. The graphs below show the same data; the second and third are zoomed in to show the performance of kernel support more clearly.

All machines synchronize to a Symmetricom (formerly Datum) time server. To prevent this reference server from being overloaded, we front-end it with a set of Sparcstation LXes ("their clocks may be crap, but they're at least consistently crappy") running as NTP servers for campus. For this experiment, my amd64 workstation was used as a baseline timer while polling the reference clock and our three other servers every ten seconds with 'rdate -pnav <server>'. The sparcstations are 2 hops and about 3.6ms away from the reference, while my workstation is 3 hops and about 2.7ms away. The decreased ping times can probably be attributed to the 1000Mbit ethernet interface in my workstation compared to the 10Mbit ethernet interface in the sparcstation.

The green trace shows ntpd without kernel support trying to follow our reference clock. While xntpd is reputed to have some clever filtering and steering algorithms, it does need kernel support to be most effective - its performance in this configuration would be almost identical. Several years ago the ntp kernel hooks were removed from OpenBSD. With nothing more than a write-only interface to adjtime, any NTP daemon will have difficulty properly steering the system's time. It would have to poll the time source and call adjtime to try keep the offset close to zero, which could cause fairly scary-looking changes in time. A cheap clock chip doesn't help matters any.

Recently, adjtime was extended to allow ntpd to query the kernel for the progress of any gentle time adjustments, and the adjfreq syscall was added to allow the kernel's timer frequency to compensate for variations in specific clock hardware. These changes are considerably simpler than the old NTP hooks and seem to perform quite well, as shown below. Before these changes, ntpd suffered the same problem, and would often lurch from being fast to slow to fast again. The magnitude of the sawtooth was greatly reduced however, by modifying ntpd to query the kernel for any adjtime currently still in progress, and using this to not request an excessively large correction.

The red trace is from my amd64 workstation, synchronized directly to the time server. Looking at the raw output from rdate, it seems to be about 50uS behind the timeserver. This is close to two orders of magnitude less than the round-trip time to the reference clock and can probably be explained by network delays and processing overhead. The consistently low offset is to be expected, as it is being synchronized directly to our reference clock.

The blue and purple traces are from the second and third of our timeservers running ntpd. The spikes in the time offset are difficult to explain. No errors or warnings were reported by ntpd during this period; only frequency adjustments of a few hundredths to a tenth of a part per million. Perhaps time2 has a slightly better clock chip than time3 as its spikes seem smaller than those from time3. No network errors and only three collisions were reported by netstat, suggesting that that routing or switching delays might be responsible. This is a reasonable guess, as time1 often calls adjtime very close to the time that the other servers also exhibit a spike.

Future tests will replace the sparcstations with either Soekris NET4801 or Nexcom EBS1563 systems. Both have 100Mbit ethernet interfaces on a PCI bus; our test Soekris and Nexcom machines have 233MHz and 667MHz x86 compatible processors, respectively. Poul Henning-Kamp claims sub-microsecond precision from a Soekris with a rubidium frequency standard; we do not expect to replicate this using the stock oscillators, but these more modern machines should do a better job than the 50MHz sparcstations.

Performance of an ntp daemon with and without kernel support:

Performance of kernel supported NTP daemons

Performance of kernel supported NTP daemons (zoomed in)

Updates to the original paper, which is a work in progress, will appear on Chris Kuethe's site at University of Alberta.