Clock Synching is Worth Your Time

Edit: Hopefully by now you have everything in sync.

Originally posted June 9, 2009 on AIXchange

How many physical machines or virtual partitions do you have in your environment? One? Ten? Hundreds? How often do you verify that the time is set correctly on each of your servers? I’ve seen customers set the time on a machine using the date command, and then forget about it.

Why should you care whether your machines keep accurate time? In many environments, one server hosts a database while another hosts the application. If either doesn’t maintain accurate time, problems can result. For instance, the database might think it’s getting requests from the future, or from the past. Have you ever tried to troubleshoot two machines whose timestamps aren’t in sync? Good luck trying to figure out what happened at a given point in time. Maybe you’ve tried running an HACMP failover on machines with clocks that are off, or maybe you’ve tried live application mobility on machines keeping different times. Each of these scenarios can introduce problems into your environment due to timing issues, and in same cases will refuse to work at all. Therefore, setting up network time protocol (NTP) is something we must do to keep our clocks in sync.

From Wikipedia: “Clock synchronization is a problem from computer science and engineering which deals with the idea that internal clocks of several computers may differ. Even when initially set accurately, real clocks will differ after some amount of time due to clock drift. Clock drift refers to several related phenomena where a clock does not run at the exact right speed compared to another clock.”

Some people will run the “ntpdate –u ” command from cron at some interval to keep clocks in sync. However, I would argue that we should set up the xntpd daemon instead. If you’re unsure how to do this, keep reading.

First, determine which time server you will use. Many times a time server is already set up in your environment, so you can use that one. You’ll need to edit the /etc/ntp.conf file and make changes. On a default AIX machine, at the end of the file you should find:

#   Broadcast client, no authentication.
#
broadcastclient
driftfile /etc/ntp.drift
tracefile /etc/ntp.trace

Comment out the broadcastclient stanza, and replace it with:

server
where is the name of the server you plan to use.

The NTP Web site, ntp.org, offers this as a starting point:

server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org

However, if you look at that Web site, you’ll also find that you could use a sub-zone of pools based on continent or country.

According to ntp.org: “As pool.ntp.org will assign you timeservers from all over the world, time quality will not be ideal. You get a bit better result if you use the continental zones (For example europe, north-america, oceania or asia.pool.ntp.org), and even better time if you use the country zone (like ch.pool.ntp.org in Switzerland) – for all these zones, you can again use the 0, 1 or 2 prefixes, like 0.ch.pool.ntp.org. Note, however, that the country zone might not exist for your country, or might contain only one or two timeservers. If you know timeservers that are really close to you (measured by network distance, with traceroute or ping), time probably will be even better.”

After set up your /etc/ntp.conf file the way you want it, save it and run startsrc –s xntpd.

Once your daemon is running, check your output with ntpq –p.

After some period of time (ntp.org says it could take up to 30 minutes), you should see:

ntpq -p
     remote           refid      st t when poll reach   delay   offset    disp
==============================================================================
+ip-72-167-54-20 193.5.216.14     2 u   41   64  377    63.26   -9.092    5.14
*rrcs-64-183-55- 130.149.17.8     1 u   41   64  377    84.99   -1.517    7.90
mirror          131.188.3.222    3 u   41   64  377     8.44   -9.533    5.33

Again from ntp.org: “The IP addresses will be different, because you’ve been assigned random timeservers. The essential thing is that one of the lines starts with an asterisk (*), this means your computer gets the time from the Internet.”

Once you set up everything, be sure to go into /etc/rc.tcpip and uncomment the line:

start /usr/sbin/xntpd “$src_running”

This enables xntpd to restart after you reboot your machine.

Keeping all of your clocks accurate and in sync is a best practice. It should be one more step on your new server build checklists, if it isn’t already.