I'm new to ntp, but have been asked to debug an issue with our NTP
configuration.  First, here is our setup.  We have two processors which
each have a third party application reading time off a serial line from
two different sources and then disciplining the internal clock on each
processor with the "adjtime" system call.  xntpd runs on each of these
processors with only internal time as a source.  One processor is
fudged at stratum 0 and the other at stratum 2.  Currently, xntp is
configured with "enable pll".  My first change will be to change
"enable pll" to "diable pll" as I believe that we shouldn't have two
different applications(our third party app and xntp) discipling the
internal clock at the same time.

These two processors act as the NTP servers to all other processors on
our network.  I've read here that having two servers is a bad situation
and that it is better to have four, but that just about
anything(including just 1) is better than having two.  I believe we our
finding that out first hand, so I work to resolve that as well.

When I checked the ntp logs for some client processors on our network,
I see that occasionaly there are large(upto 1 second) differences
between our two servers.  On top of this, the clients can not decide
which one to sync with and we get stuck in a loop as follows.

restart min polling
step time/sync to server1 at stratum 0
restart min polling due to step
step time/sync to server2 at stratum 2

This exact sequence repeats over and over on appx 5 minute intervals
due to the min polling time.  I realize that I have many problems here,
but I would like to know why ntp toggles between syncing/stepping to
server1 and server2 after each polling period.  I would have thought
that the resultant action would be the same after each polling period.
But instead it is consistent that the client steps time to the opposite
server that it stepped to last time, like so.

25 Mar 16:36:58 xntpd[7752]: time reset (step) 0.421856 s
25 Mar 16:41:55 xntpd[7752]: synchronized to 192.6.1.21, stratum=1
25 Mar 16:41:54 xntpd[7752]: time reset (step) -0.421930 s
25 Mar 16:46:33 xntpd[7752]: synchronized to 192.6.1.22, stratum=3
25 Mar 16:46:34 xntpd[7752]: time reset (step) 0.419369 s
25 Mar 16:51:31 xntpd[7752]: synchronized to 192.6.1.21, stratum=1
25 Mar 16:51:30 xntpd[7752]: time reset (step) -0.419446 s
25 Mar 16:56:09 xntpd[7752]: synchronized to 192.6.1.22, stratum=3

vwvr6vw wrote:

> I'm new to ntp, but have been asked to debug an issue with our NTP
> configuration.  First, here is our setup.  We have two processors which
> each have a third party application reading time off a serial line from
> two different sources and then disciplining the internal clock on each
> processor with the "adjtime" system call.  xntpd runs on each of these
> processors with only internal time as a source.  One processor is
> fudged at stratum 0 and the other at stratum 2.  Currently, xntp is
> configured with "enable pll".  My first change will be to change
> "enable pll" to "diable pll" as I believe that we shouldn't have two
> different applications(our third party app and xntp) discipling the
> internal clock at the same time.
> 

You are absolutely right about that.

> These two processors act as the NTP servers to all other processors on
> our network.  I've read here that having two servers is a bad situation
> and that it is better to have four, but that just about
> anything(including just 1) is better than having two.  I believe we our
> finding that out first hand, so I work to resolve that as well.
> 
> When I checked the ntp logs for some client processors on our network,
> I see that occasionaly there are large(upto 1 second) differences
> between our two servers.  On top of this, the clients can not decide
> which one to sync with and we get stuck in a loop as follows.
> 
> restart min polling
> step time/sync to server1 at stratum 0
> restart min polling due to step
> step time/sync to server2 at stratum 2
> 

There, in a nutshell, you see why two servers is the worst possible 
configuration!

> This exact sequence repeats over and over on appx 5 minute intervals
> due to the min polling time.  I realize that I have many problems here,
> but I would like to know why ntp toggles between syncing/stepping to
> server1 and server2 after each polling period.  I would have thought
> that the resultant action would be the same after each polling period.
> But instead it is consistent that the client steps time to the opposite
> server that it stepped to last time, like so.
> 
> 25 Mar 16:36:58 xntpd[7752]: time reset (step) 0.421856 s
> 25 Mar 16:41:55 xntpd[7752]: synchronized to 192.6.1.21, stratum=1
> 25 Mar 16:41:54 xntpd[7752]: time reset (step) -0.421930 s
> 25 Mar 16:46:33 xntpd[7752]: synchronized to 192.6.1.22, stratum=3
> 25 Mar 16:46:34 xntpd[7752]: time reset (step) 0.419369 s
> 25 Mar 16:51:31 xntpd[7752]: synchronized to 192.6.1.21, stratum=1
> 25 Mar 16:51:30 xntpd[7752]: time reset (step) -0.419446 s
> 25 Mar 16:56:09 xntpd[7752]: synchronized to 192.6.1.22, stratum=3
> 

I'd say that this is probably an artifact of the two server 
configuration.  There is no way do decide between the two so it's just 
"flipping a coin".

If I understand what you've said, you have only one source of time, the 
"serial line".  There is evidently some sort of problem with it or the 
"third party" software that is synchronizing your two servers to to it; 
else why the disagreement between servers?

You can set up an NTP server with a hardware reference clock (a GPS 
timing receiver) for a minimal cash outlay.  Like so:
1. A Sun Ultra 10 (used, <$100 on e-Bay)  I'd recommend a minimum of 
256MB of RAM.  I have a 440MHz processor but it's not required; any 
supported processor should do the job.
2. An ATA/EIDE disk drive =>10GB.  You may well have something suitable 
lying around.  $80 will buy you a new 80GB drive.
3. A Motorola Oncore M12+T or M12MT GPS reciever with "evaluation board" 
for around $200 from Synergy Systems.
4. Solaris 10 Media kit ($40 from Sun).
5. NTP 4.2.0 or 4.2.1.  Free.

Plug it together, install software, configure, start, and enjoy.  I have 
such a configuration running in my home.  Two other Suns (Solaris 8 and 
Solaris 9), two PCs with Windows XP and W32TIME, one PC with W2K and 
W32TIME, and one DEC Alpha running OpenVMS V6.2 synchronize to it.  The 
Suns hold synch to within a few microseconds. The Alpha to within a few 
milliseconds.  It's a little hard to tell what the PCs are doing but 
they all display the correct time to the second (they may do better 
internally but I don't know how I could see that).

Richard B. Gilbert wrote:
[]
> It's a little hard to tell
> what the PCs are doing but they all display the correct time to the
> second (they may do better internally but I don't know how I could
> see that).

If the PCs were running NTP you could use MRTG to monitor them:

  http://www.david-taylor.myby.co.uk/mrtg/daily_ntp.html

If they offer any sort of time service, you could try:

  http://www.david-taylor.myby.co.uk/software/net.htm#NTPmonitor

David 

vwvr6vw wrote:
> I'm new to ntp, but have been asked to debug an issue with our NTP
> configuration.  First, here is our setup.  We have two processors which
> each have a third party application reading time off a serial line from
> two different sources and then disciplining the internal clock on each
> processor with the "adjtime" system call.  xntpd runs on each of these
> processors with only internal time as a source.  One processor is
> fudged at stratum 0 and the other at stratum 2.  Currently, xntp is
> configured with "enable pll".  My first change will be to change
> "enable pll" to "diable pll" as I believe that we shouldn't have two
> different applications(our third party app and xntp) discipling the
> internal clock at the same time.
> 
> These two processors act as the NTP servers to all other processors on
> our network.  I've read here that having two servers is a bad situation
> and that it is better to have four, but that just about
> anything(including just 1) is better than having two.  I believe we our
> finding that out first hand, so I work to resolve that as well.
> 
> When I checked the ntp logs for some client processors on our network,
> I see that occasionaly there are large(upto 1 second) differences
> between our two servers.  On top of this, the clients can not decide
> which one to sync with and we get stuck in a loop as follows.
> 
> restart min polling
> step time/sync to server1 at stratum 0
> restart min polling due to step
> step time/sync to server2 at stratum 2
> 
> This exact sequence repeats over and over on appx 5 minute intervals
> due to the min polling time.  I realize that I have many problems here,
> but I would like to know why ntp toggles between syncing/stepping to
> server1 and server2 after each polling period.  I would have thought
> that the resultant action would be the same after each polling period.
> But instead it is consistent that the client steps time to the opposite
> server that it stepped to last time, like so.
> 
> 25 Mar 16:36:58 xntpd[7752]: time reset (step) 0.421856 s
> 25 Mar 16:41:55 xntpd[7752]: synchronized to 192.6.1.21, stratum=1
> 25 Mar 16:41:54 xntpd[7752]: time reset (step) -0.421930 s
> 25 Mar 16:46:33 xntpd[7752]: synchronized to 192.6.1.22, stratum=3
> 25 Mar 16:46:34 xntpd[7752]: time reset (step) 0.419369 s
> 25 Mar 16:51:31 xntpd[7752]: synchronized to 192.6.1.21, stratum=1
> 25 Mar 16:51:30 xntpd[7752]: time reset (step) -0.419446 s
> 25 Mar 16:56:09 xntpd[7752]: synchronized to 192.6.1.22, stratum=3
> 

Your description sounds like it broke the cardinal rule: you must not
have more than one application disciplining the clock. It sounds like
you do have more than one, but I can't be sure. The application should
have been implemented as a refclock and let ntpd figure out the rest.
fudging to stratum 0 is illegal, that's reserved for the refclock
itself. Everything else is derived from there. At best your server would
be stratum 1 if your time was being set up as a refclock. You'd need to
understand the what the third-party application is doing to figure out
how well it's disciplining the clock. It's probably not doing a good job
and clock-hopping like this is not unlikely.

Danny
_______________________________________________
questions mailing list
questions@lists.ntp.isc.org
https://lists.ntp.isc.org/mailman/listinfo/questions

Richard B. Gilbert wrote:
> You can set up an NTP server with a hardware reference clock (a GPS 
> timing receiver) for a minimal cash outlay.  Like so:
> 1. A Sun Ultra 10 (used, <$100 on e-Bay)  I'd recommend a minimum of 
> 256MB of RAM.  I have a 440MHz processor but it's not required; any 
> supported processor should do the job.
> 2. An ATA/EIDE disk drive =>10GB.  You may well have something suitable 
> lying around.  $80 will buy you a new 80GB drive.
> 3. A Motorola Oncore M12+T or M12MT GPS reciever with "evaluation board" 
> for around $200 from Synergy Systems.
> 4. Solaris 10 Media kit ($40 from Sun).
> 5. NTP 4.2.0 or 4.2.1.  Free.

Using a discarded PC (with FreeBSD) + maximum $100 for a Garmin GPS18LVC 
+ a 9-pin serial connector and a USB cable (used to get +5V power) works 
equally well for this purpose, and it is even cheaper! :-)

Use whatever you feel comfortable with!

Terje

-- 
- <Terje.Mathisen@hda.hydro.com>
"almost all programming can be viewed as an exercise in caching"

Thanks for the replies.  I've recommended to my management that we
abandon the third party software and instead purchase four independant
NTP servers.  I think this is the most simplistic solution given our
current problems, which are difficult to debug.  It is also a better
solution.  I'm not sure they will go for it given the state of our
project, but I will push for it.

vwvr6vw wrote:

> Thanks for the replies.  I've recommended to my management that we
> abandon the third party software and instead purchase four independant
> NTP servers.  I think this is the most simplistic solution given our
> current problems, which are difficult to debug.  It is also a better
> solution.  I'm not sure they will go for it given the state of our
> project, but I will push for it.
> 

Depending on what you mean by "four independent NTP servers" you could 
be talking about a lot of money!  One server is sufficient for 
non-critical applications.  Everybody will stay in synch with it without 
problems.  The weaknesses are:
1. If the server has the wrong time, everybody has the wrong time. In 
some shops that's a problem, in others, not!
2. If the server fails, you are without a server.

If you must have the correct time, you need to configure each client 
with a minimum of four servers.  Those four servers can be existing 
machines.  Each one needs a source of time.  That source can be a 
hardware reference clock; e.g. a GPS timing receiver or a minimum of 
four internet servers.  If you use internet servers, each of your 
internal servers should have at least one unique external server or 
seven in all.

If you need really tight synchronization among the clients; e.g. 
everyone within ten microseconds, you pretty much have to have a GPS 
reference clock.  Few shops need synchronization that tight but you need 
to specify just what your needs are for: accuracy, tightness of 
synchronization, and reliability before you can design a good solution.

Thanks.  In this application, a few thousand dollars is not going to be
an issue.  Having the correct time is critical and that is why I'm
hoping that managment will go for it.  I believe the two sources we
have now are from GPS.

As to why the two references that we have seem to be different at
times, I believe this is most likely due to our third party software or
possibly the fact that we have "enable pll" in our NTP configuration
file.  From what I've been told by a co-worker, NTP can not read time
off of a serial line with AIX version 3.4 and that we would need to
upgrade to version 4 or 5.  This is why we have a third party
application reading time off of the serial line and then disciplining
the local clock on the NTP servers instead of having NTP do all of this
for us.  If anyone knows this to be false, please let me know.

> From what I've been told by a co-worker, NTP can not read time
> off of a serial line with AIX version 3.4 and that we would need to
> upgrade to version 4 or 5.

Wow, I remember AIX3...

H

vwvr6vw wrote:

> Thanks.  In this application, a few thousand dollars is not going to be
> an issue.  Having the correct time is critical and that is why I'm
> hoping that managment will go for it.  I believe the two sources we
> have now are from GPS.
<snip>

Can you locate an antenna where it will have an unobstructed view of the 
entire sky?  If so, you can buy a GPS receiver (GARMIN GPL18LVC) for 
less than $100 and connect it to a serial port on a computer running 
ntpd with support for refclocks and presto!  It's a bit on the do it 
yourself side of things but it will get the job done.  You'll need a 
soldering iron, a DB9 connector, and a five volt power supply.

Moving upscale a bit, you can get a Motorola Oncore M12M timing receiver 
  (actually somebody else makes them now) from Synergy Systems.  You get 
the one with the evaluation board.  It's still pretty crude; a bare 
circuit board, but a power supply, antenna, and RS232 cable come with 
it. That will cost you $200.  Plug it in, connect it to a serial port, 
install ntpd, configure it, and go.

Moving pretty far upscale, you can get a box from Symmetricom with a 
built in GPS receiver and NTP server.  Plug it in to the AC outlet and 
your Ethernet.  I think these go for something like $2500 and up. 
That's about $2300 above my budget for toys so I don't have one and 
won't unless someone gives me one for Christmas.  I think that box may 
also come with an OCXO disciplined by the GPS to give some "holdover" 
capability if you lose signal from the satellites.

You can also get a box that gets the time from a CDMA Cell Phone base 
station (derived from GPS).  You plug that box into AC, your network and 
a whip antenna.  It, too, has a built-in NTP server.

You might want to get a couple of each.

Harlan Stenn wrote:

>>From what I've been told by a co-worker, NTP can not read time
>>off of a serial line with AIX version 3.4 and that we would need to
>>upgrade to version 4 or 5.
> 
> 
> Wow, I remember AIX3...
> 
> H

>As to why the two references that we have seem to be different at
>times, I believe this is most likely due to our third party software or
>possibly the fact that we have "enable pll" in our NTP configuration
>file.  From what I've been told by a co-worker, NTP can not read time
>off of a serial line with AIX version 3.4 and that we would need to
>upgrade to version 4 or 5.  This is why we have a third party
>application reading time off of the serial line and then disciplining
>the local clock on the NTP servers instead of having NTP do all of this
>for us.  If anyone knows this to be false, please let me know.

I don't know anything about AIX.

Most GPS clocks send a message over a serial port and also
flap a signal once per second: PPS, Pulse Per Second.
One edge of that pulse is on the second boundary.

The serial port isn't very good for timekeeping because of
the jitter.  It will probably get you within a few milliseconds.
Most kernels include serial port support that works with ntpd.

Using the PPS signal requires special support from the kernel.
The usual way is to connect it to one of the modem control
signals on the serial port.  PPS support is included with the normal
kernel sources on FreeBSD.  You probably have to turn it on in
your kernel building config file.  Linux requires a patch which
is readily available.

If accuracy of a few ms is good enough, you may not need the PPS
support in your kernel.

-- 
The suespammers.org mail server is located in California.  So are all my
other mailboxes.  Please do not send unsolicited bulk e-mail or unsolicited
commercial e-mail to my suespammers.org address or any of my other addresses.
These are my opinions, not necessarily my employer's.  I hate spam.

Richard B. Gilbert wrote:
[]
> Can you locate an antenna where it will have an unobstructed view of
> the entire sky?  If so, you can buy a GPS receiver (GARMIN GPL18LVC)
> for less than $100 and connect it to a serial port on a computer
> running ntpd with support for refclocks and presto!  It's a bit on
> the do it yourself side of things but it will get the job done. You'll 
> need a soldering iron, a DB9 connector, and a five volt power
> supply.

Here's my simple attempt at this:

  http://www.david-taylor.myby.co.uk/ntp/FreeBSD-GPS-PPS.htm

and it can only see half the sky.

David 

Do you happen to know the model number of this $2500 option?

vwvr6vw wrote:
> Do you happen to know the model number of this $2500 option?
> 

http://www.symmttm.com/products_network_timing.asp

I've seen their stuff on e-Bay for around $2500.  For all I know, it 
could cost ten times that "new in box".

Visit their web site, see what they have and ask for quotes.