f



NTP resolution

hello,

I am new to ntp. My environment is a cluster of Intel/linux (redhat 7.3 +
ntp-4.1.1) interconnected by fast ethernet. I have set up ntp for a long
time. However, those computers are still not highly synchronized. The time
difference can be as large as several seconds. This is the rough picture of
how the current situation is like: (in seconds)

GD018B: current time: 1621.005379
GD020B: current time: 1620.864570
GD024B: current time: 1635.441457
GD021B: current time: 1627.938122
GD022B: current time: 1629.067711
GD035B: current time: 1621.702259
GD019B: current time: 1620.108972
GD023B: current time: 1642.300444

I would like to know what is supposed to be the time resolution which can be
achieved in my environment? And where may I do wrong? Thank you very much!

-- 

Regards,

Weijian


0
Fang
7/10/2003 8:49:59 AM
comp.protocols.time.ntp 4895 articles. 2 followers. Post Follow

15 Replies
806 Views

Similar Articles

[PageSpeed] 20

Please post the output of "ntpq -p host1 host2 ... hostN".

H
0
Harlan
7/10/2003 9:45:08 AM
> I am new to ntp. My environment is a cluster of Intel/linux (redhat 7.3
+
> ntp-4.1.1) interconnected by fast ethernet. I have set up ntp for a long
> time. However, those computers are still not highly synchronized. The
time
> difference can be as large as several seconds.

Something sound grossly wrong - even in a Windows environment I get
syncing to within a few tens of milliseconds, see the graphs at:

http://www.david-taylor.myby.co.uk/mrtg/daily_ntp.html

You may have to wait an hour for full sync to happen, and you might want
to try the command:

  ntpq -p <node name>

to see what each node thinks it is syncing to, and what state it is in.
Look for an asterisk (*) aginst one of the servers listed to find which
node is the current reference.

Cheers,
David


0
David
7/10/2003 9:53:43 AM
"Fang Weijian" <wjfang@csis.hku.hk> wrote in message
news:bej96v$qp8$1@www.csis.hku.hk...

> I am new to ntp. My environment is a cluster of Intel/linux (redhat 7.3 +
> ntp-4.1.1) interconnected by fast ethernet. I have set up ntp for a long
> time. However, those computers are still not highly synchronized. The time
> difference can be as large as several seconds. This is the rough picture
of
> how the current situation is like: (in seconds)

    What is this cluster trying to synchronize *to*?

    DS


0
David
7/10/2003 10:50:18 AM
Thank you all guys very much.

I identified the problem. The NTP server is also one of my computers, which
has two domain names, one is for external world, the other is for internal
world. I used the external name in my ntp.conf. Therefore, they are not
synchronized at all.

But I have further problems arising. This is some output from "nap -up":

host        remote           refid      st t when poll reach   delay
offset  jitter
============================================================================
=========
GD018B *dx3             147.8.177.14     5 u  507  512  377
 0.189   -0.044   0.039
host        remote           refid      st t when poll reach   delay
offset  jitter
============================================================================
=========
GD019B *dx3             147.8.177.14     5 u  234  256  377    0.170
0.191   0.008
host        remote           refid      st t when poll reach   delay
offset  jitter
============================================================================
=========
GD020B *dx3             147.8.177.14     5 u  450  512  377    0.214
1.347   0.814
host        remote           refid      st t when poll reach   delay
offset  jitter
.......
============================================================================
=========
GD023B *dx3             147.8.177.14     5 u  226  512  377
 0.174   -0.882   0.233
host        remote           refid      st t when poll reach   delay
offset  jitter
.......
============================================================================
=========
GD075B *dx3             147.8.177.14     5 u  246  512  377
 0.172   -1.019   0.239
host        remote           refid      st t when poll reach   delay
offset  jitter
.......
GD175B *dx3             147.8.177.14     5 u   47   64  377
 0.166   -0.009   0.032
host        remote           refid      st t when poll reach   delay
offset  jitter

Is these result correct? By "correct", I mean this is the effect that NTP is
supposed to achieve in a cluster environment (Intel/Linux PCs connected by
fast Ethernet, and synchronized to an internal NTP server). We still can see
the offsets are quite widely distributed, from 9 microseconds to more than 1
milliseconds. Are there any techniques to further improve the synchronizing
precision?

Thank you very much.

-- 

Regards,

Weijian

"David J Taylor" <david-taylor@blueyonder.co.uk> wrote in message
news:HIaPa.7150$0G1.55380108@news-text.cableinet.net...
> > I am new to ntp. My environment is a cluster of Intel/linux (redhat 7.3
> +
> > ntp-4.1.1) interconnected by fast ethernet. I have set up ntp for a long
> > time. However, those computers are still not highly synchronized. The
> time
> > difference can be as large as several seconds.
>
> Something sound grossly wrong - even in a Windows environment I get
> syncing to within a few tens of milliseconds, see the graphs at:
>
> http://www.david-taylor.myby.co.uk/mrtg/daily_ntp.html
>
> You may have to wait an hour for full sync to happen, and you might want
> to try the command:
>
>   ntpq -p <node name>
>
> to see what each node thinks it is syncing to, and what state it is in.
> Look for an asterisk (*) aginst one of the servers listed to find which
> node is the current reference.
>
> Cheers,
> David
>
>


0
Fang
7/11/2003 4:29:57 AM
> > Is these result correct? By "correct", I mean this is the effect that
NTP is
> > supposed to achieve in a cluster environment (Intel/Linux PCs connected
by
> > fast Ethernet, and synchronized to an internal NTP server). We still can
see
> > the offsets are quite widely distributed, from 9 microseconds to more
than 1
> > milliseconds. Are there any techniques to further improve the
synchronizing
> > precision?
>
> That looks about right and is certainly more than sufficient for most
> clustering environments (IMO).
>
Actually, I hope they are better synchronized, to the extent of tens of
microseconds'
offset. That is what i demand. Is it possible?

BTW, what does IMO stands for? (international maritime org?)

Regards,

Weijian


0
Fang
7/11/2003 1:37:53 PM
Fang Weijian wrote:
> Actually, I hope they are better synchronized, to the extent of tens of
> microseconds'
> offset. That is what i demand. Is it possible?

Only with PPS signals to every individual system.

Using GPS to generate those will keep nano-kernel based machines down 
near the 0-2 us level most of the time.

Terje

-- 
- <Terje.Mathisen@hda.hydro.com>
"almost all programming can be viewed as an exercise in caching"

0
Terje
7/11/2003 7:10:01 PM
>  We still can see
> the offsets are quite widely distributed, from 9 microseconds to more than
1
> milliseconds. Are there any techniques to further improve the
synchronizing
> precision?
>

Try starting ntpd with the "-N" option.


0
Fran
7/14/2003 5:37:37 PM
This is a multi-part message in MIME format.
--------------060806090906040205060302
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Fang Weijian wrote:

>>>Is these result correct? By "correct", I mean this is the effect that
>>>      
>>>
>NTP is
>  
>
>>>supposed to achieve in a cluster environment (Intel/Linux PCs connected
>>>      
>>>
>by
>  
>
>>>fast Ethernet, and synchronized to an internal NTP server). We still can
>>>      
>>>
>see
>  
>
>>>the offsets are quite widely distributed, from 9 microseconds to more
>>>      
>>>
>than 1
>  
>
>>>milliseconds. Are there any techniques to further improve the
>>>      
>>>
>synchronizing
>  
>
>>>precision?
>>>      
>>>
>>That looks about right and is certainly more than sufficient for most
>>clustering environments (IMO).
>>
>>    
>>
>Actually, I hope they are better synchronized, to the extent of tens of
>microseconds' offset. 
>
Not usually, unless you have the NTP server broadcasting on a quiet network.
You can tighten up on the time with a software Phase Lock Loop, but is 
it really necessary?

>That is what i demand. Is it possible?
>
It is possible, but not likely.

>BTW, what does IMO stands for? (international maritime org?)
>
In My Opinion.

>Regards,
>
>Weijian
>  
>
Mike

--------------060806090906040205060302
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1">
  <title></title>
</head>
<body text="#000000" bgcolor="#ffffff">
Fang Weijian wrote:<br>
<blockquote type="cite" cite="midbemeen$38h$1@www.csis.hku.hk">
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">Is these result correct? By "correct", I mean this is the effect that
      </pre>
    </blockquote>
  </blockquote>
  <pre wrap=""><!---->NTP is
  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">supposed to achieve in a cluster environment (Intel/Linux PCs connected
      </pre>
    </blockquote>
  </blockquote>
  <pre wrap=""><!---->by
  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">fast Ethernet, and synchronized to an internal NTP server). We still can
      </pre>
    </blockquote>
  </blockquote>
  <pre wrap=""><!---->see
  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">the offsets are quite widely distributed, from 9 microseconds to more
      </pre>
    </blockquote>
  </blockquote>
  <pre wrap=""><!---->than 1
  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">milliseconds. Are there any techniques to further improve the
      </pre>
    </blockquote>
  </blockquote>
  <pre wrap=""><!---->synchronizing
  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">precision?
      </pre>
    </blockquote>
    <pre wrap="">That looks about right and is certainly more than sufficient for most
clustering environments (IMO).

    </pre>
  </blockquote>
  <pre wrap=""><!---->Actually, I hope they are better synchronized, to the extent of tens of
microseconds' offset. </pre>
</blockquote>
Not usually, unless you have the NTP server broadcasting on a quiet
network.<br>
You can tighten up on the time with a software Phase Lock Loop, but is
it really necessary?<br>
<blockquote type="cite" cite="midbemeen$38h$1@www.csis.hku.hk">
  <pre wrap="">That is what i demand. Is it possible?</pre>
</blockquote>
It is possible, but not likely.<br>
<blockquote type="cite" cite="midbemeen$38h$1@www.csis.hku.hk">
  <pre wrap="">BTW, what does IMO stands for? (international maritime org?)</pre>
</blockquote>
In My Opinion.<br>
<blockquote type="cite" cite="midbemeen$38h$1@www.csis.hku.hk">
  <pre wrap="">Regards,

Weijian
  </pre>
</blockquote>
Mike<br>
</body>
</html>

--------------060806090906040205060302--

0
Michael
7/15/2003 8:04:48 PM
"Michael Klos" <mike@klos.com> wrote in message
news:b9ZQa.13467$G%5.8859@twister.nyroc.rr.com...

> Only if no task disables interrupts for more that a couple of
> microseconds...  Expect no better than 10 us.  If you do better,
> consider yourself lucky and don't expect it to last.

    For PCs, it doesn't matter all that much. The TSC keeps running no
matter what the interrupts do (so timer interrupts shouldn't be much of an
issue) and late interrupts from the PPS input itself are discarded (or at
least, that's the theory).

    DS


0
David
7/16/2003 4:29:01 AM
In article <bf2kae$mhr$1@nntp.webmaster.com>,
David Schwartz <davids@webmaster.com> wrote:
>"Michael Klos" <mike@klos.com> wrote in message
>news:b9ZQa.13467$G%5.8859@twister.nyroc.rr.com...
>> Only if no task disables interrupts for more that a couple of
>> microseconds...  Expect no better than 10 us.  If you do better,
>> consider yourself lucky and don't expect it to last.
>
>    For PCs, it doesn't matter all that much. The TSC keeps running no
>matter what the interrupts do (so timer interrupts shouldn't be much of an
>issue) and late interrupts from the PPS input itself are discarded (or at
>least, that's the theory).

Actually, that's not exactly true.  The TSC doesn't keep running on certain
Intel "mobile" processors when the system has nothing to do (i.e. when the
idle thread does a HLT instruction).  We discovered this the hard way when
we were developing our TimeStamper product for Windows.  We solved this
problem by detecting when we're on those processors, and if so, we create
our own idle thread that does nothing, but doesn't to a HLT.

================= GPS based time synchronization solutions =================
    Patrick Klos                           Email: patrick@timegeeks.com
    Klos Technologies, Inc.                Web:   http://www.timegeeks.com/
============================================================================
0
patrick
7/16/2003 2:24:03 PM
In article <A7ZQa.13459$G%5.7091@twister.nyroc.rr.com>,
Michael Klos <mike@klos.com> wrote:

> You can tighten up on the time with a software Phase Lock Loop, but is 

That's what ntpd *IS*!.  If you think that you can produce a better 
PLL than the one that it implements, you need to propose it as an urgent
upgrade.

> Content-Type: text/html; charset=us-ascii

Syntactically invalid netiquette violation deleted.
0
david
7/16/2003 8:08:00 PM
In article <bf2kae$mhr$1@nntp.webmaster.com>,
David Schwartz <davids@webmaster.com> wrote:

>     For PCs, it doesn't matter all that much. The TSC keeps running no
> matter what the interrupts do (so timer interrupts shouldn't be much of an
> issue) and late interrupts from the PPS input itself are discarded (or at
> least, that's the theory).

It does matter, because most people who want accurate time don't just
want the time to reflect when the time of day system call was made, but
rather reflect the timing of some external event that was likely presented
as an interrupt, or even as the result of poling off clock interrupts; you
may have extremely accurate measurement of the system call time with a
very inaccurate value for the external event.

(TSCs are actually a recent introduction, so not a guaranteed part of the
PC architecture.)
0
david
7/16/2003 8:11:56 PM
<patrick@klos.com> wrote in message news:bf3n63$eic$1@pyrite.mv.net...

> Actually, that's not exactly true.  The TSC doesn't keep running on
certain
> Intel "mobile" processors when the system has nothing to do (i.e. when the
> idle thread does a HLT instruction).  We discovered this the hard way when
> we were developing our TimeStamper product for Windows.  We solved this
> problem by detecting when we're on those processors, and if so, we create
> our own idle thread that does nothing, but doesn't to a HLT.

    You must disable such features (like speedstep) on a time server. Are
there really processors where you can't opt to have the TSC run
continuously? Ouch.

    DS


0
David
7/16/2003 8:19:16 PM
"David Woolley" <david@djwhome.demon.co.uk> wrote in message
news:T1058386348@djwhome.demon.co.uk...

> In article <bf2kae$mhr$1@nntp.webmaster.com>,
> David Schwartz <davids@webmaster.com> wrote:

> >     For PCs, it doesn't matter all that much. The TSC keeps running no
> > matter what the interrupts do (so timer interrupts shouldn't be much of
an
> > issue) and late interrupts from the PPS input itself are discarded (or
at
> > least, that's the theory).

> It does matter, because most people who want accurate time don't just
> want the time to reflect when the time of day system call was made, but
> rather reflect the timing of some external event that was likely presented
> as an interrupt,

    If we're talking about a pure time server that serves times to other
machines, the only thing it needs to timestamp are received packets and
these can be averaged by bursting and NTP's built in averaging and ability
to discard packets delayed for unusually long amounts of time.

> or even as the result of poling off clock interrupts; you
> may have extremely accurate measurement of the system call time with a
> very inaccurate value for the external event.

    If the external event is an NTP packet, it doesn't much matter because
of NTP's own averaging. If the external event is an event you want to
actually time (that is, you've rigged up some sort of hardware to send
events to the machine), yes, it could result in a late time for any event
that occurs during the interrupt. If you're using something like RS-232,
though, the interface will add more random delay than the interrupt.

> (TSCs are actually a recent introduction, so not a guaranteed part of the
> PC architecture.)

    All Pentium and later PCs have them. At this time, there's no excuse for
running a PC time server with any CPU less than a Pentium.

    DS


0
David
7/16/2003 10:06:37 PM
In article <bf3n63$eic$1@pyrite.mv.net>, patrick@klos.com wrote:
>Actually, that's not exactly true.  The TSC doesn't keep running on certain
>Intel "mobile" processors when the system has nothing to do (i.e. when the
>idle thread does a HLT instruction).  We discovered this the hard way when
>we were developing our TimeStamper product for Windows.  We solved this
>problem by detecting when we're on those processors, and if so, we create
>our own idle thread that does nothing, but doesn't to a HLT.


there is a support for things like this in recent linux 2.4 kernels:

Unsynced TSC support
CONFIG_X86_TSC_DISABLE
  This option is used for getting Linux to run on a NUMA multi-node 
  boxes, laptops and other systems suffering from unsynced TSCs or 
  TSC drift, which can cause gettimeofday to return non-monotonic values. 
  Choosing this option will disable the CONFIG_X86_TSC optimization,
  and allows you to then specify "notsc" as a boot option regardless of 
  which processor you have compiled for. 

there is also some code in kernel for handling TSC and APM saving modes
(as I remember in PPSkit patch?).

-- 
Piotr Trojanek
0
ptrojane
7/17/2003 6:59:21 AM
Reply: