Hello all,

I am completely stumped by an NTP problem that I am having on a server.
 I've googled and tried everything to no luck, am hoping that someone
here can provide some advice.  Here is the situation:

I have a server running CentOS 4.3 2.6.9-34.0.2.ELsmp on a PDSMi
Supermicro motherboard with a Celeron D 3.06GHz processor.  Note, I'm
running an SMP kernel so that APIC can handle the interrupts properly.

I have /etc/ntp.conf configured to the CentOS 4.3 defaults, which means
it is polling the 0-2.pool.ntp.org server pool.  Upon boot, everything
looks okay, but within a minute or two, the jitter sky rockets:

[root@neon ~]# date
Tue Aug  1 05:30:09 EDT 2006
[root@neon ~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset
jitter
==============================================================================
 ouvaton.info    134.157.254.19   2 u   31   64    1   87.432  -164.78
 0.001
 bq.serverrack.n 65.111.164.223   3 u   30   64    1    0.636  -177.35
 0.001
 msb.significant 64.142.72.248    3 u   29   64    1    2.278  -166.81
 0.001
 LOCAL(0)        LOCAL(0)        10 l   28   64    1    0.000    0.000
 0.001
[root@neon ~]# date
Tue Aug  1 05:31:59 EDT 2006
[root@neon ~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset
jitter
==============================================================================
 ouvaton.info    134.157.254.19   2 u   12   64    7   87.308  -748.55
578.177
 bq.serverrack.n 65.111.164.223   3 u   10   64    7    0.604  -752.02
587.210
 msb.significant 64.142.72.248    3 u   10   64    7    1.921  -750.65
582.576
 LOCAL(0)        LOCAL(0)        10 l   10   64    7    0.000    0.000
 0.001

If I check the associations, they always show 'reject':

[root@neon ~]# ntpq
ntpq> associations

ind assID status  conf reach auth condition  last_event cnt
===========================================================
  1 34972  9014   yes   yes  none    reject   reachable  1
  2 34973  9014   yes   yes  none    reject   reachable  1
  3 34974  9014   yes   yes  none    reject   reachable  1
  4 34975  9014   yes   yes  none    reject   reachable  1

I'm just baffled to why this is occuring; within an hour it'll be >
1000.  I have run ntpdate several times and set the hwlock to the
system time, but inevitably the system clock runs way too fast, gaining
several minutes per day.  Note, if I do not run NTP at all, the hwclock
stays perfect, but the system clock still speeds too fast.  So, I
cannot be sure if this is a HW problem, network problem, or
configuration issue.  I have tried adding noapic, nolapic, noacpi to
the kernel bootup to no avail.  I have tried to boot into a
uniprocessor kernel, same behavior.  The server is hosted at Equinix in
Ashburn, VA and the network appears okay:

[root@neon ~]# mii-tool
eth0: negotiated 100baseTx-FD, link ok

I have setup NTP logging, but really do not know enough about the
protocol to troubleshoot effectively.  I've also tried pointing at
other NTP servers and get the same behavior.  I do not currently have
iptables running (the server is not in production yet) so 123 UDP
traffic is going through, and the provider has verified that there is
nothing on their side blocking it.

Any help at all would be appreciated.  I will be glad to supply log
files, trace files, anything to get this resolved.  Thanks so much.

Best Regards,
Tom

ter310@gmail.com schrieb:
> Hello all,
> 
> I am completely stumped by an NTP problem that I am having on a server.
>  I've googled and tried everything to no luck, am hoping that someone
> here can provide some advice.  Here is the situation:
> 
> I have a server running CentOS 4.3 2.6.9-34.0.2.ELsmp on a PDSMi
> Supermicro motherboard with a Celeron D 3.06GHz processor.  Note, I'm
> running an SMP kernel so that APIC can handle the interrupts properly.
> 
> I have /etc/ntp.conf configured to the CentOS 4.3 defaults, which means
> it is polling the 0-2.pool.ntp.org server pool.  Upon boot, everything
> looks okay, but within a minute or two, the jitter sky rockets:
> 
[cut ntpq -p output]

Tom,

Looking at the differences of the two billboards you sent us, your clock 
is really running fast ( 600 ms in two minutes? God heavens! ).

NTP will give up when it faces such a big problem, so your first goal is 
to get your system clock in a healthy state. Try it with a non-SMP 
kernel first in order to rule out the SMP kernel as a potential cause 
and it might be a good idea to contact the vendor of your server or 
mainboard and ask for help (BIOS update available?).

Best regards,
Heiko


-- 
Meinberg radio clocks: 25 years of accurate time worldwide

MEINBERG Radio Clocks
www.meinberg.de

Stand alone ntp time servers and radio clocks based on GPS, DCF77 and 
IRIG. Rackmount and desktop versions and PCI slot cards.

<ter310@gmail.com> wrote:
> Hello all,
>
> I am completely stumped by an NTP problem that I am having on a server.
> I've googled and tried everything to no luck, am hoping that someone
> here can provide some advice.  Here is the situation:
>
> I have a server running CentOS 4.3 2.6.9-34.0.2.ELsmp on a PDSMi
> Supermicro motherboard with a Celeron D 3.06GHz processor.  Note, I'm
> running an SMP kernel so that APIC can handle the interrupts properly.
>
> I have /etc/ntp.conf configured to the CentOS 4.3 defaults, which means
> it is polling the 0-2.pool.ntp.org server pool.  Upon boot, everything
> looks okay, but within a minute or two, the jitter sky rockets:
>
> [root@neon ~]# date
> Tue Aug  1 05:30:09 EDT 2006
> [root@neon ~]# ntpq -p
>     remote           refid      st t when poll reach   delay   offset
> jitter
> ==============================================================================
> ouvaton.info    134.157.254.19   2 u   31   64    1   87.432  -164.78
> 0.001
> bq.serverrack.n 65.111.164.223   3 u   30   64    1    0.636  -177.35
> 0.001
> msb.significant 64.142.72.248    3 u   29   64    1    2.278  -166.81
> 0.001
> LOCAL(0)        LOCAL(0)        10 l   28   64    1    0.000    0.000
> 0.001
> [root@neon ~]# date
> Tue Aug  1 05:31:59 EDT 2006
> [root@neon ~]# ntpq -p
>     remote           refid      st t when poll reach   delay   offset
> jitter
> ==============================================================================
> ouvaton.info    134.157.254.19   2 u   12   64    7   87.308  -748.55
> 578.177
> bq.serverrack.n 65.111.164.223   3 u   10   64    7    0.604  -752.02
> 587.210
> msb.significant 64.142.72.248    3 u   10   64    7    1.921  -750.65
> 582.576
> LOCAL(0)        LOCAL(0)        10 l   10   64    7    0.000    0.000
> 0.001
>
> If I check the associations, they always show 'reject':
>
> [root@neon ~]# ntpq
> ntpq> associations
>
> ind assID status  conf reach auth condition  last_event cnt
> ===========================================================
>  1 34972  9014   yes   yes  none    reject   reachable  1
>  2 34973  9014   yes   yes  none    reject   reachable  1
>  3 34974  9014   yes   yes  none    reject   reachable  1
>  4 34975  9014   yes   yes  none    reject   reachable  1
>
> I'm just baffled to why this is occuring; within an hour it'll be >
> 1000.  I have run ntpdate several times and set the hwlock to the
> system time, but inevitably the system clock runs way too fast, gaining
> several minutes per day.  Note, if I do not run NTP at all, the hwclock
> stays perfect, but the system clock still speeds too fast.  So, I
> cannot be sure if this is a HW problem, network problem, or
> configuration issue.  I have tried adding noapic, nolapic, noacpi to
> the kernel bootup to no avail.  I have tried to boot into a
> uniprocessor kernel, same behavior.  The server is hosted at Equinix in
> Ashburn, VA and the network appears okay:
>
> [root@neon ~]# mii-tool
> eth0: negotiated 100baseTx-FD, link ok
>
> I have setup NTP logging, but really do not know enough about the
> protocol to troubleshoot effectively.  I've also tried pointing at
> other NTP servers and get the same behavior.  I do not currently have
> iptables running (the server is not in production yet) so 123 UDP
> traffic is going through, and the provider has verified that there is
> nothing on their side blocking it.
>
> Any help at all would be appreciated.  I will be glad to supply log
> files, trace files, anything to get this resolved.  Thanks so much.

The jitter mirrors only an offset grow here. It seems that the frequency of 
your system clock is off by ~5200ppm. The maximum frequency error the ntpd 
can correct is 512ppm. Try tickadj or adjtimex -p to see your current tick. 
Its value is probably 10000. If so, change this value near to 9950 and try 
ntpd again.
Maybe that helps. Nevertheless, the error 8m per day is unusual and 
worrying.
--
Karel Sandler

Karel, Heiko,

Thank you for the responses.  I have booted the server with a non-SMP
kernel (2.6.9-34.0.2.EL & 2.6.9-34.EL), however the fast system clock
behavior continues.  It is odd, the hwclock functions nice and fine
when left alone.  However, the system clock flys.  At this point, I'm
assuming that this is some sort of kernel/OS bug.

I've tried noapci, nolapic, noapci, clock_timer=0, and some other
workarounds but haven't had any luck.  One thread I googled suggested
to disable the FSB on the server.  The server is colocated an hour away
from where I usually am, and I'm currently on business travel.  That
would have to be done via the BIOS, so I can try it later if all else
fails.  Still, those threads seemed to focus on a known kernel bug with
NVidia chips... which this doesn't have.  If it helps at all, the exact
MB that I have is:
http://www.supermicro.com/products/motherboard/PD/E7230/PDSMi.cfm

I did try to adjust my ticks to 9950, and later 9000 via the tickadj
command, but that did not have an impact.  Next step on this end will
be to open a support call with Supermicro as they produce the
motherboard and I'll keep searching.

Thank you for all of the help, it is greatly appreciated.
Tom

Karel Sandler wrote:
> <ter310@gmail.com> wrote:
> > Hello all,
> >
> > I am completely stumped by an NTP problem that I am having on a server.
> > I've googled and tried everything to no luck, am hoping that someone
> > here can provide some advice.  Here is the situation:
> >
> > I have a server running CentOS 4.3 2.6.9-34.0.2.ELsmp on a PDSMi
> > Supermicro motherboard with a Celeron D 3.06GHz processor.  Note, I'm
> > running an SMP kernel so that APIC can handle the interrupts properly.
> >
> > I have /etc/ntp.conf configured to the CentOS 4.3 defaults, which means
> > it is polling the 0-2.pool.ntp.org server pool.  Upon boot, everything
> > looks okay, but within a minute or two, the jitter sky rockets:
> >
> > [root@neon ~]# date
> > Tue Aug  1 05:30:09 EDT 2006
> > [root@neon ~]# ntpq -p
> >     remote           refid      st t when poll reach   delay   offset
> > jitter
> > ==============================================================================
> > ouvaton.info    134.157.254.19   2 u   31   64    1   87.432  -164.78
> > 0.001
> > bq.serverrack.n 65.111.164.223   3 u   30   64    1    0.636  -177.35
> > 0.001
> > msb.significant 64.142.72.248    3 u   29   64    1    2.278  -166.81
> > 0.001
> > LOCAL(0)        LOCAL(0)        10 l   28   64    1    0.000    0.000
> > 0.001
> > [root@neon ~]# date
> > Tue Aug  1 05:31:59 EDT 2006
> > [root@neon ~]# ntpq -p
> >     remote           refid      st t when poll reach   delay   offset
> > jitter
> > ==============================================================================
> > ouvaton.info    134.157.254.19   2 u   12   64    7   87.308  -748.55
> > 578.177
> > bq.serverrack.n 65.111.164.223   3 u   10   64    7    0.604  -752.02
> > 587.210
> > msb.significant 64.142.72.248    3 u   10   64    7    1.921  -750.65
> > 582.576
> > LOCAL(0)        LOCAL(0)        10 l   10   64    7    0.000    0.000
> > 0.001
> >
> > If I check the associations, they always show 'reject':
> >
> > [root@neon ~]# ntpq
> > ntpq> associations
> >
> > ind assID status  conf reach auth condition  last_event cnt
> > ===========================================================
> >  1 34972  9014   yes   yes  none    reject   reachable  1
> >  2 34973  9014   yes   yes  none    reject   reachable  1
> >  3 34974  9014   yes   yes  none    reject   reachable  1
> >  4 34975  9014   yes   yes  none    reject   reachable  1
> >
> > I'm just baffled to why this is occuring; within an hour it'll be >
> > 1000.  I have run ntpdate several times and set the hwlock to the
> > system time, but inevitably the system clock runs way too fast, gaining
> > several minutes per day.  Note, if I do not run NTP at all, the hwclock
> > stays perfect, but the system clock still speeds too fast.  So, I
> > cannot be sure if this is a HW problem, network problem, or
> > configuration issue.  I have tried adding noapic, nolapic, noacpi to
> > the kernel bootup to no avail.  I have tried to boot into a
> > uniprocessor kernel, same behavior.  The server is hosted at Equinix in
> > Ashburn, VA and the network appears okay:
> >
> > [root@neon ~]# mii-tool
> > eth0: negotiated 100baseTx-FD, link ok
> >
> > I have setup NTP logging, but really do not know enough about the
> > protocol to troubleshoot effectively.  I've also tried pointing at
> > other NTP servers and get the same behavior.  I do not currently have
> > iptables running (the server is not in production yet) so 123 UDP
> > traffic is going through, and the provider has verified that there is
> > nothing on their side blocking it.
> >
> > Any help at all would be appreciated.  I will be glad to supply log
> > files, trace files, anything to get this resolved.  Thanks so much.
>
> The jitter mirrors only an offset grow here. It seems that the frequency of
> your system clock is off by ~5200ppm. The maximum frequency error the ntpd
> can correct is 512ppm. Try tickadj or adjtimex -p to see your current tick.
> Its value is probably 10000. If so, change this value near to 9950 and try
> ntpd again.
> Maybe that helps. Nevertheless, the error 8m per day is unusual and
> worrying.
> --
> Karel Sandler

ter310@gmail.com wrote:
> Hello all,
> 
> I am completely stumped by an NTP problem that I am having on a server.
>  I've googled and tried everything to no luck, am hoping that someone
> here can provide some advice.  Here is the situation:
> 
> I have a server running CentOS 4.3 2.6.9-34.0.2.ELsmp on a PDSMi
> Supermicro motherboard with a Celeron D 3.06GHz processor.  Note, I'm
> running an SMP kernel so that APIC can handle the interrupts properly.
> 
> I have /etc/ntp.conf configured to the CentOS 4.3 defaults, which means
> it is polling the 0-2.pool.ntp.org server pool.  Upon boot, everything
> looks okay, but within a minute or two, the jitter sky rockets:
> 
> [root@neon ~]# date
> Tue Aug  1 05:30:09 EDT 2006
> [root@neon ~]# ntpq -p
>      remote           refid      st t when poll reach   delay   offset
> jitter
> ==============================================================================
>  ouvaton.info    134.157.254.19   2 u   31   64    1   87.432  -164.78
>  0.001
>  bq.serverrack.n 65.111.164.223   3 u   30   64    1    0.636  -177.35
>  0.001
>  msb.significant 64.142.72.248    3 u   29   64    1    2.278  -166.81
>  0.001
>  LOCAL(0)        LOCAL(0)        10 l   28   64    1    0.000    0.000
>  0.001
> [root@neon ~]# date
> Tue Aug  1 05:31:59 EDT 2006
> [root@neon ~]# ntpq -p
>      remote           refid      st t when poll reach   delay   offset
> jitter
> ==============================================================================
>  ouvaton.info    134.157.254.19   2 u   12   64    7   87.308  -748.55
> 578.177
>  bq.serverrack.n 65.111.164.223   3 u   10   64    7    0.604  -752.02
> 587.210
>  msb.significant 64.142.72.248    3 u   10   64    7    1.921  -750.65
> 582.576
>  LOCAL(0)        LOCAL(0)        10 l   10   64    7    0.000    0.000
>  0.001
> 
> If I check the associations, they always show 'reject':
> 
> [root@neon ~]# ntpq
> ntpq> associations
> 
> ind assID status  conf reach auth condition  last_event cnt
> ===========================================================
>   1 34972  9014   yes   yes  none    reject   reachable  1
>   2 34973  9014   yes   yes  none    reject   reachable  1
>   3 34974  9014   yes   yes  none    reject   reachable  1
>   4 34975  9014   yes   yes  none    reject   reachable  1
> 
> I'm just baffled to why this is occuring; within an hour it'll be >
> 1000.  I have run ntpdate several times and set the hwlock to the
> system time, but inevitably the system clock runs way too fast, gaining
> several minutes per day.  Note, if I do not run NTP at all, the hwclock
> stays perfect, but the system clock still speeds too fast.  So, I
> cannot be sure if this is a HW problem, network problem, or
> configuration issue.  I have tried adding noapic, nolapic, noacpi to
> the kernel bootup to no avail.  I have tried to boot into a
> uniprocessor kernel, same behavior.  The server is hosted at Equinix in
> Ashburn, VA and the network appears okay:
> 
> [root@neon ~]# mii-tool
> eth0: negotiated 100baseTx-FD, link ok
> 
> I have setup NTP logging, but really do not know enough about the
> protocol to troubleshoot effectively.  I've also tried pointing at
> other NTP servers and get the same behavior.  I do not currently have
> iptables running (the server is not in production yet) so 123 UDP
> traffic is going through, and the provider has verified that there is
> nothing on their side blocking it.
> 
> Any help at all would be appreciated.  I will be glad to supply log
> files, trace files, anything to get this resolved.  Thanks so much.
> 
> Best Regards,
> Tom
> 

It sounds as if your system clock has a frequency error greater than 500 
Parts Per Million (PPM).   500 PPM is the maximum error that ntpd can 
correct.  IF that error is exceeded, your choices are to have the 
hardware repaired, replace the hardware, or give up!

Richard,

I'm fairly certain your right.  Am going to log a call with the
motherboard company.  I have another server with the same MB so I'll
test that when I get back from travelling to see if the issue
duplicates.  Either way, it looks like a hardware replacement is in
order.  This server will be running transactions so an accurate date is
vital.

Thanks for the help.
Tom

Richard B. Gilbert wrote:
> ter310@gmail.com wrote:
> > Hello all,
> >
> > I am completely stumped by an NTP problem that I am having on a server.
> >  I've googled and tried everything to no luck, am hoping that someone
> > here can provide some advice.  Here is the situation:
> >
> > I have a server running CentOS 4.3 2.6.9-34.0.2.ELsmp on a PDSMi
> > Supermicro motherboard with a Celeron D 3.06GHz processor.  Note, I'm
> > running an SMP kernel so that APIC can handle the interrupts properly.
> >
> > I have /etc/ntp.conf configured to the CentOS 4.3 defaults, which means
> > it is polling the 0-2.pool.ntp.org server pool.  Upon boot, everything
> > looks okay, but within a minute or two, the jitter sky rockets:
> >
> > [root@neon ~]# date
> > Tue Aug  1 05:30:09 EDT 2006
> > [root@neon ~]# ntpq -p
> >      remote           refid      st t when poll reach   delay   offset
> > jitter
> > ==============================================================================
> >  ouvaton.info    134.157.254.19   2 u   31   64    1   87.432  -164.78
> >  0.001
> >  bq.serverrack.n 65.111.164.223   3 u   30   64    1    0.636  -177.35
> >  0.001
> >  msb.significant 64.142.72.248    3 u   29   64    1    2.278  -166.81
> >  0.001
> >  LOCAL(0)        LOCAL(0)        10 l   28   64    1    0.000    0.000
> >  0.001
> > [root@neon ~]# date
> > Tue Aug  1 05:31:59 EDT 2006
> > [root@neon ~]# ntpq -p
> >      remote           refid      st t when poll reach   delay   offset
> > jitter
> > ==============================================================================
> >  ouvaton.info    134.157.254.19   2 u   12   64    7   87.308  -748.55
> > 578.177
> >  bq.serverrack.n 65.111.164.223   3 u   10   64    7    0.604  -752.02
> > 587.210
> >  msb.significant 64.142.72.248    3 u   10   64    7    1.921  -750.65
> > 582.576
> >  LOCAL(0)        LOCAL(0)        10 l   10   64    7    0.000    0.000
> >  0.001
> >
> > If I check the associations, they always show 'reject':
> >
> > [root@neon ~]# ntpq
> > ntpq> associations
> >
> > ind assID status  conf reach auth condition  last_event cnt
> > ===========================================================
> >   1 34972  9014   yes   yes  none    reject   reachable  1
> >   2 34973  9014   yes   yes  none    reject   reachable  1
> >   3 34974  9014   yes   yes  none    reject   reachable  1
> >   4 34975  9014   yes   yes  none    reject   reachable  1
> >
> > I'm just baffled to why this is occuring; within an hour it'll be >
> > 1000.  I have run ntpdate several times and set the hwlock to the
> > system time, but inevitably the system clock runs way too fast, gaining
> > several minutes per day.  Note, if I do not run NTP at all, the hwclock
> > stays perfect, but the system clock still speeds too fast.  So, I
> > cannot be sure if this is a HW problem, network problem, or
> > configuration issue.  I have tried adding noapic, nolapic, noacpi to
> > the kernel bootup to no avail.  I have tried to boot into a
> > uniprocessor kernel, same behavior.  The server is hosted at Equinix in
> > Ashburn, VA and the network appears okay:
> >
> > [root@neon ~]# mii-tool
> > eth0: negotiated 100baseTx-FD, link ok
> >
> > I have setup NTP logging, but really do not know enough about the
> > protocol to troubleshoot effectively.  I've also tried pointing at
> > other NTP servers and get the same behavior.  I do not currently have
> > iptables running (the server is not in production yet) so 123 UDP
> > traffic is going through, and the provider has verified that there is
> > nothing on their side blocking it.
> >
> > Any help at all would be appreciated.  I will be glad to supply log
> > files, trace files, anything to get this resolved.  Thanks so much.
> >
> > Best Regards,
> > Tom
> >
>
> It sounds as if your system clock has a frequency error greater than 500
> Parts Per Million (PPM).   500 PPM is the maximum error that ntpd can
> correct.  IF that error is exceeded, your choices are to have the
> hardware repaired, replace the hardware, or give up!

Possibly an MB problem, but I would check the native clock drift
before calling Supermicro.
I have never heard of CentOS, but lord Google says it is a Redhat
ES derivative. If that is the case then I suggest you remove
/etc/adjtime , disable ntp startup and then reboot. Once
running check the clock drift against a known good source over
a longish period of time. If you chose a few hours as the time
base you could use ntpdate -d some-good-ref to get timestamps
for calculating the native ppm drift. If it is outside the motherboard 
manafacturers spec then you have some data on which they can act.

I see the BIOS supports cpu thermal management which allows the
clock frequency to change if the processor gets too hot.
Is your fan ok?
http://www.supermicro.com/manuals/motherboard/E7230/MNL-0808.pdf
Mike


ter310@gmail.com wrote:
> Hello all,
> 
> I am completely stumped by an NTP problem that I am having on a server.
>  I've googled and tried everything to no luck, am hoping that someone
> here can provide some advice.  Here is the situation:
> 
> I have a server running CentOS 4.3 2.6.9-34.0.2.ELsmp on a PDSMi
> Supermicro motherboard with a Celeron D 3.06GHz processor.  Note, I'm
> running an SMP kernel so that APIC can handle the interrupts properly.
> 
> I have /etc/ntp.conf configured to the CentOS 4.3 defaults, which means
> it is polling the 0-2.pool.ntp.org server pool.  Upon boot, everything
> looks okay, but within a minute or two, the jitter sky rockets:
> 
> [root@neon ~]# date
> Tue Aug  1 05:30:09 EDT 2006
> [root@neon ~]# ntpq -p
>      remote           refid      st t when poll reach   delay   offset
> jitter
> ==============================================================================
>  ouvaton.info    134.157.254.19   2 u   31   64    1   87.432  -164.78
>  0.001
>  bq.serverrack.n 65.111.164.223   3 u   30   64    1    0.636  -177.35
>  0.001
>  msb.significant 64.142.72.248    3 u   29   64    1    2.278  -166.81
>  0.001
>  LOCAL(0)        LOCAL(0)        10 l   28   64    1    0.000    0.000
>  0.001
> [root@neon ~]# date
> Tue Aug  1 05:31:59 EDT 2006
> [root@neon ~]# ntpq -p
>      remote           refid      st t when poll reach   delay   offset
> jitter
> ==============================================================================
>  ouvaton.info    134.157.254.19   2 u   12   64    7   87.308  -748.55
> 578.177
>  bq.serverrack.n 65.111.164.223   3 u   10   64    7    0.604  -752.02
> 587.210
>  msb.significant 64.142.72.248    3 u   10   64    7    1.921  -750.65
> 582.576
>  LOCAL(0)        LOCAL(0)        10 l   10   64    7    0.000    0.000
>  0.001
> 
> If I check the associations, they always show 'reject':
> 
> [root@neon ~]# ntpq
> ntpq> associations
> 
> ind assID status  conf reach auth condition  last_event cnt
> ===========================================================
>   1 34972  9014   yes   yes  none    reject   reachable  1
>   2 34973  9014   yes   yes  none    reject   reachable  1
>   3 34974  9014   yes   yes  none    reject   reachable  1
>   4 34975  9014   yes   yes  none    reject   reachable  1
> 
> I'm just baffled to why this is occuring; within an hour it'll be >
> 1000.  I have run ntpdate several times and set the hwlock to the
> system time, but inevitably the system clock runs way too fast, gaining
> several minutes per day.  Note, if I do not run NTP at all, the hwclock
> stays perfect, but the system clock still speeds too fast.  So, I
> cannot be sure if this is a HW problem, network problem, or
> configuration issue.  I have tried adding noapic, nolapic, noacpi to
> the kernel bootup to no avail.  I have tried to boot into a
> uniprocessor kernel, same behavior.  The server is hosted at Equinix in
> Ashburn, VA and the network appears okay:
> 
> [root@neon ~]# mii-tool
> eth0: negotiated 100baseTx-FD, link ok
> 
> I have setup NTP logging, but really do not know enough about the
> protocol to troubleshoot effectively.  I've also tried pointing at
> other NTP servers and get the same behavior.  I do not currently have
> iptables running (the server is not in production yet) so 123 UDP
> traffic is going through, and the provider has verified that there is
> nothing on their side blocking it.
> 
> Any help at all would be appreciated.  I will be glad to supply log
> files, trace files, anything to get this resolved.  Thanks so much.
> 
> Best Regards,
> Tom
>