`show ntp` not working
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	aderouineau
	Oct 27 2023, 10:15 AM

Description

vyos@router1:~$ show ntp
506 Cannot talk to daemon

vyos@router1:~$ chronyc tracking
506 Cannot talk to daemon

vyos@router1:~$ sudo chronyc tracking
Reference ID    : 6C24CF97 (2600:1f18:4a3:6900:46e8:2cd4:84f2:c189)
Stratum         : 5
Ref time (UTC)  : Fri Oct 27 10:00:40 2023
System time     : 0.000601997 seconds slow of NTP time
Last offset     : -0.000758264 seconds
RMS offset      : 0.000617808 seconds
Frequency       : 6.164 ppm slow
Residual freq   : -8.408 ppm
Skew            : 2.649 ppm
Root delay      : 0.017162237 seconds
Root dispersion : 0.001814640 seconds
Update interval : 62.6 seconds
Leap status     : Normal

Details

Difficulty level: Unknown (require assessment)
Version: 1.5-rolling-202310240118
Why the issue appeared?: Will be filled on close
Is it a breaking change?: Unspecified (possibly destroys the router)
Issue type: Bug (incorrect behavior)

Related Objects

Mentioned In: T5694: NTP should always be allowed from localhost and bindaddress/binddevice can only exist once
Mentioned Here: T5694: NTP should always be allowed from localhost and bindaddress/binddevice can only exist once

Event Timeline

aderouineau triaged this task as Normal priority.Oct 27 2023, 10:15 AM

aderouineau created this task.

aderouineau created this object in space S1 VyOS Public.

How is your current ntp configuration (as outputed by show config commands)?

My current config in VyOS 1.5-rolling-202310240118 looks like this:

set service ntp allow-client address '127.0.0.1'
set service ntp interface 'eth1'
set service ntp listen-address '127.0.0.1'
set service ntp server 194.58.200.20 prefer
set service ntp vrf 'INTERNET'

And the output is (freshly booted the VyOS box):

vyos@vyos:~$ show ntp
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
ntp.netnod.se               5   3    70     +0.002     13.902    +59ns    51us

vyos@vyos:~$ chronyc tracking
506 Cannot talk to daemon

vyos@vyos:~$ sudo chronyc tracking
Reference ID    : C23AC814 (194.58.200.20)
Stratum         : 2
Ref time (UTC)  : Fri Oct 27 15:10:57 2023
System time     : 0.000026836 seconds fast of NTP time
Last offset     : +0.000037590 seconds
RMS offset      : 0.000460026 seconds
Frequency       : 0.089 ppm fast
Residual freq   : +0.009 ppm
Skew            : 4.478 ppm
Root delay      : 0.001148363 seconds
Root dispersion : 0.000316807 seconds
Update interval : 64.2 seconds
Leap status     : Normal

pasik added a subscriber: pasik.Oct 27 2023, 3:34 PM

My VyOS NTP config:

set allow-client address '192.168.0.0/16'
set listen-address '192.168.2.253'
set server time.aws.com pool
set server time.google.com pool

Can you show the output of sudo ls -la /run/chrony?

Mine is:

vyos@router1# sudo ls -la /run/chrony
total 12
drwx------  2 _chrony _chrony  120 Oct 27 17:23 .
drwxr-xr-x 41 root    root    1220 Oct 27 13:55 ..
-rw-r--r--  1 _chrony _chrony 1143 Oct 27 13:41 chrony.conf
srwxr-xr-x  1 _chrony _chrony    0 Oct 27 13:41 chronyd.sock
-rw-r--r--  1 root    root       5 Oct 27 13:41 chrony.pid
-rw-r--r--  1 _chrony _chrony   42 Oct 27 16:50 drift

Is your WAN interface also in net 192.168.0.0/16?

It is not, but I do not want to make my NTP internet-facing anyways.

Here is the output of sudo ls -la /run/chrony (just booted up so drift will probably missing for some time):

vyos@vyos:~$ sudo ls -la /run/chrony/
total 8
drwx------  2 _chrony _chrony  100 Oct 28 03:28 .
drwxr-xr-x 41 root    root    1280 Oct 28 03:29 ..
-rw-r--r--  1 _chrony _chrony 1128 Oct 28 03:28 chrony.conf
srwxr-xr-x  1 _chrony _chrony    0 Oct 28 03:28 chronyd.sock
-rw-r--r--  1 root    root       5 Oct 28 03:28 chrony.pid

Content of the chrony.conf:

vyos@vyos:~$ sudo cat /run/chrony/chrony.conf
### Autogenerated by ntp.py ###

# This would step the system clock if the adjustment is larger than 0.1 seconds,
# but only in the first three clock updates.
makestep 1.0 3

# The rtcsync directive enables a mode where the system time is periodically
# copied to the RTC and chronyd does not try to track its drift. This directive
# cannot be used with the rtcfile directive. On Linux, the RTC copy is performed
# by the kernel every 11 minutes.
rtcsync

# This directive specifies the maximum amount of memory that chronyd is allowed
# to allocate for logging of client accesses and the state that chronyd as an
# NTP server needs to support the interleaved mode for its clients.
clientloglimit 1048576

driftfile /run/chrony/drift
dumpdir /run/chrony
ntsdumpdir /run/chrony
pidfile /run/chrony/chrony.pid

# Determine when will the next leap second occur and what is the current offset
leapsectz right/UTC

user _chrony

# NTP servers to reach out to
server 194.58.200.20 iburst   prefer

# Allowed clients configuration
allow 127.0.0.1

# NTP should listen on configured addresses only
bindaddress 127.0.0.1
binddevice eth1

Does it work if you add this?

set service ntp allow-client address '127.0.0.1'

and/or this?

set service ntp listen-address '127.0.0.1'

I would guess that the /run/chrony/chrony.conf is missing these lines which most likely should always exist:

allow 127.0.0.1
bindaddress 127.0.0.1

I havent been using ninja2 scripting previously but Im guessing something like this would be needed:

https://github.com/vyos/vyos-1x/blob/current/data/templates/chrony/chrony.conf.j2

# Allowed clients configuration
# Localhost shall always be allowed
allow 127.0.0.1
{% if allow_client.address is vyos_defined %}
{%     for address in allow_client.address %}
allow {{ address }}
{%     endfor %}
{% endif %}

# NTP should listen on configured addresses only
# Localhost shall always be listened to
bindaddress 127.0.0.1
{% if listen_address is vyos_defined %}
{%     for address in listen_address %}
bindaddress {{ address }}
{%     endfor %}
{% endif %}
{% if interface is vyos_defined %}
binddevice {{ interface }}
{% endif %}

Unless chronycd misbheaves or dislikes that the allow or bindaddress are defined twice if the user already configured them in VyOS conf-mode.

Will return after some tests...

With my config chronyd still listens locally on 323:

udp        0      0 192.168.2.253:123       0.0.0.0:*                           20420/chronyd
udp        0      0 127.0.0.1:323           0.0.0.0:*                           20420/chronyd
udp6       0      0 ::1:323                 :::*                                20420/chronyd

I do not understand why chronyc is not able to connect to it.

I found the issue. I was missing a firewall input rule to allow anything from lo.

aderouineau closed this task as Invalid.Oct 28 2023, 2:26 AM

I added the above modifications to /usr/share/vyos/templates/chrony/chrony.conf.j2 and rebooted VyOS 1.5-rolling-202310240118.

My current config in VyOS (as in untouched from previously):

set service ntp allow-client address '127.0.0.1'
set service ntp interface 'eth1'
set service ntp listen-address '127.0.0.1'
set service ntp server 194.58.200.20 prefer
set service ntp vrf 'INTERNET'

Output of /run/chrony/chrony.conf:

### Autogenerated by ntp.py ###

# This would step the system clock if the adjustment is larger than 0.1 seconds,
# but only in the first three clock updates.
makestep 1.0 3

# The rtcsync directive enables a mode where the system time is periodically
# copied to the RTC and chronyd does not try to track its drift. This directive
# cannot be used with the rtcfile directive. On Linux, the RTC copy is performed
# by the kernel every 11 minutes.
rtcsync

# This directive specifies the maximum amount of memory that chronyd is allowed
# to allocate for logging of client accesses and the state that chronyd as an
# NTP server needs to support the interleaved mode for its clients.
clientloglimit 1048576

driftfile /run/chrony/drift
dumpdir /run/chrony
ntsdumpdir /run/chrony
pidfile /run/chrony/chrony.pid

# Determine when will the next leap second occur and what is the current offset
leapsectz right/UTC

user _chrony

# NTP servers to reach out to
server 194.58.200.20 iburst   prefer

# Allowed clients configuration
# Localhost shall always be allowed
allow 127.0.0.1
allow 127.0.0.1

# NTP should listen on configured addresses only
# Localhost shall always be listened to
bindaddress 127.0.0.1
bindaddress 127.0.0.1
binddevice eth1

Output of show ntp:

vyos@vyos:~$ show ntp
                             .- Number of sample points in measurement set.
                            /    .- Number of residual runs with same sign.
                           |    /    .- Length of measurement set (time).
                           |   |    /      .- Est. clock freq error (ppm).
                           |   |   |      /           .- Est. error in freq.
                           |   |   |     |           /         .- Est. offset.
                           |   |   |     |          |          |   On the -.
                           |   |   |     |          |          |   samples. \
                           |   |   |     |          |          |             |
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
ntp.netnod.se               8   4   265     -0.058      1.415   -767ns    53us

Output of sudo chronyc tracking:

vyos@vyos:~$ sudo chronyc tracking
Reference ID    : C23AC814 (194.58.200.20)
Stratum         : 2
Ref time (UTC)  : Sat Oct 28 02:13:22 2023
System time     : 0.000000094 seconds fast of NTP time
Last offset     : -0.000141772 seconds
RMS offset      : 0.000690538 seconds
Frequency       : 0.861 ppm fast
Residual freq   : -0.058 ppm
Skew            : 1.691 ppm
Root delay      : 0.001330972 seconds
Root dispersion : 0.000247640 seconds
Update interval : 65.0 seconds
Leap status     : Normal

So it seems that chronyd doesnt mind duplicates entries in its conf-file. Which also means we dont have to filter for that in the ninja2 template nor do we need any migration script to deal with this (users who already defined 127.0.0.1 in either of allow or listen can still have those lines in their configs).

If you want to test right now you can take a backup of /usr/share/vyos/templates/chrony/chrony.conf.j2 and add the modifications to the original file as posted in previous post regarding allow and listen sections.

Then best is probably to just reboot the box (not sure if doing changes in the config to trigger a commit is enough unless you poke around in the ntp section of the conf?).

Also...

Seems like you can only define one listen-address in VyOS conf-mode and not sure how interface (aka binddevice) is related to this.

That is if this is related to limits in chronyd or if this is a limit in VyOS only.

More testing is probably needed before creating a commit.

For example if NTP-clients can sync to the VyOS after the changes to the template.

Ahh yes, I think there is another task in here regarding adding firewall rules by default to the firewall to avoid situations like this :-)

As it seems according to https://manpages.debian.org/bookworm/chrony/chrony.conf.5.en.html both bindaddress and binddevice can only be specified once.

The documentation isnt clear about what will happen if both are specified but generally speaking binddevice is the better option since that will also work if the IP-address of the interface is set dynamically.

So IF any modifications to the template would occur then they should look something like this (added localhost as allowed and cleaned up the multichoice who doesnt exist regarding bindaddress):

# Allowed clients configuration
# Localhost shall always be allowed
allow 127.0.0.1
{% if allow_client.address is vyos_defined %}
{%     for address in allow_client.address %}
allow {{ address }}
{%     endfor %}
{% endif %}

# NTP should only listen on configured address
{% if listen_address is vyos_defined %}
bindaddress {{ listen_address }}
{% endif %}
{% if interface is vyos_defined %}
binddevice {{ interface }}
{% endif %}

Since the root cause for this task have been identified and fixed by the reporting user (and the task is set to invalid) I have created another task for the spinoff regarding cleaning up of the template used by chronyd:

https://vyos.dev/T5694

`show ntp` not workingClosed, InvalidPublicActions

Description

Details

Related Objects

Event Timeline

`show ntp` not working
Closed, InvalidPublic
Actions