Page MenuHomeVyOS Platform

Upgrade ddclient from 3.8.2 to 3.9.0 (support Cloudflare API v4)
Closed, ResolvedPublicBUG

Description

I've been trying to get cloudflare dynamic DNS to work on a new VyOS installation. To troubleshoot I ran ddclient direct with debug messaging and got the following result.

RECEIVE:  HTTP/1.1 410 Gone
RECEIVE:  Date: Tue, 20 Nov 2018 21:40:57 GMT
RECEIVE:  Content-Type: text/html
RECEIVE:  Connection: close
RECEIVE:  Set-Cookie: __cfduid=d53c907651da69ff19bfc787be4f74ac11542750057; expires=Wed, 20-Nov-19 21:40:57 GMT; path=/; domain=.cloudflare.com; HttpOnly
RECEIVE:  Cache-Control: public, max-age=31536000
RECEIVE:  Set-Cookie: __cflb=1594113181; path=/; expires=Wed, 21-Nov-18 20:40:57 GMT
RECEIVE:  Strict-Transport-Security: max-age=15780000; includeSubDomains
RECEIVE:  Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
RECEIVE:  Server: cloudflare
RECEIVE:  CF-RAY: 47ce13f459c8ab90-YYZ
RECEIVE:  
RECEIVE:  This API has been deprecated in favor of API v4, available at https://api.cloudflare.com

It appears the ddclient version used in VyOS (latest rolling version) does not support the current CloudFlare API.

Details

Difficulty level
Easy (less than an hour)
Version
1.2.0-rolling+201811191945
Why the issue appeared?
Other
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Event Timeline

ddclient version 3.8.2 is currently include with VyOS which is 2 versions behind. Latest is 3.9.0 which has the required fixes.

syncer triaged this task as Normal priority.
syncer edited projects, added VyOS 1.2 Crux ( VyOS 1.2.0-rc11); removed VyOS 1.2 Crux.
syncer added subscribers: dmbaturin, syncer.

@dmbaturin can we bump the version of this

syncer lowered the priority of this task from Normal to Low.Feb 5 2019, 2:28 PM

I've tried upgrade version from 3.8.2 to 3.9.0 by replacing the ddclient executable (downloadable from the official site) in /usr/sbin/ddclient
This version needs also the libdata-validate-ip-perl package installed.
After that, reload the configuration and it works.

To elaborate on what was written above, in the case of amd64 packages, a package more recent than 3.8.2 is not available from the debian.org repository as there are no more recent releases. The official SourceForge project has been marked as unmaintained by its owner wimpunk, and 3.8.2 was the last release.

"Newer" releases are actually a fork by somewhat new maintainers, and would require fetching the a release from the GitHub releases. The new package seems to be owned by an unnamed individual, but wimpunk seems to lurk and provide advice and peer review. I would be under the impression that this is a legitimate repository.

@syncer I'm not sure what is considered acceptable or not during the build process as this would be my first time contributing. Some pointers would be appreciated. 🙏 Would fetching and installing the forked GitHub release be acceptable? Provided it tests properly.

The alternative is for me to get involved with the ddclient repository, and try and push a new package and new maintainers to the official Debian repository... which might be quite the endeavour.

As for the libdata-validate-ip-perl dependant package, this I know where to make the changes.

@dmbaturin bump? I'd like to submit a PR for this issue, but would like some guidance regarded to my comment above.

Debian Buster uses 3.8.3 https://packages.debian.org/buster/ddclient which could be considered as using a proper source tree.

@avanier could you please try building this for VyOS 1.2 and see if it fixes your issue? There is already a fork for VyOS 1.2 of ddclient (https://github.com/vyos/ddclient) so we could merge in the changes if you confirm that ddclient 3.8.3 works.

According to https://sourceforge.net/p/ddclient/news/2015/05/ddclient-383-released/

Yet again it's been a while but here is new release of ddclient. As usual, there are some important changes and some documentation is modified. A detailed overview can be found in ChangeLog but here's a quick overview:

added Alpine Linux init scritp - patch send by @Tal on github.
adding support for nsupdate - patch send by @droe on github
allow log username-password combinations - patch send by @dirdi on github
adding support for cloudflare - patch send by @roberthawdon on github
adding support for duckdns - patch send by @gkranis
A very big thank you for everyone who created a pull request on github and for everyone who helped to fix the little issues caused by the new providers.

Even better, looks like wimpunk is maintaining again and there is a fresh release https://github.com/ddclient/ddclient/releases 3.9.0

vyos@vyos:~$ wget http://ftp.de.debian.org/debian/pool/main/d/ddclient/ddclient_3.8.3-1.1_all.deb
Connecting to ftp.de.debian.org (141.76.2.4:80)
ddclient_3.8.3-1.1_a 100% |********************************************************************************************************************************************************************************************| 81924   0:00:00 ETA
vyos@vyos:~$ sudo dpkg -i ddclient_3.8.3-1.1_all.deb
(Reading database ... 59343 files and directories currently installed.)
Preparing to unpack ddclient_3.8.3-1.1_all.deb ...
Unpacking ddclient (3.8.3-1.1) over (3.8.2+vyos2+current1) ...
Setting up ddclient (3.8.3-1.1) ...

Configuration file '/etc/dhcp/dhclient-exit-hooks.d/ddclient'
 ==> Deleted (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** ddclient (Y/I/N/O/D/Z) [default=N] ? d

Configuration file '/etc/dhcp/dhclient-exit-hooks.d/ddclient'
 ==> Deleted (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** ddclient (Y/I/N/O/D/Z) [default=N] ? y
Installing new version of config file /etc/dhcp/dhclient-exit-hooks.d/ddclient ...
insserv: warning: current start runlevel(s) (empty) of script `ddclient' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (0 1 2 3 4 5 6) of script `ddclient' overrides LSB defaults (0 1 6).
Processing triggers for systemd (215-17+deb8u13) ...

Please update your local installation to ddclient 3.8.2 and check if it works as intended on your side. Once this is done we could pick that package to the VyOS installation.

hi @c-po , I've tried installing the 3.8.3 version but it's not compatible with the new CloudFlare API v4. The only way to make ddclient working with cloudflare is using the 3.9.0.

Can you please test installing this package and feedback to me?

https://helix.mybll.net/ddclient_3.9.0+vyos2+current1_all.deb

It also has a dependency on libdata-validate-ip-perl

c-po changed the task status from Open to Backport candidate.Oct 20 2019, 11:01 AM
c-po claimed this task.
c-po set Is it a breaking change? to Unspecified (possibly destroys the router).
c-po renamed this task from dynamic dns cloudflare protocol deprecated to Upgrade ddclient from 3.8.2 to 3.9.0 (support Cloudflare API v4).Oct 21 2019, 2:09 AM
c-po changed Difficulty level from Unknown (require assessment) to Easy (less than an hour).
c-po changed Why the issue appeared? from Will be filled on close to Other.

@c-po I tried your 3.9.0 deb package. I installed with some dependencies (libdata-validate-ip-perl libnet-ipv6addr-perl libnet-netmask-perl libnetwork-ipv4addr-perl).
After that, I modified some config parameters in the dns dynamic section to trigger the update but it doesn't work oob.

I tried execute "ddclient" as root and I receive this error:

WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.
WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.

I created the directory in /etc/ddclient/ and moved the ddclient.conf inside that

root@vyos:~# mv /etc/ddclient.conf /etc/ddclient/.
root@vyos:~# ddclient 
WARNING:  skipping host: test.example.com: 'zone=' is an invalid fully qualified host name.

If the new rolling iso has your commit, I can try update that. Now I'm using VyOS 1.2-rolling-201910160117

Thanks for feeding back. Please try the latest rolling ISO if it works for you.

Maybe additional adjustments to the generated configuration file is required for the new version of ddclient to work?

I've tried the VyOS 1.2-rolling-201910220117 with integrated ddclient 3.9.0 and this conf:

vyos@vyos# show service dns dynamic 
 interface pppoe0 {
     service cloudflare {
         host-name test.mydomain.com
         login [email protected]
         password 000000mycfapikey00000000
     }
 }

but it doesn't work.

Also, I have those problems:

vyos@vyos:~$ sh dns dynamic status 
Traceback (most recent call last):
  File "/usr/libexec/vyos/op_mode/dynamic_dns.py", line 106, in <module>
    main()
  File "/usr/libexec/vyos/op_mode/dynamic_dns.py", line 100, in main
    show_status()
  File "/usr/libexec/vyos/op_mode/dynamic_dns.py", line 48, in show_status
    with open(cache_file, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/ddclient/ddclient.cache'
root@vyos:~# ddclient 
WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.
WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.

so, maybe for the new configuration you need to save the file in /etc/ddclient/ddclient.conf and not in /etc/ddclient.conf

I've also created the folder /etc/ddclient and copied the ddclient.conf there. But I have this error:

root@vyos:~# ddclient 
WARNING:  skipping host: test.mydomain.com: 'zone=' is an invalid fully qualified host name.

ok, I've found a way to make it working.

  • mkdir /var/run/ddclient
  • touch /var/run/ddclient/ddclient.pid
  • edit /etc/ddclient/ddclient.conf:
### Autogenerated by dynamic_dns.py ###
daemon=1m
syslog=yes
ssl=yes
pid=/var/run/ddclient/ddclient.pid
cache=/var/cache/ddclient/ddclient.cache

#
# ddclient configuration for interface "pppoe0":
#
use=if, if=pppoe0

# DynDNS provider configuration for test.mydomain.com
protocol=cloudflare,
ttl=1,
zone=mydomain.com,
[email protected],
password='000000mycfapikey000000',
mydomain.com,test.mydomain.com

(you need the ',' at the end of the line)

Added your reccomended changes and auto create directories if they are non existent. Can you please check again?

I tried the new rolling iso VyOS 1.2-rolling-201910230349 but it doesn't work oob.
I see that you tweaked a bit the ddclient.conf file, but you need to add "zone=" in the conf in order to make it works.

# DynDNS provider configuration for test.mydomain.com
protocol=cloudflare,
ttl=1,
zone=mydomain.com,       <------------------------------------------
[email protected],
password='000000mycfapikey000000',
test.mydomain.com

I don't know if you can add a "zone" field in the vyos conf

vyos@vyos# show service dns dynamic 
 interface pppoe0 {
     service cloudflare {
         host-name test.mydomain.com
         zone mydomain.com <----------------------------------------------------------
         login [email protected]
         password 000000mycfapikey00000000
     }
 }

@pvelati looks like that the zone statement is required for Cloudflare only. I did not find a documentation how it should look like. is it only and always the domain part of your dynamic dns entry? If thats the case it can be automatically rendered into the configuration.

In T1030#46018, @c-po wrote:

@pvelati looks like that the zone statement is required for Cloudflare only. I did not find a documentation how it should look like. is it only and always the domain part of your dynamic dns entry? If thats the case it can be automatically rendered into the configuration.

ok, so if it's only for cloudflare you can render the zone removing the first part of name. so for example:

test.mydomain.com ---> zone will be mydomain.com

so in this way you can use also different multiple tld, like "example.co.at", "example.avocat.fr".
The only doubt I have is that if a user has to register a 3rd level domain, I don't know if the area can be generated correctly. But this is probably an extreme case

You could check the latest rolling which have it

Just installed the latest rolling.
I tried the old conf, delete them, and reconfigure dns dynamic.
thanks @c-po , it's working fine now

Thanks for the feedback - will backport to our LTS branch later on!

The ddclient config file got moved to /etc/ddclient/ddclient.conf but ddclient is still trying to load /etc/ddclient.conf in the latest VyOS 1.3 rolling image.

Upgraded to 1.3-rolling-201911041446 and the config file issue is still present.

vyos@104-1831-gw# show service dns dynamic
 interface eth1 {
     service cloudflare {
         host-name 104-1831.tdude.co
         host-name 104-1831-gw.104-1831.tdude.co
         login [email protected]
         password snipped
         protocol cloudflare
     }
 }
[edit]
vyos@104-1831-gw# sudo journalctl -u ddclient
-- Logs begin at Mon 2019-11-04 20:47:43 EST, end at Mon 2019-11-04 20:52:55 EST. --
Nov 04 20:50:37 104-1831-gw systemd[1]: Starting LSB: Update dynamic domain name service entries...
Nov 04 20:50:37 104-1831-gw ddclient[2811]: WARNING:  file /etc/ddclient.conf: Cannot open file '/etc/ddclient.conf'. (No such file or directory)
Nov 04 20:50:37 104-1831-gw ddclient[2811]: stat() on closed filehandle FD at /usr/sbin/ddclient line 1087.
Nov 04 20:50:37 104-1831-gw ddclient[2811]: Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1088.
Nov 04 20:50:37 104-1831-gw ddclient[2811]: readline() on closed filehandle FD at /usr/sbin/ddclient line 1100.
Nov 04 20:50:37 104-1831-gw systemd[1]: Started LSB: Update dynamic domain name service entries.
Nov 04 20:50:37 104-1831-gw ddclient[2821]: WARNING:  file /etc/ddclient.conf: Cannot open file '/etc/ddclient.conf'. (No such file or directory)

Ah you were using bleeding edge equuleus builds. Ddclient 3.9.0 has been added to recent rolling image.

c-po moved this task from Needs Triage to Finished on the VyOS 1.2 Crux (VyOS 1.2.4) board.
dmbaturin set Issue type to Unspecified (please specify).