Page MenuHomePhabricator

Upgrade ddclient from 3.8.2 to 3.9.0 (support Cloudflare API v4)
Closed, ResolvedPublicBUG

Description

I've been trying to get cloudflare dynamic DNS to work on a new VyOS installation. To troubleshoot I ran ddclient direct with debug messaging and got the following result.

RECEIVE:  HTTP/1.1 410 Gone
RECEIVE:  Date: Tue, 20 Nov 2018 21:40:57 GMT
RECEIVE:  Content-Type: text/html
RECEIVE:  Connection: close
RECEIVE:  Set-Cookie: __cfduid=d53c907651da69ff19bfc787be4f74ac11542750057; expires=Wed, 20-Nov-19 21:40:57 GMT; path=/; domain=.cloudflare.com; HttpOnly
RECEIVE:  Cache-Control: public, max-age=31536000
RECEIVE:  Set-Cookie: __cflb=1594113181; path=/; expires=Wed, 21-Nov-18 20:40:57 GMT
RECEIVE:  Strict-Transport-Security: max-age=15780000; includeSubDomains
RECEIVE:  Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
RECEIVE:  Server: cloudflare
RECEIVE:  CF-RAY: 47ce13f459c8ab90-YYZ
RECEIVE:  
RECEIVE:  This API has been deprecated in favor of API v4, available at https://api.cloudflare.com

It appears the ddclient version used in VyOS (latest rolling version) does not support the current CloudFlare API.

Details

Difficulty level
Easy (less than an hour)
Version
1.2.0-rolling+201811191945
Why the issue appeared?
Other
Is it a breaking change?
Perfectly compatible

Event Timeline

artooro created this task.Nov 20 2018, 9:48 PM

ddclient version 3.8.2 is currently include with VyOS which is 2 versions behind. Latest is 3.9.0 which has the required fixes.

syncer assigned this task to dmbaturin.Dec 1 2018, 5:51 PM
syncer triaged this task as Normal priority.
syncer edited projects, added VyOS 1.2 Crux ( VyOS 1.2.0-rc11); removed VyOS 1.2 Crux.
syncer added subscribers: dmbaturin, syncer.

@dmbaturin can we bump the version of this

pasik added a subscriber: pasik.Dec 16 2018, 11:22 AM
syncer lowered the priority of this task from Normal to Low.Feb 5 2019, 2:28 PM

I've tried upgrade version from 3.8.2 to 3.9.0 by replacing the ddclient executable (downloadable from the official site) in /usr/sbin/ddclient
This version needs also the libdata-validate-ip-perl package installed.
After that, reload the configuration and it works.

avanier added a comment.EditedSep 23 2019, 1:30 AM

To elaborate on what was written above, in the case of amd64 packages, a package more recent than 3.8.2 is not available from the debian.org repository as there are no more recent releases. The official SourceForge project has been marked as unmaintained by its owner wimpunk, and 3.8.2 was the last release.

"Newer" releases are actually a fork by somewhat new maintainers, and would require fetching the a release from the GitHub releases. The new package seems to be owned by an unnamed individual, but wimpunk seems to lurk and provide advice and peer review. I would be under the impression that this is a legitimate repository.

@syncer I'm not sure what is considered acceptable or not during the build process as this would be my first time contributing. Some pointers would be appreciated. 🙏 Would fetching and installing the forked GitHub release be acceptable? Provided it tests properly.

The alternative is for me to get involved with the ddclient repository, and try and push a new package and new maintainers to the official Debian repository... which might be quite the endeavour.

As for the libdata-validate-ip-perl dependant package, this I know where to make the changes.

@dmbaturin bump? I'd like to submit a PR for this issue, but would like some guidance regarded to my comment above.

c-po added a subscriber: c-po.EditedOct 20 2019, 7:30 AM

Debian Buster uses 3.8.3 https://packages.debian.org/buster/ddclient which could be considered as using a proper source tree.

@avanier could you please try building this for VyOS 1.2 and see if it fixes your issue? There is already a fork for VyOS 1.2 of ddclient (https://github.com/vyos/ddclient) so we could merge in the changes if you confirm that ddclient 3.8.3 works.

According to https://sourceforge.net/p/ddclient/news/2015/05/ddclient-383-released/

Yet again it's been a while but here is new release of ddclient. As usual, there are some important changes and some documentation is modified. A detailed overview can be found in ChangeLog but here's a quick overview:
added Alpine Linux init scritp - patch send by @Tal on github.
adding support for nsupdate - patch send by @droe on github
allow log username-password combinations - patch send by @dirdi on github
adding support for cloudflare - patch send by @roberthawdon on github
adding support for duckdns - patch send by @gkranis
A very big thank you for everyone who created a pull request on github and for everyone who helped to fix the little issues caused by the new providers.

c-po added a comment.Oct 20 2019, 7:38 AM

Even better, looks like wimpunk is maintaining again and there is a fresh release https://github.com/ddclient/ddclient/releases 3.9.0

c-po added a comment.EditedOct 20 2019, 7:41 AM
vyos@vyos:~$ wget http://ftp.de.debian.org/debian/pool/main/d/ddclient/ddclient_3.8.3-1.1_all.deb
Connecting to ftp.de.debian.org (141.76.2.4:80)
ddclient_3.8.3-1.1_a 100% |********************************************************************************************************************************************************************************************| 81924   0:00:00 ETA
vyos@vyos:~$ sudo dpkg -i ddclient_3.8.3-1.1_all.deb
(Reading database ... 59343 files and directories currently installed.)
Preparing to unpack ddclient_3.8.3-1.1_all.deb ...
Unpacking ddclient (3.8.3-1.1) over (3.8.2+vyos2+current1) ...
Setting up ddclient (3.8.3-1.1) ...

Configuration file '/etc/dhcp/dhclient-exit-hooks.d/ddclient'
 ==> Deleted (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** ddclient (Y/I/N/O/D/Z) [default=N] ? d

Configuration file '/etc/dhcp/dhclient-exit-hooks.d/ddclient'
 ==> Deleted (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** ddclient (Y/I/N/O/D/Z) [default=N] ? y
Installing new version of config file /etc/dhcp/dhclient-exit-hooks.d/ddclient ...
insserv: warning: current start runlevel(s) (empty) of script `ddclient' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (0 1 2 3 4 5 6) of script `ddclient' overrides LSB defaults (0 1 6).
Processing triggers for systemd (215-17+deb8u13) ...

Please update your local installation to ddclient 3.8.2 and check if it works as intended on your side. Once this is done we could pick that package to the VyOS installation.

hi @c-po , I've tried installing the 3.8.3 version but it's not compatible with the new CloudFlare API v4. The only way to make ddclient working with cloudflare is using the 3.9.0.

c-po added a comment.Oct 20 2019, 10:47 AM

@pvelati thanks for the update!

c-po added a comment.Oct 20 2019, 10:58 AM

Can you please test installing this package and feedback to me?

https://helix.mybll.net/ddclient_3.9.0+vyos2+current1_all.deb

It also has a dependency on libdata-validate-ip-perl

c-po changed the task status from Open to Backport candidate.Oct 20 2019, 11:01 AM
c-po claimed this task.
c-po set Is it a breaking change? to Perfectly compatible.
c-po added a comment.Oct 21 2019, 2:08 AM

Looks good first glance to me. Next rolling will have it included.

https://github.com/vyos/ddclient/commit/1d8514d45ba6e6bbe1d6e09fdd356a006d61c5bf

c-po renamed this task from dynamic dns cloudflare protocol deprecated to Upgrade ddclient from 3.8.2 to 3.9.0 (support Cloudflare API v4).Oct 21 2019, 2:09 AM
c-po changed Difficulty level from Unknown (require assessment) to Easy (less than an hour).
c-po changed Why the issue appeared? from Will be filled on close to Other.

@c-po I tried your 3.9.0 deb package. I installed with some dependencies (libdata-validate-ip-perl libnet-ipv6addr-perl libnet-netmask-perl libnetwork-ipv4addr-perl).
After that, I modified some config parameters in the dns dynamic section to trigger the update but it doesn't work oob.

I tried execute "ddclient" as root and I receive this error:

WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.
WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.

I created the directory in /etc/ddclient/ and moved the ddclient.conf inside that

root@vyos:~# mv /etc/ddclient.conf /etc/ddclient/.
root@vyos:~# ddclient 
WARNING:  skipping host: test.example.com: 'zone=' is an invalid fully qualified host name.

If the new rolling iso has your commit, I can try update that. Now I'm using VyOS 1.2-rolling-201910160117

c-po added a comment.EditedMon, Oct 21, 10:42 PM

Thanks for feeding back. Please try the latest rolling ISO if it works for you.

Maybe additional adjustments to the generated configuration file is required for the new version of ddclient to work?

I've tried the VyOS 1.2-rolling-201910220117 with integrated ddclient 3.9.0 and this conf:

vyos@vyos# show service dns dynamic 
 interface pppoe0 {
     service cloudflare {
         host-name test.mydomain.com
         login myemail@gmail.com
         password 000000mycfapikey00000000
     }
 }

but it doesn't work.

Also, I have those problems:

vyos@vyos:~$ sh dns dynamic status 
Traceback (most recent call last):
  File "/usr/libexec/vyos/op_mode/dynamic_dns.py", line 106, in <module>
    main()
  File "/usr/libexec/vyos/op_mode/dynamic_dns.py", line 100, in main
    show_status()
  File "/usr/libexec/vyos/op_mode/dynamic_dns.py", line 48, in show_status
    with open(cache_file, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/var/cache/ddclient/ddclient.cache'
root@vyos:~# ddclient 
WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.
WARNING:  file /etc/ddclient/ddclient.conf: Cannot open file '/etc/ddclient/ddclient.conf'. (No such file or directory)
stat() on closed filehandle FD at /usr/sbin/ddclient line 1117.
Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1118.
readline() on closed filehandle FD at /usr/sbin/ddclient line 1130.

so, maybe for the new configuration you need to save the file in /etc/ddclient/ddclient.conf and not in /etc/ddclient.conf

I've also created the folder /etc/ddclient and copied the ddclient.conf there. But I have this error:

root@vyos:~# ddclient 
WARNING:  skipping host: test.mydomain.com: 'zone=' is an invalid fully qualified host name.
pvelati added a comment.EditedTue, Oct 22, 9:45 PM

ok, I've found a way to make it working.

  • mkdir /var/run/ddclient
  • touch /var/run/ddclient/ddclient.pid
  • edit /etc/ddclient/ddclient.conf:
### Autogenerated by dynamic_dns.py ###
daemon=1m
syslog=yes
ssl=yes
pid=/var/run/ddclient/ddclient.pid
cache=/var/cache/ddclient/ddclient.cache

#
# ddclient configuration for interface "pppoe0":
#
use=if, if=pppoe0

# DynDNS provider configuration for test.mydomain.com
protocol=cloudflare,
ttl=1,
zone=mydomain.com,
login=myemail@gmail.com,
password='000000mycfapikey000000',
mydomain.com,test.mydomain.com

(you need the ',' at the end of the line)

c-po added a comment.Wed, Oct 23, 3:48 AM

Added your reccomended changes and auto create directories if they are non existent. Can you please check again?

I tried the new rolling iso VyOS 1.2-rolling-201910230349 but it doesn't work oob.
I see that you tweaked a bit the ddclient.conf file, but you need to add "zone=" in the conf in order to make it works.

# DynDNS provider configuration for test.mydomain.com
protocol=cloudflare,
ttl=1,
zone=mydomain.com,       <------------------------------------------
login=myemail@gmail.com,
password='000000mycfapikey000000',
test.mydomain.com

I don't know if you can add a "zone" field in the vyos conf

vyos@vyos# show service dns dynamic 
 interface pppoe0 {
     service cloudflare {
         host-name test.mydomain.com
         zone mydomain.com <----------------------------------------------------------
         login myemail@gmail.com
         password 000000mycfapikey00000000
     }
 }
c-po added a comment.Wed, Oct 23, 11:06 AM

@pvelati looks like that the zone statement is required for Cloudflare only. I did not find a documentation how it should look like. is it only and always the domain part of your dynamic dns entry? If thats the case it can be automatically rendered into the configuration.

In T1030#46018, @c-po wrote:

@pvelati looks like that the zone statement is required for Cloudflare only. I did not find a documentation how it should look like. is it only and always the domain part of your dynamic dns entry? If thats the case it can be automatically rendered into the configuration.

ok, so if it's only for cloudflare you can render the zone removing the first part of name. so for example:

test.mydomain.com ---> zone will be mydomain.com

so in this way you can use also different multiple tld, like "example.co.at", "example.avocat.fr".
The only doubt I have is that if a user has to register a 3rd level domain, I don't know if the area can be generated correctly. But this is probably an extreme case

c-po added a comment.Wed, Oct 23, 9:24 PM

You could check the latest rolling which have it

Just installed the latest rolling.
I tried the old conf, delete them, and reconfigure dns dynamic.
thanks @c-po , it's working fine now

c-po added a comment.Wed, Oct 23, 11:49 PM

Thanks for the feedback - will backport to our LTS branch later on!

c-po moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus board.Sun, Oct 27, 2:57 AM

The ddclient config file got moved to /etc/ddclient/ddclient.conf but ddclient is still trying to load /etc/ddclient.conf in the latest VyOS 1.3 rolling image.

c-po added a comment.Thu, Oct 31, 1:38 AM

Merged to equuleus branch.

Upgraded to 1.3-rolling-201911041446 and the config file issue is still present.

vyos@104-1831-gw# show service dns dynamic
 interface eth1 {
     service cloudflare {
         host-name 104-1831.tdude.co
         host-name 104-1831-gw.104-1831.tdude.co
         login starcraft66@gmail.com
         password snipped
         protocol cloudflare
     }
 }
[edit]
vyos@104-1831-gw# sudo journalctl -u ddclient
-- Logs begin at Mon 2019-11-04 20:47:43 EST, end at Mon 2019-11-04 20:52:55 EST. --
Nov 04 20:50:37 104-1831-gw systemd[1]: Starting LSB: Update dynamic domain name service entries...
Nov 04 20:50:37 104-1831-gw ddclient[2811]: WARNING:  file /etc/ddclient.conf: Cannot open file '/etc/ddclient.conf'. (No such file or directory)
Nov 04 20:50:37 104-1831-gw ddclient[2811]: stat() on closed filehandle FD at /usr/sbin/ddclient line 1087.
Nov 04 20:50:37 104-1831-gw ddclient[2811]: Use of uninitialized value $mode in bitwise and (&) at /usr/sbin/ddclient line 1088.
Nov 04 20:50:37 104-1831-gw ddclient[2811]: readline() on closed filehandle FD at /usr/sbin/ddclient line 1100.
Nov 04 20:50:37 104-1831-gw systemd[1]: Started LSB: Update dynamic domain name service entries.
Nov 04 20:50:37 104-1831-gw ddclient[2821]: WARNING:  file /etc/ddclient.conf: Cannot open file '/etc/ddclient.conf'. (No such file or directory)
c-po added a comment.Tue, Nov 5, 9:13 AM

Ah you were using bleeding edge equuleus builds. Ddclient 3.9.0 has been added to recent rolling image.

c-po added a comment.Sat, Nov 9, 2:18 PM

Backport to crux complete

c-po closed this task as Resolved.Sat, Nov 9, 2:18 PM
c-po moved this task from Needs Triage to Finished on the VyOS 1.2 Crux (VyOS 1.2.4) board.