Page MenuHomePhabricator

Blank hostnames from dhcpd are able to bring down DNS
Closed, ResolvedPublic

Description

We have a development box which we *think* is occasionally sending a blank hostname when it does a DHCP request.

It is getting given a lease fine by dhcpd, but /etc/hosts is being populated with the hostname as .domain-name.example.com

This causes pdns-recursor to:

  1. eat all the ram, getting killed by the OOM killer, then:
  2. fail to restart

Assuming its is just falling through the three options given as they are all empty, can we modify https://github.com/vyos/vyos-1x/blob/current/src/conf_mode/dhcp_server.py#L214 to pass an alphanumeric version of the host mac address as a fallback if the other three fail?

If this isn't what is happening, maybe an explicit check that $client_name is set in https://github.com/vyos/vyos-1x/blob/current/src/system/on-dhcp-event.sh#L35 would work as a backstop to prevent these broken entries getting into /etc/hosts

Details

Difficulty level
Unknown (require assessment)
Version
1.2.0-rc11
Why the issue appeared?
Will be filled on close

Event Timeline

thinkl33t created this object in space S1 VyOS Public.
thinkl33t updated the task description. (Show Details)
syncer assigned this task to c-po.Jan 28 2019, 4:49 PM
syncer triaged this task as High priority.
syncer edited projects, added VyOS 1.3 Equuleus; removed VyOS 1.2 Crux.

Note that this has taken down DNS on our firewall (and hence our network) a couple of times now, both annoyingly early in the morning when nobody was in to fix it.

c-po added a comment.Jan 29 2019, 7:05 AM

can you share your DHCP configuration with us for reproducibility?

In addition can you disable the dhcp hostfile-update by removing it from your cofiguration?

pasik added a subscriber: pasik.Feb 4 2019, 7:33 PM
> show service dhcp-server 
hostfile-update
shared-network-name VLAN101 
  subnet 172.16.101.0/24 {
    default-router 172.16.101.254
    dns-server 172.16.101.254
    domain-name guest.example.org
    lease 3600
    range 0 {
        start 172.16.101.1
        stop 172.16.101.250
    }
  }
}

We can't really disable hostfile-update, as we heavily use internal DNS records for development hardware.

c-po added a comment.Feb 12 2019, 2:06 PM

Maybe if hostname is empty, we can prepend the mac address to the fqdn which will be stored in /etc/hosts

c-po added a comment.Feb 17 2019, 11:57 AM

@thinkl33t can you please also provide a "faulty" /etc/hosts file?

The line giving the error is:

172.16.101.192 .guest.example.org #on-dhcp-event b4:e6:2a:54:a:ef

pdns-recursor will initially continue to run fine, but anything that forces it to restart or reload will cause it to fail to start.

c-po added a comment.Feb 18 2019, 1:47 PM

can you edit /usr/libexec/vyos/system/on-dhcp-event.sh

and add this after changes=0

if [ -z "$client_name" ]; then
    logger -s -t on-dhcp-event "Client name was empty, using MAC \"$client_mac\" instead"
    client_name=$(echo "client-"$client_mac | tr : -)
fi

Brill, i've applied that patch and will keep an eye on it for a few days to see what happens.

Which log should i be monitoring to see the log output from that script?

c-po added a comment.Feb 18 2019, 2:06 PM

/var/log/messages

c-po added a comment.Feb 21 2019, 6:15 PM

Added the above snipped to vyos-1x and the latest rolling releases

c-po changed the task status from Open to In progress.Feb 21 2019, 6:15 PM
c-po moved this task from Need Triage to In Progress on the VyOS 1.3 Equuleus board.

We've not had this issue since i applied the patch.

I'm not sure if that is because the patch fixed it, or it wouldn't have
triggered anyway, but i can't see anything in the logs.

c-po added a comment.Feb 25 2019, 5:18 PM

okay - then I kindly close this one.

Please reopen if it appears again.

c-po closed this task as Resolved.Feb 25 2019, 5:18 PM
c-po reopened this task as Open.Feb 25 2019, 5:21 PM
c-po closed this task as Resolved.
c-po moved this task from Needs Triage to Finished on the VyOS 1.2 Crux (VyOS 1.2.1) board.