Page MenuHomePhabricator

OpenVPN Server Fails to Start
Closed, ResolvedPublicBUG

Description

On the most recent rolling release, OpenVPN appears to be broken on boot and any time a commit is attempted, but the same exact configuration can be used with 1.2-RC11 without any issues. I've verified the problem is present on:

  • 1.2.0-rolling+201902130337
  • 1.2.0-rolling+201902210337

Here is the relevant (sanitized) portion of my configuration (note: I did also try removing the openvpn-option portions to no avail):

vyos@cr03-vyos# show
+description "Someone's VPN"
+encryption aes256
+hash sha256
+keep-alive {
+    failure-count 3
+    interval 10
+}
+local-host 1.2.3.4
+mode server
+openvpn-option "--tun-mtu 1472"
+openvpn-option "--sndbuf 0"
+openvpn-option "--rcvbuf 0"
+protocol tcp-passive
+server {
+    name-server 1.1.1.1
+    name-server 8.8.8.8
+    name-server 1.0.0.1
+    push-route 192.168.55.0/24
+    subnet 192.168.55.0/24
+}
+tls {
+    ca-cert-file /config/auth/openvpn/test/pki/ca.crt
+    cert-file /config/auth/openvpn/test/pki/issued/someonesvpn.crt
+    dh-file /config/auth/openvpn/test/pki/dh.pem
+    key-file /config/auth/openvpn/test/pki/private/someonesvpn.key
+}
[edit interfaces openvpn vtun0]

Here's what happens when I try to commit:

# Commit attempt
vyos@cr03-vyos# commit
[ interfaces openvpn vtun0 ]
OpenVPN configuration error: Failed to start OpenVPN tunnel.

[[interfaces openvpn vtun0]] failed
[ interfaces openvpn vtun1 ]
OpenVPN configuration error: Failed to start OpenVPN tunnel.

[[interfaces openvpn vtun1]] failed
Commit failed
[edit interfaces openvpn vtun0]

Here's the log:

# Show log
Feb 21 00:40:44 cr03-vyos.bhs05.trae32566.org openvpn-vtun1[5968]: Options error: Unrecognized option or missing or extra parameter(s) in [CMD-LINE]:1: push (2.4.0)
Feb 21 00:40:44 cr03-vyos.bhs05.trae32566.org openvpn-vtun1[5968]: Use --help for more information.
Feb 21 00:40:44 cr03-vyos.bhs05.trae32566.org sudo[5954]: pam_unix(sudo:session): session closed for user root
Feb 21 00:41:07 cr03-vyos.bhs05.trae32566.org sudo[6052]: vyos : TTY=pts/0 ; PWD=/home/vyos ; USER=root ; COMMAND=/bin/journalctl
Feb 21 00:41:07 cr03-vyos.bhs05.trae32566.org sudo[6052]: pam_unix(sudo:session): session opened for user root by vyos(uid=0)`

Details

Difficulty level
Normal (likely a few hours)
Version
1.2.0-rolling+201902210337
Why the issue appeared?
Will be filled on close

Event Timeline

varesa added a subscriber: varesa.Feb 21 2019, 9:00 PM

Also managed to reproduce, some set-commands to help reproduction:

top
edit interfaces openvpn vtun0
set encryption aes256
set hash sha256
set keep-alive failure-count 3
set keep-alive interval 10
set local-host 172.31.11.20
set mode server
set protocol tcp-passive
set server name-server 1.1.1.1
set server push-route 192.168.55.0/24
set server subnet 10.0.0.0/24
set tls ca-cert-file /config/auth/ca.crt
set tls cert-file /config/auth/key.crt
set tls dh-file /config/auth/dh.pem
set tls key-file /config/auth/key.key

Produced command seems to be:

vyos@VyOS-AMI# perl
use strict;  
use lib "/opt/vyatta/share/perl5";           
use Vyatta::OpenVPN::Config;
                                             
my $config = new Vyatta::OpenVPN::Config;
                                               
$config->setup("vtun0");   
my ($cmd, $err) = $config->get_command();           
print STDOUT "$cmd\n";

^D
/usr/sbin/openvpn --daemon openvpn-vtun0 --verb 3 --writepid /var/run/openvpn-vtun0.pid --status /opt/vyatta/etc/openvpn/status/vtun0.status 30 --dev-type tun --dev vtun0 --mode server --tls-server --topology subnet --keepalive 10 30 --local 172.31.11.20 --proto tcp-server --cipher aes-256-cbc --auth sha256 --ca /config/auth/ca.crt --cert /config/auth/key.crt --key /config/auth/key.key --dh /config/auth/dh.pem --management /tmp/openvpn-mgmt-intf unix --push dhcp-option DNS 1.1.1.1 --push route 192.168.55.0 255.255.255.0 --server 10.0.0.0 255.255.255.0 --client-config-dir /opt/vyatta/etc/openvpn/ccd/vtun0

Hopefully a fix: https://github.com/varesa/vyatta-openvpn/commit/a0d7c07f1ff0b5fe7450d3a13c1365b8e3589725

After doing: sudo curl https://raw.githubusercontent.com/varesa/vyatta-openvpn/a0d7c07f1ff0b5fe7450d3a13c1365b8e3589725/lib/Vyatta/OpenVPN/Config.pm -o /opt/vyatta/share/perl5/Vyatta/OpenVPN/Config.pm commit with my above config succeeds.

In T1259#33041, @varesa wrote:

Hopefully a fix: https://github.com/varesa/vyatta-openvpn/commit/a0d7c07f1ff0b5fe7450d3a13c1365b8e3589725
After doing: sudo curl https://raw.githubusercontent.com/varesa/vyatta-openvpn/a0d7c07f1ff0b5fe7450d3a13c1365b8e3589725/lib/Vyatta/OpenVPN/Config.pm -o /opt/vyatta/share/perl5/Vyatta/OpenVPN/Config.pm commit with my above config succeeds.

Applied this commit and it works perfect! Thanks!

hagbard claimed this task.Feb 21 2019, 10:56 PM

just compared 1.20-rc7, 1.2.0 official & the current rolling:

1.2.0-RC7
vyos@vyos:~$ show system image
The system currently has the following image(s) installed:

1: 1.2.0-rolling+201902210337
2: 1.2.0-rc7 (default boot)

vyos@vyos:~$ dpkg -l |grep openvpn
ii openvpn 2.3.4-5+deb8u2 amd64 virtual private network daemon
ii openvpn-auth-ldap 2.0.3-6.1 amd64 OpenVPN LDAP authentication module
ii openvpn-auth-radius 2.1-6+b1 amd64 OpenVPN RADIUS authentication module

ii vyatta-openvpn 0.2.60+vyos3+current2 all VyOS OpenVPN configuration/operational commands

1.2.0 *OFFICIAL*

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

100 361M 100 361M 0 0 9558k 0 0:00:38 0:00:38 --:--:-- 12.4M
ISO download succeeded.
Checking for digital signature file...

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed

100 183 100 183 0 0 1484 0 --:--:-- --:--:-- --:--:-- 1487
100 836 100 836 0 0 837 0 --:--:-- --:--:-- --:--:-- 0
Found it. Checking digital signature...
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: assuming signed data in `/var/tmp/install-image.2597/vyos-1.2.0-amd64.iso'
gpg: Signature made Mon 28 Jan 2019 10:22:27 PM UTC using RSA key ID A0FE6D7E
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: Good signature from "VyOS Maintainers (VyOS Release) <maintainers@vyos.net>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 0694 A923 0F51 39BF 834B A458 FD22 0285 A0FE 6D7E
Digital signature is valid.
Checking MD5 checksums of files on the ISO image...OK.
Done!
What would you like to name this image? [1.2.0]: OK. This image will be named: 1.2.0
Installing "1.2.0" image.
Copying new release files...
Would you like to save the current configuration
directory and config file? (Yes/No) [Yes]:
Copying current configuration...
Would you like to save the SSH host keys from your
current configuration? (Yes/No) [Yes]:
Copying SSH keys...
Running post-install script...
Setting up grub configuration...
vyos@vyos:~$ show system image
The system currently has the following image(s) installed:

1: 1.2.0 (default boot) (running image)
2: 1.2.0-rolling+201902210337
3: 1.2.0-rc7

vyos@vyos:~$ dpkg -l |grep openvpn
ii openvpn 2.3.4-5+deb8u2 amd64 virtual private network daemon
ii openvpn-auth-ldap 2.0.3-6.1 amd64 OpenVPN LDAP authentication module
ii openvpn-auth-radius 2.1-6+b1 amd64 OpenVPN RADIUS authentication module

ii vyatta-openvpn 1.2.0-1 all VyOS OpenVPN configuration/operational commands

1.2.0-rolling
vyos@vyos:~$ show system image
The system currently has the following image(s) installed:

1: 1.2.0-rolling+201902210337 (default boot)

vyos@vyos:~$ dpkg -l |grep openvpn
ii openvpn 2.4.0-6+deb9u1~bpo8+1 amd64 virtual private network daemon
ii openvpn-auth-ldap 2.0.3-6.1 amd64 OpenVPN LDAP authentication module
ii openvpn-auth-radius 2.1-6+b1 amd64 OpenVPN RADIUS authentication module
ii vyatta-openvpn 0.2.60+vyos3+current2 all VyOS OpenVPN configuration/operational commands

Clear that the current rolling has newer openvpn version

hagbard changed the task status from Open to Needs testing.Feb 21 2019, 11:34 PM

https://github.com/vyos/vyatta-openvpn/commit/9166dde7fd5ca7b313de585067b06af6a8b9c82a Should be in the next latest rolling, can you please test?

pasik added a subscriber: pasik.Feb 22 2019, 2:29 PM
hagbard triaged this task as Normal priority.Feb 22 2019, 5:40 PM
hagbard changed Difficulty level from Unknown (require assessment) to Normal (likely a few hours).

Just tested the next day nightly build.

vyos@vyos:~$ show system image
The system currently has the following image(s) installed:

1: 1.2.0-rolling+201902220337 (default boot)

Not in it (yet)

in /opt/vyatta/share/perl5/Vyatta/OpenVPN/Config.pm

line 845: $cmd .= " --push dhcp-option DNS $nserver";

trae32566 added a comment.EditedFeb 23 2019, 2:58 AM

Just tested the most recent nightly (1.2.0-rolling+201902222123) and it works! Thank you so much y'all.

hagbard closed this task as Resolved.Feb 23 2019, 3:51 PM

In T1246 I noticed that while my fix makes the syntax correct enough that OpenVPN starts, it is actually missing the $variables inside the quotes as the patch changed double quotes to single quotes.
That means that instead of --push "dhcp-option DNS 1.1.1.1" or --push "route 192.168.0.0 255.255.255.0" it started with --push "dhcp-option " and --push "route "

I've made a second PR to correct this: https://github.com/vyos/vyatta-openvpn/pull/8

This time verified more carefully with ps aux | grep openvpn to see that the parameters actually include the variables and also with cat /proc/$(pgrep openvpn)/cmdline | tr '\0' '\n' in order to check the correct quoting