Page MenuHomePhabricator

L2TPv3 interface gets not loaded after reboot with a configuration error
Open, NormalPublicBUG

Description

We have the following configuration with seems to be correct and runs on VyOS 1.2.0-RC3 / RC4
(Its running on HW Protectli FW6A)

interfaces {
  bridge br0 {
      aging 300
      description "L2TPV3 Bridge Grp Mgt"
      hello-time 2
      max-age 20
      priority 32768
      stp false
  }
  ethernet eth0 {
      address 10.52.193.14/29
      duplex auto
      hw-id 00:e0:67:0a:6b:6c
      mtu 9000
      smp-affinity auto
      speed auto
  }
  ethernet eth1 {
      address dhcp
      duplex auto
      hw-id 00:e0:67:0a:6b:6d
      smp-affinity auto
      speed auto
  }
  ethernet eth2 {
      duplex auto
      hw-id 00:e0:67:0a:6b:6e
      smp-affinity auto
      speed auto
  }
  ethernet eth3 {
      duplex auto
      hw-id 00:e0:67:0a:6b:6f
      smp-affinity auto
      speed auto
  }
  ethernet eth4 {
      duplex auto
      hw-id 00:e0:67:0a:6b:70
      smp-affinity auto
      speed auto
  }
  ethernet eth5 {
      bridge-group {
          bridge br0
      }
      description "L2 OPT4 Grp Mgt"
      duplex auto
      hw-id 00:e0:67:0a:6b:71
      smp-affinity auto
      speed auto
  }
  l2tpv3 l2tpeth0 {
      bridge-group {
          bridge br0
      }
      description "L2 Tunnel Grp Mgt"
      destination-port 5000
      encapsulation ip
      local-ip 10.52.193.14
      mtu 8958
      peer-session-id 110
      peer-tunnel-id 10
      remote-ip 10.52.192.174
      session-id 110
      source-port 5000
      tunnel-id 10
  }
  loopback lo {
  }
}
protocols {
  static {
      route 0.0.0.0/0 {
          next-hop 10.52.193.9 {
          }
      }
  }
}
service {
  ssh {
  }
}
system {
  config-management {
      commit-revisions 100
  }
  console {
      device ttyS0 {
          speed 9600
      }
  }
  host-name BRF-R309-RT02
  login {
      user vyos {
          authentication {
              encrypted-password $6$LPWjXZ.50b/LcPZz$oEnBDH8oWD0Y0FQ1C7E1SHfjBRzkf1bOpuwC4XfSWJL16i6JWiL/aJ/2NmgMsZMsrENsXRGhhe7vrSQJEaKlG0
              plaintext-password ""
          }
          level admin
      }
  }
  ntp {
      server 0.pool.ntp.org {
      }
      server 1.pool.ntp.org {
      }
      server 2.pool.ntp.org {
      }
  }
  syslog {
      global {
          facility all {
              level info
          }
          facility protocols {
              level debug
          }
      }
  }
  time-zone Europe/Zurich
}


/* Warning: Do not remove the following line. */
/* === vyatta-config-version: "broadcast-relay@1:cluster@1:config-management@1:conntrack-  sync@1:conntrack@1:dhcp-relay@1:dhcp-server@5:firewall@5:ipsec@4:mdns@1:nat@4:qos@1:quagga@3:system@9:vrrp@2:wanloadbalance@3:webgui@1:webproxy@1:zone-policy@1" === */
/* Release version: 1.2.0-rc3 */

After a reboot the configuration is not loaded completely. The following part is missing and can be loaded with a commit:

  vyos@BRF-R309-RT02:~$ configure
  [edit]
  vyos@BRF-R309-RT02# load
  Loading configuration from '/config/config.boot'...

  Load complete.  Use 'commit' to make changes active.
  [edit]
  vyos@BRF-R309-RT02# compare
  [edit interfaces]
  +l2tpv3 l2tpeth0 {
  +    bridge-group {
  +        bridge br0
  +    }
  +    description "L2 Tunnel Grp Mgt"
  +    destination-port 5000
  +    encapsulation ip
  +    local-ip 10.52.193.14
  +    mtu 8958
  +    peer-session-id 110
  +    peer-tunnel-id 10
  +    remote-ip 10.52.192.174
  +    session-id 110
  +    source-port 5000
  +    tunnel-id 10
  +}
   [edit]

  vyos@BRF-R309-RT02# commit
  Warning: priority inversion [interfaces l2tpv3 l2tpeth0 mtu](461) <= [interfaces l2tpv3 l2tpeth0](800)
         changing [interfaces l2tpv3 l2tpeth0 mtu] to (801)
  [ interfaces l2tpv3 l2tpeth0 bridge-group ]
  Adding interface l2tpeth0 to bridge br0
 
  [edit]
  vyos@BRF-R309-RT02#


  vyos@BRF-R309-RT02:~$ show interfaces
  Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
  Interface        IP Address                        S/L  Description
  ---------        ----------                        ---  -----------
  br0              -                                 u/u  L2TPV3 Bridge Grp Mgt
  eth0             10.52.193.14/29                   u/u
  eth1             172.17.0.100/24                   u/u
  eth2             -                                 u/D
  eth3             -                                 u/D
  eth4             -                                 u/D
  eth5             -                                 u/u  L2 OPT4 Grp Mgt
  l2tpeth0         -                                 u/u  L2 Tunnel Grp Mgt
  lo               127.0.0.1/8                       u/u
                 ::1/128

				 
  vyos@BRF-R309-RT02:~$ show interfaces l2tpv3 l2tpeth0
  l2tpeth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8958 qdisc pfifo_fast master br0 state UNKNOWN group default qlen 1000
    link/ether 0a:a1:c0:12:18:a6 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8a1:c0ff:fe12:18a6/64 scope link
       valid_lft forever preferred_lft forever
    Description: L2 Tunnel Grp Mgt

    RX:  bytes    packets     errors    dropped    overrun      mcast
         62606         44          0          0          0          0
    TX:  bytes    packets     errors    dropped    carrier collisions
         63000         51          0          0          0          0
  vyos@BRF-R309-RT02:~$

Are we have an error in the configuration, are we doing something wrong? It looks like it has something to do with the MTU Settings on the ethernet interfaces and the l2tp interface.

Regards Mätthi

Details

Difficulty level
Unknown (require assessment)
Version
VyOS-1.2.0-RC3, VyOS-1.2.0-RC4
Why the issue appeared?
Will be filled on close
Maetthi created this task.Oct 27 2018, 9:31 AM
syncer triaged this task as Normal priority.

We did some tests with RC5. Sometimes the complet configuration gets loaded after a reboot, but most of the time not. Maybe a timing issue?

I have tested, the reason may be that your interface and the switch have not been UP. The route leading to the default route or L2TPV3 PEER address does not take effect. The ip l2tp command does not work.
This can be configured without any network cable. To configure L2TPV3, you will find that it is unable to commit.

My current method is:
    begin:

[ -d /sys/module/l2tp_eth ] || sudo modprobe l2tp_eth
[ -d /sys/module/l2tp_netlink ] || sudo modprobe l2tp_netlink
if [ "$VAR(./encapsulation/@)" = "ip" ]; then
  if [ ! -d /sys/module/l2tp_ip ]; then
    sudo modprobe l2tp_ip
  fi

  if [ ! -d /sys/module/l2tp_ip6 ]; then
    sudo modprobe l2tp_ip6
  fi
fi

if [ ! -d /sys/class/net/dum-l2tpv3 ]; then
  [ -d /sys/module/dummy ] || sudo modprobe dummy
  ip link add name dum-l2tpv3 type dummy
  ip link set dum-l2tpv3 up
  vtysh -c "configure terminal" \
            -c "ip route 0.0.0.0/0 dum-l2tpv3 254";
fi
Maetthi added a comment.EditedMon, Dec 10, 1:47 PM

We did some testing with the RC10 Version as requested by the blog.

The issue is still there. After a reboot the settings for the l2tpv3 interfaces are not loaded and we have a difference between the running and the saved configuration. The l2tpv3 interface is bound to eth0 with has a static configuration.

Just for a test, we changed our management interfaces (eth1) from dhcp to static. This means we also changed the default static route setting from dchp to a static address.

In this case, the whole configuration got loaded. We testet that 3 times, no failer. As soon as we set the management interface back to dhcp and set also the default routing back to use the default route provided by dhcp, the issue was here again.

I can provide some logfiles, if needed. Btw. Is there something like a debug mode to get more information what is happening during boot and during applying the stored configuration?

commo added a subscriber: commo.Mon, Dec 10, 5:40 PM

I suspect this is the same issue as T1080. if the destination ip of the l2tpv3 tunnel is unreachable(no default route) it wont load. I suspect its loading the l2tpv3 config before DHCP has installed a default route. My workaround is to install a very low metric static default route, that will get overwritten by OSPF(in my case).