Page MenuHomePhabricator

L2TPv3 interface gets not loaded after reboot with a configuration error
Open, NormalPublicBUG


We have the following configuration with seems to be correct and runs on VyOS 1.2.0-RC3 / RC4
(Its running on HW Protectli FW6A)

interfaces {
  bridge br0 {
      aging 300
      description "L2TPV3 Bridge Grp Mgt"
      hello-time 2
      max-age 20
      priority 32768
      stp false
  ethernet eth0 {
      duplex auto
      hw-id 00:e0:67:0a:6b:6c
      mtu 9000
      smp-affinity auto
      speed auto
  ethernet eth1 {
      address dhcp
      duplex auto
      hw-id 00:e0:67:0a:6b:6d
      smp-affinity auto
      speed auto
  ethernet eth2 {
      duplex auto
      hw-id 00:e0:67:0a:6b:6e
      smp-affinity auto
      speed auto
  ethernet eth3 {
      duplex auto
      hw-id 00:e0:67:0a:6b:6f
      smp-affinity auto
      speed auto
  ethernet eth4 {
      duplex auto
      hw-id 00:e0:67:0a:6b:70
      smp-affinity auto
      speed auto
  ethernet eth5 {
      bridge-group {
          bridge br0
      description "L2 OPT4 Grp Mgt"
      duplex auto
      hw-id 00:e0:67:0a:6b:71
      smp-affinity auto
      speed auto
  l2tpv3 l2tpeth0 {
      bridge-group {
          bridge br0
      description "L2 Tunnel Grp Mgt"
      destination-port 5000
      encapsulation ip
      mtu 8958
      peer-session-id 110
      peer-tunnel-id 10
      session-id 110
      source-port 5000
      tunnel-id 10
  loopback lo {
protocols {
  static {
      route {
          next-hop {
service {
  ssh {
system {
  config-management {
      commit-revisions 100
  console {
      device ttyS0 {
          speed 9600
  host-name BRF-R309-RT02
  login {
      user vyos {
          authentication {
              encrypted-password $6$LPWjXZ.50b/LcPZz$oEnBDH8oWD0Y0FQ1C7E1SHfjBRzkf1bOpuwC4XfSWJL16i6JWiL/aJ/2NmgMsZMsrENsXRGhhe7vrSQJEaKlG0
              plaintext-password ""
          level admin
  ntp {
      server {
      server {
      server {
  syslog {
      global {
          facility all {
              level info
          facility protocols {
              level debug
  time-zone Europe/Zurich

/* Warning: Do not remove the following line. */
/* === vyatta-config-version: "broadcast-relay@1:cluster@1:config-management@1:conntrack-  sync@1:conntrack@1:dhcp-relay@1:dhcp-server@5:firewall@5:ipsec@4:mdns@1:nat@4:qos@1:quagga@3:system@9:vrrp@2:wanloadbalance@3:webgui@1:webproxy@1:zone-policy@1" === */
/* Release version: 1.2.0-rc3 */

After a reboot the configuration is not loaded completely. The following part is missing and can be loaded with a commit:

  vyos@BRF-R309-RT02:~$ configure
  vyos@BRF-R309-RT02# load
  Loading configuration from '/config/config.boot'...

  Load complete.  Use 'commit' to make changes active.
  vyos@BRF-R309-RT02# compare
  [edit interfaces]
  +l2tpv3 l2tpeth0 {
  +    bridge-group {
  +        bridge br0
  +    }
  +    description "L2 Tunnel Grp Mgt"
  +    destination-port 5000
  +    encapsulation ip
  +    local-ip
  +    mtu 8958
  +    peer-session-id 110
  +    peer-tunnel-id 10
  +    remote-ip
  +    session-id 110
  +    source-port 5000
  +    tunnel-id 10

  vyos@BRF-R309-RT02# commit
  Warning: priority inversion [interfaces l2tpv3 l2tpeth0 mtu](461) <= [interfaces l2tpv3 l2tpeth0](800)
         changing [interfaces l2tpv3 l2tpeth0 mtu] to (801)
  [ interfaces l2tpv3 l2tpeth0 bridge-group ]
  Adding interface l2tpeth0 to bridge br0

  vyos@BRF-R309-RT02:~$ show interfaces
  Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
  Interface        IP Address                        S/L  Description
  ---------        ----------                        ---  -----------
  br0              -                                 u/u  L2TPV3 Bridge Grp Mgt
  eth0                      u/u
  eth1                      u/u
  eth2             -                                 u/D
  eth3             -                                 u/D
  eth4             -                                 u/D
  eth5             -                                 u/u  L2 OPT4 Grp Mgt
  l2tpeth0         -                                 u/u  L2 Tunnel Grp Mgt
  lo                            u/u

  vyos@BRF-R309-RT02:~$ show interfaces l2tpv3 l2tpeth0
  l2tpeth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8958 qdisc pfifo_fast master br0 state UNKNOWN group default qlen 1000
    link/ether 0a:a1:c0:12:18:a6 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::8a1:c0ff:fe12:18a6/64 scope link
       valid_lft forever preferred_lft forever
    Description: L2 Tunnel Grp Mgt

    RX:  bytes    packets     errors    dropped    overrun      mcast
         62606         44          0          0          0          0
    TX:  bytes    packets     errors    dropped    carrier collisions
         63000         51          0          0          0          0

Are we have an error in the configuration, are we doing something wrong? It looks like it has something to do with the MTU Settings on the ethernet interfaces and the l2tp interface.

Regards Mätthi


Difficulty level
Unknown (require assessment)
VyOS-1.2.0-RC3, VyOS-1.2.0-RC4
Why the issue appeared?
Will be filled on close
Maetthi created this task.Oct 27 2018, 9:31 AM
syncer triaged this task as Normal priority.

We did some tests with RC5. Sometimes the complet configuration gets loaded after a reboot, but most of the time not. Maybe a timing issue?

pasik added a subscriber: pasik.Nov 4 2018, 11:21 AM

I have tested, the reason may be that your interface and the switch have not been UP. The route leading to the default route or L2TPV3 PEER address does not take effect. The ip l2tp command does not work.
This can be configured without any network cable. To configure L2TPV3, you will find that it is unable to commit.

My current method is:

[ -d /sys/module/l2tp_eth ] || sudo modprobe l2tp_eth
[ -d /sys/module/l2tp_netlink ] || sudo modprobe l2tp_netlink
if [ "$VAR(./encapsulation/@)" = "ip" ]; then
  if [ ! -d /sys/module/l2tp_ip ]; then
    sudo modprobe l2tp_ip

  if [ ! -d /sys/module/l2tp_ip6 ]; then
    sudo modprobe l2tp_ip6

if [ ! -d /sys/class/net/dum-l2tpv3 ]; then
  [ -d /sys/module/dummy ] || sudo modprobe dummy
  ip link add name dum-l2tpv3 type dummy
  ip link set dum-l2tpv3 up
  vtysh -c "configure terminal" \
            -c "ip route dum-l2tpv3 254";
Maetthi added a comment.EditedDec 10 2018, 1:47 PM

We did some testing with the RC10 Version as requested by the blog.

The issue is still there. After a reboot the settings for the l2tpv3 interfaces are not loaded and we have a difference between the running and the saved configuration. The l2tpv3 interface is bound to eth0 with has a static configuration.

Just for a test, we changed our management interfaces (eth1) from dhcp to static. This means we also changed the default static route setting from dchp to a static address.

In this case, the whole configuration got loaded. We testet that 3 times, no failer. As soon as we set the management interface back to dhcp and set also the default routing back to use the default route provided by dhcp, the issue was here again.

I can provide some logfiles, if needed. Btw. Is there something like a debug mode to get more information what is happening during boot and during applying the stored configuration?

commo added a subscriber: commo.Dec 10 2018, 5:40 PM

I suspect this is the same issue as T1080. if the destination ip of the l2tpv3 tunnel is unreachable(no default route) it wont load. I suspect its loading the l2tpv3 config before DHCP has installed a default route. My workaround is to install a very low metric static default route, that will get overwritten by OSPF(in my case).

syncer assigned this task to hagbard.Feb 8 2019, 12:16 AM