Page MenuHomeVyOS Platform

Wan Load Balancing - Can't create routing tables
Open, Requires assessmentPublicBUG

Description

Tested on VyOS 1.4-rolling-202204130521

Commands for Wan Load Balancing:

set interfaces ethernet eth0 address 'dhcp'
set interfaces ethernet eth1 address 'dhcp'
set interfaces ethernet eth2 address 'dhcp'
set interfaces ethernet eth3 address '192.168.250.1/24'
set load-balancing wan interface-health eth0 failure-count '5'
set load-balancing wan interface-health eth0 nexthop 'dhcp'
set load-balancing wan interface-health eth0 success-count '1'
set load-balancing wan interface-health eth0 test 10 resp-time '5'
set load-balancing wan interface-health eth0 test 10 target '1.1.1.1'
set load-balancing wan interface-health eth0 test 10 ttl-limit '1'
set load-balancing wan interface-health eth0 test 10 type 'ttl'
set load-balancing wan interface-health eth0 test 20 resp-time '5'
set load-balancing wan interface-health eth0 test 20 target '1.0.0.1'
set load-balancing wan interface-health eth0 test 20 ttl-limit '1'
set load-balancing wan interface-health eth0 test 20 type 'ping'
set load-balancing wan interface-health eth1 failure-count '5'
set load-balancing wan interface-health eth1 nexthop 'dhcp'
set load-balancing wan interface-health eth1 success-count '1'
set load-balancing wan interface-health eth1 test 10 resp-time '5'
set load-balancing wan interface-health eth1 test 10 target '8.8.8.8'
set load-balancing wan interface-health eth1 test 10 ttl-limit '1'
set load-balancing wan interface-health eth1 test 10 type 'ttl'
set load-balancing wan interface-health eth1 test 20 resp-time '5'
set load-balancing wan interface-health eth1 test 20 target '8.8.4.4'
set load-balancing wan interface-health eth1 test 20 ttl-limit '1'
set load-balancing wan interface-health eth1 test 20 type 'ping'
set load-balancing wan interface-health eth2 failure-count '5'
set load-balancing wan interface-health eth2 nexthop 'dhcp'
set load-balancing wan interface-health eth2 success-count '1'
set load-balancing wan interface-health eth2 test 10 resp-time '5'
set load-balancing wan interface-health eth2 test 10 target '9.9.9.9'
set load-balancing wan interface-health eth2 test 10 ttl-limit '1'
set load-balancing wan interface-health eth2 test 10 type 'ttl'
set load-balancing wan interface-health eth2 test 20 resp-time '5'
set load-balancing wan interface-health eth2 test 20 target '149.112.112.112'
set load-balancing wan interface-health eth2 test 20 ttl-limit '1'
set load-balancing wan interface-health eth2 test 20 type 'ping'
set load-balancing wan rule 1000 description 'DEFAULT FAILOVER RULE'
set load-balancing wan rule 1000 failover
set load-balancing wan rule 1000 inbound-interface 'eth3'
set load-balancing wan rule 1000 interface eth0 weight '3'
set load-balancing wan rule 1000 interface eth1 weight '2'
set load-balancing wan rule 1000 interface eth2 weight '1'
set load-balancing wan rule 1000 protocol 'all'
set load-balancing wan sticky-connections inbound
set protocols static route 1.0.0.1/32 next-hop 172.16.0.1
set protocols static route 1.1.1.1/32 next-hop 172.16.0.1
set protocols static route 8.8.4.4/32 next-hop 192.168.122.1
set protocols static route 8.8.8.8/32 next-hop 192.168.122.1
set protocols static route 9.9.9.9/32 next-hop 172.16.2.1
set protocols static route 149.112.112.112/32 next-hop 172.16.2.1

The result of the command wan-load-balance status:

Chain WANLOADBALANCE_PRE (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ISP_eth0   all  --  eth3   *       0.0.0.0/0            0.0.0.0/0            state NEW
    0     0 CONNMARK   all  --  eth3   *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore
[edit]

[email protected]# sudo ip rule show
0:	from all lookup local
32763:	from all fwmark 0xcb lookup 203
32764:	from all fwmark 0xca lookup 202
32765:	from all fwmark 0xc9 lookup 201
32766:	from all lookup main
32767:	from all lookup default
[edit]

The result of the command ip route:

[email protected]# sudo ip r show
default nhid 30 proto static metric 20 
	nexthop via 172.16.0.1 dev eth0 weight 1 
	nexthop via 172.16.2.1 dev eth2 weight 1 
	nexthop via 192.168.122.1 dev eth1 weight 1 
1.0.0.1 nhid 35 via 172.16.0.1 dev eth0 proto static metric 20 
1.1.1.1 nhid 35 via 172.16.0.1 dev eth0 proto static metric 20 
8.8.4.4 nhid 37 via 192.168.122.1 dev eth1 proto static metric 20 
8.8.8.8 nhid 37 via 192.168.122.1 dev eth1 proto static metric 20 
9.9.9.9 nhid 36 via 172.16.2.1 dev eth2 proto static metric 20 
149.112.112.112 nhid 36 via 172.16.2.1 dev eth2 proto static metric 20 
172.16.0.0/24 dev eth0 proto kernel scope link src 172.16.0.10 
172.16.2.0/24 dev eth2 proto kernel scope link src 172.16.2.10 
192.168.122.0/24 dev eth1 proto kernel scope link src 192.168.122.35 
192.168.250.0/24 dev eth3 proto kernel scope link src 192.168.250.1 

[email protected]# sudo ip r show table 201
Error: ipv4: FIB table does not exist.
Dump terminated
[edit]
[email protected]# sudo ip r show table 202
Error: ipv4: FIB table does not exist.
Dump terminated
[edit]
[email protected]# sudo ip r show table 203
Error: ipv4: FIB table does not exist.
Dump terminated
[edit]
[email protected]#

Routing tables 201, 202 and 203 are not created.

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.4-rolling-202204130521
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

This comment was removed by Viacheslav.

Confirming the same.

[email protected]# sudo ip rule
0:	from all lookup local
98:	from all fwmark 0xcc lookup 204
99:	from all fwmark 0xcb lookup 203
100:	from all fwmark 0xca lookup 202
101:	from all fwmark 0xc9 lookup 201
102:	from all fwmark 0x7fffff99 lookup 102
114:	from all fwmark 0x7fffff8d lookup 114
32766:	from all lookup main
32767:	from all lookup default
[email protected]# sudo ip r show table 202
Error: ipv4: FIB table does not exist.
Dump terminated
[edit]
[email protected]# sudo ip r show table 203
Error: ipv4: FIB table does not exist.
Dump terminated
[edit]
[email protected]# sudo ip r show table 204
Error: ipv4: FIB table does not exist.
Dump terminated

If it helps, I am also getting the exact same errors and problems, would love to see this working please

I can't reproduce it
With such configuration all works fine VyOS 1.4-rolling-202204300743:

set load-balancing wan interface-health eth4 failure-count '5'
set load-balancing wan interface-health eth4 nexthop 'dhcp'
set load-balancing wan interface-health eth4 success-count '1'
set load-balancing wan interface-health eth4 test 10 target '192.0.2.40'
set load-balancing wan interface-health eth5 failure-count '5'
set load-balancing wan interface-health eth5 nexthop 'dhcp'
set load-balancing wan interface-health eth5 success-count '1'
set load-balancing wan interface-health eth5 test 10 target '192.0.2.50'
set load-balancing wan interface-health eth6 failure-count '5'
set load-balancing wan interface-health eth6 nexthop 'dhcp'
set load-balancing wan interface-health eth6 success-count '1'
set load-balancing wan interface-health eth6 test 10 target '192.0.2.60'
set load-balancing wan rule 10 failover
set load-balancing wan rule 10 inbound-interface 'eth7'
set load-balancing wan rule 10 interface eth4
set load-balancing wan rule 10 interface eth5
set load-balancing wan rule 10 interface eth6
set load-balancing wan rule 10 protocol 'all'
set load-balancing wan sticky-connections

Check:

[email protected]:~$ sudo ip route show table 201
default via 100.64.4.1 dev eth4 
[email protected]:~$ 
[email protected]:~$ sudo ip route show table 202
default via 100.64.5.1 dev eth5 
[email protected]:~$ 
[email protected]:~$ sudo ip route show table 203
default via 100.64.6.1 dev eth6 
[email protected]:~$ 
[email protected]:~$ show wan-load-balance 
Interface:  eth4
  Status:  active
  Last Status Change:  Wed May  4 10:32:59 2022
  +Test:  ping  Target: 192.0.2.40
    Last Interface Success:  0s 
    Last Interface Failure:  2m0s       
    # Interface Failure(s):  0

Interface:  eth5
  Status:  active
  Last Status Change:  Wed May  4 10:32:59 2022
  +Test:  ping  Target: 192.0.2.50
    Last Interface Success:  0s 
    Last Interface Failure:  2m0s       
    # Interface Failure(s):  0

Interface:  eth6
  Status:  active
  Last Status Change:  Wed May  4 10:32:59 2022
  +Test:  ping  Target: 192.0.2.60
    Last Interface Success:  0s 
    Last Interface Failure:  2m0s       
    # Interface Failure(s):  0

[email protected]:~$

I was able to reproduce issue on latest VyOS 1.4-rolling-202205060217
Steps to reproduce:
1 - Fresh/clean vyos router
2 - Add interface configuration (dhcp on WANs and static IP addresses on LAN side), commit and save
3 - Add next WLB configuration:

[email protected]:~$ show config comm | grep wan
set load-balancing wan interface-health eth0 failure-count '5'
set load-balancing wan interface-health eth0 nexthop 'dhcp'
set load-balancing wan interface-health eth0 success-count '1'
set load-balancing wan interface-health eth0 test 10 resp-time '5'
set load-balancing wan interface-health eth0 test 10 target '1.1.1.1'
set load-balancing wan interface-health eth0 test 10 ttl-limit '1'
set load-balancing wan interface-health eth1 failure-count '5'
set load-balancing wan interface-health eth1 nexthop 'dhcp'
set load-balancing wan interface-health eth1 success-count '1'
set load-balancing wan interface-health eth1 test 10 resp-time '5'
set load-balancing wan interface-health eth1 test 10 target '8.8.8.8'
set load-balancing wan interface-health eth1 test 10 ttl-limit '1'
set load-balancing wan rule 10 failover
set load-balancing wan rule 10 inbound-interface 'eth3.25'
set load-balancing wan rule 10 interface eth0 weight '50'
set load-balancing wan rule 10 interface eth1 weight '100'
set load-balancing wan rule 10 protocol 'all'
set load-balancing wan rule 20 failover
set load-balancing wan rule 20 inbound-interface 'eth3.35'
set load-balancing wan rule 20 interface eth0 weight '100'
set load-balancing wan rule 20 interface eth1 weight '50'
set load-balancing wan rule 20 protocol 'all'

4 - commit;save
5 - Verify routing tables, save configuration and reboot:

[email protected]# sudo ip route show table 201
default via 198.51.100.1 dev eth0 
[edit]
[email protected]# sudo ip route show table 202
default via 192.0.2.1 dev eth1 
[edit]
[email protected]# save
Saving configuration to '/config/config.boot'...
Done
[edit]
[email protected]# run reboot now

6 - After reboot, verify WLB and routing tables:

[email protected]:~$ show wan-load-balance 
Interface:  eth0
  Status:  active
  Last Status Change:  Fri May  6 17:47:58 2022
  +Test:  ping  Target: 1.1.1.1
    Last Interface Success:  0s 
    Last Interface Failure:  n/a                
    # Interface Failure(s):  0

Interface:  eth1
  Status:  active
  Last Status Change:  Fri May  6 17:47:58 2022
  -Test:  ping  Target: 8.8.8.8
    Last Interface Success:  n/a                
    Last Interface Failure:  0s 
    # Interface Failure(s):  1

[email protected]:~$ sudo ip route show table 201
Error: ipv4: FIB table does not exist.
Dump terminated
vy[email protected]:~$ sudo ip route show table 202
Error: ipv4: FIB table does not exist.
Dump terminated
[email protected]:~$ 
[email protected]:~$ 
[email protected]:~$ sudo ip rule
0:      from all lookup local
32764:  from all fwmark 0xca lookup 202
32765:  from all fwmark 0xc9 lookup 201
32766:  from all lookup main
32767:  from all lookup default