Page MenuHomeVyOS Platform

Issues With More than Two Default Route Paths
Open, Requires assessmentPublicBUG

Description

This may be more of a linux bug or require some tuning to resolve. I have multiple ISPs so when troubleshooting issues on specific paths, I often use curl, mtr, ping, telnet, and traceroute and specify a source IP. I find that they only work for the first two WAN interfaces. The 3rd interface fails. It seems to be tied to the metric of the default route associated with the interface. I know it's not an issue with that specific interface/default route because if I change the metric on the 3rd default route so that it is one of the first two, it and the other one in the first two work, but now the interface that has the 3rd default route fails.

Below are examples.

In example set 1, you can see that the default route for bond1.111 is 1st and the speedtest for that interface succeeds, the default route for bond1.121 is 2nd and the speedtest for that interface succeeds, and the default route eth0.10 is 3rd and the speedtest for that interface fails with errors.

In example set 2, you can see that the default route for bond1.111 is 2nd and the speedtest for that interface succeeds, the default route for bond1.121 is 3rd and the speedtest for that interface fails with errors, and the default route eth0.10 is 1st and the speedtest for that interface succeeds.

I am running vyos-1.3.0-rc6-amd64. I was going to test this on vyos-1.3.0-amd64, but per T4206 that I posted earlier, I can use that version.

Example Set 1:

[email protected]:~/ookla-speedtest-1.1.1# show ip route | head -9
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S   0.0.0.0/0 [210/0] via x.x.x1, bond1.111, weight 1, 01:15:17
S>* 0.0.0.0/0 [10/0] via x.x.x.105, bond1.121, weight 1, 01:15:39
S   0.0.0.0/0 [20/0] via 10.x.x.1, eth0.10, weight 1, 01:15:39
[email protected]:~/ookla-speedtest-1.1.1#

[email protected]:~/ookla-speedtest-1.1.1# ./speedtest -I bond1.111

   Speedtest by Ookla

     Server: Frontier - Tampa, FL (id = 27513)
        ISP: Frontier Communications
    Latency:     1.68 ms   (0.09 ms jitter)
   Download:    77.20 Mbps (data used: 90.9 MB )
     Upload:    91.80 Mbps (data used: 114.2 MB )
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/d7df8668-a146-4cb0-9d98-48456e322328
[email protected]:~/ookla-speedtest-1.1.1#

[email protected]:~/ookla-speedtest-1.1.1# ./speedtest -I bond1.121

   Speedtest by Ookla

     Server: Velocity Online - Tallahassee, FL (id = 884)
        ISP: WideOpenWest
    Latency:    28.09 ms   (1.90 ms jitter)
   Download:   537.66 Mbps (data used: 943.2 MB )
     Upload:    13.56 Mbps (data used: 22.6 MB )
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/cd3be733-3f02-40e7-bb92-007ab26e0ad4
[email protected]:~/ookla-speedtest-1.1.1#

Example Set 2:

[email protected]:~# show ip route | head -9
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
       F - PBR, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup

S>* 0.0.0.0/0 [5/0] via 10.x.x.1, eth0.10, weight 1, 00:00:26
S   0.0.0.0/0 [210/0] via x.x.x.1, bond1.111, weight 1, 01:21:07
S   0.0.0.0/0 [10/0] via x.x.x.105, bond1.121, weight 1, 01:21:29
[email protected]:~#

[email protected]:~/ookla-speedtest-1.1.1# ./speedtest -I bond1.111

   Speedtest by Ookla

     Server: Frontier - Tampa, FL (id = 27513)
        ISP: Frontier Communications
    Latency:     1.77 ms   (0.06 ms jitter)
   Download:    74.93 Mbps (data used: 98.8 MB )
     Upload:    91.88 Mbps (data used: 114.2 MB )
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/e9c10e20-4add-4cbd-947f-e318660916fd
[email protected]:~/ookla-speedtest-1.1.1#

[email protected]:~/ookla-speedtest-1.1.1# ./speedtest -I bond1.121
[2022-01-24 23:48:34.392] [error] Configuration - Couldn't connect to server (Network is unreachable)
[2022-01-24 23:48:34.392] [error] Configuration - Cannot retrieve configuration document (0)
[2022-01-24 23:48:34.392] [error] ConfigurationError - Could not retrieve or read configuration (Configuration)
[2022-01-24 23:48:34.393] [error] ConfigurationError - Could not retrieve or read configuration (Configuration)
[error] Configuration - Could not retrieve or read configuration (ConfigurationError)
[email protected]:~/ookla-speedtest-1.1.1#

[email protected]:~/ookla-speedtest-1.1.1# ./speedtest -I eth0.10

   Speedtest by Ookla

     Server: Frontier - Tampa, FL (id = 27513)
        ISP: WideOpenWest
    Latency:    41.44 ms   (1.36 ms jitter)
   Download:    74.30 Mbps (data used: 87.8 MB )
     Upload:    99.89 Mbps (data used: 123.1 MB )
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/fc63e464-c5a0-4efb-b5a8-aa4d2926aea8
[email protected]:~/ookla-speedtest-1.1.1#

Details

Difficulty level
Unknown (require assessment)
Version
1.3.0 rc6
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Behavior change
Issue type
Bug (incorrect behavior)