Hi,
I have multihomed network with 2 internet connections. Up to not long ago it was cable and DSL but now I’m trying to replace DSL with Starlink.
Starlink had two modes of operation:
- All in one with Wi-Fi router enabled that allocates IPv4 addresses via DHCP from 198.168.1.0/24 range
- Router bypass mode that alocates IPv4 addresses from 100.64.0.0/10 range (CGNAT)
For obvious reasons I’m more interested in the latter one. The configuration shod work identically to my cable connection but for some strange reasons it doesn’t.
And here are some more details:
r24:~$ show version Version: VyOS 1.3.0-epa3 Release train: equuleus Built by: Sentrium S.L. Built on: Sun 31 Oct 2021 17:38 UTC Build UUID: 383e45ad-b32a-4359-8183-9baacc8e69d9 Build commit ID: bb511522cc3bb2-dirty Architecture: x86_64 Boot via: installed image System type: KVM guest Hardware vendor: QEMU Hardware model: Standard PC (Q35 + ICH9, 2009) Hardware S/N: Hardware UUID: 18a48ed6-0124-41b0-a4f8-bbbca875d989 Copyright: VyOS maintainers and contributors
the interface in question configuration:
r24# show interfaces ethernet eth4 address dhcp description WAN-STARLINK dhcp-options { no-default-route } hw-id 00:1b:21:8c:bd:a3 vrf STARLINK [edit]
the routing table:
r24:~$ show ip route vrf STARLINK Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP, T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP, F - PBR, f - OpenFabric, > - selected route, * - FIB route, q - queued, r - rejected, b - backup VRF STARLINK: S>* 0.0.0.0/0 [1/0] via 100.64.0.1, eth4, weight 1, 10:30:21 K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 10:32:18 S>* 34.120.255.244/32 [1/0] is directly connected, eth4, weight 1, 10:30:21 C>* 100.64.0.0/10 is directly connected, eth4, 10:30:21 S>* 192.168.100.1/32 [1/0] is directly connected, eth4, weight 1, 10:30:21
(note administrative distance of default route and the fact that it is created in the first place contadicts the cinterface configuration)
Log from dhcp client:
r24:~$ show log dhcp client ... Apr 30 22:35:44 systemd[1]: Starting DHCP client on eth4... Apr 30 22:35:44 systemd[1]: Started DHCP client on eth4. Apr 30 22:35:44 dhclient-script-vyos[26939]: Current dhclient PID: 26938, Parent PID: 26937, IP version: 4, All dhclients for interface eth4: 26937 26938 Apr 30 22:35:44 dhclient-script-vyos[26939]: Passing command to /usr/sbin/ip: "link set dev eth4 up" Apr 30 22:35:44 dhclient-script-vyos[26939]: No changes to apply via vyos-hostsd-client Apr 30 22:35:44 dhclient[26938]: DHCPDISCOVER on eth4 to 255.255.255.255 port 67 interval 7 Apr 30 22:35:44 dhclient[26938]: DHCPOFFER of 100.123.57.53 from 100.64.0.1 Apr 30 22:35:44 dhclient[26938]: DHCPREQUEST for 100.123.57.53 on eth4 to 255.255.255.255 port 67 Apr 30 22:35:44 dhclient[26938]: DHCPACK of 100.123.57.53 from 100.64.0.1 Apr 30 22:35:44 dhclient-script-vyos[26961]: Current dhclient PID: 26938, Parent PID: 1, IP version: 4, All dhclients for interface eth4: 26938 Apr 30 22:35:44 dhclient-script-vyos[26961]: Passing command to /usr/sbin/ip: "-4 addr add 100.123.57.53/255.192.0.0 broadcast 100.127.255.255 valid_lft 300 preferred_lft 300 dev eth4 label eth4" Apr 30 22:35:44 dhclient-script-vyos[26961]: Passing command to /usr/sbin/ip: "link set dev eth4 mtu 1500" Apr 30 22:35:44 dhclient-script-vyos[26961]: Deleting nameservers with tag "dhcp-eth4" via vyos-hostsd-client Apr 30 22:35:44 dhclient-script-vyos[26961]: Adding nameservers "1.1.1.1 8.8.8.8" with tag "dhcp-eth4" via vyos-hostsd-client Apr 30 22:35:44 dhclient-script-vyos[26961]: Applying changes via vyos-hostsd-client Apr 30 22:35:44 dhclient-script-vyos[26961]: No changes to apply via vyos-hostsd-client Apr 30 22:35:44 dhclient-script-vyos[26961]: FRR status: running Apr 30 22:35:44 dhclient-script-vyos[26961]: Checking if the route presented in kernel: 192.168.100.1/32 dev eth4 Apr 30 22:35:44 dhclient-script-vyos[26961]: Converted vtysh command: "ip route 192.168.100.1/32 eth4 tag 210 vrf STARLINK" Apr 30 22:35:44 dhclient-script-vyos[26961]: Sending command to vtysh Apr 30 22:35:44 dhclient-script-vyos[26961]: FRR status: running Apr 30 22:35:44 dhclient-script-vyos[26961]: Checking if the route presented in kernel: 34.120.255.244/32 dev eth4 Apr 30 22:35:44 dhclient-script-vyos[26961]: Converted vtysh command: "ip route 34.120.255.244/32 eth4 tag 210 vrf STARLINK" Apr 30 22:35:44 dhclient-script-vyos[26961]: Sending command to vtysh Apr 30 22:35:44 dhclient-script-vyos[26961]: FRR status: running Apr 30 22:35:44 dhclient-script-vyos[26961]: Checking if the route presented in kernel: 0.0.0.0/0 via 100.64.0.1 dev eth4 Apr 30 22:35:44 dhclient-script-vyos[26961]: Converted vtysh command: "ip route 0.0.0.0/0 100.64.0.1 eth4 tag 210 vrf STARLINK" Apr 30 22:35:44 dhclient-script-vyos[26961]: Sending command to vtysh Apr 30 22:35:45 dhclient[26938]: bound to 100.123.57.53 -- renewal in 150 seconds.
Expected behavior:
- no default route created
Actual behavior:
- the default route is create with administrative distance of 1 that is cannot be overwitten
Additional details:
- The interface runs in a separate vrf but behaviour is identical in the default vrf as well. I’m exploring option of isolating starling routes
- DHCP client log shows that route is injected with ... tag 210 which I assume is the distance but it ends up with distance of 1 in the routing table.
- Also, if I switch Starlink into the default mode with Wi-Fi router enabled, it allocates an address and routes with correct distance. In theory I could use it as workaround (even though double NAT does not inspire confidence) bu the bigger problem is that it allocates address from 192.168.1.0/24 range that conflict with one of my existing subnets, so it is a no-go.