The Cable Modem provided by my ISP has a default IP that can be used to get status information about the modem. In my setup I have two WAN connections my device flow is <End User Devices> --> <Firewall pfSense> --> <Router VyOS> --> <Cable Modem>. To access the Cable Modem on that preassigned IP, the flow is <End User Devices> --> <Firewall pfSense> --> <Router VyOS> --> <Cable Modem>.
I have the relevant config below. I have removed IP specifics from the outputs.
If the ISP connected to that Cable modem is up traffic comes into the VyOS on 'bond1.221' and leaves on bond1.211'. One issue is when that ISP is down my firewall sends traffic through the other WAN interface which comes into the VyOS on bond1.121. 'set policy route FIOS_01_INET rule 1001' controls traffic coming into that interface which would forward the traffic to the other ISP instead of the Cable Modem. To address this I created 'set policy route FIOS_01_INET rule 1000' which then should route traffic to the IP of the Cable Modem to bond1.121.
However, when I added that rule, I could access the Cable Modem, but traffic wouldn't flow to the ISP. I rebooted the VyOS and traffic would then flow to the ISP, but no longer to the Cable Modem. I am running vyos-1.3.0-rc6-amd64. I was going to test this on vyos-1.3.0-amd64, but per T4206 that I posted earlier, I can use that version.
falkor@nenyas-edge-01:~$ sh ip route 192.168.x.1
Routing entry for 192.168.x.0/24
Known via "connected", distance 0, metric 0, best
Last update 01:02:03 ago
* directly connected, bond1.121
falkor@nenyas-edge-01:~$
falkor@nenyas-edge-01:~$ show ip route table 111
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
VRF default table 111:
S>* 0.0.0.0/0 [1/0] via x.x.x.1, bond1.111, weight 1, 00:28:40
falkor@nenyas-edge-01:~$
falkor@nenyas-edge-01:~$ show conf commands | match 'policy route|source rule 1001|table'
set interfaces bonding bond1 vif 211 policy route 'FIOS_01_INET'
set interfaces bonding bond1 vif 221 policy route 'WOW_01_INET'
set nat source rule 1001 description 'NAT Traffic to 192.168.100.0/24'
set nat source rule 1001 destination address '192.168.x.0/24'
set nat source rule 1001 outbound-interface 'bond1.121'
set nat source rule 1001 translation address '192.168.x.x'
set policy route FIOS_01_INET rule 1000 description 'Route traffic to ISP Modem (192.168.x.x) to ISP Interface'
set policy route FIOS_01_INET rule 1000 destination address '192.168.x.0/24'
set policy route FIOS_01_INET rule 1000 disable
set policy route FIOS_01_INET rule 1000 set table 'main'
set policy route FIOS_01_INET rule 1001 description 'Route traffic from the specified subnets through FIOS_01_INET'
set policy route FIOS_01_INET rule 1001 set table '111'
set policy route FIOS_01_INET rule 1001 source group network-group 'FIOS_01_INET_NETS'
set policy route WOW_01_INET rule 1001 description 'Route traffic from the specified subnets through WOW_01_INET'
set policy route WOW_01_INET rule 1001 set table '121'
set policy route WOW_01_INET rule 1001 source group network-group 'WOW_01_INET_NETS'
set protocols static table 111 route 0.0.0.0/0 dhcp-interface 'bond1.111'
set protocols static table 121 route 0.0.0.0/0 next-hop x.x.x.x
falkor@nenyas-edge-01:~$