Page MenuHomeVyOS Platform

Zone-policy implementation does not allow secondary IP on an interface to communicate
Open, Requires assessmentPublic

Description

When implementing zone-based firewall on a system that has multiple IP addresses assigned to an interface, the second IP address fails to communicate.

ping -I <secondary IP> <destination> fails, even with all firewall rules set into an ANY-ANY type state for every zone.

vyos@KSC-RBX-RTR001# show
 firewall {
     all-ping enable
     broadcast-ping disable
     config-trap disable
     group {
         network-group NET-IPSECHOME {
             network 10.255.0.4/30
         }
         network-group NET-WAN {
             network WAN.PREFIX.0/27
         }
         network-group SERVER-LAN {
             network 10.101.0.0/26
         }
         network-group vlans {
             network 10.101.0.0/26
         }
     }
     ipv6-receive-redirects disable
     ipv6-src-route disable
     ip-src-route disable
     log-martians enable
     name ANYANY {
         default-action accept
         enable-default-log
     }
     name SERVER100-WAN {
         default-action drop
         enable-default-log
         rule 1 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
         rule 2 {
             action drop
             state {
                 invalid enable
             }
         }
         rule 100 {
             action accept
             protocol icmp
         }
         rule 200 {
             action accept
             destination {
                 port 80,443
             }
             protocol tcp
         }
         rule 9999 {
             action drop
         }
     }
     name WAN-LOCAL {
         default-action drop
         rule 1 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
         rule 2 {
             action drop
             log enable
             state {
                 invalid enable
             }
         }
         rule 100 {
             action accept
             log enable
             protocol ICMP
         }
         rule 200 {
             action accept
             destination {
                 port 22
             }
             log enable
             protocol tcp
         }
         rule 300 {
             action accept
             protocol esp
         }
         rule 301 {
             action accept
             destination {
                 port 500
             }
             protocol udp
         }
         rule 302 {
             action accept
             destination {
                 port 4500
             }
             protocol udp
         }
         rule 303 {
             action accept
             destination {
                 port 1701
             }
             ipsec {
                 match-ipsec
             }
             protocol udp
         }
         rule 9999 {
             action drop
             log enable
         }
     }
     name WAN-SERVER100 {
         default-action drop
         rule 1 {
             action accept
             state {
                 established enable
                 related enable
             }
         }
         rule 2 {
             action drop
             log enable
             state {
                 invalid enable
             }
         }
         rule 9999 {
             action drop
             log enable
         }
     }
     receive-redirects disable
     send-redirects enable
     source-validation disable
     syn-cookies enable
     twa-hazards-protection disable
 }
 interfaces {
     ethernet eth0 {
         address WAN.PREFIX.29/27
         address WAN.PREFIX.28/27
         description WAN
         duplex auto
         hw-id fa:cd:49:76:d3:49
         smp-affinity auto
         speed auto
     }
     ethernet eth1 {
         duplex auto
         hw-id 9e:e9:04:a2:e9:c2
         smp-affinity auto
         speed auto
         vif 100 {
             address 10.101.0.1/26
             description "Server LAN"
         }
     }
     loopback lo {
     }
     vti vti0 {
         address 10.255.0.6/30
         description "IPsec to HOME"
     }
 }
 nat {
     source {
         rule 2000 {
             description "1-to-1 example"
             outbound-interface eth0
             source {
                 address 10.101.0.2
             }
             translation {
                 address WAN.PREFIX.28
             }
         }
         rule 9001 {
             outbound-interface eth0
             source {
                 address 10.101.0.0/26
             }
             translation {
                 address WAN.PREFIX.29
             }
         }
     }
 }
 protocols {
     static {
         interface-route 10.100.251.0/24 {
             next-hop-interface vti0 {
             }
         }
         interface-route 10.255.0.0/30 {
             next-hop-interface vti0 {
             }
         }
         route 0.0.0.0/0 {
             next-hop WAN.PREFIX.30 {
             }
         }
     }
 }
 service {
     ssh {
         port 22
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     console {
         device ttyS0 {
             speed 9600
         }
     }
     domain-name corp.kisaracorporation.com
     host-name KSC-RBX-RTR001
     login {
         user vyos {
             authentication {
                 encrypted-password 
                 plaintext-password ""
             }
             level admin
         }
     }
     ntp {
         server 0.pool.ntp.org {
         }
         server 1.pool.ntp.org {
         }
         server 2.pool.ntp.org {
         }
     }
     syslog {
         global {
             facility all {
                 level info
             }
             facility protocols {
                 level debug
             }
         }
     }
     time-zone UTC
 }
 vpn {
     ipsec {
         esp-group ESP-OVHLOCAL {
             compression disable
             lifetime 3600
             mode tunnel
             pfs dh-group5
             proposal 1 {
                 encryption aes128
                 hash sha1
             }
         }
         ike-group IKE-OVHLOCAL {
             ikev2-reauth no
             key-exchange ikev1
             lifetime 28800
             proposal 1 {
                 dh-group 5
                 encryption aes128
                 hash sha1
             }
         }
         ipsec-interfaces {
             interface eth0
         }
         site-to-site {
             peer IPV4REMOTESITE {
                 authentication {
                     mode pre-shared-secret
                     pre-shared-secret 
                 }
                 connection-type initiate
                 description OVH-REMOTE
                 ike-group IKE-OVHLOCAL
                 ikev2-reauth inherit
                 local-address WAN.PREFIX.29
                 vti {
                     bind vti0
                     esp-group ESP-OVHLOCAL
                 }
             }
         }
     }
 }
 zone-policy {
     zone IPSEC-HOME {
         from SERVER100 {
             firewall {
                 name ANYANY
             }
         }
         from WAN {
             firewall {
                 name ANYANY
             }
         }
         from local {
             firewall {
                 name ANYANY
             }
         }
         interface vti0
     }
     zone SERVER100 {
         from IPSEC-HOME {
             firewall {
                 name ANYANY
             }
         }
         from WAN {
             firewall {
                 name ANYANY
             }
         }
         from local {
             firewall {
                 name ANYANY
             }
         }
         interface eth1.100
     }
     zone WAN {
         from IPSEC-HOME {
             firewall {
                 name ANYANY
             }
         }
         from SERVER100 {
             firewall {
                 name ANYANY
             }
         }
         from local {
             firewall {
                 name ANYANY
             }
         }
         interface eth0
     }
     zone local {
         from IPSEC-HOME {
             firewall {
                 name ANYANY
             }
         }
         from SERVER100 {
             firewall {
                 name ANYANY
             }
         }
         from WAN {
             firewall {
                 name ANYANY
             }
         }
         local-zone
     }
 }
[edit]
vyos@KSC-RBX-RTR001# sudo su -
root@KSC-RBX-RTR001:~# iptables-save
# Generated by iptables-save v1.4.21 on Mon Mar 11 19:25:29 2019
*nat
:PREROUTING ACCEPT [42613:1616059]
:INPUT ACCEPT [37242:1333934]
:OUTPUT ACCEPT [1607:123524]
:POSTROUTING ACCEPT [4138:303063]
:VYATTA_PRE_DNAT_HOOK - [0:0]
:VYATTA_PRE_SNAT_HOOK - [0:0]
-A PREROUTING -j VYATTA_PRE_DNAT_HOOK
-A POSTROUTING -j VYATTA_PRE_SNAT_HOOK
-A POSTROUTING -s 10.101.0.2/32 -o eth0 -m comment --comment SRC-NAT-2000 -j SNAT --to-source WAN.PREFIX.28
-A POSTROUTING -s 10.101.0.0/26 -o eth0 -m comment --comment SRC-NAT-9001 -j SNAT --to-source WAN.PREFIX.29
-A VYATTA_PRE_DNAT_HOOK -j RETURN
-A VYATTA_PRE_SNAT_HOOK -j RETURN
COMMIT
# Completed on Mon Mar 11 19:25:29 2019
# Generated by iptables-save v1.4.21 on Mon Mar 11 19:25:29 2019
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:ANYANY - [0:0]
:SERVER100-WAN - [0:0]
:VYATTA_FW_IN_HOOK - [0:0]
:VYATTA_FW_LOCAL_HOOK - [0:0]
:VYATTA_FW_OUT_HOOK - [0:0]
:VYATTA_POST_FW_FWD_HOOK - [0:0]
:VYATTA_POST_FW_IN_HOOK - [0:0]
:VYATTA_POST_FW_OUT_HOOK - [0:0]
:VYATTA_PRE_FW_FWD_HOOK - [0:0]
:VYATTA_PRE_FW_IN_HOOK - [0:0]
:VYATTA_PRE_FW_OUT_HOOK - [0:0]
:VZONE_IPSEC-HOME - [0:0]
:VZONE_SERVER100 - [0:0]
:VZONE_WAN - [0:0]
:VZONE_local_IN - [0:0]
:VZONE_local_OUT - [0:0]
:WAN-LOCAL - [0:0]
:WAN-SERVER100 - [0:0]
-A INPUT -j VYATTA_PRE_FW_IN_HOOK
-A INPUT -j VYATTA_FW_LOCAL_HOOK
-A INPUT -j VZONE_local_IN
-A INPUT -j VYATTA_POST_FW_IN_HOOK
-A FORWARD -j VYATTA_PRE_FW_FWD_HOOK
-A FORWARD -j VYATTA_FW_IN_HOOK
-A FORWARD -j VYATTA_FW_OUT_HOOK
-A FORWARD -o vti0 -j VZONE_IPSEC-HOME
-A FORWARD -o eth1.100 -j VZONE_SERVER100
-A FORWARD -o eth0 -j VZONE_WAN
-A FORWARD -j VYATTA_POST_FW_FWD_HOOK
-A OUTPUT -j VYATTA_PRE_FW_OUT_HOOK
-A OUTPUT -j VZONE_local_OUT
-A OUTPUT -j VYATTA_POST_FW_OUT_HOOK
-A ANYANY -m comment --comment "ANYANY-10000 default-action accept" -j LOG --log-prefix "[ANYANY-default-A]"
-A ANYANY -m comment --comment "ANYANY-10000 default-action accept" -j RETURN
-A SERVER100-WAN -m comment --comment SERVER100-WAN-1 -m state --state RELATED,ESTABLISHED -j RETURN
-A SERVER100-WAN -m comment --comment SERVER100-WAN-2 -m state --state INVALID -j DROP
-A SERVER100-WAN -p icmp -m comment --comment SERVER100-WAN-100 -j RETURN
-A SERVER100-WAN -p tcp -m comment --comment SERVER100-WAN-200 -m multiport --dports 80,443 -j RETURN
-A SERVER100-WAN -m comment --comment SERVER100-WAN-9999 -j DROP
-A SERVER100-WAN -m comment --comment "SERVER100-WAN-10000 default-action drop" -j LOG --log-prefix "[SERVER100-WAN-default-D]"
-A SERVER100-WAN -m comment --comment "SERVER100-WAN-10000 default-action drop" -j DROP
-A VYATTA_POST_FW_FWD_HOOK -j ACCEPT
-A VYATTA_POST_FW_IN_HOOK -j ACCEPT
-A VYATTA_POST_FW_OUT_HOOK -j ACCEPT
-A VYATTA_PRE_FW_FWD_HOOK -j RETURN
-A VYATTA_PRE_FW_IN_HOOK -j RETURN
-A VYATTA_PRE_FW_OUT_HOOK -j RETURN
-A VZONE_IPSEC-HOME -i vti0 -j RETURN
-A VZONE_IPSEC-HOME -i eth1.100 -j ANYANY
-A VZONE_IPSEC-HOME -i eth1.100 -j RETURN
-A VZONE_IPSEC-HOME -i eth0 -j ANYANY
-A VZONE_IPSEC-HOME -i eth0 -j RETURN
-A VZONE_IPSEC-HOME -j DROP
-A VZONE_SERVER100 -i eth1.100 -j RETURN
-A VZONE_SERVER100 -i eth0 -j ANYANY
-A VZONE_SERVER100 -i eth0 -j RETURN
-A VZONE_SERVER100 -i vti0 -j ANYANY
-A VZONE_SERVER100 -i vti0 -j RETURN
-A VZONE_SERVER100 -j DROP
-A VZONE_WAN -i eth0 -j RETURN
-A VZONE_WAN -i eth1.100 -j ANYANY
-A VZONE_WAN -i eth1.100 -j RETURN
-A VZONE_WAN -i vti0 -j ANYANY
-A VZONE_WAN -i vti0 -j RETURN
-A VZONE_WAN -j DROP
-A VZONE_local_IN -i lo -j RETURN
-A VZONE_local_IN -i vti0 -j ANYANY
-A VZONE_local_IN -i vti0 -j RETURN
-A VZONE_local_IN -i eth0 -j ANYANY
-A VZONE_local_IN -i eth0 -j RETURN
-A VZONE_local_IN -i eth1.100 -j ANYANY
-A VZONE_local_IN -i eth1.100 -j RETURN
-A VZONE_local_IN -j DROP
-A VZONE_local_OUT -o lo -j RETURN
-A VZONE_local_OUT -o eth0 -j ANYANY
-A VZONE_local_OUT -o eth0 -j RETURN
-A VZONE_local_OUT -o vti0 -j ANYANY
-A VZONE_local_OUT -o vti0 -j RETURN
-A VZONE_local_OUT -o eth1.100 -j ANYANY
-A VZONE_local_OUT -o eth1.100 -j RETURN
-A VZONE_local_OUT -j DROP
-A WAN-LOCAL -m comment --comment WAN-LOCAL-1 -m state --state RELATED,ESTABLISHED -j RETURN
-A WAN-LOCAL -m comment --comment WAN-LOCAL-2 -m state --state INVALID -j LOG --log-prefix "[WAN-LOCAL-2-D] "
-A WAN-LOCAL -m comment --comment WAN-LOCAL-2 -m state --state INVALID -j DROP
-A WAN-LOCAL -p icmp -m comment --comment WAN-LOCAL-100 -j LOG --log-prefix "[WAN-LOCAL-100-A] "
-A WAN-LOCAL -p icmp -m comment --comment WAN-LOCAL-100 -j RETURN
-A WAN-LOCAL -p tcp -m comment --comment WAN-LOCAL-200 -m tcp --dport 22 -j LOG --log-prefix "[WAN-LOCAL-200-A] "
-A WAN-LOCAL -p tcp -m comment --comment WAN-LOCAL-200 -m tcp --dport 22 -j RETURN
-A WAN-LOCAL -p esp -m comment --comment WAN-LOCAL-300 -j RETURN
-A WAN-LOCAL -p udp -m comment --comment WAN-LOCAL-301 -m udp --dport 500 -j RETURN
-A WAN-LOCAL -p udp -m comment --comment WAN-LOCAL-302 -m udp --dport 4500 -j RETURN
-A WAN-LOCAL -p udp -m comment --comment WAN-LOCAL-303 -m udp --dport 1701 -m policy --dir in --pol ipsec -j RETURN
-A WAN-LOCAL -m comment --comment WAN-LOCAL-9999 -j LOG --log-prefix "[WAN-LOCAL-9999-D] "
-A WAN-LOCAL -m comment --comment WAN-LOCAL-9999 -j DROP
-A WAN-LOCAL -m comment --comment "WAN-LOCAL-10000 default-action drop" -j DROP
-A WAN-SERVER100 -m comment --comment WAN-SERVER100-1 -m state --state RELATED,ESTABLISHED -j RETURN
-A WAN-SERVER100 -m comment --comment WAN-SERVER100-2 -m state --state INVALID -j LOG --log-prefix "[WAN-SERVER100-2-D] "
-A WAN-SERVER100 -m comment --comment WAN-SERVER100-2 -m state --state INVALID -j DROP
-A WAN-SERVER100 -m comment --comment WAN-SERVER100-9999 -j LOG --log-prefix "[WAN-SERVER100-9999-D] "
-A WAN-SERVER100 -m comment --comment WAN-SERVER100-9999 -j DROP
-A WAN-SERVER100 -m comment --comment "WAN-SERVER100-10000 default-action drop" -j DROP
COMMIT
# Completed on Mon Mar 11 19:25:29 2019
# Generated by iptables-save v1.4.21 on Mon Mar 11 19:25:29 2019
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:FW_CONNTRACK - [0:0]
:NAT_CONNTRACK - [0:0]
:VYATTA_CT_HELPER - [0:0]
:VYATTA_CT_IGNORE - [0:0]
:VYATTA_CT_OUTPUT_HOOK - [0:0]
:VYATTA_CT_PREROUTING_HOOK - [0:0]
:VYATTA_CT_TIMEOUT - [0:0]
-A PREROUTING -j VYATTA_CT_IGNORE
-A PREROUTING -j VYATTA_CT_HELPER
-A PREROUTING -j VYATTA_CT_TIMEOUT
-A PREROUTING -j VYATTA_CT_PREROUTING_HOOK
-A PREROUTING -j NAT_CONNTRACK
-A PREROUTING -j FW_CONNTRACK
-A PREROUTING -j NOTRACK
-A OUTPUT -j VYATTA_CT_IGNORE
-A OUTPUT -j VYATTA_CT_HELPER
-A OUTPUT -j VYATTA_CT_TIMEOUT
-A OUTPUT -j VYATTA_CT_OUTPUT_HOOK
-A OUTPUT -j NAT_CONNTRACK
-A OUTPUT -j FW_CONNTRACK
-A OUTPUT -j NOTRACK
-A FW_CONNTRACK -j ACCEPT
-A NAT_CONNTRACK -j ACCEPT
-A VYATTA_CT_HELPER -p tcp -m tcp --dport 1536 -j CT --helper tns
-A VYATTA_CT_HELPER -p tcp -m tcp --dport 1525 -j CT --helper tns
-A VYATTA_CT_HELPER -p tcp -m tcp --dport 1521 -j CT --helper tns
-A VYATTA_CT_HELPER -p udp -m udp --dport 111 -j CT --helper rpc
-A VYATTA_CT_HELPER -p tcp -m tcp --dport 111 -j CT --helper rpc
-A VYATTA_CT_HELPER -j RETURN
-A VYATTA_CT_IGNORE -j RETURN
-A VYATTA_CT_OUTPUT_HOOK -j RETURN
-A VYATTA_CT_PREROUTING_HOOK -j RETURN
-A VYATTA_CT_TIMEOUT -j RETURN
COMMIT
# Completed on Mon Mar 11 19:25:29 2019
root@KSC-RBX-RTR001:~#

Details

Difficulty level
Unknown (require assessment)
Version
1.2.0-rolling+201903060337
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)

Event Timeline

Mokuba created this task.Mar 11 2019, 7:29 PM
Mokuba created this object in space S1 VyOS Public.
Mokuba updated the task description. (Show Details)
Mokuba updated the task description. (Show Details)
varesa added a subscriber: varesa.Mar 11 2019, 8:04 PM

I managed to reproduce this earlier. These were the generated iptables rules and pinging from any source IP except the first one did not work. https://phabricator.vyos.net/P66
I could see the responses in tcpdump but they were getting blocked (so I assume. They did not reach the running ping program)

However now that I tried to slim my config down by removing some apparently unrelated things (nat, dhcp relay, etc.) I found that I can't reproduce, nor have I been able figure out anything that I might have deleted that would restore the issue

I seem to experience the same in a VRRP setup using VyOS 1.3-rolling-202003110613. I will try to gather more information along with a sample config file.

fromport removed a subscriber: fromport.Mar 19 2020, 8:18 PM
dmbaturin set Is it a breaking change? to Unspecified (possibly destroys the router).
pasik added a subscriber: pasik.Jun 22 2020, 5:26 PM
dsummers added a subscriber: dsummers.EditedJun 23 2020, 3:24 AM

Is this related to T2619? It sure looks like it to me.

Can you replicate your problem here with the configuration in T2619 or can you replicate the problem in T2619 with the T2619 configuration?

I have tested it in a simple configuration of zone-based firewall with both Crux and Rolling and everything worked ok.

I had an interface with two IP addresses assigned to it and I could ping everywhere from the secondary address without a problem.

So, yes, it is likely related to T2619.

s.lorente removed s.lorente as the assignee of this task.Sep 10 2020, 8:16 AM
s.lorente added a subscriber: s.lorente.