Page MenuHomePhabricator

Firewall rulesets are ignored in RFC-compliant VRRP setups
In progress, HighPublicBUG

Description

I have 2 routers running VyOS 1.2.0RC11.

They each have 3 connected interfaces.

  • eth0 for WAN
  • eth1 for LAN
  • eth5 for VRRP messaging

Both routers are paired thought interface eth0v1 and eth1v2 with VRRP and are grouped in ALPHA group.

Everything works fine and as expected on the VRRP side of things.

I applied my WAN-in firewall ruleset on eth0 IN on both routers.

But nothing gets filtered. It's as if there is no firewall at all.

I could not find a way to apply my rule set on eth0v1 since VyOS tells me it is not a valid interface.

The firewall statistics shows 0 on all rules.

This firewall setup works no problem if I'm running just one router without VRRP
C

Details

Difficulty level
Easy (less than an hour)
Version
1.2.0
Why the issue appeared?
Implementation mistake

Event Timeline

jmlccdmd created this task.Jan 18 2019, 8:26 PM
syncer changed the task status from Open to In progress.Jan 20 2019, 11:21 AM
syncer triaged this task as High priority.
syncer edited projects, added VyOS 1.2 Crux (VyOS 1.2.0-GA); removed VyOS 1.2 Crux.
syncer moved this task from Need Triage to In Progress on the VyOS 1.2 Crux board.
Merijn added a subscriber: Merijn.EditedJan 21 2019, 9:37 AM

@jmlccdmd
I have recreated your setup with Vyos 1.2.0-rc10 and it seems to be working correctly

firewall {
     all-ping enable
     broadcast-ping disable
     config-trap disable
     ipv6-receive-redirects disable
     ipv6-src-route disable
     ip-src-route disable
     log-martians enable
     name ingress-filter {
         default-action drop
         enable-default-log
         rule 1 {
             action accept
             destination {
                 address 192.168.60.0/24
                 port 22
             }
             protocol tcp
         }
         rule 2 {
             action accept
             protocol icmp
         }
     }
     name local-filter {
         default-action drop
         enable-default-log
         rule 1 {
             action accept
             destination {
                 port 22
             }
             protocol tcp
         }
         rule 2 {
             action accept
             protocol icmp
         }
     }
     receive-redirects disable
     send-redirects enable
     source-validation disable
     syn-cookies enable
     twa-hazards-protection disable
 }
 high-availability {
     vrrp {
         group Test {
             interface eth0
             priority 120
             virtual-address 10.3.18.243/32
             vrid 10
         }
         group Test-Intern {
             interface eth1
             priority 120
             virtual-address 192.168.60.254/32
             vrid 20
         }
     }
 }
 interfaces {
     ethernet eth0 {
         address 10.3.18.244/24
         duplex auto
         firewall {
             in {
                 name ingress-filter
             }
             local {
                 name local-filter
             }
         }
         hw-id 08:00:27:18:7f:cd
         smp-affinity auto
         speed auto
     }
     ethernet eth1 {
         address 192.168.60.244/24
         hw-id 08:00:27:9f:77:df
     }
     loopback lo {
         address 192.168.10.1/24
     }
 }
 nat {
     destination {
         rule 1 {
             destination {
                 address 10.3.18.243
                 port 2222
             }
             inbound-interface eth0
             protocol tcp
             translation {
                 address 192.168.60.11
                 port 22
             }
         }
     }
     source {
         rule 1 {
             destination {
                 address 0.0.0.0/0
             }
             outbound-interface eth0
             source {
                 address 192.168.60.0/24
             }
             translation {
                 address 10.3.18.243
             }
         }
     }
 }

Firewall counters increase when doing SSH to the internal system for instance, which is applied to the virtual-address.

eth0           10.3.18.244/24                 u/u
                 10.3.18.243/32
eth1           192.168.60.244/24           u/u
                 192.168.60.254/32
pasik added a subscriber: pasik.Jan 21 2019, 9:40 PM
hagbard assigned this task to Merijn.Jan 21 2019, 10:11 PM

OK,

With the exact same setup, I diabled vrrp on my second routers, the one in standby, with these commands :

delete high-availability
delete service conntrack-sync

I assined the virtual IPs to the physical interfaces of that router and I removed from the network the fisrt router that remained unchanged.

Instantly, it started to work properly.

Trafic is now filtered and "show firewall statistics" show numbers instead of zeros.

If yout need more infos, don't hesitate.

But if you run only on the first router, including the VRRP setup it does not work?

No, it does not work. The problem persist.

@jmlccdmd
I added a second router and configured conntrack-sync.
Failover and preempt failback works correct.
Both routers show statistics for the firewall rules

show firewall name ingress-filter statistics

------------------------
Firewall Global Settings
------------------------

Firewall state-policy for all IPv4 and Ipv6 traffic

state           action   log
-----           ------   ---
established     accept   disabled

-----------------------------
Rulesets Information
-----------------------------

IPv4 Firewall "ingress-filter":

 Active on (eth0,IN)

rule  packets   bytes     action  source              destination
----  -------   -----     ------  ------              -----------
1     2         152       ACCEPT  0.0.0.0/0           192.168.60.0/24
10000 0         0         DROP    0.0.0.0/0           0.0.0.0/0

Also when i connect to a port that is not allowed the active router has a log entry of the deny action.

Have you tried with the config i added above? SInce you mention eth0v1 interface, but in the new setup with Vyos 1.2.0 these do not appear.

dmbaturin renamed this task from Firewall rule set ignored in VRRP setup to Firewall rulesets are ignored in RFC-compliant VRRP setups.Jan 26 2019, 1:08 AM
dmbaturin changed Difficulty level from Unknown (require assessment) to Easy (less than an hour).
dmbaturin changed Why the issue appeared? from Will be filled on close to Implementation mistake.

This problem is specific to RFC-compliant VRRP setups. Firewall design in VyOS is rather unfortunate in that rulesets are bound to interfaces. If you assign it to eth0, a rule with -i eth0 -j MyRuleset is created. RFC-compliant (shared MAC) VRRP uses those eth0v1 etc. interfaces, but since from netfilter's point of view eth0 and eth0v1 are different interfaces, those rules are never reached.

Until we redesign the firewall CLI, I'm making the rules match eth0+ instead. I hope the performance impact will not be too high.

dmbaturin changed the task status from In progress to Needs testing.Jan 26 2019, 2:05 AM

Ok, more interesting than that. In the latest image, the setup just works as described with RFC-compliant VRRP:

vyos@vyos-test-2# show firewall name Quux 
 default-action accept
 rule 10 {
     action reject
     destination {
         address 10.217.32.183
     }
 }

vyos@vyos-test-2# show high-availability vrrp group Bar 
 interface eth0
 rfc3768-compatibility
 virtual-address 10.217.32.183/24
 vrid 20

vyos@vyos-test-2# show interfaces ethernet eth0
 address 10.217.32.173/24
 duplex auto
 firewall {
     local {
         name Quux
     }
 }
 hw-id 00:50:56:9b:05:00
 smp-affinity auto
 speed auto

vyos@vyos-test-1# ping 10.217.32.183
PING 10.217.32.183 (10.217.32.183) 56(84) bytes of data.
From 10.217.32.183 icmp_seq=1 Destination Port Unreachable
From 10.217.32.183 icmp_seq=2 Destination Port Unreachable

Removing the firewall restores connectivity. I suppose the issue has fixed itself, due to a newer kernel perhaps.

I'm very interested, what is the latest image number? I will test it.

@jmlccdmd I've been testing it with EPA3 and friday's nightly build.

runar added a subscriber: runar.Jan 26 2019, 4:17 PM

Until we redesign the firewall CLI, I'm making the rules match eth0+ instead. I hope the performance impact will not be too high.

Hmm, just to be sure.. if you have 15 eth interfaces on your device, and apply vrrp to eth1.. will this make rules on eth10-14 match aswell?

syncer closed this task as Resolved.Jan 26 2019, 10:14 PM
syncer added a project: VyOS-1.2.0-GA.
jmlccdmd changed the task status from Resolved to Wontfix.EditedJan 29 2019, 5:21 PM

On my systems, the problem persist with today's rolling release.

But if I disable rfc3768-compatibility, it works correctly.

@dmbaturin, in your test from last friday, you did one mistake that made you believe the issue has fixed itself.

You applied your firewall rule to the "local" section of the eth0 interface. The problem does not occure on that section of the interface's firewall, only on the in and out sections.

In my setup, I too can see firewall filtering traffic on the eth0-local. The problem is with the eth0-in where the traffic is flowing unfiltered because of the rfc compatibility.

Redo your tests with :

vyos@vyos-test-2# show firewall name Quux 
 default-action accept
 rule 10 {
     action reject
     destination {
         address 10.217.32.183
     }
 }

vyos@vyos-test-2# show high-availability vrrp group Bar 
 interface eth0
 rfc3768-compatibility
 virtual-address 10.217.32.183/24
 vrid 20

vyos@vyos-test-2# show interfaces ethernet eth0
 address 10.217.32.173/24
 duplex auto
 firewall {
     in {
         name Quux
     }
 }
 hw-id 00:50:56:9b:05:00
 smp-affinity auto
 speed auto

The traffic will flow, but the firewall rules will be ignored.

It really is necessary to disable the rfc-compat. Under Vyatta vRouter 5400 6.7R13, the one I used to use, this problem does not occure. FYI.

So the issue is not fixed.

jmlccdmd reopened this task as Open.Jan 30 2019, 1:24 PM

I reopen this bug.

@jmlccdmd Ok, I'll re-test with in/out then.

rps added a subscriber: rps.Feb 4 2019, 8:34 PM

My fault for not having the time to test this as one of the users who has a need for RFC compliant VRRP. The use of + for interface matching is less than ideal but if we do so we should take care to recommend that use of 802.1Q VLAN sub-interfaces not make use of the parent (untagged) interface else traffic matching will not be obvious.

Other than potential collision between eth0+ and eth0.100+ I don't think there would be anything else to watch out for.

syncer closed this task as Resolved.Feb 5 2019, 2:13 PM
syncer reopened this task as In progress.
syncer moved this task from Needs Triage to In Progress on the VyOS 1.2 Crux (VyOS 1.2.2) board.
zsdc reassigned this task from Merijn to dmbaturin.Jul 10 2019, 1:39 PM
zsdc added a subscriber: zsdc.

Hello, all!
I have prepared the pull requests for fixing this bug. They add hooks for two situations:

  • if VRRP configuration changed;
  • if firewall settings for interface changed.

Internal logic is next:

  1. Get the list of currently configured VRRP groups with rfc3768-compatibility flag.
  2. Get the list of rules for parent interfaces of this VRRP groups.
  3. Transform this list into the list for VMAC interfaces.
  4. Add firewall rules for configured VMAC interfaces.
  5. Get the list of active rules for all VMAC interfaces.
  6. Check the list of active rules with VMAC interfaces for matching with configured VRRP groups and parent interfaces. Delete those, who's don't match.

PR:
https://github.com/vyos/vyatta-cfg-firewall/pull/16
https://github.com/vyos/vyos-1x/pull/87

@dmbaturin, check please if all is correct.