Page MenuHomeVyOS Platform

Nat translation many-many with prefix does not map 1-1.
Closed, ResolvedPublicBUG

Description

When applying a destination nat translation from e.g. 10.132.0.0/16 to 172.1.0.0/16, an address 10.132.20.3 does not map to 172.1.20.3 as expected, but picks a random address in the range.

I have a VPN between ASA and vyos1.3 here. The idea is to have a virtual IP range that is translated to the ASA networks range. 11.60.0.0/16 => 10.1.0.0/16

When I do a ping from PC3 to PC4 as:

ping 11.60.1.4

The nat translation table in the vyos shows as:

vyos@vyos:~$ show nat destination translations

Pre-NATPost-NATProtTimeout
11.60.1.410.1.44.195icmp27
11.60.1.410.1.44.195icmp29
11.60.1.410.1.44.195icmp25
11.60.1.410.1.44.195icmp23

This does this random mapping on 1.3 and works correctly on 1.2

Relevant nat config:

set nat destination rule 200 destination address '11.60.0.0/16'
set nat destination rule 200 inbound-interface 'eth0'
set nat destination rule 200 source address '10.2.0.0/16'
set nat destination rule 200 translation address '10.1.0.0/16'

Here's the full config on the vyos:

set firewall all-ping 'enable'
set firewall broadcast-ping 'disable'
set firewall config-trap 'disable'
set firewall group network-group PrivateIPs network '10.2.0.0/16'
set firewall group network-group RecIPs network '10.2.2.3/32'
set firewall group network-group TapLan network '10.1.0.0/16'
set firewall ipv6-receive-redirects 'disable'
set firewall ipv6-src-route 'disable'
set firewall ip-src-route 'disable'
set firewall log-martians 'enable'
set firewall name Accept_Log default-action 'accept'
set firewall name Accept_Log enable-default-log
set firewall name Tap_In default-action 'accept'
set firewall name Tap_In enable-default-log
set firewall name Tap_In rule 100 action 'accept'
set firewall name Tap_In rule 100 destination group network-group 'RecIPs'
set firewall name Tap_In rule 100 log 'enable'
set firewall name Tap_In rule 100 protocol 'tcp_udp'
set firewall name Tap_In rule 100 source group network-group 'TapLan'
set firewall name Tap_In rule 200 action 'accept'
set firewall name Tap_In rule 200 destination group network-group 'RecIPs'
set firewall name Tap_In rule 200 log 'enable'
set firewall name Tap_In rule 200 protocol 'icmp'
set firewall name Tap_In rule 800 action 'reject'
set firewall name Tap_In rule 800 description 'No access to private IPs'
set firewall name Tap_In rule 800 destination group network-group 'PrivateIPs'
set firewall name Tap_In rule 800 log 'enable'
set firewall receive-redirects 'disable'
set firewall send-redirects 'enable'
set firewall source-validation 'disable'
set firewall state-policy established action 'accept'
set firewall state-policy invalid action 'drop'
set firewall state-policy invalid log enable
set firewall state-policy related action 'accept'
set firewall syn-cookies 'enable'
set firewall twa-hazards-protection 'disable'

set interfaces ethernet eth0 address '10.2.2.1/16'
set interfaces ethernet eth0 description 'inside'
set interfaces ethernet eth0 duplex 'auto'
set interfaces ethernet eth0 smp-affinity 'auto'
set interfaces ethernet eth0 speed 'auto'
set interfaces ethernet eth1 address '100.100.100.2/30'
set interfaces ethernet eth1 description 'outside'
set interfaces ethernet eth1 duplex 'auto'
set interfaces ethernet eth1 smp-affinity 'auto'
set interfaces ethernet eth1 speed 'auto'
set interfaces ethernet eth2 duplex 'auto'
set interfaces ethernet eth2 smp-affinity 'auto'
set interfaces ethernet eth2 speed 'auto'
set interfaces ethernet eth3 duplex 'auto'
set interfaces ethernet eth3 smp-affinity 'auto'
set interfaces ethernet eth3 speed 'auto'
set interfaces loopback lo
set interfaces vti vti0 address '192.168.13.3/24'
set interfaces vti vti0 firewall in name 'Tap_In'
set interfaces vti vti0 firewall out name 'Accept_Log'
set interfaces vti vti0 ip
set nat destination rule 200 destination address '11.60.0.0/16'
set nat destination rule 200 inbound-interface 'eth0'
set nat destination rule 200 source address '10.2.0.0/16'
set nat destination rule 200 translation address '10.1.0.0/16'
set nat destination rule 505 destination address '172.17.2.0/28'
set nat destination rule 505 inbound-interface 'vti0'
set nat destination rule 505 source address '10.1.0.0/16'
set nat destination rule 505 translation address '10.2.2.3'

set nat destination rule 8100 destination address '172.17.1.8'
set nat destination rule 8100 inbound-interface 'vti0'
set nat destination rule 8100 source address '10.1.0.0/16'
set nat destination rule 8100 translation address '10.2.2.3'
set nat source rule 100 destination address '10.1.0.0/16'
set nat source rule 100 outbound-interface 'vti0'
set nat source rule 100 protocol 'all'
set nat source rule 100 translation address 'masquerade'
set nat source rule 1000 outbound-interface 'eth1'
set nat source rule 1000 protocol 'all'
set nat source rule 1000 source address '10.2.0.0/16'
set nat source rule 1000 translation address 'masquerade'
set protocols bgp 200 address-family ipv4-unicast network 172.17.1.0/28
set protocols bgp 200 neighbor 192.168.13.4 address-family ipv4-unicast soft-reconfiguration inbound
set protocols bgp 200 neighbor 192.168.13.4 remote-as '100'
set protocols bgp 200 neighbor 192.168.13.4 update-source '192.168.13.3'
set protocols static interface-route 10.1.0.0/16 next-hop-interface vti0
set protocols static route 0.0.0.0/0 next-hop 100.100.100.1

set vpn ipsec esp-group espA compression 'disable'
set vpn ipsec esp-group espA lifetime '48800'
set vpn ipsec esp-group espA mode 'tunnel'
set vpn ipsec esp-group espA pfs 'dh-group14'
set vpn ipsec esp-group espA proposal 1 encryption 'aes256'
set vpn ipsec esp-group espA proposal 1 hash 'sha512'
set vpn ipsec ike-group ikeA close-action 'none'
set vpn ipsec ike-group ikeA dead-peer-detection action 'restart'
set vpn ipsec ike-group ikeA dead-peer-detection interval '10'
set vpn ipsec ike-group ikeA dead-peer-detection timeout '120'
set vpn ipsec ike-group ikeA ikev2-reauth 'no'
set vpn ipsec ike-group ikeA key-exchange 'ikev2'
set vpn ipsec ike-group ikeA lifetime '86400'
set vpn ipsec ike-group ikeA proposal 1 dh-group '14'
set vpn ipsec ike-group ikeA proposal 1 encryption 'aes256'
set vpn ipsec ike-group ikeA proposal 1 hash 'sha512'
set vpn ipsec ipsec-interfaces interface 'eth1'
set vpn ipsec logging log-level '1'
set vpn ipsec logging log-modes 'any'
set vpn ipsec site-to-site peer 200.200.200.2 authentication mode 'pre-shared-secret'
set vpn ipsec site-to-site peer 200.200.200.2 authentication pre-shared-secret 'testkey'
set vpn ipsec site-to-site peer 200.200.200.2 connection-type 'initiate'
set vpn ipsec site-to-site peer 200.200.200.2 default-esp-group 'espA'
set vpn ipsec site-to-site peer 200.200.200.2 ike-group 'ikeA'
set vpn ipsec site-to-site peer 200.200.200.2 ikev2-reauth 'inherit'
set vpn ipsec site-to-site peer 200.200.200.2 local-address '100.100.100.2'
set vpn ipsec site-to-site peer 200.200.200.2 vti bind 'vti0'
set vpn ipsec site-to-site peer 200.200.200.2 vti esp-group 'espA'

Version:

Version:          VyOS 1.3-rolling-202009241556
Release Train:    equuleus

Built by:        redacted
Built on:         Thu 24 Sep 2020 15:56 UTC
Build UUID:       32528dfd-734e-40e5-aeac-32ea171c9722
Build Commit ID:  d571b383797719

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (i440FX + PIIX, 1996)
Hardware S/N:     Unknown
Hardware UUID:    Unknown

Copyright:        VyOS maintainers and contributors

Details

Difficulty level
Normal (likely a few hours)
Version
1.3 current
Why the issue appeared?
Design mistake
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Event Timeline

This one is holding us back from some great 1.3 features... would love to get it looked at!

The root cause here is that there is yet no nftables map support in our template.

With recent additions with our custom jinja2 filters this should be fairly easy.

You think you can come up with an nftables example map for is how it should be?

c-po changed the task status from Open to Confirmed.Nov 27 2020, 9:31 PM
c-po triaged this task as Normal priority.
c-po changed Difficulty level from Unknown (require assessment) to Normal (likely a few hours).
c-po changed Why the issue appeared? from Will be filled on close to Design mistake.
c-po changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.

The command works on the experimental Kernel 5.9.9 VyOS ISO, but not using a 4.19 series Kernel. looks like it's not yet supported in nftables.

sudo nft add rule ip nat POSTROUTING oifname "pppoe0" snat ip prefix to ip saddr map { 10.141.11.0/24 : 192.168.2.0/24 }

Even on Kernel 5.4 this is not supported.

vyos@vyos:~$ uname -a
Linux LR1.wue3 5.4.80-amd64-vyos #1 SMP Sat Nov 28 21:16:40 UTC 2020 x86_64 GNU/Linux

vyos@vyos:~$ sudo nft add rule nat POSTROUTING  oifname "pppoe0" ip prefix saddr  map { 10.141.11.0/24 : 192.168.2.0/24 }
Error: syntax error, unexpected prefix
add rule nat POSTROUTING oifname pppoe0 ip prefix saddr map { 10.141.11.0/24 : 192.168.2.0/24 }
                                           ^^^^^^

On 1.3-rolling-202101061750 has changed to kernel 5.10,

Following command works and NATs to the correct host: nft insert rule nat PREROUTING dnat ip prefix to ip daddr map { 10.0.233.0/24 : 192.168.101.0/24 }

vyos@vyos:~$ sudo nft insert rule nat PREROUTING dnat ip prefix to ip daddr map { 10.0.233.0/24 : 192.168.101.0/24 }

vyos@vyos:~$ sudo nft list table nat
table ip nat {

chain PREROUTING {
        type nat hook prerouting priority dstnat; policy accept;
        dnat ip prefix to ip daddr map { 10.0.233.0/24 : 192.168.101.0/24 }
}

...

c-po changed the task status from Confirmed to Needs testing.Jan 19 2021, 8:04 PM

Implemented on the 1.4 current branch, once the QAT issues are resolved for 5.10 Linux Kernel this can be backported after successful testing to 1.3 equuleus LTS branch.

@c-po Why not consider dnat prefix to? Maybe it's more convenient

Feature was present in 1.2 and until now not possible with nftables, so we keep the old CLI and just replace the engine for now. Adding the prefix option simply feels more complicated in terms of validation logic and scripting.

c-po changed the task status from Needs testing to Backport pending.Jan 31 2021, 12:43 PM

It seems it works now

sudo nft  add rule ip nat POSTROUTING oifname "pppoe0" snat ip prefix to ip saddr map { 10.141.11.0/24 : 192.168.2.0/24 }
root@r4-roll:/home/vyos# sudo nft --debug=netlink add rule ip nat POSTROUTING oifname "pppoe0" snat ip prefix to ip saddr map { 10.141.11.0/24 : 192.168.2.0/24 }
__map%d nat f
__map%d nat 0
	element 00000000  : 1 [end]	element 000b8d0a  : 0002a8c0 ff02a8c0 0 [end]	element 000c8d0a  : 1 [end]
ip 
  [ meta load oifname => reg 1 ]
  [ cmp eq reg 1 0x6f707070 0x00003065 0x00000000 0x00000000 ]
  [ payload load 4b @ network header + 12 => reg 1 ]
  [ lookup reg 1 set __map%d dreg 1 ]
  [ nat snat ip addr_min reg 1 addr_max reg 9 flags 0x40 ]

root@r4-roll:/home/vyos#

Backport is complete - It will "work" once the 1.3 kernel is updated to 5.10 series

c-po changed the task status from Backport pending to On hold.Feb 27 2021, 11:59 PM

Since we cannot update the kernel in 1.3 due to the QAT issues, we likely need to backport netmap support. It seems to have been introduced by this commit: https://github.com/torvalds/linux/commit/3ff7ddb1353da

erkin set Issue type to Bug (incorrect behavior).Sun, Aug 29, 12:46 PM
erkin removed a subscriber: Active contributors.
c-po removed projects: test, VyOS 1.3 Equuleus.