Page MenuHomeVyOS Platform

Unable to apply MSS Clamp with VyOS configuration
Closed, ResolvedPublic

Description

Hello!

I need to use TCP MSS Clamp in a output interface and I'm facing a weird behaviour.

I'm using the following related rules:

leonardo@router1# show
 rule 1 {
     protocol tcp
     set {
         tcp-mss 1420
     }
     tcp {
         flags SYN
     }
 }
 rule 2 {
     protocol tcp
     set {
         tcp-mss 1420
     }
     tcp {
         flags SYN,RST
     }
 }
 rule 3 {
     protocol tcp
     set {
         tcp-mss 1420
     }
     tcp {
         flags SYN,ACK
     }
 }

This policy route is being applied to the interface, as can be confirmed using iptables-save:

# Generated by iptables-save v1.4.12.2 on Tue May 23 16:49:42 2017
*mangle
:PREROUTING ACCEPT [58274915:15858160997]
:INPUT ACCEPT [10940:1633454]
:FORWARD ACCEPT [57478793:15745212011]
:OUTPUT ACCEPT [10588:3735723]
:POSTROUTING ACCEPT [57489373:15748947030]
:VYATTA_FW_IN_HOOK - [0:0]
:VYATTA_FW_OUT_HOOK - [0:0]
:equinix-out - [0:0]
-A PREROUTING -j VYATTA_FW_IN_HOOK
-A POSTROUTING -j VYATTA_FW_OUT_HOOK
-A VYATTA_FW_IN_HOOK -i eth0.112 -j equinix-out
-A equinix-out -p tcp -m comment --comment equinix-out-1 -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1420
-A equinix-out -p tcp -m comment --comment equinix-out-2 -m tcp --tcp-flags SYN,RST SYN,RST -j TCPMSS --set-mss 1420
-A equinix-out -p tcp -m comment --comment equinix-out-3 -m tcp --tcp-flags SYN,ACK SYN,ACK -j TCPMSS --set-mss 1420
-A equinix-out -m comment --comment "equinix-out-10000 default-action accept" -j RETURN
COMMIT

With this way, an awnser for connection from internet still with MSS==1460, but if I apply the following rule, everything works as expected:

sudo iptables -t mangle -I POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o eth0.112 -j TCPMSS --set
-mss 1420

Resulting:

# Generated by iptables-save v1.4.12.2 on Tue May 23 16:49:42 2017
*mangle
:PREROUTING ACCEPT [58274915:15858160997]
:INPUT ACCEPT [10940:1633454]
:FORWARD ACCEPT [57478793:15745212011]
:OUTPUT ACCEPT [10588:3735723]
:POSTROUTING ACCEPT [57489373:15748947030]
:VYATTA_FW_IN_HOOK - [0:0]
:VYATTA_FW_OUT_HOOK - [0:0]
:equinix-out - [0:0]
-A PREROUTING -j VYATTA_FW_IN_HOOK
-A POSTROUTING -o eth0.112 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1420
-A POSTROUTING -j VYATTA_FW_OUT_HOOK
-A VYATTA_FW_IN_HOOK -i eth0.112 -j equinix-out
-A equinix-out -p tcp -m comment --comment equinix-out-1 -m tcp --tcp-flags SYN SYN -j TCPMSS --set-mss 1420
-A equinix-out -p tcp -m comment --comment equinix-out-2 -m tcp --tcp-flags SYN,RST SYN,RST -j TCPMSS --set-mss 1420
-A equinix-out -p tcp -m comment --comment equinix-out-3 -m tcp --tcp-flags SYN,ACK SYN,ACK -j TCPMSS --set-mss 1420
-A equinix-out -m comment --comment "equinix-out-10000 default-action accept" -j RETURN
COMMIT

So I tried to found a way to set the tcp flags as "SYN,RST SYN", but there is no option in VyOS allowing this, creating the argument pairing automatically from "tcp flags" statement.

Its possible to create a parameter in tcp flags statement to allow create a custom pair of mask/activeflags?

System version:

leonardo@router1:~$ show version 
Version:      VyOS 1.1.7
Description:  VyOS 1.1.7 (helium)
Copyright:    2016 VyOS maintainers and contributors
Built by:     [email protected]
Built on:     Wed Feb 17 09:57:31 UTC 2016
Build ID:     1602170957-4459750
System type:  x86 64-bit
Boot via:     image
HW model:     X10SLM-F
HW S/N:       0123456789
HW UUID:      00000000-0000-0000-0000-0CC47A4A59C0
Uptime:       16:54:53 up 42 min,  3 users,  load average: 0.03, 0.15, 0.20

Thanks!

Details

Difficulty level
Normal (likely a few hours)
Version
1.1.7
Why the issue appeared?
Design mistake
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

syncer triaged this task as Wishlist priority.Jul 30 2017, 8:54 PM
syncer edited projects, added VyOS 1.2 Crux; removed VyOS 1.1.x (1.1.8).
syncer edited subscribers, added: Maintainers, Community, Active contributors; removed: syncer.

Can someone provide more input on this,
should we implement something like that?

Hello,

Just to explain this context, usually MSS Clamping is required in PPPoE/ADSL endpoints due to PPP overhead, but for edge routers there is a non common use: DDoS mitigation clouds. They usually encapsulates IP transit inside GRE tunnel and since GRE does not have any alternative like MLPPP (Multi Line PPP) and MRRU (Maximum Receive Reconstructed Unit), the transit session MTU does need a MSS adjust to allow TCP to work well over mitigated clouds.

Thanks!

syncer added a subscriber: syncer.
syncer changed the task status from Open to On hold.Oct 13 2018, 10:53 AM
syncer edited projects, added VyOS 1.2 Crux (VyOS 1.2.0-rc4); removed VyOS 1.2 Crux.

requires testing on the latest rolling

Latest RC will work for testing? Ill try a lab here to validate the fix if its available in the RC.

dmbaturin changed the task status from On hold to Needs testing.Dec 16 2018, 4:47 PM

I see maybe the same on VyOS 1.2.0-rolling+201812162050 with MMS clamping on a GRE interface.

Config (and asigned to tun2 interface of course):

user@vyos# show policy
 route tun2-out {
     rule 100 {
         protocol tcp
         set {
             tcp-mss 1358
         }
         tcp {
             flags SYN
         }
     }
 }

Then I see on the peer VPN router the SYN packet with MSS=1460.
After this:

sudo iptables -t mangle -I POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o tun2 -j TCPMSS --set-mss 1358

I see the SYN-packet with MSS=1358

So this problem still exists but I have no clue where to add it in our source @dmbaturin @UnicronNL

syncer raised the priority of this task from Wishlist to Normal.

After some digging this is what I found out with VyOS 1.2.0-epa3:

cpo@BR1# show policy route
 route pppoe-out {
     description "PPPoE TCPMSS clamping"
     enable-default-log
     rule 100 {
         protocol tcp
         set {
             tcp-mss 1448
         }
         tcp {
             flags SYN
         }
     }
 }

cpo@BR1# show interfaces ethernet eth1 pppoe 0 policy
 route pppoe-out

Results in nice log entries

[ 1728.998011] [pppoe-out-default-A]IN=pppoe0 OUT= MAC= SRC=46.xxx.xxx.xxx DST=46.yyy.yyy.yyy LEN=1452 TOS=0x00 PREC=0x00 TTL=59 ID=44040 PROTO=UDP SPT=7701 DPT=7701 LEN=1432
[ 1728.998211] [pppoe-out-default-A]IN=pppoe0 OUT= MAC= SRC=46.xxx.xxx.xxx DST=46.yyy.yyy.yyy LEN=1452 TOS=0x00 PREC=0x00 TTL=59 ID=44041 PROTO=UDP SPT=7701 DPT=7701 LEN=1432

looks more like "input mss clamping"

cpo@BR1# sudo iptables-save -t mangle
# Generated by iptables-save v1.4.21 on Sun Feb 10 15:49:11 2019
*mangle
:PREROUTING ACCEPT [440:60658]
:INPUT ACCEPT [286:30527]
:FORWARD ACCEPT [154:30131]
:OUTPUT ACCEPT [388:94757]
:POSTROUTING ACCEPT [542:124888]
:VYATTA_FW_IN_HOOK - [0:0]
:VYATTA_FW_LOCALOUT_HOOK - [0:0]
:VYATTA_FW_OUT_HOOK - [0:0]
:pppoe-out - [0:0]
-A PREROUTING -j VYATTA_FW_IN_HOOK
-A OUTPUT -j VYATTA_FW_LOCALOUT_HOOK
-A POSTROUTING -j VYATTA_FW_OUT_HOOK
-A VYATTA_FW_IN_HOOK -i pppoe0 -j pppoe-out
-A pppoe-out -p tcp -m comment --comment pppoe-out-100 -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1448
-A pppoe-out -p tcp -m comment --comment pppoe-out-100 -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1448
-A pppoe-out -m comment --comment "pppoe-out-10000 default-action accept" -j LOG --log-prefix "[pppoe-out-default-A]"
-A pppoe-out -m comment --comment "pppoe-out-10000 default-action accept" -j RETURN
COMMIT
# Completed on Sun Feb 10 15:49:11 2019

Only the already workaround mentioned above works:

$ cat /config/scripts/vyos-postconfig-bootup.script
#!/bin/sh
# This script is executed at boot time after VyOS configuration is fully applied.
# Any modifications required to work around unfixed bugs
# or use services not available through the VyOS CLI system can be placed here.

sudo iptables -t mangle -I POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o pppoe0 -j TCPMSS --set-mss 1452

Ensure that the file has executable permission chmod +x /config/scripts/vyos-postconfig-bootup.script

A real solution needs to be implemented together with the firewall rewrite

c-po changed the task status from Needs testing to Confirmed.Feb 10 2019, 2:54 PM
c-po removed c-po as the assignee of this task.
syncer raised the priority of this task from Normal to High.Apr 17 2019, 9:07 PM
syncer edited projects, added VyOS 1.3 Equuleus; removed VyOS 1.2 Crux (VyOS 1.2.2).

Proposing a Cisco like interface:

  • set interface ethernet eth0 ip tcp adjust-mss <bytes>

EdgeOS uses this node.def file:

IPv4

$ cat /opt/vyatta/share/vyatta-cfg/templates/firewall/options/mss-clamp/node.def
help: Enable IPv4 TCP MSS clamping for specified interface types

end:
    cname=UBNT_FW_MSS_CLAMP
    mss='$VAR(mss/@)'
    types='$VAR(interface-type/@@)'
    if [ -z "$types" ]; then
        types='pppoe pptp tun vti'
    fi
    sudo iptables -t mangle -D FORWARD -j $cname >&/dev/null
    sudo iptables -t mangle -F $cname >&/dev/null
    sudo iptables -t mangle -X $cname >&/dev/null
    if [ "$COMMIT_ACTION" != DELETE ]; then
        sudo iptables -t mangle -N $cname
        sudo iptables -t mangle -A FORWARD -j $cname
        for intf in $types; do
            if [ "$intf" == all ]; then
                sudo iptables -t mangle -A $cname -p tcp \
                    --tcp-flags SYN,RST SYN -j TCPMSS --set-mss $mss
                break
            fi
            sudo iptables -t mangle -A $cname -o ${intf}+ -p tcp \
                --tcp-flags SYN,RST SYN -j TCPMSS --set-mss $mss
            sudo iptables -t mangle -A $cname -i ${intf}+ -p tcp \
                --tcp-flags SYN,RST SYN -j TCPMSS --set-mss $mss
        done
    fi

$ cat /opt/vyatta/share/vyatta-cfg/templates/firewall/options/mss-clamp/mss/node.def
type: u32
default: 1412
syntax:expression: $VAR(@) >= 536 && $VAR(@) <= 1492; \
    "MSS must be between 536 and 1492"
help: MSS value

IPv6

$ cat /opt/vyatta/share/vyatta-cfg/templates/firewall/options/mss-clamp6/node.def
help: Enable IPv6 TCP MSS clamping for specified interface types

end:
    cname=UBNT_FW_MSS_CLAMP
    mss='$VAR(mss/@)'
    types='$VAR(interface-type/@@)'
    if [ -z "$types" ]; then
        types='pppoe pptp tun'
    fi
    if [ -d /proc/sys/net/ipv6 ]; then
        sudo ip6tables -t mangle -D FORWARD -j $cname >&/dev/null
        sudo ip6tables -t mangle -F $cname >&/dev/null
        sudo ip6tables -t mangle -X $cname >&/dev/null
    fi
    if [ "$COMMIT_ACTION" != DELETE ]; then
        if [ -d /proc/sys/net/ipv6 ]; then
            sudo ip6tables -t mangle -N $cname
            sudo ip6tables -t mangle -A FORWARD -j $cname
            for intf in $types; do
                if [ "$intf" == all ]; then
                    sudo ip6tables -t mangle -A $cname -p tcp \
                        --tcp-flags SYN,RST SYN -j TCPMSS --set-mss $mss
                    break
                fi
                sudo ip6tables -t mangle -A $cname -o ${intf}+ -p tcp \
                    --tcp-flags SYN,RST SYN -j TCPMSS --set-mss $mss
                sudo ip6tables -t mangle -A $cname -i ${intf}+ -p tcp \
                    --tcp-flags SYN,RST SYN -j TCPMSS --set-mss $mss
            done
       fi
    fi

$ cat /opt/vyatta/share/vyatta-cfg/templates/firewall/options/mss-clamp6/mss/node.def
type: u32
default: 1412
syntax:expression: $VAR(@) >= 1280 && $VAR(@) <= 1492; \
    "MSS must be between 1280 and 1492"
help: MSS value

The new syntax will be:

clamp MSS IPv4

set firewall options interface pppoe0 adjust-mss '1452'

clamp MSS IPv6

set firewall options interface pppoe0 adjust-mss6 '1452'

disable entire rule

set firewall options interface pppoe0 disable

Output

 sudo iptables-save -t mangle
# Generated by iptables-save v1.4.21 on Sun Apr 21 12:56:25 2019
*mangle
:PREROUTING ACCEPT [1217:439885]
:INPUT ACCEPT [290:52459]
:FORWARD ACCEPT [920:375774]
:OUTPUT ACCEPT [301:100053]
:POSTROUTING ACCEPT [1221:475827]
:VYOS_FW_OPTIONS - [0:0]
-A FORWARD -j VYOS_FW_OPTIONS
-A VYOS_FW_OPTIONS -o pppoe0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1452
-A VYOS_FW_OPTIONS -o wg02 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1352
-A VYOS_FW_OPTIONS -o wg05 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1352
COMMIT
# Completed on Sun Apr 21 12:56:25 2019
c-po moved this task from Needs Triage to Finished on the VyOS 1.2 Crux (VyOS 1.2.2) board.
c-po moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus board.

@c-po thanks for that. I changed my configs from postconfig script to new config syntax

dmbaturin set Is it a breaking change? to Unspecified (possibly destroys the router).
dmbaturin set Issue type to Bug (incorrect behavior).