Page MenuHomeVyOS Platform

VXLAN uses ttl 1 (auto) by default
Closed, ResolvedPublicBUG

Description

In 1.3/1.4 multicast vxlan behavior was changed. That means that does not possible to route multicast packet with TTL 1 and vxlan multicast can work only in L2 segment.
In 1.2.x by default used TTL = 16

vyos@vyos:~$ sudo ip -s -d link show dev vxlan241
9: vxlan241: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br241 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 3e:1f:d6:b3:5e:13 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65535 
    vxlan id 241 group 239.0.0.241 dev eth0 srcport 0 0 dstport 8472 ttl 16 ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx 
    bridge_slave state forwarding priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_is
    RX: bytes  packets  errors  dropped overrun mcast   
    2250       28       0       0       0       0       
    TX: bytes  packets  errors  dropped carrier collsns 
    2898       35       0       0       0       0

Output from 1.3

vyos@vyos# sudo ip -s -d link show dev vxlan241
13: vxlan241: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br241 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether ae:9f:50:04:37:1f brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65535 
    vxlan id 241 group 239.0.0.241 dev eth0 srcport 0 0 dstport 8472 ttl auto ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx 
    bridge_slave state forwarding priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on port_id 0x8002 port_no 0x2 designated_port 32770 designated_cost 0 designated_bridge 8000.50:0:0:2s
    RX: bytes  packets  errors  dropped overrun mcast   
    470        6        0       0       0       0       
    TX: bytes  packets  errors  dropped carrier collsns 
    1378       21       0       0       0       0

Details

Difficulty level
Easy (less than an hour)
Version
1.3-beta-202102160443
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

As a solution proposed to use TTL by default equivalent 16, but also add the possibility to change it via VyOS CLI

set interfaces vxlan vxlan241 ttl 16

Please retest with the latest 1.4 rolling version. TTL can now be set.

@c-po does not work on 1.4-rolling-202103040218

vyos@vyos# sudo ip -d link show dev vxlan241
7: vxlan241: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br241 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fe:08:e3:3c:d4:ab brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65535 
    vxlan id 241 group 239.0.0.241 dev eth0 srcport 0 0 dstport 8472 tos inherit ttl auto ageing 300 udpcsum noudp6zerocsumtx noudp6zerocsumrx

On the middle router in traffic dump I see TTL=1

18:59:29.029090 IP (tos 0x0, ttl 1, id 24806, offset 0, flags [none], proto UDP (17), length 100)
    10.1.2.2.52948 > 239.0.0.241.8472: OTV, flags [I] (0x08), overlay 0, instance 241

VyOS 1.2 has this hardcoded: /opt/vyatta/share/vyatta-cfg/templates/interfaces/vxlan/node.def: VXLAN_TTL="ttl 16"

Can you manually change the ttl to 16 by set interfaces vxlan vxlan0 parameters ip ttl 16 if this works we have to add a migrator for older configs then.

c-po changed the task status from Open to In progress.Mar 4 2021, 8:40 PM
c-po claimed this task.

@c-po , yes now it works. Maybe we need to define ttl=16 as the default value?

@Dmitry

I know my opinion is....really not that important but I would *highly* recommend going to maximum TTL of 255 or at minimum 127. TTL is a very hard thing to troubleshoot most of the time and therefore it's almost never worth going lower than maximum for IP TTL.

I wonder why it is 1 by default now - we actually specify 0 which means "take it from the inbound packet". Nevertheless we need a migrator to mimic existing behavior vor users upgrading from 1.2 -> 1.3, so the migrator should simply insert ttl = 16 into the CLI.

@c-po, using a value of "0" for use inbound packet would actually be the best behavior if we can specify/use that.

@Cheeze_It that's the new "normal".

@Dmitry in your output above I see ttl auto for 1.3 which is ttl = 0 - is this a real bug or is it just out of curiosity?

For VXLAN multicast this is a real bug.

SrividyaA set Issue type to Bug (incorrect behavior).Aug 31 2021, 12:19 PM