Page MenuHomeVyOS Platform
Create Task
Maniphest T4729

VxLAN does not work and deleted after tun changed
Confirmed, NormalPublicBUG
Actions

Description

VxLAN does not work after tunnel is changed.
There is the topology

topology.png (474×657 px, 23 KB)

VyOS-1 configuration:

set interfaces bridge br0 member interface eth1
set interfaces bridge br0 member interface vxlan0
set interfaces ethernet eth0 address '192.0.2.2/30'
set interfaces ethernet eth0 description 'WAN'
set interfaces ethernet eth1 description 'Lan'
set interfaces tunnel tun0 address '10.0.0.2/24'
set interfaces tunnel tun0 description 'Gretap'
set interfaces tunnel tun0 encapsulation 'gretap'
set interfaces tunnel tun0 mtu '1500'
set interfaces tunnel tun0 multicast 'enable'
set interfaces tunnel tun0 parameters ip ignore-df
set interfaces tunnel tun0 parameters ip key '1'
set interfaces tunnel tun0 parameters ip no-pmtu-discovery
set interfaces tunnel tun0 parameters ip ttl '0'
set interfaces tunnel tun0 remote '203.0.113.2'
set interfaces tunnel tun0 source-address '192.0.2.2'
set interfaces vxlan vxlan0 group '239.0.0.241'
set interfaces vxlan vxlan0 mtu '1426'
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 source-interface 'tun0'
set interfaces vxlan vxlan0 vni '123'

VyOS-2 configuration:

set interfaces bridge br0 member interface eth1
set interfaces bridge br0 member interface vxlan0
set interfaces ethernet eth0 address '203.0.113.2/30'
set interfaces ethernet eth0 description 'WAN'
set interfaces ethernet eth1 description 'Lan'
set interfaces tunnel tun0 address '10.0.0.1/24'
set interfaces tunnel tun0 encapsulation 'gretap'
set interfaces tunnel tun0 mtu '1500'
set interfaces tunnel tun0 multicast 'enable'
set interfaces tunnel tun0 parameters ip ignore-df
set interfaces tunnel tun0 parameters ip key '1'
set interfaces tunnel tun0 parameters ip no-pmtu-discovery
set interfaces tunnel tun0 parameters ip ttl '0'
set interfaces tunnel tun0 remote '192.0.2.2'
set interfaces tunnel tun0 source-address '203.0.113.2'
set interfaces vxlan vxlan0 group '239.0.0.241'
set interfaces vxlan vxlan0 mtu '1426'
set interfaces vxlan vxlan0 port '4789'
set interfaces vxlan vxlan0 source-interface 'tun0'
set interfaces vxlan vxlan0 vni '123'

Ping from cl-left => cl-right

vyos@c-left:~$ ping  100.64.0.2 count 1
PING 100.64.0.2 (100.64.0.2) 56(84) bytes of data.
64 bytes from 100.64.0.2: icmp_seq=1 ttl=64 time=1.57 ms

--- 100.64.0.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.565/1.565/1.565/0.000 ms
vyos@c-left:~$

Delete key from tunnel tun0 on both routers or add tunnel key if it doesn't exist on both routers

delete interfaces tunnel tun0 parameters ip key
commit

After this VxLAN does not work, the same ping from cl-left

vyos@c-left:~$ ping  100.64.0.2 count 1
PING 100.64.0.2 (100.64.0.2) 56(84) bytes of data.

--- 100.64.0.2 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

vyos@c-left:~$

And GRE tunnels can ping each other at the same time without any issues. VxLAN does not work.
After rebooting routers, it works again.

NOTE after changing GRE key, VxLAN interface deleted for some reason

vyos@R1# show interfaces vxlan 
 vxlan vxlan0 {
     group 239.0.0.241
     mtu 1426
     port 4789
     source-interface tun0
     vni 123
 }
[edit]
vyos@R1# sudo ip link show type vxlan
[edit]
vyos@R1# 

`

Details

Difficulty level
Normal (likely a few hours)
Version
VyOS 1.4-rolling-202210020218
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Related Objects

Event Timeline

Viacheslav created this task.Oct 4 2022, 8:44 AM
Viacheslav updated the task description. (Show Details)
Viacheslav renamed this task from VxLAN does not work after tun changed to VxLAN does not work and deleted after tun changed.Oct 4 2022, 8:49 AM
Viacheslav updated the task description. (Show Details)
Viacheslav updated the task description. (Show Details)Oct 4 2022, 8:54 AM
pasik added a subscriber: pasik.Oct 4 2022, 9:37 AM
pasik added a comment.Oct 4 2022, 6:28 PM

Hmm, any specific reason for the tun0 encapsulation 'gretap' ? did you try with normal 'gre' tunnels ? Does it change anything?

Viacheslav added a comment.Oct 4 2022, 11:36 PM
In T4729#135221, @pasik wrote:

Hmm, any specific reason for the tun0 encapsulation 'gretap' ? did you try with normal 'gre' tunnels ? Does it change anything?

I guess it does not matter as tunnel per change (as I remember) deleted and created again
At this step (delete) it affected as it (gre interface tun0) is part of vxlan config and kernel just drop this vxlan interface
I can be wrong

pasik added a comment.Oct 5 2022, 7:33 AM

well, "gre" and "gretap" are different types of tunnels, with different features.. so it makes sense to test and validate with the normal "gre", as in your config I don't see a need for "gretap".

Viacheslav added a comment.Oct 5 2022, 2:42 PM
In T4729#135223, @pasik wrote:

well, "gre" and "gretap" are different types of tunnels, with different features.. so it makes sense to test and validate with the normal "gre", as in your config I don't see a need for "gretap".

See the part with mtu 1500 and ignore don’t fragment flag, it’s possible only in gretap encapsulation . Also it used multicast group for vxlan
In any case it is bug and it does not matter which type of tunnel I use in the test if it cause a bug 🐞

pasik added a comment.Oct 5 2022, 3:07 PM

Ah, yeah, that's a valid point for gretap.

Anyway, my point was, it would be good to test if the issue/bug also affects plain 'gre', as behind the scenes 'gre' and 'gretap' are handled and configured differently, even though they might seem as very similar in vyos cli/config.

The bug might affect both, but it would be good to check and verify.

Viacheslav added a comment.Oct 11 2022, 8:27 AM
In T4729#135230, @pasik wrote:

Ah, yeah, that's a valid point for gretap.

Anyway, my point was, it would be good to test if the issue/bug also affects plain 'gre', as behind the scenes 'gre' and 'gretap' are handled and configured differently, even though they might seem as very similar in vyos cli/config.

The bug might affect both, but it would be good to check and verify.

GRE is not affected

Viacheslav changed the task status from Open to Needs testing.Jan 20 2024, 11:06 AM
Viacheslav assigned this task to a.hajiyev.
Viacheslav triaged this task as Normal priority.

Needs to re-check

c-po changed the task status from Needs testing to Needs reporter action.Jan 22 2024, 9:08 PM
dmbaturin added a project: Restricted Project.Feb 15 2024, 12:58 PM
dmbaturin edited projects, added VyOS 1.4 Sagitta (1.4.0-epa1); removed VyOS 1.4 Sagitta.Feb 15 2024, 3:04 PM
Viacheslav mentioned this in T6043: VxLAN and bridge error bug.Feb 16 2024, 10:31 AM
Viacheslav added a comment.EditedFeb 16 2024, 10:43 AM

It is still a bug VyOS 1.4-rolling-202402160309. After deleting the tunnel key from both sites there are no pings

The reason is that the vxlan interface was deleted

delete interfaces tunnel tun0 parameters ip key
commit


vyos@vyos1# ip link show type vxlan
[edit]
vyos@vyos1#

client

vyos@cl-left:~$ ping 100.64.0.2
PING 100.64.0.2 (100.64.0.2) 56(84) bytes of data.
From 100.64.0.1 icmp_seq=23 Destination Host Unreachable
From 100.64.0.1 icmp_seq=24 Destination Host Unreachable
From 100.64.0.1 icmp_seq=25 Destination Host Unreachable
From 100.64.0.1 icmp_seq=26 Destination Host Unreachable
From 100.64.0.1 icmp_seq=27 Destination Host Unreachable
From 100.64.0.1 icmp_seq=28 Destination Host Unreachable
From 100.64.0.1 icmp_seq=29 Destination Host Unreachable
From 100.64.0.1 icmp_seq=30 Destination Host Unreachable
Viacheslav changed the task status from Needs reporter action to Confirmed.Feb 16 2024, 10:43 AM
Viacheslav removed a.hajiyev as the assignee of this task.Feb 16 2024, 11:50 AM
Viacheslav added a subscriber: a.hajiyev.