Page MenuHomeVyOS Platform
Create Task
Maniphest T3728

FRR not respect configured RD and RT for L3VNI
Closed, ResolvedPublicBUG
Actions

Description

[email protected]# show
 interfaces {
     bridge br1001 {
         address 10.26.0.1/24
         member {
             interface vxlan1001 {
             }
         }
         vrf tenant1
     }
     dummy dum0 {
         address 10.209.36.254/32
     }
     ethernet eth0 {
         address 37.59.2xx.2xx/29
     }
     loopback lo {
     }
     tunnel tun21 {
         address 10.91.2.2/30
         description "vr01-b AMS1"
         encapsulation gre
         parameters {
             ip {
                 ttl 200
             }
         }
         remote 185.206.xx.xxx
         source-address 37.59.xx.xxx
     }
     vxlan vxlan1001 {
         description tenant1
         mtu 1500
         parameters {
             nolearning
         }
         port 4789
         source-address 10.209.36.254
         vni 1001
     }
 }
 protocols {
     bgp {
         address-family {
             l2vpn-evpn {
                 advertise-all-vni
             }
         }
         local-as 209300
         neighbor 10.209.36.100 {
             peer-group fabric
         }
         parameters {
             default {
                 no-ipv4-unicast
             }
             router-id 10.209.36.254
         }
         peer-group fabric {
             address-family {
                 l2vpn-evpn {
                     soft-reconfiguration {
                         inbound
                     }
                 }
             }
             remote-as 209300
             update-source dum0
         }
     }
     ospf {
         area 0 {
             network 10.91.2.0/30
         }
         interface tun21 {
             mtu-ignore
             network point-to-point
         }
         parameters {
             router-id 10.209.36.254
         }
         passive-interface default
         passive-interface-exclude tun21
         redistribute {
             connected {
             }
         }
     }
     static {
         route 0.0.0.0/0 {
             next-hop 37.59.xx.xxx {
             }
         }
     }
 }
 service {
     ssh {
     }
 }
 system {
     config-management {
         commit-revisions 100
     }
     console {
         device ttyS0 {
             speed 115200
         }
     }
     host-name vrAccess01.par.fr.router.net
     ipv6 {
         disable
     }
     login {
         user vyos {
             authentication {
                 encrypted-password xxxxxxx
             }
         }
     }
     name-server 1.1.1.1
     ntp {
         server time1.vyos.net {
         }
         server time2.vyos.net {
         }
         server time3.vyos.net {
         }
     }
     syslog {
         global {
             facility all {
                 level info
             }
             facility protocols {
                 level debug
             }
         }
     }
 }
 vrf {
     name tenant1 {
         protocols {
             bgp {
                 address-family {
                     ipv4-unicast {
                         redistribute {
                             connected {
                             }
                         }
                     }
                     l2vpn-evpn {
                         advertise {
                             ipv4 {
                                 unicast {
                                 }
                             }
                         }
                         rd 10.209.36.254:1001
                         route-target {
                             export 65000:1001
                             import 65000:1001
                         }
                     }
                 }
                 local-as 209300
                 parameters {
                     router-id 10.209.36.254
                 }
             }
         }
         table 1001
         vni 1001
     }
 }
vrAccess01.par.fr.router.net# show run
Building configuration...

Current configuration:
!
frr version 7.5.1-20210305-00-g0509784dc
frr defaults traditional
hostname vrAccess01.par.fr.router.net
log syslog
log facility local7
service integrated-vtysh-config
!
ip route 0.0.0.0/0 37.59.203.246
!
interface tun21
 ip ospf mtu-ignore
!
router bgp 209300
 bgp router-id 10.209.36.254
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 neighbor fabric peer-group
 neighbor fabric remote-as 209300
 neighbor fabric update-source dum0
 neighbor 10.209.36.100 peer-group fabric
 !
 address-family l2vpn evpn
  neighbor fabric activate
  neighbor fabric soft-reconfiguration inbound
  advertise-all-vni
 exit-address-family
!
router bgp 209300 vrf tenant1
 bgp router-id 10.209.36.254
 no bgp ebgp-requires-policy
 no bgp network import-check
 !
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  rd 10.209.36.254:1001
  route-target import 65000:1001
  route-target export 65000:1001
 exit-address-family
!
router ospf
 ospf router-id 10.209.36.254
 auto-cost reference-bandwidth 100
 timers throttle spf 200 1000 10000
 redistribute connected
 passive-interface default
 no passive-interface tun21
 network 10.91.2.0/30 area 0
!
line vty
!
end
vrAccess01.par.fr.router.net# show bgp l2vpn evpn vni 1001
VNI: 1001 (known to the kernel)
  Type: L2
  Tenant-Vrf: tenant1
  RD: 10.209.36.254:2
  Originator IP: 10.209.36.254
  Mcast group: 0.0.0.0
  Advertise-gw-macip : Disabled
  Advertise-svi-macip : Disabled
  Import Route Target:
    12757:1001
  Export Route Target:
    12757:1001
vrAccess01.par.fr.router.net# show bgp l2vpn evpn
BGP table version is 3, local router ID is 10.209.36.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
EVPN type-1 prefix: [1]:[ESI]:[EthTag]:[IPlen]:[VTEP-IP]
EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 10.209.36.1:1000
*>i[5]:[0]:[24]:[10.40.1.0]
                    10.209.36.1              0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:b8:ca:3a:6c:65:50
*>i[5]:[0]:[29]:[192.168.10.0]
                    10.209.36.1              0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:b8:ca:3a:6c:65:50
*>i[5]:[0]:[32]:[10.40.1.1]
                    10.209.36.1              0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:b8:ca:3a:6c:65:50
Route Distinguisher: 10.209.36.1:1001
*>i[5]:[0]:[24]:[10.26.1.0]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:56:02:6e:63:c9:3f
*>i[5]:[0]:[24]:[10.26.4.0]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:56:02:6e:63:c9:3f
*>i[5]:[0]:[31]:[10.11.11.2]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:56:02:6e:63:c9:3f
*>i[5]:[0]:[32]:[10.26.1.1]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:56:02:6e:63:c9:3f
*>i[5]:[0]:[32]:[10.26.4.1]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:56:02:6e:63:c9:3f
Route Distinguisher: 10.209.36.1:1002
*>i[5]:[0]:[24]:[10.28.1.0]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b8:ca:3a:6c:65:50
*>i[5]:[0]:[31]:[10.11.11.4]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b8:ca:3a:6c:65:50
*>i[5]:[0]:[32]:[10.28.1.1]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b8:ca:3a:6c:65:50
Route Distinguisher: 10.209.36.1:1003
*>i[5]:[0]:[24]:[10.29.1.0]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:b8:ca:3a:6c:65:50
*>i[5]:[0]:[31]:[10.11.11.6]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:b8:ca:3a:6c:65:50
*>i[5]:[0]:[32]:[10.29.1.1]
                    10.209.36.1              0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:b8:ca:3a:6c:65:50
Route Distinguisher: 10.209.36.2:1000
*>i[5]:[0]:[24]:[10.40.1.0]
                    10.209.36.2              0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:ae:bd:dc:aa:11:f1
*>i[5]:[0]:[29]:[192.168.10.0]
                    10.209.36.2              0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:ae:bd:dc:aa:11:f1
Route Distinguisher: 10.209.36.2:1001
*>i[5]:[0]:[24]:[10.26.1.0]
                    10.209.36.2              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:b2:14:e3:8f:dc:81
*>i[5]:[0]:[24]:[10.26.4.0]
                    10.209.36.2              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:b2:14:e3:8f:dc:81
*>i[5]:[0]:[31]:[10.11.11.2]
                    10.209.36.2              0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:b2:14:e3:8f:dc:81
Route Distinguisher: 10.209.36.2:1002
*>i[5]:[0]:[24]:[10.28.1.0]
                    10.209.36.2              0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:7e:29:d7:4b:17:c9
*>i[5]:[0]:[31]:[10.11.11.4]
                    10.209.36.2              0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:7e:29:d7:4b:17:c9
Route Distinguisher: 10.209.36.2:1003
*>i[5]:[0]:[24]:[10.29.1.0]
                    10.209.36.2              0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:1e:15:95:67:b0:36
*>i[5]:[0]:[31]:[10.11.11.6]
                    10.209.36.2              0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:1e:15:95:67:b0:36
Route Distinguisher: 10.209.36.10:1000
*>i[5]:[0]:[24]:[10.40.2.0]
                    10.209.36.10             0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:b8:ca:3a:6c:62:c0
*>i[5]:[0]:[32]:[10.40.2.1]
                    10.209.36.10             0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:b8:ca:3a:6c:62:c0
Route Distinguisher: 10.209.36.10:1001
*>i[5]:[0]:[24]:[10.26.2.0]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:2a:66:a5:fc:07:45
*>i[5]:[0]:[24]:[10.26.5.0]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:2a:66:a5:fc:07:45
*>i[5]:[0]:[31]:[10.11.11.2]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:2a:66:a5:fc:07:45
*>i[5]:[0]:[32]:[10.26.2.1]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:2a:66:a5:fc:07:45
*>i[5]:[0]:[32]:[10.26.5.1]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:2a:66:a5:fc:07:45
Route Distinguisher: 10.209.36.10:1002
*>i[5]:[0]:[24]:[10.28.2.0]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b8:ca:3a:6c:62:c0
*>i[5]:[0]:[31]:[10.11.11.4]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b8:ca:3a:6c:62:c0
*>i[5]:[0]:[32]:[10.28.2.1]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b8:ca:3a:6c:62:c0
Route Distinguisher: 10.209.36.10:1003
*>i[5]:[0]:[24]:[10.29.2.0]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:66:e8:f2:39:ec:1e
*>i[5]:[0]:[31]:[10.11.11.6]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:66:e8:f2:39:ec:1e
*>i[5]:[0]:[32]:[10.29.2.1]
                    10.209.36.10             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:66:e8:f2:39:ec:1e
Route Distinguisher: 10.209.36.11:1000
*>i[5]:[0]:[24]:[10.40.2.0]
                    10.209.36.11             0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:b8:ca:3a:6a:d2:60
Route Distinguisher: 10.209.36.11:1001
*>i[5]:[0]:[24]:[10.26.2.0]
                    10.209.36.11             0    100      0 ?
                    RT:12757:1001 RT:65000:1001 ET:8 Rmac:a2:71:f3:9c:93:7f
*>i[5]:[0]:[24]:[10.26.5.0]
                    10.209.36.11             0    100      0 ?
                    RT:12757:1001 RT:65000:1001 ET:8 Rmac:a2:71:f3:9c:93:7f
*>i[5]:[0]:[31]:[10.11.11.2]
                    10.209.36.11             0    100      0 ?
                    RT:12757:1001 RT:65000:1001 ET:8 Rmac:a2:71:f3:9c:93:7f
Route Distinguisher: 10.209.36.11:1002
*>i[5]:[0]:[24]:[10.28.2.0]
                    10.209.36.11             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b2:af:dc:2d:29:48
*>i[5]:[0]:[31]:[10.11.11.4]
                    10.209.36.11             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:b2:af:dc:2d:29:48
Route Distinguisher: 10.209.36.11:1003
*>i[5]:[0]:[24]:[10.29.2.0]
                    10.209.36.11             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:76:fc:82:ae:a8:3d
*>i[5]:[0]:[31]:[10.11.11.6]
                    10.209.36.11             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:76:fc:82:ae:a8:3d
Route Distinguisher: 10.209.36.20:1000
*>i[5]:[0]:[24]:[10.40.3.0]
                    10.209.36.20             0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:16:ec:b4:21:d5:da
*>i[5]:[0]:[29]:[192.168.10.0]
                    10.209.36.20             0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:16:ec:b4:21:d5:da
*>i[5]:[0]:[32]:[10.40.3.1]
                    10.209.36.20             0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:16:ec:b4:21:d5:da
Route Distinguisher: 10.209.36.20:1001
*>i[5]:[0]:[24]:[10.26.3.0]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:1e:be:98:24:47:63
*>i[5]:[0]:[24]:[10.26.6.0]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:1e:be:98:24:47:63
*>i[5]:[0]:[31]:[10.11.11.2]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:1e:be:98:24:47:63
*>i[5]:[0]:[32]:[10.26.3.1]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:1e:be:98:24:47:63
*>i[5]:[0]:[32]:[10.26.6.1]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:1e:be:98:24:47:63
Route Distinguisher: 10.209.36.20:1002
*>i[5]:[0]:[24]:[10.28.3.0]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:20:67:7c:5d:99:70
*>i[5]:[0]:[31]:[10.11.11.4]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:20:67:7c:5d:99:70
*>i[5]:[0]:[32]:[10.28.3.1]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:20:67:7c:5d:99:70
Route Distinguisher: 10.209.36.20:1003
*>i[5]:[0]:[24]:[10.29.3.0]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:20:67:7c:5d:99:70
*>i[5]:[0]:[31]:[10.11.11.6]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:20:67:7c:5d:99:70
*>i[5]:[0]:[32]:[10.29.3.1]
                    10.209.36.20             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:20:67:7c:5d:99:70
Route Distinguisher: 10.209.36.21:1000
*>i[5]:[0]:[24]:[10.40.3.0]
                    10.209.36.21             0    100      0 ?
                    RT:12757:1000 ET:8 Rmac:20:67:7c:5d:88:20
Route Distinguisher: 10.209.36.21:1001
*>i[5]:[0]:[24]:[10.26.3.0]
                    10.209.36.21             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:20:67:7c:5d:88:20
*>i[5]:[0]:[24]:[10.26.6.0]
                    10.209.36.21             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:20:67:7c:5d:88:20
*>i[5]:[0]:[31]:[10.11.11.2]
                    10.209.36.21             0    100      0 ?
                    RT:65000:1001 ET:8 Rmac:20:67:7c:5d:88:20
Route Distinguisher: 10.209.36.21:1002
*>i[5]:[0]:[24]:[10.28.3.0]
                    10.209.36.21             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:20:67:7c:5d:88:20
*>i[5]:[0]:[31]:[10.11.11.4]
                    10.209.36.21             0    100      0 ?
                    RT:65000:1002 ET:8 Rmac:20:67:7c:5d:88:20
Route Distinguisher: 10.209.36.21:1003
*>i[5]:[0]:[24]:[10.29.3.0]
                    10.209.36.21             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:20:67:7c:5d:88:20
*>i[5]:[0]:[31]:[10.11.11.6]
                    10.209.36.21             0    100      0 ?
                    RT:65000:1003 ET:8 Rmac:20:67:7c:5d:88:20
Route Distinguisher: **10.209.36.254:2**
*> [3]:[0]:[32]:[10.209.36.254]
                    10.209.36.254                      32768 i
                   ** ET:8 RT:12757:1001**

Details

Difficulty level
Normal (likely a few hours)
Version
1.4-rolling-202108081830
Why the issue appeared?
Issues in third-party code
Is it a breaking change?
Unspecified (possibly destroys the router)

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedBUGc-poT3728 FRR not respect configured RD and RT for L3VNI
 ResolvedBUGc-poT3734 Move EVPN VRF up in FRR config

Event Timeline

paunadeu created this task.EditedAug 9 2021, 12:37 PM

Take a look to:

Route Distinguisher: 10.209.36.254:2
*> [3]:[0]:[32]:[10.209.36.254]

 10.209.36.254                      32768 i
** ET:8 RT:12757:1001**

We expect:

RD: 10.209.36.254:1001
RT: 65000:1001

Viacheslav added a subscriber: Viacheslav.Aug 9 2021, 12:51 PM

It is possible this bug.
https://github.com/FRRouting/frr/issues/9181

paunadeu added a comment.Aug 9 2021, 1:00 PM
In T3728#99683, @Viacheslav wrote:

It is possible this bug.
https://github.com/FRRouting/frr/issues/9181

They are reporting problems when they use RT auto-derivation, we don't want to use RT and RD autoderivation but FRR it's still setting as autoRT/RD.

Viacheslav changed Why the issue appeared? from Will be filled on close to Issues in third-party code.Aug 9 2021, 1:18 PM
paunadeu added a comment.Aug 9 2021, 2:48 PM

It's upgrade on FRR planned?

Viacheslav added a comment.Aug 9 2021, 6:13 PM

Maybe with FRR 8.1
In any case, we'll do more tests.

paunadeu added a comment.Aug 9 2021, 6:15 PM

Need to implement in customer, if I can help in something, please don't hesitate to ask.

paunadeu added a comment.Aug 11 2021, 2:34 PM

I think wasn't a problem. Not defined VNI in VRF, in L3 respects config in L2 don't, asked for the behavior in FRR Slack.

paunadeu added a comment.Aug 11 2021, 2:53 PM

OK. Seeing VyOS output now:

vrf {
    name tenant1 {
        protocols {
            bgp {
                address-family {
                    ipv4-unicast {
                        redistribute {
                            connected {
                            }
                        }
                    }
                    l2vpn-evpn {
                        advertise {
                            ipv4 {
                                unicast {
                                }
                            }
                        }
                        rd 10.209.36.254:1001
                        route-target {
                            export 65000:1001
                            import 65000:1001
                        }
                    }
                }
                local-as 209300
                parameters {
                    router-id 10.209.36.254
                }
            }
        }
        table 1001
        vni 1001
    }

but on FRR config I miss:

vrf tenant1
 vni 1001
 exit-vrf
c-po added a subscriber: c-po.Aug 11 2021, 4:20 PM

Hi @paunadeu yeah that is an open issue (https://github.com/vyos/vyos-1x/commit/d77709252de54757d3f3e76ce6c52492ba967488) it got disabled recently.

I hope to have a proper implementation in two weeks for this issue.

c-po claimed this task.Aug 11 2021, 4:21 PM
paunadeu added a comment.Aug 11 2021, 4:22 PM

Gotcha. Thanks. Will wait.

pasik added a subscriber: pasik.Aug 11 2021, 5:20 PM
c-po added a comment.Aug 12 2021, 5:20 PM

You can comment in the sections. it only creates an error when deleting vrf bgp instances

c-po added a subtask: T3734: Move EVPN VRF up in FRR config.Aug 12 2021, 5:56 PM
c-po closed this task as Resolved.Aug 13 2021, 7:36 PM
c-po closed subtask T3734: Move EVPN VRF up in FRR config as Resolved.
c-po reopened subtask T3734: Move EVPN VRF up in FRR config as On hold.Aug 16 2021, 4:39 PM
c-po closed subtask T3734: Move EVPN VRF up in FRR config as Resolved.Apr 17 2023, 7:01 PM