Page MenuHomeVyOS Platform

Commit failed when adding new bgp neighbor
Confirmed, HighPublicBUG

Description

Hi all, i have working bgp configuration:

set protocols bgp address-family ipv4-unicast redistribute connected
set protocols bgp address-family l2vpn-evpn advertise ipv4 unicast
set protocols bgp address-family l2vpn-evpn advertise-all-vni
set protocols bgp address-family l2vpn-evpn advertise-default-gw
set protocols bgp address-family l2vpn-evpn advertise-svi-ip
set protocols bgp address-family l2vpn-evpn default-originate ipv4
set protocols bgp address-family l2vpn-evpn rd '10.177.75.5:2'
set protocols bgp address-family l2vpn-evpn route-target export '64542:1'
set protocols bgp address-family l2vpn-evpn route-target import '64542:1'
set protocols bgp address-family l2vpn-evpn vni 7777 route-target export '64542:1'
set protocols bgp address-family l2vpn-evpn vni 7777 route-target import '64542:1'
set protocols bgp neighbor 10.177.70.62 peer-group 'UNDERLAY'
set protocols bgp neighbor 10.177.70.62 remote-as 'external'
set protocols bgp neighbor 10.177.75.1 peer-group 'OVERLAY'
set protocols bgp neighbor 10.177.75.1 remote-as '64542'
set protocols bgp parameters bestpath as-path multipath-relax
set protocols bgp parameters log-neighbor-changes
set protocols bgp parameters router-id '10.177.75.5'
set protocols bgp peer-group OVERLAY address-family l2vpn-evpn route-reflector-client
set protocols bgp peer-group OVERLAY capability dynamic
set protocols bgp peer-group OVERLAY local-as 64542
set protocols bgp peer-group OVERLAY ttl-security hops '3'
set protocols bgp peer-group OVERLAY update-source 'dum0'
set protocols bgp peer-group UNDERLAY address-family ipv4-unicast route-map export 'UNDERLAY-OUT'
set protocols bgp peer-group UNDERLAY capability dynamic
set protocols bgp system-as '4200010001'
set protocols bgp timers holdtime '90'
set protocols bgp timers keepalive '30'

But, if i trying to add new neighbor to Overlay peer-group error occured after commit:
commands:
set protocols bgp neighbor 10.177.75.2 peer-group ‘OVERLAY’
set protocols bgp neighbor 10.177.75.2 remote-as ‘64542’

Error:

[ protocols bgp ]
VyOS had an issue completing a command.

We are sorry that you encountered a problem while using VyOS.
There are a few things you can do to help us (and yourself):
- Contact us using the online help desk if you have a subscription:
  https://support.vyos.io/
- Make sure you are running the latest version of VyOS available at:
  https://vyos.net/get/
- Consult the community forum to see how to handle this issue:
  https://forum.vyos.io
- Join us on Slack where our users exchange help and advice:
  https://vyos.slack.com

When reporting problems, please include as much information as possible:
- do not obfuscate any data (feel free to contact us privately if your
  business policy requires it)
- and include all the information presented below

Report time:      2024-02-27 10:00:36
Image version:    VyOS 1.4-rolling-202402210305
Release train:    sagitta

Built by:         98030736+bot-ross[bot]@users.noreply.github.com
Built on:         Wed 21 Feb 2024 03:05 UTC
Build UUID:       ac17eaa0-f8b3-46fd-a04f-77a5d7443c63
Build commit ID:  bcac2eb1f9b49c

Architecture:     x86_64
Boot via:         installed image
System type:      KVM guest

Hardware vendor:  QEMU
Hardware model:   Standard PC (i440FX + PIIX, 1996)
Hardware S/N:
Hardware UUID:    e79a9ff2-ad64-4a12-89e9-6532764ffaa5

Traceback (most recent call last):
  File "/usr/libexec/vyos/conf_mode/protocols_bgp.py", line 611, in <module>
    apply(c)
  File "/usr/libexec/vyos/conf_mode/protocols_bgp.py", line 602, in apply
    frr_cfg.commit_configuration(bgp_daemon)
  File "/usr/lib/python3/dist-packages/vyos/frr.py", line 482, in commit_configuration
    raise ConfigurationNotValid(f'Config commit retry counter ({count_max}) exceeded for {daemon} dameon!')
vyos.frr.ConfigurationNotValid: Config commit retry counter (5) exceeded for bgpd dameon!



[[protocols bgp]] failed
Commit failed
[edit]

same error occured on latest 1.5 rolling release
Maybe my configuration is wrong? or this is the bug?

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.4-rolling-202402210305
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

SrividyaA changed the task status from Open to Confirmed.Mar 6 2024, 11:35 AM

Tested in different possibilities:

  • Did not configure the "local-as" parameter in the peer-group "OVERLAY", then able to add the new neighbor with remote-as "64542"
  • Added the new neighbor in FRR prompt, then received this error message:
test1(config-router)# neighbor 10.177.75.2 remote-as 64542
% Peer-group members must be all internal or all external.

-But if I add remote-as "4200010001" to new neighbor 10.177.75.2, then I don't receive any error.

vyos@test1# set protocols bgp neighbor 10.177.75.2 remote-as 4200010001
[edit]
vyos@test1# commit
[edit]
vyos@test1#

No, this is not correct for my set up. Configuration commited without error but lost peering

  1. Peer-group UNDERLAY - eBGP(directly connected peers)
  2. Peer-group OVERLAY - iBGP

Test1:
If i remove local-as from peer-group OVERLAY, peering is lost with error:

bgp_pp_recv:5049: NOTIFICATION sent to 10.177.75.5+44377 (proto): code 2 (Open Message Error) subcode 2 (bad peer AS number), Reason: no group for 10.177.75.5+44377 (proto) from AS 4200010001 found (peer as mismatch)

Test1:
set protocols bgp peer-group OVERLAY local-as 64542
set protocols bgp peer-group OVERLAY remote-as 'internal'

same error

This is working frr configuration:

VyOS-R1(config-router)# do sh run
Building configuration...

Current configuration:
!
frr version 9.1
frr defaults traditional
hostname VyOS-R1
log syslog
log facility local7
service integrated-vtysh-config
!
router bgp 4200010001
 bgp router-id 10.177.75.5
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp bestpath as-path multipath-relax
 no bgp network import-check
 timers bgp 30 90
 neighbor OVERLAY peer-group
 neighbor OVERLAY remote-as 64542
 neighbor OVERLAY local-as 64542
 neighbor OVERLAY ttl-security hops 3
 neighbor OVERLAY update-source dum0
 neighbor OVERLAY capability dynamic
 neighbor UNDERLAY peer-group
 neighbor UNDERLAY capability dynamic
 neighbor 10.177.75.1 peer-group OVERLAY
 neighbor 10.177.75.1 description RR-SPINE-1
 neighbor 10.177.75.2 peer-group OVERLAY
 neighbor 10.177.75.2 description RR-SPINE-2
 neighbor 10.177.70.62 remote-as external
 neighbor 10.177.70.62 peer-group UNDERLAY
 neighbor 10.177.70.62 description HOST
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor UNDERLAY activate
  neighbor UNDERLAY route-map UNDERLAY-OUT out
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor OVERLAY activate
  advertise-all-vni
  vni 7777
   route-target import 64542:1
   route-target export 64542:1
  exit-vni
  vni 7776
   route-target import 64542:1
   route-target export 64542:1
  exit-vni
  advertise-default-gw
  advertise-svi-ip
  route-target import 64542:1
  route-target export 64542:1
 exit-address-family
exit
!
ip prefix-list UNDERLAY-EXPORT seq 10 permit 10.177.75.5/32
ip prefix-list UNDERLAY-EXPORT seq 15 permit 10.216.15.0/24
ip prefix-list UNDERLAY-EXPORT seq 20 permit 10.216.16.0/24
!
route-map UNDERLAY-OUT permit 10
 match ip address prefix-list UNDERLAY-EXPORT
exit
!
rpki
exit
!
end
VyOS-R1(config-router)#

sh bgp summary:

vyos@VyOS-R1# run sh bgp summary

IPv4 Unicast Summary (VRF default):
BGP router identifier 10.177.75.5, local AS number 4200010001 vrf-id 0
BGP table version 16
RIB entries 27, using 2592 bytes of memory
Peers 1, using 20 KiB of memory
Peer groups 2, using 128 bytes of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.177.70.62    4 4200040003        40        37       16    0    0 00:15:40           11        3 HOST

Total number of neighbors 1

L2VPN EVPN Summary (VRF default):
BGP router identifier 10.177.75.5, local AS number 4200010001 vrf-id 0
BGP table version 0
RIB entries 11, using 1056 bytes of memory
Peers 2, using 40 KiB of memory
Peer groups 2, using 128 bytes of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.177.75.1     4      64542        35        23       28    0    0 00:08:25           45        4 RR-SPINE-1
10.177.75.2     4      64542        34        22       28    0    0 00:07:55           45        4 RR-SPINE-2

Total number of neighbors 2
[edit]

VyOS: when try to set:
set protocols bgp peer-group OVERLAY remote-as 64542
set protocols bgp peer-group OVERLAY local-as 64542

Error occured:

vyos@VyOS-R1# commit

**Neighbor "OVERLAY" has local-as specified which is the same as remote-
as, this is not allowed!**

[[protocols bgp]] failed
Commit failed

Frr accepts to set peer-group to set same local-as and remote-as but on VyOS this occures error!

SPINE-1 BGP config(JunOS):

set protocols bgp group UNDERLAY mtu-discovery
set protocols bgp group UNDERLAY export FROM-UNDERLAY
set protocols bgp group UNDERLAY local-as 4200020001
set protocols bgp group UNDERLAY graceful-restart disable
set protocols bgp group UNDERLAY multipath multiple-as
set protocols bgp group UNDERLAY bfd-liveness-detection minimum-interval 350
set protocols bgp group UNDERLAY bfd-liveness-detection multiplier 3
set protocols bgp group UNDERLAY neighbor 10.177.77.2 peer-as 4200030001
set protocols bgp group UNDERLAY neighbor 10.177.77.10 peer-as 4200030002
set protocols bgp group OVERLAY type internal
set protocols bgp group OVERLAY local-address 10.177.75.1
set protocols bgp group OVERLAY family evpn signaling
set protocols bgp group OVERLAY cluster 10.216.255.255
set protocols bgp group OVERLAY peer-as 64542
set protocols bgp group OVERLAY local-as 64542
set protocols bgp group OVERLAY multipath
set protocols bgp group OVERLAY neighbor 10.177.75.3 description LEAF-1
set protocols bgp group OVERLAY neighbor 10.177.75.4
set protocols bgp group OVERLAY neighbor 10.176.75.1
set protocols bgp group OVERLAY neighbor 10.176.75.2
set protocols bgp group OVERLAY neighbor 10.177.75.5 description VyOS
set protocols bgp group OVERLAY neighbor 10.176.75.3 description HV205
set protocols bgp log-updown

Your initial configuration and adding a new peer is not acceptable by FRR

vyos@r4# run show ver
Version:          VyOS 1.5-rolling-202404090019
Release train:    current
vyos@r4# compare 
[protocols bgp neighbor]
+ 10.177.75.2 {
+     peer-group "OVERLAY"
+     remote-as "64542"
+ }

[edit]
vyos@r4# commit
[ protocols bgp ]
[8785|mgmtd] sending configuration [8786|zebra] sending configuration
[8787|ripd] sending configuration [8788|ripngd] sending configuration
[8789|ospfd] sending configuration [8790|ospf6d] sending configuration
[8791|ldpd] sending configuration [8792|bgpd] sending configuration
[8793|isisd] sending configuration [8797|babeld] sending configuration
[8803|bfdd] sending configuration Waiting for children to finish
applying config... [8800|watchfrr] sending configuration [8806|pim6d]
sending configuration [8802|staticd] sending configuration [8789|ospfd]
done [8787|ripd] done [8791|ldpd] done [8790|ospf6d] done [8785|mgmtd]
done [8802|staticd] done [8793|isisd] done % Peer-group members must be
all internal or all external. line 2: Failure to communicate[13] to
bgpd, line:  neighbor 10.177.75.2 remote-as 64542  [8806|pim6d] done
[8788|ripngd] done [8786|zebra] done [8792|bgpd] Configuration
file[/etc/frr/frr.conf] processing failure: 13 [8800|watchfrr] done
[8797|babeld] done [8803|bfdd] done [8810|mgmtd] sending configuration
[8811|zebra] sending configuration [8812|ripd] sending configuration
[8813|ripngd] sending configuration [8814|ospfd] sending configuration
[8815|ospf6d] sending configuration [8816|ldpd] sending configuration
[8817|bgpd] sending configuration [8818|isisd] sending configuration
[8822|babeld] sending configuration [8825|watchfrr] sending
configuration [8812|ripd] done [8827|staticd] sending configuration
[8828|bfdd] sending configuration Waiting for children to finish
applying config... [8831|pim6d] sending configuration [8811|zebra] done
% Peer-group members must be all internal or all external. line 2:
Failure to communicate[13] to bgpd, line:  neighbor 10.177.75.2 remote-
as 64542  [8813|ripngd] done [8815|ospf6d] done % Peer-group members
must be all internal or all external. line 10: Failure to
communicate[13] to bgpd, line:  neighbor 10.177.75.2 remote-as 64542
[8810|mgmtd] done [8817|bgpd] Configuration file[/etc/frr/frr.conf]
processing failure: 13 [8822|babeld] done [8828|bfdd] done [8814|ospfd]
done [8818|isisd] done [8816|ldpd] done [8825|watchfrr] done
[8831|pim6d] done [8827|staticd] done

[[protocols bgp]] failed
Commit failed
[edit]
vyos@r4#

Initial not confirmed by FRR, but I provided configuration is accepted by FRR but not applied by vyos. Please look messages

@d.shleg As I mentioned the config is not applied by FRR

r4# show run bgpd
Building configuration...

Current configuration:
!
frr version 9.1
frr defaults traditional
hostname r4
log syslog
log facility local7
service integrated-vtysh-config
!
router bgp 4200010001
 bgp router-id 10.177.75.5
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp bestpath as-path multipath-relax
 no bgp network import-check
 timers bgp 30 90
 neighbor OVERLAY peer-group
 neighbor OVERLAY local-as 64542
 neighbor OVERLAY ttl-security hops 3
 neighbor OVERLAY update-source dum0
 neighbor OVERLAY capability dynamic
 neighbor UNDERLAY peer-group
 neighbor UNDERLAY capability dynamic
 neighbor 10.177.75.1 remote-as 64542
 neighbor 10.177.75.1 peer-group OVERLAY
 neighbor 10.177.75.2 peer-group OVERLAY
 neighbor 10.177.70.62 remote-as external
 neighbor 10.177.70.62 peer-group UNDERLAY
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor UNDERLAY activate
  neighbor UNDERLAY route-map UNDERLAY-OUT out
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor OVERLAY activate
  advertise-all-vni
  vni 7777
   route-target import 64542:1
   route-target export 64542:1
  exit-vni
  advertise-default-gw
  advertise-svi-ip
  advertise ipv4 unicast
  default-originate ipv4
  rd 10.177.75.5:2
  route-target import 64542:1
  route-target export 64542:1
 exit-address-family
exit
!
route-map UNDERLAY-OUT permit 100
exit
!
rpki
exit
!
end
r4# 



r4# conf t
r4(config)# router bgp
r4(config-router)# neighbor 10.177.75.2 peer-group OVERLAY
r4(config-router)# neighbor 10.177.75.2 remote-as 64542
% Peer-group members must be all internal or all external.
r4(config-router)#

Yes, initial not applied. I corrected config and received other error

This is working frr configuration:

VyOS-R1(config-router)# do sh run
Building configuration...

Current configuration:
!
frr version 9.1
frr defaults traditional
hostname VyOS-R1
log syslog
log facility local7
service integrated-vtysh-config
!
router bgp 4200010001
 bgp router-id 10.177.75.5
 bgp log-neighbor-changes
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 bgp bestpath as-path multipath-relax
 no bgp network import-check
 timers bgp 30 90
 neighbor OVERLAY peer-group
 neighbor OVERLAY remote-as 64542
 neighbor OVERLAY local-as 64542
 neighbor OVERLAY ttl-security hops 3
 neighbor OVERLAY update-source dum0
 neighbor OVERLAY capability dynamic
 neighbor UNDERLAY peer-group
 neighbor UNDERLAY capability dynamic
 neighbor 10.177.75.1 peer-group OVERLAY
 neighbor 10.177.75.1 description RR-SPINE-1
 neighbor 10.177.75.2 peer-group OVERLAY
 neighbor 10.177.75.2 description RR-SPINE-2
 neighbor 10.177.70.62 remote-as external
 neighbor 10.177.70.62 peer-group UNDERLAY
 neighbor 10.177.70.62 description HOST
 !
 address-family ipv4 unicast
  redistribute connected
  neighbor UNDERLAY activate
  neighbor UNDERLAY route-map UNDERLAY-OUT out
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor OVERLAY activate
  advertise-all-vni
  vni 7777
   route-target import 64542:1
   route-target export 64542:1
  exit-vni
  vni 7776
   route-target import 64542:1
   route-target export 64542:1
  exit-vni
  advertise-default-gw
  advertise-svi-ip
  route-target import 64542:1
  route-target export 64542:1
 exit-address-family
exit
!
ip prefix-list UNDERLAY-EXPORT seq 10 permit 10.177.75.5/32
ip prefix-list UNDERLAY-EXPORT seq 15 permit 10.216.15.0/24
ip prefix-list UNDERLAY-EXPORT seq 20 permit 10.216.16.0/24
!
route-map UNDERLAY-OUT permit 10
 match ip address prefix-list UNDERLAY-EXPORT
exit
!
rpki
exit
!
end
VyOS-R1(config-router)#

sh bgp summary:

vyos@VyOS-R1# run sh bgp summary

IPv4 Unicast Summary (VRF default):
BGP router identifier 10.177.75.5, local AS number 4200010001 vrf-id 0
BGP table version 16
RIB entries 27, using 2592 bytes of memory
Peers 1, using 20 KiB of memory
Peer groups 2, using 128 bytes of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.177.70.62    4 4200040003        40        37       16    0    0 00:15:40           11        3 HOST

Total number of neighbors 1

L2VPN EVPN Summary (VRF default):
BGP router identifier 10.177.75.5, local AS number 4200010001 vrf-id 0
BGP table version 0
RIB entries 11, using 1056 bytes of memory
Peers 2, using 40 KiB of memory
Peer groups 2, using 128 bytes of memory

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.177.75.1     4      64542        35        23       28    0    0 00:08:25           45        4 RR-SPINE-1
10.177.75.2     4      64542        34        22       28    0    0 00:07:55           45        4 RR-SPINE-2

Total number of neighbors 2
[edit]

VyOS: when try to set:
set protocols bgp peer-group OVERLAY remote-as 64542
set protocols bgp peer-group OVERLAY local-as 64542

Error occured:

vyos@VyOS-R1# commit

**Neighbor "OVERLAY" has local-as specified which is the same as remote-
as, this is not allowed!**

[[protocols bgp]] failed
Commit failed

Frr accepts to set peer-group to set same local-as and remote-as but on VyOS this occures error!