I'm running VyOS 1.2.5 as a PPPoE server with "redistribute connected" PPPoE customer routes to iBGP. Lots of dynamic interfaces created/deleted all the time, especially with buggy cheap SOHO routers that try to establish two sessions in parallel for no good reason, fail and try again. Crashes (bgpd signal 11) about once a day, and after bgpd is restarted by watchfrr, the process is there in "ps" but no longer works ("show ip bgp summary" says "BGP instance not found"), needs reboot to recover.
Reproduced on two separate setups with similar config, one bare metal (PC Engines APU4D4) and one VM (xcp-ng 8.1, Supermicro H8SGL-F, Opteron 6338P), in both cases with 4 CPU cores and 4 GB RAM (physical, or allocated to VM) which should be plenty. In both cases signal 11 in the same function, so bad hardware can be ruled out (both setups have ECC RAM). See below for frr.log messages (a few seconds surrounding the crash, please ask if you need more) from both crashes.
vyos@R5:~$ show version
Version: VyOS 1.2.5
Built by: Sentrium S.L.
Built on: Sun 12 Apr 2020 15:18 UTC
Build UUID: 1695c660-d785-4b16-a54b-66d6a02ea24f
Build Commit ID: 48cc9fc46569e6
Architecture: x86_64
Boot via: installed image
System type: bare metal
Hardware vendor: PC Engines
Hardware model: apu4
Hardware S/N: 1418430
Hardware UUID: Unknown
Copyright: VyOS maintainers and contributors
Aug 21 21:20:46 R5 ripd[1086]: interface delete ppp-lot29 vrf 0 index 6387 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:46 R5 bgpd[1078]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp-lot29 in VRF 0
Aug 21 21:20:46 R5 ripngd[1092]: interface delete ppp-lot29 vrf 0 index 6387 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:46 R5 ripd[1086]: interface delete ppp53 vrf 0 index 6388 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:46 R5 ripngd[1092]: interface delete ppp53 vrf 0 index 6388 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:46 R5 bgpd[1078]: Received signal 11 at 1598044846 (si_addr 0x8, PC 0x7f43ac82da1c); aborting...#012x
Aug 21 21:20:46 R5 bgpd[1078]: Backtrace for 13 stack frames:
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_backtrace_sigsafe+0x67) [0x7f43ac81e3c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_signal+0x113) [0x7f43ac81e823]3ac81e3c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0x73b85) [0x7f43ac83fb85]ac81e823]3ac81e3c7]
Aug 21 21:20:46 R5 bgpd[1078]: /lib/x86_64-linux-gnu/libpthread.so.0(+0xf890) [0x7f43ab646890]fb85]ac81e823]3ac81e3c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(_rb_remove+0x16c) [0x7f43ac82da1c]]3ac81e3c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(if_set_index+0xb3) [0x7f43ac8152a3]3ac81e3c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(if_destroy_via_zapi+0x25) [0x7f43ac816865]c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0x9452e) [0x7f43ac86052e][0x7f43ac816865]c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(thread_call+0x60) [0x7f43ac84d530]c816865]c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(frr_run+0xd8) [0x7f43ac81c658]530]c816865]c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/frr/bgpd(main+0x322) [0x55ac6d401ca2]run+0xd8) [0x7f43ac81c658]530]c816865]c7]
Aug 21 21:20:46 R5 bgpd[1078]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f43ab2adb45]530]c816865]c7]
Aug 21 21:20:46 R5 bgpd[1078]: /usr/lib/frr/bgpd(+0x3ad9c) [0x55ac6d403d9c]_main+0xf5) [0x7f43ab2adb45]530]c816865]c7]
Aug 21 21:20:46 R5 bgpd[1078]: in thread zclient_read scheduled from lib/zclient.c:3266#0127f43ac82da1c); aborting...#012x
Aug 21 21:20:46 R5 zebra[1067]: [EC 100663299] stream_read_try: read failed on fd 20: Connection reset by peer
Aug 21 21:20:46 R5 watchfrr[1030]: [EC 268435457] bgpd state -> down : read returned EOF
Aug 21 21:20:46 R5 zebra[1067]: [EC 4043309121] Client 'bgp' encountered an error and is shutting down.
Aug 21 21:20:46 R5 zebra[1067]: [EC 100663299] stream_read_try: read failed on fd 32: Connection reset by peer
Aug 21 21:20:46 R5 zebra[1067]: [EC 4043309121] Client 'vnc' encountered an error and is shutting down.
Aug 21 21:20:48 R5 zebra[1067]: client 20 disconnected. 920462 bgp routes removed from the rib
Aug 21 21:20:49 R5 zebra[1067]: client 32 disconnected. 0 vnc routes removed from the rib
Aug 21 21:20:50 R5 ripngd[1092]: interface delete ppp57 vrf 0 index 6389 flags 0x10 metric 0 mtu 1500
Aug 21 21:20:50 R5 ripd[1086]: interface delete ppp57 vrf 0 index 6389 flags 0x10 metric 0 mtu 1500
Aug 21 21:20:51 R5 watchfrr[1030]: [EC 100663303] Forked background command [pid 10441]: /usr/lib/frr/watchfrr.sh restart bgpd
Aug 21 21:20:51 R5 zebra[1067]: client 20 says hello and bids fair to announce only bgp routes vrf=0
Aug 21 21:20:51 R5 zebra[1067]: client 32 says hello and bids fair to announce only vnc routes vrf=0
Aug 21 21:20:52 R5 watchfrr[1030]: bgpd state -> up : connect succeeded
Aug 21 21:20:52 R5 ripd[1086]: interface delete ppp-lot29 vrf 0 index 6388 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:52 R5 ripngd[1092]: interface delete ppp-lot29 vrf 0 index 6388 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:52 R5 bgpd[10449]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp-lot29 in VRF 0
Aug 21 21:20:52 R5 ripngd[1092]: interface delete ppp36 vrf 0 index 6387 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:52 R5 ripd[1086]: interface delete ppp36 vrf 0 index 6387 flags 0x10 metric 0 mtu 1492
Aug 21 21:20:52 R5 bgpd[10449]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp36 in VRF 0
Aug 21 21:20:52 R5 ripd[1086]: interface delete ppp60 vrf 0 index 6390 flags 0x1090 metric 0 mtu 1492
Aug 21 21:20:52 R5 ripngd[1092]: interface delete ppp60 vrf 0 index 6390 flags 0x1090 metric 0 mtu 1492
Aug 21 21:20:52 R5 bgpd[10449]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp60 in VRF 0
vyos@R6:~$ show version
Version: VyOS 1.2.5
Built by: Sentrium S.L.
Built on: Sun 12 Apr 2020 15:18 UTC
Build UUID: 1695c660-d785-4b16-a54b-66d6a02ea24f
Build Commit ID: 48cc9fc46569e6
Architecture: x86_64
Boot via: installed image
System type: Xen HVM guest
Hardware vendor: Xen
Hardware model: HVM domU
Hardware S/N: 2f4fa905-0407-e5cc-11ab-3296d4ed829c
Hardware UUID: 2f4fa905-0407-e5cc-11ab-3296d4ed829c
Copyright: VyOS maintainers and contributors
Aug 22 11:41:41 R6 ripngd[1107]: interface delete ppp-P.sta22 vrf 0 index 3970 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:41 R6 bgpd[1093]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp-P.sta22 in VRF 0
Aug 22 11:41:41 R6 ripd[1103]: interface delete ppp-P.sta22 vrf 0 index 3970 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:41 R6 ripngd[1107]: interface delete ppp-lot29 vrf 0 index 3972 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:41 R6 ripd[1103]: interface delete ppp-lot29 vrf 0 index 3972 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:41 R6 bgpd[1093]: Received signal 11 at 1598096501 (si_addr 0x8, PC 0x7f8295ab1a1c); aborting...
Aug 22 11:41:41 R6 bgpd[1093]: Backtrace for 13 stack frames:
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_backtrace_sigsafe+0x67) [0x7f8295aa23c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(zlog_signal+0x113) [0x7f8295aa2823]295aa23c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0x73b85) [0x7f8295ac3b85]95aa2823]295aa23c7]
Aug 22 11:41:41 R6 bgpd[1093]: /lib/x86_64-linux-gnu/libpthread.so.0(+0xf890) [0x7f82948af890]3b85]95aa2823]295aa23c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(_rb_remove+0x16c) [0x7f8295ab1a1c]]295aa23c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(if_set_index+0xb3) [0x7f8295a992a3]295aa23c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(if_destroy_via_zapi+0x25) [0x7f8295a9a865]c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(+0x9452e) [0x7f8295ae452e][0x7f8295a9a865]c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(thread_call+0x60) [0x7f8295ad1530]5a9a865]c7]
Aug 22 11:41:41 R6 ripngd[1107]: interface delete ppp121 vrf 0 index 3974 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:41 R6 ripd[1103]: interface delete ppp121 vrf 0 index 3974 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/x86_64-linux-gnu/frr/libfrr.so.0(frr_run+0xd8) [0x7f8295aa0658]530]5a9a865]c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/frr/bgpd(main+0x322) [0x56354800cca2]run+0xd8) [0x7f8295aa0658]530]5a9a865]c7]
Aug 22 11:41:41 R6 bgpd[1093]: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f8294511b45]530]5a9a865]c7]
Aug 22 11:41:41 R6 bgpd[1093]: /usr/lib/frr/bgpd(+0x3ad9c) [0x56354800ed9c]_main+0xf5) [0x7f8294511b45]530]5a9a865]c7]
Aug 22 11:41:41 R6 bgpd[1093]: in thread zclient_read scheduled from lib/zclient.c:3266#0127f8295ab1a1c); aborting...
Aug 22 11:41:41 R6 zebra[1085]: [EC 100663299] stream_read_try: read failed on fd 30: Connection reset by peer
Aug 22 11:41:41 R6 zebra[1085]: [EC 4043309121] Client 'vnc' encountered an error and is shutting down.
Aug 22 11:41:41 R6 watchfrr[1025]: [EC 268435457] bgpd state -> down : read returned EOF
Aug 22 11:41:41 R6 watchfrr[1025]: bgpd state -> up : connect succeeded
Aug 22 11:41:41 R6 zebra[1085]: [EC 100663299] stream_read_try: read failed on fd 20: Connection reset by peer
Aug 22 11:41:41 R6 zebra[1085]: [EC 4043309121] Client 'bgp' encountered an error and is shutting down.
Aug 22 11:41:41 R6 watchfrr[1025]: [EC 268435457] bgpd state -> down : unexpected read error: Connection reset by peer
Aug 22 11:41:42 R6 zebra[1085]: client 30 disconnected. 0 vnc routes removed from the rib
Aug 22 11:41:42 R6 zebra[1085]: client 20 disconnected. 920320 bgp routes removed from the rib
Aug 22 11:41:45 R6 ripngd[1107]: interface delete ppp-lot29 vrf 0 index 3974 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:45 R6 ripngd[1107]: interface delete ppp55 vrf 0 index 3972 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:45 R6 ripd[1103]: interface delete ppp-lot29 vrf 0 index 3974 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:45 R6 ripd[1103]: interface delete ppp55 vrf 0 index 3972 flags 0x10 metric 0 mtu 1492
Aug 22 11:41:46 R6 watchfrr[1025]: [EC 100663303] Forked background command [pid 38779]: /usr/lib/frr/watchfrr.sh restart bgpd
Aug 22 11:41:46 R6 zebra[1085]: client 20 says hello and bids fair to announce only bgp routes vrf=0
Aug 22 11:41:46 R6 zebra[1085]: client 32 says hello and bids fair to announce only vnc routes vrf=0
Aug 22 11:41:46 R6 watchfrr[1025]: bgpd state -> up : connect succeeded
Aug 22 11:43:48 R6 ripngd[1107]: interface delete ppp-kub2c2 vrf 0 index 3646 flags 0x10 metric 0 mtu 1480
Aug 22 11:43:48 R6 bgpd[38787]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp-kub2c2 in VRF 0
Aug 22 11:43:48 R6 ripd[1103]: interface delete ppp-kub2c2 vrf 0 index 3646 flags 0x10 metric 0 mtu 1480
Aug 22 11:43:48 R6 ripngd[1107]: interface delete ppp121 vrf 0 index 3974 flags 0x10 metric 0 mtu 1492
Aug 22 11:43:48 R6 bgpd[38787]: [EC 100663301] INTERFACE_STATE: Cannot find IF ppp121 in VRF 0
Aug 22 11:43:48 R6 ripd[1103]: interface delete ppp121 vrf 0 index 3974 flags 0x10 metric 0 mtu 1492