Page MenuHomeVyOS Platform
Create Task
Maniphest T2524

Restarting vyos-router.service breaks router
Closed, WontfixPublicBUG
Actions

Description

On a running system, sudo systemctl restart vyos-router.service (this was due to restarting vyos-hostsd):

May 27 09:36:48 vyos systemd[1]: Stopping VyOS Router...
May 27 09:36:48 vyos vyos-router[2363]: Stopping VyOS router:.
May 27 09:36:48 vyos vyos-router[2363]: Un-mounting VyOS Config...done.
May 27 09:36:48 vyos watchfrr[887]: Terminating on signal
May 27 09:36:49 vyos vyos-router[2363]: Stopped watchfrr.
May 27 09:36:49 vyos ldpd[984]: SIGINT received
May 27 09:36:49 vyos ldpd[984]: terminating
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'static' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: client 82 disconnected. 1 static routes removed from the rib
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'ldp' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: [EC 100663299] stream_read_try: read failed on fd 77: Connection reset by peer
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'ldp' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: Terminating on signal
May 27 09:36:49 vyos zebra[927]: client 19 disconnected. 0 bgp routes removed from the rib
May 27 09:36:49 vyos ospf6d[971]: Terminating on signal SIGINT
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'ospf6' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'ospf' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'bfd' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: client 31 disconnected. 0 vnc routes removed from the rib
May 27 09:36:49 vyos ripd[945]: Terminating on signal
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'ripng' encountered an error and is shutting down.
May 27 09:36:49 vyos pimd[977]: Terminating on signal SIGINT
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'pim' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: [EC 4043309121] Client 'system' encountered an error and is shutting down.
May 27 09:36:49 vyos zebra[927]: client 36 disconnected. 0 rip routes removed from the rib
May 27 09:36:49 vyos zebra[927]: client 41 disconnected. 0 ripng routes removed from the rib
May 27 09:36:49 vyos zebra[927]: client 46 disconnected. 0 ospf routes removed from the rib
May 27 09:36:49 vyos zebra[927]: client 52 disconnected. 0 bfd routes removed from the rib
May 27 09:36:49 vyos zebra[927]: client 57 disconnected. 0 ospf6 routes removed from the rib
May 27 09:36:49 vyos zebra[927]: client 62 disconnected. 0 pim routes removed from the rib
May 27 09:36:49 vyos zebra[927]: client 67 disconnected. 0 system routes removed from the rib
May 27 09:36:49 vyos bgpd[935]: Terminating on signal
May 27 09:36:49 vyos zebra[927]: client 72 disconnected. 0 ldp routes removed from the rib
May 27 09:36:49 vyos zebra[927]: client 77 disconnected. 0 ldp routes removed from the rib
May 27 09:36:49 vyos zebra[927]: Zebra final shutdown
May 27 09:36:49 vyos vyos-router[2363]: Stopped ldpd.
May 27 09:36:49 vyos vyos-router[2363]: Stopped zebra.
May 27 09:36:49 vyos vyos-router[2363]: Stopped bfddStopped staticd.
May 27 09:36:49 vyos vyos-router[2363]: .
May 27 09:36:49 vyos vyos-router[2363]: Stopped ospf6d.
May 27 09:36:49 vyos vyos-router[2363]: Stopped ripdStopped pimd.
May 27 09:36:49 vyos vyos-router[2363]: .
May 27 09:36:49 vyos vyos-router[2363]: Stopped ripngd.
May 27 09:36:49 vyos vyos-router[2363]: Stopped ospfd.
May 27 09:36:49 vyos vyos-router[2363]: Stopped bgpd.
May 27 09:36:49 vyos systemd[1]: vyos-router.service: Succeeded.
May 27 09:36:49 vyos systemd[1]: Stopped VyOS Router.
May 27 09:36:49 vyos systemd[1]: Started VyOS Router.
May 27 09:36:49 vyos vyos-router[2427]: Waiting for NICs to settle down: settled in 0sec..
May 27 09:36:49 vyos watchfrr[2450]: watchfrr 7.3.1 starting: vty@0
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] zebra state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] bgpd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] ripd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] ripngd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] ospfd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] ospf6d state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] pimd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] ldpd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] staticd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 268435457] bfdd state -> down : initial connection attempt failed
May 27 09:36:49 vyos watchfrr[2450]: [EC 100663303] Forked background command [pid 2451]: /usr/lib/frr/watchfrr.sh restart all
May 27 09:36:49 vyos watchfrr.sh[2462]: Cannot stop bfdd: pid file not found
May 27 09:36:49 vyos systemd[1]: Stopping VyOS Router...
May 27 09:36:49 vyos vyos-router[2485]: Stopping VyOS router:.
May 27 09:36:49 vyos vyos-router[2485]: Un-mounting VyOS Config...umount: /opt/vyatta/config: not mounted.
May 27 09:36:49 vyos vyos-router[2485]: failed.
May 27 09:36:49 vyos watchfrr[2450]: Terminating on signal
May 27 09:36:50 vyos vyos-router[2427]: Failed to start watchfrr! ... failed!
May 27 09:36:50 vyos vyos-router[2427]: Mounting VyOS Config...done.
May 27 09:36:50 vyos vyos-router[2485]: Stopped watchfrr.
May 27 09:36:50 vyos vyos-router[2485]: Cannot stop bfdd: pid file not found ...Cannot stop ldpd: pid file not found ...Cannot stop staticd: pid file 
May 27 09:36:50 vyos vyos-router[2485]: Cannot stop pimd: pid file not found ...Cannot stop ospf6d: pid file not found ...Cannot stop bgpd: pid file n
May 27 09:36:50 vyos vyos-router[2485]:  failed!
May 27 09:36:50 vyos vyos-router[2485]:  failed!
May 27 09:36:50 vyos vyos-router[2485]:  failed!
May 27 09:36:50 vyos vyos-router[2485]:  failed!
May 27 09:36:50 vyos vyos-router[2485]:  failed!
May 27 09:36:50 vyos vyos-router[2485]:  failed!
May 27 09:36:50 vyos vyos-router[2485]:  failed!
May 27 09:36:50 vyos vyos-router[2485]: Stopped zebra.
May 27 09:36:50 vyos systemd[1]: vyos-router.service: Main process exited, code=killed, status=15/TERM
May 27 09:36:50 vyos systemd[1]: vyos-router.service: Succeeded.
May 27 09:36:50 vyos systemd[1]: Stopped VyOS Router.
May 27 09:36:50 vyos systemd[1]: vyos-router.service: Found left-over process 2451 (watchfrr.sh) in control group while starting unit. Ignoring.
May 27 09:36:50 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:50 vyos systemd[1]: vyos-router.service: Found left-over process 2511 (vtysh) in control group while starting unit. Ignoring.
May 27 09:36:50 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:50 vyos systemd[1]: vyos-router.service: Found left-over process 2517 (run-config-migr) in control group while starting unit. Ignoring.
May 27 09:36:50 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:50 vyos systemd[1]: Started VyOS Router.
May 27 09:36:50 vyos vyos-router[2542]: Waiting for NICs to settle down: settled in 0sec..
May 27 09:36:50 vyos watchfrr[2567]: watchfrr 7.3.1 starting: vty@0
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] zebra state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] bgpd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] ripd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] ripngd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] ospfd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] ospf6d state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] pimd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] ldpd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] staticd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 268435457] bfdd state -> down : initial connection attempt failed
May 27 09:36:50 vyos watchfrr[2567]: [EC 100663303] Forked background command [pid 2568]: /usr/lib/frr/watchfrr.sh restart all
May 27 09:36:50 vyos vyos-router[2427]: Starting VyOS router: migrate
May 27 09:36:50 vyos systemd[1]: Stopping VyOS Router...
May 27 09:36:50 vyos watchfrr.sh[2583]: Cannot stop bfdd: pid file not found
May 27 09:36:50 vyos watchfrr.sh[2602]: Cannot stop ripd: pid file not found
May 27 09:36:50 vyos watchfrr.sh[2592]: Cannot stop ospfd: pid file not found
May 27 09:36:50 vyos vyos-router[2570]: Stopping VyOS router:.
May 27 09:36:50 vyos vyos-router[2570]: Un-mounting VyOS Config...done.
May 27 09:36:51 vyos watchfrr[2567]: Terminating on signal
May 27 09:36:51 vyos vyos-router[2542]: Failed to start watchfrr! ... failed!
May 27 09:36:51 vyos vyos-router[2542]: Mounting VyOS Config...done.
May 27 09:36:51 vyos vyos-router[2570]: Stopped watchfrr.
May 27 09:36:51 vyos vyos-router[2570]: Cannot stop ripngd: pid file not found ...Cannot stop pimd: pid file not found ...Cannot stop ospfd: pid file 
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]:  failed!
May 27 09:36:51 vyos vyos-router[2570]: Stopped zebra.
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Main process exited, code=killed, status=15/TERM
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Succeeded.
May 27 09:36:51 vyos systemd[1]: Stopped VyOS Router.
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Found left-over process 2451 (watchfrr.sh) in control group while starting unit. Ignoring.
May 27 09:36:51 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Found left-over process 2568 (watchfrr.sh) in control group while starting unit. Ignoring.
May 27 09:36:51 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Found left-over process 2633 (vtysh) in control group while starting unit. Ignoring.
May 27 09:36:51 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Found left-over process 2635 (run-config-migr) in control group while starting unit. Ignoring.
May 27 09:36:51 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Found left-over process 2658 (bgpd) in control group while starting unit. Ignoring.
May 27 09:36:51 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:51 vyos systemd[1]: vyos-router.service: Found left-over process 2662 (vtysh) in control group while starting unit. Ignoring.
May 27 09:36:51 vyos systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
May 27 09:36:51 vyos systemd[1]: Started VyOS Router.
May 27 09:36:51 vyos vyos-router[2665]: Waiting for NICs to settle down: settled in 0sec..
May 27 09:36:51 vyos vyos-router[2542]: Starting VyOS router: migrate
May 27 09:36:51 vyos systemd[1]: Stopping VyOS Router...
May 27 09:36:51 vyos vyos-router[2683]: Stopping VyOS router:.
May 27 09:36:51 vyos vyos-router[2683]: Un-mounting VyOS Config...done.
May 27 09:36:51 vyos watchfrr[2702]: watchfrr 7.3.1 starting: vty@0
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] zebra state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: bgpd state -> up : connect succeeded
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] ripd state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] ripngd state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] ospfd state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] ospf6d state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] pimd state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] ldpd state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] staticd state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 268435457] bfdd state -> down : initial connection attempt failed
May 27 09:36:51 vyos watchfrr[2702]: [EC 100663303] Forked background command [pid 2703]: /usr/lib/frr/watchfrr.sh restart all
May 27 09:36:52 vyos watchfrr.sh[2734]: Cannot stop ospf6d: pid file not found
May 27 09:36:52 vyos bgpd[2658]: Terminating on signal
May 27 09:36:52 vyos watchfrr.sh[2729]: Cannot stop ldpd: pid file not found
May 27 09:36:52 vyos watchfrr[2702]: Terminating on signal
May 27 09:36:52 vyos vyos-router[2665]: Failed to start watchfrr! ... failed!
May 27 09:36:52 vyos vyos-router[2665]: Mounting VyOS Config...done.
May 27 09:36:52 vyos vyos-router[2683]: Stopped watchfrr.
May 27 09:36:52 vyos vyos-router[2683]: Cannot stop bfdd: pid file not found ...Cannot stop ldpd: pid file not found ...Cannot stop pimd: pid file not
May 27 09:36:52 vyos vyos-router[2683]: Cannot stop ospfd: pid file not found ... failed!
May 27 09:36:52 vyos vyos-router[2683]: Cannot stop zebra: pid file not found ...Cannot stop bgpd: pid file not found ...Cannot stop ospf6d: pid file 
May 27 09:36:52 vyos vyos-router[2683]:  failed!
May 27 09:36:52 vyos vyos-router[2683]: Cannot stop ripngd: pid file not found ... failed!
May 27 09:36:52 vyos vyos-router[2683]:  failed!
May 27 09:36:52 vyos vyos-router[2683]:  failed!
May 27 09:36:52 vyos vyos-router[2683]:  failed!
May 27 09:36:52 vyos vyos-router[2683]:  failed!
May 27 09:36:52 vyos vyos-router[2683]: Stopped ripd.
May 27 09:36:52 vyos systemd[1]: vyos-router.service: Main process exited, code=killed, status=15/TERM
May 27 09:36:52 vyos systemd[1]: vyos-router.service: Succeeded.
May 27 09:36:52 vyos systemd[1]: Stopped VyOS Router.

The above isn't the whole log, it restarts a few times after that until freezing and not doing anything any more.

A reboot doesn't solve it:

[  OK  ] Started VyOS Router.
         Starting Permit User Sessions...
         Starting LSB: AWS EC2 inst…tch and load ssh public key...
         Starting LSB: Brings up/down network automatically...
         Starting System Logging Service...
[  OK  ] Started Regular background program processing daemon.
         Starting /etc/rc.local Compatibility...
         Starting Restore /etc/reso… the ppp link was shut down...
[  OK  ] Started D-Bus System Message Bus.
         Starting LSB: powerful, ef…alable Mail Transport Agent...
         Starting OpenBSD Secure Shell session cleanup...
         Starting Deferred execution scheduler...
[  OK  ] Started Atop process accounting daemon.
[  OK  ] Started Permit User Sessions.
[  OK  ] Started LSB: Brings up/down network automatically.
[  OK  ] Started /etc/rc.local Compatibility.
[  OK  ] Started LSB: powerful, eff…scalable Mail Transport Agent.
[  OK  ] Started OpenBSD Secure Shell session cleanup.
[  OK  ] Started Getty on tty1.
[  OK  ] Started Serial Getty on ttyS0.
[  OK  ] Reached target Login Prompts.
[  OK  ] Started Atop advanced performance monitor.
[   30.604143] vyos-router[781]: Stopping VyOS router:.
[   30.625452] vyos-router[781]: Un-mounting VyOS Config...umount: /opt/vyatta/config: not mounted.
[   30.645217] vyos-router[781]: failed.
[   30.759935] vyos-router[781]: Cannot stop watchfrr: pid file not found ... failed!
[   30.784550] vyos-router[781]: Cannot stop bfdd: pid file not found ...Cannot stop ldpd: pid file not found ...Cannot stop staticd: pid file not found ...Cannot stop ospf6d: pid file not found ... failed!
[   30.815647] vyos-router[781]: Cannot stop ospfd: pid file not found ...Cannot stop ripngd: pid file not found ...Cannot stop bgpd: pid file not found ...Cannot stop ripd: pid file not found ...Cannot stop pi!
[   30.854223] vyos-router[781]:  failed!
[   30.872191] vyos-router[781]: Cannot stop zebra: pid file not found ... failed!
[   30.889386] vyos-router[781]:  failed!
[   30.909276] vyos-router[781]:  failed!
[   30.925399] vyos-router[781]:  failed!
[   30.945384] vyos-router[781]:  failed!
[   30.961372] vyos-router[781]:  failed!
[   30.984204] vyos-router[781]:  failed!
[   31.204043] vyos-router[832]: Stopping VyOS router:.
[   31.221265] vyos-router[832]: Un-mounting VyOS Config...umount: /opt/vyatta/config: not mounted.
[   31.245512] vyos-router[832]: failed.
[   31.294656] vyos-router[832]: Cannot stop watchfrr: pid file not found ... failed!
[   31.321535] vyos-router[832]: Cannot stop bfdd: pid file not found ...Cannot stop staticd: pid file not found ...Cannot stop ldpd: pid file not found ...Cannot stop ospfd: pid file not found ...Cannot stop o!
[   31.367672] vyos-router[832]:  failed!
[   31.388707] vyos-router[832]: Cannot stop ripngd: pid file not found ...Cannot stop ripd: pid file not found ... failed!
[   31.410830] vyos-router[832]: Cannot stop zebra: pid file not found ... failed!
[   31.430017] vyos-router[832]:  failed!
[   31.445919] vyos-router[832]:  failed!
[   31.461962] vyos-router[832]:  failed!
[   31.481918] vyos-router[832]:  failed!
[   31.497287] vyos-router[832]:  failed!
[   31.515635] vyos-router[832]:  failed!
[   31.708769] vyos-router[885]: Stopping VyOS router:.
[   31.725407] vyos-router[885]: Un-mounting VyOS Config...umount: /opt/vyatta/config: not mounted.
[   31.745308] vyos-router[885]: failed.
[   31.821251] vyos-router[885]: Cannot stop watchfrr: pid file not found ... failed!
[   31.852459] vyos-router[885]: Cannot stop bfdd: pid file not found ...Cannot stop staticd: pid file not found ...Cannot stop ospfd: pid file not found ...Cannot stop ospf6d: pid file not found ...Cannot stop!
[   31.893410] vyos-router[885]: Cannot stop ldpd: pid file not found ...Cannot stop bgpd: pid file not found ...Cannot stop zebra: pid file not found ...Cannot stop pimd: pid file not found ... failed!
[   31.924473] vyos-router[885]:  failed!
[   31.939065] vyos-router[885]:  failed!
[   31.954615] vyos-router[885]:  failed!
[   31.970296] vyos-router[885]:  failed!
[   31.985971] vyos-router[885]:  failed!
[   32.001772] vyos-router[885]:  failed!
[   32.018021] vyos-router[885]:  failed!
[   32.033877] vyos-router[885]:  failed!
[   32.453459] vyos-router[945]: Stopping VyOS router:.
[   32.469337] vyos-router[945]: Un-mounting VyOS Config...umount: /opt/vyatta/config: not mounted.
[   32.493288] vyos-router[945]: failed.
[   32.545150] vyos-router[945]: Cannot stop watchfrr: pid file not found ... failed!
[   32.569678] vyos-router[945]: Cannot stop bfdd: pid file not found ...Cannot stop staticd: pid file not found ...Cannot stop ldpd: pid file not found ...Cannot stop ripngd: pid file not found ...Cannot stop !
[   32.611312] vyos-router[945]: Cannot stop ripd: pid file not found ...Cannot stop pimd: pid file not found ... failed!
[   32.633792] vyos-router[945]:  failed!
[   32.650346] vyos-router[945]: Cannot stop zebra: pid file not found ... failed!
[   32.670483] vyos-router[945]: Cannot stop bgpd: pid file not found ... failed!
[   32.689931] vyos-router[945]:  failed!
[   32.706614] vyos-router[945]:  failed!
[   32.721744] vyos-router[945]:  failed!
[   32.741392] vyos-router[945]:  failed!
[   32.758841] vyos-router[945]:  failed!
[   33.203468] vyos-router[1003]: Stopping VyOS router:.
[   33.217417] vyos-router[1003]: Un-mounting VyOS Config...umount: /opt/vyatta/config: not mounted.
[   33.237384] vyos-router[1003]: failed.
[   33.297037] vyos-router[1003]: Cannot stop watchfrr: pid file not found ... failed!
[   33.322478] vyos-router[1003]: Cannot stop bfdd: pid file not found ...Cannot stop staticd: pid file not found ...Cannot stop ldpd: pid file not found ...Cannot stop ospfd: pid file not found ...Cannot stop !
[   33.357403] vyos-router[1003]:  failed!
[   33.376416] vyos-router[1003]: Cannot stop pimd: pid file not found ...Cannot stop bgpd: pid file not found ...Cannot stop ripd: pid file not found ... failed!
[   33.402199] vyos-router[1003]: Cannot stop zebra: pid file not found ... failed!
[   33.421941] vyos-router[1003]:  failed!
[   33.438058] vyos-router[1003]:  failed!
[   33.453154] vyos-router[1003]:  failed!
[   33.469365] vyos-router[1003]:  failed!
[   33.488726] vyos-router[1003]:  failed!
[   33.502463] vyos-router[1003]:  failed!
[   33.915342] vyos-router[727]: Waiting for NICs to settle down:
[   33.934055] vyos-router[875]: Waiting for NICs to settle down:
[   33.950024] vyos-router[986]: Waiting for NICs to settle down:
[   33.970080] vyos-router[822]: Waiting for NICs to settle down:
[   33.990220] vyos-router[928]: Waiting for NICs to settle down:

It stays stuck there indefinitely.

Details

Difficulty level
Unknown (require assessment)
Version
1.3-rolling-202005270117
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Related Objects

Event Timeline

jjakob created this task.May 27 2020, 9:42 AM
jjakob created this object in space S1 VyOS Public.
jjakob updated the task description. (Show Details)
jjakob added a project: VyOS 1.3 Equuleus.
jjakob changed Version from - to 1.3-rolling-202005270117.
jjakob renamed this task from Restarting vyos-router.service results in a error loop requiring reboot to Restarting vyos-router.service breaks router.May 27 2020, 9:49 AM
jjakob triaged this task as High priority.
jjakob updated the task description. (Show Details)
jjakob added a comment.May 27 2020, 11:35 AM

ps afx shows frr processes still running. systemctl stop vyos-router; pkill -f '*.frr*.'; systemctl start vyos-router makes vyos-router start successfully, but with these errors:

vyos@vyos:~$ sudo systemctl start vyos-router
vyos@vyos:~$ [ 4541.956950] vyos-router[7284]: Waiting for NICs to settle down: settled in 0sec..
[ 4548.621921] vyos-router[7284]: Started watchfrr.
[ 4548.643030] vyos-router[7284]: Mounting VyOS Config...done.
[ 4549.353119] vyos-router[7284]: Starting VyOS router: migrate rl-system firewalliptables v1.8.2 (nf_tables): Chain already exists
[ 4549.368101] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.381141] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.386182] vyos-router[7284]: nfct v1.4.5: netlink error: Device or resource busy
[ 4549.390950] vyos-router[7284]: nfct v1.4.5: netlink error: Device or resource busy
[ 4549.395532] vyos-router[7284]: nfct v1.4.5: netlink error: Device or resource busy
[ 4549.446117] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.475652] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.500946] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.506182] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.511464] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.551861] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.556786] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.561782] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.606030] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.630639] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.652099] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.656959] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.661982] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.700813] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.705958] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.711002] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.750113] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.767296] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.784086] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.801496] vyos-router[7284]: ip6tables v1.8.2 (nf_tables): Chain already exists
[ 4549.818172] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4549.835676] vyos-router[7284]: iptables v1.8.2 (nf_tables): Chain already exists
[ 4551.931714] vyos-router[7284]:  configure
[ 4551.932595] vyos-router[7284]:     !!!!!
[ 4551.932954] vyos-router[7284]:     There were errors loading the configuration
[ 4551.933463] vyos-router[7284]:     Please examine the errors in
[ 4551.933785] vyos-router[7284]:     /tmp/boot-config-trace
[ 4551.934063] vyos-router[7284]:     and correct
[ 4551.934409] vyos-router[7284]:     !!!!!
[ 4551.934830] vyos-router[7284]:
[ 4552.048953] vyos-router[7284]:  failed!
pasik added a subscriber: pasik.May 27 2020, 5:13 PM
syncer changed the subtype of this task from "Task" to "Bug".May 30 2020, 3:21 PM
c-po merged a task: T2652: nat configuration conflicts with wan-load-balance nat rules.Jun 26 2020, 7:48 AM
c-po added a subscriber: Demon_H.
erkin set Issue type to Bug (incorrect behavior).Aug 30 2021, 5:51 AM
erkin removed a subscriber: Active contributors.
syncer edited projects, added VyOS 1.3 Equuleus (1.3.0); removed VyOS 1.3 Equuleus.Nov 6 2021, 11:24 AM
syncer edited projects, added VyOS 1.3 Equuleus (1.3.3); removed VyOS 1.3 Equuleus (1.3.0).Aug 29 2022, 7:05 AM
syncer closed this task as Wontfix.Jul 11 2023, 9:48 PM
syncer claimed this task.
syncer added a subscriber: syncer.

outdated

syncer moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus (1.3.3) board.Jul 12 2023, 10:24 PM