Change Details

During a network switches upgrade, three VyOS connected to them lost some IP routes. Both of them, BGP process stopped working, an the other lost its IP static routes. All of them are running version VyOS 1.2.6-S1. The normal situation recovered after rebooting those three VyOS. I got this log from VyOS stopped BGP process.... Mar 22 23:16:02 de-rhr-ba4ngwfi01-01 Keepalived_vrrp[5366]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 23:16:08 de-rhr-ba4ngwfi01-01 lldpd[5562]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 23:18:02 de-rhr-ba4ngwfi01-01 staticd[1201]: Terminating on signal Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.068687] INFO: task jbd2/sda1-8:245 blocked for more than 120 seconds. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.068966] Not tainted 4.19.142-amd64-vyos #1 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069078] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069245] Call Trace: Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069250] 0xffffffff95c098c0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069257] ? 0xffffffff95c0a530 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069257] 0xffffffff95c09d1d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069258] 0xffffffff95c0a06d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069259] 0xffffffff95c0a538 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069260] 0xffffffff95c0a0ee Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069260] 0xffffffff95c0a1a6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069261] ? 0xffffffff956a9630 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069264] 0xffffffffc0376c8a Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069265] ? 0xffffffff95e001a5 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069267] ? 0xffffffffc037a085 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069268] 0xffffffffc037a085 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069269] ? 0xffffffff956a8de0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069269] ? 0xffffffffc0379fb0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069270] 0xffffffff95687005 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069273] ? 0xffffffff95686f00 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069273] 0xffffffff95e00215 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069284] INFO: task snmpd:5553 blocked for more than 120 seconds. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069419] Not tainted 4.19.142-amd64-vyos #1 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069528] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069693] Call Trace: Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069694] 0xffffffff95c098c0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069695] ? 0xffffffff95c0a530 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069695] 0xffffffff95c09d1d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069696] 0xffffffff95c0a06d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069696] 0xffffffff95c0a538 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069697] 0xffffffff95c0a0ee Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069697] 0xffffffff95c0a1a6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069698] ? 0xffffffff956a9630 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069698] 0xffffffffc03749d6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069700] 0xffffffffc0374b97 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069700] 0xffffffffc03880f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069703] 0xffffffffc03b773d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069704] 0xffffffffc03b8f72 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069705] 0xffffffff957dbeeb Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069710] 0xffffffffc03f7343 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069711] 0xffffffff957dbeeb Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069712] 0xffffffff957de531 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069714] 0xffffffff95603518 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069717] 0xffffffff95e00088 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069718] RIP: 0033:0x00007f331e6f02f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069724] Code: Bad RIP value. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069725] RSP: 002b:00007ffc5fd15938 EFLAGS: 00000202 ORIG_RAX: 0000000000000057 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069726] RAX: ffffffffffffffda RBX: 00007ffc5fd15950 RCX: 00007f331e6f02f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069727] RDX: 0000000000000001 RSI: 00007ffc5fcf837c RDI: 00007ffc5fd15950 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069727] RBP: 000000000186c640 R08: 00007ffc5fcf83a0 R09: 0000000000000000 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069728] R10: 0000000000000000 R11: 0000000000000202 R12: 000000000186ca40 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069729] R13: 00007ffc5fd15d50 R14: 000000000186ca40 R15: 00007ffc5fd34728 Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop bfdd: pid file not found Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop staticd: pid file not found Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop ripngd: pid file not found And I got this message from VyOS lost IP static routes... Mar 22 22:07:37 de-rhr-ba4ngwft01-01 Keepalived_vrrp[30692]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 22:07:44 de-rhr-ba4ngwft01-01 lldpd[30792]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 22:07:54 de-rhr-ba4ngwft01-01 kernel: [14000186.691601] [VLAN2305-INBOUND-100-A] IN=eth3.2305 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:9a:31:08:00 SRC=10.4.233.68 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=20758 DF PROTO=TCP SPT=41822 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:08:34 de-rhr-ba4ngwft01-01 kernel: [14000227.135220] [VLAN2301-INBOUND-100-A] IN=eth3.2301 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:14:6f:08:00 SRC=10.4.232.68 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=37325 DF PROTO=TCP SPT=38064 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984298] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984313] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984343] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984345] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984370] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984372] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984376] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984378] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984381] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984383] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984448] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984452] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984468] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984470] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984757] IPv4: martian source 10.4.232.65 from 10.4.232.65, on dev eth3.2301 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984762] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984888] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984896] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984938] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984945] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:57 de-rhr-ba4ngwft01-01 staticd[1204]: Terminating on signal Mar 22 22:10:02 de-rhr-ba4ngwft01-01 kernel: [14000315.399864] [VLAN2300-INBOUND-100-A] IN=eth3.2300 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:35:68:08:00 SRC=10.4.232.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=23385 DF PROTO=TCP SPT=38098 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:10:03 de-rhr-ba4ngwft01-01 kernel: [14000316.397271] [VLAN2300-INBOUND-100-A] IN=eth3.2300 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:35:68:08:00 SRC=10.4.232.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=23386 DF PROTO=TCP SPT=38098 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.817745] INFO: task jbd2/sda1-8:246 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.817970] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818080] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818246] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818253] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818254] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818254] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818255] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818256] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818256] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818257] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818257] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818260] 0xffffffffc0286c8a Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818262] ? 0xffffffffb12001a5 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818263] ? 0xffffffffc028a085 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818264] 0xffffffffc028a085 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818265] ? 0xffffffffb0aa8de0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818265] ? 0xffffffffc0289fb0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818266] 0xffffffffb0a87005 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818267] ? 0xffffffffb0a86f00 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818267] 0xffffffffb1200215 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818275] INFO: task rs:main Q:Reg:26770 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818425] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818534] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818707] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818708] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818709] ? 0xffffffffb0f0e336 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818711] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818711] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818712] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818712] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818713] 0xffffffffc02849d6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818714] 0xffffffffc0284b97 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818715] 0xffffffffc02ee0f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818717] 0xffffffffc03077b5 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818718] 0xffffffffc0307995 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818719] ? 0xffffffffc0283a33 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818719] 0xffffffffc030b93e Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818720] 0xffffffffb0bf907d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818721] 0xffffffffb0be91bf Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818722] 0xffffffffb0be8bc9 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818722] 0xffffffffb0b4a944 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818723] 0xffffffffc02f7b86 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818724] 0xffffffffb0bc993b Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818725] 0xffffffffb0bcaafe Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818726] 0xffffffffc035b75d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818727] 0xffffffffb0bcbc8c Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818728] 0xffffffffb0bcbebe Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818728] 0xffffffffb0bcc155 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818729] 0xffffffffb0a03518 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818730] 0xffffffffb1200088 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818731] RIP: 0033:0x00007f6c2b4a7a3d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818740] Code: Bad RIP value. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818741] RSP: 002b:00007f6c23ffe600 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818742] RAX: ffffffffffffffda RBX: 00007f6c180203b0 RCX: 00007f6c2b4a7a3d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818742] RDX: 000000000000012a RSI: 00007f6c180203b0 RDI: 0000000000000006 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818743] RBP: 0000000000000000 R08: fffffffffffeac20 R09: fffffffffffeac10 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818744] R10: fffffffffffeac00 R11: 0000000000000293 R12: 00007f6c1800c3b0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818744] R13: 00007f6c23ffe620 R14: 0000000000000000 R15: 000000000000012a Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818746] INFO: task snmpd:30783 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818882] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818991] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819154] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819155] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819156] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819156] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819157] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819157] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819158] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819158] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819159] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819160] 0xffffffffc02849d6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819160] 0xffffffffc0284b97 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819161] 0xffffffffc02ee0f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819162] 0xffffffffc031d73d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819163] 0xffffffffc031ef72 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819163] 0xffffffffb0bdbeeb Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819164] 0xffffffffc035d343 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819165] 0xffffffffb0bdbeeb Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819166] 0xffffffffb0bde531 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819167] 0xffffffffb0a03518 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819167] 0xffffffffb1200088 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819168] RIP: 0033:0x00007f483ff012f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819169] Code: Bad RIP value. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819170] RSP: 002b:00007ffc774140c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000057 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819171] RAX: ffffffffffffffda RBX: 00007ffc774140e0 RCX: 00007f483ff012f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819171] RDX: 0000000000000001 RSI: 00007ffc773f6b0c RDI: 00007ffc774140e0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819172] RBP: 00000000013c2020 R08: 00007ffc773f6b30 R09: 0000000000000000 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819172] R10: 0000000000000000 R11: 0000000000000202 R12: 00000000013c2420 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819173] R13: 00007ffc774144e0 R14: 00000000013c2420 R15: 00007ffc77432eb8 Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 0.pool.ntp.org Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 1.pool.ntp.org Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 2.pool.ntp.org Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop bfdd: pid file not found Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop staticd: pid file not found Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop ripngd: pid file not found Mar 22 22:11:53 de-rhr-ba4ngwft01-01 kernel: [14000426.271769] [VLAN2316-INBOUND-100-A] IN=eth3.2316 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:d3:27:08:00 SRC=10.4.236.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=19925 DF PROTO=TCP SPT=52930 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:11:54 de-rhr-ba4ngwft01-01 kernel: [14000427.270625] [VLAN2316-INBOUND-100-A] IN=eth3.2316 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:d3:27:08:00 SRC=10.4.236.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=19926 DF PROTO=TCP SPT=52930 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:12:15 de-rhr-ba4ngwft01-01 kernel: [14000447.968880] [VLAN2311-INBOUND-100-A] IN=eth3.2311 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:c3:2f:08:00 SRC=10.4.234.196 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=58438 DF PROTO=TCP SPT=58792 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.645810] INFO: task jbd2/sda1-8:246 blocked for more than 120 seconds. Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.645970] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646079] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646248] Call Trace: Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646250] 0xffffffffb10098c0 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646252] ? 0xffffffffb100a530 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646252] 0xffffffffb1009d1d Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646253] 0xffffffffb100a06d Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646254] 0xffffffffb100a538 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646254] 0xffffffffb100a0ee Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646255] 0xffffffffb100a1a6 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646255] ? 0xffffffffb0aa9630 :

During a network switches upgrade, three VyOS connected to them lost some IP routes. Both of them, BGP process stopped working, an the other lost its IP static routes. All of them are running version VyOS 1.2.6-S1. The normal situation recovered after rebooting those three VyOS. Can anyone help me? thanks. I got this log from VyOS stopped BGP process.... Mar 22 23:16:02 de-rhr-ba4ngwfi01-01 Keepalived_vrrp[5366]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 23:16:08 de-rhr-ba4ngwfi01-01 lldpd[5562]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 23:18:02 de-rhr-ba4ngwfi01-01 staticd[1201]: Terminating on signal Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.068687] INFO: task jbd2/sda1-8:245 blocked for more than 120 seconds. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.068966] Not tainted 4.19.142-amd64-vyos #1 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069078] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069245] Call Trace: Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069250] 0xffffffff95c098c0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069257] ? 0xffffffff95c0a530 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069257] 0xffffffff95c09d1d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069258] 0xffffffff95c0a06d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069259] 0xffffffff95c0a538 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069260] 0xffffffff95c0a0ee Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069260] 0xffffffff95c0a1a6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069261] ? 0xffffffff956a9630 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069264] 0xffffffffc0376c8a Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069265] ? 0xffffffff95e001a5 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069267] ? 0xffffffffc037a085 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069268] 0xffffffffc037a085 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069269] ? 0xffffffff956a8de0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069269] ? 0xffffffffc0379fb0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069270] 0xffffffff95687005 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069273] ? 0xffffffff95686f00 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069273] 0xffffffff95e00215 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069284] INFO: task snmpd:5553 blocked for more than 120 seconds. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069419] Not tainted 4.19.142-amd64-vyos #1 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069528] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069693] Call Trace: Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069694] 0xffffffff95c098c0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069695] ? 0xffffffff95c0a530 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069695] 0xffffffff95c09d1d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069696] 0xffffffff95c0a06d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069696] 0xffffffff95c0a538 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069697] 0xffffffff95c0a0ee Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069697] 0xffffffff95c0a1a6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069698] ? 0xffffffff956a9630 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069698] 0xffffffffc03749d6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069700] 0xffffffffc0374b97 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069700] 0xffffffffc03880f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069703] 0xffffffffc03b773d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069704] 0xffffffffc03b8f72 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069705] 0xffffffff957dbeeb Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069710] 0xffffffffc03f7343 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069711] 0xffffffff957dbeeb Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069712] 0xffffffff957de531 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069714] 0xffffffff95603518 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069717] 0xffffffff95e00088 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069718] RIP: 0033:0x00007f331e6f02f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069724] Code: Bad RIP value. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069725] RSP: 002b:00007ffc5fd15938 EFLAGS: 00000202 ORIG_RAX: 0000000000000057 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069726] RAX: ffffffffffffffda RBX: 00007ffc5fd15950 RCX: 00007f331e6f02f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069727] RDX: 0000000000000001 RSI: 00007ffc5fcf837c RDI: 00007ffc5fd15950 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069727] RBP: 000000000186c640 R08: 00007ffc5fcf83a0 R09: 0000000000000000 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069728] R10: 0000000000000000 R11: 0000000000000202 R12: 000000000186ca40 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069729] R13: 00007ffc5fd15d50 R14: 000000000186ca40 R15: 00007ffc5fd34728 Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop bfdd: pid file not found Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop staticd: pid file not found Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop ripngd: pid file not found And I got this message from VyOS lost IP static routes... Mar 22 22:07:37 de-rhr-ba4ngwft01-01 Keepalived_vrrp[30692]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 22:07:44 de-rhr-ba4ngwft01-01 lldpd[30792]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 22:07:54 de-rhr-ba4ngwft01-01 kernel: [14000186.691601] [VLAN2305-INBOUND-100-A] IN=eth3.2305 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:9a:31:08:00 SRC=10.4.233.68 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=20758 DF PROTO=TCP SPT=41822 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:08:34 de-rhr-ba4ngwft01-01 kernel: [14000227.135220] [VLAN2301-INBOUND-100-A] IN=eth3.2301 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:14:6f:08:00 SRC=10.4.232.68 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=37325 DF PROTO=TCP SPT=38064 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984298] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984313] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984343] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984345] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984370] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984372] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984376] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984378] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984381] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984383] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984448] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984452] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984468] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984470] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984757] IPv4: martian source 10.4.232.65 from 10.4.232.65, on dev eth3.2301 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984762] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984888] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984896] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984938] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984945] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:57 de-rhr-ba4ngwft01-01 staticd[1204]: Terminating on signal Mar 22 22:10:02 de-rhr-ba4ngwft01-01 kernel: [14000315.399864] [VLAN2300-INBOUND-100-A] IN=eth3.2300 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:35:68:08:00 SRC=10.4.232.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=23385 DF PROTO=TCP SPT=38098 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:10:03 de-rhr-ba4ngwft01-01 kernel: [14000316.397271] [VLAN2300-INBOUND-100-A] IN=eth3.2300 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:35:68:08:00 SRC=10.4.232.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=23386 DF PROTO=TCP SPT=38098 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.817745] INFO: task jbd2/sda1-8:246 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.817970] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818080] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818246] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818253] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818254] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818254] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818255] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818256] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818256] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818257] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818257] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818260] 0xffffffffc0286c8a Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818262] ? 0xffffffffb12001a5 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818263] ? 0xffffffffc028a085 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818264] 0xffffffffc028a085 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818265] ? 0xffffffffb0aa8de0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818265] ? 0xffffffffc0289fb0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818266] 0xffffffffb0a87005 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818267] ? 0xffffffffb0a86f00 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818267] 0xffffffffb1200215 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818275] INFO: task rs:main Q:Reg:26770 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818425] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818534] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818707] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818708] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818709] ? 0xffffffffb0f0e336 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818711] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818711] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818712] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818712] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818713] 0xffffffffc02849d6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818714] 0xffffffffc0284b97 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818715] 0xffffffffc02ee0f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818717] 0xffffffffc03077b5 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818718] 0xffffffffc0307995 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818719] ? 0xffffffffc0283a33 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818719] 0xffffffffc030b93e Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818720] 0xffffffffb0bf907d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818721] 0xffffffffb0be91bf Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818722] 0xffffffffb0be8bc9 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818722] 0xffffffffb0b4a944 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818723] 0xffffffffc02f7b86 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818724] 0xffffffffb0bc993b Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818725] 0xffffffffb0bcaafe Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818726] 0xffffffffc035b75d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818727] 0xffffffffb0bcbc8c Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818728] 0xffffffffb0bcbebe Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818728] 0xffffffffb0bcc155 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818729] 0xffffffffb0a03518 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818730] 0xffffffffb1200088 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818731] RIP: 0033:0x00007f6c2b4a7a3d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818740] Code: Bad RIP value. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818741] RSP: 002b:00007f6c23ffe600 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818742] RAX: ffffffffffffffda RBX: 00007f6c180203b0 RCX: 00007f6c2b4a7a3d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818742] RDX: 000000000000012a RSI: 00007f6c180203b0 RDI: 0000000000000006 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818743] RBP: 0000000000000000 R08: fffffffffffeac20 R09: fffffffffffeac10 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818744] R10: fffffffffffeac00 R11: 0000000000000293 R12: 00007f6c1800c3b0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818744] R13: 00007f6c23ffe620 R14: 0000000000000000 R15: 000000000000012a Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818746] INFO: task snmpd:30783 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818882] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818991] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819154] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819155] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819156] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819156] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819157] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819157] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819158] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819158] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819159] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819160] 0xffffffffc02849d6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819160] 0xffffffffc0284b97 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819161] 0xffffffffc02ee0f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819162] 0xffffffffc031d73d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819163] 0xffffffffc031ef72 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819163] 0xffffffffb0bdbeeb Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819164] 0xffffffffc035d343 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819165] 0xffffffffb0bdbeeb Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819166] 0xffffffffb0bde531 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819167] 0xffffffffb0a03518 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819167] 0xffffffffb1200088 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819168] RIP: 0033:0x00007f483ff012f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819169] Code: Bad RIP value. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819170] RSP: 002b:00007ffc774140c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000057 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819171] RAX: ffffffffffffffda RBX: 00007ffc774140e0 RCX: 00007f483ff012f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819171] RDX: 0000000000000001 RSI: 00007ffc773f6b0c RDI: 00007ffc774140e0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819172] RBP: 00000000013c2020 R08: 00007ffc773f6b30 R09: 0000000000000000 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819172] R10: 0000000000000000 R11: 0000000000000202 R12: 00000000013c2420 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819173] R13: 00007ffc774144e0 R14: 00000000013c2420 R15: 00007ffc77432eb8 Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 0.pool.ntp.org Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 1.pool.ntp.org Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 2.pool.ntp.org Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop bfdd: pid file not found Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop staticd: pid file not found Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop ripngd: pid file not found Mar 22 22:11:53 de-rhr-ba4ngwft01-01 kernel: [14000426.271769] [VLAN2316-INBOUND-100-A] IN=eth3.2316 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:d3:27:08:00 SRC=10.4.236.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=19925 DF PROTO=TCP SPT=52930 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:11:54 de-rhr-ba4ngwft01-01 kernel: [14000427.270625] [VLAN2316-INBOUND-100-A] IN=eth3.2316 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:d3:27:08:00 SRC=10.4.236.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=19926 DF PROTO=TCP SPT=52930 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:12:15 de-rhr-ba4ngwft01-01 kernel: [14000447.968880] [VLAN2311-INBOUND-100-A] IN=eth3.2311 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:c3:2f:08:00 SRC=10.4.234.196 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=58438 DF PROTO=TCP SPT=58792 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.645810] INFO: task jbd2/sda1-8:246 blocked for more than 120 seconds. Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.645970] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646079] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646248] Call Trace: Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646250] 0xffffffffb10098c0 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646252] ? 0xffffffffb100a530 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646252] 0xffffffffb1009d1d Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646253] 0xffffffffb100a06d Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646254] 0xffffffffb100a538 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646254] 0xffffffffb100a0ee Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646255] 0xffffffffb100a1a6 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646255] ? 0xffffffffb0aa9630 :

During a network switches upgrade, three VyOS connected to them lost some IP routes. Both of them, BGP process stopped working, an the other lost its IP static routes. All of them are running version VyOS 1.2.6-S1. The normal situation recovered after rebooting those three VyOS. Can anyone help me? thanks. I got this log from VyOS stopped BGP process.... Mar 22 23:16:02 de-rhr-ba4ngwfi01-01 Keepalived_vrrp[5366]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 23:16:08 de-rhr-ba4ngwfi01-01 lldpd[5562]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 23:18:02 de-rhr-ba4ngwfi01-01 staticd[1201]: Terminating on signal Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.068687] INFO: task jbd2/sda1-8:245 blocked for more than 120 seconds. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.068966] Not tainted 4.19.142-amd64-vyos #1 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069078] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069245] Call Trace: Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069250] 0xffffffff95c098c0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069257] ? 0xffffffff95c0a530 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069257] 0xffffffff95c09d1d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069258] 0xffffffff95c0a06d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069259] 0xffffffff95c0a538 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069260] 0xffffffff95c0a0ee Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069260] 0xffffffff95c0a1a6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069261] ? 0xffffffff956a9630 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069264] 0xffffffffc0376c8a Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069265] ? 0xffffffff95e001a5 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069267] ? 0xffffffffc037a085 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069268] 0xffffffffc037a085 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069269] ? 0xffffffff956a8de0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069269] ? 0xffffffffc0379fb0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069270] 0xffffffff95687005 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069273] ? 0xffffffff95686f00 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069273] 0xffffffff95e00215 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069284] INFO: task snmpd:5553 blocked for more than 120 seconds. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069419] Not tainted 4.19.142-amd64-vyos #1 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069528] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069693] Call Trace: Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069694] 0xffffffff95c098c0 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069695] ? 0xffffffff95c0a530 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069695] 0xffffffff95c09d1d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069696] 0xffffffff95c0a06d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069696] 0xffffffff95c0a538 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069697] 0xffffffff95c0a0ee Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069697] 0xffffffff95c0a1a6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069698] ? 0xffffffff956a9630 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069698] 0xffffffffc03749d6 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069700] 0xffffffffc0374b97 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069700] 0xffffffffc03880f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069703] 0xffffffffc03b773d Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069704] 0xffffffffc03b8f72 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069705] 0xffffffff957dbeeb Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069710] 0xffffffffc03f7343 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069711] 0xffffffff957dbeeb Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069712] 0xffffffff957de531 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069714] 0xffffffff95603518 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069717] 0xffffffff95e00088 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069718] RIP: 0033:0x00007f331e6f02f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069724] Code: Bad RIP value. Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069725] RSP: 002b:00007ffc5fd15938 EFLAGS: 00000202 ORIG_RAX: 0000000000000057 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069726] RAX: ffffffffffffffda RBX: 00007ffc5fd15950 RCX: 00007f331e6f02f7 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069727] RDX: 0000000000000001 RSI: 00007ffc5fcf837c RDI: 00007ffc5fd15950 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069727] RBP: 000000000186c640 R08: 00007ffc5fcf83a0 R09: 0000000000000000 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069728] R10: 0000000000000000 R11: 0000000000000202 R12: 000000000186ca40 Mar 22 23:18:18 de-rhr-ba4ngwfi01-01 kernel: [14000307.069729] R13: 00007ffc5fd15d50 R14: 000000000186ca40 R15: 00007ffc5fd34728 Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop bfdd: pid file not found Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop staticd: pid file not found Mar 22 23:19:22 de-rhr-ba4ngwfi01-01 watchfrr.sh: Cannot stop ripngd: pid file not found And I got this message from VyOS lost IP static routes... Mar 22 22:07:37 de-rhr-ba4ngwft01-01 Keepalived_vrrp[30692]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 22:07:44 de-rhr-ba4ngwft01-01 lldpd[30792]: AgentX master agent failed to respond to ping. Attempting to re-register. Mar 22 22:07:54 de-rhr-ba4ngwft01-01 kernel: [14000186.691601] [VLAN2305-INBOUND-100-A] IN=eth3.2305 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:9a:31:08:00 SRC=10.4.233.68 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=20758 DF PROTO=TCP SPT=41822 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:08:34 de-rhr-ba4ngwft01-01 kernel: [14000227.135220] [VLAN2301-INBOUND-100-A] IN=eth3.2301 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:14:6f:08:00 SRC=10.4.232.68 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=37325 DF PROTO=TCP SPT=38064 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984298] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984313] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984343] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984345] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984370] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984372] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984376] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984378] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984381] IPv4: martian source 10.4.6.19 from 10.4.6.19, on dev eth1.180 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984383] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd 0b 67 08 06 .......PV..g.. Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984448] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984452] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984468] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984470] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984757] IPv4: martian source 10.4.232.65 from 10.4.232.65, on dev eth3.2301 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984762] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984888] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984896] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984938] IPv4: martian source 10.4.232.1 from 10.4.232.1, on dev eth3.2300 Mar 22 22:09:07 de-rhr-ba4ngwft01-01 kernel: [14000259.984945] ll header: 00000000: ff ff ff ff ff ff 00 50 56 bd bd db 08 06 .......PV..... Mar 22 22:09:57 de-rhr-ba4ngwft01-01 staticd[1204]: Terminating on signal Mar 22 22:10:02 de-rhr-ba4ngwft01-01 kernel: [14000315.399864] [VLAN2300-INBOUND-100-A] IN=eth3.2300 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:35:68:08:00 SRC=10.4.232.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=23385 DF PROTO=TCP SPT=38098 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:10:03 de-rhr-ba4ngwft01-01 kernel: [14000316.397271] [VLAN2300-INBOUND-100-A] IN=eth3.2300 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:35:68:08:00 SRC=10.4.232.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=23386 DF PROTO=TCP SPT=38098 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.817745] INFO: task jbd2/sda1-8:246 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.817970] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818080] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818246] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818253] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818254] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818254] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818255] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818256] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818256] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818257] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818257] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818260] 0xffffffffc0286c8a Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818262] ? 0xffffffffb12001a5 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818263] ? 0xffffffffc028a085 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818264] 0xffffffffc028a085 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818265] ? 0xffffffffb0aa8de0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818265] ? 0xffffffffc0289fb0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818266] 0xffffffffb0a87005 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818267] ? 0xffffffffb0a86f00 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818267] 0xffffffffb1200215 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818275] INFO: task rs:main Q:Reg:26770 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818425] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818534] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818707] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818708] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818709] ? 0xffffffffb0f0e336 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818710] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818711] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818711] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818712] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818712] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818713] 0xffffffffc02849d6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818714] 0xffffffffc0284b97 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818715] 0xffffffffc02ee0f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818717] 0xffffffffc03077b5 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818718] 0xffffffffc0307995 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818719] ? 0xffffffffc0283a33 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818719] 0xffffffffc030b93e Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818720] 0xffffffffb0bf907d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818721] 0xffffffffb0be91bf Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818722] 0xffffffffb0be8bc9 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818722] 0xffffffffb0b4a944 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818723] 0xffffffffc02f7b86 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818724] 0xffffffffb0bc993b Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818725] 0xffffffffb0bcaafe Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818726] 0xffffffffc035b75d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818727] 0xffffffffb0bcbc8c Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818728] 0xffffffffb0bcbebe Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818728] 0xffffffffb0bcc155 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818729] 0xffffffffb0a03518 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818730] 0xffffffffb1200088 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818731] RIP: 0033:0x00007f6c2b4a7a3d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818740] Code: Bad RIP value. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818741] RSP: 002b:00007f6c23ffe600 EFLAGS: 00000293 ORIG_RAX: 0000000000000001 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818742] RAX: ffffffffffffffda RBX: 00007f6c180203b0 RCX: 00007f6c2b4a7a3d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818742] RDX: 000000000000012a RSI: 00007f6c180203b0 RDI: 0000000000000006 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818743] RBP: 0000000000000000 R08: fffffffffffeac20 R09: fffffffffffeac10 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818744] R10: fffffffffffeac00 R11: 0000000000000293 R12: 00007f6c1800c3b0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818744] R13: 00007f6c23ffe620 R14: 0000000000000000 R15: 000000000000012a Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818746] INFO: task snmpd:30783 blocked for more than 120 seconds. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818882] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.818991] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819154] Call Trace: Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819155] 0xffffffffb10098c0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819156] ? 0xffffffffb100a530 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819156] 0xffffffffb1009d1d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819157] 0xffffffffb100a06d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819157] 0xffffffffb100a538 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819158] 0xffffffffb100a0ee Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819158] 0xffffffffb100a1a6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819159] ? 0xffffffffb0aa9630 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819160] 0xffffffffc02849d6 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819160] 0xffffffffc0284b97 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819161] 0xffffffffc02ee0f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819162] 0xffffffffc031d73d Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819163] 0xffffffffc031ef72 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819163] 0xffffffffb0bdbeeb Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819164] 0xffffffffc035d343 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819165] 0xffffffffb0bdbeeb Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819166] 0xffffffffb0bde531 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819167] 0xffffffffb0a03518 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819167] 0xffffffffb1200088 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819168] RIP: 0033:0x00007f483ff012f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819169] Code: Bad RIP value. Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819170] RSP: 002b:00007ffc774140c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000057 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819171] RAX: ffffffffffffffda RBX: 00007ffc774140e0 RCX: 00007f483ff012f7 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819171] RDX: 0000000000000001 RSI: 00007ffc773f6b0c RDI: 00007ffc774140e0 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819172] RBP: 00000000013c2020 R08: 00007ffc773f6b30 R09: 0000000000000000 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819172] R10: 0000000000000000 R11: 0000000000000202 R12: 00000000013c2420 Mar 22 22:10:15 de-rhr-ba4ngwft01-01 kernel: [14000327.819173] R13: 00007ffc774144e0 R14: 00000000013c2420 R15: 00007ffc77432eb8 Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 0.pool.ntp.org Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 1.pool.ntp.org Mar 22 22:11:16 de-rhr-ba4ngwft01-01 ntpd_intres[26697]: host name not found: 2.pool.ntp.org Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop bfdd: pid file not found Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop staticd: pid file not found Mar 22 22:11:17 de-rhr-ba4ngwft01-01 watchfrr.sh: Cannot stop ripngd: pid file not found Mar 22 22:11:53 de-rhr-ba4ngwft01-01 kernel: [14000426.271769] [VLAN2316-INBOUND-100-A] IN=eth3.2316 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:d3:27:08:00 SRC=10.4.236.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=19925 DF PROTO=TCP SPT=52930 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:11:54 de-rhr-ba4ngwft01-01 kernel: [14000427.270625] [VLAN2316-INBOUND-100-A] IN=eth3.2316 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:d3:27:08:00 SRC=10.4.236.4 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=19926 DF PROTO=TCP SPT=52930 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:12:15 de-rhr-ba4ngwft01-01 kernel: [14000447.968880] [VLAN2311-INBOUND-100-A] IN=eth3.2311 OUT=eth1.200 MAC=00:50:56:bd:f9:1f:00:50:56:bd:c3:2f:08:00 SRC=10.4.234.196 DST=208.91.0.89 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=58438 DF PROTO=TCP SPT=58792 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.645810] INFO: task jbd2/sda1-8:246 blocked for more than 120 seconds. Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.645970] Not tainted 4.19.142-amd64-vyos #1 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646079] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646248] Call Trace: Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646250] 0xffffffffb10098c0 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646252] ? 0xffffffffb100a530 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646252] 0xffffffffb1009d1d Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646253] 0xffffffffb100a06d Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646254] 0xffffffffb100a538 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646254] 0xffffffffb100a0ee Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646255] 0xffffffffb100a1a6 Mar 22 22:12:16 de-rhr-ba4ngwft01-01 kernel: [14000448.646255] ? 0xffffffffb0aa9630 :