Hi,
We were using vyos v1.1.7 almost for 1,5 years with no problems at all to connect to AWS VPN. We have 2 vyos in cluster, with each one have 1 connection to AWS, 4 tunnels and configured BGP.
From last month, every 2/3/5 days (random) we experienced that traffic through VPN tunnels just stops, so we upgraded to v1.1.8., thinking that we resolved our problem. But everything is the same, and there are no logs what is wrong (or I don't see it), so I would need some help.
The symptoms are:
- IPSEC proceses are ok, both VTI's are up/up
- BGP drops routes for some reason
- ping doesn't work
We checked with our ISP provider and every time traffic stops, there is an route path calculation on one of links that they connect to AWS (it lasts for few seconds). And few minutes (10, sometimes 30 minutes later) we experience problems.
But after that when we restart ipesc service everything comes up and traffic goes as nothing happen.
I think that Dead-peer-detection is not working as it should or BGP, but can not confirm that.
Can someone help?
Logs before restart:
Feb 1 16:45:35 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:45:36 vyos01 pluto[8741]: "[AWS_vti_01]" #5701: replacing stale IPsec SA Feb 1 16:45:36 vyos01 pluto[8741]: "[AWS_vti_01]" #5704: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP to replace #5701 {using isakmp#5699} Feb 1 16:45:36 vyos01 pluto[8741]: "[AWS_vti_01]" #5704: Dead Peer Detection (RFC 3706) enabled Feb 1 16:45:36 vyos01 pluto[8741]: "[AWS_vti_01]" #5704: sent QI2, IPsec SA established {ESP=>0x9bd614a3 <0xc4ac7fe5} Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:45:45 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:45:46 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:45:55 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:46:01 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (routeadv timer expire) Feb 1 16:46:01 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (routeadv timer expire) Feb 1 16:46:01 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:46:05 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:46:12 vyos01 pluto[8741]: forgetting secrets Feb 1 16:46:12 vyos01 pluto[8741]: loading secrets from "/etc/ipsec.secrets" Feb 1 16:46:12 vyos01 pluto[8741]: loaded PSK secret for 192.168.0.190 [AWS_vti_01] Feb 1 16:46:12 vyos01 pluto[8741]: loaded PSK secret for 192.168.0.190 [AWS_vti_02] Feb 1 16:46:12 vyos01 pluto[8741]: loading secrets from "/etc/dmvpn.secrets" Feb 1 16:46:12 vyos01 pluto[8741]: Changing to directory '/etc/ipsec.d/crls' Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:46:15 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:46:16 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:46:22 vyos01 bgpd[2459]: Performing BGP general scanning Feb 1 16:46:22 vyos01 bgpd[2459]: scanning IPv4 Unicast routing tables Feb 1 16:46:22 vyos01 bgpd[2459]: scanning IPv6 Unicast routing tables Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:46:25 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:46:31 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (routeadv timer expire) Feb 1 16:46:31 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (routeadv timer expire) Feb 1 16:46:31 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:46:35 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:46:45 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:46:46 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:46:55 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:46:57 vyos01 pluto[8741]: forgetting secrets Feb 1 16:46:57 vyos01 pluto[8741]: loading secrets from "/etc/ipsec.secrets" Feb 1 16:46:57 vyos01 pluto[8741]: loaded PSK secret for 192.168.0.190 [AWS_vti_01] Feb 1 16:46:57 vyos01 pluto[8741]: loaded PSK secret for 192.168.0.190 [AWS_vti_02] Feb 1 16:46:57 vyos01 pluto[8741]: loading secrets from "/etc/dmvpn.secrets" Feb 1 16:46:57 vyos01 pluto[8741]: Changing to directory '/etc/ipsec.d/crls' Feb 1 16:46:58 vyos01 pluto[8741]: "[AWS_vti_02]" #5695: IPsec SA expired (superseded by #5703) Feb 1 16:46:58 vyos01 pluto[8741]: "[AWS_vti_02]" #5694: ignoring Delete SA payload: PROTO_IPSEC_ESP SA(0x2b4ad0ed) not found (maybe expired) Feb 1 16:47:01 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (routeadv timer expire) Feb 1 16:47:01 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (routeadv timer expire) Feb 1 16:47:01 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:47:05 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:47:15 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:47:16 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:47:22 vyos01 pluto[8741]: "[AWS_vti_01]" #5686: ISAKMP SA expired (superseded by #5699) Feb 1 16:47:22 vyos01 pluto[8741]: packet from 52.58.104.97:500: Informational Exchange is for an unknown (expired?) SA Feb 1 16:47:22 vyos01 bgpd[2459]: Performing BGP general scanning Feb 1 16:47:22 vyos01 bgpd[2459]: scanning IPv4 Unicast routing tables Feb 1 16:47:22 vyos01 bgpd[2459]: scanning IPv6 Unicast routing tables Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:47:25 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:47:31 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (routeadv timer expire) Feb 1 16:47:31 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (routeadv timer expire) Feb 1 16:47:31 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:47:35 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:47:42 vyos01 pluto[8741]: forgetting secrets Feb 1 16:47:42 vyos01 pluto[8741]: loading secrets from "/etc/ipsec.secrets" Feb 1 16:47:42 vyos01 pluto[8741]: loaded PSK secret for 192.168.0.190 [AWS_vti_01] Feb 1 16:47:42 vyos01 pluto[8741]: loaded PSK secret for 192.168.0.190 [AWS_vti_02] Feb 1 16:47:42 vyos01 pluto[8741]: loading secrets from "/etc/dmvpn.secrets" Feb 1 16:47:42 vyos01 pluto[8741]: Changing to directory '/etc/ipsec.d/crls' Feb 1 16:47:45 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:47:45 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:47:45 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:47:45 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:47:45 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:47:45 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:47:46 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:47:47 vyos01 pluto[8741]: "[AWS_vti_01]" #5699: received Delete SA(0xc98c8b65) payload: deleting IPSEC State #5696 Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 3, length (excl. header) 2 Feb 1 16:47:55 vyos01 bgpd[2459]: %NOTIFICATION: received from neighbor 169.254.40.61 4/0 (Hold Timer Expired) 0 bytes Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Receive_NOTIFICATION_message (Established->Clearing) Feb 1 16:47:55 vyos01 bgpd[2459]: %ADJCHANGE: neighbor 169.254.40.61 Down BGP Notification received Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 went from Established to Clearing Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 3, length (excl. header) 2 Feb 1 16:47:55 vyos01 bgpd[2459]: %NOTIFICATION: received from neighbor 169.254.40.169 4/0 (Hold Timer Expired) 0 bytes Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Receive_NOTIFICATION_message (Established->Clearing) Feb 1 16:47:55 vyos01 bgpd[2459]: %ADJCHANGE: neighbor 169.254.40.169 Down BGP Notification received Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 went from Established to Clearing Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Clearing_Completed (Clearing->Idle) Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.61 went from Clearing to Idle Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Clearing_Completed (Clearing->Idle) Feb 1 16:47:55 vyos01 bgpd[2459]: 169.254.40.169 went from Clearing to Idle Feb 1 16:47:55 vyos01 bgpd[2459]: Zebra send: IPv4 route delete 10.50.0.0/16 nexthop 169.254.40.169 metric 100 Feb 1 16:47:58 vyos01 pluto[8741]: "[AWS_vti_02]" #5703: replacing stale IPsec SA Feb 1 16:47:58 vyos01 pluto[8741]: "[AWS_vti_02]" #5705: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP to replace #5703 {using isakmp#5694} Feb 1 16:48:01 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (start timer expire). Feb 1 16:48:01 vyos01 bgpd[2459]: 169.254.40.61 [FSM] BGP_Start (Idle->Connect) Feb 1 16:48:01 vyos01 bgpd[2459]: 169.254.40.61 [Event] Connect start to 169.254.40.61 fd 8 Feb 1 16:48:01 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Non blocking connect waiting result Feb 1 16:48:01 vyos01 bgpd[2459]: 169.254.40.61 went from Idle to Connect Feb 1 16:48:01 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_01]" #5699: received Delete SA payload: replace IPSEC State #5704 in 10 seconds Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_01]" #5699: received Delete SA(0x5902e492) payload: deleting IPSEC State #5701 Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_01]" #5699: received Delete SA(0x2f42a949) payload: deleting IPSEC State #5700 Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_01]" #5699: received Delete SA payload: deleting ISAKMP State #5699 Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_02]" #5694: received Delete SA payload: replace IPSEC State #5703 in 10 seconds Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_02]" #5694: received Delete SA(0x368ef2ed) payload: deleting IPSEC State #5702 Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_02]" #5694: received Delete SA(0x12b03fa2) payload: deleting IPSEC State #5698 Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_02]" #5694: received Delete SA(0x29562d00) payload: deleting IPSEC State #5697 Feb 1 16:48:02 vyos01 pluto[8741]: "[AWS_vti_02]" #5694: received Delete SA payload: deleting ISAKMP State #5694 Feb 1 16:48:06 vyos01 pluto[8741]: "[AWS_vti_01]" #5704: DPD: Could not find newest phase 1 state
After restart:
Feb 1 16:48:09 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (start timer expire). Feb 1 16:48:09 vyos01 bgpd[2459]: 169.254.40.169 [FSM] BGP_Start (Idle->Connect) Feb 1 16:48:09 vyos01 bgpd[2459]: 169.254.40.169 [Event] Connect start to 169.254.40.169 fd 11 Feb 1 16:48:09 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Non blocking connect waiting result Feb 1 16:48:09 vyos01 bgpd[2459]: 169.254.40.169 went from Idle to Connect Feb 1 16:48:10 vyos01 pluto[8741]: shutting down Feb 1 16:48:10 vyos01 pluto[8741]: forgetting secrets Feb 1 16:48:10 vyos01 pluto[8741]: "[AWS_vti_01]": deleting connection Feb 1 16:48:10 vyos01 pluto[8741]: "[AWS_vti_01]" #5704: deleting state (STATE_QUICK_I2) Feb 1 16:48:10 vyos01 zebra[2449]: interface vti1 index 6 changed <POINTOPOINT,NOARP>. Feb 1 16:48:10 vyos01 bgpd[2459]: Zebra rcvd: interface vti1 down Feb 1 16:48:10 vyos01 pluto[8741]: "[AWS_vti_02]": deleting connection Feb 1 16:48:10 vyos01 pluto[8741]: "[AWS_vti_02]" #5705: deleting state (STATE_QUICK_I1) Feb 1 16:48:10 vyos01 pluto[8741]: "[AWS_vti_02]" #5703: deleting state (STATE_QUICK_I2) Feb 1 16:48:10 vyos01 zebra[2449]: interface vti0 index 5 changed <POINTOPOINT,NOARP>. Feb 1 16:48:10 vyos01 bgpd[2459]: Zebra rcvd: interface vti0 down Feb 1 16:48:10 vyos01 pluto[8741]: shutting down interface lo/lo ::1 Feb 1 16:48:10 vyos01 pluto[8741]: shutting down interface lo/lo 127.0.0.1 Feb 1 16:48:10 vyos01 pluto[8741]: shutting down interface eth3/eth3 192.168.0.190 Feb 1 16:48:10 vyos01 pluto[8741]: shutting down interface eth2/eth2 192.168.10.190 Feb 1 16:48:10 vyos01 ipsec_starter[8740]: pluto stopped after 160 ms Feb 1 16:48:10 vyos01 charon: 00[DMN] signal of type SIGINT received. Shutting down Feb 1 16:48:10 vyos01 ipsec_starter[8740]: charon stopped after 200 ms Feb 1 16:48:10 vyos01 ipsec_starter[8740]: ipsec starter stopped Feb 1 16:48:11 vyos01 ipsec_starter[17266]: Starting strongSwan 4.5.2 IPsec [starter]... Feb 1 16:48:11 vyos01 pluto[17275]: Starting IKEv1 pluto daemon (strongSwan 4.5.2) THREADS SMARTCARD VENDORID CISCO_QUIRKS Feb 1 16:48:11 vyos01 pluto[17275]: failed to load pkcs11 module '/usr/lib/opensc-pkcs11.so' Feb 1 16:48:11 vyos01 ipsec_starter[17274]: pluto (17275) started after 20 ms Feb 1 16:48:11 vyos01 charon: 00[DMN] Starting IKEv2 charon daemon (strongSwan 4.5.2) Feb 1 16:48:11 vyos01 charon: 00[KNL] listening on interfaces: Feb 1 16:48:11 vyos01 charon: 00[KNL] eth3 Feb 1 16:48:11 vyos01 charon: 00[KNL] 192.168.0.190 Feb 1 16:48:11 vyos01 charon: 00[KNL] fe80::250:56ff:feb7:2648 Feb 1 16:48:11 vyos01 charon: 00[KNL] eth2 Feb 1 16:48:11 vyos01 charon: 00[KNL] 192.168.10.190 Feb 1 16:48:11 vyos01 charon: 00[KNL] fe80::250:56ff:feb7:950 Feb 1 16:48:11 vyos01 charon: 00[CFG] loading ca certificates from '/etc/ipsec.d/cacerts' Feb 1 16:48:11 vyos01 charon: 00[CFG] loading aa certificates from '/etc/ipsec.d/aacerts' Feb 1 16:48:11 vyos01 charon: 00[CFG] loading ocsp signer certificates from '/etc/ipsec.d/ocspcerts' Feb 1 16:48:11 vyos01 charon: 00[CFG] loading attribute certificates from '/etc/ipsec.d/acerts' Feb 1 16:48:11 vyos01 charon: 00[CFG] loading crls from '/etc/ipsec.d/crls' Feb 1 16:48:11 vyos01 charon: 00[CFG] loading secrets from '/etc/ipsec.secrets' Feb 1 16:48:11 vyos01 charon: 00[CFG] loaded IKE secret for 192.168.0.190 [AWS_vti_01] Feb 1 16:48:11 vyos01 charon: 00[CFG] loaded IKE secret for 192.168.0.190 [AWS_vti_02] Feb 1 16:48:11 vyos01 charon: 00[CFG] loading secrets from '/etc/dmvpn.secrets' Feb 1 16:48:11 vyos01 charon: 00[CFG] sql plugin: database URI not set Feb 1 16:48:11 vyos01 charon: 00[LIB] plugin 'sql': failed to load - sql_plugin_create returned NULL Feb 1 16:48:11 vyos01 charon: 00[CFG] loaded 0 RADIUS server configurations Feb 1 16:48:11 vyos01 charon: 00[CFG] HA config misses local/remote address Feb 1 16:48:11 vyos01 charon: 00[LIB] plugin 'ha': failed to load - ha_plugin_create returned NULL Feb 1 16:48:11 vyos01 charon: 00[DMN] loaded plugins: test-vectors curl ldap aes des sha1 sha2 md5 random x509 revocation constraints pubkey pkcs1 pgp pem openssl fips-prf gmp agent pkcs11 xcbc hmac ctr ccm gcm attr kernel-netlink resolve socket-raw farp stroke updown eap-identity eap-aka eap-md5 eap-gtc eap-mschapv2 eap-radius eap-tls eap-ttls eap-tnc dhcp led addrblock Feb 1 16:48:11 vyos01 charon: 00[JOB] spawning 16 worker threads Feb 1 16:48:11 vyos01 pluto[17275]: Changing to directory '/etc/ipsec.d/crls' Feb 1 16:48:11 vyos01 pluto[17275]: listening for IKE messages Feb 1 16:48:11 vyos01 pluto[17275]: adding interface eth2/eth2 192.168.10.190:500 Feb 1 16:48:11 vyos01 pluto[17275]: adding interface eth3/eth3 192.168.0.190:500 Feb 1 16:48:11 vyos01 pluto[17275]: adding interface lo/lo 127.0.0.1:500 Feb 1 16:48:11 vyos01 pluto[17275]: adding interface lo/lo ::1:500 Feb 1 16:48:11 vyos01 pluto[17275]: loading secrets from "/etc/ipsec.secrets" Feb 1 16:48:11 vyos01 pluto[17275]: loaded PSK secret for 192.168.0.190 [AWS_vti_01] Feb 1 16:48:11 vyos01 pluto[17275]: loaded PSK secret for 192.168.0.190 [AWS_vti_02] Feb 1 16:48:11 vyos01 pluto[17275]: loading secrets from "/etc/dmvpn.secrets" Feb 1 16:48:11 vyos01 ipsec_starter[17274]: charon (17341) started after 20 ms Feb 1 16:48:11 vyos01 charon: 07[CFG] received stroke: add connection '[AWS_vti_02]' Feb 1 16:48:11 vyos01 charon: 07[CFG] added configuration '[AWS_vti_02]' Feb 1 16:48:11 vyos01 charon: 07[CFG] received stroke: add connection '[AWS_vti_01]' Feb 1 16:48:11 vyos01 pluto[17275]: added connection description "[AWS_vti_02]" Feb 1 16:48:11 vyos01 charon: 07[CFG] added configuration '[AWS_vti_01]' Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]": route-client output: perl: warning: Setting locale failed. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]": route-client output: perl: warning: Please check that your locale settings: Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]": route-client output: \011LANGUAGE = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]": route-client output: \011LC_ALL = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]": route-client output: \011LANG = "en_US.UTF-8" Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]": route-client output: are supported and installed on your system. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]": route-client output: perl: warning: Falling back to the standard locale ("C"). Feb 1 16:48:11 vyos01 zebra[2449]: interface vti0 index 5 changed <UP,POINTOPOINT,RUNNING,NOARP>. Feb 1 16:48:11 vyos01 bgpd[2459]: Zebra rcvd: interface vti0 up Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #1: initiating Main Mode Feb 1 16:48:11 vyos01 pluto[17275]: added connection description "[AWS_vti_01]" Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]": route-client output: perl: warning: Setting locale failed. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]": route-client output: perl: warning: Please check that your locale settings: Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]": route-client output: \011LANGUAGE = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]": route-client output: \011LC_ALL = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]": route-client output: \011LANG = "en_US.UTF-8" Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]": route-client output: are supported and installed on your system. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]": route-client output: perl: warning: Falling back to the standard locale ("C"). Feb 1 16:48:11 vyos01 zebra[2449]: interface vti1 index 6 changed <UP,POINTOPOINT,RUNNING,NOARP>. Feb 1 16:48:11 vyos01 bgpd[2459]: Zebra rcvd: interface vti1 up Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #1: received Vendor ID payload [Dead Peer Detection] Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #2: initiating Main Mode Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #2: received Vendor ID payload [Dead Peer Detection] Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #1: Peer ID is ID_IPV4_ADDR: '[AWS_vti_01]' Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #1: ISAKMP SA established Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP {using isakmp#1} Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #2: Peer ID is ID_IPV4_ADDR: '[AWS_vti_02]' Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #2: ISAKMP SA established Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP {using isakmp#2} Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: up-client output: perl: warning: Setting locale failed. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: up-client output: perl: warning: Please check that your locale settings: Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: up-client output: \011LANGUAGE = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: up-client output: \011LC_ALL = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: up-client output: \011LANG = "en_US.UTF-8" Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: up-client output: are supported and installed on your system. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: up-client output: perl: warning: Falling back to the standard locale ("C"). Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: Dead Peer Detection (RFC 3706) enabled Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_02]" #3: sent QI2, IPsec SA established {ESP=>0x4ff64e1e <0xcbcc8420} Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: up-client output: perl: warning: Setting locale failed. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: up-client output: perl: warning: Please check that your locale settings: Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: up-client output: \011LANGUAGE = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: up-client output: \011LC_ALL = (unset), Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: up-client output: \011LANG = "en_US.UTF-8" Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: up-client output: are supported and installed on your system. Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: up-client output: perl: warning: Falling back to the standard locale ("C"). Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: Dead Peer Detection (RFC 3706) enabled Feb 1 16:48:11 vyos01 pluto[17275]: "[AWS_vti_01]" #4: sent QI2, IPsec SA established {ESP=>0xf52b5d00 <0xc442253d} Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 [FSM] TCP_connection_open (Connect->OpenSent) Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 open active, local address 169.254.40.170 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 sending OPEN, version 4, my as 65000, holdtime 32, id 192.168.0.190 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 send message type 1, length (incl. header) 53 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 went from Connect to OpenSent Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 1, length (excl. header) 34 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcv OPEN, version 4, remote-as (in open) 7224, holdtime 30, id 169.254.40.169 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcv OPEN w/ OPTION parameter len: 24 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcvd OPEN w/ optional parameter type 2 (Capability) len 6 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 OPEN has MultiProtocol Extensions capability (1), length 4 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 OPEN has MP_EXT CAP for afi/safi: 1/1 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcvd OPEN w/ optional parameter type 2 (Capability) len 2 Feb 1 16:48:12 vyos01 bgpd[2459]: message index 128 [Route Refresh (Old)] found in capcode_str at position 6 (max is 8) Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 OPEN has Route Refresh (Old) capability (128), length 0 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcvd OPEN w/ optional parameter type 2 (Capability) len 2 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 OPEN has Route Refresh capability (2), length 0 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcvd OPEN w/ optional parameter type 2 (Capability) len 6 Feb 1 16:48:12 vyos01 bgpd[2459]: message index 65 [4-octet AS number] found in capcode_str at position 4 (max is 8) Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 OPEN has 4-octet AS number capability (65), length 4 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm) Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 went from OpenSent to OpenConfirm Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established) Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 went from OpenConfirm to Established Feb 1 16:48:12 vyos01 bgpd[2459]: %ADJCHANGE: neighbor 169.254.40.169 Up Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:48:12 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:48:13 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (routeadv timer expire) Feb 1 16:48:13 vyos01 bgpd[2459]: 169.254.40.169 send UPDATE 10.0.0.0/26 Feb 1 16:48:13 vyos01 bgpd[2459]: 169.254.40.169 rcvd UPDATE w/ attr: nexthop 169.254.40.169, origin i, metric 100, path 7224 Feb 1 16:48:14 vyos01 kernel: [1316316.879526] e1000 0000:02:01.0 eth2: Reset adapter Feb 1 16:48:14 vyos01 netplugd[2371]: eth2: state ACTIVE flags 0x00011043 UP,BROADCAST,RUNNING,MULTICAST,10000 -> 0x00001003 UP,BROADCAST,MULTICAST Feb 1 16:48:14 vyos01 zebra[2449]: interface eth2 index 3 changed <UP,BROADCAST,MULTICAST>. Feb 1 16:48:14 vyos01 bgpd[2459]: Zebra rcvd: interface eth2 down Feb 1 16:48:14 vyos01 netplugd[17642]: /etc/netplug/netplug eth2 out -> pid 17642 Feb 1 16:48:14 vyos01 conntrack-tools[7886]: no dedicated links available! Feb 1 16:48:14 vyos01 netplugd[2371]: eth2: state OUTING pid 17642 exited status 0 Feb 1 16:48:14 vyos01 zebra[2449]: interface eth2 index 3 changed <UP,BROADCAST,RUNNING,MULTICAST>. Feb 1 16:48:14 vyos01 bgpd[2459]: Zebra rcvd: interface eth2 up Feb 1 16:48:14 vyos01 netplugd[2371]: eth2: state INACTIVE flags 0x00001003 UP,BROADCAST,MULTICAST -> 0x00011043 UP,BROADCAST,RUNNING,MULTICAST,10000 Feb 1 16:48:14 vyos01 netplugd[17645]: /etc/netplug/netplug eth2 in -> pid 17645 Feb 1 16:48:14 vyos01 netplugd[2371]: eth2: state INNING pid 17645 exited status 0 Feb 1 16:48:15 vyos01 heartbeat: [8225]: WARN: Late heartbeat: Node 192.168.10.198: interval 12000 ms Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 [FSM] TCP_connection_open (Connect->OpenSent) Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 open active, local address 169.254.40.62 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 sending OPEN, version 4, my as 65000, holdtime 32, id 192.168.0.190 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 send message type 1, length (incl. header) 53 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 went from Connect to OpenSent Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 1, length (excl. header) 34 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcv OPEN, version 4, remote-as (in open) 7224, holdtime 30, id 169.254.40.61 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcv OPEN w/ OPTION parameter len: 24 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcvd OPEN w/ optional parameter type 2 (Capability) len 6 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 OPEN has MultiProtocol Extensions capability (1), length 4 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 OPEN has MP_EXT CAP for afi/safi: 1/1 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcvd OPEN w/ optional parameter type 2 (Capability) len 2 Feb 1 16:48:16 vyos01 bgpd[2459]: message index 128 [Route Refresh (Old)] found in capcode_str at position 6 (max is 8) Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 OPEN has Route Refresh (Old) capability (128), length 0 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcvd OPEN w/ optional parameter type 2 (Capability) len 2 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 OPEN has Route Refresh capability (2), length 0 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcvd OPEN w/ optional parameter type 2 (Capability) len 6 Feb 1 16:48:16 vyos01 bgpd[2459]: message index 65 [4-octet AS number] found in capcode_str at position 4 (max is 8) Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 OPEN has 4-octet AS number capability (65), length 4 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm) Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 went from OpenSent to OpenConfirm Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established) Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 went from OpenConfirm to Established Feb 1 16:48:16 vyos01 bgpd[2459]: %ADJCHANGE: neighbor 169.254.40.61 Up Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:48:16 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:48:16 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:48:17 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (routeadv timer expire) Feb 1 16:48:17 vyos01 bgpd[2459]: 169.254.40.61 send UPDATE 10.0.0.0/26 Feb 1 16:48:17 vyos01 bgpd[2459]: 169.254.40.61 rcvd UPDATE w/ attr: nexthop 169.254.40.61, origin i, metric 200, path 7224 Feb 1 16:48:22 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:48:22 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:48:22 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:48:22 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:48:22 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:48:22 vyos01 bgpd[2459]: Performing BGP general scanning Feb 1 16:48:22 vyos01 bgpd[2459]: scanning IPv4 Unicast routing tables Feb 1 16:48:22 vyos01 bgpd[2459]: scanning IPv6 Unicast routing tables Feb 1 16:48:26 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:48:26 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:48:26 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:48:26 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:48:26 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:48:31 vyos01 bgpd[2459]: Import timer expired. Feb 1 16:48:32 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:48:32 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:48:32 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:48:32 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:48:32 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:48:36 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:48:36 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:48:36 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:48:36 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:48:36 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:48:42 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (keepalive timer expire) Feb 1 16:48:42 vyos01 bgpd[2459]: 169.254.40.169 sending KEEPALIVE Feb 1 16:48:42 vyos01 bgpd[2459]: 169.254.40.169 send message type 4, length (incl. header) 19 Feb 1 16:48:42 vyos01 bgpd[2459]: 169.254.40.169 rcv message type 4, length (excl. header) 0 Feb 1 16:48:42 vyos01 bgpd[2459]: 169.254.40.169 KEEPALIVE rcvd Feb 1 16:48:43 vyos01 bgpd[2459]: 169.254.40.169 [FSM] Timer (routeadv timer expire) Feb 1 16:48:46 vyos01 bgpd[2459]: 169.254.40.61 [FSM] Timer (keepalive timer expire) Feb 1 16:48:46 vyos01 bgpd[2459]: 169.254.40.61 sending KEEPALIVE Feb 1 16:48:46 vyos01 bgpd[2459]: 169.254.40.61 send message type 4, length (incl. header) 19 Feb 1 16:48:46 vyos01 bgpd[2459]: 169.254.40.61 rcv message type 4, length (excl. header) 0 Feb 1 16:48:46 vyos01 bgpd[2459]: 169.254.40.61 KEEPALIVE rcvd Feb 1 16:48:46 vyos01 bgpd[2459]: Import timer expired.
Our configuration:
cluster { dead-interval 10000 group ClusterGroup1 { auto-failback false primary vyos01 secondary vyos02 } interface eth0 interface eth1 keepalive-interval 2000 monitor-dead-interval 15000 } protocols { bgp LOCAL_AS { maximum-paths { ebgp 2 } neighbor 169.254.40.5 { ebgp-multihop 2 nexthop-self remote-as REMOTE_AS route-map { } soft-reconfiguration { inbound } timers { holdtime 32 keepalive 10 } } neighbor 169.254.41.21 { ebgp-multihop 2 nexthop-self remote-as REMOTE_AS route-map { } soft-reconfiguration { inbound } timers { holdtime 32 keepalive 10 } } parameters { log-neighbor-changes } } vpn { ipsec { auto-update 45 esp-group AWS { compression disable lifetime 900 mode tunnel pfs enable proposal 1 { encryption aes128 hash sha1 } } ike-group AWS { dead-peer-detection { action restart interval 15 timeout 32 } ikev2-reauth no key-exchange ikev1 lifetime 1800 proposal 1 { dh-group 2 encryption aes128 hash sha1 } } ipsec-interfaces { interface eth0 } logging { log-modes all } nat-traversal disable site-to-site { peer [AWS_IP_VPN2] { authentication { mode pre-shared-secret } connection-type initiate ike-group AWS ikev2-reauth inherit local-address 192.168.x.y vti { bind vti0 esp-group AWS } } peer [AWS_IP_VPN1] { authentication { mode pre-shared-secret } connection-type initiate ike-group AWS ikev2-reauth inherit local-address 192.168.x.z vti { bind vti1 esp-group AWS } } } } }