Page MenuHomeVyOS Platform

nhrp - add or delete nhrp tunnel restart opennhrp process
Needs testing, Requires assessmentPublicBUG

Description

Hi team

We've found when you add or delete a nhrp tunnel , it restart the opennhrp process of all tunnels which trigger any connection to be lost. example below :

[email protected]# compare
[edit protocols nhrp]
-tunnel tun104 {
-    cisco-authentication pass123
-    holding-time 400
-}
[edit]
[email protected]# commit

[email protected]:~$ sudo systemctl status  opennhrp
● opennhrp.service - OpenNHRP
     Loaded: loaded (/lib/systemd/system/opennhrp.service; static)
     Active: active (running) since Tue 2022-04-26 21:13:55 UTC; 11min ago
    Process: 8084 ExecStart=/usr/sbin/opennhrp -d -v -a /run/opennhrp.socket -c>
   Main PID: 8096 (opennhrp)
      Tasks: 1 (limit: 545)
     Memory: 932.0K
        CPU: 590ms
     CGroup: /system.slice/opennhrp.service
             └─8096 /usr/sbin/opennhrp -d -v -a /run/opennhrp.socket -c /run/op>

Apr 26 21:24:14 hub-dmvpn opennhrp[8096]: NL-ARP(tun100) who-has 100.64.10.2
Apr 26 21:24:15 hub-dmvpn opennhrp[8096]: NL-ARP(tun100) who-has 100.64.10.2
Apr 26 21:24:30 hub-dmvpn opennhrp[8096]: NL-ARP(tun100) who-has 100.64.10.2
Apr 26 21:24:32 hub-dmvpn opennhrp[8096]: NL-ARP(tun100) who-has 100.64.10.2
Apr 26 21:24:44 hub-dmvpn opennhrp[8096]: Received Registration Request from pr>
Apr 26 21:24:44 hub-dmvpn opennhrp[8096]: [100.64.10.2] Peer registration faile>
Apr 26 21:24:44 hub-dmvpn opennhrp[8096]: Sending Registration Reply from proto>
Apr 26 21:24:44 hub-dmvpn opennhrp[8096]: Sending packet 4, from: 100.64.10.2 (

Apr 26 21:13:54 hub-dmvpn opennhrp[2788]: Removing local 10.64.20.1/32 dev tun101 up
Apr 26 21:13:54 hub-dmvpn opennhrp[2788]: Removing local 100.64.10.255/32 alias 100.64.10.1 dev tun100 up
Apr 26 21:13:54 hub-dmvpn opennhrp[2788]: Removing local 100.64.10.1/32 dev tun100 up
Apr 26 21:13:54 hub-dmvpn opennhrp[2788]: Removing dynamic 100.64.10.2/32 nbma 100.0.0.3 dev tun100 up expires_in 0:04
Apr 26 21:13:54 hub-dmvpn opennhrp[2788]: Removing local-route 172.16.80.0/24 nexthop 100.64.10.2 dev tun100 up
Apr 26 21:13:54 hub-dmvpn opennhrp[2788]: NL-ARP(tun100) 100.64.10.2 not-reachable
Apr 26 21:13:55 hub-dmvpn systemd[1]: opennhrp.service: Succeeded.
Apr 26 21:13:55 hub-dmvpn systemd[1]: opennhrp.service: Consumed 49.880s CPU time.
Apr 26 21:13:55 hub-dmvpn opennhrp[8084]: opennhrp[8084]: OpenNHRP 0.14-20-g613277f starting
Apr 26 21:13:55 hub-dmvpn opennhrp[8084]: OpenNHRP 0.14-20-g613277f starting
Apr 26 21:13:55 hub-dmvpn opennhrp[8084]: opennhrp[8084]: Interface lo: configured UP, mtu=0
Apr 26 21:13:55 hub-dmvpn opennhrp[8084]: Interface lo: configured UP, mtu=0
Apr 26 21:13:55 hub-dmvpn opennhrp[8084]: opennhrp[8084]: Interface eth0: configured UP, mtu=1500

it seems to be affected for this action (python code)

action = 'restart' if nhrp and 'tunnel' in nhrp else 'stop'
run(f'systemctl {action} opennhrp')
return None

we need to improve this behavior .

Details

Difficulty level
Normal (likely a few hours)
Version
VyOS 1.4-rolling-202204201428
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Event Timeline

@fernando Does it work if you "reload" configuration?

Yes ,you need to reload the services and it works , here's an basic example with the current configuration :

#add new tunnel

[email protected]# set protocols nhrp tunnel tun104 cisco-authentication 'pass123'
[email protected]# set protocols nhrp tunnel tun104 holding-time '400'
[email protected]# commit

# start restart process 

sudo journalctl -b | tail -n 200
Apr 27 12:19:25 hub-dmvpn systemd[1]: Stopping OpenNHRP...
Apr 27 12:19:25 hub-dmvpn sudo[9725]:     root : PWD=/ ; USER=root ; COMMAND=/usr/sbin/ip route del 100.0.0.3 src 100.0.0.1 proto 42
Apr 27 12:19:25 hub-dmvpn sudo[9725]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Apr 27 12:19:25 hub-dmvpn sudo[9725]: pam_unix(sudo:session): session closed for user root
Apr 27 12:19:25 hub-dmvpn systemd[1]: opennhrp.service: Succeeded.
Apr 27 12:19:25 hub-dmvpn systemd[1]: Stopped OpenNHRP.
Apr 27 12:19:25 hub-dmvpn systemd[1]: opennhrp.service: Consumed 7.185s CPU time.
Apr 27 12:19:25 hub-dmvpn systemd[1]: Starting OpenNHRP...
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: OpenNHRP 0.14-20-g613277f starting
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: OpenNHRP 0.14-20-g613277f starting
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface lo: configured UP, mtu=0
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface lo: configured UP, mtu=0
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface eth0: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface eth0: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface eth1: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface eth1: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface eth2: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface eth2: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface eth3: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface eth3: configured UP, mtu=1500
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface gre0: config change, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface gre0: config change, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface gretap0: config change, mtu=1462
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface gretap0: config change, mtu=1462
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface erspan0: config change, mtu=1450
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface erspan0: config change, mtu=1450
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface tun100: configured UP, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface tun100: configured UP, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.10.1/32 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.10.1/32 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.10.255/32 alias 100.64.10.1 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.10.255/32 alias 100.64.10.1 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface tun100: GRE configuration changed. Purged 0 peers.
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface tun100: GRE configuration changed. Purged 0 peers.
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface tun104: configured UP, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface tun104: configured UP, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.10.1/32 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.10.1/32 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.10.255/32 alias 100.64.10.1 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.10.255/32 alias 100.64.10.1 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.40.1/32 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.40.1/32 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.40.255/32 alias 100.64.40.1 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.40.255/32 alias 100.64.40.1 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface tun104: GRE configuration changed. Purged 0 peers.
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface tun104: GRE configuration changed. Purged 0 peers.
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Interface tun102: configured UP, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Interface tun102: configured UP, mtu=1476
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.10.1/32 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.10.1/32 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.10.255/32 alias 100.64.10.1 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.10.255/32 alias 100.64.10.1 dev tun100
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.40.1/32 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.40.1/32 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.40.255/32 alias 100.64.40.1 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.40.255/32 alias 100.64.40.1 dev tun104
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.30.1/32 dev tun102
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.30.1/32 dev tun102
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: opennhrp[9727]: Adding local 100.64.30.255/32 alias 100.64.30.1 dev tun102
Apr 27 12:19:25 hub-dmvpn opennhrp[9727]: Adding local 100.64.30.255/32 alias 100.64.30.1 dev tun102

the spoke lost reachability to the hub :

[email protected]:~$ ping  100.64.10.1
PING 100.64.10.1 (100.64.10.1) 56(84) bytes of data.
64 bytes from 100.64.10.1: icmp_seq=1 ttl=64 time=0.344 ms
64 bytes from 100.64.10.1: icmp_seq=2 ttl=64 time=0.583 ms
64 bytes from 100.64.10.1: icmp_seq=3 ttl=64 time=0.530 ms
64 bytes from 100.64.10.1: icmp_seq=4 ttl=64 time=0.673 ms
64 bytes from 100.64.10.1: icmp_seq=5 ttl=64 time=0.576 ms
64 bytes from 100.64.10.1: icmp_seq=6 ttl=64 time=0.679 ms
64 bytes from 100.64.10.1: icmp_seq=7 ttl=64 time=0.538 ms
64 bytes from 100.64.10.1: icmp_seq=8 ttl=64 time=1.00 ms
^C
--- 100.64.10.1 ping statistics ---
45 packets transmitted, 8 received, 82.2222% packet loss, time 44880ms
rtt min/avg/max/mdev = 0.344/0.615/1.001/0.175 ms

#log messages : 
pr 27 12:22:54 dmvpn-spoke opennhrp[2141]: NL-ARP(tun100) who-has 100.64.10.1
Apr 27 12:22:54 dmvpn-spoke opennhrp[2141]: NL-ARP(tun100) 100.64.10.1 is-at 100.0.0.1
Apr 27 12:23:09 dmvpn-spoke opennhrp[2141]: Sending Registration Request to 100.64.10.1 (my mtu=0)
Apr 27 12:23:09 dmvpn-spoke opennhrp[2141]: Sending packet 3, from: 100.64.10.2 (nbma 100.0.0.3), to: 100.64.10.1 (nbma 100.0.0.1)
Apr 27 12:23:09 dmvpn-spoke opennhrp[2141]: Received Registration Reply from 100.64.10.1: administratively prohibited
Apr 27 12:24:00 dmvpn-spoke opennhrp[2141]: Sending Registration Request to 100.64.10.1 (my mtu=0)
Apr 27 12:24:00 dmvpn-spoke opennhrp[2141]: Sending packet 3, from: 100.64.10.2 (nbma 100.0.0.3), to: 100.64.10.1 (nbma 100.0.0.1)
Apr 27 12:24:00 dmvpn-spoke opennhrp[2141]: Received Registration Reply from 100.64.10.1: administratively prohibited
Apr 27 12:24:22 dmvpn-spoke opennhrp[2141]: NL-ARP(tun100) who-has 100.64.10.1
Apr 27 12:24:22 dmvpn-spoke opennhrp[2141]: NL-ARP(tun100) 100.64.10.1 is-at 100.0.0.1
Apr 27 12:24:38 dmvpn-spoke opennhrp[2141]: Sending Registration Request to 100.64.10.1 (my mtu=0)
Apr 27 12:24:38 dmvpn-spoke opennhrp[2141]: Sending packet 3, from: 100.64.10.2 (nbma 100.0.0.3), to: 100.64.10.1 (nbma 100.0.0.1)
Apr 27 12:24:38 dmvpn-spoke opennhrp[2141]: Received Registration Reply from 100.64.10.1: administratively prohibited

Hub basic setting :

[email protected]:~$ show configuration commands
set interfaces ethernet eth0 address '100.0.0.1/24'

set interfaces tunnel tun100 address '100.64.10.1/24'
set interfaces tunnel tun100 encapsulation 'gre'
set interfaces tunnel tun100 parameters ip key '10100'
set interfaces tunnel tun100 source-interface 'eth0'
set interfaces tunnel tun101 address '10.64.20.1/24'
set interfaces tunnel tun101 encapsulation 'gre'
set interfaces tunnel tun101 parameters ip key '10101'
set interfaces tunnel tun101 source-interface 'eth0'
set interfaces tunnel tun102 address '100.64.30.1/24'
set interfaces tunnel tun102 encapsulation 'gre'
set interfaces tunnel tun102 parameters ip key '10102'
set interfaces tunnel tun102 source-interface 'eth0'
set interfaces tunnel tun104 address '100.64.40.1/24'
set interfaces tunnel tun104 encapsulation 'gre'
set interfaces tunnel tun104 parameters ip key '10104'
set interfaces tunnel tun104 source-interface 'eth0'
set interfaces tunnel tun107 address '100.64.6.1/24'
set interfaces tunnel tun107 encapsulation 'gre'
set interfaces tunnel tun107 parameters ip key '10106'
set interfaces tunnel tun107 source-interface 'eth0'
set protocols bgp local-as '65000'
set protocols bgp neighbor 100.64.10.2 address-family ipv4-unicast
set protocols bgp neighbor 100.64.10.2 remote-as '65001'
set protocols nhrp tunnel tun100 cisco-authentication 'password123'
set protocols nhrp tunnel tun100 holding-time '5'
set protocols nhrp tunnel tun101 cisco-authentication 'pass123'
set protocols nhrp tunnel tun101 holding-time '300'
set protocols nhrp tunnel tun102 cisco-authentication 'pass123'
set protocols nhrp tunnel tun102 holding-time '300'
set protocols nhrp tunnel tun104 cisco-authentication 'pass123'
set protocols nhrp tunnel tun104 holding-time '400'

spoke basic configuration :

[email protected]# run show configuration commands
set interfaces dummy dum10 address '172.16.80.50/24'
set interfaces ethernet eth0 address '100.0.0.3/24'

set interfaces tunnel tun100 address '100.64.10.2/24'
set interfaces tunnel tun100 encapsulation 'gre'
set interfaces tunnel tun100 parameters ip key '10100'
set interfaces tunnel tun100 source-interface 'eth0'
set protocols bgp address-family ipv4-unicast network 172.16.80.0/24
set protocols bgp local-as '65001'
set protocols bgp neighbor 100.64.10.1 address-family ipv4-unicast
set protocols bgp neighbor 100.64.10.1 remote-as '65000'
set protocols bgp neighbor 100.64.10.1 update-source 'tun100'
set protocols nhrp tunnel tun100 cisco-authentication 'password123'
set protocols nhrp tunnel tun100 holding-time '5'
set protocols nhrp tunnel tun100 map 100.64.10.1/24 nbma-address '100.0.0.1'
set protocols nhrp tunnel tun100 map 100.64.10.1/24 register

I did this change as you mentioned and it worked, , example:

[email protected]:/vyos/work/TEST2/vyos-1x$ git diff
diff --git a/src/conf_mode/protocols_nhrp.py b/src/conf_mode/protocols_nhrp.py
index 7eeb5cd3..ff8ae8ee 100755
--- a/src/conf_mode/protocols_nhrp.py
+++ b/src/conf_mode/protocols_nhrp.py
@@ -104,7 +104,7 @@ def apply(nhrp):
         if rule_handle:
             remove_nftables_rule('ip filter', 'VYOS_FW_OUTPUT', rule_handle)

-    action = 'restart' if nhrp and 'tunnel' in nhrp else 'stop'
+    action = 'reload-or-restart' if nhrp and 'tunnel' in nhrp else 'stop'
     run(f'systemctl {action} opennhrp')
     return None

I created a custom pkg and reconfigure the nhrp services on vyos-cli ,after I can add or del any nrhp peer without lost the spoke tunnel :

****
**Apr 27 20:01:34 hub-dmvpn systemd[1]: Reloading OpenNHRP.
Apr 27 20:01:34 hub-dmvpn opennhrp[25022]: Removing dynamic 100.64.10.2/32 nbma 100.0.0.3 dev tun100 up expires_in 0:04
Apr 27 20:01:34 hub-dmvpn opennhrp[25022]: NL-ARP(tun100) 100.64.10.2 not-reachable
Apr 27 20:01:34 hub-dmvpn systemd[1]: Reloaded OpenNHRP.******
Apr 27 20:01:34 hub-dmvpn sudo[25996]: pam_unix(sudo:session): session closed for user root
Apr 27 20:01:34 hub-dmvpn vyos-configd[24217]: Sending response 1
Apr 27 20:01:34 hub-dmvpn sudo[26016]:     root : PWD=/ ; USER=root ; COMMAND=/usr/sbin/ip route del 100.0.0.3 src 100.0.0.1 proto 42
Apr 27 20:01:34 hub-dmvpn sudo[26016]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Apr 27 20:01:34 hub-dmvpn sudo[26016]: pam_unix(sudo:session): session closed for user root
Apr 27 20:01:34 hub-dmvpn systemd[1]: opt-vyatta-config-tmp-new_config_25157.mount: Succeeded.
Apr 27 20:01:34 hub-dmvpn systemd[1629]: opt-vyatta-config-tmp-new_config_25157.mount: Succeeded.
Apr 27 20:01:34 hub-dmvpn opennhrp[25022]: NL-ARP(tun100) who-has 100.64.10.2
Apr 27 20:01:35 hub-dmvpn sudo[26034]:     vyos : TTY=ttyS0 ; PWD=/home/vyos ; USER=root ; COMMAND=/usr/bin/mv /tmp/config.boot.26028 /opt/vyatta/etc/config/archive/config.boot
Apr 27 20:01:35 hub-dmvpn sudo[26034]: pam_unix(sudo:session): session opened for user root(uid=0) by vyos(uid=1002)
Apr 27 20:01:35 hub-dmvpn sudo[26034]: pam_unix(sudo:session): session closed for user root
Apr 27 20:01:35 hub-dmvpn sudo[26036]:     vyos : TTY=ttyS0 ; PWD=/home/vyos ; USER=root ; COMMAND=/usr/sbin/logrotate -f -s /opt/vyatta/etc/config/archive/lr.state /opt/vyatta/etc/config/archive/lr.cof
Apr 27 20:01:35 hub-dmvpn sudo[26036]: pam_unix(sudo:session): session opened for user root(uid=0) by vyos(uid=1002)
Apr 27 20:01:35 hub-dmvpn sudo[26036]: pam_unix(sudo:session): session closed for user root
Apr 27 20:01:35 hub-dmvpn commit[26039]: Successful change to active configuration by user vyos on /dev/ttyS0
Apr 27 20:01:35 hub-dmvpn opennhrp[25022]: NL-ARP(tun100) who-has 100.64.10.2
Apr 27 20:01:36 hub-dmvpn opennhrp[25022]: Received Registration Request from proto src 100.64.10.2 to 100.64.10.1
Apr 27 20:01:36 hub-dmvpn opennhrp[25022]: [100.64.10.2] Peer registration authorized
Apr 27 20:01:36 hub-dmvpn opennhrp[25022]: Adding dynamic 100.64.10.2/32 nbma 100.0.0.3 dev tun100 expires_in 0:04
Apr 27 20:01:36 hub-dmvpn opennhrp[25022]: Sending Registration Reply from proto src 100.64.10.1 to 100.64.10.2 (1 bindings accepted, 0 rejected)
Apr 27 20:01:36 hub-dmvpn opennhrp[25022]: [100.64.10.2] Peer up script: success
Apr 27 20:01:36 hub-dmvpn opennhrp[25022]: NL-ARP(tun100) 100.64.10.2 is-at 100.0.0.3
Apr 27 20:01:36 hub-dmvpn opennhrp[25022]: Sending packet 4, from: 100.64.10.2 (nbma 100.0.0.3), to: 100.64.10.1 (nbma 100.0.0.3)
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: Received Registration Request from proto src 100.64.10.2 to 100.64.10.1
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: [100.64.10.2] Peer registration authorized
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: Removing dynamic 100.64.10.2/32 nbma 100.0.0.3 dev tun100 used up expires_in 0:02
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: Adding dynamic 100.64.10.2/32 nbma 100.0.0.3 dev tun100 up expires_in 0:04
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: [100.64.10.2] Peer up script: success
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: NL-ARP(tun100) 100.64.10.2 is-at 100.0.0.3
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: Sending Registration Reply from proto src 100.64.10.1 to 100.64.10.2 (1 bindings accepted, 0 rejected)
Apr 27 20:01:38 hub-dmvpn opennhrp[25022]: Sending packet 4, from: 100.64.10.2 (nbma 100.0.0.3), to: 100.64.10.1 (nbma 100.0.0.3)
Apr 27 20:01:38 hub-dmvpn sudo[26044]:     vyos : TTY=ttyS0 ; PWD=/home/vyos ; USER=root ; COMMAND=/usr/bin/journalctl -b
Apr 27 20:01:38 hub-dmvpn sudo[26044]: pam_unix(sudo:session): session opened for user root(uid=0) by vyos(uid=1002)
Apr 27 20:01:40 hub-dmvpn opennhrp[25022]: Received Registration Request from proto src 100.64.10.2 to 100.64.10.1
Apr 27 20:01:40 hub-dmvpn opennhrp[25022]: [100.64.10.2] Peer registration authorized
Apr 27 20:01:40 hub-dmvpn opennhrp[25022]: Removing dynamic 100.64.10.2/32 nbma 100.0.0.3 dev tun100 up expires_in 0:02
Apr 27 20:01:40 hub-dmvpn opennhrp[25022]: Adding dynamic 100.64.10.2/32 nbma 100.0.0.3 dev tun100 up expires_in 0:04
Apr 27 20:01:40 hub-dmvpn opennhrp[25022]: [100.64.10.2] Peer up script: success
fernando changed the task status from Open to Needs testing.Thu, Apr 28, 11:51 AM

I've tried with a new spoke and I can't seem to register using `reload-or-restart', although it resolved the lost connectivity issues the opennhrp process needs a full restart. however, if you restart opennhrp daemon it causes different issues and usually the spoke loses connection.

## hub 

Apr 28 19:52:47 hub-dmvpn opennhrp[5052]: Received Registration Request from proto src 100.64.10.2 to 100.64.10.1
Apr 28 19:52:47 hub-dmvpn opennhrp[5052]: [100.64.10.2] Peer registration failed: static entry exists

##spokes 

Apr 28 19:54:12 dmvpn-spoke opennhrp[1558]: NL-ARP(tun100) who-has 100.64.10.1
Apr 28 19:54:12 dmvpn-spoke opennhrp[1558]: NL-ARP(tun100) 100.64.10.1 is-at 100.0.0.1
Apr 28 19:54:30 dmvpn-spoke opennhrp[1558]: Sending Registration Request to 100.64.10.1 (my mtu=0)
Apr 28 19:54:30 dmvpn-spoke opennhrp[1558]: Sending packet 3, from: 100.64.10.2 (nbma 100.0.0.3), to: 100.64.10.1 (nbma 100.0.0.1)
Apr 28 19:54:30 dmvpn-spoke opennhrp[1558]: Received Registration Reply from 100.64.10.1: administratively prohibited

it seems that hub-dmvpn maintains a static route (it was generated dynamic by protocol) and doesn't allow register any new/old peer.