Page MenuHomeVyOS Platform
Create Task
Maniphest T1721

Recursive Next Hop not updated for static routes
Closed, ResolvedPublicBUG
Actions

Description

I have configuration with 4 routers.
First two routers are in Office1 (R1, R2) and second two - in Office2 (R3, R4).
R1 have vti-tunnel connection to R3 and R4.
R2 have vti-tunnel connection to R3 and R4.

Routes are distributed with OSPF.

eth1 (LAN) interface for R1 have IP: 192.168.0.2 (for example)
eth1 (LAN) interface for R2 have IP: 192.168.0.3 (for example)

R1 and R2 routers have additional table 500 with default static route.

static {
        route 0.0.0.0/0 {
            next-hop XX.XX.XX.XX {
            }
        }
        table 500 {
            route 0.0.0.0/0 {
                next-hop 192.168.0.10 {
                }
            }
        }
    }

When ethernet interfaces go to down (no link state) frr change this static route to recursive.
When ethernet interfaces go back to up frr does not change static route back.

After reboot (as expected):

S>* 0.0.0.0/0 [1/0] via 192.168.0.10, eth1

After ethernet Interfaces go down for 1 second (Live Migration in Cluster, for example)

S>  0.0.0.0/0 [1/0] via 192.168.0.10 (recursive)
  *                   via XX.XX.0.174, vti3
  *                   via XX.XX.0.170, vti2

I think, that this is bug in frr, and It is resolved: https://github.com/FRRouting/frr/pull/4602

Details

Difficulty level
Easy (less than an hour)
Version
1.2.x
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Behavior change
Issue type
Bug (incorrect behavior)

Related Objects

Event Timeline

lbv2rus created this task.Oct 10 2019, 4:38 PM
pasik added a subscriber: pasik.Oct 14 2019, 5:14 PM
syncer changed the task status from Open to Needs testing.Nov 16 2019, 11:13 PM
syncer assigned this task to Viacheslav.
syncer triaged this task as Low priority.
syncer edited projects, added VyOS 1.3 Equuleus; removed VyOS 1.2 Crux.
Viacheslav mentioned this in T1361: VRRP Starts After FRR, Creating Inconsistent Routes -.Nov 23 2019, 8:46 PM
Viacheslav added a comment.Oct 19 2020, 7:25 PM

@lbv2rus Can you try the latest rolling release?

lbv2rus added a comment.Oct 20 2020, 11:49 AM

I cannot check rolling release, but on latest 1.2.6 problem is solved.

Viacheslav added a comment.Oct 20 2020, 2:34 PM

Can you provide a simple config of R1 R3 and R4?
I want to reproduce it in the test lab.

lbv2rus added a comment.EditedOct 20 2020, 3:03 PM

Simple R1&R2
R2 will have low priority in HA and other ip-adresses in the same subnet

About vti
vti0: R1<->R3
vti1: R1<->R4
vti2: R2<->R3
vti3: R2<->R4

high-availability {
    vrrp {
        group eth0-1 {
            advertise-interval 1
            interface eth0
            priority 150
            virtual-address IA.IA.IA.IA/I
            vrid 1
        }
        group eth1-101 {
            advertise-interval 1
            interface eth1
            priority 150
            virtual-address LA.LA.LA.LA/L
            vrid 101
        }
        sync-group G1 {
            member eth0-1
            member eth1-101
        }
    }
}
interfaces {
    ethernet eth0 {
        address I.I.I.I/I
        description "Internet Interface"
        duplex auto
        smp-affinity auto
        speed auto
    }
    ethernet eth1 {
        address L.L.L.L/L
        description "LAN Interface"
        duplex auto
        ip {
            ospf {
                cost 10
                dead-interval 40
                hello-interval 10
                priority 1
                retransmit-interval 5
                transmit-delay 1
            }
        }
        policy {
            route PROXY-ROUTING
        }
        smp-affinity auto
        speed auto        
    }
    loopback lo {
    }
    vti vti0 {
        address YA.YA.YA.YA/30
        description "Tunnel to R3"
    }
    vti vti1 {
        address YB.YB.YB.YB/30
        description "Tunnel to R4"
    }
}
nat {
    source {
        rule 1 {
            description "Default"
            outbound-interface eth0
            translation {
                address masquerade
            }
        }
    }
}
policy {
    access-list 10 {
        rule 1 {
            action permit
            source {
                inverse-mask 0.255.255.255
                network 10.0.0.0
            }
        }
        rule 2 {
            action deny
            source {
                any
            }
        }
    }
    route PROXY-ROUTING {
        rule 10 {
            description "Local Network to Main Table"
            destination {
                address 10.0.0.0/8
            }
            protocol all
            set {
                table main
            }
        }
        rule 99 {
            description "Users to Proxy"
            protocol all
            set {
                table 100
            }
        }
    }
}
protocols {
    ospf {
        access-list 10 {
            export connected
        }
        area 0 {
            authentication md5
            network L.L.L.L/X
            network YA.YA.YA.YA/30
	    network YB.YB.YB.YB/30
        }
        parameters {
            abr-type cisco
            router-id L.L.L.L
        }
        passive-interface default
        passive-interface-exclude vti0
        passive-interface-exclude vti1
        passive-interface-exclude eth1
        redistribute {
            connected {
                metric-type 2
            }
            static {
                metric-type 2
            }
        }
    }
    static {
        route 0.0.0.0/0 {
            next-hop I.I.I.I {
            }
        }
        table 100 {
            route 0.0.0.0/0 {
                next-hop LZ.LZ.LZ.lZ {
                }
            }
        }
    }
}
vpn {
    ipsec {
        esp-group ESP1-NR {
            compression disable
            lifetime 1800
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes256
                hash sha1
            }
        }
        ike-group IKE1-NR {
            close-action none
            dead-peer-detection {
                action restart
                interval 30
                timeout 120
            }
            ikev2-reauth no
            key-exchange ikev1
            lifetime 3600
            proposal 1 {
                dh-group 2
                encryption aes256
                hash sha1
            }
        }
        ipsec-interfaces {
            interface eth0
        }
        nat-networks {
            allowed-network 10.0.0.0/8 {
            }
            allowed-network 172.16.0.0/12 {
            }
            allowed-network 192.168.0.0/16 {
            }
        }
        options {
            disable-route-autoinstall
        }
        site-to-site {
            peer A.A.A.A {
                authentication {
                    mode rsa
                    rsa-key-name R3-RSA-KEY
                }
                connection-type initiate
                default-esp-group ESP1-NR
                description "Tunnel to R3"
                ike-group IKE1-NR
                ikev2-reauth inherit
                local-address L.L.L.L
                vti {
                    bind vti0
                    esp-group ESP1-NR
                }
            }
            peer B.B.B.B {
                authentication {
                    mode rsa
                    rsa-key-name R4-RSA-KEY
                }
                connection-type initiate
                default-esp-group ESP1-NR
                description "Tunnel to R4"
                ike-group IKE1-NR
                ikev2-reauth inherit
                local-address L.L.L.L
                vti {
                    bind vti1
                    esp-group ESP1-NR
                }
            }
        }
    }
}

Simple R3&R4
The same as R1&R2 but without some routing policy

high-availability {
    vrrp {
        group eth0-1 {
            advertise-interval 1
            interface eth0
            priority 150
            virtual-address IA.IA.IA.IA/I
            vrid 1
        }
        group eth1-101 {
            advertise-interval 1
            interface eth1
            priority 150
            virtual-address LA.LA.LA.LA/L
            vrid 101
        }
        sync-group G1 {
            member eth0-1
            member eth1-101
        }
    }
}
interfaces {
    ethernet eth0 {
        address I.I.I.I/I
        description "Internet Interface"
        duplex auto
        smp-affinity auto
        speed auto
    }
    ethernet eth1 {
        address L.L.L.L/L
        description "LAN Interface"
        duplex auto
        ip {
            ospf {
                cost 10
                dead-interval 40
                hello-interval 10
                priority 1
                retransmit-interval 5
                transmit-delay 1
            }
        }
        smp-affinity auto
        speed auto        
    }
    loopback lo {
    }
    vti vti0 {
        address YC.YC.YC.YC/30
        description "Tunnel to R1"
    }
    vti vti2 {
        address YD.YD.YD.YD/30
        description "Tunnel to R2"
    }
}
nat {
    source {
        rule 1 {
            description "Default"
            outbound-interface eth0
            translation {
                address masquerade
            }
        }
    }
}
policy {
    access-list 10 {
        rule 1 {
            action permit
            source {
                inverse-mask 0.255.255.255
                network 10.0.0.0
            }
        }
        rule 2 {
            action deny
            source {
                any
            }
        }
    }
}
protocols {
    ospf {
        access-list 10 {
            export connected
        }
        area 0 {
            authentication md5
            network L.L.L.L/X
            network YC.YC.YC.YC/30
	    network YD.YD.YD.YD/30
        }
        parameters {
            abr-type cisco
            router-id L.L.L.L
        }
        passive-interface default
        passive-interface-exclude vti0
        passive-interface-exclude vti2
        passive-interface-exclude eth1
        redistribute {
            connected {
                metric-type 2
            }
            static {
                metric-type 2
            }
        }
    }
    static {
        route 0.0.0.0/0 {
            next-hop I.I.I.I {
            }
        }
    }
}
vpn {
    ipsec {
        esp-group ESP1-NR {
            compression disable
            lifetime 1800
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes256
                hash sha1
            }
        }
        ike-group IKE1-NR {
            close-action none
            dead-peer-detection {
                action restart
                interval 30
                timeout 120
            }
            ikev2-reauth no
            key-exchange ikev1
            lifetime 3600
            proposal 1 {
                dh-group 2
                encryption aes256
                hash sha1
            }
        }
        ipsec-interfaces {
            interface eth0
        }
        nat-networks {
            allowed-network 10.0.0.0/8 {
            }
            allowed-network 172.16.0.0/12 {
            }
            allowed-network 192.168.0.0/16 {
            }
        }
        options {
            disable-route-autoinstall
        }
        site-to-site {
            peer C.C.C.C {
                authentication {
                    mode rsa
                    rsa-key-name R1-RSA-KEY
                }
                connection-type initiate
                default-esp-group ESP1-NR
                description "Tunnel to R1"
                ike-group IKE1-NR
                ikev2-reauth inherit
                local-address L.L.L.L
                vti {
                    bind vti0
                    esp-group ESP1-NR
                }
            }
            peer D.D.D.D {
                authentication {
                    mode rsa
                    rsa-key-name R2-RSA-KEY
                }
                connection-type initiate
                default-esp-group ESP1-NR
                description "Tunnel to R2"
                ike-group IKE1-NR
                ikev2-reauth inherit
                local-address L.L.L.L
                vti {
                    bind vti2
                    esp-group ESP1-NR
                }
            }
        }
    }
}
SrividyaA added a subscriber: SrividyaA.EditedOct 27 2020, 12:08 PM

I have tested on this rolling release VyOS 1.3-rolling-202010231135 and created a lab setup similar to the reporter's setup.

image.png (561×425 px, 46 KB)

route 0.0.0.0/0 {
    next-hop 100.100.100.4 {
    }
}
table 100 {
    route 0.0.0.0/0 {
        next-hop 192.168.0.10 {
        }
    }
}

Restarted the vrrp to reproduce the issue but did not receive any recursive routes in the output:

VRF default table 100:
S>* 0.0.0.0/0 [1/0] via 192.168.0.10, eth1, 00:11:43

As the issue is not reproduced, will close the ticket.

Viacheslav closed this task as Resolved.Oct 27 2020, 12:42 PM

@SrividyaA Thanks.

c-po moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus board.Nov 23 2020, 5:20 PM
erkin set Issue type to Bug (incorrect behavior).Aug 31 2021, 6:38 PM
syncer edited projects, added VyOS 1.3 Equuleus (1.3.0); removed VyOS 1.3 Equuleus.Aug 29 2022, 12:26 PM
syncer moved this task from Need Triage to Finished on the VyOS 1.3 Equuleus (1.3.0) board.