Page MenuHomeVyOS Platform

enable VRF for conntrack-sync
Open, NormalPublicFEATURE REQUEST

Description

I have two VyOS routers using a dedicated VLAN interface for conntrack-sync. It would be perfect if conntrack-sync would support VRF to seperate HA communication from normal routing. I am thinking of a VRF instance to handle all router-to-router management/HA traffic that could be used for other services as well (e.g. config-sync, etc.).

Details

Difficulty level
Unknown (require assessment)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Improvement (missing useful functionality)

Related Objects

StatusSubtypeAssignedTask
OpenFEATURE REQUESTNone
OpenFEATURE REQUESTNone

Event Timeline

Make sure conntrack-sync works with active-active HA configuration with BGP environment & IPv6

This comment was removed by qdrddr.

Did you try to start this service in VRF?

It has been a while since I had setup the HA VRF. I attached the interfaces on both routers to use this VRF but then conntrack-sync wasn't woking anymore. Do you mean if I had also tried to manually start the service and configure it to use this VRF?

Yes I mean sudo ip vrf exec FOO /usr/sbin/conntrackd -C /run/conntrackd/conntrackd.conf

I had entered the command as you have suggested and I think it's working somehow.

My VRF connfiguration is as follows:

show configuration commands | match vrf

set interfaces ethernet eth2 vif 1590 vrf 'HA'
set vrf bind-to-all
set vrf name HA table '100'
sudo ip vrf exec HA /usr/sbin/conntrackd -C /run/conntrackd/conntrackd.conf

[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] disabling external cache
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] reliable ctnetlink event delivery is ENABLED.
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] using user-space event filtering
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] netlink event socket buffer size has been set to 2097152 bytes
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] configuring helper `tns' with queuenum=5 and queuelen=0
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] policy name=tns expect_timeout=300 expect_max=1
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] helper `tns' configured successfully
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] configuring helper `rpc' with queuenum=4 and queuelen=0
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] policy name=rpc expect_timeout=300 expect_max=1
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] helper `rpc' configured successfully
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] configuring helper `rpc' with queuenum=3 and queuelen=0
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] policy name=rpc expect_timeout=300 expect_max=1
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] helper `rpc' configured successfully
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] initialization completed
[Wed Nov 15 17:05:07 2023] (pid=251686) [notice] -- starting in console mode --
[Wed Nov 15 17:05:07 2023] (pid=251686) [ERROR] inject-add2: File exists

There seems to happen an inital synchronization when the daemon starts as I can see new sessions on the destination router (conntrack-sync partner) but after the start no new session will be synchronized anymore. Maybe this behavior has something to do with the error message "inject-add2: File exists" shown in the output when entering the command.

Session entry on the source router

show conntrack-sync cache internal | grep "10\.12\.1\.15"
10.12.1.15:49078                                 10.255.15.152:22                                 tcp

Session entry on the destination router (when sudo ip vrf exec ... is started)

show conntrack-sync cache internal | grep "10\.12\.1\.15"
10.12.1.15:49078                                 10.255.15.152:22                                 tcp

Creating a new SSH session to the source router

show conntrack-sync cache internal | grep "10\.12\.1\.15"
10.12.1.15:49078                                 10.255.15.152:22                                 tcp
10.12.1.15:59432                                 10.255.15.152:22                                 tcp

Corresponding session list on the destination router

show conntrack-sync cache internal | grep "10\.12\.1\.15"
10.12.1.15:49078                                 10.255.15.152:22                                 tcp

Session list on the destination router (after stopping and starting "sudo ip vrf ..." again)

show conntrack-sync cache internal | grep "10\.12\.1\.15"
10.12.1.15:49078                                 10.255.15.152:22                                 tcp
10.12.1.15:59432                                 10.255.15.152:22                                 tcp

cat /run/conntrackd/conntrackd.conf

# autogenerated by conntrack_sync.py

# Synchronizer settings
Sync {
    Mode FTFW {
        DisableExternalCache on
    }
    UDP {
        IPv4_address 10.254.15.2
        IPv4_Destination_Address 10.254.15.1
        Port 3780
        Interface eth2.1590
        SndSocketBuffer 1048576
        RcvSocketBuffer 1048576
        Checksum on
    }
}
Helper {
    Type rpc inet tcp {
        QueueNum 3
        Policy rpc {
            ExpectMax 1
            ExpectTimeout 300
        }
    }
    Type rpc inet udp {
        QueueNum 4
        Policy rpc {
            ExpectMax 1
            ExpectTimeout 300
        }
    }
    Type tns inet tcp {
        QueueNum 5
        Policy tns {
            ExpectMax 1
            ExpectTimeout 300
        }
    }
}

# General settings
General {
    HashSize 32768
    HashLimit 524288
    LogFile off
    Syslog on
    LockFile /var/lock/conntrack.lock
    UNIX {
        Path /var/run/conntrackd.ctl
    }
    NetlinkBufferSize 2097152
    NetlinkBufferSizeMaxGrowth 8388608
    NetlinkOverrunResync off
    NetlinkEventsReliable on
    Filter From Userspace {
        Protocol Accept {
            TCP
            UDP
            ICMP
        }
    }
}

Created a related feature request but for VRRP here
https://vyos.dev/T5745

dmbaturin triaged this task as Normal priority.Jan 9 2024, 6:04 PM
dmbaturin added a parent task: T5116: Better VRF support.