NetFlow have impact on performance
Open, HighPublic

Description

Hello,
we have several reports now that NetFlow collection had a huge impact on performance.
Maybe we should replace current pmacct with another solution,
there is some alternative natively supported in iptables
https://github.com/aabc/ipt-netflow
please consider replacement, it will be great to have it in 1.2 if possible

Details

Difficulty level
Hard (possibly days)
Version
1.1.7
syncer created this task.May 30 2016, 2:16 PM
syncer created this object with edit policy "Administrators".
syncer updated the task description. (Show Details)

Well, I have ipt-netflow on self-rebuilt vyos kernel, no problems with performance. But I have no vyos-related scripts for interaction with this module.

I had to disable dkms there
https://github.com/mickvav/ipt-netflow-code
And if anyone is interested - I also have xtables-addons compilable against vyos kernel (it has several interesting firewall features - such as geoip and ipmark) - https://github.com/mickvav/xtables-addons

afics added a subscriber: afics.Jun 1 2016, 1:55 PM

Related/duplicate: T33.

Hm, as ipt-netflow is actually a firewall target, it looks like it's configuration logic should be slightly different from pmacct's one.
Looks like there should be some service level config tree, specifying module load parameters, like

set service ipt-netflow collector 10.2.3.4
 ...

and some firewall-level additional target, e.g.

set firewall name blabla rule 123 action 'NETFLOW'

I think we can choose how to implement it. We can apply it as a default entry in one of the vyos chains or let the user-decide. The advantage with the latter is that both implementations can co-exist for a while. With the former solution I would remove the old implementation to not confuse the user.

syncer edited subscribers, added: VyOS 1.2.x; removed: VyOS 1.1.x (1.1.8).
syncer added a comment.Mar 2 2017, 1:09 PM

@mickvav I recall that you told in some task about IPT usage
can you share how you currently integrate IPT?
@jclendenan and me(and not only) will be interested to see this in 1.2

Well, I take vyos-kernel, iptables, build them in packages directory, and put ipt-netflow from here: https://github.com/mickvav/ipt-netflow-code as git submodule in the same packages directory, build it there and get working .deb package containing module, crafted for current vyos kernel. I have no CLI integration for it though I use my own firewall-messing scripts. But in general, you jest have to do modprobe the module with right parameters (where to send collected data) and add somewhere in firewall the rule with "-j NETFLOW" to trigger, which packets to take into account.

Hello,
Is there a chance that ipt_NETFLOW will be included in next release (and if yes, where it is planned to release this version?)
@mickvav Can you share your .deb package please? We need ipt_NETFLOW ASAP. Thanks

Well, I don't have access to development vm, where I did this stuff today (remind me on monday, please), but I do have kernel module (the only file in .deb, actually) compiled against 4.4.15-amd64-vyos kernel.

You have to put it in /lib/modules/4.4.15-amd64-vyos/extra/ directory and run depmod after.
But be ware - use at your own risk!

Ups, seems I was wrong in last comment. I'll collect all the files from .deb and post them here.

Here you are -

- it expects to be extracted in / directory. But no warranties on any binary compatibility with current version of kernel and iptables. AT ALL.

Thank you.
It looks like you have this compiled to much newer kernel, 4.4.15 while current kernel in VyOS 1.1.7 is 3.13.11-1-amd64-vyos.
So it looks like i need to compile it by my own, but thanks anyway for sharing this ;)

chilek added a subscriber: chilek.Sep 27 2017, 8:54 AM

Hello, is there a way to easy install kernel source/headers for default kernel used by vyos 1.1.x (3.13.11-1-amd64-vyos)?