Enabling NetFlow fails, iptables chain VYATTA_CT_PREROUTING_HOOK unknown
In progress, NormalPublic

Description

Hi,

Enabling NetFlow accounting under Vyos Beta fails:

vyos@vyos# commit
[ system flow-accounting interface eth0 ]
iptables: No chain/target/match by that name.
Error: [iptables -t raw -I VYATTA_CT_PREROUTING_HOOK 1 -i eth0 -j ULOG --ulog-nlgroup 2 --ulog-cprange 64 --ulog-qthreshold 10] failed - 256

system flow-accounting failed
Commit failed
[edit]
vyos@vyos#

system {

config-management {
    commit-revisions 20
}
domain-name xxxx.xxx

+ flow-accounting {
+ interface eth0
+ netflow {
+ engine-id 50
+ sampling-rate 1
+ server x.x.x.x {
+ port 2055
+ }
+ server x.x.x.x {
+ port 2055
+ }
+ timeout {
+ expiry-interval 60
+ flow-generic 3600
+ icmp 300
+ max-active-life 300
+ tcp-fin 300
+ tcp-generic 3600
+ tcp-rst 120
+ udp 300
+ }
+ version 9
+ }
+ syslog-facility daemon
+ }

host-name XXXXXXX

Details

Difficulty level
Normal (likely a few hours)
Version
VyOS 999.201704052137
Why the issue appeared?
Other

Related Objects

LordNikon set Version to VyOS 999.201704052137.
syncer claimed this task.Apr 6 2017, 6:19 PM
syncer triaged this task as Normal priority.
syncer removed syncer as the assignee of this task.
syncer added subscribers: syncer, VyOS 1.2.x.
tdale added a subscriber: tdale.EditedSep 18 2017, 6:50 AM

Any updates on this? I can't seem to win. On 1.1.7 snmp is broken due to my intel 10g nics and now on beta snmp works but netflow doesnt work :(

vyos@vyos# commit
[ system flow-accounting interface eth4 ]
iptables: No chain/target/match by that name.
Error: [iptables -t raw -I VYATTA_CT_PREROUTING_HOOK 1 -i eth4 -j ULOG --ulog-nlgroup 2 --ulog-cprange 64 --ulog-qthreshold 10] failed - 256

system flow-accounting failed
Commit failed
[edit]
vyos@vyos#

Seems that 'ULOG' is missing.

No, ULOG is there:

show version
Version:          VyOS 999.201711072137

sudo bash
iptables -j ULOG --help

iptables -n -L VYATTA_CT_PREROUTING_HOOK
iptables: No chain/target/match by that name.

I don't know what is supposed to create the VYATTA_CT_PREROUTING_HOOK chain.

aopdal added a subscriber: aopdal.Nov 28 2017, 8:57 AM

But if you run:

iptables -t raw -nL -v

You find the VYATTA_CT_PREROUTING_HOOK

root@o6-10344:~# iptables -t raw -I VYATTA_CT_PREROUTING_HOOK 1 -i eth0.42 -j ULOG --ulog-nlgroup 2 --ulog-cprange 64 --ulog-qthreshold 10
iptables: No chain/target/match by that name.
root@o6-10344:~# iptables -t raw -I VYATTA_CT_PREROUTING_HOOK 1 -i eth0.42 -j ACCEPT
root@o6-10344:~# 
root@o6-10344:~# iptables -t raw -n -L VYATTA_CT_PREROUTING_HOOK -v
Chain VYATTA_CT_PREROUTING_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination
 1206 48240 ACCEPT     all  --  eth0.42 *       0.0.0.0/0            0.0.0.0/0
9981K  684M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Ah, I missed the -t raw. I thought maybe the ULOG target needs uacctd running first on netlink group 2, so I started it:

uacctd  -P "memory,nfprobe" -d -g 2 -S daemon

then tried to add the iptables -j ULOG entry, but it still fails with the same error message.

syncer changed the task status from Open to In progress.Fri, Dec 22, 12:26 AM
syncer moved this task from Need Triage to In Progress on the VyOS 1.2.x board.

Please test latest nightly builds and report back

squeeby added a subscriber: squeeby.Fri, Jan 5, 1:38 AM
# show system flow-accounting
 interface pppoe0
 interface eth2.2
 interface eth2.3
 sflow {
     agent-address 192.168.64.1
     sampling-rate 10
     server 192.168.64.10 {
         port 2055
     }
 }

# commit
[ system flow-accounting interface eth2.2 ]
Adding flow-accounting for [eth2.2]

[ system flow-accounting interface eth2.3 ]
Adding flow-accounting for [eth2.3]

[ system flow-accounting ]
Stopping flow-accounting
Starting flow-accounting
WARN: [/etc/pmacct/uacctd.conf:19] Unknown symbol '192.168.64.10-2055'. Ignored.
WARN: [/etc/pmacct/uacctd.conf:20] Unknown symbol '192.168.64.10-2055'. Ignored.
WARN: [/etc/pmacct/uacctd.conf:21] Unknown symbol '192.168.64.10-2055'. Ignored.


# cat /etc/pmacct/uacctd.conf
!
! autogenerated by /opt/vyatta/sbin/vyatta-netflow.pl
!
daemonize: true
promisc:   false
pidfile:   /var/run/uacctd.pid
imt_path:  /tmp/uacctd.pipe
imt_mem_pools_number: 169
uacctd_group: 2
uacctd_nl_size: 2097152
snaplen: 32768
refresh_maps: true
pre_tag_map: /etc/pmacct/int_map
aggregate: tag,src_mac,dst_mac,vlan,src_host,dst_host,src_port,dst_port,proto,tos,flows
plugin_pipe_size: 10485760
plugin_buffer_size: 10240
syslog: daemon
plugins: memory,sfprobe[192.168.64.10-2055]
sfprobe_receiver[192.168.64.10-2055]: 192.168.64.10:2055
sfprobe_agentip[192.168.64.10-2055]: 192.168.64.1
sampling_rate[192.168.64.10-2055]: 10

With this config, nothing is received by the sflow collector.
Removing the "[192.168.64.10-2055]", killing uacctd and starting it again using:

sudo uacctd -f /etc/pmacct/uacctd.conf

yields data on the collector.

Changing the following lines to the excerpt below in /opt/vyatta/sbin/vyatta-netflow.pl seems to work:

265     foreach my $name (@names) {
266         my $server_port = $name;
267         $server_port    =~ s/-/:/;
268         $output .= "sfprobe_receiver: $server_port\n";
269         $output .= "sfprobe_agentip: $agent_ip\n" if $agent_ip;
270         $output .= "sfprobe_agentsubid: $agent\n" if $agent;
271         $output .= "sampling_rate: $sampling\n" if defined $sampling;
272     }
c-po added a subscriber: c-po.Fri, Jan 5, 6:50 AM

@squeeby do you mind verifying the following package containing your fix:

On your VyOS instance do via SSH:

$ cd /tmp
$ wget https://www.mybll.net/vyatta-netflow_0.42+vyos2+current1_all.deb
$ dpkg -i vyatta-netflow_0.42+vyos2+current1_all.deb

Thanks!

squeeby added a comment.EditedFri, Jan 5, 7:50 AM

Hi @c-po,

I applied your patch but now iptables has reverted to using the ULOG target instead of NFLOG:

squeeb@gw1# commit
[ system flow-accounting interface eth2 ]
iptables: No chain/target/match by that name.
Error: [iptables -t raw -I VYATTA_CT_PREROUTING_HOOK 1 -i eth2 -j ULOG --ulog-nlgroup 2 --ulog-cprange 64 --ulog-qthreshold 10] failed - 256

Apologies, I should have mentioned the nightly build I was using in the above ticket:

# run show version
No hypervisor detected
Version:          VyOS 999.201801040845
Built by:         autobuild@vyos.net
Built on:         Thu 04 Jan 2018 08:45 UTC
Build ID:         59850d33-3180-40b6-851b-a914a19ff8b1

Architecture:     x86_64
Boot via:         installed image
System type:      physical

Also, uacctd.conf is still appending [$name] in the config, I forgot to mention it also puts this on the plugin line:

# cat /etc/pmacct/uacctd.conf
!
! autogenerated by /opt/vyatta/sbin/vyatta-netflow.pl
!
daemonize: true
promisc:   false
pidfile:   /var/run/uacctd.pid
imt_path:  /tmp/uacctd.pipe
imt_mem_pools_number: 169
uacctd_group: 2
uacctd_nl_size: 2097152
snaplen: 32768
refresh_maps: true
pre_tag_map: /etc/pmacct/int_map
aggregate: tag,src_mac,dst_mac,vlan,src_host,dst_host,src_port,dst_port,proto,tos,flows
plugin_pipe_size: 10485760
plugin_buffer_size: 10240
syslog: daemon
plugins: memory,sfprobe[192.168.64.10-2055]
sfprobe_receiver: 192.168.64.10:2055
sfprobe_agentip: 192.168.64.1
sampling_rate: 10
c-po added a comment.Fri, Jan 5, 9:32 AM

Strange. I only changed /opt/vyatta/sbin/vyatta-netflow.pl to your recommendation.

Do you know how I can restore the previous version so I can see if it was this package that changed it?

c-po added a comment.EditedFri, Jan 5, 2:13 PM

You can revert by switching back to the official VyOS package.

$ cd /tmp
$ wget http://dev.packages.vyos.net/vyos/pool/main/v/vyatta-netflow/vyatta-netflow_0.42+vyos2+current1_all.deb
$ dpkg --install vyatta-netflow_0.42+vyos2+current1_all.deb

Hmmmm..

So by reverting, the file /opt/vyatta/sbin/vyatta-netflow.pl contains:

328 sub acct_add_nflog_target {
329     my ($intf) = @_;
330
331     my ($table_chain) = acct_get_table_chain();
332     while (my ($chain, $table) = each(%$table_chain)) {
333         my $cmd = "iptables -t $table -I $chain 1 -i $intf -j NFLOG" ." --nflog-group 2";
334         if (defined $nflog_range) {
335             $cmd .= " --nflog-range $nflog_range";
336         }
337         if (defined $nflog_threshold) {
338             $cmd .= " --nflog-threshold $nflog_threshold";
339         }
340         my $ret = system($cmd);
341         if ($ret >> 8) {
342             die "Error: [$cmd] failed - $?\n";
343         }
344     }
345 }

The patch you sent me contains the following:

328 sub acct_add_ulog_target {
329     my ($intf) = @_;
330
331     my ($table_chain) = acct_get_table_chain();
332     while (my ($chain, $table) = each(%$table_chain)) {
333         my $cmd = "iptables -t $table -I $chain 1 -i $intf -j ULOG" ." --ulog-nlgroup 2";
334         if (defined $ulog_cprange) {
335             $cmd .= " --ulog-cprange $ulog_cprange";
336         }
337         if (defined $ulog_qthreshold) {
338             $cmd .= " --ulog-qthreshold $ulog_qthreshold";
339         }
340         my $ret = system($cmd);
341         if ($ret >> 8) {
342             die "Error: [$cmd] failed - $?\n";
343         }
344     }
345 }

The patch is using the ULOG target instead of NFLOG.

There's probably other places in that script that also use NFLOG instead of ULOG but I didn't look very hard. ;)

c-po added a comment.Fri, Jan 5, 2:42 PM

Could you alter the file manually to get a working state and pass it to me by e.g. pasting it here or a https://pastebin.com/ link? Then I could regenerate a package for testing. This would help me a lot as I do not have any flow collector.

https://hastebin.com/bufirameyo.pl

This appears to operate as expected.

Looks good!

I ran the following:

delete system flow-accounting
commit

Installed the ver02 package,

edit system flow-accounting
set interface 'eth2.2'
set interface 'eth2.3'
set sflow agent-address '192.168.64.1'
set sflow sampling-rate '10'
set sflow server 192.168.64.10 port '2055'
top
commit

Yields the expected output and I can see flows appearing on my collector:

squeeb@gw1# sudo iptables -L VYATTA_CT_PREROUTING_HOOK -t raw -v
Chain VYATTA_CT_PREROUTING_HOOK (1 references)
 pkts bytes target     prot opt in     out     source               destination
  616 53226 NFLOG      all  --  eth2.3 any     anywhere             anywhere             nflog-group 2 nflog-range 64 nflog-threshold 10
  166 51202 NFLOG      all  --  eth2.2 any     anywhere             anywhere             nflog-group 2 nflog-range 64 nflog-threshold 10
8561K 2054M RETURN     all  --  any    any     anywhere             anywhere
[edit]
squeeb@gw1# cat /etc/pmacct/uacctd.conf
!
! autogenerated by /opt/vyatta/sbin/vyatta-netflow.pl
!
daemonize: true
promisc:   false
pidfile:   /var/run/uacctd.pid
imt_path:  /tmp/uacctd.pipe
imt_mem_pools_number: 169
uacctd_group: 2
uacctd_nl_size: 2097152
snaplen: 32768
refresh_maps: true
pre_tag_map: /etc/pmacct/int_map
aggregate: tag,src_mac,dst_mac,vlan,src_host,dst_host,src_port,dst_port,proto,tos,flows
plugin_pipe_size: 10485760
plugin_buffer_size: 10240
syslog: daemon
plugins: memory,sfprobe
sfprobe_receiver: 192.168.64.10:2055
sfprobe_agentip: 192.168.64.1
sampling_rate: 10
[edit]
c-po added a comment.EditedFri, Jan 5, 8:07 PM

Merged into vyatta-netflow package and will be included in tonights build.

@squeeby thanks for your support!

c-po moved this task from In Progress to Finished on the VyOS 1.2.x board.Fri, Jan 5, 8:12 PM
c-po added a comment.Sat, Jan 6, 11:02 PM

@squeeby which sflow collector do you use? Is there one you can recommend?

I'm using ntop-ng + nprobe.

It's a paid for product however there is a community edition available.

In production we're using SolarWinds NTA, expensive product but gives us a good insight as to what's going on.