Page MenuHomeVyOS Platform
Create Task
Maniphest T6033

hsflowd fails to start when using a tunnel interface
Closed, ResolvedPublicBUG
Actions

Description

When adding sflow to a tunnel interface hsflowd fails to restart.

VRF interfaces

Name     State    MAC address        Flags                     Interfaces
-------  -------  -----------------  ------------------------  --------------------
AS23428  up       06:58:65:b8:30:97  noarp,master,up,lower_up  eth0,eth2,tun20,tun0

example config

set system sflow agent-interface 'tun0'
set system sflow interface 'tun0'
set system sflow sampling-rate '1000'
set system sflow server 2602:fb68:140::2d
set system sflow vrf 'AS23428'
Feb 11 16:12:44 ix-1 systemd[1]: Started hsflowd.service - Host sFlow.
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(log_backtrace+0x2e)[0x55ab2eaf1cce]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x11ed9)[0x55ab2eaf1ed9]
Feb 11 16:12:44 ix-1 ip[162730]: /lib/x86_64-linux-gnu/libc.so.6(+0x3bfd0)[0x7fbaa54fdfd0]
Feb 11 16:12:44 ix-1 ip[162730]: /etc/hsflowd/modules/mod_pcap.so(+0x1954)[0x7fbaa56a6954]
Feb 11 16:12:44 ix-1 ip[162730]: /etc/hsflowd/modules/mod_pcap.so(+0x1d3f)[0x7fbaa56a6d3f]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(EVEventTx+0xf7)[0x55ab2eaf2bb7]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x12d59)[0x55ab2eaf2d59]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x1316a)[0x55ab2eaf316a]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x13340)[0x55ab2eaf3340]
Feb 11 16:12:44 ix-1 ip[162730]: /lib/x86_64-linux-gnu/libc.so.6(+0x89044)[0x7fbaa554b044]
Feb 11 16:12:44 ix-1 ip[162730]: /lib/x86_64-linux-gnu/libc.so.6(+0x10961c)[0x7fbaa55cb61c]
Feb 11 16:12:44 ix-1 ip[162730]: SIGSEGV, faulty address is (nil)
Feb 11 16:12:44 ix-1 ip[162730]: current bus: packet
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(log_backtrace+0x2e)[0x55ab2eaf1cce]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x11eeb)[0x55ab2eaf1eeb]
Feb 11 16:12:44 ix-1 ip[162730]: /lib/x86_64-linux-gnu/libc.so.6(+0x3bfd0)[0x7fbaa54fdfd0]
Feb 11 16:12:44 ix-1 ip[162730]: /etc/hsflowd/modules/mod_pcap.so(+0x1954)[0x7fbaa56a6954]
Feb 11 16:12:44 ix-1 ip[162730]: /etc/hsflowd/modules/mod_pcap.so(+0x1d3f)[0x7fbaa56a6d3f]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(EVEventTx+0xf7)[0x55ab2eaf2bb7]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x12d59)[0x55ab2eaf2d59]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x1316a)[0x55ab2eaf316a]
Feb 11 16:12:44 ix-1 ip[162730]: /usr/sbin/hsflowd(+0x13340)[0x55ab2eaf3340]
Feb 11 16:12:44 ix-1 ip[162730]: /lib/x86_64-linux-gnu/libc.so.6(+0x89044)[0x7fbaa554b044]
Feb 11 16:12:44 ix-1 ip[162730]: /lib/x86_64-linux-gnu/libc.so.6(+0x10961c)[0x7fbaa55cb61c]
Feb 11 16:12:44 ix-1 ip[162730]: PCAP: tun0 has no supported datalink encapsulaton
Feb 11 16:12:44 ix-1 ip[162730]: Received signal 11
Feb 11 16:12:44 ix-1 ip[162730]: SIGSEGV, faulty address is (nil)
Feb 11 16:12:44 ix-1 ip[162730]: current bus: packet
Feb 11 16:12:44 ix-1 systemd[1]: hsflowd.service: Main process exited, code=exited, status=11/n/a
Feb 11 16:12:44 ix-1 systemd[1]: hsflowd.service: Failed with result 'exit-code'.

Details

Difficulty level
Unknown (require assessment)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Bug (incorrect behavior)

Event Timeline

ServerForge created this task.Feb 11 2024, 4:20 PM
ServerForge created this object in space S1 VyOS Public.
ServerForge added a comment.Feb 11 2024, 4:30 PM

Seems its actually the tunnel interface it doesnt like, not the address.

ServerForge renamed this task from hsflowd fails to start when using a VRF to hsflowd fails to start when using a tunnel interface.Feb 11 2024, 4:31 PM
ServerForge updated the task description. (Show Details)
pasik added a subscriber: pasik.Feb 11 2024, 5:59 PM
Viacheslav added a subscriber: Viacheslav.EditedMar 29 2024, 10:24 AM

Only ethernet interfaces or bridges are allowed; from their docs https://sflow.net/host-sflow-linux-config.php

PCAP Packet Sampling

pcap { dev=docker0 }

Apply packet sampling to the "docker0" device, which can be a hardware NIC or a linux bridge.

We probably needs to exclude non ethernet interfaces from the flow.

set interfaces ethernet eth1 address '192.0.2.1/30'
set interfaces tunnel tun0 address '203.0.113.1/27'
set interfaces tunnel tun0 encapsulation 'gre'
set interfaces tunnel tun0 remote '192.0.2.2'
set interfaces tunnel tun0 source-address '192.0.2.1'
set system sflow agent-address '203.0.113.1'
set system sflow interface 'tun0'
set system sflow sampling-rate '1000'
set system sflow server 192.0.2.254

Config:

vyos@r4# cat /run/sflow/hsflowd.conf 
# Genereated by /usr/libexec/vyos/conf_mode/system_sflow.py
# Parameters http://sflow.net/host-sflow-linux-config.php

sflow {
  polling=30
  sampling=1000
  sampling.bps_ratio=0
  agentIP=203.0.113.1
  collector { ip = 192.0.2.254 udpport = 6343 }
  pcap { dev=tun0 }
  dbus { }
}

Logs PCAP: tun0 has no supported datalink encapsulaton

Mar 29 12:23:21 r4 systemd[1]: hsflowd.service: Scheduled restart job, restart counter is at 168.
Mar 29 12:23:21 r4 systemd[1]: Stopped hsflowd.service - Host sFlow.
Mar 29 12:23:21 r4 systemd[1]: Started hsflowd.service - Host sFlow.
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(log_backtrace+0x2e)[0x56027cafecce]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x11ed9)[0x56027cafeed9]
Mar 29 12:23:21 r4 hsflowd[9160]: /lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7f1381330050]
Mar 29 12:23:21 r4 hsflowd[9160]: /etc/hsflowd/modules/mod_pcap.so(+0x1954)[0x7f13814d8954]
Mar 29 12:23:21 r4 hsflowd[9160]: /etc/hsflowd/modules/mod_pcap.so(+0x1d3f)[0x7f13814d8d3f]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(EVEventTx+0xf7)[0x56027caffbb7]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x12d59)[0x56027caffd59]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x1316a)[0x56027cb0016a]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x13340)[0x56027cb00340]
Mar 29 12:23:21 r4 hsflowd[9160]: /lib/x86_64-linux-gnu/libc.so.6(+0x89134)[0x7f138137d134]
Mar 29 12:23:21 r4 hsflowd[9160]: /lib/x86_64-linux-gnu/libc.so.6(+0x1097dc)[0x7f13813fd7dc]
Mar 29 12:23:21 r4 hsflowd[9160]: SIGSEGV, faulty address is (nil)
Mar 29 12:23:21 r4 hsflowd[9160]: current bus: packet
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(log_backtrace+0x2e)[0x56027cafecce]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x11eeb)[0x56027cafeeeb]
Mar 29 12:23:21 r4 hsflowd[9160]: /lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7f1381330050]
Mar 29 12:23:21 r4 hsflowd[9160]: /etc/hsflowd/modules/mod_pcap.so(+0x1954)[0x7f13814d8954]
Mar 29 12:23:21 r4 hsflowd[9160]: /etc/hsflowd/modules/mod_pcap.so(+0x1d3f)[0x7f13814d8d3f]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(EVEventTx+0xf7)[0x56027caffbb7]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x12d59)[0x56027caffd59]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x1316a)[0x56027cb0016a]
Mar 29 12:23:21 r4 hsflowd[9160]: /usr/sbin/hsflowd(+0x13340)[0x56027cb00340]
Mar 29 12:23:21 r4 hsflowd[9160]: /lib/x86_64-linux-gnu/libc.so.6(+0x89134)[0x7f138137d134]
Mar 29 12:23:21 r4 hsflowd[9160]: /lib/x86_64-linux-gnu/libc.so.6(+0x1097dc)[0x7f13813fd7dc]
Mar 29 12:23:21 r4 hsflowd[9160]: PCAP: tun0 has no supported datalink encapsulaton
Mar 29 12:23:21 r4 hsflowd[9160]: Received signal 11
Mar 29 12:23:21 r4 hsflowd[9160]: SIGSEGV, faulty address is (nil)
Mar 29 12:23:21 r4 hsflowd[9160]: current bus: packet
Mar 29 12:23:21 r4 systemd[1]: hsflowd.service: Main process exited, code=exited, status=11/n/a
Mar 29 12:23:21 r4 systemd[1]: hsflowd.service: Failed with result 'exit-code'.

I opened an issue https://github.com/sflow/host-sflow/issues/65

Viacheslav added a project: VyOS 1.5 Circinus.Mar 29 2024, 10:37 AM
Viacheslav changed the subtype of this task from "Task" to "Bug".Mar 29 2024, 11:08 AM
Viacheslav added a comment.Mar 29 2024, 11:22 AM

@ServerForge, it seems to be working with the v2.0.55-1 version. Could you build a package and re-check?

git clone -b v2.0.55-1 --single-branch  https://github.com/sflow/host-sflow.git
sudo apt-get install -y libpcap0.8-dev
make deb FEATURES="PCAP DROPMON DBUS"

At least daemon starts correctly

vyos@r4# sudo systemctl status hsflowd
● hsflowd.service - Host sFlow
     Loaded: loaded (/lib/systemd/system/hsflowd.service; disabled; preset: enabled)
    Drop-In: /run/systemd/system/hsflowd.service.d
             └─override.conf
     Active: active (running) since Fri 2024-03-29 13:16:17 EET; 6min ago
   Main PID: 12970 (hsflowd)
      Tasks: 2 (limit: 18713)
     Memory: 660.0K
        CPU: 344ms
     CGroup: /system.slice/hsflowd.service
             └─12970 /usr/sbin/hsflowd -m 166cfd257d3a4eca9ef60b655c9acf0f -d -f /run/sflow/hsflowd.conf

Mar 29 13:16:17 r4 systemd[1]: Started hsflowd.service - Host sFlow.
Viacheslav changed the task status from Open to In progress.Mar 29 2024, 11:53 AM
Viacheslav claimed this task.

PR https://github.com/vyos/vyos-build/pull/544

Viacheslav changed the task status from In progress to Needs testing.Mar 29 2024, 3:07 PM
ServerForge added a comment.Mon, Apr 1, 12:25 AM

Its no longer failing to start, but it seems to be only capturing inbound traffic on the tunnel, no outbound. I'm also observing this behavior on vlan interfaces, IE bond0.10.

Viacheslav added a comment.Mon, Apr 1, 7:09 AM

@ServerForge It is question for hsflowd
You can open the issue on their git repo

Viacheslav closed this task as Resolved.Tue, Apr 2, 9:45 AM
Viacheslav moved this task from Need Triage to Finished on the VyOS 1.5 Circinus board.
Viacheslav edited projects, added VyOS 1.4 Sagitta (1.4.0-epa3); removed VyOS 1.4 Sagitta.

The original issue was resolved.

Viacheslav moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta (1.4.0-epa3) board.Tue, Apr 2, 9:46 AM