Page MenuHomeVyOS Platform

v.huti (Volodymyr Huti)
User

Projects

User Details

User Since
Mar 13 2022, 5:49 PM (106 w, 3 d)

Recent Activity

Thu, Mar 21

v.huti claimed T3040: NHRP IPv6 Support.
Thu, Mar 21, 2:51 PM · VyOS 1.5 Circinus
v.huti added a comment to T3040: NHRP IPv6 Support.

I have checked in with @sharpd, he mentioned that it might not be supported to use v4 nexthop for v6 route.
From local testing:

vova | ip r
default via 192.168.0.1 dev wlp4s0 proto dhcp metric 600 
192.168.0.0/24 dev wlp4s0 proto kernel scope link src 192.168.0.102 metric 600
Thu, Mar 21, 2:51 PM · VyOS 1.5 Circinus

Nov 13 2023

v.huti added a comment to T5168: zebra memory leak.

Hi @aserkin ! I should have some time to help you. For this issue, I would need

  • artifacts collected from running zebra under Valgrind
  • a reference configuration that I can replicate in a lab

I will ping you over Slack for additional details and assistance.

Nov 13 2023, 10:37 PM
v.huti triaged T5737: Eigrp #11301 - Configuration failed error type: validation as Normal priority.
Nov 13 2023, 10:31 PM · VyOS 1.4 Sagitta
v.huti added a comment to T2326: Migrate NHRP(DMVPN) to FRR.

I have created a draft pull request for FRR, but I can still see a bunch of odd bugs.
I'm going to activate it after additional testing by the team.
Most issues involve Wireshark's inability to parse packets correctly and display an exception, although the demons seem to run fine.
https://github.com/FRRouting/frr/pull/14788

Nov 13 2023, 10:27 PM · VyOS 1.5 Circinus
v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

I`m back after a long break and will follow up on this feature.
Here is a summary of things that have happened since the last update:

1. In my absence, the feature testing got broken as a result of migrating from the `mininet` to the `munet` framework
From debugging, I have identified a root cause to be - bpf fs was not inherited by the `munet` router.
The solution is to hop into the router mount namespace for the test run.
Nov 13 2023, 10:22 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4462: FRR operational-data pagination.

Because there is a long-running development for operation data retrieval, we can postpone this ticket until an effort is finished.
Then, I can open a feature request or visit the yang meeting and start a discussion about the data pagination functionality.
Currently, my idea is to simulate pagination at the fs level by having a split of requested JSON.
This solution involves:

  • Fetching an operation data from the demon
vtysh -c "show yang operational-data /frr-vrf:lib/vrf[name='default']/frr-zebra:zebra/ribs zebra" > big.json
  • Fetching flat data stream and formating it 1 item (prefix) per line with jq -c option
jq -c '."frr-vrf:lib" .vrf[0] ."frr-zebra:zebra" .ribs .rib[0] .route[]' big.json | split -l 100 -d
  • Splitting result by the number of lines (objects), saving into files on the filesystem

Now, UI can display 1 of the resulting files at a time. These files may be regenerated on page refresh.
Such a solution will utilize extra disk space, although it can be avoided by using pipes and other
streaming utilities like awk/sed

Nov 13 2023, 10:17 PM · VyOS 1.4 Sagitta

Aug 2 2023

v.huti claimed T5424: Routes vanishes when using FRR with ECMP and one of the ECMP paths is no longer available.
Aug 2 2023, 1:48 PM · VyOS 1.4 Sagitta (1.4.0-epa1), Restricted Project

Apr 19 2023

v.huti claimed T5168: zebra memory leak.
Apr 19 2023, 10:18 AM

Apr 9 2023

v.huti claimed T2326: Migrate NHRP(DMVPN) to FRR.
Apr 9 2023, 11:57 AM · VyOS 1.5 Circinus

Feb 17 2023

v.huti changed the status of T4180: Support for QoS Policy Propagation via BGP (QPPB) from Open to Needs testing.
Feb 17 2023, 4:56 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

I cannot attach the rest of the artifacts due to size limits.
Publishing them on git instead: https://github.com/volodymyrhuti/QPPB_docs

Feb 17 2023, 4:55 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).
Intro
=========================================================================================================
The QoS Policy Propagation via BGP feature allows you to classify packets by IP precedence based on the
Border Gateway Protocol (BGP) community lists, BGP autonomous system paths, access lists, thus helping to
classify based on the destination instead of source address.
Feb 17 2023, 4:52 PM · VyOS 1.4 Sagitta

Nov 7 2022

v.huti added a comment to T4737: FRRouting/zebra 7.5.1 does not redistribute routes to other protocols.

Hi @zsdc! This seems to be related to T4028. The relevant commits are:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
commit 92980561382fc04380414a6e2f6ca6746c2fe5e9 ┃
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━━━━
Author: Donald Sharp <[email protected]>
Date:   Mon Apr 19 19:23:45 2021 -0400
Nov 7 2022, 1:17 PM · VyOS 1.3 Equuleus (1.3.3)

Oct 6 2022

v.huti claimed T4731: excessive FRR logs about non-existent VRFs.
Oct 6 2022, 2:29 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4731: excessive FRR logs about non-existent VRFs.

Hi @aserkin! It looks like you have some frr server misbehavior. It sends up/down events with an unexisting vrf id.
Could you make/describe the setup that causes the issue to appear? Thanks

Oct 6 2022, 12:48 PM · VyOS 1.4 Sagitta

Sep 27 2022

v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).


DEMO Notes:
=====================

1) You need to load the XDP program before starting frr so that
   it can find the LPM map on plugin initialization.
   To keep it simple, the VTY interface was not implemented for now.
   XDP side is accessible via `bpftool`
3) I`m monitoring packets for TOS/DSCP changes to see if marking happens
   But in another approach tag is associated with the packet and then
   read by the TC classifier
4) These are two traffic shaping examples.
   The point is that you have two options for marking:
4.1) Modifying the TOS byte and installing the u32 tc filter to match the value.
   This has a limited range of possible values (8 bits) + needs to modify the packet.
4.2) Using a custom BPF classifier.
    The XDP side extends the packet context and saves the value.
    Afterward, the classifier may read the context and control the shaping behavior
    by setting the `skb->tc_classid` or one of the fields mentioned below.
Therefore, BPF programs attached to the tc BPF hook can, for instance,
read or write the skb’s mark, pkt_type, protocol, priority, queue_mapping,
napi_id, cb[] array, hash, tc_classid or tc_index, vlan metadata, the XDP
transferred custom metadata and various other information. All members of
the struct __sk_buff BPF context used in tc BPF are defined in the
linux/bpf.h system header.

https://docs.cilium.io/en/stable/bpf/#tc-traffic-control
Sep 27 2022, 8:05 PM · VyOS 1.4 Sagitta

Sep 22 2022

v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

DEMO
===============================================

To demonstrate the feature let's look at the following topology

topo.png (447×900 px, 87 KB)



Sep 22 2022, 10:54 AM · VyOS 1.4 Sagitta

Sep 15 2022

v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

Changes on the FRR side:

  • Convert xdp helper library to an optional plugin + bgp hook
  • Minor fixes + cleanups
  • Figured out most of the permission problems

Changes on the XDP side:

  • Convert mappings from legacy iproute format to the latest libbpf one
  • New mappings improve debugging experience by implementing pretty-printing for XDP map dumping
  • Added an xdp-loader for xdp-tools repo
Sep 15 2022, 1:57 PM · VyOS 1.4 Sagitta

Aug 25 2022

v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

The latest version of the demo can be found here:

  1. volodymyrhuti/frr/tree/QPPB_DEMO_V1.3
  2. volodymyrhuti/xdp_qppb
Aug 25 2022, 1:47 PM · VyOS 1.4 Sagitta

Jul 28 2022

v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

The latest version of the demo can be found here:

Jul 28 2022, 10:44 AM · VyOS 1.4 Sagitta

Jul 21 2022

v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

You can find the latest version of the demo implementation here:

  1. volodymyrhuti/linux/tree/QPPB_DEMO_V1.1
  2. volodymyrhuti/frr/tree/QPPB_DEMO_V1.1
Jul 21 2022, 1:56 PM · VyOS 1.4 Sagitta

Jul 14 2022

v.huti claimed T4180: Support for QoS Policy Propagation via BGP (QPPB).
Jul 14 2022, 1:59 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4180: Support for QoS Policy Propagation via BGP (QPPB).

Demo QPPB implementation supporting bgp-policy destination mode:

Jul 14 2022, 1:59 PM · VyOS 1.4 Sagitta

Jun 21 2022

v.huti updated subscribers of T4394: Improve VYOS_DEBUG profiling support.

Memray:

In order to keep useful tracing/debugging tooling in a single place, the @jestabro has created the repo:

https://github.com/jestabro/profiling-tools

Since there is no vyatta package yet, you need to either compile it by hand or install it from the apt
as explained before. Some examples to play around with:

# NOTE: I had to downgrade this package to resolve the installation conflict
# sudo apt-get install python3-pkg-resources=45.2.0-1
  sudo apt-get install python3-pip
  sudo python3 -m pip install memray
  PATH+=":/home/vyos/.local/bin"
Jun 21 2022, 10:49 PM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti added a comment to T4394: Improve VYOS_DEBUG profiling support.

Analysis:

I have collected the profiling data for the following configurations:

Jun 21 2022, 10:48 PM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti added a comment to T4394: Improve VYOS_DEBUG profiling support.

Gotchas:

If you are running a small QEMU device and it has run out of the memory, the scenario is following:

- The boot process has failed, the prompt is stuck, bash is not initialized
- You reboot the device, it tries to read the config, fails once again as there is no free memory
- Config was not loaded, you cannot log in; it is a loop
Jun 21 2022, 10:47 PM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti added a comment to T4394: Improve VYOS_DEBUG profiling support.
NOTE: by default, the perf binary is not installed on the ISO image.
Jun 21 2022, 10:45 PM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti added a comment to T4462: FRR operational-data pagination.

TBD: GUI

VyOS users can configure the front-end interface, called vycontroll, to examine the configuration state.
A detailed description can be found at:
https://vycontrol.com/
https://github.com/vycontrol/vycontrol
https://docs.vyos.io/en/equuleus/configuration/service/https.html
https://brezular.com/2021/05/01/vycontrol-web-ui-for-vyos-firewall/

Jun 21 2022, 2:40 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4462: FRR operational-data pagination.

FRR Debugging


Recently, I had to triage/debug a bunch of issues that involved running a legacy build of frr.
This involved:

  • Triaging issue down to the place when it was introduced. Otherwise, verifying that feature was never working at all.
  • Comparing the execution flow between legacy/master versions to identify the divergence
  • Building & running multiple (legacy/master) frr versions in parallel
  • Doing deep analysis within gdb
Jun 21 2022, 2:38 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4462: FRR operational-data pagination.

Since the last update, I have simplified the CLI interface:

1. I have removed the global iterator and incapsulated the iteration state into the vty structure.
   This way, each vtysh client has its private iteration state for the following requests.
   It should be possible to query multiple data nodes simultaneously and asynchronously.
Jun 21 2022, 2:27 PM · VyOS 1.4 Sagitta

Jun 16 2022

v.huti added a comment to T4462: FRR operational-data pagination.

Ongoing activity:

1. Stabilization
-  I have seen a corner case that would crash inside the northbound callbacks.
-  I can see some validation failure logs, although the resulting output seems good for me.
-  Daniil was concerned about memory leaks associated with iteration state.
   After additional research - this is not a problem, but I can imagine cases where we would
   fail to handle a malformed XPath and leak resources on the stuck unwinding
   I need to do some testing with Valgrind.
2. Scale testing
3. Async support for multiple vtysh clients. The current demo assumes that there is only one client.
   I want to map the iteration state to the vtysh client/socket so multiple requests may be executed in parallel
4. A debugging instruction
   I have used some complicated debugging flow when merging the feature.
   This should be useful for other (non-C) devs.
5. Finishing the documentation
6. advanced XPath filtering support?
Jun 16 2022, 1:50 PM · VyOS 1.4 Sagitta
v.huti added a comment to T4462: FRR operational-data pagination.

Recently, I had a conversation with the VMware team lead - Pushpasis Sarkar.
He has described the ongoing development and explained the use case they are interested in.
From the conversation:

1. The latest proposal draft: 
   Page 72-73 `Retrieve Operational Data - Retrieving Containers and Leaf members`
   Page 84-85 `Retrieve Operational Data - Retrieving Large List elements` + comments
   Page 86 `Retrieve Operational Data - Retrieving Containers and Leaf members` + comments.
Jun 16 2022, 1:29 PM · VyOS 1.4 Sagitta
v.huti updated the task description for T4462: FRR operational-data pagination.
Jun 16 2022, 12:39 PM · VyOS 1.4 Sagitta

Jun 9 2022

v.huti created T4462: FRR operational-data pagination.
Jun 9 2022, 2:00 PM · VyOS 1.4 Sagitta

May 2 2022

v.huti added a comment to T4394: Improve VYOS_DEBUG profiling support.

There was some effort to introduce profiling into the system before, but nothing was developed.
The ticket was opened to verify that the timing values displayed in /var/log/vyatta are correct.
The vyos-debug flag enables tracing for actions described in the templates.
This will be a step-by-step walkthrough of the system profiling, as I have found this to have a bunch of non-obvious technical nuances that might get you stuck.

May 2 2022, 8:55 AM · VyOS 1.4 Sagitta, vyatta-cfg

Apr 25 2022

v.huti claimed T4394: Improve VYOS_DEBUG profiling support.
Apr 25 2022, 2:45 PM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti updated the task description for T4394: Improve VYOS_DEBUG profiling support.
Apr 25 2022, 6:50 AM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti updated the task description for T4394: Improve VYOS_DEBUG profiling support.
Apr 25 2022, 6:40 AM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti updated the task description for T4394: Improve VYOS_DEBUG profiling support.
Apr 25 2022, 6:39 AM · VyOS 1.4 Sagitta, vyatta-cfg
v.huti created T4394: Improve VYOS_DEBUG profiling support.
Apr 25 2022, 6:35 AM · VyOS 1.4 Sagitta, vyatta-cfg

Apr 20 2022

v.huti added a comment to T4028: FRR 8.1 routes not being applied to routing table after reboot if an interface has 2 ip addresses.

Pull https://github.com/FRRouting/frr/pull/11004 was merged. This needs retesting on the latest FRR

Apr 20 2022, 10:48 AM · VyOS 1.4 Sagitta