Okay, I like that video already ...

@thomas-mangin Firstly, I needed to write this version before reasonably debating the pros/cons of various approaches --- the important idea developed is that the use of get_sub_dict, and ability to return the sub_dict under the diff'ed nodes will allow a fluid use in writing conf_mode scripts --- beyond that, I am not particularly attached to any details of implementation, yet.

As I click "sail", I just realise that as there is only one config per router, ConfigDict could be a subclass of Config which could be itself singleton. So even if it is subclassed, all instances could/would share the same underlying data and could be used inter-exchangeably for the content of the config.

@jestabro I would like to hear why you advocate this API and why you believe it is better than the one I have suggested.

Using the mentioned defaults in https://tools.ietf.org/html/rfc3414#appendix-A.3.2

@thomas-mangin and @runar I do like the enum idea, however, this would add boilerplate to the conf_mode scripts, which would quickly become annoying in practice ... rather, common workflow just wants the actionable data (added, deleted) _and_ the ability to then access values under the node in question. Consequently, I'm following @runar 's suggestion of returning a tuple, combined with the return_as_dict arg to allow access to sub-data.

This somehow relates to T2651

It does work (eth0 is actually my management vrf interface) if I put the IP for downloads.vyos.io into my /etc/hosts (I guess using a vrf for outgoing DNS requests would be even trickier but this workaround is okay for me, especially since I have a local update mirror reachable via static IP):

@moepman can you check command?

The reason this is failing is VyOS 1.2 lacks proper input validation on the loglevel nodes.

One forgotten point: get_config_diff obtains the config_dicts at root level, so any movement within/between sub-sections are available, with set_level.

It is true that get_config_dict is slow, so it should only be called once (twice for diff) per session. Consequently, the work flow will be:

Adding screenshots showing the error and the fix.

Regarding the API proposed, most of these functions are also syntactic sugar for the same operations. Looking at the use cases, there is two: getting information about a leaf value, or getting information about tagNode changes.

So, as far as useful helper functions, one certainly wants:
get_child_nodes_changed(... path)
get_value_changed(... path)

@runar in fact, that's all one wants in current use case: has the list of elements, directly _under_ the specified node, changed? For example, (1) change of values (2) added or removed tag node entries.

Yes, I'm expanding all paths under the specified path

About is_changed, i see the need to have a function that tells if there are any changes in the path tree under the given path.. specified.

Yes, I definitely prefer a return type of tuple ...

Good point, get_value_changed is a better name for this. As you want to distinguish between a returned value of False and a "Not Changed" using a two tuple (namedTuple?) returned with new and old value makes it easy to "see" the difference

Regarding is_value_changed, I was thinking the other way around: get_value_changed returns None if no change, so is_value_changed would be redundant --- put good point: one may/will want both old and new values

I am entirely open to suggestions here; the underlying functions support any such forms. Note however, that we want to distinguish between new/deleted paths and changed values --- one could treat these all as a difference in path, but it will be more convenient for use if we make the distinction ...

Also, as everything set in python will render True, couldn't is_value_changed return the old and new value instead of just true/false? This will make get_value_changed redundant

What about providing a is_changed, that returns False, added, deleted or changed with the new value provided in the result? Added/deleted/changed can be of a enum type or something like that

With 4.19.123-amd64-vyos I am having the same problems. I would assume, that the patch from 2016 is already in this kernel?

I have tested it in a simple configuration of zone-based firewall with both Crux and Rolling and everything worked ok.

Any chance this will be revived for 1.3 or 2.0 ?
Any amount of firewalling is not gonna stop brute forcing.

I don't see problems with Debian Buster, kernel "4.19.0-9"
Need to check this patch. Ref. https://patchwork.kernel.org/patch/9293785/

Ok, I will look at how we can use the current configuration to insert the tagNode name when we generate the default configuration

@c-po Yes, I checked it on Crux.
Works as expected.

So should be cherry-pick this to 1.2.6?

Community/large/extend community lists syntax was updated by FRR with "detail" https://phabricator.vyos.net/T2389

This PR should correct the issue.

It should. This should really be a non-breaking change as it's a fallback for something else that already exists in /etc/hosts.

Just digging around I found this:

There is already a new build containing the fix.

@Viacheslav could you please check if this probably should make it into 1.2.6 in addition?

https://github.com/vyos/vyatta-cfg-quagga/pull/51

@c-po When you submit the following commit, the problem seems to be solved, when I need to do further testing tomorrow, waiting for tomorrow's daily build

Problem was introduced by porting PPPoE to the get_config_dict() implementation T2653 commit https://github.com/vyos/vyos-1x/commit/65fa21f5

The most likely culprit is /opt/vyatta/sbin/vyatta_interface_rescan. I'm not sure if this should be fixed or migrated to Python.
The rewrite would need to be done together with all other vyatta interface renaming and detection scripts.

Upgrading a different test VM with different config that starts at eth0: on 2nd reboot the hw-id lines are duplicated too, but they are the same on a single interface, and there are no new interfaces created, so the config loads and works fine. The duplicated hw-id lines stay in the config for all subsequent reboots.
Example:

ethernet eth0 {
    address "192.0.2.1"
    hw-id "52:54:00:2d:29:19"
    ipv6 {
        address {
        }
    }
    smp-affinity "auto"
    speed "auto"
    hw-id 52:54:00:2d:29:19
}

What I'm noticing is that the migration scripts save all nodes with quotes, but saving in config mode (through vyatta-cfg) results in most nodes not having quotes (mostly just those with spaces have it). Maybe there is a vyatta script that adds any new interfaces to config.boot that runs on each boot that doesn't like these quoted hw-id lines that the migration scripts produce.

I tried reupgrading from 1.3-rolling-202006110117 to 1.3-rolling-202007050117 and the exact same error occurred - on first reboot everything was fine (config.boot was migrated, looked correct, and loaded fine). On 2nd reboot, the exact same thing happened.

Necessary run service with priority for correct starting https://github.com/vyos/vyos-1x/pull/489

Does DNS static-host-mapping still work with the nssswich.conf change? I‘m just curious about the side effects.

@c-po The basic cause of the failure has been determined.This time, the problem may be relatively serious, because it is not the configuration of the service, but the execution of commit will not start the dhcp6c service at all.

Duplicate of T2627

Somehow I do not want to change the overall system behavior by altering nsswitch.conf. I wonder if we should not enable "disable-host-ookups" by default as an IP address is in the end more useful then a resolved PTR. A PTR record can be changed later on when dissecting the logfiles but an IP lookup should stay longer.

Linux tries to bind SSHd to the VRF but it is yet not ready. After restarting SSH to often (rate-limiting) it is blocked.