@hexes please provide exact version number "current" does not really help in this case as definately something else is going on with 1.3-rolling-202006011159
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
All Stories
Jun 2 2020
Unfortunately I receive a different error:
Looks like I'v got same error: https://phabricator.vyos.net/T2542
As a side note, if there is a need to migrate between backends, it would be possible to change the code to have an option to indicate which storage should be used and allow loading from one, saving to another as long as the XML schema is the same.
I see the POC as complete and conclusive, it would only make sense to spend more time on things once this is done (or to add support for control via HTTP/API).
vyos@MSklad# save Warning: you have uncommitted changes that will not be saved.
vyos@MSklad# show interfaces openvpn +openvpn vtun1 { + description VPN + device-type tap + mode client + openvpn-option "--script-security 2" + persistent-tunnel + protocol udp + remote-host ip + remote-port port + tls { + ca-cert-file /config/auth/ca.crt + cert-file /config/auth/MSklad.crt + key-file /config/auth/MSklad.key + tls-version-min 1.0 + } + use-lzo-compression +} [edit] vyos@MSklad# commit [ interfaces openvpn vtun1 ] Traceback (most recent call last): File "/usr/libexec/vyos/conf_mode/interfaces-openvpn.py", line 1028, in <module> apply(c) File "/usr/libexec/vyos/conf_mode/interfaces-openvpn.py", line 1018, in apply Interface(openvpn['intf']).set_state('up') File "/usr/lib/python3/dist-packages/vyos/ifconfig.py", line 110, in __init__ raise Exception('interface "{}" not found'.format(self.config['ifname'])) Exception: interface "vtun1" not found
Configuring 100 dummy interfaces (no thread) on a local VirtualBox, all tests done from boot
@c-po After thinking about it, maybe the only thing I can think of is the word limit of a single line. I submitted another commit.
Jun 1 2020
vyos@vyos:~$ wake-on-lan interface eth0.201 host a:b:c:d:e:f
Thank you for this long answer @jjakob. I want to demonstrate that a full python solution can provide the performance we need. I appreciate that changing the Vyatta code need to be done carefully with many consideration about backward compatibility. What I am doing is surely 1.4 material. However I do not believe this is as hard to achieve as everyone may think, and as working code is the best way to discuss code design, that is what I am doing.
The cli is mostly functional. I am able to validate the data as it is typed in conf mode (the CLI has both completion and validation working), as soon as delete and show (the current show is "show configuration commands") are implemented, it will be mostly usable. The code can already load a configuration from file, allow some "set" edit and then allow the use of save (the config format is a number of "set" commands, one per line), respect the initialisation order of the XML.
May 31 2020
As the current "priority map" there aren't a loot of concurrent python blocks, but i think many of the remaining bash/perl scripts could be moved to new places. https://pastebin.com/z6ZvkJKB
I've created some proof of concept code that i think could help on this issue. https://github.com/runborg/vyos-1x/blob/main-cfg/src/conf_mode/main.py this is a conf-mode executor that handles multiple conf mode scripts. The reason i think this could seriously help on this issue is that as this is all running inside a single python tnterpreter, its able to load the config object once and pass it to all needed conf_mode scripts without a need for reinitialization.
May 30 2020
I also couldn't immediately grasp how your suggestion would fit into the whole design concept of VyOS - now that I've had some time to think about it, it indeed makes sense. I was having problems seeing how having a complex daemon just for validators would make sense while still leaving everything else as is - it makes sense if everything is moved there. And only because of the current state of having to "marry" vyatta-cfg with Python and Python's slow initialization times. I guess in the future, if VyConf gets to a place where it can replace vyatta-cfg, it would still make sense to run everything in a single daemon.
@thomas-mangin I think there was a misunderstanding between us. The disagreement we had regarding the way to implement vyatta-cfg validators was because the validators are a integral part of vyatta-cfg operation. They are also simple and small as they only need to validate the types and constraints of config nodes. As they are tied to vyatta-cfg closely, which operates by executing a new process for each config node, that execution needs to be very fast. I was against your solution (a validator daemon in Python listening on a socket file and a companion client in a language that's faster to start up) just because it seemed needlessly complicated for what it needs to achieve. Node validation in vyatta-cfg is a case of simple constraints, not complex interdependencies that would require a higher level language. As we later do the complex validation in the configuration scripts that are written in python themselves, all the complexity can already be put there. Now you may be wondering why this validation is done in two places, it's because of the legacy of vyatta-cfg. In the old days of vyatta, many config nodes didn't have corresponding scripts at all, they were self contained and applied the config directly using system utilities and simple shell scripts that were part of the node definitions themselves. In that case, the config node validators were the only validation of a value that was done and each config node coould specify their own shell snippet or script to validate its own value. This made sense in that design concept.
It is also still an integral part of the shell environment: in config mode, a set command with a invalid value will return an error immediately as its validator returns an error. The configuration script can catch an error only when a commit is triggered.
Now that we are tacking a completely different design concept onto that, things become complex. If the new design says: "all new code must be python" but since we're marrying this new code with the old vyatta-cfg core (vyatta-cfg is still the heart and core of VyOS with Python being the "worker"), things will become very unoptimal and complex and bizarre in some places that wouldn't need to be that way and could be left simple. The above being an example of this complexity due to a design choice.
Possibly it's in cases where it's first called in get_config(), then also later in verify() or apply() - the object is function local scope and isn't saved in a global variable. It would be a simple fix if Config() were initialised in main() and passed as an argument to any function that needs it.
vyos@vyos# diff -u /usr/libexec/vyos/completion/list_interfaces.py list_interfaces.py --- /usr/libexec/vyos/completion/list_interfaces.py 2020-03-21 19:47:22.000000000 +0000 +++ list_interfaces.py 2020-05-30 18:45:30.564000000 +0000 @@ -39,4 +39,7 @@ print(" ".join([intf for intf in matching("bondable") if '.' not in intf]))
Why is it sometimes called twice?
Begging for a merge will not speed things up. You multiple times refused to adopt the patch to our guidelines and it is still unclear if this is the right path to go.
I just had a look at VyConf and it is excellent, I fear that no one but @dmbaturin can maintain participate to it.
Also, VyConf will still need to fork all the python code and unless we have a resolution to T2088 - I am not sure what the best path forward will be
I tried to get a flamegraph showing what I was wanting to say but .. do not look very clear :-(
Please merge the following PR, if there is a problem, please let me modify it.
Well I do no longer maintain it so it‘s defacto dead. It only served as poc, I rather use some bash aliases now to build my packages and rely on the deb mirror.
@c-po IMO the script should be kept but fixed so it builds all valid packages. Otherwise there's no way to build our own packages with one command. Sure we can build them one by one by manually cloning each repository, but that's automated by this script. There's a task I already opened for it.
If so, it's better to consider porting the 1.2 NPT implementation instead of using a new solution. Can they coexist? I'm just a suggestion.
I believe it was git pull —tags which fixed it for me ..
Many scripts also so do implement if name == “main”
I'm working on a larger set of patches for DNS, a fix for this will be included
This does not compile python scripts without a .py extension (there are several in src/services, src/utils that have #!/usr/bin/env python3)
The recent implementation here uses the python ifconfig module and walks it to detect interfaces marked as beidgable. I found such constructs are way too slow, simply listing ls -1 /sys/class/net (and do some filtering) is magnitudes faster.
I think I drop the script as it was considered as a PoC but its heavily unmaintained
That is what VyConf is for
It‘s implemented in 1.2 but not with the new nftables based NAT backend as the required commands could not be translated from ip6tables.
Validator now prevents this