Protobuf schema has been written.

@jpbostic My idea for interacting with vyconf from outside the interactive shell is a bit different. The issue with 'vyshell -c "set interfaces ethernet eth0 disable' is that it needs to setup a session first, and store the session ID between commands, so either it will be limited to 'vyshell -c "configure; set interfaces ethernet eth0 address 192.0.2.1/24; set interfaces ethernet eth0 mtu 1400"' (i.e. long command strings in single call), or it will be dependent on specific environment setup, and from VyOS 1.x we already know how problematic it will be.

@dmbaturin @mickvav yes, definitely a very good point, and I'm guessing that same new VyOS shell would then be callable from the changed-to Unix shell (e.g. cli -c "show configuration commands | match blah") ... nice.

I'm a "NO" as a network engineer with a bunch of different brands already XORP style, or as close to JunOS as you can get it the best. Yet another (Similar) config style would be way too much frustration for most of my peers to even consider.

@jpbostic I think @mickvav has a point here, making it configurable is relatively cheap (it's just a field in /etc/passwd after all, or a call to chsh). People who don't want it to be the deault like myself and service accounts for ansible etc. can just change the default.

Once @tmartinson setups a physical server for us (I'd like to say thanks to him, by the way!), it will become a permanent place for the jenkins VM and build hosts where we can give access all maintainers without worrying about mixing Sentrium corporate stuff with it.

@dmbaturin, you can probably assign this one to me, if you feel comfortable doing so. I think I'm nearly done. I'd just like to put together some decent test cases before making a PR.

FWIW, I definitely prefer JunOS-like (issue command to enter VyOS shell) behavior and agree with the comments about remote configuration, such as with Ansible. This makes mass configuration, change-controls, and backups much more like other *nix based installs. It also makes munging the VyOS command output available out-of-the-box on the VyOS install, i.e. IMO it would be much easier to call VyOS shell sniplets from scripts in bash, tcsh, python, perl, etc, than to deal with getting out of the more captive shell back to a "real" shell for a custom script, cron job, etc.

@UnicronNL Thanks.

I heard there will come a new server available to run jenkins on, i have to wait until i have more information.

And https://ci.vyos.net certificate is invalid.

https://ci.vyos.net/computer/jessie64devel.vyos.net/

Nightly builds are not working again? It seems jessie64devel.vyos.net is down now.

I think if you keep in mind the Principle of Least Astonishment, the answer becomes obvious: when you login to VyOS, do you expect a VyOS shell or a Unix shell? VyOS! Conversely, when you login to Unix, do you expect a Unix shell or a VyOS shell? Unix!

For me the current defaults is fine for router-like device. But it's a good idea to have this option in user config, e.g.

Well, my vote is "No", because if for small configs it's OK to have just intent-expressed syntax, if you have huge one, e.g. several pages - if you omit prefix before, say, 55, you will have to guess from context, if it is a vlan or preffix list entry, or VRRP group or whatever.

The suggestion from @rps (XORP style) seems to be the best way from my point of view:
https://phabricator.vyos.net/V3#51

With respect to the concerns I mentioned above, I've voted no.

@dmbaturin, Im with you on the aesthetics, and the readability. In the firewall ruleset example I still feel that the first is easier read than the second. Are we talking hundreds of lines to parse the former vs the latter? It seems like the later, across a whole config would at 10-20 lines if not more depending on the complexity. I for one am interested in seeing as much of the config on one screen, vs needlessly needing to scroll. As for your Q on pfSense, I've had to edit the xml configuration file by hand based on how pfSense sorts VLANs based on their add date vs numerical value.

@tmartinson Well, you should change your vote then (votes are not final here, for the better I guess).

I keep coming back to a sense that dramatic syntax changes are very damaging and disruptive to users. My fear is that we'll be spending years explaining to people that they're looking at old documentation or examples and that they don't have their curly braces in the right place. Or that we'll alienate a segment of our user base that is averse to change.

In the example above, I vote that the first example where name Foo and rule 10 are on the same line. It is much easier to read, and shortens up the output on the display. Sometimes with long configurations, it is easier when you can see more information on the same screen without scrolling.

@systo Just to make sure you are looking at it the right way, in the large it's actually less verbose than old syntax. The vif may not be the best example but firewall would make it apparent:

As an end user, I just keep coming back to the verbosity of the syntax, and the divergence from all the other established command syntax in this space. VyOS doesn't have the following to do it differently, as it adds another barrier to adoption. Its a subtle change, but it has a long reach, especially when luring former vyatta or EdgeOS converts that want to roll-their-own, vs buy MIPS hardware. While I understand it may save coding time in the end, I'm trying to avoid the verbosity that is pfsense, and awall/shorewall. I bet if you asked a room of non-vyos engineers, they would prefer the first syntax with a much higher percentage, but alas I digress.

Any change that imparts simplicity for the coding ahead is worthwhile. Time saved in the parser's reduced complexity can be spent in other ways.

@rps An serious issue with "interfaces { eth0" is that when there is no parent subtree of all ethernet interfaces specifically, we don't know which script to call when something in "eth0" changes. We'd have to have one big script that handles the whole "interfaces" subtree, which is very problematic when it comes to adding new interface types. If eth* interfaces are children of the "ethernet" node and tun* interfaces are children of the "tunnel" node, it's easy to attach ethernet script to the "ethernet" node and "tunnel" script to the "tunnel" node, if we want to add "openvpn" later, we won't have to modify that large script to accomodate it

I haven't voted yet because I haven't decided ... It's a big change.

@rps No, that's not the biggest challenge. Semicolon at the end of leaf nodes makes them unambiguous enough and easy to tell from tag nodes (this is especially bad with valueless nodes by the way, think "disable", colon wouldn't help there, but semicolon at the end does the job). The biggest challenge is that with "ethernet eth0" the parser must be fully stateful and capable of tracking which parent nodes it's already seen. "eth0", "eth1" etc. are really children of the same node called "ethernet", but in the config they appear separately. Consider this unusual but logically valid config:

@rps this distinction also seems to be easy in the original proposed solution by @dmbaturin because key value pairs are not followed by '{' and the rest is.

From a parsing perspective the only challenge tag nodes present is that you can't easily distinguish between "key value" and "key tag" without context. "key" and "key tag value" are fine. Using a ":" you get "key: value" vs "key tag" which removes the ambiguity.

@dmbaturin I understand that the discussion is "unit 0" vs "unit { 0", what i meant was that i could be an option to keep following the JunOS style as much as possible to maybe enable more interoperability.

Well plain JSON would also be an option then :-)

@Merijn I'm still not sure why JunOS has that "unit" thing. To me it looks redundant, redundant ©. Though what we are discussing is "unit 0" vs "unit { 0" grammatic distinction, rather than specific syntax of ethernet interfaces.

The XORP configuration syntax (which Vyatta initially built upon) solves the parsing issue with the simple introduction of a ":" as a delimiter between keys and values.

In the blog post #7 i liked the address [ 192.168.2.1/24 10.10.10.1/30 ]; part. But since i work most of the time with mixed JunOS and Vyos environments a mostly the same syntax would be very nice :-)
However JunOS would be:

I was thinking that the variable would actually be "vlan-id 99". That was written simply to make it easier to read. But if it will be the top of a node, then we end up with vif, vlan-id. Which is redundant, redundant. In that case I would drop the "vlan-id" portion all together. It is only there for esthetics.

@tmartinson No, "vlan-id 99" is the old style. And, at that stage we don't know if it's ethernet or not.

Maybe something like this? We already know that it is an ethernet interface by the fact that it is eth0. And by adding the "vlan-id" portion we get a newer style of configuration but keep the read-ability of the configuration stanza.

@Merijn Now that you remind me of it, I think "edit interfaces tunnel; copy tun10 to tun11" or similar should be possible regardless of the config syntax. No matter how it looks in the config, internally "tunnel" is a node with children "tun0", "tun1" and so on, and there's no reason why it shouldn't be possible to use it as edit level.

A pro for me would be that i can do 'edit interfaces ethernet eth0 vif' and work with all virtual interfaces.

@dsteinkopf Not sure, we'll have to devise some rules regarding line breaks, and past some number of leaf nodes inside we are back to the original aesthetic issue (and then there can be non-leaf nodes inside too!
On a fresh look today, I'm convinced that the old tag node formatting is aesthetically superior, so myself as a user of my own project I'm probably voting no, though as a developer I want to see how many people also think it's worth it.

Maybe it's a good idea to 1. use the new syntax but 2. generate less line breaks. e.g.

interfaces {
  ethernet { eth0 {
      vif {
          99 { address 192.0.2.1/24; }
          101 { address 203.0.113.1/24; }
      }
  } }
}

In this case the new syntax would be fine for me. (Details open for discussion.)

In JunOS the root user enters in the shell and uses 'cli' to enter show mode followed by configure for config mode.
When i add an extra user without shell access, this user is placed directly into show mode.
The vyos user is not the root user, so the way it currently is makes perfect sense to me.

I like the separation for admin vs view. Its the same reason we have RO and RW in SNMP v2c etc. While I don't yet have hands-on experience with Junos, I specifically like the demarcation of configuration vs show commands. For those that don't like the dual approach, can't the run prefix be used to enable a flatter CLI approach?

As long as it is simple to get to a system shell, I'd prefer the VyOS shell to be the default

for unix shell better, but it's just because i'm more linux then network guy.
i think MOTD with how to enter into VyOS shell should be enough.

I started using VyOS because I saw Ansible gained support for configuring VyOS devices. I want all my devices in config management. I'd lie if I said I was familiar with the inner workings of the Python module that interacts with VyOS, but from my config management POV I think it makes sense to login into a normal shell and then perform 'some action'. This action could be done in a CLI tool or it could be something OS-related. So for me, I think it makes sense to login to a normal shell first. My two (possibly worthless) cents ;)

Beside Juniper, other router vendor starting with there specific cli shell. But if the JunOS approach is much easier for maintenance on the long term, take this way.

How did you go with OFP? I can compi;e and run it on vyos platform but first I tried in on Ubuntu and found it much worse than stock linux perhaps there are some setup that is required for it to function optimally. I also tried vpp and that doubled the pps at 64bytes.

Nightly builds are working again, need to fix web hooks next.

Yes, related. I was just talking to myself really, we get the CI back first, and then we can look into adding vyconf to it.

Our gateway is bad and we should feel bad. When jenkins migration to the new site is complete (we are migrating build hosts too), this should work again.

I get a 502 Bad Gateway too.
Is this related? https://phabricator.vyos.net/T222

Awesome. I don't know if it's just me but I get a 502 Bad Gateway when accessing https://ci.vyos.net/

Thanks! Unit tests pass.

Unit tests pass for me too.

Mentioning: http://pastebin.com/yZLVRfnA
Which is an example of how would WLB work with a custom script.

@EwaldvanGeffen apply this rule on what? a WLB?
the WLB from what I understood required an interface per gateway while PBR allows me to route the traffic towards any of the gateways which can be the next-hop ie 10.0.0.100/24 or 10.0.0.101/24.
This is what I remember from vyatta and I haven't digged into the subject since I have a huge gap ahead as far as I can see.

@elico if you apply a 'source my-lan-clients, destination port-80, proto tcp' rule with gateway your proxy server + the custom testing-target script. If the proxy is up it will be routed towards it. If the target goes down, without any other policies the packet will fall onto PBR and then routing. Isn't that the behaviour you were looking for?

@EwaldvanGeffen WLB has a difference from PBR and what is required a PBR.
The code is not something I was looking for but an example of implementation in the configuration.
Then I will be able to look at the code and understand what might be applied to PBR compared to WLB.

Wan-load-balance. Example is here: https://github.com/vyos/vyatta-wanloadbalance/blob/current/scripts/http_test.pl and implementation https://github.com/vyos/vyatta-wanloadbalance/blob/current/templates/load-balancing/wan/interface-health/node.tag/test/node.tag/type/node.def

@EwaldvanGeffen Can you help with giving an example of implementing this?
Like with a tiny ping that returns a status code?
(I do not know what WLB is...)

@elico it's pretty simple since WLB supports custom tests for gateway/targets. You can simply script it up to that.

@EwaldvanGeffen technically we can simplify it into a form of a script that monitors the service using http or another tcp\udp based and would flag the avaliability of the service.
The marking and forwarding rule can be automativally bypassed if the service is flagged as down.
Anyone interested working with me on this?
It's basically a simple conditional PBR.. and since WCCP is "OK" for tiny routers for beafy machines such VYOS have I believe that it would be a piece of cake to cook this up.

Here is a howto on the openfastpath - https://www.howtoforge.com/tutorial/opendataplane-with-open-fast-path-on-ubuntu/

Does openfastpath really work? Have you tried it? It all looks great, and if it works reliably, we indeed should integrate it.

Would the http://www.opendataplane.org/ and http://www.openfastpath.org/index.php/service/technicaloverview/ not work better in VyOS?
Use this to create a fastpath interface and the linux OS can just that.