We have find an important bug in VyOS: if quite a large ruleset is configured in the system, it will work OK and even add new rules while it's powered on, but if the device is rebooted it will crash at startup.
I will detail our investigation and the tests we have done:
- We have a config.boot of more than 25K lines with several rulesets and static routes that was working OK and with no error at commits or saves, but when we rebooted the device the console returned an error configuration, the interfaces were messed up and we were not able to connect to the device remotely.
- After several tests, we have narrowed down the issue to a ruleset called LAN-INBOUND, with a length of almost 12K lines at config.boot. The funny thing here is that there are neither errors nor unsupported config in this ruleset, but some kind of a "ruleset length limit" (we guess): we have copied config.boot to two different files: one with the first half of the ruleset and the second one with the second half of the ruleset.
- When we try to boot the system with any of these configuration files with the half of the ruleset, the system boots up successfullt, but when we try to boot the device with the full ruleset (both halves included) it crashes, does not apply the configuration and even mess the interfaces up appearing a new one called "renameX" (rename4 in the attached image "iface_rename.png"):
In order for you to try to replicate the bug, it can be easily done with the following three files that have been uploaded:
- "config-base_lan.cfg": config with the full ruleset
- "config-base_lan_half1.cfg": config with first half of the ruleset
- "config-base_lan_half2.cfg": config with second half of the ruleset
Steps to reproduce:
- Copy all three files to /config/
- Copy the first half ruleset to /config.boot and reboot (it should work):
sudo cp /config/config-base_lan_half1.cfg /config.boot reboot
- Repeat the step 2 with the second half ruleset (and it should also work)
- Repeat the step 2 with "config-base_lan.cfg" (and it should crash with "configuration error" at startup).
Maybe you can find useful that if you boot the system with an empty or default configuration and then just load the config file containing the full ruleset it works!! But it doesn't if you try to load that config file on boot.
In addition, for testing purposes we haven't assigned the ruleset to any interface... and even then the system messes the interfaces up at boot!!
Additional note: this bug also happens with VyOS 1.1.8, but with a difference: while VyOS 1.2.0 crashes but leaves config.boot untouched (with the original configuration), VyOS 1.1.8 crashes as well but also modifies config.boot leaving it inconsistent.
Please let me know if you need further information or any other thing I can help with.
Thank you very much and best regards!