Page MenuHomePhabricator

BGP process does not start a boot
On hold, LowPublicBUG

Description

Running VyOS 1.2.0-rc2 and with working BGP config at reboot the router does not start bgpd.
When i manually start the service, remove the BGP config and re-add and commit it does work correctly.

Details

Difficulty level
Unknown (require assessment)
Version
1.2.0-rc2
Why the issue appeared?
Will be filled on close
Merijn created this task.Oct 15 2018, 11:31 AM
Merijn updated the task description. (Show Details)Oct 15 2018, 11:34 AM
Merijn added a comment.EditedOct 15 2018, 11:54 AM

Trying to reproduce it by doing a reboot with a working BGP config i now have lost during the reboot all interface and firewall configuration.
But the BGP config is now present and loaded.
:-(

And by reloading the complete config, after the reboot i am missing the BGP part of the config.
I cannot find a failure message during boot in the logs and on the console.

c-po added a subscriber: c-po.Oct 15 2018, 12:18 PM

@Merijn this look slike a problem with a validation script in VyOS - sorry.

Can you please provide some additional information my:

$ ls -la /config/config.boot.*

Now load the configuration with the latest timestamp in configure mode

vyos@vyos:~$ configure
vyos@vyos:~# load /config/config.boot.2018-<date>
vyos@vyos:~# commit

You will now be hinted why it's failing.

At that location only a couple of pre-migration configs exist. In archive subfolder also only old files exist.
But i loaded the latest pre-migration script, this causes all BGP configuration to be removed by the commit.
The error message is:
% Specify remote-as or peer-group commands first

So that must mean that on of the current BGP peers is not correctly loaded in FRR. I can check that.

@c-po i saved the working complete config to a file, and when loading this after reboot with a lot missing, not a single error message is shown besides: boot-config-loader: Commit failed at boot

Also rollback shows that all my previous commits are saved.
But it seems they try to load very old files from archive. Should i open a separate bug report?

c-po assigned this task to dmbaturin.Oct 15 2018, 1:28 PM

Not at this time.

As you loaded the /config/config.boot.DATECODE.pre-migration files this is fine. Can you please attach a working configuration?

Merijn added a comment.EditedOct 15 2018, 1:56 PM

After a reboot now the config misses the BGP config. Loading it and trying commit shows the following:
Warning: connecting to bgpd...failed!
bgpd is not running

Maybe this is also the case during boot, although i can see in the boot messages that bgpd is started.
Starting bgpd myself en then loading the config works.

I cannot upload the config because it is a test router in a production environment and contains a lot of information about that.

syncer triaged this task as High priority.Oct 15 2018, 3:21 PM

@Merijn I could not reproduce the issue with any of my configs, so I can only suggest two options: either you can make an anonymized version of your config with all AS and network values replaced with those from private ranges (but please make sure the issue is still reproducible with that config); or you can send the actual config to me privately under a promise not to share it with anyone or a formal NDA.

syncer lowered the priority of this task from High to Low.Nov 9 2018, 8:32 PM
syncer changed the task status from Open to On hold.
syncer added a subscriber: syncer.

requires additional input

@syncer In the 1.2.0-rc9 image this seems to be fixed. I upgraded to this release and the config is working flawlessly as far as i can tell now.
BGPD is started at boot, and no config parts are missing.
I am also running several BGP neighbors, and several of them with a full BGP table.

Upgrade to 1.2.0-rc10 and BGP is still working fine. It starts at boot and loads all BGP peers and several full tables.