Page MenuHomeVyOS Platform
Create Task
Maniphest T3662

Container configuration upgrade destroys system
Closed, ResolvedPublicBUG
Actions

Assigned To
c-po
Authored By
jack9603301
Jul 3 2021, 2:00 PM
Tags
Referenced Files
F1695681: 图片.png
Jul 21 2021, 2:33 AM
F1516120: 图片.png
Jul 3 2021, 2:00 PM
F1516118: 图片.png
Jul 3 2021, 2:00 PM
F1516117: 图片.png
Jul 3 2021, 2:00 PM
F1516119: 图片.png
Jul 3 2021, 2:00 PM
Subscribers
c-po
imathew
jack9603301
pasik
Viacheslav
Maintainers

Description

When upgrading a vyos system containing container configuration, the configuration will be destroyed, such as the following configuration:

set interfaces ethernet eth0 address dhcp
set interfaces ethernet eth0 ipv6 address autoconf
set system name-service 114.114.114.114
set container name nginx image nginx
set container name nginx allow-host-networks
commit
save
exit

Then, upgrade the vyos:

add system image https://downloads.vyos.io/rolling/current/amd64/vyos-rolling-latest.iso
reboot

Restart after the first upgrade:

图片.png (488×1 px, 15 KB)

Restart after the second upgrade:

图片.png (493×744 px, 8 KB)

图片.png (577×1 px, 12 KB)

图片.png (581×1 px, 10 KB)

Details

Difficulty level
Hard (possibly days)
Version
-
Why the issue appeared?
Issues in third-party code
Is it a breaking change?
Perfectly compatible
Issue type
Bug (incorrect behavior)

Related Objects
Search...

Event Timeline

jack9603301 created this task.Jul 3 2021, 2:00 PM
jack9603301 triaged this task as High priority.Jul 3 2021, 2:09 PM
pasik added a subscriber: pasik.Jul 3 2021, 5:04 PM
jack9603301 claimed this task.Jul 4 2021, 6:40 AM
jack9603301 added a comment.Jul 4 2021, 1:27 PM

PR: https://github.com/vyos/vyos-1x/pull/909

Viacheslav added a parent task: T2216: Containerized third-party applications for VyOS.Jul 4 2021, 3:07 PM
Viacheslav added a subscriber: Viacheslav.Jul 9 2021, 8:39 AM

In the latest rolling release all works fine without any changes

vyos@r1-roll:~$ show version 

Version:          VyOS 1.4-rolling-202107010537
Release Train:    sagitta

vyos@r1-roll:~$ sudo podman  run --net host --rm fedora echo it works
it works
vyos@r1-roll:~$
jack9603301 added a comment.Jul 14 2021, 6:56 PM

PR: https://github.com/vyos/vyos-1x/pull/922

jack9603301 added a comment.Jul 14 2021, 7:00 PM

note: Record the process of upgrading from 1.4-rolling-202107010537 to 1.4-rolling-202107122017

After the first startup, the container fails to start. Restart the operating system and start normally.

Wait for the next test to determine if this problem is resolved!

jack9603301 added a comment.Jul 21 2021, 2:33 AM

图片.png (669×1 px, 132 KB)

Unfortunately, it seems that the same problem still happened. Although it seems that the container can be started normally after restart due to priority adjustment and some factors, it is not a normal operation result!

c-po changed the task status from Open to Confirmed.Aug 29 2021, 8:09 AM
c-po set Issue type to Unspecified (please specify).
c-po added a subscriber: c-po.

Reproducible on my side, too.

set container name unifi environment TZ value 'Europe/Berlin'
set container name unifi image 'jacobalberty/unifi'
set container name unifi memory '1024'
set container name unifi network pod-network-01 address '172.29.1.2'
set container name unifi volume config destination '/unifi'
set container name unifi volume config source '/config/unifi'
set container network pod-network-01 prefix '172.29.1.0/24'

It only happens once after the VyOS upgrade. Podman errors out with 125.

If podman can not find an image it returns 125. I don't think this is an error, I think that Fedora for some reason did not have fedoraMinimal available for a while.

Another call to:

vyos@vyos:~$ configure
lo[edit]
vyos@vyos# load
commit
Loading configuration from 'config.boot'
Load complete. Use 'commit' to make changes effective.
[edit]
vyos@vyos# commit
[ container ]
systemctl start podman.service
podman network ls --quiet --filter name=^pod-network-01$
podman run --detach --interactive --tty --replace --memory 1024m --memory-swap 0 --restart on-failure --name unifi   -v /config/unifi:/unifi  -e "TZ=Europe/Berlin" --net pod-network-01 --ip 172.29.1.2 jacobalberty/unifi

[edit]
vyos@vyos#

and all is god for every reboot

c-po added a comment.Nov 4 2021, 7:58 PM

Fixed in https://github.com/vyos/vyos-1x/commit/358f0b481d8620cad4954e3fe418054b9a8c3ecd

c-po closed this task as Resolved.Nov 4 2021, 7:58 PM
c-po claimed this task.
c-po changed Difficulty level from Unknown (require assessment) to Hard (possibly days).
c-po changed Why the issue appeared? from Will be filled on close to Issues in third-party code.
c-po changed Is it a breaking change? from Unspecified (possibly destroys the router) to Perfectly compatible.
c-po changed Issue type from Unspecified (please specify) to Bug (incorrect behavior).
imathew added a subscriber: imathew.Dec 21 2021, 4:23 AM
imathew added a comment.Jan 11 2022, 3:42 AM

Hi, I've just submitted a pull request (https://github.com/vyos/vyos-1x/pull/1154) to hopefully complete this bugfix.

The earlier commit (thanks @c-po) introduced a startup retry (up to 10 times) to allow the OS to settle before the container is started.
That is a welcome change, but it only applies if host networking is NOT used.
This further change applies the same fix for containers where host networking is employed.