Page MenuHomeVyOS Platform

Rewrite system image manipulation tools in Python
In progress, Requires assessmentPublicFEATURE REQUEST

Description

We still use old bash scripts for image manipulations, like install, add, remove, etc. This blocks the ability to reuse system image tools for new useful feature implementations, like unattended installations, automation, etc.
Would be good to move this all to Python in a way that allows reuse in third-party components. For example, create a module with all basic features (add, install, delete an image, prepare disk, modify boot config), and op-mode scripts with logic that will use this module.

Details

Difficulty level
Unknown (require assessment)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Perfectly compatible
Issue type
Internal change (not visible to end users)

Related Objects

StatusSubtypeAssignedTask
OpenFEATURE REQUESTNone
Openerkin
In progressFEATURE REQUESTNone
In progressFEATURE REQUESTzsdc

Event Timeline

zsdc changed the task status from Open to In progress.Jul 8 2022, 4:46 PM
zsdc claimed this task.

Is it possible to also add some logic to populate boot entries using https://uapi-group.org/specifications/specs/boot_loader_specification/ ? I have been experimenting with systemd-boot, and it's working fine apart from the missing loader files. Those files look something like this:
/usr/lib/live/mount/persistence/loader/entries/1.4-rolling-202210050218-vty.conf

title     "VyOS 1.4-rolling-202210050218 (KVM console)"
version   1.4-rolling-202210050218
options   boot=live quiet rootdelay=5 noautologin net.ifnames=0 biosdevname=0 vyos-union=/boot/1.4-rolling-202210050218 console=tty0
linux     boot/1.4-rolling-202210050218/vmlinuz
initrd    boot/1.4-rolling-202210050218/initrd.img

There will be similar files for serial and USB console.

My ESP looks like this and my firmware is set to boot systemd-bootx64.efi (can tweak this with efibootmgr):

[email protected]:~# mount /dev/sda2 /efi/
[email protected]:~# find /efi/
/efi/
/efi/EFI
/efi/EFI/VyOS
/efi/EFI/VyOS/grubx64.efi
/efi/EFI/BOOT
/efi/EFI/BOOT/BOOTX64.EFI
/efi/EFI/BOOT/BOOTX64B.EFI
/efi/EFI/systemd
/efi/EFI/systemd/drivers
/efi/EFI/systemd/drivers/ext2_x64.efi
/efi/EFI/systemd/systemd-bootx64.efi
/efi/loader
/efi/loader/loader.conf
/efi/loader/random-seed

The systemd-bootx64.efi is taken from a the latest version of systemd (the one in debian11 is too old and does not support autoloading of EFI drivers.) The ext2_x64.efi driver makes the root of the main partition (which gets mounted at /usr/lib/live/mount/persistence) available to systemd-boot. I've changed the type for the main partition to Linux extended boot (gpt UUID bc13c2ff-59e6-4262-a352-b275fd6f7172) in order for systemd-boot to consider it when it scans for boot entries at /loader/entries.

Disklabel type: gpt
Disk identifier: 7182D4CB-F36A-4EBE-8862-8E1572314548

Device      Start       End   Sectors  Size Type
/dev/sda2    2048    526335    524288  256M EFI System
/dev/sda3  526336 117231374 116705039 55.6G Linux extended boot

If you use systemd-boot to boot an EFI system it would not work on RAID afaict, due to missing mdraid drivers available for systemd-boot. However, the loader files are generic and there may be other bootloaders that also use the same specification, currently fedora/rhel GRUB also supports it (used by default on fedora and latest rhel) . The specification is not bound to EFI and would also work on other platforms with supported bootloaders.

The simplicity of the boot entry files makes the boot process very clean IMO. If the system could write the *.conf files as part of image installation, then they would be ready for use in case at some point in the future you consider systemd-boot or a version of grub that supports them.