Page MenuHomeVyOS Platform

integrate new installer made by Kroy
Closed, ResolvedPublicFEATURE REQUEST

Description

@kroy created new installer

https://forum.vyos.io/t/any-interest-in-an-updated-installer/2719

Provides a bit more “modern” experience by using whiptail dialogs, progress bars, etc.
RAID1/10/5/6/etc support
ZFS support with stripe, mirrors, RAIDz1/2/3. Yeah, pretty much completely unnecessary for VyOS, but it was only a few more lines of script.
MBR/GPT Support
EFI Support. This was my main interest, and why I started using whiptail; to make it easy to navigate a few choices. It works, but requires manually booting an EFI shell the first time because the default ISO that gets built doesn’t support EFI. That means no grub access to the EFI variables until a reboot. Also means it wouldn’t work on an EFI-only system. To fully make it work out-of-box would require a few more changes to the image as a whole.

Will need to review and integrate it in current and LTS

Details

Difficulty level
Unknown (require assessment)
Version
-
Why the issue appeared?
Will be filled on close

Event Timeline

syncer triaged this task as High priority.Nov 2 2018, 8:51 AM
syncer created this task.
syncer assigned this task to UnicronNL.Nov 2 2018, 8:54 AM
kroy added a comment.EditedNov 5 2018, 7:04 PM

https://github.com/kroy-the-rabbit/vyatta-cfg-system

The short, this is a refactored installer that should be the starts for supporting EFI, as well as integrating some whiptail menus. Initially I totally rewrote it and cleaned it up, and the menu-ing was a bit more complete, but there is SOOO much going on there, that I decided to go back and just rebuild it using the existing scripts.

Obviously if the menu-ing changes are undesired, the EFI changes amount to about 20 lines of code that could be pulled out. My initial rewrite was far more dependent on the menu-ing than the current system


custom_packages = dosfstools (for EFI partition formatting) gdisk (GPT partitioning for EFI) whiptail (menus)

Problem #1. grub-pc and grub-efi-amd64 are incompatible. This ultimately means that with the current ISO building system, you would need to publish two ISOs, one without grub2 and grub-pc and adding grub-efi-amd64 in data/package-lists/vyos-x86.list.chroot in vyos-build. Obviously this isn't ideal, but my familiarity with the project is limited at this point.

Problem #2. The current ISO install also only boots in BIOS mode. This prevents the if [ -d /sys/firmware/efi ]; then check from ever passing and the EFI installation from happening (line 667 install-get-partition). This means grub-efi can't install the required EFI variable.

Problem #3. Something changed in the last week since I was last working on this. The mounting of the vfat no longer works, where it did before (the code in this module is identical). It fails complaining about a missing codepage. I pulled down vyos-kernel, and the appropriate codepage exists in the .config, so I'm not sure where it's broke.

The bottom line is this needs mount the vfat EFI partition to install grubx64.efi but it can't with the current VyOS kernel.

line 150: install-postinst-new

Nov 5 18:19:14 localhost kernel: [ 1581.458192] FAT-fs (sda2): utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!
Nov 5 18:19:14 localhost kernel: [ 1581.460581] FAT-fs (sda2): codepage cp437 not found


With that all said, I am using this install and have about a dozen working EFI installs. Prior to the version that broke due to the missing codepage, the steps to get around the shortcomings I mentioned:

  • Change the grub package I mention above
  • Boot the ISO in BIOS mode
  • Override the /sys/firmware/efi check to always true
  • Let it install
  • Reboot and boot into a EFI shell, manually run grubx64.efi
  • run grub-install again, per line 154 install-postinst-new
  • Install is a working EFI system now.
syncer added a comment.Nov 5 2018, 7:39 PM

@kroy actually @UnicronNL already implemented hybrid iso boot that works with uefi and bios
we now need only installer which will handle installation
but by this time we already either booted via one or other, so we just do install with required support files

kroy added a comment.Nov 5 2018, 7:45 PM

@syncer Give me about an hour here to post a commit with JUST the EFI changes then, since that's what you are interested.

syncer added a comment.Nov 5 2018, 7:48 PM

@kroy no hurry
see also T967
maybe we should spend more time and address that too

kroy added a comment.Nov 5 2018, 7:49 PM

@syncer Actually, my original script addressed that too. I'll integrate that change back in

kroy added a comment.Nov 5 2018, 9:49 PM

https://github.com/kroy-the-rabbit/vyatta-cfg-system

Okay, this should just be the EFI stuff.

Assuming you have a hybrid-ISO that has the grub-efi package installed, EFI should work with the exception of Problem 3 above. I'm not sure what happened, but the ISO refuses to mount FAT32 partitions. But a week ago, it had no problem.

codepage cp437 not found is the error and it seems to suggest that something in the kernel compile changed.

syncer added a comment.Nov 5 2018, 9:57 PM

@can you do PR mentioning this ticket please
@UnicronNL can you check on fat32 issues

@syncer will check @kroy thanks a bunch!!

kroy added a comment.Nov 6 2018, 3:54 AM

I'll generate this pull request a bit later or tomorrow morning (EST here). I noticed it was missing some of the stuff with RAID

runar added a subscriber: runar.Nov 6 2018, 9:00 AM
kroy added a comment.Nov 6 2018, 3:58 PM

I'm doing some testing before re-publishing. Looks like whatever was broken with the codepage cp437 error was resolved.

@kroy what kernel is it now you use? I updated the kernel 4.14.75 and 4.19.0
https://github.com/vyos/vyos-kernel/commit/32f3f5caaec8bff6f372ab22801cfaaa30a766cd

Please let me know when you have made pull request.

kroy added a comment.EditedNov 6 2018, 4:48 PM

@UnicronNL kernel in use is now 4.14.75-amd64-vyos

As of this morning, that kernel version resolves the issue where FAT32 wouldn't mount. It's back to a state where you can kludge an EFI install

Pull request is https://github.com/vyos/vyatta-cfg-system/pull/83. I cleaned up a lot of the cruft and simplified it.

@kroy
If you would like to test:
git clone https://github.com/vyos/vyos-build.git
cd vyos-build
git checkout current-uefi
docker build -t vyos-builder .
docker run -it --privileged -v /HOST_PATH_OF_VYOS_BUILD_REPO:/vyos -w="/vyos" vyos-builder bash
./configure
make iso

kroy added a comment.Nov 9 2018, 4:08 PM

@UnicronNL

Just ran it on a few different machines, VMs, set up RAID/non-RAID. It worked great. Thanks!

syncer closed this task as Resolved.Nov 9 2018, 4:40 PM
syncer added a project: VyOS-1.2.0-GA.
syncer moved this task from Needs Triage to Finished on the VyOS 1.2 Crux (VyOS 1.2.0-rc7) board.

@UnicronNL merge into rolling and crux, please

Responding to Problem #1 mentioned by kroy at Mon, Nov 5, 7:04 PM.

The grub-pc and grub-efi-amd64 packages are incompatible.

The required parts are actually in the '-bin' packages, eg grub-efi-amd64-bin. The '-bin' packages of different architectures may be installed at the same time and the architecture selected with the '--target' option to grub-install, eg grub-install --target x86_64-efi.

kroy added a comment.Nov 16 2018, 9:30 PM

@dgeary2

This was resolved by @UnicronNL when the dual-mode ISO was built. As of RC7, the installer now integrates this and can successfully do a UEFI install