We will need to build image for GCE
it seems that it will need something similar as for aws
We will need to build image for GCE
- Difficulty level
- Easy (less than an hour)
I'm trying to take the 1.1.7 version and get it working in GCE which I haven't had any luck so far.
I've performed the following steps:
Create Vitualbox image (Debian 64bit) using vyos-1.1.7-amd64 (I've also tried the virt image)
- 2048 memory
- 10 GB drive (QCOW)
- disable audio
- disable floppy on boot
- network/advanced, change to Paravirtualized Network
- Install to image
- Create your password.
- setup ssh.
- setup dhcp on eth interface
- remove hw-id on interface
- modified grup to default to serial
Shutdown Virtualbox Image
- Create Raw Image (VboxManage clonehd -format RAW ./Vyos.qcow ~/disk.raw)
- tar raw image (gtar -cSzf vyos.tar.gz disk.raw) You must use "Gtar" not just "tar" or you'll get the bad image error.
- gsutil cp vyos.tar.gz gs://some-unique-folder/vyos.tar.gz
- gcloud compute images create vyos --source-uri gs://some-unique-folder/vyos.tar.gz
I then create the vm enabling serial output, then connect the serial output.
In the serial output after it goes through grub and says Booting Vyos, you eventually get the following error:
Unable to find a medium containing a live file system.
I've tried the beta builds, the virt iso and haven't had any luck getting this working. Any thoughts?
@silverbp have you considered using Packer? I got Packer building AMI's of VyOS on AWS here: https://github.com/amosshapira/thermal/tree/master/vyos-image, perhaps you can use it as a basis for a GCE image. Let me know if you need help with understanding the code there (it uses a trick to work around Packer's lack of support for creating the final image from a "side-mounted" volume by switching volumes and rebooting).
Hm, I can't get this command to work..
mount -o noatime,upperdir=/mnt/wroot/boot/1.2.0/live-rw,lowerdir=/mnt/squashfs -t overlayfs overlayfs /mnt/inst_root
mount: wrong fs type, bad option, bad superblock on overlayfs, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so
I originally tried using debian 8 and then ubuntu 16.04 which at least had overlay in it. The debian one didn't have the overlayfs module.
ami-72343365 is Ubuntu official public trusty AMI in us-east-1, name "ubuntu/images/hvm-ssd/ubuntu-trusty-14.04-amd64-server-20161205" owner id "099720109477"
Be aware that this AMI is only used as a platform to execute the steps to build the disk image from ISO. Nothing from this AMI is included in the final image.
I see that the instructions for running Packer are lacking so I'm going through extra testing to complete them.
How do you get the code for 1.2 into your image? I use the ISO of 1.1.7 and started work for 1.2.0 and made some progress but stopped when I hit too many problems and realised that 1.2 isn't stable (that was a few months ago).
I got the VyOS 1.2 AMI built by Packer but it doesn't get the ssh key pulled from the metadata and stored in the configuration. With lots of manual testing I got to a stage where "/bin/cli-shell-api setupSession" fails with a core dump.
Context - this is inside a chroot environment under Ubuntu 14.04. I don't know if that could affect the binary?
Here is the gdb stack trace:
(gdb) run setupSession Starting program: /bin/cli-shell-api setupSession [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". terminate called after throwing an instance of 'std::out_of_range' what(): basic_string::erase: __pos (which is 18446744073709551615) > this->size() (which is 0) Program received signal SIGABRT, Aborted. 0x00007ffff69cf067 in __GI_raise ([email protected]=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) where #0 0x00007ffff69cf067 in __GI_raise ([email protected]=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007ffff69d0448 in __GI_abort () at abort.c:89 #2 0x00007ffff72bcb3d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #3 0x00007ffff72babb6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #4 0x00007ffff72bac01 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #5 0x00007ffff72bae19 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #6 0x00007ffff7310cdf in std::__throw_out_of_range_fmt(char const*, ...) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #7 0x00007ffff731b2d1 in std::string::erase(unsigned long, unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6 #8 0x00007ffff7b73c98 in cstore::unionfs::UnionfsCstore::setupSession() () from /usr/lib/libvyatta-cfg.so.1 #9 0x000000000040968d in ?? () #10 0x000000000040b3fc in ?? () #11 0x00007ffff69bbb45 in __libc_start_main (main=0x40b04a, argc=2, argv=0x7fffffffe258, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe248) at libc-start.c:287 #12 0x0000000000408e59 in ?? ()
Does anyone know where to go from here? Could it be important that I should use Debian Jessie instead of Ubuntu 14.04 as the host to debug this?
EDIT: The work done so far is available on a branch here: https://github.com/amosshapira/thermal/tree/issue-1-support-vyos-1.2
with ubuntu 14.04 I still get the error above.
mount: wrong fs type, bad option, bad superblock on overlayfs,
Running through those packer steps. I don't understand disk partitioning enough on this to be able to work through that.
I don't know enough to be able to do this in GCE without some help, not sure where to start. I was using the AWS packer image as an example but got stuck. If I have something to go off of, I can use that and work through it and/or ask questions as needed.
I've started to try to convert the official ansible playbooks used to build the AMI for use on building a GCE image. Hopefully I'll have something working this weekend and can post here for some wider testing. I'm starting from the method here: https://github.com/vyos/build-ami Not the method mentioned earlier in this task using Packer.
I also found this: https://wiki.vyos.net/wiki/Howto_build_VyOS_1.2_ISO_image_and_VMWare_.OVF which describes how to build an OVF image and the introduction says he started down that path with the intention of getting it working on GCE but never mentions if he was successful. GCE allows uploading raw VM disk images, so making an ansible build pipeline may not be strictly necessary but I figured some people would still want to build it themselves to ensure the integrity of the image.
I would definitely be interested in hearing how far anyone else got on this project and if they ran into any unexpected gotchas.