Status quo
Software and also build systems come with iterations, it is time for the next big iteration on our package build system. VyOS makes use of several package repositories to craft the final ISO image. Those repositories usually all have their dedicated Jenkinsfile. To name a few:
- https://github.com/vyos/vyos-1x
- https://github.com/vyos/vyatta-cfg-system
- https://github.com/vyos/vyatta-op
The purpose of a Jenkinsfile (also known as Pipeline) is to provide individual steps on how to transfor the source code to a binary package and deploy it to the VyOS repository for installation during the ISO build.
When comparing most of our Jenkinsfiles you can clearly see that those are 1:1 copies in 95% of all cases. Some packages can't be build using the regular dpkg-buildpackage -uc -us -tc -b command and thus only prepend the command by some small BASHish lines.
Maintaining this setup is a major pain in the ass as if the Pipeline requires only a simple change - you need to change all repositories - that's definately not how a software should be maintained.
Library approach
Jenkins comes with support for common and "user defined" library support. The most obvious idea is to start a library which holds all the building blocks we utilize in the Jenkins Pipelines. The pipeline library code is stored in the vyos-build repository under the vars/ folder as this is a Jenkins internal requirement. The pipeline library is thus versioned together with the build system - which is a nice benefit. The library is already configured in on our Jenkins server https://ci.vyos.net (Manage Jenkins -> Configure System)
If a Jenkins Job makes use of the library you can see it in the console output of the individual Job:
Started by user Christian Poessinger 11:42:02 Connecting to https://api.github.com using (GitHub Access Token) Obtained Jenkinsfile from e90581590b21e30b99891fd2c394df2797716585 Running in Durability level: MAX_SURVIVABILITY Loading library vyos-build@current 11:42:03 GitHub API Usage: Current quota has 33 remaining (12 under budget). Next quota of 60 in 9 min 27 sec Examining vyos/vyos-build Attempting to resolve current as a branch Resolved current as branch current at revision 055d658a71c89aaa27f8b9bc77d8686410a5725c
Retire individual build repositories
There are currently two individual repositories we maintain to build the Linux Kernel and FRRouting. Those repositories only hold one Jenkinsfile (FRR) or some build scripts (Kernel) and nothing more. As new repositories might be needed in the future (T1538) a common approach should be established.
FRRouting
Decomission the vyos-build-frr repository on GitHub and move the individual Jenkinsfile to the vyos-build repository under packages/frr
- Branch current
- Branch crux
Linux Kernel
Decomission the vyos-build-kernel repository on GitHub and move the individual Jenkinsfile to the vyos-build repository under packages/linux-kernel
- Branch current
- Branch crux
VyOS
Migrate Pipeline to library based Pipeline (only for packages in current)
- ddclient
- hvinfo
- ipaddrcheck
- libnss-mapuser
- libpam-radius-auth
- libvyosconfig
- live-boot
- mdns-repeater
- udp-broadcast-relay
- vyatta-bash
- vyatta-biosdevname
- vyatta-cfg
- vyatta-cfg-firewall
- vyatta-cfg-qos
- vyatta-cfg-quagga
- vyatta-cfg-system
- vyatta-cfg-vpn
- vyatta-cluster
- vyatta-config-mgmt
- vyatta-config-migrate
- vyatta-conntrack
- vyatta-conntrack-sync
- vyatta-iproute
- vyatta-op
- vyatta-op-firewall
- vyatta-op-qos
- vyatta-op-quagga
- vyatta-op-vpn
- vyatta-wanloadbalance
- vyatta-webproxy
- vyatta-zone
- vyos-1x
- vyos-build
- vyos-cloud-init
- vyos-netplug
- vyos-nhrp
- vyos-opennhrp
- vyos-smoketest
- vyos-strongswan
- vyos-world
- vyos-xe-guest-utilities