Page MenuHomeVyOS Platform

Provide generic Library for package builds
Closed, ResolvedPublicFEATURE REQUEST

Description

Status quo

Software and also build systems come with iterations, it is time for the next big iteration on our package build system. VyOS makes use of several package repositories to craft the final ISO image. Those repositories usually all have their dedicated Jenkinsfile. To name a few:

The purpose of a Jenkinsfile (also known as Pipeline) is to provide individual steps on how to transfor the source code to a binary package and deploy it to the VyOS repository for installation during the ISO build.

When comparing most of our Jenkinsfiles you can clearly see that those are 1:1 copies in 95% of all cases. Some packages can't be build using the regular dpkg-buildpackage -uc -us -tc -b command and thus only prepend the command by some small BASHish lines.

Maintaining this setup is a major pain in the ass as if the Pipeline requires only a simple change - you need to change all repositories - that's definately not how a software should be maintained.

Library approach

Jenkins comes with support for common and "user defined" library support. The most obvious idea is to start a library which holds all the building blocks we utilize in the Jenkins Pipelines. The pipeline library code is stored in the vyos-build repository under the vars/ folder as this is a Jenkins internal requirement. The pipeline library is thus versioned together with the build system - which is a nice benefit. The library is already configured in on our Jenkins server https://ci.vyos.net (Manage Jenkins -> Configure System)

If a Jenkins Job makes use of the library you can see it in the console output of the individual Job:

Started by user Christian Poessinger
11:42:02 Connecting to https://api.github.com using (GitHub Access Token)
Obtained Jenkinsfile from e90581590b21e30b99891fd2c394df2797716585
Running in Durability level: MAX_SURVIVABILITY
Loading library vyos-build@current
11:42:03 GitHub API Usage: Current quota has 33 remaining (12 under budget). Next quota of 60 in 9 min 27 sec
Examining vyos/vyos-build
Attempting to resolve current as a branch
Resolved current as branch current at revision 055d658a71c89aaa27f8b9bc77d8686410a5725c

Retire individual build repositories

There are currently two individual repositories we maintain to build the Linux Kernel and FRRouting. Those repositories only hold one Jenkinsfile (FRR) or some build scripts (Kernel) and nothing more. As new repositories might be needed in the future (T1538) a common approach should be established.

FRRouting

Decomission the vyos-build-frr repository on GitHub and move the individual Jenkinsfile to the vyos-build repository under packages/frr

  • Branch current
  • Branch crux

Linux Kernel

Decomission the vyos-build-kernel repository on GitHub and move the individual Jenkinsfile to the vyos-build repository under packages/linux-kernel

  • Branch current
  • Branch crux

VyOS

Migrate Pipeline to library based Pipeline (only for packages in current)

  • ddclient
  • hvinfo
  • ipaddrcheck
  • libnss-mapuser
  • libpam-radius-auth
  • libvyosconfig
  • live-boot
  • mdns-repeater
  • udp-broadcast-relay
  • vyatta-bash
  • vyatta-biosdevname
  • vyatta-cfg
  • vyatta-cfg-firewall
  • vyatta-cfg-qos
  • vyatta-cfg-quagga
  • vyatta-cfg-system
  • vyatta-cfg-vpn
  • vyatta-cluster
  • vyatta-config-mgmt
  • vyatta-config-migrate
  • vyatta-conntrack
  • vyatta-conntrack-sync
  • vyatta-iproute
  • vyatta-op
  • vyatta-op-firewall
  • vyatta-op-qos
  • vyatta-op-quagga
  • vyatta-op-vpn
  • vyatta-wanloadbalance
  • vyatta-webproxy
  • vyatta-zone
  • vyos-1x
  • vyos-build
  • vyos-cloud-init
  • vyos-netplug
  • vyos-nhrp
  • vyos-opennhrp
  • vyos-smoketest
  • vyos-strongswan
  • vyos-world
  • vyos-xe-guest-utilities

Details

Difficulty level
Hard (possibly days)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Behavior change

Event Timeline

c-po created this task.Sun, Jun 21, 10:17 AM
c-po changed the task status from Open to In progress.Sun, Jun 21, 10:21 AM
c-po claimed this task.
c-po triaged this task as Wishlist priority.
c-po updated the task description. (Show Details)
c-po changed Difficulty level from Unknown (require assessment) to Hard (possibly days).
c-po changed Is it a breaking change? from Unspecified (possibly destroys the router) to Behavior change.
c-po updated the task description. (Show Details)Sun, Jun 21, 10:24 AM
c-po updated the task description. (Show Details)Sun, Jun 21, 10:43 AM
c-po updated the task description. (Show Details)Sun, Jun 21, 10:45 AM
c-po updated the task description. (Show Details)
c-po updated the task description. (Show Details)Sun, Jun 21, 10:49 AM
c-po changed the task status from In progress to Backport candidate.Sun, Jun 21, 11:33 AM
c-po moved this task from Need Triage to In Progress on the VyOS 1.3 Equuleus board.
c-po moved this task from In Progress to Backport Candidates on the VyOS 1.3 Equuleus board.
c-po added a project: Ready for Crux (1.2.x).
c-po updated the task description. (Show Details)Sun, Jun 21, 1:08 PM
pasik added a subscriber: pasik.Sun, Jun 21, 8:12 PM
c-po updated the task description. (Show Details)Mon, Jun 22, 8:16 PM
c-po updated the task description. (Show Details)Mon, Jun 22, 8:20 PM
c-po updated the task description. (Show Details)Tue, Jun 23, 4:58 PM
c-po updated the task description. (Show Details)Tue, Jun 23, 5:33 PM
c-po updated the task description. (Show Details)Tue, Jun 23, 5:52 PM
c-po updated the task description. (Show Details)Tue, Jun 23, 6:57 PM
c-po updated the task description. (Show Details)Tue, Jun 23, 7:10 PM
c-po updated the task description. (Show Details)
c-po updated the task description. (Show Details)
c-po updated the task description. (Show Details)Wed, Jun 24, 3:36 PM
c-po updated the task description. (Show Details)Wed, Jun 24, 3:49 PM
c-po updated the task description. (Show Details)Wed, Jun 24, 4:09 PM
c-po updated the task description. (Show Details)
c-po updated the task description. (Show Details)Wed, Jun 24, 4:28 PM
c-po updated the task description. (Show Details)Fri, Jun 26, 3:55 PM
c-po closed this task as Resolved.Fri, Jun 26, 4:03 PM
c-po updated the task description. (Show Details)
c-po removed a project: Ready for Crux (1.2.x).
c-po moved this task from Needs Triage to Finished on the VyOS 1.2 Crux (VyOS 1.2.6) board.
c-po moved this task from Backport Candidates to Finished on the VyOS 1.3 Equuleus board.