Page MenuHomeVyOS Platform

Update availability check
Closed, ResolvedPublicFEATURE REQUEST

Description

The concept of a "universal image" is clearly infeasible. The number of image flavors is growing, and we cannot stop its growth for multiple reasons:

  • Cloud-init and vendor-specific tools: indispensable in the cloud environment they are meant for, useless and even dangerous outside of a trusted environment since they don't normally try to authenticate the server and can easily become high-risk attack vectors if an attacker can mimic a server response.
  • Platform-specific drivers: required for a range of devices, useless for anyone else.
  • Platform-specific NIC mapping link files: can be a huge quality of life improvement for users of certain hardware, will make NIC layout confusing outside of a specific model range.

All in all, different flavors do more good than harm. However, as the number of flavors grows, it becomes harder for users to pick correct images. We already see people getting confused. Ideally, we'd like to limit confusion to the initial installation phase, updates should be straightforward.

The idea to make an update check script has been around for a while, but now may be the right time to do it because: a) the issue on hand is getting more acute b) we have most of the infrastructure for that already.

By the infrastructure, I mean the S3 storage and the scripts to work with it. We can easily adapt our image directory listing scripts (those we use for generating the web pages for nightly builds and snapshots) to produce machine-readable versions of those listings. Then we can add a command like show system updates to retrieve that file and display available updates.

I propose the following:

  1. We should codify a system of flavors and allow multiple hyphen-separated flavors, e.g. vyos-1.2.100-amd64-azure.iso would produce a ["amd64", "azure"] flavor list.
  1. Use the following listing format:
{"latest-version": "1.3.0",
  "images": [
    {"url": "https://<some path>/vyos-1.3.0-amd64-vmware.ova", "flavors": ["amd64", "vmware"]},
    ...
  ]
}
  1. Add the following CLI to the images:
system {
  update-check {
    server <hostname>
    user <username>
    password <password>
    auto-check # not present by default
  }
}

The server option will default to the community update server in rolling release and snapshots. In LTS releases, it will default to another server that will require subscriber credentials to receive a list of updates from. When accessed without authentication, it will show the latest available version and some info regarding subscriptions.

Communication with the update server will occur:

  • When a user runs run show system update
  • When the auto-check option is true, it will also query the server periodically

The auto-check option will be disabled by default. The installer will suggest enabling it. The script will not transmit any information that could identify the sender.

Details

Difficulty level
Unknown (require assessment)
Version
-
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Perfectly compatible

Event Timeline

syncer changed the subtype of this task from "Task" to "Feature Request".

Maybe I'm wrong, I see it as some small API (on some hosts) without links to the images but with information about images (in JSON).
We compare our local VyOS version and the version that we get from API, if diff => true send a message to the "wall"

An example of dict that we can use

{
    "images": {
        "rolling": {
            "latest": {
                "arch": "amd64",
                "flavors": ["azure"],
                "image": "vyos-rolling-latest.iso",
                "latest": True,
                "lts": False,
                "release_date": "2022-09-02",
                "release_train": "sagitta",
                "version": "1.4-rolling-202209020217"
            },
            "1.4-rolling-202209020217": {
                "arch": "amd64",
                "flavors": ["generic"],
                "image": "vyos-1.4-rolling-202209020217-amd64.iso",
                "latest": True,
                "lts": False,
                "release_date": "2022-09-02",
                "release_train": "sagitta",
                "version": "1.4-rolling-202209020217"
            },
            "1.4-rolling-202208291850": {
                "arch": "amd64",
                "flavors": ["openstack"],
                "image": "vyos-1.4-rolling-202208291850-amd64.iso",
                "latest": False,
                "lts": False,
                "release_date": "2022-08-29",
                "release_train": "sagitta",
                "version": "1.4-rolling-20220829850"
            }
        },
        "lts": {
            "latest": {
                "arch": "amd64",
                "flavors": ["generic"],
                "image": "vyos-1.3-x.iso",
                "latest": True,
                "lts": True,
                "release_date": "2022-xx-xx",
                "release_train": "equuleus",
                "version": "1.3-stable-202208230511"
            }
        }
    }
}

What diff we can compare images.lts.latest.(image|release_date|version) and so on

PR https://github.com/vyos/vyos-1x/pull/1521

set system update-check auto-check
set system update-check url 'http://192.168.122.14:8080/download/image-version.json'

vyos@r14# commit
                                                                               
Update available: 1.4-rolling-202209050744 
Update URL: http://s3-us.vyos.io/rolling/current/vyos-rolling-latest.iso
                                                                               
[edit]
vyos@r14#

Very nice. I noticed show system updates and set system update-check in the CLI now. Will there be a URL available to start testing this, perhaps even via the most recent nightly rolling release?

Example of usage for 1.5

vyos@r4:~$ 
vyos@r4:~$ show conf com | match upd
set system update-check auto-check
set system update-check url 'https://raw.githubusercontent.com/vyos/vyos-rolling-nightly-builds/main/version.json'
vyos@r4:~$ 
vyos@r4:~$ 
vyos@r4:~$ show system updates 
Current version: 1.5-rolling-202312220023

Update available: 1.5-rolling-202312260023
Update URL: https://github.com/vyos/vyos-rolling-nightly-builds/releases/download/1.5-rolling-202312260023/1.5-rolling-202312260023-amd64.iso
vyos@r4:~$

This is exciting! Is there a public endpoint available for testing this yet? GitHub is rejecting requests on that URL:

Connecting error: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /vyos/vyos-rolling-nightly-builds/main/version.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f916bdb2f90>: Failed to establish a new connection: [Errno 16] Device or resource busy'))
Update not found

This is exciting! Is there a public endpoint available for testing this yet? GitHub is rejecting requests on that URL:

Connecting error: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /vyos/vyos-rolling-nightly-builds/main/version.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f916bdb2f90>: Failed to establish a new connection: [Errno 16] Device or resource busy'))
Update not found

Check that you have connectivity to this host.
All works fine with example above.

My bad, I had misconfigured DNS on this test VM. It's now working as expected. Will there be a way to wrap this in an add system image command?