Page MenuHomeVyOS Platform

Configuration commit fails on kernel 5.15 (and likely others) in libboost's filesystem::copy_file from EXEDEV on the FUSE UnionFS mounts
Resolved (N/A)Public

Description

While testing upgrade to 5.15 for the hardened patchset PR, ran into a somewhat nuanced problem with how configuration copies are made during commit. Test device boots with an apparently configured state, but configure followed by show yields an empty config. After loading the default config and trying to commit that, i found this gem in the vyatta-cfg logs:

cp w->tw failed[boost::filesystem::copy_file: Invalid cross-device link: "/opt/vyatta/config/tmp/new_config_1607/interfaces/ethernet/eth0/hw-id/node.val", "/opt/vyatta/config/tmp/tmp_1607/work/interfaces/ethernet/eth0/hw-id/node.val"]

Which seems related to this fix in libboost itself: https://github.com/boostorg/filesystem/commit/4b9052f1e0b2acf625e8247582f44acdcc78a4ce

Given that Boost's API is "somewhat fragile," its probably safer to drop-down to stdapi for file operations so that we know exactly what is happening and aren't at the mercy of Boost's innards determining whether to rename(), copy ranges, or who knows what else it will want to do under the skin someday. In the very least, it probably makes sense to catch the EXEDEV exception and use a native primitive to complete the op it tried to execute (boost::filesystem::file_copy())

I can verify that nothing other than the kernel version have changed, including sysctls, mount options, packages, etc - same build for both images a few minutes apart with 5.10 working fine and 5.15 refusing to commit.

Details

Difficulty level
Unknown (require assessment)
Version
1.4
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Stricter validation
Issue type
Bug (incorrect behavior)

Event Timeline

Created a pull request implementing a rudimentary fall-through-on-exception to standard API from the Boost version @ https://github.com/vyos/vyatta-cfg/pull/49
Have not built it yet, nor am i a formal C++ developer (hackers are informal everything developers and rarely formal anything developers), so would appreciate eyes on and sanity checks.
Exception can probably be scoped better to only trip on EXEDEV but i dont see a logical problem with falling-through like this on other errors (is this a bad assumption?).

Using the pull requests filesystem copy, same place, new error:

vyos@vyos-515:~$ cat /var/log/vyatta/*log
cp[/opt/vyatta/config/tmp/new_config_1615]->[/opt/vyatta/config/tmp/tmp_1615/work]
cp w->tw failed[unknown exception]
cp[/opt/vyatta/config/tmp/new_config_2665]->[/opt/vyatta/config/tmp/tmp_2665/work]
cp w->tw failed[unknown exception]
vyos@vyos-515:~$ dpkg -l|grep vyatta-cfg
ii  libvyatta-cfg-dev                    0.102.0+vyos2+current5              amd64        libvyatta-cfg development package
ii  libvyatta-cfg1                       0.102.0+vyos2+current5              amd64        vyatta-cfg back-end library
ii  libvyatta-cfg1-dbgsym                0.102.0+vyos2+current5              amd64        debug symbols for libvyatta-cfg1
ii  vyatta-cfg                           0.102.0+vyos2+current5              amd64        VyOS configuration system
ii  vyatta-cfg-dbgsym                    0.102.0+vyos2+current5              amd64        debug symbols for vyatta-cfg
ii  vyatta-cfg-qos                       0.15.42+vyos2+current1              all          VyOS Qos configuration templates/scripts
ii  vyatta-cfg-system                    0.20.44+vyos2+current22             amd64        VyOS system-level configuration
vyos@vyos-515:~$ uname -r
5.15.59-amd64-vyos-sv
vyos@vyos-515:~$

If Linux maintainers backport the delta causing this to 5.10, it could become a rather pressing concern, but for now merely a show-stopper in terms of moving past 5.10LTS.

We have a new kernel