Nonstripped binaries exists in VyOS
Closed, ResolvedPublicBUG
Actions

Assigned To

Authored By

	Apachez
	Sep 16 2023, 11:04 AM

Description

I tested using rmlint -p -T "nonstripped" /path on the extracted filesystem.squashfs (VyOS 1.5-rolling-202309151051) and detected following files who are not stripped:

./chroot/usr/bin/accel-cmd
./chroot/etc/hsflowd/modules/mod_json.so
./chroot/usr/lib/x86_64-linux-gnu/libusdm_drv_s.so
./chroot/usr/sbin/adf_ctl
./chroot/etc/hsflowd/modules/mod_dbus.so
./chroot/usr/sbin/accel-pppd
./chroot/usr/sbin/hsflowd
./chroot/etc/hsflowd/modules/mod_dropmon.so
./chroot/usr/sbin/udp-broadcast-relay
./chroot/usr/lib/openvpn/openvpn-otp.so
./chroot/etc/hsflowd/modules/mod_dnssd.so
./chroot/etc/hsflowd/modules/mod_pcap.so

I assume a quickfix would be to add a hooks/live-script to vyox-build that would do strip --strip-all on above files (along with installing "rmlint" package outside of the chroot)?

Doing so manually gave the following result:

Original-dir:
26728 sep 14 ./squashfs-root_orig/usr/bin/accel-cmd
137840 maj 26 ./squashfs-root_orig/etc/hsflowd/modules/mod_json.so
30008 sep 14 ./squashfs-root_orig/usr/lib/x86_64-linux-gnu/libusdm_drv_s.so
104344 sep 14 ./squashfs-root_orig/usr/sbin/adf_ctl
80072 maj 26 ./squashfs-root_orig/etc/hsflowd/modules/mod_dbus.so
275536 sep 14 ./squashfs-root_orig/usr/sbin/accel-pppd
710984 maj 26 ./squashfs-root_orig/usr/sbin/hsflowd
140784 maj 26 ./squashfs-root_orig/etc/hsflowd/modules/mod_dropmon.so
25576 jul 23 ./squashfs-root_orig/usr/sbin/udp-broadcast-relay
62144 maj 26 ./squashfs-root_orig/usr/lib/openvpn/openvpn-otp.so
57352 maj 26 ./squashfs-root_orig/etc/hsflowd/modules/mod_dnssd.so
70200 maj 26 ./squashfs-root_orig/etc/hsflowd/modules/mod_pcap.so

Stripped-dir:
22968 sep 16 ./squashfs-root_strip/usr/bin/accel-cmd
31128 sep 16 ./squashfs-root_strip/etc/hsflowd/modules/mod_json.so
26000 sep 16 ./squashfs-root_strip/usr/lib/x86_64-linux-gnu/libusdm_drv_s.so
80528 sep 16 ./squashfs-root_strip/usr/sbin/adf_ctl
22992 sep 16 ./squashfs-root_strip/etc/hsflowd/modules/mod_dbus.so
235200 sep 16 ./squashfs-root_strip/usr/sbin/accel-pppd
212424 sep 16 ./squashfs-root_strip/usr/sbin/hsflowd
56800 sep 16 ./squashfs-root_strip/etc/hsflowd/modules/mod_dropmon.so
22896 sep 16 ./squashfs-root_strip/usr/sbin/udp-broadcast-relay
26872 sep 16 ./squashfs-root_strip/usr/lib/openvpn/openvpn-otp.so
14400 sep 16 ./squashfs-root_strip/etc/hsflowd/modules/mod_dnssd.so
18656 sep 16 ./squashfs-root_strip/etc/hsflowd/modules/mod_pcap.so

That is:

Before: 1721568 bytes
After: 770864 bytes
Saved: 950704 bytes

Ref:

https://rmlint.readthedocs.io/en/master/

https://rmlint.readthedocs.io/en/latest/tutorial.html

https://manpages.debian.org/bookworm/rmlint/rmlint.1.en.html

Details

Difficulty level: Unknown (require assessment)
Version: VyOS 1.5-rolling-202309151051
Why the issue appeared?: Will be filled on close
Is it a breaking change?: Unspecified (possibly destroys the router)
Issue type: Performance optimization

Event Timeline

Apachez created this task.Sep 16 2023, 11:04 AM

pasik added a subscriber: pasik.Sep 16 2023, 11:07 AM

Suggestion:

Implement hooks-script for livebuild that recursively go through following directories using "strip --strip-all" (syntax to be verified):

#!/bin/sh

#
# Discard symbols and other data from object files.
#

STRIPCMD="/usr/bin/strip --strip-all"

echo "Strip symbols..."

find /etc/hsflowd/modules/ | xargs ${STRIPCMD}
find /usr/bin/ | xargs ${STRIPCMD}
find /usr/lib/ | xargs ${STRIPCMD}
find /usr/sbin/ | xargs ${STRIPCMD}

Apachez claimed this task.Sep 25 2023, 4:34 PM

Have to add Debian package "binutils" to make "strip" work within the chroot of livebuild.

Turned out to be little of a challenge do "just" strip all binaries (and libraries, modules etc).

So long story short (currently running a smoketest)...

Added "binutils" to the vyos-build/data/live-build-config/package-lists/vyos-utils.list.chroot file.

Added following file to the hooks/live-directory: vyos-build/data/live-build-config/hooks/live/99-strip-symbols.chroot

#!/bin/sh

#
# Discard symbols and other data from object files.
#
# Reference:
# https://www.linuxfromscratch.org/lfs/view/systemd/chapter08/stripping.html
# https://www.debian.org/doc/debian-policy/ch-files.html
#

# Set variables.
STRIPCMD_REGULAR="strip --remove-section=.comment --remove-section=.note --preserve-dates"
STRIPCMD_DEBUG="strip --strip-debug --remove-section=.comment --remove-section=.note --preserve-dates"
STRIPCMD_UNNEEDED="strip --strip-unneeded --remove-section=.comment --remove-section=.note --preserve-dates"
STRIPDIR_REGULAR="
"
STRIPDIR_DEBUG="
/usr/lib/modules
"
STRIPDIR_UNNEEDED="
/etc/hsflowd/modules
/usr/bin
/usr/lib/openvpn
/usr/lib/x86_64-linux-gnu
/usr/lib32
/usr/lib64
/usr/libx32
/usr/sbin
"

# Perform stuff.
echo "Stripping symbols..."

# CMD: strip
for DIR in ${STRIPDIR_REGULAR}; do
  echo "Parse dir (strip): ${DIR}"
  find ${DIR} -type f -exec file {} \; | grep 'not stripped' | cut -d ":" -f 1 | while read FILE; do
    echo "Strip file (strip): ${FILE}"
    ${STRIPCMD_REGULAR} ${FILE}
  done
done

# CMD: strip --strip-debug
for DIR in ${STRIPDIR_DEBUG}; do
  echo "Parse dir (strip-debug): ${DIR}"
  find ${DIR} -type f -exec file {} \; | grep 'not stripped' | cut -d ":" -f 1 | while read FILE; do
    echo "Strip file (strip-debug): ${FILE}"
    ${STRIPCMD_DEBUG} ${FILE}
  done
done

# CMD: strip --strip-unneeded
for DIR in ${STRIPDIR_UNNEEDED}; do
  echo "Parse dir (strip-unneeded: ${DIR}"
  find ${DIR} -type f -exec file {} \; | grep 'not stripped' | cut -d ":" -f 1 | while read FILE; do
    echo "Strip file (strip-unneeded): ${FILE}"
    ${STRIPCMD_UNNEEDED} ${FILE}
  done
done

# Remove binutils package.
apt-get -y purge --autoremove binutils

PR created: https://github.com/vyos/vyos-build/pull/426

Merged, will show up in nightly 2023-10-04.

@xrobau noted that PR426 have an anomaly regarding one of the libraries during the strip-run:

https://github.com/vyos/vyos-rolling-nightly-builds/actions/runs/6399896752/job/17372698450#step:9:7904

Parse dir (strip-unneeded: /usr/lib/x86_64-linux-gnu
Strip file (strip-unneeded): /usr/lib/x86_64-linux-gnu/libsframe.so.0.0.0
/root/99-strip-symbols.chroot: line 53: 52424 Segmentation fault      (core dumped) ${STRIPCMD_UNNEEDED} ${FILE}
Strip file (strip-unneeded): /usr/lib/x86_64-linux-gnu/libusdm_drv_s.so
Parse dir (strip-unneeded: /usr/lib32

Turns out that libsframe.so.0.0.0 is part of the binutils deb-package which later is removed by 99-strip-symbols.chroot script which explains why the file is nowhere to be found after the build completes OR in previous builds (before the merge).

Digging further the file is actually part of "libbinutils" (installed by "binutils" deb package):

https://packages.debian.org/bookworm/amd64/libbinutils/filelist

where the following *.so files can be located:

/usr/lib/x86_64-linux-gnu/libbfd-2.40-system.so
/usr/lib/x86_64-linux-gnu/libopcodes-2.40-system.so
/usr/lib/x86_64-linux-gnu/libsframe.so.0
/usr/lib/x86_64-linux-gnu/libsframe.so.0.0.0

The above is also confirmed within the docker image:

vyos_bld@abd6a012ee45:/vyos$ dpkg-query -L libbinutils | awk -F/  '/x86_64-linux-gnu.+so/ { print $NF }'
libbfd-2.40-system.so
libopcodes-2.40-system.so
libsframe.so.0.0.0
libsframe.so.0

The fix will be to add an exclude list such as:

STRIP_EXCLUDE="
/usr/lib/x86_64-linux-gnu/libbfd-2.40-system.so
/usr/lib/x86_64-linux-gnu/libopcodes-2.40-system.so
/usr/lib/x86_64-linux-gnu/libsframe.so.0
/usr/lib/x86_64-linux-gnu/libsframe.so.0.0.0
"

which the parser of which file to perform strip on will use to NOT perform strip on files part of the STRIP_EXCLUDE even if they are detected as "not stripped".

Will return with a new PR when the above have been coded, tested and verified.

Regarding STRIP_EXCLUDE variable... one idea is to assign it dynamically like so:

STRIP_EXCLUDE=`dpkg-query -L libbinutils | grep '.so'`

PR created: https://github.com/vyos/vyos-build/pull/434

Works as expected:

https://github.com/vyos/vyos-rolling-nightly-builds/actions/runs/6463073617/job/17545691908#step:9:6461

P: Executing hook config/hooks/live/99-strip-symbols.chroot...
Stripping symbols...
Exclude files: /usr/lib/x86_64-linux-gnu/libbfd-2.40-system.so
/usr/lib/x86_64-linux-gnu/libopcodes-2.40-system.so
/usr/lib/x86_64-linux-gnu/libsframe.so.0.0.0
/usr/lib/x86_64-linux-gnu/libsframe.so.0
Parse dir (strip-debug): /usr/lib/modules
Strip file (strip-debug): /usr/lib/modules/6.1.56-amd64-vyos/extra/vlan_mon.ko
...
Strip file (strip-debug): /usr/lib/modules/6.1.56-amd64-vyos/kernel/arch/x86/kernel/cpuid.ko
Parse dir (strip-unneeded: /etc/hsflowd/modules
Strip file (strip-unneeded): /etc/hsflowd/modules/mod_dbus.so
Strip file (strip-unneeded): /etc/hsflowd/modules/mod_pcap.so
Strip file (strip-unneeded): /etc/hsflowd/modules/mod_json.so
Strip file (strip-unneeded): /etc/hsflowd/modules/mod_dnssd.so
Strip file (strip-unneeded): /etc/hsflowd/modules/mod_dropmon.so
Parse dir (strip-unneeded: /usr/bin
Strip file (strip-unneeded): /usr/bin/accel-cmd
Parse dir (strip-unneeded: /usr/lib/openvpn
Strip file (strip-unneeded): /usr/lib/openvpn/openvpn-otp.so
Parse dir (strip-unneeded: /usr/lib/x86_64-linux-gnu
Strip file (strip-unneeded): /usr/lib/x86_64-linux-gnu/libusdm_drv_s.so
Parse dir (strip-unneeded: /usr/lib32
Parse dir (strip-unneeded: /usr/lib64
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libauth_mschap_v2.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libl2tp.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libtriton.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libipv6_dhcp.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libauth_chap_md5.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libipv6_nd.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libippool.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libchap-secrets.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libipv6pool.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/liblog_file.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libipoe.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/liblog_syslog.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libpppd_compat.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libluasupp.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/liblog_tcp.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/liblogwtmp.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libvlan-mon.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libauth_mschap_v1.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libradius.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libshaper.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libconnlimit.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libsstp.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libauth_pap.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libsigchld.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libpppoe.so
Strip file (strip-unneeded): /usr/lib64/accel-ppp/libpptp.so
Parse dir (strip-unneeded: /usr/libx32
Parse dir (strip-unneeded: /usr/sbin
Strip file (strip-unneeded): /usr/sbin/accel-pppd
Strip file (strip-unneeded): /usr/sbin/hsflowd
Strip file (strip-unneeded): /usr/sbin/udp-broadcast-relay
Strip file (strip-unneeded): /usr/sbin/adf_ctl

Verified by downloading VyOS 1.5-rolling-202310100022 from:

https://github.com/vyos/vyos-rolling-nightly-builds/releases

Copied /live/filesystem.squashfs from above ISO-file and extracted it using:

sudo unsquashfs ./filesystem.squashfs

Then running sudo rmlint -p -T "nonstripped" ./squashfs-root/ on the extracted folder, the result:

==> Note: Please use the saved script below for removal, not the above output.
==> In total 47410 files, whereof 0 are duplicates in 0 groups.
==> This equals 0 B of duplicates which could be removed.
==> Scanning took in total 0,480s.

So 0 identified duplicates and 0 files to be stripped.

Task can be set to resolved.

Nonstripped binaries exists in VyOSClosed, ResolvedPublicBUGActions

Description

Details

Event Timeline

Nonstripped binaries exists in VyOS
Closed, ResolvedPublicBUG
Actions