Without this, official builds are failing:
```
This is a dev rebuild of an official release tag: No BUILD ID set in '/mnt/host/source/.repo/manifests/version.txt'. Will use base squasfs BUILD ID for version check.
Repo root FLATCAR_VERSION is '4081.0.0', squashfs build ID is '2024-09-03-2245'
Setting FLATCAR_VERSION to '4081.0.0+2024-09-03-2245'
Base squashfs version: 4081.0.0
SDK board packages version: 4081.0.0+2024-09-03-2245
Version mismatch between board flatcar release and SDK container flatcar release.
```
The "base squashfs version" is the actual version for official release builds.
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
build_sysext uses a base squashfs (basically a full snapshot of the
Flatcar OS image) to build custom sysexts on top. Before building it
ensures the base image actualy matches the OS version in the repository
root.
The version string includes a BUILD_ID which might be auto-generated (by
including common.sh) if it is not present in the version file - e.g.
when the version is an official release (tag). This build ID
auto-generation causes issues with the version check when image build
and sysext build scripts run independently - each will generate its own
build ID, and this will cause build_sysext's version check to fail.
build_sysext will now use the build id from the base squashfs when it is
not set in the source tree's version.txt to work around that issue. This
is a more general solution than 361eda220b
(which this patch reverts) as it directly addresses the issue in
build_sysext instead of working around it in sysext_prod_builder.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
When mksquashfs encounters btrfs attributes which indicate that the file
is transparently compressed, it can't embed this info in the squashfs
and prints a warning about that.
Silence the warnings by excluding btrfs attributes as done already in
the rest of the code base. This aligns the mksquashfs invocation in this
regard.
The Flatcar extension images built with build_sysext created directories
in the overlay in a way that masked contents from other layers.
Instead of fixing the way we create directories, make use of
postprocessing to avoid any similar problems show up again in the
future.
This change removes Flatcar specific builds of docker[-cli], containerd,
runc, and cri-tools and instead switches to upstream Gentoo ebuilds
added to portage-stable.
The change updates docker to 24.0.6.
NOTE that there currently is no upstream ebuild for containerd-1.7.7, so
this change adds that ebuild based on the upstream containerd-1.7.6
ebuild.
Flatcar customisations like systemd units etc. are now applied in the
manglefs script of the respective sysexts, based on file system trees in
coreos-overlay/coreos/sysext/(containerd|docker).
The build_sysext script has been extended by an option to strip all
binaries in a sysext; the option is active by default. This takes care
of removing debug symbols from docker and containerd - which are not
removed by the default Gentoo build. The overall size of both containerd
and docker sysext is reduced by ~50%.
Lastly, the sysext command line syntax of build_image has been extended
to allow specifying multiple packages for a sysext. This was necessary
because docker-cli and docker do not have any runtime relationships and
therefore must both be specified for installation to correctly mirror
Flatcar's own docker packaging.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
- updated github actions for runc, containerd, and docker to not handle
nonexistent ebuilds in app-torcx/ anymore
- removed spurious package_run_dependencies from build_image_util.sh
- build_sysext: generate pkginfo before mangle script runs
use zstd for compression; add cli flag to select compression
- ci_automation_common.sh: remove spurious `/` from match string
- coreos, board-packages, bootengine: bump ebuild revisions
- kernel commonconfig: add squashfs zstd support
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
This change refactors base OS sysext builds to use a separate build
script `build_library/sysext_prod_builder`, which is called from
`build_library/prod_image_util.sh` when `build_image` runs.
This allows for better separation of cleanup traps: prod image sysext
builds need its own trap / cleanup function for temporary build
directories and loopback mounts.
Prod sysext builds properly generate lincense and SBOM information, and
provide detailed file listings and disk space usage stats.
- SBOM / licenses JSON now include all packages of the
final image, i.e. a combined list of base image and all base OS
sysexts.
- Packages lists, files list and detailed files list include the sysext
squashfs files for the base image, and separate sections with files /
packages lists for each sysext.
- Disk usage contains both final disk image usage as well as usage of
each individual sysext squashfs.
This change adds 2 optional command line parameters to build_sysext to
handle dependencies in stacked sysexts. The command line parameters
allow exporting portage package db information into a separate squashfs
image as well as using package db information exported by a previous
sysext build.
--generate_pkginfo will generate a separate squashfs
<sysextname>_pkginfo.raw which contains the sysext's /var/db/pkgs.
--base_pkginfo=<pkginfo>[:<pkginfo>[:...]] will use the supplied paths
to pkginfo squashfses as additional lowerdirs when creating a sysext.
Useage example:
./build_sysext --generate_pkginfo containerd app-containers/containerd
./build_sysext --base_pkginfo=containerd_pkginfo.raw docker app-containers/docker
will create a containerd.raw sysext and a dependent docker.raw sysext
that does not have the containerd dependency installed. Both sysexts
must be merged together in order for docker to work.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
This change improves build_sysext by sourcing a missing lib dependency,
adding a number of comfort / quality-of-life options, and updating the
output of '--help' accordingly.
The OEM sysext finction in build_library/vm_image_util.sh is also
updated to use new command line format.
1. Include missing dependency toolchain_util.sh to fix an error in
board_options.sh (get_board_arch undefined).
2. Use positional parameters for mandatory arguments.
build_dir and sysext_name are mandatory and are now positional
arguments instead of options.
binary_package is the third positional argument but can be omitted
if --metapkgs was specified.
3. --squashfs_base is now guessed better and will use the most recent
build by default.
4. A new boolean flag --ignore_version_mismatch for the more daring
developer was added. The flag will cause the script to continue if a
version mismatch between SDK board packages and squashfs base is
detected.
5. Error messages were improved for when mandatory parameters were not
provided.
6. The '--help' message was improved and adjusted to the new parameters.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
Included a script to enable generating systemd-sysexts. Successfully
tested sysext generation with a fresh Flatcar image (e.g., Python and
Neofetch system extension). Part of my internship work.
The current OS images we provide are not OK as base for flatcar specific
sysext images: it lacks the package metadata and portage configuration,
in order to keep end user OS image clean. This script retains this
information and allows you to produce systemd-sysexts to extend the
system. This script can be used to build a Flatcar sysext image.
Recommended to run from image build folder.
Signed-off-by: Krish Jain <kjain7@u.rochester.edu>
