The growth of binaries over time and the inclusion of new features
filled the available boot partition space, so that the kernel+initrd
almost couldn't fit twice anymore as required for updates. We employed
workarounds such as wrapper scripts for ignition, afterburn and other
binaries so that they are loaded from /usr. However, this was still not
enough and we would have to do the same for (network) kernel modules and
firmware. To avoid making this ever more complex we can use a dedicated
initrd focused on loading the full initrd from /usr and then this full
initrd can use dracut as before and even drop all the workarounds we
accumulated.
Generate a minimal initrd to use instead of the full bootengine initrd.
The bootengine initrd gets stored as squashfs on /usr. The minimal
initrd still includes the early_cpio for amd64 microcode updates.
We have a fixed list of modules or module directories to include, only
focused on loading /usr and any emergency console interaction. This
requires also checking for module dependencies to copy over.
The busybox, veritysetup, and kmod binaries are needed and get their
required libraries resolved and copied over. They are not static and
use shared libraries which should be ok for now. The resulting vmlinuz
file is 27 MB for amd64, down from ~60 MB, so we have enough room to
include more kernel modules and so on for the next years while we also
grow the boot partition and wait for users to redeploy until we can rely
on a larger boot partition and eventually drop the minimal initrd again.
Pulls in https://github.com/flatcar/bootengine/pull/110 for the
minimal initrd script and https://github.com/flatcar/seismograph/pull/12
for making the device mapper discovery for the "rootdev" command more
reliable.
This also requied a backport of a kernel patch from 2017 that exposes
the PARTUUID in the /sys uevent file.
Co-authored-by: James Le Cuirot <jlecuirot@microsoft.com>
Signed-off-by: Kai Lueke <kailuke@microsoft.com>
`show_changes_params` is not available in this lexical scope, we
should have been using `show_changes_params_ref`. This has worked so
far only because all the callers of the functions were passing
`show_changes_params` to be referenced by
`show_changes_params_ref`. Just a lucky happenstance.
Spotted by Chewi.
It's a recent addition - nvidia sysexts are amd64-only for now, so the
EXTRA_SYSEXTS variable got some new fields to indicate that. Make use
of them for filtering which extra sysexts changes to report.
The refactored image changes script will eventually be run for the old
LTS version, so make sure that the script for that channel will get a
last release of old LTS instead of new LTS.
Image changes job needs a list of OEMIDs that are built for a specific
architecture. Similar information already existed in the
coreos-base/common-oem-files ebuild, so factor it out to a separate
file, so the image changes job does not need to source the entire
ebuild (or process it in other way), but rather source the smaller
file.
Instead of depending on default value of build_image's base_sysext
parameter, create a file that explicitly lists which base sysexts will
be built for each architecture. The file can be sourced by other
scripts that need this kind of information. Currently, image.sh and
image_changes.sh use this file.
This is to limit the amount of reports consisting purely of failures,
because some files were missing. And those files will be missing,
because an OEM might not even have any image for certain arches (like
digitalocean has no arm64 images).
It certainly does happen on GitHub Actions that the HEAD commit has no
tag. In such case, fake the tag from version file. The git tag in this
scenario is used to figure out a channel transition.
This avoid assuming that the current working directory in the
generate_image_changes_report function is actually toplevel directory
of the scripts repo.
It was only needed for the show-changes script. Now that show-changes
script allows to set the repos parent directory with an environment
variable, we set the variable instead of changing the working
directory.
There's a bug in show-changes script where it defaults to values with
single quotes in them. So the default scripts directory is not
"scripts" but "'scripts'". This will be fixed in show-scripts, but for
now work it around here by explicitly defining the directories.
First issue is that on Jenkins, the beginning of the output seems to
be eaten, leaving us only the final part of the reports. This looks
like an issue stemming from redirecting stdout to stdout with
">/dev/stdout". Special case the stdout by not redirecting anything in
such case.
Second issue is that errors printed by the tools we use for generating
the reports go to stderr, so they don't show in the report. So
redirect their stderr to stdout, so the possible errors are visible in
the report file too. We do not want to redirect the stderr of the
print_image_reports function, because that would also capture
debugging stuff from "set -x" that GitHub Actions are using.
This uses the new size-change-report.sh script to print out some
information about largest files being added/removed and files with
greatest increase/decrease in file size between two versions of the
image.
This relies on flatcar_production_image_initrd_contents.txt being
uploaded to the server. It also exports the WITHWTD environment
variable with a value 1, which will make the package-diff script to
try out the wtd contents file variant first.
To review the image changes and the changelog more easily and in case
of fixes, iterate over it without rebuilding the image, move this logic
to its own file where a new job could call it.
