Once we have passed the shim review, we will delay this task until the
kernel has been signed later in the pipeline.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The update payload needs the kernel, which isn't signed during the image
job. Secure Boot is not currently enabled for update tests, but we may
as well do this properly. The production update upload is generated
manually at the end after everything has already been signed.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The --extract_update option used to do exactly that, just extract the
USR-A partition for updates and no more. Now it does the same thing as
--generate_update, except it names the file flatcar_test_update.gz
rather than flatcar_production_update.gz. --generate_update is never
actually used because official update payloads are manually generated
with the generate_payload script later on.
Resolve this confusion by deduplicating the common code between them.
Any update payload produced during this stage of the build is only
useful for testing, so change --generate_update to always create
flatcar_test_update.gz. --generate_update now implies --extract_update
and both are enabled by default.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Most of this hinges on the --upload option being passed, and it never is
any more. Much of it also uses Google Buckets, which we no longer use,
save for some GCE-specific bits.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This allows it to be sourced from other scripts and used e.g. in
generating an image changes summary.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
by skipping the check for an existing image directory if we're not
building an image. This makes './build_image sysext' work.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This function is meant to prebuild certain sysexts to be released along
with each release. These will not be built into the image, but instead
can be fetched by the user on demand.
The command to build sysexts would be:
./build_image prod sysext
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This change removes Flatcar specific builds of docker[-cli], containerd,
runc, and cri-tools and instead switches to upstream Gentoo ebuilds
added to portage-stable.
The change updates docker to 24.0.6.
NOTE that there currently is no upstream ebuild for containerd-1.7.7, so
this change adds that ebuild based on the upstream containerd-1.7.6
ebuild.
Flatcar customisations like systemd units etc. are now applied in the
manglefs script of the respective sysexts, based on file system trees in
coreos-overlay/coreos/sysext/(containerd|docker).
The build_sysext script has been extended by an option to strip all
binaries in a sysext; the option is active by default. This takes care
of removing debug symbols from docker and containerd - which are not
removed by the default Gentoo build. The overall size of both containerd
and docker sysext is reduced by ~50%.
Lastly, the sysext command line syntax of build_image has been extended
to allow specifying multiple packages for a sysext. This was necessary
because docker-cli and docker do not have any runtime relationships and
therefore must both be specified for installation to correctly mirror
Flatcar's own docker packaging.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
This change refactors sysext builds during build_image and generalises
the code (no hard-coded containerd and docker anymore).
A command line option is added to build_image for sysexts to include in
the OS image. It defaults to containerd and docker but may be set to
arbitrary packages. The command line supports simple depenencies, i.e.
the "docker" sysext will re-use package information from the
"containerd" sysext and not include another containerd.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
This change removes torcx libraries, references, and commandline options
from build automation scripts and from build_library/.
Containerd and docker are shipped via sysexts which are included in the
base image.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
This change adds a job for publishing binary packages to the build cache
server to the ci automation.
Also, setup_board is updated to use the buildcache package cache if a
nightly build version is detected.
Signed-off-by: flatcar-ci <infra+ci@flatcar-linux.org>
ci-automation builds on the SDK container and simplifies CI automation
build tasks (SDK bootstrap, SDK container, packages, image, VMs).
See ci-automation/README.md for a brief introduction.
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
The `--jobs` parameter that some scripts defined was not used anywhere
in jenkins or mantle. So the value of the parameter always ended up
being equal to `${NUM_JOBS}` set by `common.sh`. Also, even if the
`--jobs` parameter was used for some script, that script usually
didn't forward the jobs value to other scripts, so the other scripts
ended up using `${NUM_JOBS}` again. Also, the `${FLAGS_jobs}` variable
was used by some functions in the build library, and those functions
were sometimes invoked by scripts that didn't define the
`${FLAGS_jobs}` variable. It is tedious to track which script should
actually define the parameter, and where it should be forwarded.
Just get rid of this half-working pretense. If you want to affect how
many jobs `emerge` uses, export the `NUM_JOBS` environment variable
before calling any script.
For `EMERGE_FLAGS` and `REBUILD_FLAGS` we unconditionally specify the
`--jobs` flag's value to `${NUM_JOBS}` because they are passed to
`emerge`. On the other hand we drop the `--jobs` parameter from the
`UPDATE_ARGS` variable, because this variable passed to `setup_board`
or `update_chroot`, which don't have this flag any more.
Create a tar ball with the contents of the / and /usr partitions
to be used as follows with systemd-nspawn (via machinectl):
machinectl import-tar flatcar-container.tar.gz flatcar-container
machinectl start flatcar-container
machinectl shell flatcar-container
or with docker by converting it to an OCI image:
docker import -c "CMD /bin/bash" flatcar-container.tar.gz flatcar-container
Since the new "prodtar" command relies on the results of the "prod" command,
it bundles it so that "prod prodtar" and "prodtar" is the same.
This moves the default symlinking logic into build image as well.
This assumes that a torcx store is available locally with all images
referenced in the torcx manifest.
This is accomplished with a highly-indented double-for-loop, but I think
it's still decently readable.
This adds the option --torcx_store to specify the path to a
directory containing torcx images to be baked into the OS image. A
blank string can be given instead of a path to restore the previous
behavior and leave an empty vendor store.
The default value is the default path created by build_torcx_store,
which is used when build_packages updates torcx images. This means
that the current pattern "./build_packages && ./build_image prod"
should result in a fully updated OS image with all torcx images
available in the vendor store.
To clean things up and prepare for arrm64 support move
all the enable_rootfs_verification processing into one
location and add some comments.
Signed-off-by: Geoff Levand <geoff@infradead.org>
To make verity work both enable_rootfs_verification and enable_verity
need to be set. Without one verity just gets half enabled. Remove
the enable_verity flag and do the full verity setup when
enable_rootfs_verification is set.
Signed-off-by: Geoff Levand <geoff@infradead.org>
The Xen loader in GRUB never received support for our hacky scheme of
adding the verity hash to the kernel cmdline. Disable till that's fixed.
Partially reverts 2016567 and 533b1b9.
Consolidates two very similar flags into one and fix an issue where
verity could get enabled in the GRUB config when rootfs verification was
turned off (e.g. on arm64 which cannot use verity yet).
PROD_IMAGE is a flag that indicates a production image should be
built, and will be set for dev builds if the user specifies that
both dev and prod images should be built. build_image was
incorrectly using the PROD_IMAGE variable to conditionaly do some
setup depending on the image type.
Add a new variable IMAGE_BUILD_TYPE that can be tested for the type
of image currently being built and replace the PROD_IMAGE usage.
Signed-off-by: Geoff Levand <geoff@infradead.org>
Allows build_image to be used without first running build_packages.
Note: setup_board --force is required before build_packages will work
properly after doing this since baselayout won't be installed otherwise.
This variable was semi-deprecated ages ago so `version.txt` could follow
a similar variable naming pattern to `os-release`. Finally drop usage of
it here in favor of `$COREOS_VERSION`.
The generation of version.txt was the only thing depending on sourcing
the deprecated BUILD, BRANCH, and PATCH values from version.txt which
common.sh no longer does since 0b6acf86. Derive them instead.
- "./build_image prod" already has the ability to specify which package will specify all the packages that should be pulled in and built into an image by specifying a package name using the --base_pkg command line flag. This creates an equivalent option for "./build_image dev" creating a --base_dev_pkg flag that passes a package name into the create_dev_img() function in dev_image_util.sh the same way that --base_pkg is passed into create_prod_image() inside prod_image_util.sh.
This uses our new GRUB2 features to handle GPT priority partition
selection, terminal selection, OEM tweaks, etc. The old SYSLINUX and
PV-GRUB configs are now unused except for maintaining compatibility
with older installs. Of the old configs only the ones that
coreos-postinst copies are needed. The new setup supports using GRUB2
under Xen, giving us automatic fallback support on all of our platforms
for the very first time!
Since grub.cfg is copied into place instead of generated, build_image's
--boot_args option is no longer supported. It could be re-added later
with some sed goo but for now it is easy enough to just edit grub.cfg.
Using parallel_emerge has been disabled by default for all commands
except build_image for quite a while now, build_image kept it just
because it was still a bit faster than normal emerge. Keeping
parallel_emerge complicates future changes to build_image so it needs to
drop it entirely. Since that means nothing uses it by default we might
as well just rip out support for it entirely.
Missed this in 7231b95a, the update zip should still be built when the
usr partition is extracted for generating updates but build_image itself
is not generating and signing the update.
