This change removes the legacy_boot flag from the EFI system partition.
We already have a BIOS boot partition which should offer compatibility with
legacy bios systems.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Mentions of virtual/rust in some scripts were replaced with
dev-lang/rust-bin. These were usually about skipping the update/build
of the package, and these already contained dev-lang/rust, so added
the -bin variant for completeness.
docs: Add entrance to the changelog about the fix
Update changelog/changes/2025-01-15-qemu-startup-script-comma-fix.md
Co-authored-by: Mathieu Tortuyaux <mathieu.tortuyaux@gmail.com>
I know I recently deduplicated the code between extract_update and
generate_update recently, but now that generate_update will sometimes be
called at a later time, I've realised that it is compressing and
uploading the partition twice.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We would normally remove these for an official build so that the signed
versions can be uploaded later. However, we are not doing that signing
until we pass the shim review.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Apparently `local -x FOO` does not locally export an already existing
variable, but rather does some whole weird lot of nothing - it shadows
an existing variable with a new unset one, but it won't export it
until it gets assigned.
We previously did the AKV signing in the image job but temporarily
nobbled that code path while we completed the shim review.
Now the AKV signing has been split out into a separate job that will
only be invoked once changes to the jenkins-os repo have been merged.
The only thing we now need to nobble here is copying the signed shim. In
the meantime, we copy the unsigned shim instead. Revert this commit once
the shim review is complete.
We only want to do the signing in Azure, not the whole image job. This
new job downloads the unsigned image, signs it, and replaces it.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The --extract_update option used to do exactly that, just extract the
USR-A partition for updates and no more. Now it does the same thing as
--generate_update, except it names the file flatcar_test_update.gz
rather than flatcar_production_update.gz. --generate_update is never
actually used because official update payloads are manually generated
with the generate_payload script later on.
Resolve this confusion by deduplicating the common code between them.
Any update payload produced during this stage of the build is only
useful for testing, so change --generate_update to always create
flatcar_test_update.gz. --generate_update now implies --extract_update
and both are enabled by default.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We were supposed to collect allowed users and allowed groups into
separate arrays. Due to the copy-paste mistake, we overwrote allowed
users array with allowed groups while leaving the array for allowed
groups empty, so we ended up passing only allowed groups instead of
both.
Most of this hinges on the --upload option being passed, and it never is
any more. Much of it also uses Google Buckets, which we no longer use,
save for some GCE-specific bits.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Giving the --best or -9 option results in a heavier decompression cost
with no gain on such small files.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Secure Boot prevents you from loading additional modules so remove them
to save space. These modules could be useful for debugging with Secure
Boot disabled, but manually copying the modules with debug symbols is
even more useful and not that difficult.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We don't want to be blocked from doing releases in the meantime. Revert
this commit when ready.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
These are only needed when you are going to ship DB updates to existing
systems, which we are not going to do. Our EFI variables are only for
testing. End users are expected to use EFI variables provided by their
hosts or hardware vendors. We presumably provided these before because
some PK and KEK does need to be provided, but we can now use the
Microsoft and Red Hat ones provided via Gentoo's edk2 package.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Rather than starting with a blank image, reuse the image that already
has the Microsoft certificates and the latest DBX revocation list
applied. Gentoo also applies the Red Hat certificates, which we don't
need, but this is okay.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
edk2-bin now supports multiple platforms, including QEMU on arm64, so we
no longer need to use Fedora's build. Note that the Secure Boot
implementation is currently insecure as it lacks SMM, which is needed to
protect the EFI variable store.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The new arm64 firmware supporting Secure Boot (see next commit) is in
QCOW2 format only, avoiding the extra space taken up by the 64MB
padding. Supporting both raw and QCOW2 images would be messy, so switch
entirely to QCOW2.
Only the 4MB images are in QCOW2 format on amd64, so also switch away
from the 2MB images. 4MB images are now the default for most
distributions as they are needed to apply certain Windows updates.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
virt-fw-vars handles X.509 conversion and QCOW2 conversion transparently
and can update all the variables in a single invocation.
Bonus: Asking it to list the variables doesn't cause a segfault due to
the feature not really being implemented. :D
The 00000000-0000-0000-0000-000000000000 owner GUID is what flash-var
used to set, as we didn't specify the -g argument. We don't need to set
a meaningful value as this file is only for testing.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
They were previously in /usr/share/docker/contrib which means they were
deleted at build time and not shipped into Docker sysext.
New ebuild version of Docker now provides those two files as symlinks to
/usr/share/docker/contrib from /usr/bin.
We can't really remove symlinks using find as docker-runc,
docker-containerd, etc. are broken symlinks too during the build phase.
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
The build_docker_aci script only supported docker 12.x, which we don't
have since ages, so it's a clear sign of a script being obsolete.
Removing it results in some other scripts in build_library being
unused, so drop them too.
The docker and containerd copy files from the repository, which are owned by
the sdk user. This ownership leaks into the final image, which means the first
created user could edit systemd files. This is bad.
Modify the cp invocation to copy files without preserving ownership. The
sysext-mangle script is called by build_sysext, which is executed using sudo.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
build_sysext uses a base squashfs (basically a full snapshot of the
Flatcar OS image) to build custom sysexts on top. Before building it
ensures the base image actualy matches the OS version in the repository
root.
The version string includes a BUILD_ID which might be auto-generated (by
including common.sh) if it is not present in the version file - e.g.
when the version is an official release (tag). This build ID
auto-generation causes issues with the version check when image build
and sysext build scripts run independently - each will generate its own
build ID, and this will cause build_sysext's version check to fail.
build_sysext will now use the build id from the base squashfs when it is
not set in the source tree's version.txt to work around that issue. This
is a more general solution than 361eda220b
(which this patch reverts) as it directly addresses the issue in
build_sysext instead of working around it in sysext_prod_builder.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
It is not clear why this was forked originally. One reason was to avoid
the sys-apps/lsb-release dependency, but it probably wasn't just that.
It seems likely that the upstream package did not support cross targets
at the time. Now it does.
It appears that LTO was previously enabled by us following Gentoo rather
than through an explicit decision. They now disable it by default, so we
do likewise. It previously used "fat" LTO, which makes Rust especially
slow to build and reportedly made rustc slower than with "thin" LTO!
There seems little benefit in using thin LTO given that we rebuild Rust
almost as much as the packages that use it, plus we don't enable LTO
anywhere else.
We still avoid rustdoc to keep the size down using INSTALL_MASK. This
isn't as good as not building it in the first place, but this alone
isn't worth keeping a fork.
Cross targets are now handled via the admittedly experimental
RUST_CROSS_TARGETS support. This has been in place for a while, and I
think it is fairly widely used now. If it does disappear, it would
almost certainly be for something even better.
This also updates Rust from 1.80.0 to 1.80.1.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
It will patch gcc to respect ESYSROOT when cross-compiling, effectively
adding the --sysroot flag without the use of flags or wrappers. This
hasn't been merged into Gentoo yet, but it has been given the nod. When
it does get merged, it was only be for newer gcc versions than we're
currently using, so we'll need this user patch in the meantime
regardless.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
From https://wiki.gentoo.org/wiki/Catalyst/Stage_Creation#Build_Stage3:
> It is not necessary to build stage2 in order to build stage3. Gentoo
> release engineering does not build stage2, and you should not need to
> unless you're intentionally building a stage2 as your goal.
We can now sync portage-stable/scripts with upstream because
bootstrap.sh is only used during stage2, and the changes we had are no
longer relevant. It seems likely the changes were already redundant
anyway.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
These flags normally need to be temporarily forced during stage1, but we
already force them permanently in our profiles.
Removing this appears to make build_library/portage redundant, but it
will later be used to allow building under QEMU with Catalyst, and it
could have other uses too.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The changes to support Catalyst 4 are not backwards compatible and we
need a seamless transition for builds in CI.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This is what upstream Gentoo does. They would previously update the
entire seed, but this took a long time. Our seeds are much bigger, so we
kept repo snapshots to build stage1 against these instead. The new
method of only rebuilding packages with changed sub-slots is a good
compromise and removes the need to write stage1 hooks that selectively
catch the repository up.
This also avoids some conflicts by adding the `--ignore-world` option.
Gentoo seeds have nothing in @world. We have much more, but none of that
is needed for stage1.
This continues to exclude cross-*-cros-linux-gnu/* as that is not needed
for stage1. It now also excludes dev-lang/rust, because it is never a
DEPEND, so it would not break other packages in this way. It may fail to
run due to a sub-slot change in one of its own dependencies, but it is
also unlikely to be needed in stage1 and it is not configured to use the
system LLVM. If needs be, we could improve the behaviour of Portage's
@changed-subslot to respect `--with-bdeps`.
In my testing, it was unable to handle an SDK from 17 months ago, but
one from 7 months ago did work. In practise, we will always use a much
more recent one, which is far more likely to work.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Catalyst 4 has totally changed the way repositories are handled. It only
works when the name of the directory containing the repository matches
the configured name of that repository. This was not the case for us,
with the coreos repository residing in the coreos-overlay directory. We
wanted to move and rename our repositories anyway, but this is a big
change, so we'll do separately. For now, this just renames coreos to
coreos-overlay.
Catalyst 4 also ingests the main repository snapshot as a squashfs
rather than a tarball. It features a utility to generate such a
snapshot, but it doesn't fit Flatcar well, particularly because it
expects each ebuild repository to reside at the top level of its own git
repository. It was very easy to call tar2sqfs manually though.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This change fixes a version mismatch of FLATCAR_BUILD_ID when performing
a dev build of an existing release tag. The build ID is part of the
version string of dev builds, separated by a "+" from the main version.
If common.sh detects a dev build (COREOS_OFFICIAL != 1) and
FLATCAR_BUILD_ID is empty, common.sh will generate a new ID based on a
timestamp.
For official releases, FLATCAR_BUILD_ID is not set in version.txt. A dev
build of a release tag would make common.sh generate a new ID each time
it is sourced by different processes. build_image sources common.sh
first, and writes the resulting version string the OS image's
os-release file. build_sysext runs later and also sources common.sh,
leading its version check to fail as its own VERSION now differs from
the version of the OS image it's supposed to generate sysexts for.
This change reads BUILD_ID from the OS image rootfs in
sysext_prod_builder and exports FLATCAR_BUILD_ID accordingly before
calling build_sysext. Hence FLATCAR_BUILD_ID is not empty, so common.sh
in build_sysext will not re-generate it.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
The cros_workon tool has been replaced with a simpler flatcar_workon
tool based around git-r3.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
