I couldn't take it anymore! The launcher script could not handle paths
outside the script's own directory, and it was driving me crazy. Now
only the default values are relative to the script's directory. Given
paths are relative to the current directory and absolute paths work as
you would expect.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
On Linux >= 6.10, the first rootfs is an extra ghost rootfs of 336K,
that has a corrupted CPIO.
To overcome this issue, do not fail on `cpio --extract`.
Setting a profile in a newly created sysroot when building native
toolchains broke after an eselect update. Apparently eselect gets the
path to the coreos-overlay repository and then prefixes it with
ROOT. Since ROOT was set to /build/<arch>-usr, the resulting patch was
wrong. Fix this by telling eselect where to find our make.profile
symlink in new sysroot by setting PORTAGE_CONFIGROOT to
/build/<arch>-usr and where to find our profiles by setting ROOT (and
SYSROOT, because it must match ROOT) to /.
This change removes the legacy_boot flag from the EFI system partition.
We already have a BIOS boot partition which should offer compatibility with
legacy bios systems.
Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
Mentions of virtual/rust in some scripts were replaced with
dev-lang/rust-bin. These were usually about skipping the update/build
of the package, and these already contained dev-lang/rust, so added
the -bin variant for completeness.
docs: Add entrance to the changelog about the fix
Update changelog/changes/2025-01-15-qemu-startup-script-comma-fix.md
Co-authored-by: Mathieu Tortuyaux <mathieu.tortuyaux@gmail.com>
I know I recently deduplicated the code between extract_update and
generate_update recently, but now that generate_update will sometimes be
called at a later time, I've realised that it is compressing and
uploading the partition twice.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We would normally remove these for an official build so that the signed
versions can be uploaded later. However, we are not doing that signing
until we pass the shim review.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Apparently `local -x FOO` does not locally export an already existing
variable, but rather does some whole weird lot of nothing - it shadows
an existing variable with a new unset one, but it won't export it
until it gets assigned.
We previously did the AKV signing in the image job but temporarily
nobbled that code path while we completed the shim review.
Now the AKV signing has been split out into a separate job that will
only be invoked once changes to the jenkins-os repo have been merged.
The only thing we now need to nobble here is copying the signed shim. In
the meantime, we copy the unsigned shim instead. Revert this commit once
the shim review is complete.
We only want to do the signing in Azure, not the whole image job. This
new job downloads the unsigned image, signs it, and replaces it.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The --extract_update option used to do exactly that, just extract the
USR-A partition for updates and no more. Now it does the same thing as
--generate_update, except it names the file flatcar_test_update.gz
rather than flatcar_production_update.gz. --generate_update is never
actually used because official update payloads are manually generated
with the generate_payload script later on.
Resolve this confusion by deduplicating the common code between them.
Any update payload produced during this stage of the build is only
useful for testing, so change --generate_update to always create
flatcar_test_update.gz. --generate_update now implies --extract_update
and both are enabled by default.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We were supposed to collect allowed users and allowed groups into
separate arrays. Due to the copy-paste mistake, we overwrote allowed
users array with allowed groups while leaving the array for allowed
groups empty, so we ended up passing only allowed groups instead of
both.
Most of this hinges on the --upload option being passed, and it never is
any more. Much of it also uses Google Buckets, which we no longer use,
save for some GCE-specific bits.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Giving the --best or -9 option results in a heavier decompression cost
with no gain on such small files.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Secure Boot prevents you from loading additional modules so remove them
to save space. These modules could be useful for debugging with Secure
Boot disabled, but manually copying the modules with debug symbols is
even more useful and not that difficult.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We don't want to be blocked from doing releases in the meantime. Revert
this commit when ready.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
These are only needed when you are going to ship DB updates to existing
systems, which we are not going to do. Our EFI variables are only for
testing. End users are expected to use EFI variables provided by their
hosts or hardware vendors. We presumably provided these before because
some PK and KEK does need to be provided, but we can now use the
Microsoft and Red Hat ones provided via Gentoo's edk2 package.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Rather than starting with a blank image, reuse the image that already
has the Microsoft certificates and the latest DBX revocation list
applied. Gentoo also applies the Red Hat certificates, which we don't
need, but this is okay.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
edk2-bin now supports multiple platforms, including QEMU on arm64, so we
no longer need to use Fedora's build. Note that the Secure Boot
implementation is currently insecure as it lacks SMM, which is needed to
protect the EFI variable store.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The new arm64 firmware supporting Secure Boot (see next commit) is in
QCOW2 format only, avoiding the extra space taken up by the 64MB
padding. Supporting both raw and QCOW2 images would be messy, so switch
entirely to QCOW2.
Only the 4MB images are in QCOW2 format on amd64, so also switch away
from the 2MB images. 4MB images are now the default for most
distributions as they are needed to apply certain Windows updates.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
virt-fw-vars handles X.509 conversion and QCOW2 conversion transparently
and can update all the variables in a single invocation.
Bonus: Asking it to list the variables doesn't cause a segfault due to
the feature not really being implemented. :D
The 00000000-0000-0000-0000-000000000000 owner GUID is what flash-var
used to set, as we didn't specify the -g argument. We don't need to set
a meaningful value as this file is only for testing.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
They were previously in /usr/share/docker/contrib which means they were
deleted at build time and not shipped into Docker sysext.
New ebuild version of Docker now provides those two files as symlinks to
/usr/share/docker/contrib from /usr/bin.
We can't really remove symlinks using find as docker-runc,
docker-containerd, etc. are broken symlinks too during the build phase.
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
The build_docker_aci script only supported docker 12.x, which we don't
have since ages, so it's a clear sign of a script being obsolete.
Removing it results in some other scripts in build_library being
unused, so drop them too.
The docker and containerd copy files from the repository, which are owned by
the sdk user. This ownership leaks into the final image, which means the first
created user could edit systemd files. This is bad.
Modify the cp invocation to copy files without preserving ownership. The
sysext-mangle script is called by build_sysext, which is executed using sudo.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
build_sysext uses a base squashfs (basically a full snapshot of the
Flatcar OS image) to build custom sysexts on top. Before building it
ensures the base image actualy matches the OS version in the repository
root.
The version string includes a BUILD_ID which might be auto-generated (by
including common.sh) if it is not present in the version file - e.g.
when the version is an official release (tag). This build ID
auto-generation causes issues with the version check when image build
and sysext build scripts run independently - each will generate its own
build ID, and this will cause build_sysext's version check to fail.
build_sysext will now use the build id from the base squashfs when it is
not set in the source tree's version.txt to work around that issue. This
is a more general solution than 361eda220b
(which this patch reverts) as it directly addresses the issue in
build_sysext instead of working around it in sysext_prod_builder.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
It is not clear why this was forked originally. One reason was to avoid
the sys-apps/lsb-release dependency, but it probably wasn't just that.
It seems likely that the upstream package did not support cross targets
at the time. Now it does.
It appears that LTO was previously enabled by us following Gentoo rather
than through an explicit decision. They now disable it by default, so we
do likewise. It previously used "fat" LTO, which makes Rust especially
slow to build and reportedly made rustc slower than with "thin" LTO!
There seems little benefit in using thin LTO given that we rebuild Rust
almost as much as the packages that use it, plus we don't enable LTO
anywhere else.
We still avoid rustdoc to keep the size down using INSTALL_MASK. This
isn't as good as not building it in the first place, but this alone
isn't worth keeping a fork.
Cross targets are now handled via the admittedly experimental
RUST_CROSS_TARGETS support. This has been in place for a while, and I
think it is fairly widely used now. If it does disappear, it would
almost certainly be for something even better.
This also updates Rust from 1.80.0 to 1.80.1.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
It will patch gcc to respect ESYSROOT when cross-compiling, effectively
adding the --sysroot flag without the use of flags or wrappers. This
hasn't been merged into Gentoo yet, but it has been given the nod. When
it does get merged, it was only be for newer gcc versions than we're
currently using, so we'll need this user patch in the meantime
regardless.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
From https://wiki.gentoo.org/wiki/Catalyst/Stage_Creation#Build_Stage3:
> It is not necessary to build stage2 in order to build stage3. Gentoo
> release engineering does not build stage2, and you should not need to
> unless you're intentionally building a stage2 as your goal.
We can now sync portage-stable/scripts with upstream because
bootstrap.sh is only used during stage2, and the changes we had are no
longer relevant. It seems likely the changes were already redundant
anyway.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
These flags normally need to be temporarily forced during stage1, but we
already force them permanently in our profiles.
Removing this appears to make build_library/portage redundant, but it
will later be used to allow building under QEMU with Catalyst, and it
could have other uses too.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
