Commit Graph

1205 Commits

Author SHA1 Message Date
Krzesimir Nowak
5ca9142507 build_library/build_image_util.sh: Use scan instead of packages in syft
The `packages` command is deprecated in favor of `scan`.
2025-04-24 15:26:13 +02:00
James Le Cuirot
2f587621ab
dev-util/checkbashisms: Drop package as qemu script now uses Bash
The check for that script was the only consumer.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2025-04-24 12:23:46 +01:00
Krzesimir Nowak
654f36e75a build_library/qemu_template.sh: Make it use bash
Insisting on just sh is pointless. Use bash, so the checkbashisms
script running somewhere will just shut up.
2025-04-24 10:53:29 +02:00
Krzesimir Nowak
fb2682ddb6
Merge pull request #2846 from flatcar/krnowak/qemu-script
Simplify and improve flatcar_production_qemu*.sh script
2025-04-23 15:15:43 +02:00
Krzesimir Nowak
7fc5ec8b50 build_library/qemu_template.sh: Add support for primary disk options 2025-04-23 14:41:19 +02:00
Krzesimir Nowak
df17f6b672 build_library/qemu_template.sh: Add support for extra disks 2025-04-22 17:19:38 +02:00
Krzesimir Nowak
36ee8ed43a build_library/qemu_template.sh: Simplify qemu execution
The final part of the script differed only the name of the qemu binary
to execute and in network device driver (virtio-net-pci on amd64 vs
virtio-net-device on arm64). virtio-net-pci seems to be working also
on arm64, so simplify the code to avoid repetition.
2025-04-22 17:19:38 +02:00
Krzesimir Nowak
b3dd088469 build_library/qemu_template.sh: Some whitespace fixes 2025-04-22 17:19:38 +02:00
Krzesimir Nowak
2ca63a3a82 build_library/qemu_template.sh: Simplify mounting VM image
There's no need to differentiate between amd64 and arm64 boards here
any more. This also adds bootindex=1 option to the -device flag, so we
can pass more secondary disks without affecting the boot order.
2025-04-22 17:19:38 +02:00
James Le Cuirot
14398067d5
build_library: Change extraction script to xz and rewrite to be simpler
This version writes fewer temporary files and tries cpio multiple times
for concatenated archives again.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2025-04-22 13:14:02 +01:00
James Le Cuirot
3ab16bd843
build_library: Fix path handling in QEMU launcher script
I couldn't take it anymore! The launcher script could not handle paths
outside the script's own directory, and it was driving me crazy. Now
only the default values are relative to the script's directory. Given
paths are relative to the current directory and absolute paths work as
you would expect.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2025-04-07 10:13:09 +01:00
Adrian Vladu
0a8267f69b
cpio: do not fail on ghost rootfs
On Linux >= 6.10, the first rootfs is an extra ghost rootfs of 336K,
that has a corrupted CPIO.

To overcome this issue, do not fail on `cpio --extract`.
2025-03-27 16:14:17 +00:00
Krzesimir Nowak
e8b631fcb9
Merge pull request #2750 from flatcar/buildbot/weekly-portage-stable-package-updates-2025-03-17
Weekly portage-stable package updates 2025-03-17
2025-03-27 07:13:11 +01:00
Krzesimir Nowak
b1220b6e4e build_library/toolchain_util: Fix setting a profile
Setting a profile in a newly created sysroot when building native
toolchains broke after an eselect update. Apparently eselect gets the
path to the coreos-overlay repository and then prefixes it with
ROOT. Since ROOT was set to /build/<arch>-usr, the resulting patch was
wrong. Fix this by telling eselect where to find our make.profile
symlink in new sysroot by setting PORTAGE_CONFIGROOT to
/build/<arch>-usr and where to find our profiles by setting ROOT (and
SYSROOT, because it must match ROOT) to /.
2025-03-21 14:16:01 +01:00
Sayan Chowdhury
7d7bcb02e1
grub_install.sh: copy signed shim to location
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2025-03-21 18:02:38 +05:30
James Le Cuirot
76729bea7f
Revert "prod_image_util.sh: Temporarily nobble removing unsigned kernel and GRUB"
This reverts commit 3f68400908.
2025-03-21 18:02:38 +05:30
Sayan Chowdhury
cdb45e62b3
build_libary/vm_image_util: Add the temporary shim signing cert
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2025-03-21 18:02:37 +05:30
Sayan Chowdhury
450a8983cc
build_library/sbsign_util: Update the SBSIGN_KEY & AZURE_KEYVAULT_URL
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2025-03-21 18:02:30 +05:30
Gabriel Adrian Samfira
fb396e642f
Fix non conforming partition table
This change removes the legacy_boot flag from the EFI system partition.
We already have a BIOS boot partition which should offer compatibility with
legacy bios systems.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2025-02-14 19:24:47 +02:00
Krzesimir Nowak
1e9ed6156c virtual/rust: Drop unused package
Mentions of virtual/rust in some scripts were replaced with
dev-lang/rust-bin. These were usually about skipping the update/build
of the package, and these already contained dev-lang/rust, so added
the -bin variant for completeness.
2025-01-24 11:41:22 +01:00
Jan Bronicki
0f0fa2ff7a fix: Dynamically create netdev arguments to correctly include commas
docs: Add entrance to the changelog about the fix

Update changelog/changes/2025-01-15-qemu-startup-script-comma-fix.md

Co-authored-by: Mathieu Tortuyaux <mathieu.tortuyaux@gmail.com>
2025-01-15 17:57:56 +01:00
Jan Bronicki
1a7b995a08 Add port forwarding as multiple use parameters
Add mention in the changelog

Update changelog/changes/2025-01-10-qemu-startup-script-port-forwarding.md

Co-authored-by: Mathieu Tortuyaux <mathieu.tortuyaux@gmail.com>

Update build_library/qemu_template.sh

Co-authored-by: Mathieu Tortuyaux <mathieu.tortuyaux@gmail.com>

Adjust to review comments

Review adjustments
2025-01-13 15:28:25 +01:00
James Le Cuirot
29a5131380
build_image_util.sh: Don't compress extracted partition unnecessarily
I know I recently deduplicated the code between extract_update and
generate_update recently, but now that generate_update will sometimes be
called at a later time, I've realised that it is compressing and
uploading the partition twice.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-12-23 13:01:36 +00:00
James Le Cuirot
3f68400908
prod_image_util.sh: Temporarily nobble removing unsigned kernel and GRUB
We would normally remove these for an official build so that the signed
versions can be uploaded later. However, we are not doing that signing
until we pass the shim review.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-12-13 10:05:05 +00:00
Krzesimir Nowak
d4db435dd9 build_library/build_image_util: Fix local exporting
Apparently `local -x FOO` does not locally export an already existing
variable, but rather does some whole weird lot of nothing - it shadows
an existing variable with a new unset one, but it won't export it
until it gets assigned.
2024-12-04 17:27:29 +01:00
James Le Cuirot
0059a33aae
Merge pull request #2491 from flatcar/chewi/split-sbsign-job
Do Secure Boot signing for official builds in a separate additional job
2024-12-04 10:39:55 +00:00
James Le Cuirot
e6e3dafa86
grub_install.sh: Temporarily undo copying the officially signed shim
We previously did the AKV signing in the image job but temporarily
nobbled that code path while we completed the shim review.

Now the AKV signing has been split out into a separate job that will
only be invoked once changes to the jenkins-os repo have been merged.
The only thing we now need to nobble here is copying the signed shim. In
the meantime, we copy the unsigned shim instead. Revert this commit once
the shim review is complete.
2024-12-03 16:01:24 +00:00
James Le Cuirot
b3183b42c4
Do SB signing for official builds in a separate additional job
We only want to do the signing in Azure, not the whole image job. This
new job downloads the unsigned image, signs it, and replaces it.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-12-03 16:01:23 +00:00
James Le Cuirot
0fd811f453
Drop more unused "modify_image" code
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-12-03 16:01:20 +00:00
James Le Cuirot
e44b946abf
build_image: Deduplicate --extract_update and --generate_update options
The --extract_update option used to do exactly that, just extract the
USR-A partition for updates and no more. Now it does the same thing as
--generate_update, except it names the file flatcar_test_update.gz
rather than flatcar_production_update.gz. --generate_update is never
actually used because official update payloads are manually generated
with the generate_payload script later on.

Resolve this confusion by deduplicating the common code between them.
Any update payload produced during this stage of the build is only
useful for testing, so change --generate_update to always create
flatcar_test_update.gz. --generate_update now implies --extract_update
and both are enabled by default.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-12-03 16:01:19 +00:00
Krzesimir Nowak
489eb799a8 build_library/build_image_util.sh: Fix copy-pasta
We were supposed to collect allowed users and allowed groups into
separate arrays. Due to the copy-paste mistake, we overwrote allowed
users array with allowed groups while leaving the array for allowed
groups empty, so we ended up passing only allowed groups instead of
both.
2024-11-29 16:16:51 +01:00
James Le Cuirot
9f8eac6540
Drop lots of obsolete release signing and upload code
Most of this hinges on the --upload option being passed, and it never is
any more. Much of it also uses Google Buckets, which we no longer use,
save for some GCE-specific bits.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-26 14:58:24 +00:00
James Le Cuirot
945014691b
grub_install.sh: Compress modules with xz instead of gzip to save space
Giving the --best or -9 option results in a heavier decompression cost
with no gain on such small files.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-18 14:56:20 +00:00
James Le Cuirot
93cbba765d
grub_install.sh: Only install additional GRUB modules for non-SB targets
Secure Boot prevents you from loading additional modules so remove them
to save space. These modules could be useful for debugging with Secure
Boot disabled, but manually copying the modules with debug symbols is
even more useful and not that difficult.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-18 14:55:50 +00:00
James Le Cuirot
d35954ca15
app-crypt/azure-keyvault-pkcs11: Bump to new pre-release for HSM support
This one includes all the renaming for Azure Key Vault.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-18 13:33:12 +00:00
James Le Cuirot
101efbff39
Temporarily undo AKV signing while we complete the shim review
We don't want to be blocked from doing releases in the meantime. Revert
this commit when ready.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-13 12:19:24 +00:00
James Le Cuirot
d8a8704f92
Refactor SB signing code and sign official builds with Azure Key Vault
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-13 12:19:22 +00:00
James Le Cuirot
0eb1d4a287
grub_install.sh: Remove redundant BOARD_GRUB logic
We always use the board's GRUB now.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-13 12:19:21 +00:00
James Le Cuirot
a76d1b3f1b
Drop all instances of ineffective --root-deps=rdeps option
This option became ineffective with EAPI 7 and all trace of older EAPIs
has now gone.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-12 22:08:21 +00:00
James Le Cuirot
5125317506
coreos-base/coreos-sb-keys: Drop unnecessary PK and KEK certificates
These are only needed when you are going to ship DB updates to existing
systems, which we are not going to do. Our EFI variables are only for
testing. End users are expected to use EFI variables provided by their
hosts or hardware vendors. We presumably provided these before because
some PK and KEK does need to be provided, but we can now use the
Microsoft and Red Hat ones provided via Gentoo's edk2 package.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-12 12:02:00 +00:00
James Le Cuirot
c0b58cf56b
Reuse Secure Boot EFI variables image prepared by Gentoo
Rather than starting with a blank image, reuse the image that already
has the Microsoft certificates and the latest DBX revocation list
applied. Gentoo also applies the Red Hat certificates, which we don't
need, but this is okay.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-12 12:01:59 +00:00
James Le Cuirot
e50fe0a7e4
sys-firmware/edk2-aarch64: Drop in favour of edk2-bin
edk2-bin now supports multiple platforms, including QEMU on arm64, so we
no longer need to use Fedora's build. Note that the Secure Boot
implementation is currently insecure as it lacks SMM, which is needed to
protect the EFI variable store.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-12 12:01:58 +00:00
James Le Cuirot
d1ba9b19fb
Switch from raw (.fd) EDK2 firmware images to QCOW2, plus 4MB on amd64
The new arm64 firmware supporting Secure Boot (see next commit) is in
QCOW2 format only, avoiding the extra space taken up by the 64MB
padding. Supporting both raw and QCOW2 images would be messy, so switch
entirely to QCOW2.

Only the 4MB images are in QCOW2 format on amd64, so also switch away
from the 2MB images. 4MB images are now the default for most
distributions as they are needed to apply certain Windows updates.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-12 11:59:40 +00:00
Mathieu Tortuyaux
c80760c999
Merge pull request #1783 from flatcar/kai/proxmox-support
OEM: Provide Proxmox images
2024-11-12 12:52:31 +01:00
Mathieu Tortuyaux
6e674b26d3
oem: provide proxmox images
Co-authored-by: Kai Lüke <pothos@users.noreply.github.com>
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-11-12 12:51:36 +01:00
James Le Cuirot
bcd203ebcb
app-crypt/efitools: Drop in favour of app-emulation/virt-firmware
virt-fw-vars handles X.509 conversion and QCOW2 conversion transparently
and can update all the variables in a single invocation.

Bonus: Asking it to list the variables doesn't cause a segfault due to
the feature not really being implemented. :D

The 00000000-0000-0000-0000-000000000000 owner GUID is what flash-var
used to set, as we didn't specify the -g argument. We don't need to set
a meaningful value as this file is only for testing.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-11-06 12:04:18 +00:00
Mathieu Tortuyaux
0aab7daaf7 build_library: remove two Docker files not used on Flatcar
They were previously in /usr/share/docker/contrib which means they were
deleted at build time and not shipped into Docker sysext.

New ebuild version of Docker now provides those two files as symlinks to
/usr/share/docker/contrib from /usr/bin.

We can't really remove symlinks using find as docker-runc,
docker-containerd, etc. are broken symlinks too during the build phase.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-10-02 16:32:20 +02:00
Jeremi Piotrowski
be652a29d5 build_library/grub.cfg: Enable grub fallback behavior
With secure boot a failed shim signature check will leave us stuck in grub.
Enable automatic fallback in that case.
2024-09-17 11:01:06 +02:00
Krzesimir Nowak
babc0e6e33 build_docker_aci, build_library: Remove obsolete scripts
The build_docker_aci script only supported docker 12.x, which we don't
have since ages, so it's a clear sign of a script being obsolete.
Removing it results in some other scripts in build_library being
unused, so drop them too.
2024-09-02 14:05:56 +02:00
Jeremi Piotrowski
ff6279a6b0 build_library: Don't preserve file ownership when copying sysext files
The docker and containerd copy files from the repository, which are owned by
the sdk user. This ownership leaks into the final image, which means the first
created user could edit systemd files. This is bad.

Modify the cp invocation to copy files without preserving ownership. The
sysext-mangle script is called by build_sysext, which is executed using sudo.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-08-30 09:16:25 +02:00