Commit Graph

1140 Commits

Author SHA1 Message Date
James Le Cuirot
6732c23e8b
Merge pull request #2047 from flatcar/chewi/drop-cros-eclasses
Drop cros-debug + cros-workon eclasses, new flatcar_workon tool, fix 9999 Cargo ebuilds
2024-06-21 14:45:15 +01:00
James Le Cuirot
26c2e935c1
Drop cros-workon.eclass and replace with git-r3.eclass
The cros_workon tool has been replaced with a simpler flatcar_workon
tool based around git-r3.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
2024-06-21 10:47:50 +01:00
Adrian Vladu
e7d9faadc8 grub_install: Remove core grub modules from EFI partition for i386-pc
The image also boots on Hyper-V Generation 1 VM (BIOS) if the modules
are removed.

Signed-off-by: Adrian Vladu <avladu@cloudbasesolutions.com>
2024-06-21 11:04:35 +03:00
Jeremi Piotrowski
0624d8cc4a grub_install: Remove core grub modules from EFI partition
Since we build them into the grub executable, they are not needed on
disk. The only case I am unsure of is legacy BIOS boot, so left those
on disk.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Signed-off-by: Adrian Vladu <avladu@cloudbasesolutions.com>
2024-06-21 11:04:35 +03:00
Jeremi Piotrowski
14f1c4f735 grub_install: Remove core.{elf,efi} after copying
We currently carry multiple copies of the same grub core.elf or core.efi
on the boot partition. Save some space by removing duplicates that are
never used at runtime. CPIO build needed to be adapted because it
publishes grub efi files.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-06-21 11:04:35 +03:00
Mathieu Tortuyaux
441e10bec5
oem: provide akamai images
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-06-19 15:37:57 +02:00
Jendrik Weise
64c9ed5820 sysext: Add python sysext
Minimal sysext containing dev-lang/python and dev-python/pip.
Enabled ensurepip for python and added required packages from Gentoo.
2024-05-29 19:18:18 +02:00
Jendrik Weise
cd849d6849 sysext: Add podman sysext
Enabled user session dbus in base image to support podman rootless mode.
Extension images can now be created from multiple packages by seperating
them with a comma. The podman sysext includes app-containers/podman and
net-misc/passt.
It can be enabled by adding podman to /etc/flatcar/enabled-sysext.conf.
Potential TODO: gpgme had to be added as BDEPEND to podman ebuild.
2024-05-03 22:59:36 +09:00
Adrian Vladu
6efe345d89 image: add kubevirt image build
As Ignition supports KubeVirt, add a custom oem for it and also the
required parts to be able to build an image in .qcow2 format that
is already using internal .qcow2 gzip compression.

Fixes: https://github.com/flatcar/Flatcar/issues/1358

Signed-off-by: Adrian Vladu <avladu@cloudbasesolutions.com>
2024-04-30 09:19:16 +03:00
Mathieu Tortuyaux
9f02c8d262
vm_image_util: produce qcow2 images for Scaleway
For importing Scaleway images, extension needs to be '.qcow2'

See: https://www.scaleway.com/en/docs/compute/instances/how-to/snapshot-import-export-feature/
> Make sure that the QCOW / QCOW2 image file you want to import,
> uses the file extension .qcow or .qcow2 to avoid issues while importing the image.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-04-26 09:47:29 +02:00
Mathieu Tortuyaux
5c2a8cf8b9
vm_image_util: add IMAGE_DISK_EXTENSION variable
This variable allows to override the disk extension which is initially
based on the DISK_FORMAT.

Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-04-26 09:47:29 +02:00
Jeremi Piotrowski
f53f1af51e grub_install: Add btrfs to core modules
The OEM partition is on a btrfs file system and grub has to be able to
read grub.cfg from there so it makes sense to include btrfs in core
modules. This avoids all other fs modules from being autoloaded during
boot.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-04-25 12:20:19 +02:00
Jeremi Piotrowski
e78a59cc65 grub_install: Use hd0,gpt1 as a hint when searching for root
In the context of load.cfg in the memdisk, root is set to memdisk, so
passing it as a hint to search is not helpful. While we don't know for
sure whether hd0 is the boot disk, it's a safe hint for most situations.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-04-25 12:20:19 +02:00
Mathieu Tortuyaux
0d4031a6d8
Merge pull request #1880 from apricote/hetzner
OEM: Provide Hetzner Images
2024-04-17 15:17:33 +02:00
Kai Lueke
2e3f175fc1 build_library: Use original qemu image name in qemu script
The move to symlinking to the qemu-uefi image also resulted in the
qemu-uefi image being referenced in the qemu-bios and qemu-uefi-secure
scripts instead of referencing the image symlinks. Same for the VM name
shown in the qemu window title.
When generating the qemu scripts, use the original qemu image name and
VM name.
2024-04-17 12:19:08 +09:00
Julian Tölle
48c81bc978 oem: provide hetzner images 2024-04-16 17:08:04 +02:00
Kai Lueke
dcd7ee6fc4 Silence mksquashfs warnings by excluding btrfs attributes
When mksquashfs encounters btrfs attributes which indicate that the file
is transparently compressed, it can't embed this info in the squashfs
and prints a warning about that.
Silence the warnings by excluding btrfs attributes as done already in
the rest of the code base. This aligns the mksquashfs invocation in this
regard.
2024-04-15 20:15:05 +09:00
Kai Lüke
de3a39b39c
Merge pull request #1861 from flatcar/kai/grub-tpm
build_library/grub.cfg: Enable TPM module by default
2024-04-10 13:20:27 +02:00
Kai Lueke
286a8ca4c1 Skip non-UEFI qemu image symlink/script for arm64
We didn't generate this before and don't need it.
2024-04-10 16:42:36 +09:00
Kai Lueke
834d32efa8 build_library/grub.cfg: Enable TPM module by default
For binding a secret to the OS we need TPM PCRs that measure the kernel
and boot configuration (UEFI). Used for:
https://github.com/flatcar/flatcar-website/pull/317
2024-04-09 22:17:18 +09:00
Kai Lueke
de4eb8f755 Set up symlinks for same image artifacts to remove qemu/qemu_uefi_secure
The qemu and qemu_uefi_secure images have the same contents as the
qemu_uefi image which wastes space on the release server. A similar
case is the PXE vmlinuz which is the same as the regular one, too.

Set up symlinks for same images, and also detect this when compressing
to set up symlinks there as well. To reduce complexity, the qemu and
qemu_uefi_secure images are not supported anymore and the Jenkins or
GitHub CI will skip over them if specified. Users that build their own
images need to adapt, though.
2024-04-09 15:09:29 +02:00
Adrian Vladu
7d4917d67c image_to_vm: add support for hyper-v vhdx format
Add support for Gen 2 Hyper-V VMs.

`./image_to_vm` tool has now a new supported format: `hyperv_vhdx`,
that produces .vhdx dynamic disks.

How to use:

```bash
 ./image_to_vm.sh --from ../build/images/amd64-usr/developer-latest/ --format hyperv_vhdx
```

See: https://github.com/flatcar/Flatcar/issues/1009

Uses PR: https://github.com/flatcar/bootengine/pull/92

Signed-off-by: Adrian Vladu <avladu@cloudbasesolutions.com>
2024-04-09 10:52:06 +03:00
Kai Lueke
5e7b4b6b3d qemu_template.sh: Allow parameters for kernel and initrd
With the PXE script it is easy to boot different versions from one
folder without any copies because the kernel and PXE initrd are always
"fresh".
Instead of only supporting hardcoded file names, support parameters for
the kernel and initrd file to be used.
2024-04-04 16:53:08 +09:00
Kai Lueke
9d3200bc47 build_library/qemu_template.sh: Add notes for swtpm init commands
For the swtpm version in Ubuntu some init command is required first.
2024-04-04 16:53:08 +09:00
Kai Lueke
71866e4824 qemu_template.sh: Allow parameters for VM pflash firmware
The qemu UEFI and regular qemu script only differ by having a default
value for the firmware. If one tries to switch between different
firmwares one normally would modify the script.
Make it easier to switch boot modes and use custom firmwares by
supporting a flag to set the pflash contents.
2024-04-04 12:55:37 +09:00
Kai Lueke
48780dc375 qemu_template.sh: Add support for attaching a software TPM
For testing TPM2-backed rootfs encryption it is handy to have a software
TPM option for the qemu script.
Add a flag for a software TPM with swtpm like kola also does. The user
has to specify a folder for the secret state and this won't be removed
because the same store should be able to be passed when booting the VM
again after shutdown.
2024-04-04 12:55:37 +09:00
Kai Lueke
7379db37e8 vm_image_util.sh: Bump default VM memory to 2 GB
While Flatcar itself runs fine with 1 GB, many workloads do not and
having to debug this is time consuming when one forgets to bump the VM
memory, e.g., in the Qemu script.
Default to 2 GB as known-good setting for things like Kubernetes or
setting up LUKS devices.
2024-04-04 12:55:37 +09:00
Kai Lueke
4d5e46432a qemu_template.sh: Allow parameters for VM image and memory
When testing multiple images one always has to copy them to the
expected file name, and when trying to run two VMs this means one needs
to either use separate directories or modify the qemu script. One also
needs to modify the qemu script to bump the memory for K8s or for LUKS.

Support parameters for the VM image name and the VM memory.
2024-04-04 12:55:37 +09:00
Kai Lueke
1513f3a966 Remove ACI image building bits
The ACI OEM container image was used with rkt for GCE. For long time
this has been migrated to systemd-nspawn and now systemd-sysext.
Remove the unused build library code.
2024-04-03 16:18:56 +09:00
Jeremi Piotrowski
9b50112c5a build_library: Reduce xattr spam when creating cpio
Mksquashfs running against a btrfs filesystem tries to capture btrfs specific
xattrs (btrfs.compression) generating a lot of spam. Remove the spam by
ignoring btrfs xattrs.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-15 09:26:36 +01:00
Jeremi Piotrowski
0eb0bb38e8 build_library: Use lbzip2 to decompress gcc binpkg
Speeds things up a bit.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-14 17:50:21 +00:00
Kai Lüke
e5867917ec
Merge pull request #1742 from flatcar/jepio/zfs-sysext
Build ZFS sysext with each release
2024-03-13 23:24:30 +01:00
Jeremi Piotrowski
61c3af32fa
Merge pull request #1730 from flatcar/jepio/btrfs-opt
Mount btrfs fs with rescue= option
2024-03-12 15:05:08 +01:00
Jeremi Piotrowski
1ec40b2dac build_library: create_prod_sysexts: Move artifacts to BUILD_DIR
Putting things in a sysexts subdir results in the same layout on
bincache and does not follow the expected url schema for fetching the
sysext.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 21:03:16 +01:00
Jeremi Piotrowski
79294342a9 build_library: Place EXTRA_SYSEXTS in a separate file
This allows it to be sourced from other scripts and used e.g. in
generating an image changes summary.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 12:51:09 +00:00
Jeremi Piotrowski
87262e4f91 build_library: Add mangle script for zfs sysext
There are two challenges with the sysext: it needs config files in /etc
and it needs udev rules for mounting during boot to work. The etc files
are placed in the standard flatcar etc overlay path but the overlay is
mounted from the initrd. So instead, we create a tmpfiles.d rule that
symlinks the best important files over. For the udev issue, we create a
drop-in in /etc that ensures udev runs after systemd-sysext.

We also can't rely on systemd presets to work, so instead parse the
preset file and statically create the service dependencies. For the
primary zfs.target we rely on an Upholds entry. Users can still disabled
unwanted services if they want.

We also removed unnecessary files:
- development files
- initramfs related scripts

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 11:57:45 +00:00
Jeremi Piotrowski
5d5dd59102 build_image: Build sysext store after image when requested
This function is meant to prebuild certain sysexts to be released along
with each release. These will not be built into the image, but instead
can be fetched by the user on demand.

The command to build sysexts would be:

  ./build_image prod sysext

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-11 11:57:45 +00:00
Jeremi Piotrowski
18265de9d8 disk_util: Mount btrfs fs with rescue= option
It looks like 'norecovery' is deprecated and has been removed in the v6.8-rc1
kernel. Replace it with 'rescue=nologreplay', which is a replacement
implemented since v5.9. The standalone 'nologreplay' option is also deprecated.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-03-05 17:35:55 +01:00
Mathieu Tortuyaux
1017216cbe
oem: provide scaleway images
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2024-03-05 13:51:07 +01:00
Jeremi Piotrowski
e51a90d8ed build_library: Ignore btrfs xattrs
To prevent mksquashfs from spamming the console about btrfs.compression.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-28 12:25:06 +01:00
Sayan Chowdhury
52ce21a5ab grub_install.sh: ship mokmanager, and rename to grubx64
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2024-02-26 15:46:13 +01:00
Sayan Chowdhury
99bfcf5f32 shim, coreos-sb-keys, grub_install.sh: retab to spaces
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2024-02-26 12:01:10 +01:00
Sayan Chowdhury
04005652dd build_library: Drop redundant config from grub.cfg
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2024-02-26 12:01:10 +01:00
Sayan Chowdhury
c1bdbd9d90 build_image_util: Sign the vmlinuz with the shim key
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2024-02-26 12:01:10 +01:00
Jeremi Piotrowski
6ff9f8b098 Add support for secure boot in qemu_template.sh
We have an existing qemu_uefi_secure format definition, but it is
necessary to update it so that it actually works. Qemu needs to be
passed the correct flags to enable SMM, we need to switch to the Q35
machine, and we need to copy over the secboot variant of the OVMF
firmware.
2024-02-26 12:01:09 +01:00
Sayan Chowdhury
fc28e72322 sys-boot/grub: install file with sbat contents, add --sbat to script
This is just the contents of the section, but the section
itself is written by grub-mkimage. sbat.csv needs to be passed
with --sbat.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2024-02-26 12:01:09 +01:00
Jeremi Piotrowski
64556256db grub_install: switch to BOARD_GRUB by default
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2024-02-26 12:01:09 +01:00
Sayan Chowdhury
0479480ef1 grub_install.sh: Sign the GRUB/MM with the proper keys
Add the linux.mod file back

Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
2024-02-26 12:01:09 +01:00
Jeremi Piotrowski
8019f7fd9f vm_image_util.sh: update path to arm64 UEFI firmware
The arm64 firmware is now called AAVMF with the updated edk2-aarch64
ebuild.
2024-02-26 12:01:09 +01:00
Krzesimir Nowak
b7e13a2160
Merge pull request #1434 from flatcar/krnowak/debug-output
Print debug output on any error
2024-02-20 17:39:18 +01:00