The changes to support Catalyst 4 are not backwards compatible and we
need a seamless transition for builds in CI.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This is what upstream Gentoo does. They would previously update the
entire seed, but this took a long time. Our seeds are much bigger, so we
kept repo snapshots to build stage1 against these instead. The new
method of only rebuilding packages with changed sub-slots is a good
compromise and removes the need to write stage1 hooks that selectively
catch the repository up.
This also avoids some conflicts by adding the `--ignore-world` option.
Gentoo seeds have nothing in @world. We have much more, but none of that
is needed for stage1.
This continues to exclude cross-*-cros-linux-gnu/* as that is not needed
for stage1. It now also excludes dev-lang/rust, because it is never a
DEPEND, so it would not break other packages in this way. It may fail to
run due to a sub-slot change in one of its own dependencies, but it is
also unlikely to be needed in stage1 and it is not configured to use the
system LLVM. If needs be, we could improve the behaviour of Portage's
@changed-subslot to respect `--with-bdeps`.
In my testing, it was unable to handle an SDK from 17 months ago, but
one from 7 months ago did work. In practise, we will always use a much
more recent one, which is far more likely to work.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Catalyst 4 has totally changed the way repositories are handled. It only
works when the name of the directory containing the repository matches
the configured name of that repository. This was not the case for us,
with the coreos repository residing in the coreos-overlay directory. We
wanted to move and rename our repositories anyway, but this is a big
change, so we'll do separately. For now, this just renames coreos to
coreos-overlay.
Catalyst 4 also ingests the main repository snapshot as a squashfs
rather than a tarball. It features a utility to generate such a
snapshot, but it doesn't fit Flatcar well, particularly because it
expects each ebuild repository to reside at the top level of its own git
repository. It was very easy to call tar2sqfs manually though.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This change fixes a version mismatch of FLATCAR_BUILD_ID when performing
a dev build of an existing release tag. The build ID is part of the
version string of dev builds, separated by a "+" from the main version.
If common.sh detects a dev build (COREOS_OFFICIAL != 1) and
FLATCAR_BUILD_ID is empty, common.sh will generate a new ID based on a
timestamp.
For official releases, FLATCAR_BUILD_ID is not set in version.txt. A dev
build of a release tag would make common.sh generate a new ID each time
it is sourced by different processes. build_image sources common.sh
first, and writes the resulting version string the OS image's
os-release file. build_sysext runs later and also sources common.sh,
leading its version check to fail as its own VERSION now differs from
the version of the OS image it's supposed to generate sysexts for.
This change reads BUILD_ID from the OS image rootfs in
sysext_prod_builder and exports FLATCAR_BUILD_ID accordingly before
calling build_sysext. Hence FLATCAR_BUILD_ID is not empty, so common.sh
in build_sysext will not re-generate it.
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
The cros_workon tool has been replaced with a simpler flatcar_workon
tool based around git-r3.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Since we build them into the grub executable, they are not needed on
disk. The only case I am unsure of is legacy BIOS boot, so left those
on disk.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Signed-off-by: Adrian Vladu <avladu@cloudbasesolutions.com>
We currently carry multiple copies of the same grub core.elf or core.efi
on the boot partition. Save some space by removing duplicates that are
never used at runtime. CPIO build needed to be adapted because it
publishes grub efi files.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Enabled user session dbus in base image to support podman rootless mode.
Extension images can now be created from multiple packages by seperating
them with a comma. The podman sysext includes app-containers/podman and
net-misc/passt.
It can be enabled by adding podman to /etc/flatcar/enabled-sysext.conf.
Potential TODO: gpgme had to be added as BDEPEND to podman ebuild.
As Ignition supports KubeVirt, add a custom oem for it and also the
required parts to be able to build an image in .qcow2 format that
is already using internal .qcow2 gzip compression.
Fixes: https://github.com/flatcar/Flatcar/issues/1358
Signed-off-by: Adrian Vladu <avladu@cloudbasesolutions.com>
For importing Scaleway images, extension needs to be '.qcow2'
See: https://www.scaleway.com/en/docs/compute/instances/how-to/snapshot-import-export-feature/
> Make sure that the QCOW / QCOW2 image file you want to import,
> uses the file extension .qcow or .qcow2 to avoid issues while importing the image.
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
This variable allows to override the disk extension which is initially
based on the DISK_FORMAT.
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
The OEM partition is on a btrfs file system and grub has to be able to
read grub.cfg from there so it makes sense to include btrfs in core
modules. This avoids all other fs modules from being autoloaded during
boot.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
In the context of load.cfg in the memdisk, root is set to memdisk, so
passing it as a hint to search is not helpful. While we don't know for
sure whether hd0 is the boot disk, it's a safe hint for most situations.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The move to symlinking to the qemu-uefi image also resulted in the
qemu-uefi image being referenced in the qemu-bios and qemu-uefi-secure
scripts instead of referencing the image symlinks. Same for the VM name
shown in the qemu window title.
When generating the qemu scripts, use the original qemu image name and
VM name.
When mksquashfs encounters btrfs attributes which indicate that the file
is transparently compressed, it can't embed this info in the squashfs
and prints a warning about that.
Silence the warnings by excluding btrfs attributes as done already in
the rest of the code base. This aligns the mksquashfs invocation in this
regard.
The qemu and qemu_uefi_secure images have the same contents as the
qemu_uefi image which wastes space on the release server. A similar
case is the PXE vmlinuz which is the same as the regular one, too.
Set up symlinks for same images, and also detect this when compressing
to set up symlinks there as well. To reduce complexity, the qemu and
qemu_uefi_secure images are not supported anymore and the Jenkins or
GitHub CI will skip over them if specified. Users that build their own
images need to adapt, though.
Add support for Gen 2 Hyper-V VMs.
`./image_to_vm` tool has now a new supported format: `hyperv_vhdx`,
that produces .vhdx dynamic disks.
How to use:
```bash
./image_to_vm.sh --from ../build/images/amd64-usr/developer-latest/ --format hyperv_vhdx
```
See: https://github.com/flatcar/Flatcar/issues/1009
Uses PR: https://github.com/flatcar/bootengine/pull/92
Signed-off-by: Adrian Vladu <avladu@cloudbasesolutions.com>
With the PXE script it is easy to boot different versions from one
folder without any copies because the kernel and PXE initrd are always
"fresh".
Instead of only supporting hardcoded file names, support parameters for
the kernel and initrd file to be used.
The qemu UEFI and regular qemu script only differ by having a default
value for the firmware. If one tries to switch between different
firmwares one normally would modify the script.
Make it easier to switch boot modes and use custom firmwares by
supporting a flag to set the pflash contents.
For testing TPM2-backed rootfs encryption it is handy to have a software
TPM option for the qemu script.
Add a flag for a software TPM with swtpm like kola also does. The user
has to specify a folder for the secret state and this won't be removed
because the same store should be able to be passed when booting the VM
again after shutdown.
While Flatcar itself runs fine with 1 GB, many workloads do not and
having to debug this is time consuming when one forgets to bump the VM
memory, e.g., in the Qemu script.
Default to 2 GB as known-good setting for things like Kubernetes or
setting up LUKS devices.
When testing multiple images one always has to copy them to the
expected file name, and when trying to run two VMs this means one needs
to either use separate directories or modify the qemu script. One also
needs to modify the qemu script to bump the memory for K8s or for LUKS.
Support parameters for the VM image name and the VM memory.
The ACI OEM container image was used with rkt for GCE. For long time
this has been migrated to systemd-nspawn and now systemd-sysext.
Remove the unused build library code.
Mksquashfs running against a btrfs filesystem tries to capture btrfs specific
xattrs (btrfs.compression) generating a lot of spam. Remove the spam by
ignoring btrfs xattrs.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Putting things in a sysexts subdir results in the same layout on
bincache and does not follow the expected url schema for fetching the
sysext.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This allows it to be sourced from other scripts and used e.g. in
generating an image changes summary.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
There are two challenges with the sysext: it needs config files in /etc
and it needs udev rules for mounting during boot to work. The etc files
are placed in the standard flatcar etc overlay path but the overlay is
mounted from the initrd. So instead, we create a tmpfiles.d rule that
symlinks the best important files over. For the udev issue, we create a
drop-in in /etc that ensures udev runs after systemd-sysext.
We also can't rely on systemd presets to work, so instead parse the
preset file and statically create the service dependencies. For the
primary zfs.target we rely on an Upholds entry. Users can still disabled
unwanted services if they want.
We also removed unnecessary files:
- development files
- initramfs related scripts
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This function is meant to prebuild certain sysexts to be released along
with each release. These will not be built into the image, but instead
can be fetched by the user on demand.
The command to build sysexts would be:
./build_image prod sysext
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
It looks like 'norecovery' is deprecated and has been removed in the v6.8-rc1
kernel. Replace it with 'rescue=nologreplay', which is a replacement
implemented since v5.9. The standalone 'nologreplay' option is also deprecated.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
We have an existing qemu_uefi_secure format definition, but it is
necessary to update it so that it actually works. Qemu needs to be
passed the correct flags to enable SMM, we need to switch to the Q35
machine, and we need to copy over the secboot variant of the OVMF
firmware.
This is just the contents of the section, but the section
itself is written by grub-mkimage. sbat.csv needs to be passed
with --sbat.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
Signed-off-by: Sayan Chowdhury <schowdhury@microsoft.com>
