Commit Graph

218 Commits

Author SHA1 Message Date
Kai Lueke
87ff566aae Use OpenStack image for Brightbox
The special Brightbox image uses the OpenStack userdata in Ignition but
lacked Afterburn usage. It actually works to use the OpenStack image and
directly which also enables Afterburn, thus we can drop the special
image.
Don't build a special image for Brightbox but recommend to use OpenStack
images directly. A symlink is added to help with the download of
hardcoded user scripts.
2023-11-15 13:33:55 +01:00
Mathieu Tortuyaux
b36cb28768
overlay coreos-base/oem-packet: migrate to sysext format
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2023-10-09 14:35:08 +02:00
Mathieu Tortuyaux
989d20a2f1
overlay core-base/oem-digitalocean: migrate to sysext format
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2023-10-09 14:35:07 +02:00
Mathieu Tortuyaux
57e7b5aebc
overlay coreos-base/oem-openstack: add oem-openstack
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2023-10-09 14:35:07 +02:00
Mathieu Tortuyaux
ce2c0e7475
overlay coreos-base/oem-ami: new sysext image
Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2023-09-26 13:34:09 +02:00
Kai Lüke
9ffceaf3fe
Merge pull request #1132 from krishjainx/qcow2-inline-compression
Allow use qcow2 inline compression
2023-09-25 17:37:15 +02:00
Krzesimir Nowak
fa3dc6b649 build_library/vm_image_util: Opt VMware OEM into sysext images 2023-09-20 09:18:43 +02:00
Kai Lueke
d711725fe0 vm_image_util: Fix variable quoting to create correct OEM sysext file
The OEM sysext image file in the OEM partition had the version variable
name being part of the filename instead of the substituted version value
because of wrong quoting when the fixed string got replaced by ${…}.
2023-09-14 11:46:40 +02:00
krishjainx
2124f63dcb Allow use qcow2 compressed format in place of additional compressed layer 2023-09-12 11:21:08 -04:00
Kai Lueke
995ae2cdbc Support OEM systemd-sysext images and Flatcar extensions
The vendor tools on the OEM partition weren't updated. We now want to
ship them as systemd-sysext images which we can easily update. This
change extends the Flatcar A/B update mechanism to cover the OEM
systemd-sysext images. The same mechanism is also able to support
"official" Flatcar extensions, e.g., a ZFS extension.
2023-09-08 14:50:43 +02:00
Thilo Fromm
1f2706d44e build_sysext: guess image build dir and auto-create it
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-07-07 14:49:50 +02:00
Thilo Fromm
190fd2c38e build_sysext: bugfixes, QoL changes, help updated
This change improves build_sysext by sourcing a missing lib dependency,
adding a number of comfort / quality-of-life options, and updating the
output of '--help' accordingly.

The OEM sysext finction in build_library/vm_image_util.sh is also
updated to use new command line format.

1. Include missing dependency toolchain_util.sh to fix an error in
   board_options.sh (get_board_arch undefined).
2. Use positional parameters for mandatory arguments.
   build_dir and sysext_name are mandatory and are now positional
   arguments instead of options.
   binary_package is the third positional argument but can be omitted
   if --metapkgs was specified.
3. --squashfs_base is now guessed better and will use the most recent
   build by default.
4. A new boolean flag --ignore_version_mismatch for the more daring
   developer was added. The flag will cause the script to continue if a
   version mismatch between SDK board packages and squashfs base is
   detected.
5. Error messages were improved for when mandatory parameters were not
   provided.
6. The '--help' message was improved and adjusted to the new parameters.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-07-07 11:44:39 +02:00
Krzesimir Nowak
00ac7586a4 *: Remove support for niftycloud OEM 2023-07-05 14:34:17 +02:00
Krzesimir Nowak
ad0360e744 *: Remove support for interoute OEM 2023-07-05 14:33:26 +02:00
Krzesimir Nowak
ec723be9d9 *: Drop unused pkgdb stuff 2023-07-05 08:43:25 +02:00
Krzesimir Nowak
16fee4f5b1 build_library/vm_image_util.sh: Switch OEM sysext image building to build_sysext 2023-07-05 08:43:25 +02:00
Krzesimir Nowak
8c2883114a build_library/vm_image_util.sh: Set up a path to sysext base image 2023-07-05 08:40:15 +02:00
Kai Lueke
af4daf3516 Upload dev-key-signed OEM sysext update payload
For release tests and updating a machine to a dev build we already have
the dev-key-signed generic update payload but not yet the OEM sysext
update payload.
Generate the dev-key-signed OEM sysext update payload during build and
upload it.
2023-06-14 18:30:48 +02:00
Kai Lueke
f790e3e648 build_library: For now don't use a version for the OEM sysext image
The initial MVP of the OEM sysext usage we release won't have updates
for the sysext image and, therefore, it is not bound to the OS version.
The special name suffix instead of the version hints bootengine at using
it if no matching version is found. The name will also be used at hint
for update-engine to clean it up when versioned sysext images arrive.
2023-06-14 13:00:30 +02:00
Krzesimir Nowak
23df3170ff build_library: Force initial version of OEM sysexts for now
We don't have an update process of the OEM sysexts implemented yet, so
use a fake "initial" version for them and make them independent from
OS version.
2023-06-06 15:10:07 +02:00
Krzesimir Nowak
8b17ca2b02 overlay coreos-base/oem-qemu: Make it a sysext package
It isn't doing much as nothing QEMU-specific was being installed into
the OEM partition.

With that done, we opt into building an OEM sysext image for QEMU
platform.
2023-06-06 15:10:07 +02:00
Krzesimir Nowak
995910cd78 overlay coreos-base/oem-azure: Make it a sysext package
This package will be used for the sysext image, instead of for
installing files into /usr/share/oem. This means that we can drop some
files or move them elsewhere. The systemd service file is not needed,
because it is installed by the app-emulation/wa-linux-agent package
now. This also means that the ignition file as lost its purpose. The
grub.cfg and oem-release must be installed in /usr/share/oem, next to
the sysext raw image file, so handling of these files is moved to the
newly added coreos-base/common-oem-files package. `eject` symlink to
`/usr/bin/true` is installed in the newly added manglefs.sh script.

With this done, we also opt into building an OEM sysext image for
Azure platform.
2023-06-06 15:10:07 +02:00
Krzesimir Nowak
46b98ba0d2 image_to_vm: Require pkgdb tarball and handle sysext building 2023-06-06 13:01:09 +02:00
Krzesimir Nowak
33d2af5600 build_library: Adapt to /oem being the new OEM partition mountpoint
This requires us to add another directory to keep when removing a
regenerable state from rootfs. Other changes were straightforward find
and replace.
2023-05-30 15:53:17 +02:00
Krzesimir Nowak
fa48f70a37 *: Make catalyst and emerge verbose by default
That way we can see a report of what emerge is going to do and the
status of the use flags for the installed packages. The downside is
that we are going to have reports about using deprecated and
unsupported profile in even more places.
2023-02-16 13:57:05 +01:00
Krzesimir Nowak
520b92ad7e *: Expand short emerge flags and use bash arrays
Emerge flags are cryptic in general, but short flags even more so, so
expand them. While at it, I noticed some places where bash arrays
could be used, so convert those places too.
2023-02-16 13:57:05 +01:00
Krzesimir Nowak
e65072910d build_library: Try to clean up unused stuff
There is some cruft left after grub hashes generation. After the
contents are zipped into archive, they don't need to be around any
more.

Try to remove the rootfs directory after unmounting the
image. disk_util can recreate it again if there is a need for it.

Remove the build directory used for generating ACI images - it's not
needed after successful installation.
2022-07-14 15:08:12 +02:00
Kai Lueke
6648df57a2 Remove unused image variants
There is no difference anymore between the regular and Pro images.
Remove the definitions and stop building extra images.
2022-05-03 20:30:42 +09:00
Gabriel Adrian Samfira
f126debb37 Allow specifying multiple compression formats
Add the ability to specify a comma separated list of compression formats.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-07 12:01:21 +00:00
Gabriel Adrian Samfira
571de4064b Make image compression format configurable
This change adds a new flag called --image_compression_format which
allows us to output the final VM image as one of the supported formats:

bz2 (default), gz, zip or none

if the compression format is "none" or "", the image will not be compressed.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-07 12:01:21 +00:00
Sayan Chowdhury
560122f49d
build_library/vm_image_util.sh: Pro image to use non-pro package & use flag 2022-02-17 19:42:05 +05:30
Jeremi Piotrowski
07568fb418 build_library: vm_image_util: create fixed-size VHDs for Azure
Azure requires disks to be fixed-size VHD files when uploading to blob storage
in order to create image/gallery objects from them. This is documented here[1].
To prevent mistakes from happening create disks in that format directly so that
any azure compatible tool can upload them, though azcopy is recommend because
it handles their sparseness best.

This has not been an issue for us so far because kola uses code from an older
utility that transparently handled the dynamic-to-fixed-size conversion for VHD
files (azure-vhd-utils). But people working with these things for the first
time fall into this trap.

[1]: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/create-upload-generic#resizing-vhds.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2022-02-01 17:52:09 +00:00
Kai Lüke
c1b2d88aa2 build_image_util.sh: delete vmlinuz file from /usr partition
The vmlinuz kernel image gets installed to /usr/boot/ but isn't usable
for dm-verity until it gets copied over to /boot/flatcar/ and the hash
gets embedded at a particular offset. The file in /usr/boot/ uses space
while it's not having a real purpose as long as dm-verity is used.

Delete the vmlinuz file under /usr/boot/ to free up space. When
generating the ISO image we use the vmlinuz file from /boot/flatcar/
which also has the advantage that we only distribute a single vmlinuz
file with one particular checksum.
2021-07-28 20:59:16 +02:00
Krzesimir Nowak
6ed7cd66d5 *: Drop jobs parameter
The `--jobs` parameter that some scripts defined was not used anywhere
in jenkins or mantle. So the value of the parameter always ended up
being equal to `${NUM_JOBS}` set by `common.sh`. Also, even if the
`--jobs` parameter was used for some script, that script usually
didn't forward the jobs value to other scripts, so the other scripts
ended up using `${NUM_JOBS}` again. Also, the `${FLAGS_jobs}` variable
was used by some functions in the build library, and those functions
were sometimes invoked by scripts that didn't define the
`${FLAGS_jobs}` variable. It is tedious to track which script should
actually define the parameter, and where it should be forwarded.

Just get rid of this half-working pretense. If you want to affect how
many jobs `emerge` uses, export the `NUM_JOBS` environment variable
before calling any script.

For `EMERGE_FLAGS` and `REBUILD_FLAGS` we unconditionally specify the
`--jobs` flag's value to `${NUM_JOBS}` because they are passed to
`emerge`. On the other hand we drop the `--jobs` parameter from the
`UPDATE_ARGS` variable, because this variable passed to `setup_board`
or `update_chroot`, which don't have this flag any more.
2021-02-17 13:26:36 +01:00
Krzesimir Nowak
9cbf3049ea vm_image-util: Forward jobs parameter to build_oem_aci 2021-02-16 08:36:57 +01:00
Margarita Manterola
ad8204af82 vm_image_util: Use new aws_pro flag for ec2-compat 2021-01-22 11:57:52 +01:00
Margarita Manterola
cf7f4bd4a4 Add GCE Pro OEM type 2020-12-22 13:52:52 +01:00
Kai Lüke
f84ca169e5 Add AWS Pro OEM type
This is the initial creation of an AWS Pro image coming with support,
and features added on top of it.
2020-12-09 16:38:28 +01:00
Sayan Chowdhury
9e2098bfff
Update the azure pro package reference to the oem-azure-pro
This commit also fixes the BINHOST URL for the for the developer
container portage

Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-11-25 14:42:56 +05:30
Sayan Chowdhury
8799028007
Add OEM package for QEMU
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-10-08 11:51:24 +05:30
Sayan Chowdhury
9953cc8c8f
build_library: Rename the images to use pro instead of premium
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-09-21 20:31:41 +05:30
Sayan Chowdhury
cc147a1255
Add the new OEM type, and selective upload azure as private
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-09-19 18:58:12 +05:30
Kai Lüke
687bbefad8
build_library/vm_image_util.sh: Generate iPXE script
Write out an iPXE script file for Packet.
The script uses relative URLs to refer to
the other PXE files and thus can be copied
along with the files to any server.
This is useful because it saves the creation
of an iPXE script for a release/channel on a
third-party service. For CI testing it is
also helpful because the script does not only
end up on the release server but also already
on the Google buckets, refering to unpublished
PXE payloads.
2020-03-18 16:09:40 +01:00
Flatcar Buildbot
c7bbb2b1e2 2345.0.0 2019-12-04 14:59:11 +01:00
Flatcar Buildbot
1dad511f69 2317.0.1 2019-11-07 19:40:01 +01:00
David Michael
f9432f6702 vm_image_util: Switch back to edk2-ovmf paths
The custom sys-firmware/edk2 package has been replaced by Gentoo's
sys-firmware/edk2-ovmf package now that only amd64 is supported.

This partially reverts 1761d9d071 .
2018-10-25 17:15:03 +00:00
David Michael
2467923d56 Remove arm64 from supported board operations 2018-10-25 16:00:09 +00:00
David Michael
5ffb938934 image_to_vm: Fix OEM install roots for new portage 2018-08-30 00:53:17 +00:00
Benjamin Gilbert
25fffe839d build_library: update VMware virtual hardware version to 11
This will run on ESXi 6.0 and above, and all non-EOL versions of Fusion
and Workstation.

Also enable a few useful VMX features (HPET; CPU and memory hotplug) that
are added by VMware Workstation 14.1.1's Change Hardware Compatibility
wizard. Correspondingly, enable CPU/memory hotplug in the OVF; omit
HPET because there's no obvious way to enable it.
2018-05-18 16:23:32 -04:00
Benjamin Gilbert
0e83301d8c build_library: drop oracle_oci_qcow image format
Images have been published to the release buckets, but never to OCI.
2018-04-25 22:40:29 -04:00