Commit Graph

192 Commits

Author SHA1 Message Date
Krzesimir Nowak
e65072910d build_library: Try to clean up unused stuff
There is some cruft left after grub hashes generation. After the
contents are zipped into archive, they don't need to be around any
more.

Try to remove the rootfs directory after unmounting the
image. disk_util can recreate it again if there is a need for it.

Remove the build directory used for generating ACI images - it's not
needed after successful installation.
2022-07-14 15:08:12 +02:00
Kai Lueke
6648df57a2 Remove unused image variants
There is no difference anymore between the regular and Pro images.
Remove the definitions and stop building extra images.
2022-05-03 20:30:42 +09:00
Gabriel Adrian Samfira
f126debb37 Allow specifying multiple compression formats
Add the ability to specify a comma separated list of compression formats.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-07 12:01:21 +00:00
Gabriel Adrian Samfira
571de4064b Make image compression format configurable
This change adds a new flag called --image_compression_format which
allows us to output the final VM image as one of the supported formats:

bz2 (default), gz, zip or none

if the compression format is "none" or "", the image will not be compressed.

Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
2022-03-07 12:01:21 +00:00
Sayan Chowdhury
560122f49d
build_library/vm_image_util.sh: Pro image to use non-pro package & use flag 2022-02-17 19:42:05 +05:30
Jeremi Piotrowski
07568fb418 build_library: vm_image_util: create fixed-size VHDs for Azure
Azure requires disks to be fixed-size VHD files when uploading to blob storage
in order to create image/gallery objects from them. This is documented here[1].
To prevent mistakes from happening create disks in that format directly so that
any azure compatible tool can upload them, though azcopy is recommend because
it handles their sparseness best.

This has not been an issue for us so far because kola uses code from an older
utility that transparently handled the dynamic-to-fixed-size conversion for VHD
files (azure-vhd-utils). But people working with these things for the first
time fall into this trap.

[1]: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/create-upload-generic#resizing-vhds.

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
2022-02-01 17:52:09 +00:00
Kai Lüke
c1b2d88aa2 build_image_util.sh: delete vmlinuz file from /usr partition
The vmlinuz kernel image gets installed to /usr/boot/ but isn't usable
for dm-verity until it gets copied over to /boot/flatcar/ and the hash
gets embedded at a particular offset. The file in /usr/boot/ uses space
while it's not having a real purpose as long as dm-verity is used.

Delete the vmlinuz file under /usr/boot/ to free up space. When
generating the ISO image we use the vmlinuz file from /boot/flatcar/
which also has the advantage that we only distribute a single vmlinuz
file with one particular checksum.
2021-07-28 20:59:16 +02:00
Krzesimir Nowak
6ed7cd66d5 *: Drop jobs parameter
The `--jobs` parameter that some scripts defined was not used anywhere
in jenkins or mantle. So the value of the parameter always ended up
being equal to `${NUM_JOBS}` set by `common.sh`. Also, even if the
`--jobs` parameter was used for some script, that script usually
didn't forward the jobs value to other scripts, so the other scripts
ended up using `${NUM_JOBS}` again. Also, the `${FLAGS_jobs}` variable
was used by some functions in the build library, and those functions
were sometimes invoked by scripts that didn't define the
`${FLAGS_jobs}` variable. It is tedious to track which script should
actually define the parameter, and where it should be forwarded.

Just get rid of this half-working pretense. If you want to affect how
many jobs `emerge` uses, export the `NUM_JOBS` environment variable
before calling any script.

For `EMERGE_FLAGS` and `REBUILD_FLAGS` we unconditionally specify the
`--jobs` flag's value to `${NUM_JOBS}` because they are passed to
`emerge`. On the other hand we drop the `--jobs` parameter from the
`UPDATE_ARGS` variable, because this variable passed to `setup_board`
or `update_chroot`, which don't have this flag any more.
2021-02-17 13:26:36 +01:00
Krzesimir Nowak
9cbf3049ea vm_image-util: Forward jobs parameter to build_oem_aci 2021-02-16 08:36:57 +01:00
Margarita Manterola
ad8204af82 vm_image_util: Use new aws_pro flag for ec2-compat 2021-01-22 11:57:52 +01:00
Margarita Manterola
cf7f4bd4a4 Add GCE Pro OEM type 2020-12-22 13:52:52 +01:00
Kai Lüke
f84ca169e5 Add AWS Pro OEM type
This is the initial creation of an AWS Pro image coming with support,
and features added on top of it.
2020-12-09 16:38:28 +01:00
Sayan Chowdhury
9e2098bfff
Update the azure pro package reference to the oem-azure-pro
This commit also fixes the BINHOST URL for the for the developer
container portage

Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-11-25 14:42:56 +05:30
Sayan Chowdhury
8799028007
Add OEM package for QEMU
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-10-08 11:51:24 +05:30
Sayan Chowdhury
9953cc8c8f
build_library: Rename the images to use pro instead of premium
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-09-21 20:31:41 +05:30
Sayan Chowdhury
cc147a1255
Add the new OEM type, and selective upload azure as private
Signed-off-by: Sayan Chowdhury <sayan@kinvolk.io>
2020-09-19 18:58:12 +05:30
Kai Lüke
687bbefad8
build_library/vm_image_util.sh: Generate iPXE script
Write out an iPXE script file for Packet.
The script uses relative URLs to refer to
the other PXE files and thus can be copied
along with the files to any server.
This is useful because it saves the creation
of an iPXE script for a release/channel on a
third-party service. For CI testing it is
also helpful because the script does not only
end up on the release server but also already
on the Google buckets, refering to unpublished
PXE payloads.
2020-03-18 16:09:40 +01:00
Flatcar Buildbot
c7bbb2b1e2 2345.0.0 2019-12-04 14:59:11 +01:00
Flatcar Buildbot
1dad511f69 2317.0.1 2019-11-07 19:40:01 +01:00
David Michael
f9432f6702 vm_image_util: Switch back to edk2-ovmf paths
The custom sys-firmware/edk2 package has been replaced by Gentoo's
sys-firmware/edk2-ovmf package now that only amd64 is supported.

This partially reverts 1761d9d071 .
2018-10-25 17:15:03 +00:00
David Michael
2467923d56 Remove arm64 from supported board operations 2018-10-25 16:00:09 +00:00
David Michael
5ffb938934 image_to_vm: Fix OEM install roots for new portage 2018-08-30 00:53:17 +00:00
Benjamin Gilbert
25fffe839d build_library: update VMware virtual hardware version to 11
This will run on ESXi 6.0 and above, and all non-EOL versions of Fusion
and Workstation.

Also enable a few useful VMX features (HPET; CPU and memory hotplug) that
are added by VMware Workstation 14.1.1's Change Hardware Compatibility
wizard. Correspondingly, enable CPU/memory hotplug in the OVF; omit
HPET because there's no obvious way to enable it.
2018-05-18 16:23:32 -04:00
Benjamin Gilbert
0e83301d8c build_library: drop oracle_oci_qcow image format
Images have been published to the release buckets, but never to OCI.
2018-04-25 22:40:29 -04:00
Benjamin Gilbert
fcbc7c3219 vm_image_util: bump PXE and ISO scripts to 2 GiB RAM
Now that we're shipping multiple docker packages in the image, 1 GiB
is not enough to boot successfully.
2017-09-22 15:52:19 -07:00
Benjamin Gilbert
2024326789 build_library: rename oracle_oci image to oracle_oci_qcow
Reserve the unsuffixed name for a raw image, in case we ever want to
support installing to disk using "coreos-install -o" on a PXE-booted
system.
2017-09-06 17:46:07 -07:00
Benjamin Gilbert
51c5345c0d build_library: Add Oracle OCI OEM and format 2017-08-30 03:10:33 -07:00
David Michael
71ebc66065 build_library: avoid a SIGPIPE while building arm64 UEFI firmware
This just sets the code file size to the var file size, so it gets
zero-padding without having to pipe commands together.

From: David Michael <david.michael@coreos.com>
[Rebased]
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 15:03:58 -07:00
Geoff Levand
1761d9d071 vm_image_util: Update to use edk2 package
Signed-off-by: Geoff Levand <geoff@infradead.org>
2017-07-27 14:51:32 -07:00
Benjamin Gilbert
e98efe074d build_library: Add vagrant_virtualbox image type 2017-06-22 13:41:15 -07:00
Alex Crawford
65986e02bc build_library: split vagrant and vagrant vbox
This reverts the vagrant image back to using oem-vagrant because we
don't want to break the existing images. It moves the new,
Ignition-powered virtualbox flavor of vagrant into a new image.
2017-06-22 13:23:50 -07:00
Benjamin Gilbert
ab4705f9dc Merge pull request #697 from bgilbert/generic
Drop ISO/PXE support for amd64-generic, and an unused variable
2017-06-21 13:28:55 -07:00
Benjamin Gilbert
cb8b8f34ca build_library: Drop ISO/PXE support for amd64-generic 2017-06-16 19:05:07 -07:00
Alexander Pavel
7298b0bf2a build_library: Change oem package for vagrant to oem-vagrant-virtualbox
This changes the oem-package for vagrant to vagrant-virtualbox,
which uses ignition instead of cloud-clonfig and sets the oem id
to "virtualbox" so that ignition can handle the machine correctly
2017-06-14 15:58:17 -07:00
Alexander Pavel
cab8e3ed37 build_library: Add virtualbox oem to vm_image_util
This commit adds support for the oem-virtualbox package for
virtualbox builds
2017-06-14 13:21:21 -07:00
Benjamin Gilbert
40978865d2 Revert "A pile of goo to support a minimal secure boot demo"
This reverts commit 0c6adece3d.
2017-06-08 14:51:15 -07:00
Benjamin Gilbert
6878bd8f5f Revert "vm_image_util: add qemu_xen image type for testing"
This reverts commit fe9db4157b.
2017-06-08 14:47:23 -07:00
Benjamin Gilbert
c98788befc image_to_vm: Default to qemu_uefi on ARM
qemu makes no sense on that platform.
2017-05-02 15:02:09 -07:00
Benjamin Gilbert
329f4bd113 build_library: Run dd with status=none
Avoid dumping dd statistics to stderr.
2017-04-21 14:06:00 -07:00
Benjamin Gilbert
b363484c3b image_to_vm: Add vmware_raw image type
Allow "coreos-install -o vmware_raw" to install Container Linux with
the vmware OEM.

Use base DISK_LAYOUT to reduce the minimum disk size.

Fixes coreos/bugs#359.
2017-01-19 14:34:53 -08:00
Nick Owens
2b46ddc20c build_library: pass force_size when converting disks to vpc
this stops qemu-img info from choking on vpc-type images made with our
patched qemu.
2016-11-28 17:17:20 -08:00
Bassam Tabbara
b0bf230433 add support for Parallels Desktop VM images
Two new image types have been added:

1. parallels - this produces VM images with extension pvm.tgz that can be loaded directly into Parallels Desktop
2. vagrant_parallels - this produces a Vagrant box that works with parallels vagrant provider (http://parallels.github.io/vagrant-parallels/)

Just like vmdk and others we rely on qemu-img to convert raw images. Support for Parallels disk images was added to qemu-img in version 2.4.

I also removed the box files from the actual image since there are not needed in /usr/share/oem.

Signed-off-by: Bassam Tabbara <bassam.tabbara@quantum.com>
2016-10-26 09:25:45 +02:00
David Michael
5b640e79fd image_to_vm: build and install a GCE agent ACI 2016-10-18 11:37:45 -07:00
David Michael
e715a9ca3c image_to_vm: support generating and installing an OEM agent ACI
The ACI root is created by reusing the create_prod_image function
to install a base meta-package.  It then runs a script to customize
the file structure as required by agent software (if necessary),
writes a manifest file from a supplied template, and then packages
it all into a tar file.
2016-10-18 11:37:33 -07:00
Nick Owens
1fd668ef0d build_library: fix grub loader copy for arm64 2016-05-12 12:05:11 -07:00
Michael Marineau
9a468bf37d vm_image_util: ensure UEFI firmware is installed before reading it 2016-05-10 12:02:56 -07:00
Nick Owens
a03cef558e build_library: copy grub.efi in pxe/cpio generation 2016-04-27 15:54:59 -07:00
Nick Owens
e0524264b9 build_library: generate pflash files for qemu_uefi from edk2-armvirt 2016-04-21 10:45:15 -07:00
Michael Marineau
66ea5974ce image_to_vm: add --getbinpkg option
Now image_to_vm can be used without first running build_packages.
2016-02-23 13:06:49 -08:00
Michael Marineau
744faa2df4 vm_image_util: oems are always from ebuild, others always from binary
This resolves two issues:
 - Large dependencies are *never* built during image_to_vm,
   build_packages must now handle that.
 - Since build_packages can't resonably do the oem-* packages (they all
   conflict with eachother) we do want to build them from the ebuild.
   This is now enforced so a old binpkg is never used. This resolves
   confusing issues people have always had while when editing oem
   ebuilds but getting a stale build instead.
2016-02-23 12:50:52 -08:00