25901 Commits

Author SHA1 Message Date
flatcar-ci
4deb3ba157 New version: main-3766.0.0-nightly-20231023-2100 2023-10-23 21:00:25 +00:00
Thilo Fromm
f81bbebddf build_sysext: run package inventorisation before mangle script
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:46 +02:00
Thilo Fromm
65bf2bfd0c Update run_local_tests.sh: improve quoting
Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2023-10-23 16:05:46 +02:00
Thilo Fromm
6dd2f203f1 run_local_tests.sh: un-break variable quoting
Turns out using ${var@Q} instead of ${var} ends up with paths like
/work/foo/'amd64'-usr/...
instead of
/work/foo/amd64-usr/...
which breaks the script. So we revert it.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:46 +02:00
Thilo Fromm
d744d32733 build_sysext: optimise squashfs for size
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:46 +02:00
Thilo Fromm
37cc05ce11 coreos/base/package.use: squashfs-tools: enable all compression formats
This change enables USE flags for all supported compression formats.
zstd specifically is required to build zstd sysexts.
2023-10-23 16:05:46 +02:00
Thilo Fromm
c2d1c13abd Apply bash scripting suggestions from code review
Thank you Krzesimir!

Co-authored-by: Krzesimir Nowak <knowak@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
b775036a62 torcx removal: address PR comments
- updated github actions for runc, containerd, and docker to not handle
  nonexistent ebuilds in app-torcx/ anymore
- removed spurious package_run_dependencies from build_image_util.sh
- build_sysext: generate pkginfo before mangle script runs
  use zstd for compression; add cli flag to select compression
- ci_automation_common.sh: remove spurious `/` from match string
- coreos, board-packages, bootengine: bump ebuild revisions
- kernel commonconfig: add squashfs zstd support

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
57ed77ea91 ci-automation: fix local image detection for nightly SDKs
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
22582dca7c changelog: add torcx removal, sysext migration
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
6512ef04ca overlay coreos-base/update_engine: remove torcx sanity check
Include PR https://github.com/flatcar/update_engine/pull/30 to un-break
updates when torcx was removed in favour of sysext.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
428503083f run_sdk_container: Add -U flag to use the version file unchanged
This change adds a -U flag to run_sdk_container. If provided, the script
will not regenerate version.txt but instead use the existing file as-is.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
919d368688 mantle-container: bump image ref to support torcx removal
This change bumps the image ref of the mantle container to
ghcr.io/flatcar/mantle:git-20a2f8ffee8c8a1a042b1da99f0f59312110f285.
This version includes 2 PRs (https://github.com/flatcar/mantle/pull/465
and https://github.com/flatcar/mantle/pull/466) which add support for
sysext docker / torcx removal in the OS image.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
8ef820ad20 vendor-testing/qemu_update.sh: make update payload configurable
This change makes QEMU_UPDATE_PAYLOAD configurable via
ci-automation/settings.env where it was hard-wired before.

The change also fixes fall-out in qemu_update.sh by ensuring a local tmp
directory is created before it is used by the test.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
bd54e3a0ce .github: removed torcx artifact handling from github CI
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
35779f1ca3 run_local_tests.sh: script for running tests on local builds
This change adds run_local_tests.sh, a script to run tests on local
builds. It's a comfort wrapper around ci-automation scripts and uses
the latest local build.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
6f651a298c build scripts: separate txt inventory; official naming for sysexts
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
e85ca4984c ci-automation: remove torcx artifact handling and tests 2023-10-23 16:05:45 +02:00
Thilo Fromm
89555ed2bd base OS sysexts: separate build script, inventory generation
This change refactors base OS sysext builds to use a separate build
script `build_library/sysext_prod_builder`, which is called from
`build_library/prod_image_util.sh` when `build_image` runs.

This allows for better separation of cleanup traps: prod image sysext
builds need its own trap / cleanup function for temporary build
directories and loopback mounts.

Prod sysext builds properly generate lincense and SBOM information, and
provide detailed file listings and disk space usage stats.

- SBOM / licenses JSON now include all packages of the
  final image, i.e. a combined list of base image and all base OS
  sysexts.
- Packages lists, files list and detailed files list include the sysext
  squashfs files for the base image, and separate sections with files /
  packages lists for each sysext.
- Disk usage contains both final disk image usage as well as usage of
  each individual sysext squashfs.
2023-10-23 16:05:45 +02:00
Thilo Fromm
c589fb8e56 coreos-overlay: removed all remaining app-torcx artifacts
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
8e01a2c8f9 build_image: add sysext command line option
This change refactors sysext builds during build_image and generalises
the code (no hard-coded containerd and docker anymore).

A command line option is added to build_image for sysexts to include in
the OS image. It defaults to containerd and docker but may be set to
arbitrary packages. The command line supports simple depenencies, i.e.
the "docker" sysext will re-use package information from the
"containerd" sysext and not include another containerd.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
8f8f262f19 torcx: remove from scripts, use docker+containerd sysexts
This change removes torcx libraries, references, and commandline options
from build automation scripts and from build_library/.

Containerd and docker are shipped via sysexts which are included in the
base image.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
aae3fd8260 prod_image_util.sh: remove pkgdb tarball (replaced by sysext)
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
9837c35ec4 build_sysext: add sysext dependency handling
This change adds 2 optional command line parameters to build_sysext to
handle dependencies in stacked sysexts. The command line parameters
allow exporting portage package db information into a separate squashfs
image as well as using package db information exported by a previous
sysext build.

--generate_pkginfo will generate a separate squashfs
   <sysextname>_pkginfo.raw which contains the sysext's /var/db/pkgs.

--base_pkginfo=<pkginfo>[:<pkginfo>[:...]] will use the supplied paths
   to pkginfo squashfses as additional lowerdirs when creating a sysext.

Useage example:
./build_sysext --generate_pkginfo containerd app-containers/containerd
./build_sysext --base_pkginfo=containerd_pkginfo.raw docker app-containers/docker

will create a containerd.raw sysext and a dependent docker.raw sysext
that does not have the containerd dependency installed. Both sysexts
must be merged together in order for docker to work.

Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Thilo Fromm
9cede02f86 dev_container_util.sh: remove spurious DEVCONTAINER=1
Signed-off-by: Thilo Fromm <thilofromm@microsoft.com>
2023-10-23 16:05:45 +02:00
Krish Jain
cc029bb5ca use DEVCONTAINER=1 2023-10-23 16:05:45 +02:00
Krish Jain
3c4639b0ce use sudo_clobber function 2023-10-23 16:05:45 +02:00
Krish Jain
298e283944 fix permissions issue 2023-10-23 16:05:45 +02:00
Krish Jain
4633cc3e7c Fix syntax errors, quoting 2023-10-23 16:05:45 +02:00
Krish Jain
f6f5c32417 add information about the packages contained in the built sysext 2023-10-23 16:05:45 +02:00
Krish Jain
fe949f5900 Make sure binary packages are built in build_packages already 2023-10-23 16:05:45 +02:00
Krish Jain
4e8b276319 Remove duplicate making directory root_fs_dir/etc/extensions 2023-10-23 16:05:45 +02:00
Krish Jain
abe8bf683a Use helper function query_available_package() 2023-10-23 16:05:45 +02:00
Krish Jain
4b00bc14a4 tail, not head 2023-10-23 16:05:45 +02:00
Krish Jain
9709a64239 incorporate suggestions from Jeremi 2023-10-23 16:05:45 +02:00
Krish Jain
e6bb5470c4 Add mangle script for containerd 2023-10-23 16:05:45 +02:00
Krish Jain
e5f1ecbae3 Only keep docker stuff in manglefs_docker 2023-10-23 16:05:45 +02:00
Krish Jain
2be2e884b3 Add manglefs script for containerd 2023-10-23 16:05:45 +02:00
Krish Jain
3d17f65088 Create seperate docker and containerd sysext 2023-10-23 16:05:45 +02:00
krishjainx
c026d9ed53 Remove app-arch/torcx from coreos-overlay 2023-10-23 16:05:45 +02:00
krishjainx
851009c6cc Install docker as a system extension, use manglefs script ( convert the unit dependencies from wants to upholds); one sysext for all docker for now 2023-10-23 16:05:45 +02:00
Mathieu Tortuyaux
7b0d9bde7e
Merge pull request #1289 from flatcar/linux-6.1.59-main
Upgrade Linux Kernel for main from 6.1.58 to 6.1.59
2023-10-23 15:26:43 +02:00
flatcar-ci
ed16d78876 New version: main-3763.0.0-nightly-20231020-2100 2023-10-21 02:09:26 +00:00
flatcar-ci
418e418c29 New version: main-3763.0.0-nightly-20231020-2100-INTERMEDIATE 2023-10-20 21:00:29 +00:00
Kai Lüke
a85e51b03d
Merge pull request #1170 from flatcar/kai/usr-btrfs
disk_layout: use btrfs for the /usr partition
2023-10-20 13:26:21 +02:00
Kai Lüke
736259812a
Merge pull request #1282 from flatcar/mantle-update-main
Upgrade mantle container image to latest HEAD in main
2023-10-20 13:22:50 +02:00
Flatcar Buildbot
9c8e8b8f10 sys-kernel/coreos-sources: Update from 6.1.58 to 6.1.59 2023-10-20 07:11:41 +00:00
Flatcar Buildbot
c3b366f872 Update mantle container image to latest HEAD 2023-10-19 21:00:42 +00:00
flatcar-ci
18adacfa57 New version: main-3762.0.0-nightly-20231019-2100 2023-10-19 21:00:23 +00:00
Dongsu Park
291ec131e3
Merge pull request #1281 from flatcar/dongsu/rust-1.73.0
overlay dev-lang/rust: Update to 1.73.0
2023-10-19 12:27:20 +02:00