We were running the run_sdk_container script with passing a value of a
variable named version to the script through the -v flag. But nowhere
is the variable defined. This worked under jenkins, because jenkins
job has a version parameter that gets exported into environment under
the same name. But running it manually outside jenkins revealed the
bug.
The script should have been using a vernum variable. Now, the
difference between this variable and the version variable is that
"version" was in form of <channel>-<version>-<build_id>, whereas
"vernum" comes without the channel part. Fortunately,
"run_sdk_container" was stripping the channel part before using this
value, so it makes no difference whether we pass
main-3333.0.0.0-some-id or just 3333.0.0-some-id.
Recently we changed the region from DA (Dallas) to DC (Washington),
because there are more ARM64 servers available. Reflect this change in
the new pipeline too.
Some packages are currently missing from the /usr/share/SLSA directory
compared to flatcar_production_image_packages.txt. For torcx packages,
extract the reports from the torcx bundle when adding it to the rootfs.
For initramfs packages, as a substitute we enumerate build dependencies
of coreos-kernel (image_packages_implicit()). At this time these are
bootengine and intel-microcode.
Prod images need libstdc++.so and other libraries produced by
sys-devel/gcc build, but because we don't want all of gcc in the image,
the binpkg is manually unpacked instead of installed with emerge. Make
sure to preserve SLSA metadata when unpacking as well.
Catalyst runs builds with copies of the portage/coreos overlays in a
chroot, which prevents us from accessing the git metadata necessary to
create provenance information. Copy some files over into the
root_overlay used by the toolchains catalyst build so that provenance
can be correctly captured.
install_cross_libs installs TOOLCHAIN_PKGS deps into /usr/$BOARD_CHOST,
so that TOOLCHAIN_PKGS binpkgs can be built. We also need binpkgs for
the TOOLCHAIN_PKGS deps so that we can install them into /build/$BOARD
later together with TOOLCHAIN_PKGS. This is where the flow is currently
broken. Due to a change in semantics, --emptyroot tries to rebuild host
packages as well, and dropping it leaves --onlydeps which results in no
binpkgs being built because they are already installed.
We can solve resolve this by reusing the dependency list generated by
install_cross_libs, and explicitly building those binpkgs. It is worth
remarking that this flow of building the toolchain binpkgs through
setup_board is not in use in Flatcar, because we normally build
toolchains with catalyst. We are interested in reviving it because we
want to build everything with SLSA provenance information.
`./setup_board --nousepkg --nogetbinpkg` currently fails with a
circular dependency due to pulling in the whole systemd-cryptsetup-udev
dependency chain. This is due to several issue:
* `emerge --root=$ROOT --emptytree` considers ROOT=/ to also be empty,
so it pulls in all host packages. This must've not always been the case.
So we need to pipe the dependency package list through `egrep $ROOT`
to filter only those that would get installed into the desired ROOT
* if SYSROOT=/ and not SYSROOT=ROOT, then virtual/os-headers is missing
from $ROOT package list
* the final filter expression tries to previously looked like this:
(=sys-devel/gcc|sys-devel/binutils-0.9) which also matches
sys-devel/gcc-config and sys-devel/binutils-config, which are
necessary dependencies. Rework the match expression to not filter
those out.
