Stop relying on github redirects, they are a mixed blessing and using
them broke emerge-gitclone inside dev-container in silent way. The
script could not find a desired revision of portage-stable or
coreos-overlay, because it tried to pull from kinvolk instead of
flatcar-linux github org. The redirects seem to hinder fetching a
specific commit, so the script pulled something else (HEAD or main?).
Previously before https://github.com/kinvolk/flatcar-scripts/pull/134,
`bootstrap_sdk` was looking at the wrong path
(/usr/lib/rust-*/rustlib/aarch64-unknown-linux-gnu instead of
/usr/lib/rustlib/aarch64-unknown-linux-gnu). As a result, Rust got always
removed and rebuilt in `install_cross_rust`, which resulted in
`flatcar-sdk/crossdev/dev-lang/rust/rust-1.54.0-1.xpak` being created.
Now legitimate changes of https://github.com/kinvolk/flatcar-scripts/pull/134
prevent the rebuild from happening. The path already exists in a stage4
SDK build, because the seed stage already has cross-compilers so the
Rust upgrade has all the right cross-paths.
That's why SDK builds with only stage4 failed when it tries uploading Rust
packages like the following. On the other hand, full SDK builds with stage1
to 4 worked well, because in that case Rust is rebuilt anyway.
```
INFO bootstrap_sdk: Uploading cross toolchain packages to
gs://flatcar-jenkins/developer/sdk/amd64/2021.08.04+dev-flatcar-master-3209
CommandException: No URLs matched:
/mnt/host/source/src/build/catalyst/packages/flatcar-sdk/crossdev/*
CommandException: No URLs matched:
/tmp/tmp.xyjXbCFhUc//mnt/host/source/src/build/catalyst/packages/flatcar-sdk/crossdev/*.sig
CommandException: 2 files/objects could not be transferred.
```
To fix that, we have to skip uploading packages when the crossdev
directory does not exist.
Debugged and suggested by @jepio
When performing a full bootstrap (stage1-4), the stage1 results are currently
discarded because of the logic in catalyst_build: the first build stage uses
the "seed" and every following stage uses the previous stages results *but*
stage1 is built before catalyst_build. So from the point of view of
catalyst_build, stage2 is the first one and uses the seed tarball.
To make sure stage1 results are used if it was built, set the SEED variable to
the latest stage1 location.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
The arm64 profiles don't specify SYMLINK_LIB=yes, which makes sense
since arm64 systems don't support multilib in the way that we are used
to from x86. What this means is that build artifacts are installed into
separate lib and lib64 directories. The root overlay installed in stage4
needs to check for SYMLINK_LIB before trying to create a symlink,
otherwise it fails to be applied because it collides with the directory
in the rootfs.
This uncovered a second minor issues - the rust toolchain bootstrap
scripts checked for /usr/lib64/rust*, but the ebuild installs to
/usr/lib/rust.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This change uses portage-stable and coreos-overlay from the local SDK
chroot (from /var/lib/gentoo/repos) in the stage 1 SDK bootstrap build.
This is part 2 of the SDK bootstrap stage 1 fix (part 1 is covered in
64d8a73ac0), which ensures stage 1 does
not introduce any changes in its ebuilds over the seed SDK.
The change also introduces an option to consciously divert from the
above enforcement by use of command line parameters:
--stage1_overlay_ref <gitref> will check out coreos-overlay and use
<gitref> for stage 1 instead of the
local SDK's
/var/lib/gentoo/repos/coreos-overlay
--stage1_portage_ref <gitref> will check out portage-stable and use
<gitref> for stage 1 instead of the
local SDK's
/var/lib/gentoo/repos/gentoo
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
This change updates the stage1 SDK bootstrap build to use local
("known good") package ebuilds only, preventing updated package ebuilds
to apply in stage 1. This fixes SDK build breakage we observed when
upgrading core libraries like readline.
The change also removes the seed update from stage 1 as it should not
be needed anymore now that we postpone any package updates to stage 2.
The following package ebuild repos are used for stage 1:
- for portage-stable, we simply copy /var/gentoo/repos/gentoo
from the SDK root.
- coreos-overlay is more complicated since ebuilds are missing from
the SDK. So we grok the version the SDK was built with from
/mnt/host/source/.repo/manifests/default.xml
and then we create a local stage 1 clone of
https://github.com/kinvolk/coreos-overlay.git
in which we then check out the revision noted in the default mnifest.
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
ROOT_OVERLAY variable is defined in terms of TEMPDIR. The TEMPDIR
variable is set to an empty value by catalyst.sh, which the two
scripts import. So ROOT_OVERLAY always ended up being located in
toplevel directory (i.e. `/`). But the TEMPDIR variable gets a
meaningful value after calling the catalyst_init function, so define
the ROOT_OVERLAY after the function is called.
Before, we were relying on the toolchains job to build and upload
packages that were part of the SDK. With this change, all packages that
should be part of the SDK are built and uploaded by the SDK job. The
toolchains job only builds toolchain packages specific for the release.
This change includes several adjustments done to both the SDK and the
toolchains jobs to make this work:
* Make the SDK job build all cross toolchains, including Rust
* Stop building Rust in the toolchains job and use the one in the SDK
instead.
* In toolchain_util.sh: detect when the symlink folder for crossdev
packages is missing and run crossdev to create it during
update_chroot setup.
* Make it possible to build the SDK starting from stage 4 instead of
stage 1, to make the SDK building faster for PR branches / nightlies
(full build should still be done for releases / weeklies).
This variable was semi-deprecated ages ago so `version.txt` could follow
a similar variable naming pattern to `os-release`. Finally drop usage of
it here in favor of `$COREOS_VERSION`.
The version of repos.conf/coreos.conf that catalyst needs isn't valid
for normal SDK chroots and causes env-update to spew errors when it is
run prior to update_chroot which configures portage properly.
SDK tarballs have a .DIGESTS file but it is created by catalyst instead
of the upload_image function. In order to support plain GPG signing but
not avoid re-generating .DIGESTS we need to move that code out of
upload_image to a new function. upload_files shouldn't do it itself
because it is also used for portage binary packages which shouldn't be
signed (there is no point, nothing would verify the signatures).
- Remove custom COREOS_* attributes from /etc/lsb-release
- Move dev image logic to dev_image_util
For extra fun fix detection of local host URL for devserver.
- Remove weirdly verbose "DESCRIPTION" format.
- Add COREOS_RELEASE_BOARD back to /usr/share/coreos/release
This is mostly just so update_engine and gmerge report the correct
board name to devserver, informative-only on prod images.
- Remove version info from /etc/gentoo-release
- Switch from 'track' to 'group' terminology.
Right now there is some funky logic to either use a previous build as a
seed or the current SDK tarball if it happens to have been downloaded.
This is a bit confusing and doesn't work reliably since it is reasonable
for there to be neither a previous build or the current SDK available if
the SDK chroot was created some time ago. Fix this by using the new SDK
library and always use the latest SDK, downloading it if needed.
Sync up bootstrap_sdk with other tools by using the common upload
functions. As part of this refactor release_util a bit to provide a
truly generic upload function.
The name "coreos-sdk-amd64-..." makes much more sense for general
distribution than "stage4..." so after catalyst is done rename the final
tarball and fixup the DIGESTS file to refer to the new name.
This uses Gentoo's catalyst for very thoroughly building images from
scratch. Using images based on this will eliminate some of the hackery
in make_chroot.sh for building up the sdk from a stock stage3 tarball.
For reference the procedure it performs is this:
1. snapshot: Grab a snapshot of portage-stable. Note that overalys are
not snapshotted.
2. stage1: Using a "seed" tarball as a build environment, build a
minimal root file system into a clean directory using ROOT=...
and USE=-* The restricted USE flags are key be small and avoid
circular dependencies.
3. stage2: Run portage-stable/scripts/bootstrap.sh
This rebuilds the toolchain. Probably not strictly necessary most of
the time but does super-duper-promise that the toolchain isn't linked
to or otherwise influenced by whatever was in the "seed" tarball.
4. stage3: Run emerge -e system to rebuild everything using the fresh
toolchain using the normal USE flags provided by the profile. This
will also pull in assorted base system packages that weren't included
in the minimal environment stage1 created.
5. stage4: Install any extra packages or other desired tweaks. For the
sdk we just install all the packages normally make_chroot.sh does.
