It is not clear why this was forked originally. One reason was to avoid
the sys-apps/lsb-release dependency, but it probably wasn't just that.
It seems likely that the upstream package did not support cross targets
at the time. Now it does.
It appears that LTO was previously enabled by us following Gentoo rather
than through an explicit decision. They now disable it by default, so we
do likewise. It previously used "fat" LTO, which makes Rust especially
slow to build and reportedly made rustc slower than with "thin" LTO!
There seems little benefit in using thin LTO given that we rebuild Rust
almost as much as the packages that use it, plus we don't enable LTO
anywhere else.
We still avoid rustdoc to keep the size down using INSTALL_MASK. This
isn't as good as not building it in the first place, but this alone
isn't worth keeping a fork.
Cross targets are now handled via the admittedly experimental
RUST_CROSS_TARGETS support. This has been in place for a while, and I
think it is fairly widely used now. If it does disappear, it would
almost certainly be for something even better.
This also updates Rust from 1.80.0 to 1.80.1.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
It will patch gcc to respect ESYSROOT when cross-compiling, effectively
adding the --sysroot flag without the use of flags or wrappers. This
hasn't been merged into Gentoo yet, but it has been given the nod. When
it does get merged, it was only be for newer gcc versions than we're
currently using, so we'll need this user patch in the meantime
regardless.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Catalyst 4 has totally changed the way repositories are handled. It only
works when the name of the directory containing the repository matches
the configured name of that repository. This was not the case for us,
with the coreos repository residing in the coreos-overlay directory. We
wanted to move and rename our repositories anyway, but this is a big
change, so we'll do separately. For now, this just renames coreos to
coreos-overlay.
Catalyst 4 also ingests the main repository snapshot as a squashfs
rather than a tarball. It features a utility to generate such a
snapshot, but it doesn't fit Flatcar well, particularly because it
expects each ebuild repository to reside at the top level of its own git
repository. It was very easy to call tar2sqfs manually though.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The cros_workon tool has been replaced with a simpler flatcar_workon
tool based around git-r3.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
When running the build_packages script, one can encounter an error such as
'Error fetching binhost package info from.' This pertains to SDK packages (not
board packages). Since we have transitioned to the SDK container, the SDK
packages are no longer published independently from the container image.
The `dev-python/setuptools_scm` was renamed in first quarter of 2023
to `dev-python/setuptools-scm` (the underline was replaced with a
dash). To be safe, check for blockers with the new package name, just
in case.
That way we can see a report of what emerge is going to do and the
status of the use flags for the installed packages. The downside is
that we are going to have reports about using deprecated and
unsupported profile in even more places.
Emerge flags are cryptic in general, but short flags even more so, so
expand them. While at it, I noticed some places where bash arrays
could be used, so convert those places too.
Normally `ln -sf path/to/target at/name` will create a symlink at
`at/name` that points to `path/to/target`. But if `at/name` already
exists and is a directory or a symlink to some other directory, then
this command will create a symlink at `at/name/target` pointing to
`path/to/target`. There is an ambiguity between 1st and 3rd form of
`ln` (please refer to `man ln` for the available invocation forms). It
can be disambiguated by using the `-T` flag to force the 1st form.
In our case, if `/etc/portage/patches` symlink already existed and was
pointing to `<coreos-overlay>/coreos/user-patches`, we ended up with a
useless symlink at `<coreos-overlay>/coreos/user-patches/user-patches`
pointing to `<coreos-overlay>/coreos/user-patches`.
When setting up portage configuration, we set up a symlink in
${ROOT}/etc/portage/patches that points to the coreos/user-patches
directory inside the coreos-overlay. That way, we can add our custom
patches to coreos-overlay without the need for moving the packages
from portage-stable to coreos-overlay only to apply an extra patch.
Currently the os/sdk and os/toolchains job perform a chroot update whose
results are immediately discarded because the rest of the build uses a fresh
chroot and catalyst. Towards the end of a release period this can extend the
build time by about an hour (longer if rust is involved).
Introduce a `--setuponly` flag that bails after the chroot configuration, and
the skips chroot update.
Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
This updates the default settings in build scripts to use
https://mirror.release.flatcar-linux.net/
instead of the google storage bucket if no binhost or FLATCAR_DEV_BUILDS
is specified.
Defaults are updated for
* update_chroot (runs at SDK initialisation time)
* setup_board (creates /boards/[ARCH]/) chroots
* the development container
* set_version
The scripts that invoked `cros_workon` without specifying a path to
the script were not calling the internal `cros_workon` directly, but
rather a copy installed in `/usr/bin/cros_workon`.
`/usr/bin/cros_workon` comes from the `coreos-base/cros-devutil` and
is a wrapper script that sources `common.sh` file to figure the
location of the `scripts` and finally invokes the internal
`cros_workon`. Curious thing is that the sourced `common.sh` comes
from the `/usr/lib/crosutils` directory and contents of the directory
come from the `dev-util/crosutils` package. And that `common.sh` is
different from the one in the scripts directory, but fortunately the
part that detects the path to the `scripts` directory is the same. I'm
not sure where where exactly the copy of `common.sh` in
`/usr/lib/crosutils` comes from - likely from somewhere in
`https://chromium.googlesource.com/chromiumos/platform/crosutils`.
Just cut the middle layers and call the internal copy of `cros_workon`
directly.
The `--jobs` parameter that some scripts defined was not used anywhere
in jenkins or mantle. So the value of the parameter always ended up
being equal to `${NUM_JOBS}` set by `common.sh`. Also, even if the
`--jobs` parameter was used for some script, that script usually
didn't forward the jobs value to other scripts, so the other scripts
ended up using `${NUM_JOBS}` again. Also, the `${FLAGS_jobs}` variable
was used by some functions in the build library, and those functions
were sometimes invoked by scripts that didn't define the
`${FLAGS_jobs}` variable. It is tedious to track which script should
actually define the parameter, and where it should be forwarded.
Just get rid of this half-working pretense. If you want to affect how
many jobs `emerge` uses, export the `NUM_JOBS` environment variable
before calling any script.
For `EMERGE_FLAGS` and `REBUILD_FLAGS` we unconditionally specify the
`--jobs` flag's value to `${NUM_JOBS}` because they are passed to
`emerge`. On the other hand we drop the `--jobs` parameter from the
`UPDATE_ARGS` variable, because this variable passed to `setup_board`
or `update_chroot`, which don't have this flag any more.
Setting the invalid CCACHE_ variables resulted in strange failure
in projects depending on meson, newer version like 0.55.3. For example
systemd build fails like the following errors:
```
* ACCESS DENIED: utimes: /mnt/host/source/ccache
* ACCESS DENIED: utimes: /mnt/host/source/ccache
F: utimes
S: deny
P: /mnt/host/source/ccache
A: /mnt/host/source/ccache
R: /mnt/host/source/ccache
C: ccache cc /build/amd64-usr/var/tmp/portage/sys-apps/systemd-246/work/systemd-246-abi_x86_64.amd64/meson-private/sanitycheckc.c -o /build/amd64-usr/var/tmp/portage/sys-apps/systemd-246/work/systemd-246-abi_x86_64.amd64/meson-private/sanitycheckc.exe -O1 -pipe -pipe -D_FILE_OFFSET_BITS=64
```
We should not set up ccache at all, as it has been already disabled in
coreos-overlay repo.
The dev build SDKs are not in $FLATCAR_DEV_BUILDS/sdk but published under
$FLATCAR_DEV_BUILDS/developer/sdk.
Add an environment variable to specify where the SDK is to be found
but default to $FLATCAR_DEV_BUILDS/sdk if it is not specified.
From Jenkins this variable is exported as DOWNLOAD_ROOT_SDK.
Normally toolchains packages are prevented from upgrading. This
drops that restriction and explicitly removes old versions so that
conflicting tool profiles are not accidentally used.
This reverts commit 20975049b3.
This omits the toolchain packages' version-pinning flag for the
binutils package while it is being upgraded. It also removes older
versions installed in parallel that cause unwanted rebuilds.
When stable has the upgraded version, this can be reverted.
This seems to fix the ccache permission issues `update_chroot` hits
while building ninja.
The erroneous files were created as root:portage, so a umask of 002
should let other portage group members share them, which seems entirely
reasonable.
Instead of patching portage to support the `disabled` flag now we just
patch it to leave the `[gentoo]` section out of the default repos.conf.
Follow up to 585275b268
This variable was semi-deprecated ages ago so `version.txt` could follow
a similar variable naming pattern to `os-release`. Finally drop usage of
it here in favor of `$COREOS_VERSION`.
When --usepkgonly is mixed with the right update flags it will
re-install any binary packages that have a newer build. Since the full
set of SDK and board packages are rebuilt quite often this leads to
excessive reinstalling.
Before attempting to do @preserved-rebuild to fix linked against old
libraries remove any packages that no longer have corresponding ebuilds,
making them impossible to rebuild. This uses `--depclean`'s secondary
meaning: `--unmerge` but only remove packages without dependencies.
Rebuilds packages that are linked against old libraries that have been
upgraded or removed from the system. Skipping this can lead to shared
library checks looking ok in the build root but then built images have
broken library dependencies.
The distfiles cache is always under .cache in the repo tree but there is
a lot of extra logic to make that configurable along with compatibility
symlinks for previous locations. Just yank it all out.
