That way we can see a report of what emerge is going to do and the
status of the use flags for the installed packages. The downside is
that we are going to have reports about using deprecated and
unsupported profile in even more places.
Emerge flags are cryptic in general, but short flags even more so, so
expand them. While at it, I noticed some places where bash arrays
could be used, so convert those places too.
Normally `ln -sf path/to/target at/name` will create a symlink at
`at/name` that points to `path/to/target`. But if `at/name` already
exists and is a directory or a symlink to some other directory, then
this command will create a symlink at `at/name/target` pointing to
`path/to/target`. There is an ambiguity between 1st and 3rd form of
`ln` (please refer to `man ln` for the available invocation forms). It
can be disambiguated by using the `-T` flag to force the 1st form.
In our case, if `/etc/portage/patches` symlink already existed and was
pointing to `<coreos-overlay>/coreos/user-patches`, we ended up with a
useless symlink at `<coreos-overlay>/coreos/user-patches/user-patches`
pointing to `<coreos-overlay>/coreos/user-patches`.
When setting up portage configuration, we set up a symlink in
${ROOT}/etc/portage/patches that points to the coreos/user-patches
directory inside the coreos-overlay. That way, we can add our custom
patches to coreos-overlay without the need for moving the packages
from portage-stable to coreos-overlay only to apply an extra patch.
install_cross_libs installs TOOLCHAIN_PKGS deps into /usr/$BOARD_CHOST,
so that TOOLCHAIN_PKGS binpkgs can be built. We also need binpkgs for
the TOOLCHAIN_PKGS deps so that we can install them into /build/$BOARD
later together with TOOLCHAIN_PKGS. This is where the flow is currently
broken. Due to a change in semantics, --emptyroot tries to rebuild host
packages as well, and dropping it leaves --onlydeps which results in no
binpkgs being built because they are already installed.
We can solve resolve this by reusing the dependency list generated by
install_cross_libs, and explicitly building those binpkgs. It is worth
remarking that this flow of building the toolchain binpkgs through
setup_board is not in use in Flatcar, because we normally build
toolchains with catalyst. We are interested in reviving it because we
want to build everything with SLSA provenance information.
This change introduces a containerised SDK as a replacement for cork SDK
operations. It also simplifies versioning by removing the need for
manifest repos as well as usage of the "repo" tool by use of git
submodules for coreos-overlay and portage-stable.
The following feature scripts are added:
- run_sdk_container: Run a command in an SDK container, using the
current scripts repo + ebuild submodules.
current scripts repo + ebuild submodules.
- bootstrap_sdk_container / build_sdk_container_image: Bootstrap a new
SDK and create an SDK container from the resulting SDK tarball.
The following additions have been made to SDK scripts:
- setup_board: add --pkgdir parameter to use a custom binary packge
directory.
Signed-off-by: Thilo Fromm <thilo@kinvolk.io>
This updates the default settings in build scripts to use
https://mirror.release.flatcar-linux.net/
instead of the google storage bucket if no binhost or FLATCAR_DEV_BUILDS
is specified.
Defaults are updated for
* update_chroot (runs at SDK initialisation time)
* setup_board (creates /boards/[ARCH]/) chroots
* the development container
* set_version
The scripts that invoked `cros_workon` without specifying a path to
the script were not calling the internal `cros_workon` directly, but
rather a copy installed in `/usr/bin/cros_workon`.
`/usr/bin/cros_workon` comes from the `coreos-base/cros-devutil` and
is a wrapper script that sources `common.sh` file to figure the
location of the `scripts` and finally invokes the internal
`cros_workon`. Curious thing is that the sourced `common.sh` comes
from the `/usr/lib/crosutils` directory and contents of the directory
come from the `dev-util/crosutils` package. And that `common.sh` is
different from the one in the scripts directory, but fortunately the
part that detects the path to the `scripts` directory is the same. I'm
not sure where where exactly the copy of `common.sh` in
`/usr/lib/crosutils` comes from - likely from somewhere in
`https://chromium.googlesource.com/chromiumos/platform/crosutils`.
Just cut the middle layers and call the internal copy of `cros_workon`
directly.
The `--jobs` parameter that some scripts defined was not used anywhere
in jenkins or mantle. So the value of the parameter always ended up
being equal to `${NUM_JOBS}` set by `common.sh`. Also, even if the
`--jobs` parameter was used for some script, that script usually
didn't forward the jobs value to other scripts, so the other scripts
ended up using `${NUM_JOBS}` again. Also, the `${FLAGS_jobs}` variable
was used by some functions in the build library, and those functions
were sometimes invoked by scripts that didn't define the
`${FLAGS_jobs}` variable. It is tedious to track which script should
actually define the parameter, and where it should be forwarded.
Just get rid of this half-working pretense. If you want to affect how
many jobs `emerge` uses, export the `NUM_JOBS` environment variable
before calling any script.
For `EMERGE_FLAGS` and `REBUILD_FLAGS` we unconditionally specify the
`--jobs` flag's value to `${NUM_JOBS}` because they are passed to
`emerge`. On the other hand we drop the `--jobs` parameter from the
`UPDATE_ARGS` variable, because this variable passed to `setup_board`
or `update_chroot`, which don't have this flag any more.
eselect calls "portage get_repo_path /build/amd64-usr coreos" at some
point. Before updating portage, portageq seemed to take all the
information not from /build/amd64-usr (which at the time contained no
repo information at all), but rather from /. The newer version of
portageq seems to respect the passed root now, so it actually tries to
consult the nonexistent repos configuration in /build/amd64-usr and
fails. To avoid the failure, perform the copying of the configuration
files earlier.
For the less common case where binpkgs are not used, restructure
this so that it builds binpkgs in /usr/${CHOST} without installing
them, use those binpkgs to initialize /build/${BOARD}.
When --usepkgonly is mixed with the right update flags it will
re-install any binary packages that have a newer build. Since the full
set of SDK and board packages are rebuilt quite often this leads to
excessive reinstalling.
Run the emerge of the SDK sys-boot/grub again to pick up any config
changes that were made in setup_board. This late emerge is needed
to build grub modules for architectures that are different from the
host.
Signed-off-by: Geoff Levand <geoff@infradead.org>
setup_qemu_static only needs to be called once, and can
be done early so that it is available. Move the call
of setup_qemu_static from build_packages to setup_board.
Signed-off-by: Geoff Levand <geoff@infradead.org>
Set the host grub use flags to build arm64 grub support when BOARD is
equal to arm64-usr.
From: Andrej Rosano <andrej@inversepath.com>
[Move to case statement]
Signed-off-by: Geoff Levand <geoff@infradead.org>
A step in reducing the amount of initialization code required: drop
needless symlinks under /usr/local/portage to the portage trees. Just
configure portage to point directly at the source instead. Only crossdev
remains in that location because it is a locally managed overlay.
For some reason package moves are not handled automatically in the board
build roots. Add explicit calls to emaint to update cached binary
packages, installed packages, and the world file.
setup_board's --latest_toolchain option seems to be a left over from a
removed feature. It's not used, so remove it.
Signed-off-by: Geoff Levand <geoff@infradead.org>
commit 03e8d451bf (setup_board: setup
gdb wrapper and debug symbol path) added hard coded paths for the
symbol path. Change those to use the BOARD_ROOT variable.
Signed-off-by: Geoff Levand <geoff@infradead.org>
The linux-info eclass may trigger a fatal error if it is unable to check
a package's required kernel options. Even when the error isn't fatal the
warnings add a lot of clutter to our build output.
Fix 98684560 which in turn tried to fix 0d29e735. This time the option
to download binary packages was lost so building from scratch worked but
not the normal usage of using binary packages. *sigh*
Using parallel_emerge has been disabled by default for all commands
except build_image for quite a while now, build_image kept it just
because it was still a bit faster than normal emerge. Keeping
parallel_emerge complicates future changes to build_image so it needs to
drop it entirely. Since that means nothing uses it by default we might
as well just rip out support for it entirely.
