This variable was semi-deprecated ages ago so `version.txt` could follow
a similar variable naming pattern to `os-release`. Finally drop usage of
it here in favor of `$COREOS_VERSION`.
When --usepkgonly is mixed with the right update flags it will
re-install any binary packages that have a newer build. Since the full
set of SDK and board packages are rebuilt quite often this leads to
excessive reinstalling.
Before attempting to do @preserved-rebuild to fix linked against old
libraries remove any packages that no longer have corresponding ebuilds,
making them impossible to rebuild. This uses `--depclean`'s secondary
meaning: `--unmerge` but only remove packages without dependencies.
Rebuilds packages that are linked against old libraries that have been
upgraded or removed from the system. Skipping this can lead to shared
library checks looking ok in the build root but then built images have
broken library dependencies.
The distfiles cache is always under .cache in the repo tree but there is
a lot of extra logic to make that configurable along with compatibility
symlinks for previous locations. Just yank it all out.
A step in reducing the amount of initialization code required: drop
needless symlinks under /usr/local/portage to the portage trees. Just
configure portage to point directly at the source instead. Only crossdev
remains in that location because it is a locally managed overlay.
This is required for the eventual removal of `$PORTDIR` and
`$PORTDIR_OVERLAY` and ensures toolchain rebuilds/updates with
`./build_packages --nousepkg` don't erroniously try to use ebuilds from
`/usr/portage` inside of the SDK.
In order to fix up the build_toolchains script the crossdev overlay
needs to be setup properly, previously only setup_board did it.
Overall silences a lot of warnings and fixes an issue with crossdev:
/usr/bin/emerge-wrapper: line 48: /eclass/toolchain-funcs.eclass: No such file or directory
/usr/bin/emerge-wrapper: line 49: tc-arch: command not found
Normally Gentoo expects moving between major GCC releases to be a manual
step. In our case we want this to always be automatic, otherwise the GCC
version won't be switched at all.
Using parallel_emerge has been disabled by default for all commands
except build_image for quite a while now, build_image kept it just
because it was still a bit faster than normal emerge. Keeping
parallel_emerge complicates future changes to build_image so it needs to
drop it entirely. Since that means nothing uses it by default we might
as well just rip out support for it entirely.
To behave more like setup_board/build_packages update_chroot should
fully configure portage to make sure everything is accurate.
Now binhosts are defined in make.conf.host_setup so the static config in
coreos-overlays doesn't need to refer to version.txt. setup_board
already made this change in 7a43a07f.
Define path locations to reduce dependency between static configs in
coreos-overlays and the behavior of the scripts repo. Spreading
configuration across two repos makes everything harder to understand.
Eventually everything should either be defined in profiles in
coreos-overlays or minimal auto-generated config files here in scripts.
The latest release of portage uses repos.conf, so generate that
file when making the chroot. The crossdev overlay also needed
repo_name and layout.conf files.
- Remove custom COREOS_* attributes from /etc/lsb-release
- Move dev image logic to dev_image_util
For extra fun fix detection of local host URL for devserver.
- Remove weirdly verbose "DESCRIPTION" format.
- Add COREOS_RELEASE_BOARD back to /usr/share/coreos/release
This is mostly just so update_engine and gmerge report the correct
board name to devserver, informative-only on prod images.
- Remove version info from /etc/gentoo-release
- Switch from 'track' to 'group' terminology.
I would like to phase out parallel_emerge so disable it for all commands
other than build_image which is the only one that shows a noticeable
benefit from it (~2 min with --fast, ~3 min with --nofast).
Soon nss-usrfiles will be required to resolve users and groups properly.
To avoid potentially breaking during the transition we need to make sure
the package is installed early during the chroot upgrade process.
This makes it possible to toggle parallel_emerge just as other scripts
do. In other scripts update the help string to be more specific, the
--jobs option can be used to control parallelism.
This doesn't currently have any real impact but seems like a better
ordering since update scripts may need to emerge things.
Add some friendly version info logging.
The old chroot version system we inherited from Chromium OS always
assumes that a newly unpacked tarball is the latest and greatest but
since we version the SDK in the same way as target builds we can use
that version for these sorts of upgrade scripts and not make assumptions
about how late and great the starting tarball was.
The first upgrade script simply aborts to force the user to recreate
their chroot when moving from python 2.6 to 2.7.
As-is all of the various emerge wrapping scripts default to using
--getbinpkg whenever --usepkg is enabled. This means every single emerge
command made makes multiple synchronous HTTP requests to the upstream
binary package repository to get the latest package list. This gets
really frustrating when working remotely with limited network
connectivity. Using --usepkg with --nogetbinpkg will use locally cached
packages without making remote requests.
The current logic accidentally sets TOOLCHAIN_FLAGS when the --nousepkg
flag is used rather than appending it leading to possible flag loss.
While we're at it, use arrays.
BUG=None
TEST=`./setup_board --board=x32-generic --nousepkg` works now
Change-Id: I33627ec088afb649b4fdde42b933a4af1521675b
Reviewed-on: https://gerrit.chromium.org/gerrit/31150
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Make the following commands send stats to chromiumos-build-stats.appspot.com:
build_image
run_chroot_version_hooks
make_chroot
setup_board
update_chroot
BUG=chromium-os:33088
TEST=`cbuildbot --remote -p chromiumos/platform/crosutils alex-paladin`
confirm in the log that uploads succeded, and see them show up in queries
at chromiumos-build-stats.appspot.com.
Change-Id: I0280f91a3e7e0a0483c01c87072bc589003dbe95
Reviewed-on: https://gerrit.chromium.org/gerrit/28969
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
If the board we're building includes extra toolchains that aren't part
of our official set, we need to make sure cros_setup_toolchains still
tries to set them up for us.
BUG=None
TEST=`cbuildbot chromiumos-sdk` works
TEST=`./setup_board --board=x32-generic` installs an extra toolchain
Change-Id: I85f70177f82bdf295eb2cd8668a031317a3b6e61
Reviewed-on: https://gerrit.chromium.org/gerrit/24336
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Previously, if you were working on a cros_workon package on the host, it
would only get updated once -- after that, the package would never build
again.
With this change, we now rebuild host workon packages any time their
timestamp is modified. This replicates the same feature that we already
have for board packages.
BUG=chromium-os:12771
TEST=Verify that host workon packages are rebuilt when the timestamps change,
and only when the timestamps change.
Change-Id: I31ef1d83dc591161a7cb55c4af806ee4a4212cdd
Reviewed-on: https://gerrit.chromium.org/gerrit/24782
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Resubmit of If62b4f3973f02fd8e1deed35864c824a02ab0c22
This will be safe to land after
I2c4e21ec7e8c0c0cf58947e2b0a3a9edf7617a09
The breakage was a timing issue paired with people not always syncing
the complete tree. No changes to the CL are needed.
It is now used for:
- make_chroot (cros_sdk --bootstrap)
- update_chroot
setup_board is stripped of redundant code which was deprecated by this.
Also stripped is some usepkg logic in make_chroot, as that is now
exclusively source-only.
BUG=chromium-os:23032
TEST=trybot chromiumos-sdk
Change-Id: Ib888cf2886218622d9cfeebb17b9cd4462d06c89
Reviewed-on: https://gerrit.chromium.org/gerrit/22578
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: asharif <asharif@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why. This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.
Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down. This includes
outputing the relevant stack trace, or at least what we can get of
it.
The 'die' function is now enhanced to automatically dump the trace
that lead to it. For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise. For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.
Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).
Before:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
<no output at all, just exit code 1>
With this CL:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
ERROR : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR : Backtrace: (most recent call is last)
ERROR : file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR : file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR :
ERROR : Command failed:
ERROR : Command '/bin/false' exited with nonzero code: 1
BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote
Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Resubmit of If62b4f3973f02fd8e1deed35864c824a02ab0c22
This will be safe to land after I2c4e21ec7e8c0c0cf58947e2b0a3a9edf7617a09
It is now used for:
- make_chroot (cros_sdk --bootstrap)
- update_chroot
setup_board is stripped of redundant code which was deprecated by this.
Also stripped is some usepkg logic in make_chroot, as that is now
exclusively source-only.
BUG=chromium-os:23032
TEST=trybot chromiumos-sdk
Change-Id: Ic908eac712ac097e5c2062d3be70177e172aa924
Reviewed-on: https://gerrit.chromium.org/gerrit/20191
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>
It is now used for:
- make_chroot (cros_sdk --bootstrap)
- update_chroot
setup_board is stripped of redundant code which was deprecated by this.
Also stripped is some usepkg logic in make_chroot, as that is now
exclusively source-only.
BUG=chromium-os:23032
TEST=trybot chromiumos-sdk
Change-Id: If62b4f3973f02fd8e1deed35864c824a02ab0c22
Reviewed-on: https://gerrit.chromium.org/gerrit/17910
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
The latest gcc-config will accept the CHOST setting from the host env,
so export that variable since we already spent the time running portageq
to get it. This allows gcc-config to never execute portageq, and since
we run it multiple times, it helps in every invocation.
Along those lines, make sure we run gcc-config with `sudo -E` to pass
through the setting.
Since awk can handle regexps, merge the grep|awk into a single awk call.
Finally, there was a slight logic typo where we would always re-run
gcc-config even when the profiles were the same. This does not match
the comments, nor the spirit of this code, so fix the operator typo.
BUG=None
TEST=`./update_chroot` still works and doesn't re-select the toolchain
Change-Id: If73df81c014219f8f9ab5895e59d055696add777
Reviewed-on: https://gerrit.chromium.org/gerrit/19164
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
1. Remove options that are already deprecated:
--chromefromsource --chromebuild --chromebase
--crosbug12352_arm_kernel_signing
2. Remove the --retries option and associated function eretry.
3. Move seldomly used options out of --help to avoid confusing
developers. Developers who need these options can read the
source.
4. Alphabetize all options.
5. Add description of each script to --help.
BUG=chromium-os:27364
TEST=Run --help with each script. Full canary trybot run.
Change-Id: I95675b069781f7e950d75d32dbad744adce6b830
Reviewed-on: https://gerrit.chromium.org/gerrit/18194
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Update setup_board and build_packages to use common.sh from source
directory, so that changes in common.sh show up immediately.
BUG=chromium-os:27364
TEST=Run each script, testing that changes to common.sh are picked up.
Change-Id: I6847bbf8c486d14c58dfcbb9361cd606c11c817b
Reviewed-on: https://gerrit.chromium.org/gerrit/18245
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
