If util-linux has a binary package it will be used, but if that binary
package has +udev it will pull in systemd. systemd has a loop that needs
to be broken too so if the util-linux loop breaker doesn't also handle
the systemd one it all falls apart.
In short the comment above the loop breaker code noting that we can try
this until it gets wonky. Well, it is wonky and we need to re-do how
build_packages works as a result. This is just a temporary workaround
until we figure out a larger restructuring.
Using parallel_emerge has been disabled by default for all commands
except build_image for quite a while now, build_image kept it just
because it was still a bit faster than normal emerge. Keeping
parallel_emerge complicates future changes to build_image so it needs to
drop it entirely. Since that means nothing uses it by default we might
as well just rip out support for it entirely.
The extra "dependency check" doesn't appear to be all that useful, so
don't waste time on it. Remove some unused command line options for
selecting which packages to build. Use new board-packages ebuild which
currently just includes 'coreos' and 'coreos-dev' but later can include
extra things like 'python-oem' which doesn't land in the base image.
This makes it possible to toggle parallel_emerge just as other scripts
do. In other scripts update the help string to be more specific, the
--jobs option can be used to control parallelism.
The stats upload has been removed so there is no longer a need to
capture the emerge output to parse the logged output. Remove a bit of
dead chromeos logic too.
Nothing in build_packages references the variables from
coreos_version.sh and nothing in our ebuilds depends on it being in the
environments so just remove it.
As-is all of the various emerge wrapping scripts default to using
--getbinpkg whenever --usepkg is enabled. This means every single emerge
command made makes multiple synchronous HTTP requests to the upstream
binary package repository to get the latest package list. This gets
really frustrating when working remotely with limited network
connectivity. Using --usepkg with --nogetbinpkg will use locally cached
packages without making remote requests.
Now that chromeos-dev-init and chromeos-test-init are explicitly
specified in chromeos-dev and chromeos-test remove the hardcoding
in build_packages
BUG=None
TEST=build and run on daisy and trybots on a variety of x86/arm
paladin/full builders
CQ-DEPEND=CL:Iaceab4dbc2c215ba899d17e78c6c22e60c0802c8
Change-Id: I3064117ecf5bbafbc0f340f46cc07d4f6c959559
Reviewed-on: https://gerrit.chromium.org/gerrit/41943
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Anush Elangovan <anush@chromium.org>
Tested-by: Anush Elangovan <anush@chromium.org>
Now that we have an ebuild for factory install shim, let's move to that
instead of base image.
BUG=chrome-os-partner:16712
TEST=Test network boot, install shim, RMA shim on Link.
Test install shim on Snow.
CQ-DEPEND=CL:39952
Change-Id: Ib6a7a598087647b44a4aeca970e293ea72023890
Signed-off-by: Vic Yang <victoryang@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39955
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Setting the flag will cause make.conf to set the ACCEPT_LICENSE
environment variable.
BUG=chromium-os:37312
TEST=remote trybots
Change-Id: Id478b7cfb8c37d363da46b1d74f33143e8ea159f
Reviewed-on: https://gerrit.chromium.org/gerrit/39801
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
This was identified in CL:39304, but not corrected before landing.
BUG=None
TEST=bash -n build_packages
Change-Id: I17cefc883e191d33a3569d7f53ab2e0a85417da9
Reviewed-on: https://gerrit.chromium.org/gerrit/39753
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
BUG=None
TEST=./build_packages --fetch-only, verify it just fetches.
Change-Id: I290ef23f13446c912724aa37b8c79269a9db6cc0
Reviewed-on: https://gerrit.chromium.org/gerrit/39304
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Currently, build_packages has a typo that causes it to only include "chromeos"
in dependency checks. This means that if a package outside of chromeos needs
to be updated in order to fix a conflict, build_packages will refuse to do that
and break. This caused a build break on keybuk's machine.
BUG=none
TEST=Try it on keybuk's machine and verify build_packages works again.
TEST=Remote trybot run.
Change-Id: I1acca0be47152ba93c30ba07c0bf643650906199
Reviewed-on: https://gerrit.chromium.org/gerrit/37452
Reviewed-by: Scott James Remnant <keybuk@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
We also remove some dead mod_for_test scripts.
BUG=chromium-os:9729, chromium-os:14091
TEST=build_packages, build_image dev test
CQ-DEPEND=If58678ab4fe7abd9142a619850be07249d159781
Change-Id: Iff6a143dbf8ce58878dc1b07629c52bbb7f02ded
Reviewed-on: https://gerrit.chromium.org/gerrit/32998
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
This way we can tweak buildbot configs to install specific packages rather
than all of ChromeOS (useful for firmware builders).
BUG=chromium-os:19140
TEST=`cbuildbot {{x86,amd64}-generic,daisy}-paladin {amd64,arm,x86}-generic-full stout-release` passed
TEST=`./build_packages nano` installed just nano
Change-Id: I9c316d7c390ebe6ceade97cf33c1387dc83fb887
Reviewed-on: https://gerrit.chromium.org/gerrit/33266
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Convert strings-used-as-lists to arrays. This makes the code nicer.
Further, package the variable initializations together. There is no
need to have them all intermingled. e.g. we set up EMERGE_FLAGS all
in the same region rather than doing some at the top, some in the
middle, and then more at the bottom.
BUG=chromium-os:19140
TEST=`cbuildbot {{x86,amd64}-generic,daisy}-paladin {amd64,arm,x86}-generic-full stout-release` passed
Change-Id: I593e01e6f450370545335144b06f814f306019df
Reviewed-on: https://gerrit.chromium.org/gerrit/33265
Reviewed-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Previously it was a dependency of chromeos-test which isn't correct
BUG=none
TEST=Build base and factory images
Change-Id: I133b937d1b56f96d5ebf3428172f9a4457c6bf5e
Reviewed-on: https://gerrit.chromium.org/gerrit/31156
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
Rather than forcing all consumers of DEFAULT_BOARD to remember to call
get_default_board, just do it for them automatically.
BUG=None
TEST=`cbuildbot {arm,amd64,x86}-generic-full` works
TEST=`./build_packages --help` shows correct default
Change-Id: I8d6ccb83babb2764a50692318eb9193c45fb3b39
Reviewed-on: https://gerrit.chromium.org/gerrit/17868
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.
The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.
BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file
TEST=Verified that putting 'set -u' in my fake build_packages
caused an exit in print_time_elapsed, then fixed unbound variable
in print_time_elapsed, then rerun passed.
TEST=`cbuildbot -g <cl> --lkgm mario-paladin` passed
TEST=`cbuildbot -g <cl> --lkgm link-release` passed
Change-Id: Ieb714522cb32d7558b661e4ee1a197d1fce2c516
Reviewed-on: https://gerrit.chromium.org/gerrit/26084
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
Posting the emerge info for the native build isn't terribly useful, so
just punt it to clean up the build output slightly.
BUG=None
TEST=`./build_packages --board=x86-alex` still works
Change-Id: If4ddb437b68c5624f3bc1a7519ef409134ba687b
Reviewed-on: https://gerrit.chromium.org/gerrit/26044
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
This reverts commit 615fe57ff0.
This CL broke mod_image_for_recovery.sh with the following failure:
common.sh: line 684: $2: unbound variable
This bug is easy to fix, but suggests the CL needs more testing, so
we're going to revert for now and re-submit the CL once it's been
verified to pass on release trybots.
Example failure:
http://chromegw.corp.google.com/i/chromeos/builders/x86-mario%20canary/builds/2214/steps/Archive/logs/stdio
BUG=chromium-os:27355
TEST=None, since this is reverting a change that broke the tree.
Change-Id: I61d182e3dcee267a8d9dea3b547fa6a75140d974
Reviewed-on: https://gerrit.chromium.org/gerrit/26077
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: David James <davidjames@chromium.org>
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.
The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.
BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file
Change-Id: Iac071d1cc55a44335fc7c846960c7ae45fc93ed8
Reviewed-on: https://gerrit.chromium.org/gerrit/19401
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
BUG=chromium-os:27493
TEST=Verify that packages are only rebuilt when their modification times
change (on either the ebuild or the content).
Change-Id: Iac44e86455d12601a25c8d02f14aa69a4829a330
Reviewed-on: https://gerrit.chromium.org/gerrit/24677
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Running these as root does not make sense. Furthermore, it will fail
on calling cros-workon, which sometimes fails but most certainly will
not give correct information anyway.
BUG=chromium-os:30384
TEST=run build_packages/setup_board with/without sudo
Change-Id: I0cba72334369e35ba0e864c53fd81037ee9e0efa
Reviewed-on: https://gerrit.chromium.org/gerrit/23003
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why. This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.
Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down. This includes
outputing the relevant stack trace, or at least what we can get of
it.
The 'die' function is now enhanced to automatically dump the trace
that lead to it. For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise. For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.
Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).
Before:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
<no output at all, just exit code 1>
With this CL:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
ERROR : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR : Backtrace: (most recent call is last)
ERROR : file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR : file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR :
ERROR : Command failed:
ERROR : Command '/bin/false' exited with nonzero code: 1
BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote
Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
We have pretty good binpkg support for chrome now, as well as incremental
building (and ccaching), so we should be able to drop this flag now.
BUG=None
TEST=build_packages still works for x86-alex
Change-Id: I3f3bf7e513ef45c091cc8363753035cac07d2f97
Reviewed-on: https://gerrit.chromium.org/gerrit/20967
Reviewed-by: Anush Elangovan <anush@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
1. Remove options that are already deprecated:
--chromefromsource --chromebuild --chromebase
--crosbug12352_arm_kernel_signing
2. Remove the --retries option and associated function eretry.
3. Move seldomly used options out of --help to avoid confusing
developers. Developers who need these options can read the
source.
4. Alphabetize all options.
5. Add description of each script to --help.
BUG=chromium-os:27364
TEST=Run --help with each script. Full canary trybot run.
Change-Id: I95675b069781f7e950d75d32dbad744adce6b830
Reviewed-on: https://gerrit.chromium.org/gerrit/18194
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Update setup_board and build_packages to use common.sh from source
directory, so that changes in common.sh show up immediately.
BUG=chromium-os:27364
TEST=Run each script, testing that changes to common.sh are picked up.
Change-Id: I6847bbf8c486d14c58dfcbb9361cd606c11c817b
Reviewed-on: https://gerrit.chromium.org/gerrit/18245
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Now that the setup_board has an option for skipping the chroot update
logic, it'd be helpful (for buildbots) if build_packages had a flag to
pass through to setup_board for that too.
BUG=None
TEST=`./build_packages --board=tegra2` ran the chroot update
TEST=`./build_packages --board=tegra2 --skip_chroot_upgrade` skipped the chroot update
Change-Id: I662d90811d7bcf9782ef8cb06655940b0e5c7f06
Reviewed-on: https://gerrit.chromium.org/gerrit/17159
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
When running build_packages, the toolchain checking operations from the
update_chroot step and the initial merge are hard to differentiate, so
add an explicit `info` message so people can see what's going on.
BUG=None
TEST=`./build_packages --board=x86-alex` shows more info
Change-Id: I3593eb9ca56c620232bf155daf89b15fadb6547d
Reviewed-on: https://gerrit.chromium.org/gerrit/17604
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
This CL adds the --reuse_pkgs_from_local_boards flag to setup_board and
build_packages. With this flag, binary packages are shared between boards
that are built locally, so that the total time required to build several
boards is reduced.
This flag is only useful when you are not able to use remote binary packages,
since remote binary packages are usually more up to date than anything you have
locally. For this reason it is typically only used by buildbots which build
multiple similar full builds on the same machine.
BUG=chromium-os:26232
TEST=Test build using only local binary packages from a different
board.
CQ-DEPEND=I0353dd34f63f1b5c9c1a08882322279c6c4d82f5
Change-Id: I8a1d2687bd0d177de4464a80703b8b5a0b6c9f52
Reviewed-on: https://gerrit.chromium.org/gerrit/15775
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
We are changing the default emerge options to use --oneshot, so make sure
our scripts select the packages we care about for @world.
BUG=None
TEST=setup_board+build_packages for tegra2 adds pkgs to world
Change-Id: I5fc68c538a1a1c846bd9724f27ef717029d11d42
Reviewed-on: https://gerrit.chromium.org/gerrit/16838
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
The --jobs=<n> option wasn't actually being passed to parallel_emerge --
which meant it didn't do anything.
BUG=chromium-os:26827
TEST=Ran with --jobs=N and made sure only N parallel_emerge processes were
started.
Change-Id: I581fc5588b54e246acaefd0c7e528e55adf9ba8a
Reviewed-on: https://gerrit.chromium.org/gerrit/16570
Reviewed-by: Michael Krebs <mkrebs@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Now that emerge supports the flags needed to build packages, we don't
need to special-case it anymore.
BUG=chromium-os:24497
TEST=Trybot run.
Change-Id: I4ca1cd22c309f97fc14f1d7b9eede9128b7f0be1
Reviewed-on: https://gerrit.chromium.org/gerrit/13389
Reviewed-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
