This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.
The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.
BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file
TEST=Verified that putting 'set -u' in my fake build_packages
caused an exit in print_time_elapsed, then fixed unbound variable
in print_time_elapsed, then rerun passed.
TEST=`cbuildbot -g <cl> --lkgm mario-paladin` passed
TEST=`cbuildbot -g <cl> --lkgm link-release` passed
Change-Id: Ieb714522cb32d7558b661e4ee1a197d1fce2c516
Reviewed-on: https://gerrit.chromium.org/gerrit/26084
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
Posting the emerge info for the native build isn't terribly useful, so
just punt it to clean up the build output slightly.
BUG=None
TEST=`./build_packages --board=x86-alex` still works
Change-Id: If4ddb437b68c5624f3bc1a7519ef409134ba687b
Reviewed-on: https://gerrit.chromium.org/gerrit/26044
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
This reverts commit 615fe57ff0.
This CL broke mod_image_for_recovery.sh with the following failure:
common.sh: line 684: $2: unbound variable
This bug is easy to fix, but suggests the CL needs more testing, so
we're going to revert for now and re-submit the CL once it's been
verified to pass on release trybots.
Example failure:
http://chromegw.corp.google.com/i/chromeos/builders/x86-mario%20canary/builds/2214/steps/Archive/logs/stdio
BUG=chromium-os:27355
TEST=None, since this is reverting a change that broke the tree.
Change-Id: I61d182e3dcee267a8d9dea3b547fa6a75140d974
Reviewed-on: https://gerrit.chromium.org/gerrit/26077
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: David James <davidjames@chromium.org>
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.
The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.
BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file
Change-Id: Iac071d1cc55a44335fc7c846960c7ae45fc93ed8
Reviewed-on: https://gerrit.chromium.org/gerrit/19401
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
BUG=chromium-os:27493
TEST=Verify that packages are only rebuilt when their modification times
change (on either the ebuild or the content).
Change-Id: Iac44e86455d12601a25c8d02f14aa69a4829a330
Reviewed-on: https://gerrit.chromium.org/gerrit/24677
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Running these as root does not make sense. Furthermore, it will fail
on calling cros-workon, which sometimes fails but most certainly will
not give correct information anyway.
BUG=chromium-os:30384
TEST=run build_packages/setup_board with/without sudo
Change-Id: I0cba72334369e35ba0e864c53fd81037ee9e0efa
Reviewed-on: https://gerrit.chromium.org/gerrit/23003
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why. This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.
Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down. This includes
outputing the relevant stack trace, or at least what we can get of
it.
The 'die' function is now enhanced to automatically dump the trace
that lead to it. For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise. For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.
Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).
Before:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
<no output at all, just exit code 1>
With this CL:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
ERROR : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR : Backtrace: (most recent call is last)
ERROR : file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR : file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR :
ERROR : Command failed:
ERROR : Command '/bin/false' exited with nonzero code: 1
BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote
Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
We have pretty good binpkg support for chrome now, as well as incremental
building (and ccaching), so we should be able to drop this flag now.
BUG=None
TEST=build_packages still works for x86-alex
Change-Id: I3f3bf7e513ef45c091cc8363753035cac07d2f97
Reviewed-on: https://gerrit.chromium.org/gerrit/20967
Reviewed-by: Anush Elangovan <anush@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
1. Remove options that are already deprecated:
--chromefromsource --chromebuild --chromebase
--crosbug12352_arm_kernel_signing
2. Remove the --retries option and associated function eretry.
3. Move seldomly used options out of --help to avoid confusing
developers. Developers who need these options can read the
source.
4. Alphabetize all options.
5. Add description of each script to --help.
BUG=chromium-os:27364
TEST=Run --help with each script. Full canary trybot run.
Change-Id: I95675b069781f7e950d75d32dbad744adce6b830
Reviewed-on: https://gerrit.chromium.org/gerrit/18194
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Update setup_board and build_packages to use common.sh from source
directory, so that changes in common.sh show up immediately.
BUG=chromium-os:27364
TEST=Run each script, testing that changes to common.sh are picked up.
Change-Id: I6847bbf8c486d14c58dfcbb9361cd606c11c817b
Reviewed-on: https://gerrit.chromium.org/gerrit/18245
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Now that the setup_board has an option for skipping the chroot update
logic, it'd be helpful (for buildbots) if build_packages had a flag to
pass through to setup_board for that too.
BUG=None
TEST=`./build_packages --board=tegra2` ran the chroot update
TEST=`./build_packages --board=tegra2 --skip_chroot_upgrade` skipped the chroot update
Change-Id: I662d90811d7bcf9782ef8cb06655940b0e5c7f06
Reviewed-on: https://gerrit.chromium.org/gerrit/17159
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
When running build_packages, the toolchain checking operations from the
update_chroot step and the initial merge are hard to differentiate, so
add an explicit `info` message so people can see what's going on.
BUG=None
TEST=`./build_packages --board=x86-alex` shows more info
Change-Id: I3593eb9ca56c620232bf155daf89b15fadb6547d
Reviewed-on: https://gerrit.chromium.org/gerrit/17604
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
This CL adds the --reuse_pkgs_from_local_boards flag to setup_board and
build_packages. With this flag, binary packages are shared between boards
that are built locally, so that the total time required to build several
boards is reduced.
This flag is only useful when you are not able to use remote binary packages,
since remote binary packages are usually more up to date than anything you have
locally. For this reason it is typically only used by buildbots which build
multiple similar full builds on the same machine.
BUG=chromium-os:26232
TEST=Test build using only local binary packages from a different
board.
CQ-DEPEND=I0353dd34f63f1b5c9c1a08882322279c6c4d82f5
Change-Id: I8a1d2687bd0d177de4464a80703b8b5a0b6c9f52
Reviewed-on: https://gerrit.chromium.org/gerrit/15775
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
We are changing the default emerge options to use --oneshot, so make sure
our scripts select the packages we care about for @world.
BUG=None
TEST=setup_board+build_packages for tegra2 adds pkgs to world
Change-Id: I5fc68c538a1a1c846bd9724f27ef717029d11d42
Reviewed-on: https://gerrit.chromium.org/gerrit/16838
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
The --jobs=<n> option wasn't actually being passed to parallel_emerge --
which meant it didn't do anything.
BUG=chromium-os:26827
TEST=Ran with --jobs=N and made sure only N parallel_emerge processes were
started.
Change-Id: I581fc5588b54e246acaefd0c7e528e55adf9ba8a
Reviewed-on: https://gerrit.chromium.org/gerrit/16570
Reviewed-by: Michael Krebs <mkrebs@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Now that emerge supports the flags needed to build packages, we don't
need to special-case it anymore.
BUG=chromium-os:24497
TEST=Trybot run.
Change-Id: I4ca1cd22c309f97fc14f1d7b9eede9128b7f0be1
Reviewed-on: https://gerrit.chromium.org/gerrit/13389
Reviewed-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Due to historical reasons, these scripts have existed at
chromiumos-overlay/chrome/scripts, but have belonged
in src/scripts (previously accessed by symlinks here
pointing to their locations).
Transfer them in, rather than relying on symlinks; this
gets us atomic commits w/in crosutils in the process.
Scripts were imported as of 604e3722af59164bc97aa5dcd1407e36495c96d7
BUG=chromium-os:24111
TEST=cbuildbot x86-generic-full
Change-Id: I689f7e05a25d427e24372f206bdb0779bf857820
Reviewed-on: https://gerrit.chromium.org/gerrit/12893
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
