Currently, the scripts in src/scripts have multiple implementations
for handling when common.sh fails to load, some of which are buggy.
To simplify the boilerplate, these scripts now just exit if common.sh
fails to load. The shell itself will print the following message if
common.sh is not found:
/usr/lib/crosutils/common.sh: No such file or directory
BUG=chromium-os:32442
TEST=Run these scripts with and without common.sh installed.
Change-Id: Ie54420b6c649774f9cb039c14c80f4cf6c6ebc07
Reviewed-on: https://gerrit.chromium.org/gerrit/27058
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Use the ":" builtin command to prevent the increment of $ERROR_COUNT from
ever failing under strict mode (i.e. "set -e").
cros_generate_breakpad_symbols failed for some reason, and this is my best
guess as to the cause. That is, the post-increment would otherwise fail
when $ERROR_COUNT is zero because it would have a non-zero exit status.
BUG=chromium-os:31332
TEST=Manually ran script
Change-Id: Iec7fd9358c339414ccd3c2ca1fd598f124375f0b
Reviewed-on: https://gerrit.chromium.org/gerrit/23979
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Michael Krebs <mkrebs@chromium.org>
Tested-by: Michael Krebs <mkrebs@chromium.org>
The "function" keyword is superfluous, not in POSIX, is inconsistent
between bash files, and generally makes me angry. So convert every
instance to the form:
foo() {
BUG=None
TEST=`cbuildbot x86-generic-paladin` works
Change-Id: I97f5ca30a3edfef7222b1e08ac23917dc613b556
Reviewed-on: https://gerrit.chromium.org/gerrit/22467
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
This actually turns out to be a prime example of amdahls; generating
symbols for just the chrome binary takes ~120s, and with parallelization
in place for my hardware it now takes just over 120s.
For wall time, on my local hardware this brings it down from ~472 to
~120; can't get any faster w/out speeding up dump_syms itself at this
point.
The way this works is via passing the workers pid down a named pipe
once it's finished. The usual approch to bash parallelization is
a round robin loop over an array- this doesn't suffice here due
to the aforementioned chrome binary, thus the hash/control pipe
approach.
For output, we're relying on linux's atomic write gurantee for
pipes; all of our output passes through error/info/warn which
internally will chunk each line of text up into a separate
write (I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c added this).
Via this approach (and the explicit check and setup if necessary
of a pipe), we don't have to worry about interleaved output.
Due to the new approch, we no longer report how much raw data
was generated; instead we report the unique end result. This
is noteworthy since both versions are generating 1742989183 bytes
of data, but the actual ondisk is 1664241402. While that is
78MB of redundant data generated, it's less than 5% of our
generated data and likely is more trouble removing than it's
worth (it won't bring the runtime down at all after all).
Finally... while I realize this is a bit more complex than
most script tricks we do, frankly this route's pretty straightforward-
while we could rewrite this into python, we run the risk of bugs
during conversion, issues w/ multiprocessing having it's own races,
and generally a bit more pain then was worth the hour to hack this
up.
BUG=chromium-os:23050
TEST=cbuildbot x86-generic-full --remote
TEST=manual runs comparing output before/after
Change-Id: I5dd0f685bbb7f5e63e6a1f998e38156b76e80582
Reviewed-on: https://gerrit.chromium.org/gerrit/21940
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why. This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.
Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down. This includes
outputing the relevant stack trace, or at least what we can get of
it.
The 'die' function is now enhanced to automatically dump the trace
that lead to it. For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise. For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.
Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).
Before:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
<no output at all, just exit code 1>
With this CL:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
ERROR : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR : Backtrace: (most recent call is last)
ERROR : file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR : file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR :
ERROR : Command failed:
ERROR : Command '/bin/false' exited with nonzero code: 1
BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote
Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
This is a straightforward change- the intent is to up the debug
information available so we can deal w/ crashes like:
http://chromegw.corp.google.com/i/chromeos/builders/x86-alex32%20canary/builds/56
BUG=None
TEST=# Manual inducing of failures.
Change-Id: Ibea75d1467160fc7f07c21235d701692cec96d05
Reviewed-on: https://gerrit.chromium.org/gerrit/21931
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Of the 11713 lines output via this for a mario build, 96% of it
is stating "Using dump_syms.32 for 32-bit file <the-path>".
At one point that may have been useful; now it just obscures errors,
thus only output that info when verbose is turned on.
BUG=None
TEST=./cros_generate_breakpad_symbols; # enjoy the 438 lines of
# output rather than the 11,700 lines of output.
Change-Id: Iba9d1af3421c6b377af8388446521d106399ce25
Reviewed-on: https://gerrit.chromium.org/gerrit/21925
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
In our build environment, we have lots of symbolic links such as
"debug/.build-id/5c/a06545ff3812557dc5fd259db4e905c5dc3484.debug" that just
link to another .debug file. In our case, those links don't actually point
to the right place, so we've just been seeing lots of "Binary does not
exist" warnings. But it would also be undesirable to actually generate
symbols for these even if their targets *did* exist.
BUG=None
TEST=Ran cros_generate_breakpad_symbols
Change-Id: I050c349f7fefbdf922d54250379bad9b3db073d7
Reviewed-on: https://gerrit.chromium.org/gerrit/19182
Commit-Ready: Michael Krebs <mkrebs@chromium.org>
Tested-by: Michael Krebs <mkrebs@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
'dump_syms' was recently changed to be built 64-bit, with a 32-bit version
available as 'dump_syms.32'. This change makes use of the 32b dump_syms if
a file is built 32b. For almost all targets nowadays, we should only see
64b files, but things like the installer can still be 32b. Also see
https://gerrit.chromium.org/gerrit/14835.
BUG=chromium-os:22778
TEST=Ran cros_generate_breakpad_symbols on 32/64-bit executables
CQ-DEPEND=I6551fe22fb0caebd3584f76f95a543e9a91b7e1b
Change-Id: I780c20eeb745a919dbe130cf3cede7ec5ca6483a
Reviewed-on: https://gerrit.chromium.org/gerrit/18569
Commit-Ready: Michael Krebs <mkrebs@chromium.org>
Tested-by: Michael Krebs <mkrebs@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Rather than trying to use an old/stale common.sh, use the common.sh
from the invocation point- if invoked via /usr/lib/crosutils, use that
common.sh. If invoked via src/scripts/, use that, etc.
Trying to intermix it just introduces potential for bugs and invalidly
freezes common.sh api, thus the efforts to revert this and ultimately
revert the existing of a crosutils ebuild.
BUG=chromium-os:27201
TEST=cbuildbot x86-generic-full
Change-Id: I4c6c5fbade3d28c71752bd4c44dccad49af52ec0
Reviewed-on: https://gerrit.chromium.org/gerrit/18303
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
dump_syms can only dump the symbols of an executable with the same ELF
format as itself. Some recent build configurations now have binaries with
differing ELF formats. For example, issue 25468 has a 64-bit kernel where
everything else is 32-bit, and issue 25466 indicates there's a test that
contains a 32-bit executable.
BUG=chromium-os:25496, chromium-os:25468, chromium-os:25466
TEST=Ran cros_generate_breakpad_symbols on 32/64-bit executables
Change-Id: I15f5115585b3ed54ca7ae7b631216285baef8580
Reviewed-on: https://gerrit.chromium.org/gerrit/14835
Tested-by: Michael Krebs <mkrebs@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Michael Krebs <mkrebs@chromium.org>
Not all 64-bit platforms have names starting with amd64-, so we should
use portageq to get the board architecture name.
BUG=chromium-os:25228
TEST=./cros_generate_breakpad_symbols --board=x86-alex, amd64-corei7, link
Change-Id: I83769575dbd19112b929724995d0c97ed4df2b02
Reviewed-on: https://gerrit.chromium.org/gerrit/14444
Commit-Ready: Vincent Palatin <vpalatin@chromium.org>
Tested-by: Vincent Palatin <vpalatin@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Right now, the buildbot won't symbolize any crash that occurs in autotest
because all symbols in /usr/local/autotest are skipped by breakpad. Tweak
cros_generate_breakpad_symbols to not skip over these symbols so that
browser test crashes can be symbolized.
BUG=chromium-os:25061
TEST=Run cros_generate_breakpad_symbols and verify it still completes
successfully, and generates working symbols for autotest that can
be used to symbolize browser test crashes.
Change-Id: I072498060e78b373bd12c94ff95465878301cbce
Reviewed-on: https://gerrit.chromium.org/gerrit/14155
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
If we're generating new breakpad symbols, we don't need to keep the
old ones around. Keeping the old ones forever means the debug tarballs
get really large (e.g. >10GB).
This problem makes incremental bots get slower and slower over time. The
chromium.chromiumos bot spends over an hour archiving the debug symbols,
for example.
BUG=chromium-os:24994
TEST=Trybot run of archive stage, generating and uploading debug
symbols.
Change-Id: Ibf57db2561d29085434439ecd4f23e5cec1f598a
Reviewed-on: https://gerrit.chromium.org/gerrit/14040
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
This is very inefficent in calling dump_syms sequentially this can be parallelized easily and reduce the time it takes (~6mins)
BUG=chromium-os:21914
TEST=./cros_generate_breakpad_symbols --board=x86-alex, amd64-corei7
Change-Id: Ic9d3bbcd2dbccfaeb01e60d757549dcb5c45c03b
Reviewed-on: https://gerrit.chromium.org/gerrit/11747
Reviewed-by: Michael Krebs <mkrebs@chromium.org>
Tested-by: Anush Elangovan <anush@chromium.org>
dump_syms bails if you give it a debug info path and the file there
does not have debug info, instead of falling back to dumping
public/linkage symbols (and more importantly, call frame info).
Give it the chance to redeem itself.
Also, instead of not ever uploading CFI for x86 modules, strip it
for any architecture but only when the file is above some
threshold.
BUG=chromium-os:22373 chromium-os:22741
Change-Id: Iad6981efec537868296a6713b1d9ca0cdb750d28
Reviewed-on: https://gerrit.chromium.org/gerrit/11443
Commit-Ready: Ken Mixter <kmixter@chromium.org>
Reviewed-by: Ken Mixter <kmixter@chromium.org>
Tested-by: Ken Mixter <kmixter@chromium.org>
The problem here is that most were doing their exiting w/in a subshell;
exit within a subshell kills the subshell, not the parent. Not all scripts
were using set -e (which would pick up the failing subshell); as such
just rewriting them to remove the potential via eliminateing the subshelling.
Beyond that, removed a couple of custom (working, although non-standard)
approaches, and removed a duplicate common.sh sourc'ing w/in mk_memento_images.sh
TEST=force 'find_common_sh' to fail, note the scripts fails to exit
BUG=none
Change-Id: Ia1108a091a6399ad6aedd3cade4a107f4411686c
Reviewed-on: http://gerrit.chromium.org/gerrit/3905
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
from within the chroot.
It also fixes a number of style issues.
It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.
It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.
BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.
Review URL: http://codereview.chromium.org/6240018
Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
