Commit Graph

216 Commits

Author SHA1 Message Date
Brian Harring
23da619f13 Force our umount w/in the chroot.
Do this via ensuring that any common.sh invoker
of raw umount (say a root script) sees our umount
path.

Additionally, inject into default profiles our override,
and via an upgrade scriptlet.

BUG=chromium-os:23443
TEST=manual validation, trybot.

Change-Id: Ie2514f6e8d2e10a19ab8d11c8056177bc1a2fb4d
Reviewed-on: https://gerrit.chromium.org/gerrit/32088
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-09-05 14:38:14 -07:00
Brian Harring
ece65e0633 Add a umount wrapper to suppress gvfsd/trashd breaking umount calls.
Specifically, detect if the umount failed due to current access, if so,
give it up to 9 more runs (w/ 1s pauses) continuing only if it's still
failing due to currently open files.

Via this, it should suppress the race of gvfs/trashd looking at
quick mounted/umounted pathways.

This CL is a two parter; this adds the script, and converts common.sh
consumers over to using the override.

The next CL will modify the chroot itself to ensure our script gets
picked up/used.

BUG=chromium-os:23443
TEST=trybot, manul validation.

Change-Id: I92dedd91d6133c2063b1e5dbbc1a68366844801d
Reviewed-on: https://gerrit.chromium.org/gerrit/32087
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-09-05 00:00:43 -07:00
Zdenek Behan
52970f47a5 common.sh: Change umount function to account for already deleted mounts.
BUG=chromium-os:31250
TEST=enter chroot; kill ssh-agent; exit chroot

Change-Id: Iea8e0fdd9a82047bb756a459dffc2ef59615074c
Reviewed-on: https://gerrit.chromium.org/gerrit/31929
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2012-08-31 15:41:45 -07:00
Jon Salz
8c8186c832 Add factory bundle overlay in /usr/local/factory/bundle.
CQ-DEPEND=CL:31631 CL:31632 CL:31633 CL:31636
BUG=None
TEST=cbuildbot --remote

Change-Id: Ie9b30d16e31fcba20009ccc3d7b4b24d2e9a8f3f
Reviewed-on: https://gerrit.chromium.org/gerrit/31636
Tested-by: Jon Salz <jsalz@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Jon Salz <jsalz@chromium.org>
2012-08-30 14:38:53 -07:00
Brian Harring
06d3c2e20b Pass more proxy vars into the chroot, rework sudoers.d maintenance.
In particular, put the sudoers.d setup into one script (making
updates to it easier in the future if necessary), and
centralize the proxied vars into a const in common.sh.

Thanks to Kevin McCray/Josh Triplett/Alexander Kanevsky for
pointing out the missing proxy variables, and fixes/cleanup.

BUG=None
TEST=https_proxy=blah cros_sdk -- bash -c 'echo $https_proxy'
TEST=build_packages behind a proxy.
TEST=cros_sdk --replace && \
     RSYNC_PROXY=blah cros_sdk -- bash -c 'echo $RSYNC_PROXY'

Change-Id: I3165882dfd9c8b52d25c2b26d7ff9242c84c91bd
Reviewed-on: https://gerrit.chromium.org/gerrit/31185
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Josh Triplett <josh@joshtriplett.org>
2012-08-24 13:26:03 -07:00
Mike Frysinger
c17a493bcc setup DEFAULT_BOARD by default
Rather than forcing all consumers of DEFAULT_BOARD to remember to call
get_default_board, just do it for them automatically.

BUG=None
TEST=`cbuildbot {arm,amd64,x86}-generic-full` works
TEST=`./build_packages --help` shows correct default

Change-Id: I8d6ccb83babb2764a50692318eb9193c45fb3b39
Reviewed-on: https://gerrit.chromium.org/gerrit/17868
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-08-14 10:17:31 -07:00
Mike Frysinger
27b5edb4db common.sh: fix pkgconfig install mask
Since pkgconfig is installed in the native libdir path, we need to
make sure to filter out all pkgconfig subdirs.

BUG=None
TEST=`./build_image --board=amd64-generic` no longer includes .pc files

Change-Id: Id8116190b6900ac86b433794551b250b6445d9d8
Reviewed-on: https://gerrit.chromium.org/gerrit/29682
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-08-09 01:57:02 -07:00
Olof Johansson
1222b2e736 common.sh: add /lib/modules/*/{build,source} to DEFAULT_INSTALL_MASK
Since we moved the source location during build, some of the security
testcases started reporting errors.

BUG=chromium-os:33392
TEST=build image, check /lib/module/*/{build,source} (none should be there)

Change-Id: I4656f043e4014f33dda03e1f788ba9190e0ce38a
Reviewed-on: https://gerrit.chromium.org/gerrit/29674
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Jim Hebert <jimhebert@chromium.org>
Commit-Ready: Olof Johansson <olofj@chromium.org>
Tested-by: Olof Johansson <olofj@chromium.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
2012-08-08 16:49:56 -07:00
Hung-Te Lin
76272be9de Add /firmware into INSTALL_MASK.
We will add more firmware-related tools (static AU programs, updater, netboot
firmware, firmware symbols, firmware for AU testing, ...) into /firmware for
buildbot archiver to output, and should not be picked into target chromiumos
image rootfs; so adding INSTALL_MASK should be the right approach.

BUG=chromium-os:33338
TEST=./build_packages; ./build_image

Change-Id: I64a4477a8ca1a5ed99e32d92767509f21a77c704
Reviewed-on: https://gerrit.chromium.org/gerrit/29411
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2012-08-07 21:20:31 -07:00
Jon Salz
d712ae9007 Mask /usr/local/autotest/tests from factory image.
BUG=None
TEST=build_image factory_test

Change-Id: Ic6e3d4cc9026d4a2d242655d05c431e52bb40136
Reviewed-on: https://gerrit.chromium.org/gerrit/27134
Tested-by: Jon Salz <jsalz@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Commit-Ready: Jon Salz <jsalz@chromium.org>
2012-07-11 01:24:53 -07:00
Matt Tennant
298f61ab25 build_packages: Upload command stats after each run.
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.

The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.

BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
  shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
  completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file
TEST=Verified that putting 'set -u' in my fake build_packages
caused an exit in print_time_elapsed, then fixed unbound variable
in print_time_elapsed, then rerun passed.
TEST=`cbuildbot -g <cl> --lkgm mario-paladin` passed
TEST=`cbuildbot -g <cl> --lkgm link-release` passed

Change-Id: Ieb714522cb32d7558b661e4ee1a197d1fce2c516
Reviewed-on: https://gerrit.chromium.org/gerrit/26084
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
2012-06-26 13:58:51 -07:00
David James
9d9efefe27 Revert "build_packages: Upload command stats after each run."
This reverts commit 615fe57ff0.

This CL broke mod_image_for_recovery.sh with the following failure:
  common.sh: line 684: $2: unbound variable

This bug is easy to fix, but suggests the CL needs more testing, so
we're going to revert for now and re-submit the CL once it's been
verified to pass on release trybots.

Example failure:
  http://chromegw.corp.google.com/i/chromeos/builders/x86-mario%20canary/builds/2214/steps/Archive/logs/stdio

BUG=chromium-os:27355
TEST=None, since this is reverting a change that broke the tree.

Change-Id: I61d182e3dcee267a8d9dea3b547fa6a75140d974
Reviewed-on: https://gerrit.chromium.org/gerrit/26077
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2012-06-25 18:57:10 -07:00
Matt Tennant
615fe57ff0 build_packages: Upload command stats after each run.
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.

The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.

BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
  shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
  completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file

Change-Id: Iac071d1cc55a44335fc7c846960c7ae45fc93ed8
Reviewed-on: https://gerrit.chromium.org/gerrit/19401
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
2012-06-25 15:16:36 -07:00
Mike Frysinger
6b1abb2a6f fix up function style
The "function" keyword is superfluous, not in POSIX, is inconsistent
between bash files, and generally makes me angry.  So convert every
instance to the form:
	foo() {

BUG=None
TEST=`cbuildbot x86-generic-paladin` works

Change-Id: I97f5ca30a3edfef7222b1e08ac23917dc613b556
Reviewed-on: https://gerrit.chromium.org/gerrit/22467
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-05-11 14:10:38 -07:00
Brian Harring
28bb01f53e Make default parallelization controllable.
Specifically, do this via env for the time being since each program
doesn't necessarily have an option (nor warrant one).

BUG=None
TEST=None

Change-Id: I26e7f06ad5d6a44a7826bfa8465b34154d21b6a3
Reviewed-on: https://gerrit.chromium.org/gerrit/22295
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2012-05-09 16:06:40 -07:00
Brian Harring
7f175a59e1 common.sh: output a backtrace and debug information on failure.
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why.  This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.

Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down.  This includes
outputing the relevant stack trace, or at least what we can get of
it.

The 'die' function is now enhanced to automatically dump the trace
that lead to it.  For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise.  For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.

Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).

Before:
./cros_generate_breakpad_symbols  monkeys --board=x86-alex
<no output at all, just exit code 1>

With this CL:
./cros_generate_breakpad_symbols  monkeys --board=x86-alex
ERROR   : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :   file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR   :   file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR   :
ERROR   : Command failed:
ERROR   :   Command '/bin/false' exited with nonzero code: 1

BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote

Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
2012-05-07 17:19:41 -07:00
Nirnimesh
e52cb76907 Remove reference to desktopui_PageCyclerTests
BUG=None
TEST=None

Change-Id: I9bc3bbd72da506fb746bde9604378a0355381db6
Reviewed-on: https://gerrit.chromium.org/gerrit/21700
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Nirnimesh <nirnimesh@chromium.org>
Tested-by: Nirnimesh <nirnimesh@chromium.org>
2012-05-02 18:47:43 -07:00
Zdenek Behan
1d5d3b56e4 common.sh: Use /proc/mounts instead of mtab for umount
Using /proc/mounts is safer because mtab may in rare cases get desynced.
The only significant difference in output is "X Y" instead of "X on Y".

BUG=chromium-os:30249
TEST=create a chroot; enter a chroot; exit a chroot
TEST=assortment of manual tests

Change-Id: I392290e6f52a677ee2d77d77e025ef60240b11b5
Reviewed-on: https://gerrit.chromium.org/gerrit/21499
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2012-05-02 17:47:01 -07:00
Mike Frysinger
0957a18357 virtual/linux-sources: rename from virtual/kernel
Upstream Gentoo renamed this virtual, so follow suite.

BUG=None
TEST=`cbuildbot amd64-generic-full` worked
TEST=`cbuildbot arm-tegra2-full` worked
TEST=`cbuildbot x86-generic-full` worked

Change-Id: I2721c85fe83f4ee8a90533eda14813697430e98e
Reviewed-on: https://gerrit.chromium.org/gerrit/18781
Reviewed-by: Olof Johansson <olofj@chromium.org>
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-03-27 09:12:29 -07:00
Joseph Hwang
ca63e041e5 Install uinput module in test image only
This CL adds uinput module in INSTALL_MASK so that the module
is not installed to non-test image. A mod_image_for_test script
is implemented to install the module in the test image if the
uinput module exists.

BUG=chromium-os:26707
TEST=Build a base image and a test image. Check the directory
  /lib/modules/<version>/kernel/drivers/input/misc
where current <version> is 3.2.7.
In the base image, there should be no uinput.ko,
while in the test image, there should be uinput.ko.
CQ-DEPEND=Ie96242c4d56403866a2298db2ba3bd6459248c1b

Change-Id: I0ca6599f80b9bb72cdc044fc97cdf990ce550edc
Reviewed-on: https://gerrit.chromium.org/gerrit/19032
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Mandeep Singh Baines <msb@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Ready: Joseph Shyh-In Hwang <josephsih@chromium.org>
2012-03-27 06:46:50 -07:00
Brian Harring
c03f260bb2 Revert "Install uinput module in test image only"
This reverts commit 5bf16bab5abb1cbf4f4acbc0d51d8aefa2c64091

Induced breakage: http://build.chromium.org/p/chromiumos/builders/x86%20generic%20incremental/builds/1259

Change-Id: I72170bc48cbc1cfe8b4913a9de1e5d8087525845
Reviewed-on: https://gerrit.chromium.org/gerrit/19023
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-03-23 22:44:04 -07:00
Joseph Hwang
50420556f8 Install uinput module in test image only
This CL adds uinput module in INSTALL_MASK so that the module
is not installed to non-test image. A mod_image_for_test script
is implemented to install the module in test image.

BUG=chromium-os:26707
TEST=Build a base image and a test image. Check the directory
  /lib/modules/<version>/kernel/drivers/input/misc
where current <version> is 3.2.7.
In the base image, there should be no uinput.ko,
while in the test image, there should be uinput.ko.

Change-Id: I02b557466a56e11c5dcc1649a9275d0c2a896f09
Reviewed-on: https://gerrit.chromium.org/gerrit/17209
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Mandeep Singh Baines <msb@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Ready: Joseph Shyh-In Hwang <josephsih@chromium.org>
2012-03-23 20:35:46 -07:00
David James
dee866c121 Remove --build_root from all scripts except mod_image_for_pyauto.sh.
For all scripts, --build_root defaults to /build and is never used. To
remove option clutter, I've deleted this option.

The only script which still references build_root in src/scripts is
mod_image_for_pyauto.sh, which seems to support it for grabbing pyauto
dependencies from a location other than /build/*. This might conceivably
be useful, so I haven't touched that script.

BUG=chromium-os:27364
TEST=Remote trybot run. git grep for references to build_root.

Change-Id: I502f7df0123a598fc62a4ef4ed847ceb182f65b8
Reviewed-on: https://gerrit.chromium.org/gerrit/18283
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-03-15 18:55:26 -07:00
David James
855afb7561 Clean up options to build scripts.
1. Remove options that are already deprecated:
   --chromefromsource --chromebuild --chromebase
   --crosbug12352_arm_kernel_signing
2. Remove the --retries option and associated function eretry.
3. Move seldomly used options out of --help to avoid confusing
   developers. Developers who need these options can read the
   source.
4. Alphabetize all options.
5. Add description of each script to --help.

BUG=chromium-os:27364
TEST=Run --help with each script. Full canary trybot run.

Change-Id: I95675b069781f7e950d75d32dbad744adce6b830
Reviewed-on: https://gerrit.chromium.org/gerrit/18194
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-03-15 15:54:43 -07:00
Chris Sosa
c9422fa467 Deprecate --test, --withdev, --factory, --factory_install from build_image.
Using these flags directly was deprecated a while back and is causing confusion
which way is the right way.  This CL removes all these FLAGS from build_image
(though keeps support of them in mod_image_for_test/image_to_usb invocations
in common.sh).  In this change I've also defaulted build_image to build the
developer image (and only the developer image).

BUG=chromium-os:27362
TEST=Been busy testing.  Have built the following combinations and verified them:

build_image base
build_image
build_image base dev test
build_image dev
build_image dev factory_test
build_image factory_install

Change-Id: Ie534c276a9ec571926964320ac176daa91b12a81
Reviewed-on: https://gerrit.chromium.org/gerrit/17386
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
2012-03-09 12:41:50 -08:00
Chris Wolfe
21a27b75e9 scripts: Add functions to detect an interactive terminal
With more interactive features appearing in the scripts, we need to
avoid hanging buildbots waiting for user input. This changes adds
utility functions to test whether and ensure that the execution is
interactive, and adds an assertion to the 'choose' function.

is_interactive: Check that both stdin and stderr are terminals.
assert_interactive: Die with an error if not interactive

Also tweaking 'choose' so that its menu is printed to stderr,
for consistency with read -p and select.

BUG=None
TEST=Modified setup_board to log the value of is_interactive, and
  then (unconditionally) attempt to display a menu.
  Running setup_board in a terminal showed interactive and the menu.
  Manually run cbuildbot x86-generic-pfq showed non-interactive and
  the assertion failure.

Change-Id: I768a37b8b85ce036edac7c9bb0fd1e0a2b92ecd5
Reviewed-on: https://gerrit.chromium.org/gerrit/16817
Commit-Ready: Chris Wolfe <cwolfe@chromium.org>
Reviewed-by: Chris Wolfe <cwolfe@chromium.org>
Tested-by: Chris Wolfe <cwolfe@chromium.org>
2012-02-28 13:57:05 -08:00
David James
184e39019d Fix --jobs flags to actually work.
The --jobs=<n> option wasn't actually being passed to parallel_emerge --
which meant it didn't do anything.

BUG=chromium-os:26827
TEST=Ran with --jobs=N and made sure only N parallel_emerge processes were
started.

Change-Id: I581fc5588b54e246acaefd0c7e528e55adf9ba8a
Reviewed-on: https://gerrit.chromium.org/gerrit/16570
Reviewed-by: Michael Krebs <mkrebs@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2012-02-23 21:42:34 -08:00
Chris Sosa
2b73600f7f Update the copyright year in common.sh.
BUG=None
TEST=visual.

Change-Id: I66d6a6901930cfeb6391df5064f247e7148e5c80
Reviewed-on: https://gerrit.chromium.org/gerrit/15747
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2012-02-12 16:45:48 -08:00
Gilad Arnold
207a7c7e7f common.sh: new function for a selection menu with a default option.
Defined a bash function choose(), which allows to present a menu and
prompting for a selection, with support for default choice. Usage is
commented appropriately.

Originally reviewed in CL I0d2f20dc8d62ce5fa18c10d9f8b51a46b2ddca5d.

BUG=chromium-os:26010
TEST=Tested in conjunction with a use in image_to_usb.sh, works fine.

Change-Id: I53a42a46a3c90fd486fead578bfbae248f64cfc2
Reviewed-on: https://gerrit.chromium.org/gerrit/15586
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
2012-02-09 13:31:41 -08:00
Brian Harring
feb04f77fe Require sudo usage to be non-interactive.
First, add a enable_strict_sudo helpers that scripts that are
sudo strict can invoke.  This does a sanity check bailing immediately
if invoked from a non sudo-keep-alive context.

Second, update enter_chroot.sh and make_chroot.sh to be strict.

While this is strict, that's the point.  The aim of this is to block
the previous sudo interactive mess for spreading, let alone reappearing
in scripts/code that has been cleansed.

BUG=chromium-os:18393
TEST=cros_sdk --replace; in the midst of it, do sudo -k.
     cros_sdk should thus bail out w/ an appropriate error.
CQ-DEPEND=I01bb1466cf027401fa387af7fad15e42fd33aea4

Change-Id: I76c5b87a812cc78c30a2eb1a0c56b9e438f4a98f
Reviewed-on: https://gerrit.chromium.org/gerrit/15294
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-02-08 16:19:14 -08:00
Gilad Arnold
083662748f common.sh: add a variable for default recovery image name.
This is a preparatory step for changes to image_to_usb.sh and
mod_image_for_recovery.sh.  In general, by migrating the default
recovery image name to common.sh we can improve the interoperability
between the different scripts.

BUG=chromium-os:26010
TEST=Obvious change; tested to see the mod_image_for_recovery.sh and
image_to_usb.sh still work.

Change-Id: I765d665b656d5d25b2d9b0e40bf5c05048c3f255
Reviewed-on: https://gerrit.chromium.org/gerrit/15507
Commit-Ready: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
2012-02-08 12:09:16 -08:00
Mike Frysinger
77c674b54e common.sh: add missing die to local shflags inherit
When we source the local copy of shflags, we don't check for errors like
we do when sourcing the external one.  Add a `die` call for that too.

BUG=None
TEST=`rm lib/shflags/shflags && (. ./common.sh)` now exits properly

Change-Id: I62c9b33dd96f2934691c43d53dea012097116ca1
Reviewed-on: https://gerrit.chromium.org/gerrit/15443
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
2012-02-08 08:58:19 -08:00
Mike Frysinger
669b28b666 common.sh: move output helper funcs up top the top
The shflags sourcing logic tries to use `die` before we've defined it, so
if there's an issue, we end up with:
	bash: die: command not found
and the shflags code doesn't actually get loaded.

So relocate these small helpers to the top so we don't have to worry about
when it's safe to use these things.

BUG=None
TEST=`rm lib/shflags/shflags && (. ./common.sh)` now exits properly

Change-Id: Ibdc268e6c081aa07679dc9fce76e5603b7217b20
Reviewed-on: https://gerrit.chromium.org/gerrit/15442
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-02-08 08:00:57 -08:00
Hung-Te Lin
d32c59fe47 crosutils: build factory test image by emerge commands
The factory test image was created by using rsync from build artifacts
in chroot, which has some concerns:
 - Runtime dependencies of autotest-factory won't be picked into image.
 - If a developer skips build_package and builds image by using only pre-built
   binary packages, he will get nothing in factory test image.
 - It's hard for developers to figure out how and when his changes will be
   merged to next build_image (cros_workon does not really work).
 - Output image will be definitely different for every developers, also the
   official build bots.
 - If developers never wipes his chroot (setup_board), the factory test image
   will grow until out of space. (For example, my environment outputs a 825M
   image while the official buildbot generates only 563M for same ToT source).

This CL changes image build command to using portage emerge, so that output
image can be prepared faster and smaller, and easier for maintenance.

BUG=chromium-os:3335
TEST=./build_packages; ./build_image --factory  # Image starts factory UI successfully
     # Also tried tests in test_list.all, seems fine.

     time ./mod_image_for_test.sh --factory --force_copy --no_inplace
     # time: 3m2s => 1m55s, factory test image data: 825/563M => 378M

     ./build_image --factory_install # factory install shim is also fine

Change-Id: I82b4505c74cd31e718aaff4a319d50b69b2c852c
Reviewed-on: https://gerrit.chromium.org/gerrit/14473
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
2012-01-20 17:54:11 -08:00
Mike Frysinger
bc36d041f0 get_default_board: detect invalid user settings
If people have whitespace in their .default_board setting, then the cros
utils will act weirdly.  Some work the same as if there was no whitespace
(probably because the variable is used unquoted) while others error out
with weird messages (because the variable is used quoted).

Update the helper function to only allow certain characters in the name.

BUG=None
TEST=`printf 'x86-alex\n' > .default_board; cros_workon list --all` works
TEST=`printf 'x86-alex\t\n' > .default_board; cros_workon list --all` errors out

Change-Id: Id83794c13bfddb7fb56b7f8ed8a375eefe6096e0
Reviewed-on: https://gerrit.chromium.org/gerrit/13151
Reviewed-by: Jason Glasgow <jglasgow@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-12-20 14:38:57 -08:00
Mike Frysinger
286b5928c8 enter_chroot: merge multiple mount requests into a single sudo
Every invocation of `sudo` delays things, so merge all of the mounts
into a single `sudo` command when possible.  This saves over 1 second
on initial execution (out of ~4 seconds total).

BUG=None
TEST=`cros_sdk --enter true` still mounts & unmounts properly

Change-Id: Ibc66507dc21250a81207d2f940645eaebe93c79c
Reviewed-on: https://gerrit.chromium.org/gerrit/10901
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
2011-11-14 13:42:21 -08:00
Zdenek Behan
07d2422c91 common.sh: Revert commit 9aa80dc457
This partially reverts and partially clarifies the code in the above
commit. It was still in use, just in an obscure place, so a reference
to that has been added.

BUG=chromium-os:21971
TEST=below
1) build_lib/generate-au-zip.py
2) cd /tmp/au-generator/; unzip au-generator.zip
3) exit chroot; cd $chroot/tmp/au-generator/
4) LD_LIBRARY_PATH="$(pwd)" PATH="${PATH}:$(pwd)" ./cros_generate_update_payload --image something_i_had --output x --outside_chroot

Change-Id: Idfc03937c4f3517c8567efcdebd1cb8ce134fc8e
Reviewed-on: https://gerrit.chromium.org/gerrit/11019
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2011-11-07 17:28:59 -08:00
Chris Sosa
131eaf5667 Fix message funcs to use $@ and fix callers to use it correctly.
Fixed all callers in my big CL to no longer have to wrap poorly using
\ and start from the beginning of the next line but rather pass in
an array of args i.e.

info "tacos" " are" " delicious".

BUG=None
TEST=Ran to see the errors in parse_build_image and manual eyeing.

Change-Id: I5eac8a5ae7a8d314dbc4e821ee33cf88213711d0
Reviewed-on: https://gerrit.chromium.org/gerrit/10823
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-10-28 11:08:58 -07:00
Chris Sosa
b0f5732449 Add ability to pass in specific images you want built rather than use flags.
This CL does two things.

First it introduces an argument interface that works with all the following
combinations:

factory_install | [base|dev[test|factory_test]] rather than use --test etc.

This does not build extra images.  If you just want a test image you can
run build_image --board=<board> test and all that's in your latest_dir is
chromiumos_test_image.bin.

Second, we only build what we ask and only finalize what we ask so on
an invocation of build_image test we only actually run cros_make_image_bootable
once saving 2 minutes on my machine (from 6:50->4:50 on multiple iterations).

BUG=chromium-os:22048
TEST=build_image test, all possible combinations of args through to start of
build.  Built factory_install using new and old methods and dev test with both
methods.  Also verified mod_image_for_test still works as advertises stand-alone.

Change-Id: I9fe2feb50a941c007214decd9ba1627012c050af
Reviewed-on: https://gerrit.chromium.org/gerrit/10621
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-10-27 16:33:07 -07:00
Mike Frysinger
9aa80dc457 cros_sdk: always pull shflags from local tree
Since we need this anyways (can't assume the host outside of the chroot
provides shflags), always source the local copy rather than searching
for versions provided by the chroot.

BUG=chromium-os:21742
TEST=`cros_sdk --enter` works
TEST=`./build_packages --board=amd64-generic` works
Change-Id: Ia390042131f59948472b124cfe4e273483eada6a
Reviewed-on: http://gerrit.chromium.org/gerrit/10231
Reviewed-by: David James <davidjames@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-10-19 14:15:12 -07:00
Mike Frysinger
b2e897bf46 enter_chroot: tweak awk to match POSIX substr requirements
POSIX says that the first arg to substr() counts from 1 which means
that awk implementations interpret a value of 0 in their own special
way.  With gawk, it does the logical thing (imo), but with mawk, it
does it by one less.

Example:
echo abc | mawk '{print substr($0, 0, 2)}'  -> a
echo abc | gawk '{print substr($0, 0, 2)}'  -> ab
echo abc | mawk '{print substr($0, 1, 2)}'  -> ab
echo abc | gawk '{print substr($0, 1, 2)}'  -> ab

So stick to the POSIX spec and count from 1.

While we're here, using "len" rather than "l" to improve readability.

URL=http://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html
BUG=chrome-os-partner:6026
TEST=set /usr/bin/awk to mawk; run `cros_sdk --enter -- true`; see everything unmounted
TEST=set /usr/bin/awk to gawk; run `cros_sdk --enter -- true`; see everything unmounted

Change-Id: I48f57d642730a0b3d909079027edf6ebf1bb9459
Reviewed-on: http://gerrit.chromium.org/gerrit/8163
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
2011-09-22 16:13:51 -07:00
Mike Frysinger
e8aec37165 cros_sdk: skip unmount phase if no mounts are found
Normally safe_umount_tree isn't called if there are no mount points in the
chroot, but there are some edge cases (like early setup errors) where it
may be possible.  So add a simple check to make sure that there are mount
points for us to work on before calling umount.

BUG=None
TEST=add early `die` before mounts; see no umount error output

Change-Id: I57e5022c3242c635097bceed44772f10eecf6a64
Reviewed-on: http://gerrit.chromium.org/gerrit/8026
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-21 17:15:59 -07:00
Mike Frysinger
1aa6124f1d cros_sdk: speed up unmounting when tearing down env
The teardown_env is a little slow when unmounting things because it runs
`sudo umount` once per mount point.  This is so that when things go wrong,
it can easily recover.  However, this slows down the common case at the
expense of the uncommon.

Refactor the code so that in the common case, we run one `sudo umount`.
When things do fail, we're a bit slower as we reparse the entire mount,
list, but that's fine as it's an error case.

BUG=None
TEST=enable `set -x`; run `cros_sdk true`; see all mount points unmounted in one shot

Change-Id: Iec98a7b9f51a77e90c30e6f6acae26e528b8c50d
Reviewed-on: http://gerrit.chromium.org/gerrit/7822
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2011-09-16 18:58:17 -07:00
Stefan Reinauer
b392153c54 rename /u-boot --> /firmware
Also drop find_*_component, because it is not used anywhere anymore.

BUG=chrome-os-partner:5459
TEST=none

Change-Id: Iea49a81538c22fee4b91c8b3e5582b5174282151
Reviewed-on: http://gerrit.chromium.org/gerrit/7425
Tested-by: Stefan Reinauer <reinauer@chromium.org>
Reviewed-by: Anton Staaf <robotboy@chromium.org>
2011-09-08 17:05:10 -07:00
Zdenek Behan
2811c16f55 crosutils: globally replace enter_chroot with cros_sdk
"cros_sdk" is a drop-in replacement of enter_chroot.sh. The only
important difference is in the calling path, specifically that
cros_sdk is in path but has to be called from $(pwd) being inside the
repo checkout, while enter_chroot.sh is called by explicit path.

Invariably, "./enter_chroot.sh" can be replaced, as pwd=src/scripts.
Calling by absolute path can be replaced by first changing directory
anywhere into the repo checkout, and then calling cros_sdk.

BUG=chromium-os:18750
TEST=run them

Change-Id: Ieff91a27bb419e1121361d5b3a11e4c87ff7a087
Reviewed-on: http://gerrit.chromium.org/gerrit/6273
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
2011-08-19 15:35:32 -07:00
Mike Frysinger
a1a06abeb7 common.sh: add quotes where appropriate
Much of this file quotes paths which could possibly contain breaking
chars (e.g. spaces), but a few are missed.  In general, this probably
doesn't break as no one is adventurous enough to use spaces, but might
as well do it for the fun of it.  And to be consistent.  One of those.

BUG=None
TEST=ran make_chroot/enter_chroot/build_packages; seemed to be OK

Change-Id: Ic64d60790f50bb812c87a4672e2a24fafbccd749
Reviewed-on: http://gerrit.chromium.org/gerrit/5759
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
2011-08-12 08:16:14 -07:00
Mike Frysinger
c2474ae596 common.sh: tweak color order for set -x love
If you trace any script with `set -x` enabled, bash will dump the color
variables and all of your output will be colored.  So move the "reset"
color to last so that the bleeding is localized to a few lines rather
than a few hundred.  There should be no functional difference.

BUG=None
TEST=ran setup_board with and w/out -x to create x86-generic to check result

Change-Id: I1e598cc4a38e9f217f2f670a00bd676bf70c2338
Reviewed-on: http://gerrit.chromium.org/gerrit/5804
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-08-11 19:40:59 -07:00
David James
0ea96e4fd6 Install recovery kernel directly to image, not to build dir.
The recovery kernel is different from the regular kernel, and
should be kept sandboxed so that it does not mess with our
build directory.

We also need to give the recovery kernel a different PKGDIR so
that it does not overwrite our prebuilts for the ordinary
kernel.

BUG=chromium-os:18691
TEST=build_image --factory_install, mod_image_for_recovery.sh

Change-Id: I678a94305d5f30bc2c95bf4e53cfe81b2ae9d8ef
Reviewed-on: http://gerrit.chromium.org/gerrit/5305
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-08-05 12:33:15 -07:00
Chris Wolfe
d13775f60c Add /usr/src to masked install paths.
This makes it possible for chromeos-kernel to include a source tree,
without the tree being installed to CrOS images.

BUG=None
TEST=Manual with modified chromeos-kernel

Change-Id: I9f1193277638e828ffa116c5a397609a3246c80b
Reviewed-on: http://gerrit.chromium.org/gerrit/4760
Reviewed-by: Chris Wolfe <cwolfe@chromium.org>
Tested-by: Chris Wolfe <cwolfe@chromium.org>
2011-08-05 06:26:18 -07:00
Gabe Black
83d8b824a3 Add some functions which will help transition to a single firmware directory.
This is the first in three stages of changes to transition from separate
"u-boot" and "coreboot" output directories to a single "firmware"
directory. This change/stage will give scripts a facility to easily be
compatible with both schemes. The second stage will transition all the
scripts to use the mechanism, and then finally the third stage will
switch the directories over. Some additional work may be needed if there
are file name clashes, although those sorts of problems would likely
already be present. These new functions could also optionally be taken back
out after the transition.

This change adds find_fw_component, find_u_boot_component, and
find_coreboot_component functions to common.sh. The first function provides
the bulk of the functionality, while the second and third wrap the first and
pass it parameters appropriate for u-boot or coreboot. The first function
accepts an override which it returns if set, and if not it checks for a file
in each of the directories it's passed in the second parameter one by one and
returns the first one it finds. If it doesn't find any it defaults to the last
one.

BUG=chromium-os:18107
TEST=Modified some u-boot related ebuilds to put some files in the firmware
directory. Ran the ebuilds and verified the files showed up where expected.
Modified the cros_write_firmware.sh script to use these new functions.  Ran
cros_write_firmware and saw it find the files in their new location.
Also verified that the firmware on the target was updated as expected.

Change-Id: Ic664e0e1c4a5b27efc9fb892ff5bd91cf1b5be56
Signed-off-by: Gabe Black <gabeblack@google.com>
Reviewed-on: http://gerrit.chromium.org/gerrit/5170
Tested-by: Gabe Black <gabeblack@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
2011-08-02 19:54:17 -07:00
Gaurav Shah
786ceeb389 Only use colored output if the terminal supports it
This also removes all the tput noise we are seeing in buildbot logs.
("tput: No value for $TERM and no -T specified", "tput: unknown terminal
"unknown", etc.)

BUG=none
TEST=manual below
src/platform/scripts$ TERM=unknown; ./enter_chroot (No more tput errors)

Change-Id: Ic74a0bd66d7e0cedcf7b65ec3a5f43b61b8276f9
Reviewed-on: http://gerrit.chromium.org/gerrit/4763
Tested-by: Gaurav Shah <gauravsh@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
2011-07-27 15:08:29 -07:00
J. Richard Barnette
f5029789d6 Remove unsued code from common.sh
BUG=None
TEST=setup_board, build_packages, build_image
TEST=grep all sources for the unused functions

Change-Id: Iaac5589aaba6f6f48aa00cdab76a7765f8d931ce
Reviewed-on: http://gerrit.chromium.org/gerrit/4580
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
2011-07-22 14:39:39 -07:00
David James
f540249413 Nuke chroot hacks from before March and before Gerrit transition.
These hacks are all at least three months old, and our official policy is to
remove hacks after two weeks. For future hacks, the plan is to use the chroot
upgrader script so that old chroots can be supported indefinitely.

Thanks to the Gerrit transition, there are probably few old chroots around
anyway that aren't going to be recreated anyway.

BUG=chromium-os:16151
TEST=Verify enter_chroot.sh still works.
Change-Id: Ie0c777aabbd70d2a0c88b59ddd71e9b67965ca94
Reviewed-on: http://gerrit.chromium.org/gerrit/2158
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-06-05 09:22:20 -07:00
Chris Sosa
fd2cdec118 Remove testing utilities and fix symlinks.
For run_remote_tests this required "fixing" restart_in_chroot if required.  Right now that script ->only<- works if its run on a scripts being called from src/scripts.  This change makes it more generic by doing this for any script called from within our source tree.

I borrow reinterpret_path_for_chroot from image_to_live.  Other CL to remove it from there.

Change-Id: If717beccd777ac178366d58b91521b9a62d55d85

BUG=chromium-os:11172
TEST=Ran them

Review URL: http://codereview.chromium.org/6730012
2011-03-24 16:06:59 -07:00
Simon Glass
6e448ae31a Select factory image when --factory is given
prepare_test_image would prepare the correct image (test or factory)
but its return value would always specify the test image. This meant
that image_to_usb.sh would pick up the wrong image.

This commit fixes this.

BUG=chromium-os:12377
TEST=./image_to_usb --from=... --to=/dev/sdb
also with --factory
also with --test

Make sure it picks up the correct image in each case.

Change-Id: I165b604b448a6c938a47e57a667bd5f001626efe

Review URL: http://codereview.chromium.org/6609016
2011-03-03 11:20:54 -08:00
Anton Staaf
6adf1a3458 Add optional logging prefix to info, warn and error messages.
BUG=chromium-os:11981
TEST=run cros_sign_bootstub with and without CROS_LOG_PREFIX set.

Change-Id: I320622d4161479dcc29ebdee68e160d94f1d3196

Review URL: http://codereview.chromium.org/6596108
2011-03-02 10:13:14 -08:00
Anton Staaf
6f5262d325 Add check_for_file and check_for_tool functions to common.sh
These functions make it easy to do testing in scripts for required
files and tools and provide useful error messages when the files
or tools are not found.

BUG=chromium-os:11981
TEST=run cros_sign_bootstub in configurations that test all paths in
these functions.

Change-Id: Ia63cfabd1eecba6473331b756486745d71e6b699

Review URL: http://codereview.chromium.org/6591056
2011-03-02 09:35:54 -08:00
Anton Staaf
dec0a5be72 Revert "Add optional logging prefix to info, warn and error messages."
The build bots run things a little different it seems.  I'll look into
what it is that they do that a normal run doesn't do that causes
this change to fail there.  But for now I'm reverting it.

This reverts commit 6d65c0f1b3.

BUG=None
TEST=None

Review URL: http://codereview.chromium.org/6594062

Change-Id: I7cdfb25c28f1c139981aaf7382cd3e9dbfd4446b
2011-02-28 17:02:53 -08:00
Anton Staaf
6d65c0f1b3 Add optional logging prefix to info, warn and error messages.
BUG=chromium-os:11981
TEST=run cros_sign_bootstub with and without CROS_LOG_PREFIX set.

Change-Id: I320622d4161479dcc29ebdee68e160d94f1d3196

Review URL: http://codereview.chromium.org/6597053
2011-02-28 16:21:53 -08:00
Chris Sosa
b885b80a33 Remove old logic ... and don't needlessly modifiy dev image.
Change-Id: I9b4047f2f124875553542df73e16cba6be41c561

BUG=chromium-os:chromium-os:8364
TEST=Ran it with pfq options.

Review URL: http://codereview.chromium.org/6527001
2011-02-16 15:33:11 -08:00
Simon Glass
142ca06512 Factored out the code to copy an image and modify it for test
into mod_image_for_test.sh rather than slightly different versions
of the same in image_to_usb.sh and image_to_vm.sh

Added a function to get a test image into common.sh

Added --inplace option to mod_image_for_test, which is the default,
and preserves the original behaviour. But using --noinplace it will
now do the copy for you.

Found that chromiumos_image.bin appears throughout the scripts, so added it and the test variant to common.sh

BUG=chromiumos-10126
TEST=run mod_image_for_test.sh with and without --noinplace
run image_to_usb.sh and image_to_vm.sh with both options
test on Seaboard that correct image is provided

Really we should have automated testing for these scripts

Change-Id: I5cfa91792c7fded35e7f4ca8f8f27c6b270817fb

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=4fc5227

Review URL: http://codereview.chromium.org/5271010
2011-02-09 13:39:43 -08:00
Greg Spencer
798d75f3be This starts to fix the scripts so that they load from /usr/lib/crosutils
from within the chroot.

It also fixes a number of style issues.

It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.

It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.

BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.

Review URL: http://codereview.chromium.org/6240018

Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
2011-02-01 22:04:49 -08:00
Anton Staaf
30acb0b926 Fix common.sh to work in it's new location at /usr/lib/crosutils.
If we are inside the chroot then we assume that the "gclient" root
is ~/trunk.  If we are outside the chroot we continue with the
previous search mechanism to find the "gclient" root.

Change-Id: Ia40de609ea596228fec2644ff3046e376b112b06

BUG=chromium-os:4230
TEST=run "src/scripts/cros_overlay_list --board tegra2 --variant seaboard" inside and outside the chroot, and under sudo.

Review URL: http://codereview.chromium.org/6265028

Review URL: http://codereview.chromium.org/6344016
2011-01-26 16:00:20 -08:00
Chris Sosa
fd38b5bdfe Revert "Fix common.sh to work in it's new location at /usr/lib/crosutils."
This reverts commit 856799a47f.

TBR=Anton for pfq.

Change-Id: Id592ee7a68c5f77300f5bf4f55192a372dd8a9da
2011-01-26 12:47:49 -08:00
Anton Staaf
856799a47f Fix common.sh to work in it's new location at /usr/lib/crosutils.
If we are inside the chroot then we assume that the "gclient" root
is ~/trunk.  If we are outside the chroot we continue with the
previous search mechanism to find the "gclient" root.

Change-Id: Ia40de609ea596228fec2644ff3046e376b112b06

BUG=chromium-os:4230
TEST=run "src/scripts/cros_overlay_list --board tegra2 --variant seaboard" inside and outside the chroot.

Review URL: http://codereview.chromium.org/6265028
2011-01-26 11:23:50 -08:00
Anton Staaf
9bcd84140a Add common function to determin board and variant values.
Change-Id: I9fe8b267a6fe96f3c4049d77cf866d0b6d36ecbe

BUG=None
TEST=setup_board --board tegra2 --variant seaboard

Review URL: http://codereview.chromium.org/6371008
2011-01-24 15:27:14 -08:00
Daniel Erat
e82f07c262 crosutils: Exclude Xvfb and Xnest from images.
These are used to run virtual or nested X servers, which we
don't do.  Removing the binaries saves 3.4 MB.

Change-Id: I528231a781f8a7a0202ca109f8064c3b11ff172b

BUG=chromium-os:8073
TEST=built an image and checked that the files were gone

Review URL: http://codereview.chromium.org/5990003
2010-12-21 13:39:22 -08:00
David Rochberg
b4f6b3c6a1 Make restart_in_chroot_if_needed run against scripts that are symlinks
This reverts the main change of
84e72fab23

and undoes the breakage of scripts that are symlinks to
...chromeos/scripts..., but at the cost of re-breaking scripts under
scripts/bin.  The correct fix is a combination of removing all the
symlinks to ...chromeos/scripts... and a more robust way of finding
the chroot path of the executable.  But that will have to wait until
someone else does it next week or I return from vacation

TEST=Ran all of these outside chroot.  None seemed more broken than when run in the chroot.  /setup_board  --board=x86-generic, ./cros_workon list --all --board=x86-mario, ./start_devserver , ./build_image --help, ./build_packages --help, ./cros_upgrade_chroot --help, ./enter_chroot.sh , ./customize_rootfs --help, ./run_remote_tests.sh , ./verify_rootfs_chksum.sh , ./set_shared_user_password.sh
BUG=n0ne

Change-Id: Ic2b7484e53e6a977ed6ae675fe99e109b385020d

Review URL: http://codereview.chromium.org/5946003
2010-12-17 11:55:40 -05:00
Anush Elangovan
84e72fab23 Revert "Factored out the code to copy an image and modify it for test"
Reverting for now since preflight fails.

This reverts commit 4fc5227fac.

TBR=sjg

Change-Id: I0ada89c15d0f5ecc07cb15b22d4084a37fa4ddb8
2010-12-08 23:22:15 -08:00
Simon Glass
4fc5227fac Factored out the code to copy an image and modify it for test
into mod_image_for_test.sh rather than slightly different versions
of the same in image_to_usb.sh and image_to_vm.sh

Added a function to get a test image into common.sh

Added --inplace option to mod_image_for_test, which is the default,
and preserves the original behaviour. But using --noinplace it will
now do the copy for you.

Found that chromiumos_image.bin appears throughout the scripts, so added it and the test variant to common.sh

BUG=chromiumos-10126
TEST=run mod_image_for_test.sh with and without --noinplace
run image_to_usb.sh and image_to_vm.sh with both options
test on Seaboard that correct image is provided

Really we should have automated testing for these scripts

Change-Id: I5cfa91792c7fded35e7f4ca8f8f27c6b270817fb

Review URL: http://codereview.chromium.org/5271010
2010-12-08 14:22:11 -08:00
David Rochberg
3b9107014b restart_in_chroot_if_needed can run scripts from bin/. cros_workon_now does this
BUG=chromium-os:9877
TEST=ran bin/cros_workon_now, cros_workon, set_shared_user_password

Note---this introduces another bashism to common.sh.  However, despite
comments to the contrary, common.sh does not actually parse under dash
without this change

Change-Id: I1e6b9751afa8341c100f3b8e0b96284835a53d17

Review URL: http://codereview.chromium.org/5444002
2010-12-02 10:45:21 -05:00
David James
ff07201a8c Turn on --fast on official builds.
The --fast support for build_packages is stable now, and should significantly
improve the speed of our official builders. We should turn it on so that
builders can finish their builds in less than 8 hours.

BUG=chromium-os:6706
TEST=Run ./enter_chroot.sh --official_build and verify that --fast is on by
     default now

Change-Id: I6ad126b9b6ce16ffc9887a7af22c2e3f85afbf42

Review URL: http://codereview.chromium.org/3418001
2010-11-30 13:22:05 -08:00
Raja Aluri
d44bcf9a45 Adding a script to generate au-geneate.zip file
Change-Id: Ifc47ed28dc3efc0e7ebd018f6703b36913ffd39c

BUG=8716
TEST=Ran the script inside the chroot to makesure it is generating the package.

Review URL: http://codereview.chromium.org/4425004
2010-11-11 17:46:53 -08:00
Chris Masone
d11ce17167 [crosutils] Mask out /usr/local/autotest-pkgs on install
BUG=8544
TEST=build image with --factory_install, ensure that image does not have data in /usr/local/autotest-pkgs

Change-Id: I5ed17d794392ec654bc122fa15f7ab8a7abbba2d

Review URL: http://codereview.chromium.org/4756001
2010-11-09 14:22:08 -08:00
Raja Aluri
c6e23cdb38 1. Not to assert inside chroot, if so desired.
2. Source shflags, if they are in the current directory

Change-Id: Id22e597a6fb71905f2d0ca5c4a1d94ce5a8a931f

Review URL: http://codereview.chromium.org/4177005
2010-11-02 15:55:57 -07:00
Will Drewry
9b7cb51697 common.sh: switch out echo -ne for printf and fix incorrect offset
echo -ne isn't portable but printf with octals is so this
makes the switch.

In addition, the current offset is off by 3 and is updating the UUID space.
This was from style cleanup and all functional test passed - of course.
This change fixes that too.

TEST=built and installed and checked ro enforcement works
BUG=chromium-os:7972

Change-Id: Ifb3cb2c6f3219af819baff5750453439e1ba6edf

Review URL: http://codereview.chromium.org/3997001
2010-10-20 18:11:24 -05:00
Will Drewry
55b42c94ca cros_make_image_bootable, mount_gpt_image, common.sh: root filesystem changes: ext2, ro by default
This change makes more of the root filesyste metadata static across builds, but
more can be done there.  It also changes the root filesystem to use ext2 as
we don't need journaling in normal mode.  Optionally we could use ext3 for
non-verified if desired (it's an easy change).

In particular, this change cleans up the following:
- clears the rootfs uuid
- labels it C-ROOT (instead of C-KEYFOB)
- removes reserved inodes and blocks

The major feature of this change, however, is that it adds two simple
helpers to common.sh: disable_rw_mount and enable_rw_mount.  They will
set high order byte (le) in the ro compat field to be 0xff.  This will
tell the kernel that the filesystem uses features R24-R31 which are safe
for use on the running kernel iff the filesystem is mounted read-only.

These functions are called in cros_make_image_bootable and
mount_gpt_image, respectively.  mount_gpt_image will always
enable_rw_mount and cros_make_image_bootable will disable_rw_mount if
--enable_rootfs_verification is true.

The approach is ugly but reasonably well contained.  If ext2 ever gets a
new revision and new features in the same range are introduced, then we
would be getting inconsistent behavior.  That said, it is unlikely that
that churmn will happen and if the impact is negative, it will ideally
show up during testing.

N.B., this will likely result in changes needing to be made to the
signing scripts in vboot_reference to ensure that rw mounting is
enabled/disabled in the same way (E.g., during stamping).

BUG=chromium-os:7468
TEST=- built x86-generic, imaged to usb stick, attempted to mount rw /dev/sdc3 on the host and was properly bounced.
     - booted to the image just fine on a dogfood device.
     - mod'd for recovery, then installed and booted.
     - mod_image_for_test runs with no errors; booted the resulting image as well
     - booted a factory_install with the pending dm changes
     - BVT passed with build_image x86-generic (vboot enabled)
     - [in progress] autotest that checks if the rootdev = /dev/dm-0 and
       then does a dumpe2fs | grep -q FEATURE_R31

Review URL: http://codereview.chromium.org/3916002

Change-Id: If4dcba7568a110f4e32627c916d9e5741e5e5414
2010-10-20 15:44:11 -05:00
Doug Anderson
0c9e88de6c Added a function to update chroot sudoers automatically.
This function is now called by enter_chroot.sh.  A separate change
will call the function from make_chroot

Change-Id: I4fc07c413e56db3e5e7617428f20f2ffc02790d7

BUG=chromium-os:7072
TEST=Took root out of sudoers and ran enter_chroot.sh script; saw that it was re-added.

Review URL: http://codereview.chromium.org/3909001
2010-10-19 14:49:39 -07:00
Nick Sanders
b3d04a0529 Filter more stuff from shim
Change-Id: I23ee03198acb7a792310799d7a72f8a19ca5445b

Review URL: http://codereview.chromium.org/3331021
2010-09-09 00:01:24 -07:00
Sean O'Connor
d52c5ea1bc Make enter_chroot spew logs to stderr.
It's sometimes useful to run a command in chroot and redirect/pipe stdout. Any objections to having enter_chroot send its chatter to stderr?

Review URL: http://codereview.chromium.org/3160024
2010-08-18 21:18:23 +02:00
David James
71fa14b024 Don't install autotest bits to the image.
Autotest installs stuff to the image via ssh, so there's no need to include
all of this stuff to the image. This saves disk space. Note that
/usr/local/autotest-chrome is currently already excluded from the image
because of a hack in the chrome ebuild, and that /usr/local/autotest is
usually also excluded from the image because autotest is not included
as a dependency of any packages that are installed.

In the factory setting, it looks like the factory has its own code for
installing autotests via rsync. Presumbably, if the factory tests are
managing the autotest directly, they won't want emerge mucking with the
directory as well on the image. Currently, emerge doesn't muck with the same
directory, but we probably want to ensure that it doesn't start doing that in
future to ensure sanity.

TEST=Built images. Modded for factory test. Verified that (a) build directory contains autotests, (b) regular images do not contain autotests; and (c) images modified for factory test do contain autotests

Review URL: http://codereview.chromium.org/3124010
2010-08-11 16:15:16 -07:00
Olof Johansson
6d49138087 fix print of total elapsed time
BUG=none
TEST=run build_image, happen to finish on a time with less than 10 seconds in it, watch the missing 0-padding

Review URL: http://codereview.chromium.org/3050047
2010-08-09 16:05:50 -05:00
Anush Elangovan
56de1dd852 Check for system shflags
This change checks for and sources shflags from /usr/lib before
depending on the gclient-style path.

TEST=None
BUG=4230

Review URL: http://codereview.chromium.org/3010045

Change-Id: I3085c2356c40c691a2a72bf79c1d9167162d6dca
2010-08-02 23:56:07 -07:00
David James
0366864a98 Enable fast build by default in build_image and mod_image_for_test.sh.
The fast build is stable enough now that we can enable it for everybody.
I ran the build constantly all weekend on two machines and I only ran into
one (rare) error, which I've fixed in a separate patch.

See <http://codereview.chromium.org/2856048/show>

TEST=Ran lots and lots of fast builds
BUG=none

Review URL: http://codereview.chromium.org/3059001
2010-07-28 17:08:29 -07:00
Anush Elangovan
2c7d19693e Move shflags into crosutils.git
Change-Id: I3aa6275362910addfaa25db9a1578a5496019c9e

Review URL: http://codereview.chromium.org/2983005
2010-07-13 15:24:54 -07:00
David James
fd7403ab89 Revert "Use --fast by default in build_image when --jobs is enabled."
See if this fixes the symlink problems we've been running into.

This reverts commit 8edffbaec5.
2010-07-09 12:00:17 -07:00
David James
8edffbaec5 Revert "Revert "Use --fast by default in build_image when --jobs is enabled.""
parallel_emerge prints a message every 30 seconds now so that buildbot doesn't
kill us early. Per discussion with cmasone, reverting the revert.

This reverts commit debaa3d8cf.
2010-07-08 21:21:41 -07:00
Chris Masone
debaa3d8cf Revert "Use --fast by default in build_image when --jobs is enabled."
This reverts commit f0fb864e8d.

This change needs to be reverted because it suppresses logging output
from the build process.  Thus, when something goes wrong...the
sherriff is left completely helpless in terms of debugging the issue.

TBR: davidjames@chromium.org
2010-07-08 21:10:23 -07:00
David James
f0fb864e8d Use --fast by default in build_image when --jobs is enabled.
Also update eretry to not pass in $EMERGE_JOBS anymore, in coordination with my previous change, which passed it in directly instead of depending on eretry to do so: http://codereview.chromium.org/2944004/show

TEST=Ran build_image --jobs 2 and build_packages --jobs 2
BUG=none

Review URL: http://codereview.chromium.org/2914004
2010-07-08 20:17:45 -07:00
Chris Sosa
d97985240a Check in to run unittests that uses src_test to build and run unittests.
TEST=Ran unittests with 2 test failures (powerd and update engine).
Will check with authors to fix these before moving the default over
to this.

Review URL: http://codereview.chromium.org/2837004
2010-06-16 14:29:50 -07:00
Nick Sanders
d250927093 Add Build Timer
Review URL: http://codereview.chromium.org/2859005
2010-06-16 03:50:04 -07:00
Chris Sosa
d44550272c Cleanup of adding pristine image and fixing of symlinks in developer image.
TEST=Tested with building a new image, looking in the output directory, running
the image and running vi.

Review URL: http://codereview.chromium.org/2075019
2010-05-20 10:14:06 -07:00
Amol Kher
9d2a7980cf Fixing DEFAULT and FACTORY mask to exclude usr/lib/debug folder
I verified that if I whack /usr/lib/debug folder and run build_image, it works fine. This is important since I plan to check in this issue first and then the fix to add debug flags.

Review URL: http://codereview.chromium.org/2113007
2010-05-19 17:30:29 -07:00
Tom Wai-Hong Tam
f87a3678dd Reduce the size of factory install shim.
Temporary approach to using INSTALL_MASK to suppress some non-relevant paths.
It reduces the size from ~426MB to ~163MB. Eventually, will remove it and
create a clean and separated ebuild for chromeos-factoryinstall.

Review URL: http://codereview.chromium.org/2084001
2010-05-17 16:06:33 +08:00
Chris Sosa
702618ff2b Move to using chromeos-test to emerge test dependencies in mod_image_for_test.
In addition unifies changes in mount_gpt_image and one-offs in mod_image_for_test
to consolidate gpt / var mounting.

Review URL: http://codereview.chromium.org/2064001
2010-05-14 12:52:32 -07:00
Olof Johansson
00171bdab6 Add /etc/init.d to INSTALL_MASK
Since we're using upstart, don't let packages drop startup scripts into
/etc/init.d.

BUG=none
TEST=Build image, inspect contents of /etc/init.d (or lack thereof, after this patch)

Review URL: http://codereview.chromium.org/2023006
2010-05-11 11:38:09 -07:00
Don Garrett
640a0585f5 Added restart_in_chroot_if_needed to common.sh.
Add restart_in_chroot_if_needed to common.sh, and modify the build scripts which referred to assert_inside_chroot to use it instead. The effect is that you don't ever have to explicitly enter_chroot.sh to build (still can, it work's fine).

Update mod_image_for_test.sh to use restart_in_chroot_if_needed

Review URL: http://codereview.chromium.org/1736025
2010-05-04 16:54:28 -07:00
Nasser Grainawi
cb5f3ebe52 Retry N-1 times with the --jobs flag, then drop it
Review URL: http://codereview.chromium.org/1545043
Patch from Nasser Grainawi <nasserg@quicinc.com>.
2010-04-21 08:46:49 -06:00
Chris Sosa
3d9a10b393 Fix install mask and move dev_mode to true when dev packages are installed
Review URL: http://codereview.chromium.org/1553028
2010-04-13 15:00:46 -07:00
robotboy
9891221b6e Add a safe_unmount function and use it in enter_chroot.
The safe_unmount function first tries a regular unmount,  If that
fails it warns and tries a lazy unmount.  If the lazy unmount fails
it dies.

Both unmounts take the -d option in case the mount is a loop device.

BUG=None
TEST=Entered and exited multiple chroots

Review URL: http://codereview.chromium.org/1593021
2010-04-12 14:08:14 -07:00
tedbo
373c3903c2 Add sudo_clobber and sudo_append functions so that we can "echo"
and "cat" stuff and pipe it to something that can write to a
file as root. For example:

echo "foo" | sudo_append /tmp/bar

will append "foo" to the file /tmp/bar as root. While

echo "foo" | sudo_clobber /tmp/bar

will truncate /tmp/bar and then write "foo" to the file.

Review URL: http://codereview.chromium.org/1610021
2010-04-12 10:52:40 -07:00
Chris Sosa
aa1a7fd63d Adds gmergefs. A method of remoting to a target and pushing new
packages using emerge from the host

Review URL: http://codereview.chromium.org/1515011
2010-04-02 14:06:29 -07:00
Darin Petkov
b2a21e7ecf Make errors and warnings a bit more obvious.
Review URL: http://codereview.chromium.org/1342001
2010-03-25 11:33:41 -07:00
David James
546747b982 Update enter_chroot.sh to exit with error messages when mount/unmount fails.
This is necessary for two reasons:
   1) It's nice to get an error message when mount/unmount fails
   2) set -e mode doesn't have any effect when you're in a subshell

Note that these mount/unmount failures do happen regularly in development,
so folks who depended on mount/umount failing silently will no longer be
able to rely on this and will have to kill the mounts manually.

Also fix subtle bugs in regexes for matching mount paths. (E.g. where the
regex for $HOME/chroot also matches $HOME/chroot2).

TEST=Tested mount/unmount with concurrently open shells.
     Tested mount/unmount when mount is being used by a process but the
     lock file does not reflect this.
BUG=none

Review URL: http://codereview.chromium.org/1211001
2010-03-23 15:19:43 -07:00
Luigi Semenzato
1f82e12eae Auxiliary function for args checks.
Review URL: http://codereview.chromium.org/1141010
2010-03-23 12:43:08 -07:00
Chris Sosa
acada7387b Changes to use default board when set by setup_board
Review URL: http://codereview.chromium.org/656023
2010-02-23 15:20:03 -08:00
Sam Leffler
a92ecd6718 fix finding overlays from the dev-server
Review URL: http://codereview.chromium.org/650182
2010-02-23 08:25:38 -08:00
David McMahon
4930294039 Fix archive_build.sh to work with new build using board subdirs.
Add some sane and useful defaults to common.sh for BOARDS.

Review URL: http://codereview.chromium.org/647053
2010-02-18 16:55:35 -08:00
robotboy
2ea03ac0fc Add warn and error functions to color warning and error output.
Review URL: http://codereview.chromium.org/601050
2010-02-11 15:30:55 -08:00
David McMahon
b7eb3a2be6 Add a warning to users trying to run old build scripts.
Add an exception for chrome-bot user for continuous builders.

Review URL: http://codereview.chromium.org/595009
2010-02-09 14:07:40 -08:00
Bill Richardson
10d27c2540 Rewrite make_chroot.sh to use local mirrors directly.
Use our karmic mirror for creating the build chroot, removing the need for
make_local_repo.sh and the repo_list_*.txt files. If a local repo exists
when build_chroot.sh is run it will be used, but it's no longer necessary.

This change does not remove make_local_repo.sh or the repo_list_*.txt files.

Review URL: http://codereview.chromium.org/548083
2010-01-20 13:38:50 -08:00
Steve VanDeBogart
8174ba0782 Move is_whitelisted to the common script so that all users get the previous fix
Fix typo to enable preinst scripts to run

Review URL: http://codereview.chromium.org/542097
2010-01-15 19:45:30 -08:00
tedbo
4f44d9e104 Use local apt cache instead of local_repo when mastering an image.
This change switches to mastering an image without using debootstrap.
We turn on the previously experimental bits that install a small
set of packages manually before handing things over to apt. In both
cases we use apt to download the packages so that it can populate
it's local package cache.

With this change we will no longer depend on the local_repo when
mastering an image. Developers will not have to rebuild their
local repo when repo_list_image.txt changes. Instead we will use
and lazy-fill the local apt-cache. The first time you build_image.sh
it will be slow since it needs to download the packages. Subsequent
runs should be as before since it will use the local cache. If
packages are added they should be lazily fetched in the next image
build.

Until we have a fully populated external mirror, developers will
still have to add packages to repo_list_image.txt. Also, until
make_chroot is switched over to use the external repo then
developers will have to redo their local repo when the
repo_list_dev.txt changes.

Review URL: http://codereview.chromium.org/521073
2010-01-08 17:26:11 -08:00
Alexey Marinichev
63c42aa811 Simplified counting CPUs.
Review URL: http://codereview.chromium.org/501153
2009-12-21 11:42:39 -08:00
derat@google.com
86dcc8e68b build: Make enter_chroot.sh fail if run as root.
BUG=684

Review URL: http://codereview.chromium.org/429006

git-svn-id: svn://chrome-svn/chromeos/trunk@304 06c00378-0e64-4dae-be16-12b19f9950a1
2009-11-21 19:49:49 +00:00
rspangler@google.com
d74220d772 Start copying over source.
git-svn-id: svn://chrome-svn/chromeos/trunk@4 06c00378-0e64-4dae-be16-12b19f9950a1
2009-10-09 20:56:14 +00:00