Commit Graph

127 Commits

Author SHA1 Message Date
Kees Cook
74f163b587 install kernel testcase modules on test image only
Similar to the uinput module, install the testcases/ tree of modules
only on test images.

BUG=chromium-os:37353
TEST=daisy test image build has modules, production does not.
CQ-DEPEND=I16eee5afd1664f0ab4a9ab48b6cb2beaa6f30017

Change-Id: I5e7a48936b3fb45f49ff7ca4d592d46150c5610c
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39920
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
2012-12-21 12:42:42 -08:00
Vic Yang
982556afcc Filter out more things in install shim
This reduces tens of MB of install shim.

BUG=chrome-os-partner:16712
TEST=Build install shim and install with it.

Change-Id: Ic7fab3c6e26d9b78f5181c4f10181217c1c2697a
Signed-off-by: Vic Yang <victoryang@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39534
Reviewed-by: Tom Wai-Hong Tam <waihong@chromium.org>
2012-12-11 03:02:12 -08:00
David James
22dc2ba9d9 Fallback to SUDO_USER when running on NFS.
When running on NFS, the root user may not be able to access ~/.ssh and
~/.gitconfig, so it is necessary to fallback to SUDO_USER to access these
files.

To discourage users from using NFS homedirs, print warnings every time
cros_sdk is run with an NFS homedir.

BUG=chromium-os:36783
TEST=Try cros_sdk --replace and cros_sdk with and without NFS homedirs.

Change-Id: I4cdbceca485d3491656d6f743814da4ebcdd75ad
Reviewed-on: https://gerrit.chromium.org/gerrit/38953
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2012-11-30 09:06:06 -08:00
David James
76764885b9 Convert make_chroot.sh and enter_chroot.sh to run as root.
sudo takes 150ms per invocation on Goobuntu, and with 10 invocations in
enter_chroot.sh, this means that we're wasting a lot of time, every time
cros_sdk is invoked. Cutting these unnecessary invocations reduces the time
required to run enter_chroot.sh from 2.3s to 0.8s.

CL:36618 is the companion change that updates cros_sdk to invoke
sudo unshare -m prior to calling enter_chroot.sh.

Summary of changes:

1. Remove all calls to sudo and just run the commands directly.
  - Remove the mount queue and any sudo_multi optimizations.
  - Rename sudo_chroot -> bare_chroot because we don't run sudo anymore there.
  - Remove code for validating sudo timestamp.
2. Allow the scripts to work as root:
  - Ensure that files created by cros_sdk that previously were owned by the
    user still are owned by the user (either using chown or cp -p).
  - Use $SUDO_USER to find the user's account.
  - Use $SUDO_HOME instead of $HOME to find the user's home dir.
  - Remove outdated code for disabling automount on Lucid, which doesn't work
    when run as root.
  - Update code for calculating the user's git username to use sudo to switch
    to the user. Also move it to make_chroot.sh so that this change doesn't
    impact performance.
3. Cleanup
  - Remove environment syncer process in favor of just syncing once when chroot
    is entered.
  - Remove teardown and instead rely on unshare to unmount the mounts. To make
    sure that outside processes never notice the mounts, we use mount -n. This
    also ensures that /etc/mtab never contains stale mounts.
  - Remove path-overrides, since it is no longer needed.

BUG=chromium-os:35714, chromium-os:35679
TEST=Trybot runs.
CQ-DEPEND=CL:36618

Change-Id: I919a8aadb08fafde97348e8511573c28fdd47186
Reviewed-on: https://gerrit.chromium.org/gerrit/36619
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-10-30 14:49:48 -07:00
David James
d9b6798de1 Handle race condition where mounts were unmounted from under our feet.
Mounts can sometimes be unmounted from under our feet. I'm not sure
what's causing this problem, but I can reproduce it both on gPrecise
and on our builders (chromiumos-sdk). I've confirmed this patch fixes
the problem.

BUG=chromium-os:35679
TEST=20 chromiumos-sdk trybot runs. Confirmed this code is executing
     and catching a race condition in the trybot runs.

Change-Id: Iff019e672e9124053ac62cf3017dba485a4989cb
Reviewed-on: https://gerrit.chromium.org/gerrit/36698
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-10-26 13:10:58 -07:00
Rene Bolldorf
32228223d6 Mask all .h and .hpp files and remove now obsolete include paths.
BUG=None
TEST=build_image

Change-Id: Ib1ff8ac57b48402d2353d33413b936e5a6627176
Reviewed-on: https://gerrit.chromium.org/gerrit/28120
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-10-24 21:37:26 -07:00
Liam McLoughlin
b2a30d6fab Mask out OEM partition for factory images
BUG=chrome-os-partner:14645
TEST=Build factory image, verify OEM partition is empty

Change-Id: I4ca863401137051bf0bacfd0dc886811c21fcc37
Reviewed-on: https://gerrit.chromium.org/gerrit/34181
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
2012-09-27 07:43:33 -07:00
Liam McLoughlin
5b37c5443a Simplify and add flexibility to image creation process
This change adds support for building the disk layout from a
configuration file. It also cleans up much of the image creation
code.

install_gpt no longer exists, and has been replaced by cgpt.py's
write action. This spits out a file that has two functions that can
be called to write a partition layout to a disk/file. This gets rid
of the gigantic nest of calculations that built the layout previously.

All instances of partition/filesystem sizes in build scripts should now
be gone in favour of calls to the cgpt.py tool.

create_boot_desc has moved inside the base image creation, in an effort
to simplify build_image.

load_kernel_test is gone since it's apparently not supposed to be called
here anyway (asked wfrichar/rspangler about this one).

Base image creation now uses files rather than loop devices when
building an image. This means we can simply umount them once we're
done and not worry about cleaning up the loop device, since it's
been done for us.

Hash pad calculation has been removed. This is now set manually inside
the partition config file.

Hybrid MBR creation is gone, since it's now possible to do that in a board
specific hook (see overlay-beaglebone/scripts/board_specific_setup.sh).

OEM partition now has a filesystem, which is mounted at /usr/share/oem
during emerge so that packages can stash files here.

root_fs_dir and friends are still globals, but the long-term idea
is to make this not the case.

BUG=chromium-os:33817
TEST=All types of images and their respective flows
  (VM, recovery, test, factory etc)

Change-Id: I8a596728a4d1845c930e837bea627f5b6a11c098
Reviewed-on: https://gerrit.chromium.org/gerrit/29931
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
Reviewed-by: Liam McLoughlin <lmcloughlin@chromium.org>
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
2012-09-23 10:05:12 -07:00
Kris Rambish
4404791930 Add boats to common.sh
TEST=See boat
    .  o ..
    o . o o.o
         ...oo_
           _[__\___
        __|_o_o_o_o\__
    OK  \' ' ' ' ' ' /
    ^^^^^^^^^^^^^^^^^^^^
BUG=None

Change-Id: I0a38d572c338762084e2ce83211d9067217586e4
Reviewed-on: https://gerrit.chromium.org/gerrit/33372
Commit-Ready: Kris Rambish <krisr@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Kris Rambish <krisr@chromium.org>
2012-09-19 19:47:06 -07:00
Brian Harring
f264b82dd2 Force our umount w/in the chroot.
Do this via ensuring that any common.sh invoker
of raw umount (say a root script) sees our umount
path.

Additionally, inject into default profiles our override,
and via an upgrade scriptlet.

This is round two; originally appeared as CL:32088, was
reverted due to:
https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos%20sdk/builds/2314/steps/BuildBoard/logs/stdio

The fix however is just adding a single sudo mkdir. :/

BUG=chromium-os:23443
TEST=cros_sdk --replace --bootstrap
TEST=cros_sdk --replace

Change-Id: I0dc7522a9c623f40081d4f138cea0c2c45171fea
Reviewed-on: https://gerrit.chromium.org/gerrit/32365
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
2012-09-06 10:34:33 -07:00
Chris Sosa
389634d71c Revert "Force our umount w/in the chroot."
This change breaks the chromiumos_sdk builder.  See

http://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos%20sdk/builds/2314/steps/BuildBoard/logs/stdio

This reverts commit 23da619f13

Change-Id: I47aa6e94aa0c62494cc40e9964f25097d3353322
Reviewed-on: https://gerrit.chromium.org/gerrit/32349
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2012-09-05 19:57:02 -07:00
Brian Harring
23da619f13 Force our umount w/in the chroot.
Do this via ensuring that any common.sh invoker
of raw umount (say a root script) sees our umount
path.

Additionally, inject into default profiles our override,
and via an upgrade scriptlet.

BUG=chromium-os:23443
TEST=manual validation, trybot.

Change-Id: Ie2514f6e8d2e10a19ab8d11c8056177bc1a2fb4d
Reviewed-on: https://gerrit.chromium.org/gerrit/32088
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-09-05 14:38:14 -07:00
Brian Harring
ece65e0633 Add a umount wrapper to suppress gvfsd/trashd breaking umount calls.
Specifically, detect if the umount failed due to current access, if so,
give it up to 9 more runs (w/ 1s pauses) continuing only if it's still
failing due to currently open files.

Via this, it should suppress the race of gvfs/trashd looking at
quick mounted/umounted pathways.

This CL is a two parter; this adds the script, and converts common.sh
consumers over to using the override.

The next CL will modify the chroot itself to ensure our script gets
picked up/used.

BUG=chromium-os:23443
TEST=trybot, manul validation.

Change-Id: I92dedd91d6133c2063b1e5dbbc1a68366844801d
Reviewed-on: https://gerrit.chromium.org/gerrit/32087
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-09-05 00:00:43 -07:00
Zdenek Behan
52970f47a5 common.sh: Change umount function to account for already deleted mounts.
BUG=chromium-os:31250
TEST=enter chroot; kill ssh-agent; exit chroot

Change-Id: Iea8e0fdd9a82047bb756a459dffc2ef59615074c
Reviewed-on: https://gerrit.chromium.org/gerrit/31929
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2012-08-31 15:41:45 -07:00
Jon Salz
8c8186c832 Add factory bundle overlay in /usr/local/factory/bundle.
CQ-DEPEND=CL:31631 CL:31632 CL:31633 CL:31636
BUG=None
TEST=cbuildbot --remote

Change-Id: Ie9b30d16e31fcba20009ccc3d7b4b24d2e9a8f3f
Reviewed-on: https://gerrit.chromium.org/gerrit/31636
Tested-by: Jon Salz <jsalz@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Jon Salz <jsalz@chromium.org>
2012-08-30 14:38:53 -07:00
Brian Harring
06d3c2e20b Pass more proxy vars into the chroot, rework sudoers.d maintenance.
In particular, put the sudoers.d setup into one script (making
updates to it easier in the future if necessary), and
centralize the proxied vars into a const in common.sh.

Thanks to Kevin McCray/Josh Triplett/Alexander Kanevsky for
pointing out the missing proxy variables, and fixes/cleanup.

BUG=None
TEST=https_proxy=blah cros_sdk -- bash -c 'echo $https_proxy'
TEST=build_packages behind a proxy.
TEST=cros_sdk --replace && \
     RSYNC_PROXY=blah cros_sdk -- bash -c 'echo $RSYNC_PROXY'

Change-Id: I3165882dfd9c8b52d25c2b26d7ff9242c84c91bd
Reviewed-on: https://gerrit.chromium.org/gerrit/31185
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Josh Triplett <josh@joshtriplett.org>
2012-08-24 13:26:03 -07:00
Mike Frysinger
c17a493bcc setup DEFAULT_BOARD by default
Rather than forcing all consumers of DEFAULT_BOARD to remember to call
get_default_board, just do it for them automatically.

BUG=None
TEST=`cbuildbot {arm,amd64,x86}-generic-full` works
TEST=`./build_packages --help` shows correct default

Change-Id: I8d6ccb83babb2764a50692318eb9193c45fb3b39
Reviewed-on: https://gerrit.chromium.org/gerrit/17868
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-08-14 10:17:31 -07:00
Mike Frysinger
27b5edb4db common.sh: fix pkgconfig install mask
Since pkgconfig is installed in the native libdir path, we need to
make sure to filter out all pkgconfig subdirs.

BUG=None
TEST=`./build_image --board=amd64-generic` no longer includes .pc files

Change-Id: Id8116190b6900ac86b433794551b250b6445d9d8
Reviewed-on: https://gerrit.chromium.org/gerrit/29682
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-08-09 01:57:02 -07:00
Olof Johansson
1222b2e736 common.sh: add /lib/modules/*/{build,source} to DEFAULT_INSTALL_MASK
Since we moved the source location during build, some of the security
testcases started reporting errors.

BUG=chromium-os:33392
TEST=build image, check /lib/module/*/{build,source} (none should be there)

Change-Id: I4656f043e4014f33dda03e1f788ba9190e0ce38a
Reviewed-on: https://gerrit.chromium.org/gerrit/29674
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Jim Hebert <jimhebert@chromium.org>
Commit-Ready: Olof Johansson <olofj@chromium.org>
Tested-by: Olof Johansson <olofj@chromium.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
2012-08-08 16:49:56 -07:00
Hung-Te Lin
76272be9de Add /firmware into INSTALL_MASK.
We will add more firmware-related tools (static AU programs, updater, netboot
firmware, firmware symbols, firmware for AU testing, ...) into /firmware for
buildbot archiver to output, and should not be picked into target chromiumos
image rootfs; so adding INSTALL_MASK should be the right approach.

BUG=chromium-os:33338
TEST=./build_packages; ./build_image

Change-Id: I64a4477a8ca1a5ed99e32d92767509f21a77c704
Reviewed-on: https://gerrit.chromium.org/gerrit/29411
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2012-08-07 21:20:31 -07:00
Jon Salz
d712ae9007 Mask /usr/local/autotest/tests from factory image.
BUG=None
TEST=build_image factory_test

Change-Id: Ic6e3d4cc9026d4a2d242655d05c431e52bb40136
Reviewed-on: https://gerrit.chromium.org/gerrit/27134
Tested-by: Jon Salz <jsalz@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Commit-Ready: Jon Salz <jsalz@chromium.org>
2012-07-11 01:24:53 -07:00
Matt Tennant
298f61ab25 build_packages: Upload command stats after each run.
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.

The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.

BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
  shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
  completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file
TEST=Verified that putting 'set -u' in my fake build_packages
caused an exit in print_time_elapsed, then fixed unbound variable
in print_time_elapsed, then rerun passed.
TEST=`cbuildbot -g <cl> --lkgm mario-paladin` passed
TEST=`cbuildbot -g <cl> --lkgm link-release` passed

Change-Id: Ieb714522cb32d7558b661e4ee1a197d1fce2c516
Reviewed-on: https://gerrit.chromium.org/gerrit/26084
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
2012-06-26 13:58:51 -07:00
David James
9d9efefe27 Revert "build_packages: Upload command stats after each run."
This reverts commit 615fe57ff0.

This CL broke mod_image_for_recovery.sh with the following failure:
  common.sh: line 684: $2: unbound variable

This bug is easy to fix, but suggests the CL needs more testing, so
we're going to revert for now and re-submit the CL once it's been
verified to pass on release trybots.

Example failure:
  http://chromegw.corp.google.com/i/chromeos/builders/x86-mario%20canary/builds/2214/steps/Archive/logs/stdio

BUG=chromium-os:27355
TEST=None, since this is reverting a change that broke the tree.

Change-Id: I61d182e3dcee267a8d9dea3b547fa6a75140d974
Reviewed-on: https://gerrit.chromium.org/gerrit/26077
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2012-06-25 18:57:10 -07:00
Matt Tennant
615fe57ff0 build_packages: Upload command stats after each run.
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.

The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.

BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
  shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
  completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file

Change-Id: Iac071d1cc55a44335fc7c846960c7ae45fc93ed8
Reviewed-on: https://gerrit.chromium.org/gerrit/19401
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
2012-06-25 15:16:36 -07:00
Mike Frysinger
6b1abb2a6f fix up function style
The "function" keyword is superfluous, not in POSIX, is inconsistent
between bash files, and generally makes me angry.  So convert every
instance to the form:
	foo() {

BUG=None
TEST=`cbuildbot x86-generic-paladin` works

Change-Id: I97f5ca30a3edfef7222b1e08ac23917dc613b556
Reviewed-on: https://gerrit.chromium.org/gerrit/22467
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-05-11 14:10:38 -07:00
Brian Harring
28bb01f53e Make default parallelization controllable.
Specifically, do this via env for the time being since each program
doesn't necessarily have an option (nor warrant one).

BUG=None
TEST=None

Change-Id: I26e7f06ad5d6a44a7826bfa8465b34154d21b6a3
Reviewed-on: https://gerrit.chromium.org/gerrit/22295
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2012-05-09 16:06:40 -07:00
Brian Harring
7f175a59e1 common.sh: output a backtrace and debug information on failure.
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why.  This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.

Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down.  This includes
outputing the relevant stack trace, or at least what we can get of
it.

The 'die' function is now enhanced to automatically dump the trace
that lead to it.  For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise.  For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.

Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).

Before:
./cros_generate_breakpad_symbols  monkeys --board=x86-alex
<no output at all, just exit code 1>

With this CL:
./cros_generate_breakpad_symbols  monkeys --board=x86-alex
ERROR   : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :   file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR   :   file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR   :
ERROR   : Command failed:
ERROR   :   Command '/bin/false' exited with nonzero code: 1

BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote

Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
2012-05-07 17:19:41 -07:00
Nirnimesh
e52cb76907 Remove reference to desktopui_PageCyclerTests
BUG=None
TEST=None

Change-Id: I9bc3bbd72da506fb746bde9604378a0355381db6
Reviewed-on: https://gerrit.chromium.org/gerrit/21700
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Nirnimesh <nirnimesh@chromium.org>
Tested-by: Nirnimesh <nirnimesh@chromium.org>
2012-05-02 18:47:43 -07:00
Zdenek Behan
1d5d3b56e4 common.sh: Use /proc/mounts instead of mtab for umount
Using /proc/mounts is safer because mtab may in rare cases get desynced.
The only significant difference in output is "X Y" instead of "X on Y".

BUG=chromium-os:30249
TEST=create a chroot; enter a chroot; exit a chroot
TEST=assortment of manual tests

Change-Id: I392290e6f52a677ee2d77d77e025ef60240b11b5
Reviewed-on: https://gerrit.chromium.org/gerrit/21499
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2012-05-02 17:47:01 -07:00
Mike Frysinger
0957a18357 virtual/linux-sources: rename from virtual/kernel
Upstream Gentoo renamed this virtual, so follow suite.

BUG=None
TEST=`cbuildbot amd64-generic-full` worked
TEST=`cbuildbot arm-tegra2-full` worked
TEST=`cbuildbot x86-generic-full` worked

Change-Id: I2721c85fe83f4ee8a90533eda14813697430e98e
Reviewed-on: https://gerrit.chromium.org/gerrit/18781
Reviewed-by: Olof Johansson <olofj@chromium.org>
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-03-27 09:12:29 -07:00
Joseph Hwang
ca63e041e5 Install uinput module in test image only
This CL adds uinput module in INSTALL_MASK so that the module
is not installed to non-test image. A mod_image_for_test script
is implemented to install the module in the test image if the
uinput module exists.

BUG=chromium-os:26707
TEST=Build a base image and a test image. Check the directory
  /lib/modules/<version>/kernel/drivers/input/misc
where current <version> is 3.2.7.
In the base image, there should be no uinput.ko,
while in the test image, there should be uinput.ko.
CQ-DEPEND=Ie96242c4d56403866a2298db2ba3bd6459248c1b

Change-Id: I0ca6599f80b9bb72cdc044fc97cdf990ce550edc
Reviewed-on: https://gerrit.chromium.org/gerrit/19032
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Mandeep Singh Baines <msb@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Ready: Joseph Shyh-In Hwang <josephsih@chromium.org>
2012-03-27 06:46:50 -07:00
Brian Harring
c03f260bb2 Revert "Install uinput module in test image only"
This reverts commit 5bf16bab5abb1cbf4f4acbc0d51d8aefa2c64091

Induced breakage: http://build.chromium.org/p/chromiumos/builders/x86%20generic%20incremental/builds/1259

Change-Id: I72170bc48cbc1cfe8b4913a9de1e5d8087525845
Reviewed-on: https://gerrit.chromium.org/gerrit/19023
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-03-23 22:44:04 -07:00
Joseph Hwang
50420556f8 Install uinput module in test image only
This CL adds uinput module in INSTALL_MASK so that the module
is not installed to non-test image. A mod_image_for_test script
is implemented to install the module in test image.

BUG=chromium-os:26707
TEST=Build a base image and a test image. Check the directory
  /lib/modules/<version>/kernel/drivers/input/misc
where current <version> is 3.2.7.
In the base image, there should be no uinput.ko,
while in the test image, there should be uinput.ko.

Change-Id: I02b557466a56e11c5dcc1649a9275d0c2a896f09
Reviewed-on: https://gerrit.chromium.org/gerrit/17209
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Mandeep Singh Baines <msb@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Ready: Joseph Shyh-In Hwang <josephsih@chromium.org>
2012-03-23 20:35:46 -07:00
David James
dee866c121 Remove --build_root from all scripts except mod_image_for_pyauto.sh.
For all scripts, --build_root defaults to /build and is never used. To
remove option clutter, I've deleted this option.

The only script which still references build_root in src/scripts is
mod_image_for_pyauto.sh, which seems to support it for grabbing pyauto
dependencies from a location other than /build/*. This might conceivably
be useful, so I haven't touched that script.

BUG=chromium-os:27364
TEST=Remote trybot run. git grep for references to build_root.

Change-Id: I502f7df0123a598fc62a4ef4ed847ceb182f65b8
Reviewed-on: https://gerrit.chromium.org/gerrit/18283
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-03-15 18:55:26 -07:00
David James
855afb7561 Clean up options to build scripts.
1. Remove options that are already deprecated:
   --chromefromsource --chromebuild --chromebase
   --crosbug12352_arm_kernel_signing
2. Remove the --retries option and associated function eretry.
3. Move seldomly used options out of --help to avoid confusing
   developers. Developers who need these options can read the
   source.
4. Alphabetize all options.
5. Add description of each script to --help.

BUG=chromium-os:27364
TEST=Run --help with each script. Full canary trybot run.

Change-Id: I95675b069781f7e950d75d32dbad744adce6b830
Reviewed-on: https://gerrit.chromium.org/gerrit/18194
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-03-15 15:54:43 -07:00
Chris Sosa
c9422fa467 Deprecate --test, --withdev, --factory, --factory_install from build_image.
Using these flags directly was deprecated a while back and is causing confusion
which way is the right way.  This CL removes all these FLAGS from build_image
(though keeps support of them in mod_image_for_test/image_to_usb invocations
in common.sh).  In this change I've also defaulted build_image to build the
developer image (and only the developer image).

BUG=chromium-os:27362
TEST=Been busy testing.  Have built the following combinations and verified them:

build_image base
build_image
build_image base dev test
build_image dev
build_image dev factory_test
build_image factory_install

Change-Id: Ie534c276a9ec571926964320ac176daa91b12a81
Reviewed-on: https://gerrit.chromium.org/gerrit/17386
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
2012-03-09 12:41:50 -08:00
Chris Wolfe
21a27b75e9 scripts: Add functions to detect an interactive terminal
With more interactive features appearing in the scripts, we need to
avoid hanging buildbots waiting for user input. This changes adds
utility functions to test whether and ensure that the execution is
interactive, and adds an assertion to the 'choose' function.

is_interactive: Check that both stdin and stderr are terminals.
assert_interactive: Die with an error if not interactive

Also tweaking 'choose' so that its menu is printed to stderr,
for consistency with read -p and select.

BUG=None
TEST=Modified setup_board to log the value of is_interactive, and
  then (unconditionally) attempt to display a menu.
  Running setup_board in a terminal showed interactive and the menu.
  Manually run cbuildbot x86-generic-pfq showed non-interactive and
  the assertion failure.

Change-Id: I768a37b8b85ce036edac7c9bb0fd1e0a2b92ecd5
Reviewed-on: https://gerrit.chromium.org/gerrit/16817
Commit-Ready: Chris Wolfe <cwolfe@chromium.org>
Reviewed-by: Chris Wolfe <cwolfe@chromium.org>
Tested-by: Chris Wolfe <cwolfe@chromium.org>
2012-02-28 13:57:05 -08:00
David James
184e39019d Fix --jobs flags to actually work.
The --jobs=<n> option wasn't actually being passed to parallel_emerge --
which meant it didn't do anything.

BUG=chromium-os:26827
TEST=Ran with --jobs=N and made sure only N parallel_emerge processes were
started.

Change-Id: I581fc5588b54e246acaefd0c7e528e55adf9ba8a
Reviewed-on: https://gerrit.chromium.org/gerrit/16570
Reviewed-by: Michael Krebs <mkrebs@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2012-02-23 21:42:34 -08:00
Chris Sosa
2b73600f7f Update the copyright year in common.sh.
BUG=None
TEST=visual.

Change-Id: I66d6a6901930cfeb6391df5064f247e7148e5c80
Reviewed-on: https://gerrit.chromium.org/gerrit/15747
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2012-02-12 16:45:48 -08:00
Gilad Arnold
207a7c7e7f common.sh: new function for a selection menu with a default option.
Defined a bash function choose(), which allows to present a menu and
prompting for a selection, with support for default choice. Usage is
commented appropriately.

Originally reviewed in CL I0d2f20dc8d62ce5fa18c10d9f8b51a46b2ddca5d.

BUG=chromium-os:26010
TEST=Tested in conjunction with a use in image_to_usb.sh, works fine.

Change-Id: I53a42a46a3c90fd486fead578bfbae248f64cfc2
Reviewed-on: https://gerrit.chromium.org/gerrit/15586
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
Commit-Ready: Gilad Arnold <garnold@chromium.org>
2012-02-09 13:31:41 -08:00
Brian Harring
feb04f77fe Require sudo usage to be non-interactive.
First, add a enable_strict_sudo helpers that scripts that are
sudo strict can invoke.  This does a sanity check bailing immediately
if invoked from a non sudo-keep-alive context.

Second, update enter_chroot.sh and make_chroot.sh to be strict.

While this is strict, that's the point.  The aim of this is to block
the previous sudo interactive mess for spreading, let alone reappearing
in scripts/code that has been cleansed.

BUG=chromium-os:18393
TEST=cros_sdk --replace; in the midst of it, do sudo -k.
     cros_sdk should thus bail out w/ an appropriate error.
CQ-DEPEND=I01bb1466cf027401fa387af7fad15e42fd33aea4

Change-Id: I76c5b87a812cc78c30a2eb1a0c56b9e438f4a98f
Reviewed-on: https://gerrit.chromium.org/gerrit/15294
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-02-08 16:19:14 -08:00
Gilad Arnold
083662748f common.sh: add a variable for default recovery image name.
This is a preparatory step for changes to image_to_usb.sh and
mod_image_for_recovery.sh.  In general, by migrating the default
recovery image name to common.sh we can improve the interoperability
between the different scripts.

BUG=chromium-os:26010
TEST=Obvious change; tested to see the mod_image_for_recovery.sh and
image_to_usb.sh still work.

Change-Id: I765d665b656d5d25b2d9b0e40bf5c05048c3f255
Reviewed-on: https://gerrit.chromium.org/gerrit/15507
Commit-Ready: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
2012-02-08 12:09:16 -08:00
Mike Frysinger
77c674b54e common.sh: add missing die to local shflags inherit
When we source the local copy of shflags, we don't check for errors like
we do when sourcing the external one.  Add a `die` call for that too.

BUG=None
TEST=`rm lib/shflags/shflags && (. ./common.sh)` now exits properly

Change-Id: I62c9b33dd96f2934691c43d53dea012097116ca1
Reviewed-on: https://gerrit.chromium.org/gerrit/15443
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
2012-02-08 08:58:19 -08:00
Mike Frysinger
669b28b666 common.sh: move output helper funcs up top the top
The shflags sourcing logic tries to use `die` before we've defined it, so
if there's an issue, we end up with:
	bash: die: command not found
and the shflags code doesn't actually get loaded.

So relocate these small helpers to the top so we don't have to worry about
when it's safe to use these things.

BUG=None
TEST=`rm lib/shflags/shflags && (. ./common.sh)` now exits properly

Change-Id: Ibdc268e6c081aa07679dc9fce76e5603b7217b20
Reviewed-on: https://gerrit.chromium.org/gerrit/15442
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-02-08 08:00:57 -08:00
Hung-Te Lin
d32c59fe47 crosutils: build factory test image by emerge commands
The factory test image was created by using rsync from build artifacts
in chroot, which has some concerns:
 - Runtime dependencies of autotest-factory won't be picked into image.
 - If a developer skips build_package and builds image by using only pre-built
   binary packages, he will get nothing in factory test image.
 - It's hard for developers to figure out how and when his changes will be
   merged to next build_image (cros_workon does not really work).
 - Output image will be definitely different for every developers, also the
   official build bots.
 - If developers never wipes his chroot (setup_board), the factory test image
   will grow until out of space. (For example, my environment outputs a 825M
   image while the official buildbot generates only 563M for same ToT source).

This CL changes image build command to using portage emerge, so that output
image can be prepared faster and smaller, and easier for maintenance.

BUG=chromium-os:3335
TEST=./build_packages; ./build_image --factory  # Image starts factory UI successfully
     # Also tried tests in test_list.all, seems fine.

     time ./mod_image_for_test.sh --factory --force_copy --no_inplace
     # time: 3m2s => 1m55s, factory test image data: 825/563M => 378M

     ./build_image --factory_install # factory install shim is also fine

Change-Id: I82b4505c74cd31e718aaff4a319d50b69b2c852c
Reviewed-on: https://gerrit.chromium.org/gerrit/14473
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
2012-01-20 17:54:11 -08:00
Mike Frysinger
bc36d041f0 get_default_board: detect invalid user settings
If people have whitespace in their .default_board setting, then the cros
utils will act weirdly.  Some work the same as if there was no whitespace
(probably because the variable is used unquoted) while others error out
with weird messages (because the variable is used quoted).

Update the helper function to only allow certain characters in the name.

BUG=None
TEST=`printf 'x86-alex\n' > .default_board; cros_workon list --all` works
TEST=`printf 'x86-alex\t\n' > .default_board; cros_workon list --all` errors out

Change-Id: Id83794c13bfddb7fb56b7f8ed8a375eefe6096e0
Reviewed-on: https://gerrit.chromium.org/gerrit/13151
Reviewed-by: Jason Glasgow <jglasgow@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-12-20 14:38:57 -08:00
Mike Frysinger
286b5928c8 enter_chroot: merge multiple mount requests into a single sudo
Every invocation of `sudo` delays things, so merge all of the mounts
into a single `sudo` command when possible.  This saves over 1 second
on initial execution (out of ~4 seconds total).

BUG=None
TEST=`cros_sdk --enter true` still mounts & unmounts properly

Change-Id: Ibc66507dc21250a81207d2f940645eaebe93c79c
Reviewed-on: https://gerrit.chromium.org/gerrit/10901
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
2011-11-14 13:42:21 -08:00
Zdenek Behan
07d2422c91 common.sh: Revert commit 9aa80dc457
This partially reverts and partially clarifies the code in the above
commit. It was still in use, just in an obscure place, so a reference
to that has been added.

BUG=chromium-os:21971
TEST=below
1) build_lib/generate-au-zip.py
2) cd /tmp/au-generator/; unzip au-generator.zip
3) exit chroot; cd $chroot/tmp/au-generator/
4) LD_LIBRARY_PATH="$(pwd)" PATH="${PATH}:$(pwd)" ./cros_generate_update_payload --image something_i_had --output x --outside_chroot

Change-Id: Idfc03937c4f3517c8567efcdebd1cb8ce134fc8e
Reviewed-on: https://gerrit.chromium.org/gerrit/11019
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2011-11-07 17:28:59 -08:00
Chris Sosa
131eaf5667 Fix message funcs to use $@ and fix callers to use it correctly.
Fixed all callers in my big CL to no longer have to wrap poorly using
\ and start from the beginning of the next line but rather pass in
an array of args i.e.

info "tacos" " are" " delicious".

BUG=None
TEST=Ran to see the errors in parse_build_image and manual eyeing.

Change-Id: I5eac8a5ae7a8d314dbc4e821ee33cf88213711d0
Reviewed-on: https://gerrit.chromium.org/gerrit/10823
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-10-28 11:08:58 -07:00
Chris Sosa
b0f5732449 Add ability to pass in specific images you want built rather than use flags.
This CL does two things.

First it introduces an argument interface that works with all the following
combinations:

factory_install | [base|dev[test|factory_test]] rather than use --test etc.

This does not build extra images.  If you just want a test image you can
run build_image --board=<board> test and all that's in your latest_dir is
chromiumos_test_image.bin.

Second, we only build what we ask and only finalize what we ask so on
an invocation of build_image test we only actually run cros_make_image_bootable
once saving 2 minutes on my machine (from 6:50->4:50 on multiple iterations).

BUG=chromium-os:22048
TEST=build_image test, all possible combinations of args through to start of
build.  Built factory_install using new and old methods and dev test with both
methods.  Also verified mod_image_for_test still works as advertises stand-alone.

Change-Id: I9fe2feb50a941c007214decd9ba1627012c050af
Reviewed-on: https://gerrit.chromium.org/gerrit/10621
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-10-27 16:33:07 -07:00