Commit Graph

144 Commits

Author SHA1 Message Date
Marc MERLIN
a1f5ee388d Remove a warning when $HOME is on NFS
If ~/.subversion doesn't exist, the code didn't run, but if it existed
there is no reason to re-create it, nor is it necessary to change its
permissions since they are inherited by the bind mount source.
However user_mkdir was trying to run chown as root which does not work
over NFS with root_squash or krb-nfs.
Therefore, the un-needed call to user_mkdir is removed.
(this is an issue because cros_sdk  --replace does call this code path
multiple times).

BUG=None
TEST=Built the chroot, and the permission denied on 'install' went away.
Change-Id: I01e9a7baf51a99a96d790c9613e26e652379e6df
Reviewed-on: https://gerrit.chromium.org/gerrit/44880
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
2013-07-14 10:45:10 -07:00
Michael Marineau
680c40ad81 fix(common): Adopt semver formatted version strings.
Drop zero padding and format versions as described by the semver spec.
The terminology is a little awkward because we inherited the backwards
meaning of 'BUILD' and 'BRANCH' version identifiers but that the version
strings themselves conform to semver.

(This doesn't change the current version, that'll happen with our next
branch cut)
2013-07-09 11:31:39 -07:00
Michael Marineau
12935743b6 fix(common.sh): Fallback to loading version.txt from current directory.
This makes version handling continue to work normally with
au-generator.zip
2013-07-08 21:39:04 -07:00
Michael Marineau
8af55de72c fix(common.sh): Correctly report script name in tracebacks
Pretty simple typo.
2013-07-05 21:34:11 -04:00
Brandon Philips
a45c529549 feat(build*): add CoreOS production image building
This will create a CoreOS production image and support it with the tools
like image_to_vm.sh and build_image.
2013-07-05 12:06:40 -07:00
Brandon Philips
195d052495 fix(*): drop FACTORY images
we don't need factory image functionality. Drop it to simplify our
scripts and prepare for our "production" feature and flags.
2013-07-05 09:37:38 -07:00
Brandon Philips
61b3560bec Merge pull request #18 from philips/core_upload_update-signing
Core upload update signing
2013-07-03 12:20:15 -07:00
Michael Marineau
5790b3e182 fix(*_release): Calculate TODAYS_VERSION in common.sh 2013-07-03 12:11:49 -04:00
Brandon Philips
3b9ff1ea62 fix(core_upload_update): use the base image
- Use variable for the image name
- add COREOS prefixed variables so we can start weaning ourself off of the
others.
2013-07-02 16:14:07 -07:00
Michael Marineau
2482291e7d fix(scripts): Cleanup build version handling.
A few things here:
 - Source manifests/version.txt directly instead of coreos-version.sh
 - Remove Chrome branch from target image directory names.
 - Use proper version instead of timestap for catalyst builds.
 - Move lsb_release script from coreos-overlay to build_library.
2013-07-02 16:12:02 -04:00
Brandon Philips
11472e5166 common: make safe_umount retry a few times
on Fedora 18 on Gnome 3.0 something is making the first attempt at
unmounting return busy. Unfortunatly, the return code is 32 everytime
so we have to parse the output of umount :( :( :(

Change-Id: I7f94bf6c2059c7e7cb4fb173d9ffbabd59f2b24f
2013-02-14 16:18:56 -08:00
Ryan Harrison
b8bd594130 Add /usr/share/profiling to COMMON_INSTALL_MASK
This directory contains profiling/coverage data that is generated by some
packages during the build process. It should not be installed onto the device,
but we want the data to live on in /build to allow developers to consult it.

BUG=chromium-os:37854
TEST=None, this change will be tested in CL 42339 since it currently has not
     effect

Change-Id: I882fa7a7496db52a72bb8c5409b3715490d95879
Reviewed-on: https://gerrit.chromium.org/gerrit/42499
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Ryan Harrison <rharrison@chromium.org>
Tested-by: Ryan Harrison <rharrison@chromium.org>
2013-02-02 11:48:48 -08:00
Mike Frysinger
baae8eb628 common.sh: convert to bash
We don't need to support POSIX shell, so convert to bash.

BUG=None
TEST=`cbuildbot {amd64,x86,arm}-generic-full chromiumos-sdk daisy-release` worked

Change-Id: I33ad25d2310c593f3e346d955e3aa27da41091fc
Reviewed-on: https://gerrit.chromium.org/gerrit/41271
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2013-01-16 13:59:21 -08:00
Brian Harring
9086a3f9d3 Calculate CHROOT_TRUNK_DIR earlier.
Issues flagged during unittest run; bit unsure how this got missed
since trybots were involved (Presuming human error).

BUG=chromium-os:37347
TEST=trybot run

Change-Id: Ib6e49d28f9427197a0abd0e4da68d2f628dc2803
Reviewed-on: https://gerrit.chromium.org/gerrit/41203
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2013-01-14 01:53:41 -08:00
Brian Harring
2499bfbeb2 Move source access to a standard location, add chromite to PYTHONPATH.
Rather than having to find /home/${SUDO_USER:-${USER}}/trunk, instead
just look for /mnt/host/trunk (defined by common.sh as $CHROOT_TRUNK_DIR).
This simplifies code flow, and is a requirement for shoving chromite
into PYTHONPATH globally w/in the chroot.

BUG=chromium-os:37347
TEST=cros_sdk --replace; cros_sdk w/ chroot upgrade.
Change-Id: I9ee3e6556541a91193f49cbf74ffc5a8e090537f
Reviewed-on: https://gerrit.chromium.org/gerrit/39921
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2013-01-14 00:44:03 -08:00
Gilad Arnold
8f861c05b6 common.sh: limit pv progress bar initial width to 80 characters
By default, pv automatically infers and uses the full width of the
terminal. This generally makes sense for a console application, but in
the case of pv it just causes the progress bar to be ridiculously wide
when run on wide terminals, to the point it's hard to read. This CL is
setting it to 80 characters, the widely accepted standard width for
a terminal, in cases where the terminal appears to be larger than
80 columns. Note that:

* Even with -w, pv appears to be resizing the progress bar as the
  terminal width changes midway through the run. This means that if
  a user widens the window, then the progress bar will go wide again and
  there's nothing to be done about it.

* Theoretically, in very rare cases this may lead to a progress bar the
  exceeds the width of the terminal (i.e. set to 80 columns on
  a terminal that has just shrunk to fewer columns). The odds for such
  timing are close to nil and even then the damage is minimal.

* This will work for non-terminal runs, or otherwise runs where stty
  does not produce any output.

* To avoid the initialization overhead for all common.sh inclusion,
  replacing the variable with a function that prints the pv/cat command.

BUG=None
TEST=Ran ./image_to_usb on wide and narrow terminal windows, it works.

Change-Id: I549df1dd29e93909ea646ae9b9e09d9a588ad382
Reviewed-on: https://gerrit.chromium.org/gerrit/40937
Commit-Queue: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
2013-01-12 23:06:31 -08:00
David James
7efb76cde5 Remove deprecated mod_image_for_test.sh and references.
mod_image_for_test.sh doesn't work anymore so nobody should be using it.
There are a few places where scripts try to use mod_image_for_test.sh,
and these are timebombs because they fail if a test image needs to actually
be produced.

BUG=chromium-os:31183
TEST=Tested that this script doesn't produce any images anymore, so
     it should be fine to delete it.

Change-Id: If80337407023d62f76117dc44cadfa46801ca236
Reviewed-on: https://gerrit.chromium.org/gerrit/40955
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2013-01-09 18:30:52 -08:00
Kees Cook
74f163b587 install kernel testcase modules on test image only
Similar to the uinput module, install the testcases/ tree of modules
only on test images.

BUG=chromium-os:37353
TEST=daisy test image build has modules, production does not.
CQ-DEPEND=I16eee5afd1664f0ab4a9ab48b6cb2beaa6f30017

Change-Id: I5e7a48936b3fb45f49ff7ca4d592d46150c5610c
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39920
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
2012-12-21 12:42:42 -08:00
Vic Yang
982556afcc Filter out more things in install shim
This reduces tens of MB of install shim.

BUG=chrome-os-partner:16712
TEST=Build install shim and install with it.

Change-Id: Ic7fab3c6e26d9b78f5181c4f10181217c1c2697a
Signed-off-by: Vic Yang <victoryang@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39534
Reviewed-by: Tom Wai-Hong Tam <waihong@chromium.org>
2012-12-11 03:02:12 -08:00
David James
22dc2ba9d9 Fallback to SUDO_USER when running on NFS.
When running on NFS, the root user may not be able to access ~/.ssh and
~/.gitconfig, so it is necessary to fallback to SUDO_USER to access these
files.

To discourage users from using NFS homedirs, print warnings every time
cros_sdk is run with an NFS homedir.

BUG=chromium-os:36783
TEST=Try cros_sdk --replace and cros_sdk with and without NFS homedirs.

Change-Id: I4cdbceca485d3491656d6f743814da4ebcdd75ad
Reviewed-on: https://gerrit.chromium.org/gerrit/38953
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2012-11-30 09:06:06 -08:00
David James
76764885b9 Convert make_chroot.sh and enter_chroot.sh to run as root.
sudo takes 150ms per invocation on Goobuntu, and with 10 invocations in
enter_chroot.sh, this means that we're wasting a lot of time, every time
cros_sdk is invoked. Cutting these unnecessary invocations reduces the time
required to run enter_chroot.sh from 2.3s to 0.8s.

CL:36618 is the companion change that updates cros_sdk to invoke
sudo unshare -m prior to calling enter_chroot.sh.

Summary of changes:

1. Remove all calls to sudo and just run the commands directly.
  - Remove the mount queue and any sudo_multi optimizations.
  - Rename sudo_chroot -> bare_chroot because we don't run sudo anymore there.
  - Remove code for validating sudo timestamp.
2. Allow the scripts to work as root:
  - Ensure that files created by cros_sdk that previously were owned by the
    user still are owned by the user (either using chown or cp -p).
  - Use $SUDO_USER to find the user's account.
  - Use $SUDO_HOME instead of $HOME to find the user's home dir.
  - Remove outdated code for disabling automount on Lucid, which doesn't work
    when run as root.
  - Update code for calculating the user's git username to use sudo to switch
    to the user. Also move it to make_chroot.sh so that this change doesn't
    impact performance.
3. Cleanup
  - Remove environment syncer process in favor of just syncing once when chroot
    is entered.
  - Remove teardown and instead rely on unshare to unmount the mounts. To make
    sure that outside processes never notice the mounts, we use mount -n. This
    also ensures that /etc/mtab never contains stale mounts.
  - Remove path-overrides, since it is no longer needed.

BUG=chromium-os:35714, chromium-os:35679
TEST=Trybot runs.
CQ-DEPEND=CL:36618

Change-Id: I919a8aadb08fafde97348e8511573c28fdd47186
Reviewed-on: https://gerrit.chromium.org/gerrit/36619
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-10-30 14:49:48 -07:00
David James
d9b6798de1 Handle race condition where mounts were unmounted from under our feet.
Mounts can sometimes be unmounted from under our feet. I'm not sure
what's causing this problem, but I can reproduce it both on gPrecise
and on our builders (chromiumos-sdk). I've confirmed this patch fixes
the problem.

BUG=chromium-os:35679
TEST=20 chromiumos-sdk trybot runs. Confirmed this code is executing
     and catching a race condition in the trybot runs.

Change-Id: Iff019e672e9124053ac62cf3017dba485a4989cb
Reviewed-on: https://gerrit.chromium.org/gerrit/36698
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2012-10-26 13:10:58 -07:00
Rene Bolldorf
32228223d6 Mask all .h and .hpp files and remove now obsolete include paths.
BUG=None
TEST=build_image

Change-Id: Ib1ff8ac57b48402d2353d33413b936e5a6627176
Reviewed-on: https://gerrit.chromium.org/gerrit/28120
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-10-24 21:37:26 -07:00
Liam McLoughlin
b2a30d6fab Mask out OEM partition for factory images
BUG=chrome-os-partner:14645
TEST=Build factory image, verify OEM partition is empty

Change-Id: I4ca863401137051bf0bacfd0dc886811c21fcc37
Reviewed-on: https://gerrit.chromium.org/gerrit/34181
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
2012-09-27 07:43:33 -07:00
Liam McLoughlin
5b37c5443a Simplify and add flexibility to image creation process
This change adds support for building the disk layout from a
configuration file. It also cleans up much of the image creation
code.

install_gpt no longer exists, and has been replaced by cgpt.py's
write action. This spits out a file that has two functions that can
be called to write a partition layout to a disk/file. This gets rid
of the gigantic nest of calculations that built the layout previously.

All instances of partition/filesystem sizes in build scripts should now
be gone in favour of calls to the cgpt.py tool.

create_boot_desc has moved inside the base image creation, in an effort
to simplify build_image.

load_kernel_test is gone since it's apparently not supposed to be called
here anyway (asked wfrichar/rspangler about this one).

Base image creation now uses files rather than loop devices when
building an image. This means we can simply umount them once we're
done and not worry about cleaning up the loop device, since it's
been done for us.

Hash pad calculation has been removed. This is now set manually inside
the partition config file.

Hybrid MBR creation is gone, since it's now possible to do that in a board
specific hook (see overlay-beaglebone/scripts/board_specific_setup.sh).

OEM partition now has a filesystem, which is mounted at /usr/share/oem
during emerge so that packages can stash files here.

root_fs_dir and friends are still globals, but the long-term idea
is to make this not the case.

BUG=chromium-os:33817
TEST=All types of images and their respective flows
  (VM, recovery, test, factory etc)

Change-Id: I8a596728a4d1845c930e837bea627f5b6a11c098
Reviewed-on: https://gerrit.chromium.org/gerrit/29931
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
Reviewed-by: Liam McLoughlin <lmcloughlin@chromium.org>
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
2012-09-23 10:05:12 -07:00
Kris Rambish
4404791930 Add boats to common.sh
TEST=See boat
    .  o ..
    o . o o.o
         ...oo_
           _[__\___
        __|_o_o_o_o\__
    OK  \' ' ' ' ' ' /
    ^^^^^^^^^^^^^^^^^^^^
BUG=None

Change-Id: I0a38d572c338762084e2ce83211d9067217586e4
Reviewed-on: https://gerrit.chromium.org/gerrit/33372
Commit-Ready: Kris Rambish <krisr@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Kris Rambish <krisr@chromium.org>
2012-09-19 19:47:06 -07:00
Brian Harring
f264b82dd2 Force our umount w/in the chroot.
Do this via ensuring that any common.sh invoker
of raw umount (say a root script) sees our umount
path.

Additionally, inject into default profiles our override,
and via an upgrade scriptlet.

This is round two; originally appeared as CL:32088, was
reverted due to:
https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos%20sdk/builds/2314/steps/BuildBoard/logs/stdio

The fix however is just adding a single sudo mkdir. :/

BUG=chromium-os:23443
TEST=cros_sdk --replace --bootstrap
TEST=cros_sdk --replace

Change-Id: I0dc7522a9c623f40081d4f138cea0c2c45171fea
Reviewed-on: https://gerrit.chromium.org/gerrit/32365
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
2012-09-06 10:34:33 -07:00
Chris Sosa
389634d71c Revert "Force our umount w/in the chroot."
This change breaks the chromiumos_sdk builder.  See

http://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos%20sdk/builds/2314/steps/BuildBoard/logs/stdio

This reverts commit 23da619f13

Change-Id: I47aa6e94aa0c62494cc40e9964f25097d3353322
Reviewed-on: https://gerrit.chromium.org/gerrit/32349
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2012-09-05 19:57:02 -07:00
Brian Harring
23da619f13 Force our umount w/in the chroot.
Do this via ensuring that any common.sh invoker
of raw umount (say a root script) sees our umount
path.

Additionally, inject into default profiles our override,
and via an upgrade scriptlet.

BUG=chromium-os:23443
TEST=manual validation, trybot.

Change-Id: Ie2514f6e8d2e10a19ab8d11c8056177bc1a2fb4d
Reviewed-on: https://gerrit.chromium.org/gerrit/32088
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-09-05 14:38:14 -07:00
Brian Harring
ece65e0633 Add a umount wrapper to suppress gvfsd/trashd breaking umount calls.
Specifically, detect if the umount failed due to current access, if so,
give it up to 9 more runs (w/ 1s pauses) continuing only if it's still
failing due to currently open files.

Via this, it should suppress the race of gvfs/trashd looking at
quick mounted/umounted pathways.

This CL is a two parter; this adds the script, and converts common.sh
consumers over to using the override.

The next CL will modify the chroot itself to ensure our script gets
picked up/used.

BUG=chromium-os:23443
TEST=trybot, manul validation.

Change-Id: I92dedd91d6133c2063b1e5dbbc1a68366844801d
Reviewed-on: https://gerrit.chromium.org/gerrit/32087
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-09-05 00:00:43 -07:00
Zdenek Behan
52970f47a5 common.sh: Change umount function to account for already deleted mounts.
BUG=chromium-os:31250
TEST=enter chroot; kill ssh-agent; exit chroot

Change-Id: Iea8e0fdd9a82047bb756a459dffc2ef59615074c
Reviewed-on: https://gerrit.chromium.org/gerrit/31929
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2012-08-31 15:41:45 -07:00
Jon Salz
8c8186c832 Add factory bundle overlay in /usr/local/factory/bundle.
CQ-DEPEND=CL:31631 CL:31632 CL:31633 CL:31636
BUG=None
TEST=cbuildbot --remote

Change-Id: Ie9b30d16e31fcba20009ccc3d7b4b24d2e9a8f3f
Reviewed-on: https://gerrit.chromium.org/gerrit/31636
Tested-by: Jon Salz <jsalz@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Jon Salz <jsalz@chromium.org>
2012-08-30 14:38:53 -07:00
Brian Harring
06d3c2e20b Pass more proxy vars into the chroot, rework sudoers.d maintenance.
In particular, put the sudoers.d setup into one script (making
updates to it easier in the future if necessary), and
centralize the proxied vars into a const in common.sh.

Thanks to Kevin McCray/Josh Triplett/Alexander Kanevsky for
pointing out the missing proxy variables, and fixes/cleanup.

BUG=None
TEST=https_proxy=blah cros_sdk -- bash -c 'echo $https_proxy'
TEST=build_packages behind a proxy.
TEST=cros_sdk --replace && \
     RSYNC_PROXY=blah cros_sdk -- bash -c 'echo $RSYNC_PROXY'

Change-Id: I3165882dfd9c8b52d25c2b26d7ff9242c84c91bd
Reviewed-on: https://gerrit.chromium.org/gerrit/31185
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Josh Triplett <josh@joshtriplett.org>
2012-08-24 13:26:03 -07:00
Mike Frysinger
c17a493bcc setup DEFAULT_BOARD by default
Rather than forcing all consumers of DEFAULT_BOARD to remember to call
get_default_board, just do it for them automatically.

BUG=None
TEST=`cbuildbot {arm,amd64,x86}-generic-full` works
TEST=`./build_packages --help` shows correct default

Change-Id: I8d6ccb83babb2764a50692318eb9193c45fb3b39
Reviewed-on: https://gerrit.chromium.org/gerrit/17868
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-08-14 10:17:31 -07:00
Mike Frysinger
27b5edb4db common.sh: fix pkgconfig install mask
Since pkgconfig is installed in the native libdir path, we need to
make sure to filter out all pkgconfig subdirs.

BUG=None
TEST=`./build_image --board=amd64-generic` no longer includes .pc files

Change-Id: Id8116190b6900ac86b433794551b250b6445d9d8
Reviewed-on: https://gerrit.chromium.org/gerrit/29682
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-08-09 01:57:02 -07:00
Olof Johansson
1222b2e736 common.sh: add /lib/modules/*/{build,source} to DEFAULT_INSTALL_MASK
Since we moved the source location during build, some of the security
testcases started reporting errors.

BUG=chromium-os:33392
TEST=build image, check /lib/module/*/{build,source} (none should be there)

Change-Id: I4656f043e4014f33dda03e1f788ba9190e0ce38a
Reviewed-on: https://gerrit.chromium.org/gerrit/29674
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Jim Hebert <jimhebert@chromium.org>
Commit-Ready: Olof Johansson <olofj@chromium.org>
Tested-by: Olof Johansson <olofj@chromium.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
2012-08-08 16:49:56 -07:00
Hung-Te Lin
76272be9de Add /firmware into INSTALL_MASK.
We will add more firmware-related tools (static AU programs, updater, netboot
firmware, firmware symbols, firmware for AU testing, ...) into /firmware for
buildbot archiver to output, and should not be picked into target chromiumos
image rootfs; so adding INSTALL_MASK should be the right approach.

BUG=chromium-os:33338
TEST=./build_packages; ./build_image

Change-Id: I64a4477a8ca1a5ed99e32d92767509f21a77c704
Reviewed-on: https://gerrit.chromium.org/gerrit/29411
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2012-08-07 21:20:31 -07:00
Jon Salz
d712ae9007 Mask /usr/local/autotest/tests from factory image.
BUG=None
TEST=build_image factory_test

Change-Id: Ic6e3d4cc9026d4a2d242655d05c431e52bb40136
Reviewed-on: https://gerrit.chromium.org/gerrit/27134
Tested-by: Jon Salz <jsalz@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Commit-Ready: Jon Salz <jsalz@chromium.org>
2012-07-11 01:24:53 -07:00
Matt Tennant
298f61ab25 build_packages: Upload command stats after each run.
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.

The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.

BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
  shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
  completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file
TEST=Verified that putting 'set -u' in my fake build_packages
caused an exit in print_time_elapsed, then fixed unbound variable
in print_time_elapsed, then rerun passed.
TEST=`cbuildbot -g <cl> --lkgm mario-paladin` passed
TEST=`cbuildbot -g <cl> --lkgm link-release` passed

Change-Id: Ieb714522cb32d7558b661e4ee1a197d1fce2c516
Reviewed-on: https://gerrit.chromium.org/gerrit/26084
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
2012-06-26 13:58:51 -07:00
David James
9d9efefe27 Revert "build_packages: Upload command stats after each run."
This reverts commit 615fe57ff0.

This CL broke mod_image_for_recovery.sh with the following failure:
  common.sh: line 684: $2: unbound variable

This bug is easy to fix, but suggests the CL needs more testing, so
we're going to revert for now and re-submit the CL once it's been
verified to pass on release trybots.

Example failure:
  http://chromegw.corp.google.com/i/chromeos/builders/x86-mario%20canary/builds/2214/steps/Archive/logs/stdio

BUG=chromium-os:27355
TEST=None, since this is reverting a change that broke the tree.

Change-Id: I61d182e3dcee267a8d9dea3b547fa6a75140d974
Reviewed-on: https://gerrit.chromium.org/gerrit/26077
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2012-06-25 18:57:10 -07:00
Matt Tennant
615fe57ff0 build_packages: Upload command stats after each run.
This changelist adds code in common.sh to support collecting command
statistics before calling upload_command_stats to upload those stats
to an appspot instance.

The presence of a file at ~/.disable_build_stats_upload will disable
all uploading of build command stats.

BUG=chromium-os:27355
TEST=`build_packages --board=x86-generic` with missing appspot instance
  shows upload error but still goes through with build.
TEST=`build_packages --board=x86-generic` with real appspot instance
  completes upload with all expected stats and does not affect build.
TEST=From outside chroot:
`touch ~/.disable_build_stats_upload`
`cros_sdk`
'build_packages --board=x86-generic`
Nothing uploaded due to .disable file

Change-Id: Iac071d1cc55a44335fc7c846960c7ae45fc93ed8
Reviewed-on: https://gerrit.chromium.org/gerrit/19401
Tested-by: Matt Tennant <mtennant@chromium.org>
Reviewed-by: Matt Tennant <mtennant@chromium.org>
Commit-Ready: Matt Tennant <mtennant@chromium.org>
2012-06-25 15:16:36 -07:00
Mike Frysinger
6b1abb2a6f fix up function style
The "function" keyword is superfluous, not in POSIX, is inconsistent
between bash files, and generally makes me angry.  So convert every
instance to the form:
	foo() {

BUG=None
TEST=`cbuildbot x86-generic-paladin` works

Change-Id: I97f5ca30a3edfef7222b1e08ac23917dc613b556
Reviewed-on: https://gerrit.chromium.org/gerrit/22467
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-05-11 14:10:38 -07:00
Brian Harring
28bb01f53e Make default parallelization controllable.
Specifically, do this via env for the time being since each program
doesn't necessarily have an option (nor warrant one).

BUG=None
TEST=None

Change-Id: I26e7f06ad5d6a44a7826bfa8465b34154d21b6a3
Reviewed-on: https://gerrit.chromium.org/gerrit/22295
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2012-05-09 16:06:40 -07:00
Brian Harring
7f175a59e1 common.sh: output a backtrace and debug information on failure.
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why.  This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.

Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down.  This includes
outputing the relevant stack trace, or at least what we can get of
it.

The 'die' function is now enhanced to automatically dump the trace
that lead to it.  For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise.  For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.

Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).

Before:
./cros_generate_breakpad_symbols  monkeys --board=x86-alex
<no output at all, just exit code 1>

With this CL:
./cros_generate_breakpad_symbols  monkeys --board=x86-alex
ERROR   : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR   : Backtrace:  (most recent call is last)
ERROR   :   file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR   :   file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR   :
ERROR   : Command failed:
ERROR   :   Command '/bin/false' exited with nonzero code: 1

BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote

Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
2012-05-07 17:19:41 -07:00
Nirnimesh
e52cb76907 Remove reference to desktopui_PageCyclerTests
BUG=None
TEST=None

Change-Id: I9bc3bbd72da506fb746bde9604378a0355381db6
Reviewed-on: https://gerrit.chromium.org/gerrit/21700
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Nirnimesh <nirnimesh@chromium.org>
Tested-by: Nirnimesh <nirnimesh@chromium.org>
2012-05-02 18:47:43 -07:00
Zdenek Behan
1d5d3b56e4 common.sh: Use /proc/mounts instead of mtab for umount
Using /proc/mounts is safer because mtab may in rare cases get desynced.
The only significant difference in output is "X Y" instead of "X on Y".

BUG=chromium-os:30249
TEST=create a chroot; enter a chroot; exit a chroot
TEST=assortment of manual tests

Change-Id: I392290e6f52a677ee2d77d77e025ef60240b11b5
Reviewed-on: https://gerrit.chromium.org/gerrit/21499
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
2012-05-02 17:47:01 -07:00
Mike Frysinger
0957a18357 virtual/linux-sources: rename from virtual/kernel
Upstream Gentoo renamed this virtual, so follow suite.

BUG=None
TEST=`cbuildbot amd64-generic-full` worked
TEST=`cbuildbot arm-tegra2-full` worked
TEST=`cbuildbot x86-generic-full` worked

Change-Id: I2721c85fe83f4ee8a90533eda14813697430e98e
Reviewed-on: https://gerrit.chromium.org/gerrit/18781
Reviewed-by: Olof Johansson <olofj@chromium.org>
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2012-03-27 09:12:29 -07:00
Joseph Hwang
ca63e041e5 Install uinput module in test image only
This CL adds uinput module in INSTALL_MASK so that the module
is not installed to non-test image. A mod_image_for_test script
is implemented to install the module in the test image if the
uinput module exists.

BUG=chromium-os:26707
TEST=Build a base image and a test image. Check the directory
  /lib/modules/<version>/kernel/drivers/input/misc
where current <version> is 3.2.7.
In the base image, there should be no uinput.ko,
while in the test image, there should be uinput.ko.
CQ-DEPEND=Ie96242c4d56403866a2298db2ba3bd6459248c1b

Change-Id: I0ca6599f80b9bb72cdc044fc97cdf990ce550edc
Reviewed-on: https://gerrit.chromium.org/gerrit/19032
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Mandeep Singh Baines <msb@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Ready: Joseph Shyh-In Hwang <josephsih@chromium.org>
2012-03-27 06:46:50 -07:00
Brian Harring
c03f260bb2 Revert "Install uinput module in test image only"
This reverts commit 5bf16bab5abb1cbf4f4acbc0d51d8aefa2c64091

Induced breakage: http://build.chromium.org/p/chromiumos/builders/x86%20generic%20incremental/builds/1259

Change-Id: I72170bc48cbc1cfe8b4913a9de1e5d8087525845
Reviewed-on: https://gerrit.chromium.org/gerrit/19023
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2012-03-23 22:44:04 -07:00
Joseph Hwang
50420556f8 Install uinput module in test image only
This CL adds uinput module in INSTALL_MASK so that the module
is not installed to non-test image. A mod_image_for_test script
is implemented to install the module in test image.

BUG=chromium-os:26707
TEST=Build a base image and a test image. Check the directory
  /lib/modules/<version>/kernel/drivers/input/misc
where current <version> is 3.2.7.
In the base image, there should be no uinput.ko,
while in the test image, there should be uinput.ko.

Change-Id: I02b557466a56e11c5dcc1649a9275d0c2a896f09
Reviewed-on: https://gerrit.chromium.org/gerrit/17209
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Mandeep Singh Baines <msb@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Ready: Joseph Shyh-In Hwang <josephsih@chromium.org>
2012-03-23 20:35:46 -07:00