Commit Graph

1911 Commits

Author SHA1 Message Date
David James
f5850903d9 Tweak ssh settings to detect when remote server dies.
Right now, it seems that we don't consistently detect when the kvm is rebooted.
To improve this, I've added ServerKeepAlive messages, such that our connection
will be dropped if the target host is unreachable for 15 seconds. Further, I've
also tightened the ConnectTimeout from 120 seconds to 30 seconds so that we
detect timeouts earlier, but added 4 retries, so that all connections are
robust to temporary unresponsiveness (e.g. ssh server restarting).

BUG=chromium-os:21082, chromium-os:20305
TEST=alex-binary trybot run.

Change-Id: Ibd000bc3158138ee9c617e56bd7b11d4bd8bcb61
Reviewed-on: http://gerrit.chromium.org/gerrit/8574
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
2011-09-30 22:29:41 -07:00
Mike Frysinger
61e4f2855b cros_sdk: avoid useless forks
The cros_sdk tool runs a lot of helper programs which can be replaced
with bash builtins, or condensed multiple calls into a single one.  By
themselves they aren't that slow, but add them all up and run them a
whole lot, and it starts to make a difference.

External programs to bash internals:
 - dirname $f    -> ${f%/*}
 - f=$(cat $f)   -> f=$(<$f)
 - which f       -> type -P f

Simpler expressions:
 - [[ ( ... ) ]] -> [[ ... ]]
 - eval v=\$$f   -> v=${!f}

Common/clearer expressions:
 - ! var=$(cmd)  -> var=$(cmd || :)

Condensed tools:
 - sort | uniq   -> uniq -u

BUG=None
TEST=`cros_sdk --enter` still works with simultaneous runs
TEST=`cros_sdk --enter -- ls` still works

Change-Id: Ice5e5e252237082a2249990644e051895d61d1fd
Reviewed-on: http://gerrit.chromium.org/gerrit/8029
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-30 08:53:31 -07:00
Mike Frysinger
7f7a763ed1 enter_chroot: optimize env_sync_proc ownership changing
Most of the time the files in the chroot /etc/ are already changed to
the proper owner.  Only on the first run do they need to be changed.
So check the owner before doing the sudo as that is a bit faster.  If
we do end up needing to run sudo, bundle the commands up to avoid having
to execute sudo multiple times.

BUG=None
TEST=`sudo chown root chroot/etc/resolv.conf && cros_sdk --enter`; see /etc/resolv.conf owned by me

Change-Id: Ifd974ace168ab309fdc2e7583d135ee23576a5bb
Reviewed-on: http://gerrit.chromium.org/gerrit/8418
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-28 19:47:57 -07:00
Elly Jones
9ca3e4c0fe mod_image_for_recovery: reuse salt.
This will prevent the recovery kernel from having a different salt from the
rootfs it corresponds to.

BUG=chromium-os:20766
TESTED_ON=kaen
TEST=Adhoc
Build a recovery image and do recovery from it.

Change-Id: I96f735e527d807247e09e17aac1ed5b51367f0ef
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/8288
2011-09-28 10:50:10 -07:00
Scott Zawalski
53eb7a03b8 Modify build_image_util to preface CHROME_BRANCH with an R 2011-09-28 08:29:19 -07:00
Mike Frysinger
2a97059dfc cros_sdk: add a mount cache to avoid multiple execs
Since we grab a lock before mounting anything, we know that the relevant
list of mounts won't change (since we haven't changed it yet).  So we can
cache the output of `mount` and re-use that in every subsequent check.

BUG=None
TEST=`cros_sdk --enter` still works with simultaneous runs
TEST=`cros_sdk --enter -- ls` still works

Change-Id: I27c98c923761fb777686632fe6e6604ca543cfb6
Reviewed-on: http://gerrit.chromium.org/gerrit/8030
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-28 08:09:04 -07:00
Chris Sosa
905f6164a1 Prefix build folder name with CHROME_BRANCH from chromeos_version.sh.
BUG=chromium-os:15703
TEST=Ran build_image saw right folder.

Change-Id: I646dc7f67dac126436e75299e150ee053d25e810
Reviewed-on: http://gerrit.chromium.org/gerrit/8397
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
2011-09-27 19:05:14 -07:00
Mike Frysinger
8264917362 cros_sdk: avoid sudo to chmod when possible
Checking the permission of files is faster than assuming they're broken
and then running `sudo chmod`, so do just that.

BUG=None
TEST=run `cros_sdk --enter` with bad perms and see them fixed
TEST=run `cros_sdk --enter` with correct perms and see sudo skipped
TEST=the buildbot failure was due to other changes, not this one

Change-Id: Ie69b4e766e2e3652944e4723d091ce589d07a4f6
Reviewed-on: http://gerrit.chromium.org/gerrit/8028
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-27 16:36:53 -07:00
Andrew de los Reyes
8e93647f05 kvm: set cache type to unsafe if possible, else writeback
We try to set the cache type to unsafe, which is a new kvm feature,
else we set the cache type to writeback. The hope is that these cache
types will perform better on the bots, especially in the case of the
guest trying to flush data to disk.

The CL includes code to determine which version of kvm is running, and
the cache type is chosen based on that.

BUG=chromium-os:20305
TEST=trybot

Change-Id: Icf7dbb8f0056a330282ebeb81d0902b3d54f34c5
Reviewed-on: http://gerrit.chromium.org/gerrit/8360
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Andrew de los Reyes <adlr@chromium.org>
2011-09-27 16:30:09 -07:00
Chris Sosa
faeee5f47c Add more debugging by re-running ssh commands that failed verbosely.
This CL changes remote_ssh to re-run the command verbosely if the
command failed with an SSH error (code 255).

BUG=chromium-os:19198
TEST=Ran it.  Simulated SSH error by closing KVM instance in cros_run_vm_update
while it was intiializing.  Saw re-run verbosely run correctly.

Change-Id: I9f95aac3fb2d00bc7d26b0b9298409d8ce2db594
Reviewed-on: http://gerrit.chromium.org/gerrit/8313
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-09-26 20:19:34 -07:00
Mike Frysinger
94717e3099 cros_sdk: unify duplicate git calls for user commit info
No need to call `git var | sed` multiple times when we can call it once
and then use bash string parsing routines to extract the relevant info.

BUG=None
TEST=delete .gitconfig in chroot; enter chroot; verify .gitconfig is up-to-date

Change-Id: I900e4241a5c53bc46c90eb0f2a01152f4bfba577
Reviewed-on: http://gerrit.chromium.org/gerrit/8027
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-26 08:43:10 -07:00
Gaurav Shah
af7d5d11a5 remote_access.sh: Be verbose when remote accesss setup fails. Silent otherwise.
A parallel change in run_remote_tests.sh (http://gerrit.chromium.org/gerrit/8090)
ensures that the output of the command isn't ignored (since now it is only verbose
when it needs to be).

BUG=chromium-os:20695
TEST=Ran run_remote_tests.sh with both valid and invalid ip (again).

Change-Id: I185c81b0a8193bfa5f88ff2b9eea38dabe24edcc
Reviewed-on: http://gerrit.chromium.org/gerrit/8089
Commit-Ready: Gaurav Shah <gauravsh@chromium.org>
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
Tested-by: Gaurav Shah <gauravsh@chromium.org>
2011-09-25 14:32:13 -07:00
Mike Frysinger
b2e897bf46 enter_chroot: tweak awk to match POSIX substr requirements
POSIX says that the first arg to substr() counts from 1 which means
that awk implementations interpret a value of 0 in their own special
way.  With gawk, it does the logical thing (imo), but with mawk, it
does it by one less.

Example:
echo abc | mawk '{print substr($0, 0, 2)}'  -> a
echo abc | gawk '{print substr($0, 0, 2)}'  -> ab
echo abc | mawk '{print substr($0, 1, 2)}'  -> ab
echo abc | gawk '{print substr($0, 1, 2)}'  -> ab

So stick to the POSIX spec and count from 1.

While we're here, using "len" rather than "l" to improve readability.

URL=http://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html
BUG=chrome-os-partner:6026
TEST=set /usr/bin/awk to mawk; run `cros_sdk --enter -- true`; see everything unmounted
TEST=set /usr/bin/awk to gawk; run `cros_sdk --enter -- true`; see everything unmounted

Change-Id: I48f57d642730a0b3d909079027edf6ebf1bb9459
Reviewed-on: http://gerrit.chromium.org/gerrit/8163
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
2011-09-22 16:13:51 -07:00
Mike Frysinger
e8aec37165 cros_sdk: skip unmount phase if no mounts are found
Normally safe_umount_tree isn't called if there are no mount points in the
chroot, but there are some edge cases (like early setup errors) where it
may be possible.  So add a simple check to make sure that there are mount
points for us to work on before calling umount.

BUG=None
TEST=add early `die` before mounts; see no umount error output

Change-Id: I57e5022c3242c635097bceed44772f10eecf6a64
Reviewed-on: http://gerrit.chromium.org/gerrit/8026
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-21 17:15:59 -07:00
Andrew de los Reyes
e08639b859 remote_access.sh: add remote_sh_raw for a non-buffered ssh connection.
BUG=chromium-os:20305
TEST=manual test of image_to_live.sh

Change-Id: I964088673d0dc2867e911368d9cc4ec73085a4f4
Reviewed-on: http://gerrit.chromium.org/gerrit/8082
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Andrew de los Reyes <adlr@chromium.org>
2011-09-21 15:53:30 -07:00
Mike Frysinger
4d258cdf9d enter_chroot: restore auto locale generation
First let's add a chroot upgrade hook that'll declare a small list of
locales to speed up glibc upgrades.

The other blocker was that `cros_sdk --enter` took an inordinate amount
of time to execute (`locale-gen -u` specifically) when the number of
existing locales is huge.  This seems to be a bug in the bash/glibc
stack which is resolved in newer versions, but we can workaround the
issue by forcing locale-gen to run in a C locale.  The tool itself does
not care about its locale and we silence its output.

BUG=chromium-os:20378
TEST=`cros_sdk --enter`; see no locales in /etc/locale.gen; run `./build_packages` and see upgrade hook work
TEST=generate all 400 locales; `cros_sdk --enter`; see that it was quick

Change-Id: I8fcc5e26bd8e1bcfd52b6a6c7ef3cacf0a252081
Reviewed-on: http://gerrit.chromium.org/gerrit/7806
Reviewed-by: David James <davidjames@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-19 21:57:14 -07:00
Mike Frysinger
acc4c9b3fc cros_sdk: optimize fuse loading
Running `sudo` at all is a bit poky, so let's avoid modprobing fuse when
we know it already exists because it's listed in /proc/filesystems.

The relative speed:
 time sudo modprobe fuse -> 0.164s
 time grep -q fuse /proc/filesystems -> 0.002s

BUG=None
TEST=enable `set -x`; run `sudo modprobe fuse`; run `cros_sdk true`; see modprobe skipped

Change-Id: I07213e7b0607ca65df2a8a3ae22eb4a49c8bb888
Reviewed-on: http://gerrit.chromium.org/gerrit/7824
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-19 10:20:00 -07:00
Mike Frysinger
da7be781c0 cros_sdk: optimize gconftool interaction
It doesn't make much sense to run gconftool when the default automount
value is what we already want: false.  So in this common case, skip the
save/restore logic altogether since we aren't actually changing things.

Further, we can merge the `which` check in by checking the exit status
of the `gconftool` run.  If it fails, the tool doesn't exist.

BUG=None
TEST=enable `set -x`; run `cros_sdk true`; see gconftool run once in common case

Change-Id: I1c6aee945218a8a9df533d9ef207c750f909c252
Reviewed-on: http://gerrit.chromium.org/gerrit/7823
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-09-19 09:57:26 -07:00
Hung-Te Lin
d2631cd952 crosutils: add 'archive_factory' to create factory bundles
To generate a bundle with mini-omaha and all utility scripts, we need a new
archive script.

BUG=chrome-os-partner:5979
TEST=./archive_factory --factory_test factory_test \
       --factory_install factory_install --script . --dev ../platform/dev/ \
       --bin ../../chroot/usr/bin
  # output:
  [   0/2.4G]   adding: bin/cgpt  (stored 0%)
  [633K/2.4G]   adding: dev/autoupdate.py  (stored 0%)
  [658K/2.4G]   adding: dev/buildutil.py  (stored 0%)
  [660K/2.4G]   adding: dev/devserver.py  (stored 0%)
  [671K/2.4G]   adding: dev/static/ (stored 0%)
  [671K/2.4G]   adding: factory_test/chromiumos_factory_image.bin
  [1.9G/506M]   adding: factory_test/pack_partitions.sh  (stored 0%)
  [1.9G/506M]   adding: factory_test/unpack_partitions.sh  (stored 0%)
  [1.9G/506M]   adding: hwid/hwid_bundle_zgb_all.sh  (stored 0%)
  [1.9G/502M]   adding: install_shim/factory_install_shim.bin
  [2.4G/119K]   adding: install_shim/pack_partitions.sh  (stored 0%)
  [2.4G/115K]   adding: install_shim/unpack_partitions.sh  (stored 0%)
  [2.4G/112K]   adding: scripts/lib/cros_image_common.sh  (stored 0%)
  [2.4G/102K]   adding: scripts/lib/shflags/shflags  (stored 0%)
  [2.4G/ 71K]   adding: scripts/common.sh  (stored 0%)
  [2.4G/ 49K]   adding: scripts/chromeos-common.sh  (stored 0%)
  [2.4G/ 33K]   adding: scripts/make_factory_package.sh  (stored 0%)
  [2.4G/ 11K]   adding: scripts/make_universal_factory_shim.sh  (stored 0%)
  [2.4G/4.0K]   adding: scripts/mk_memento_images.sh  (stored 0%)

Change-Id: Idd645c908c4fed70408406278163309037c285dd
Reviewed-on: http://gerrit.chromium.org/gerrit/7906
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
2011-09-19 02:01:41 -07:00
Hung-Te Lin
17eeff6fcb crosutils: add boilerplate comments for factory scripts
Adding boilerplate comments for factory scripts that may be executed inside
CrOS source tree or a limited factory bundle extraction.

BUG=chrome-os-partner:5979
TEST=none, only comment update

Change-Id: Ifd5b07e9d44f4ce74073e5e32370040d2d6728d8
Reviewed-on: http://gerrit.chromium.org/gerrit/7908
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2011-09-19 00:44:48 -07:00
Hung-Te Lin
ab4c4e1c69 crosutils: Revert "crosutils: refine make_factory_package"
This partially reverts commit cfc4720c28
( http://gerrit.chromium.org/gerrit/7904 )

Currently we store the hash generated by compress_and_hash_memento_image in a
shell variable, and sub shell "(cmd)" can't export the variables. It would
require more rewriting of scripts to fix this issue, so let's first revert it.

BUG=chrome-os-partner:5979
TEST=./make_factory_package --config config/mp_factory.config
     cd ../platform/dev; python deverserver.py --factory_config mini_omaha.conf
     # then start whole factory installation process on DUT, and passed.

Change-Id: I1c7d8de61427733b1dff61dcd7e1c01fea7c4de6
Reviewed-on: http://gerrit.chromium.org/gerrit/7916
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Tammo Spalink <tammo@chromium.org>
2011-09-19 00:32:12 -07:00
Hung-Te Lin
545dc57e97 crosutils: allow factory scripts to be executed in a limited bundle environment
This CL allows factory related scripts to be executed in a prepared and limited
environment which can be made by extracting a factory bundle:
 - always load scripts from $SCRIPT_ROOT
 - allow running bundled binary programs from $SCRIPT_ROOT/../bin
 - override GCLIENT_ROOT for the layout of a bundle

BUG=chrome-os-partner:5979
TEST=(outside cros environment)
     ./make_factory_package --config mp_factory.conf  # test omaha
     ./make_factory_package --config rma.conf  # test --usbimg
     ./make_factory_package --config mp_ssd.conf  # test --diskimg

Change-Id: Ibf85fa267f6fe53ae88e41fd7f62ef7110b2b0bf
Reviewed-on: http://gerrit.chromium.org/gerrit/7905
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
2011-09-18 21:24:11 -07:00
Hung-Te Lin
cfc4720c28 crosutils: refine make_factory_package
Change non-posix pushd and popd into posix sub commands.
Added a check of --board to be non-zero for omaha setup.

BUG=chrome-os-partner:5979
TEST=./make_factory_package.sh --config bundle/mp_factory.conf

Change-Id: I1aa847ae2df28460319ca0b574ca4921993e41e1
Reviewed-on: http://gerrit.chromium.org/gerrit/7904
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
2011-09-18 21:02:13 -07:00
Mike Frysinger
1aa6124f1d cros_sdk: speed up unmounting when tearing down env
The teardown_env is a little slow when unmounting things because it runs
`sudo umount` once per mount point.  This is so that when things go wrong,
it can easily recover.  However, this slows down the common case at the
expense of the uncommon.

Refactor the code so that in the common case, we run one `sudo umount`.
When things do fail, we're a bit slower as we reparse the entire mount,
list, but that's fine as it's an error case.

BUG=None
TEST=enable `set -x`; run `cros_sdk true`; see all mount points unmounted in one shot

Change-Id: Iec98a7b9f51a77e90c30e6f6acae26e528b8c50d
Reviewed-on: http://gerrit.chromium.org/gerrit/7822
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2011-09-16 18:58:17 -07:00
Hung-Te Lin
882cae5df4 crosutils: Revert "Cache sudo in make_factory_package"
This reverts commit 0126898d23.

When "sudo -v" is executed inside chroot, it prompts for password; however the
user account inside chroot may be using a different password (ex, "chronos")
from the same account outside chroot.  The /etc/sudoers file inside chroot has
explicitly specified "userid ALL=NOPASSWD: ALL" for the account, so we should
keep doing nothing inside chroot.

BUG=chrome-os-partner:2827
TEST=(inside chroot) ./make_factory_package --release ...... # success

Change-Id: Ibadd5d539df83d70aba0daabc40bb9edba842f07
Reviewed-on: http://gerrit.chromium.org/gerrit/7847
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
2011-09-16 06:14:51 -07:00
ttuttle
de99e8b59a cros_run_cell_tests: Explicitly accept remote, add args workaround
BUG=chromium-os:20349
TEST=runs cellular_Dummy

Change-Id: I8dbe7a99c02d22d68303e71aa2f4df514d800c95
Reviewed-on: http://gerrit.chromium.org/gerrit/7732
Reviewed-by: David Rochberg <rochberg@chromium.org>
Tested-by: ttuttle <ttuttle@chromium.org>
2011-09-15 10:20:14 -07:00
Chinyue Chen
af9c1d9463 Make the path and folder of config file available when parsing config.
The path and folder of config file is available as env variables and can
be used to reference images and updater scripts in config file.

BUG=chrome-os-partner:5850
TEST=1. Create a config file (mp_factory.conf) under src/mp_bundle with
        the following content:

        [x86-zgb]
        --factory ${MFP_CONFIG_DIR}/../build/images/${BOARD}/latest/chromiumos_factory_image.bin
        --release ${MFP_CONFIG_DIR}/../build/images/${BOARD}/latest/[Cc]hromiumos_image.bin
        --hwid_updater ${MFP_CONFIG_DIR}/../platform/[Cc]hromeos-hwid/hwid_bundle_zgb.sh
        --firmware_updater none
        --detect_release_image  # Comment.

        # More comments.
        [x86-zgb-he]
        --subfolder x86-zgb-he
        --factory ${MFP_CONFIG_DIR}/../build/images/${BOARD}/latest/chromiumos_factory_image.bin
        --release ${MFP_CONFIG_DIR}/../build/images/${BOARD}/latest/chromiumos_image.bin
        --hwid_updater ${MFP_CONFIG_DIR}/../platform/chromeos-hwid/hwid_bundle_zgb.sh
        --firmware_updater none
        --detect_release_image

     2. Under src, execute
        BOARD=x86-zgb scripts/make_factory_package.sh --config mp_bundle/mp_factory.conf

Change-Id: I3c9693febe1f7f1efc92ab845d63226911114031
Reviewed-on: http://gerrit.chromium.org/gerrit/7764
Tested-by: Chinyue Chen <chinyue@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
2011-09-15 08:21:09 -07:00
David Rochberg
503e8f6dca Remove v flag for untarring of libc. Reduces volume of log output
BUG=none
TEST=./build_image  --board=x86-alex

Change-Id: I0112165e499fe0aa2dd3ff65c7f1d14406b3534b
Reviewed-on: http://gerrit.chromium.org/gerrit/7648
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David Rochberg <rochberg@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Raymes Khoury <raymes@chromium.org>
2011-09-14 10:28:02 -07:00
Chinyue Chen
8e6b452f4e Fix config file section header regex.
Fix section header regex so that bracket expansion syntax won't be
recognized as section header. The only requirement is that section
header must start at the beginning of a line.

BUG=chrome-os-partner:5850
TEST=1. Create a config file (mp_factory.conf) with the following
        content:
        [x86-zgb]
        --factory ~/trunk/src/build/images/${BOARD}/latest/chromiumos_factory_image.bin
        --release ~/trunk/src/build/images/${BOARD}/latest/[Cc]hromiumos_image.bin
        --hwid_updater ~/trunk/src/platform/[Cc]hromeos-hwid/hwid_bundle_zgb.sh
        --firmware_updater none
        --detect_release_image  # Comment.

        # More comments.
        [x86-zgb-he]
        --subfolder x86-zgb-he
        --factory ~/trunk/src/build/images/${BOARD}/latest/chromiumos_factory_image.bin
        --release ~/trunk/src/build/images/${BOARD}/latest/chromiumos_image.bin
        --hwid_updater ~/trunk/src/platform/chromeos-hwid/hwid_bundle_zgb.sh
        --firmware_updater none
        --detect_release_image

     2. BOARD=x86-zgb ./make_factory_package.sh --config mp_factory.conf

Change-Id: I4d1475a5e513e7e0e06a5c3c6d46d13e2120668c
Reviewed-on: http://gerrit.chromium.org/gerrit/7698
Tested-by: Chinyue Chen <chinyue@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
2011-09-14 05:59:32 -07:00
Matt Tennant
feb896d363 Upgrade script to insert BOARD_USE into existing make.conf.board_setup files.
The setup_board script now inserts a BOARD_USE value into
/build/<board>/etc/make.conf.board_setup.  This only happens the first
time the board is setup, so this upgrade script inserts that value
for boards that were previously set up.

BUG=chromium-os:20341
TEST=Ran run_chroot_version_hooks in chroot, inspected make.conf.board_setup
files for boards that did and did not have BOARD_USE already set.

Change-Id: I333198a0434076a5e8e50db6affc81ea2076d301
Reviewed-on: http://gerrit.chromium.org/gerrit/7653
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Matt Tennant <mtennant@chromium.org>
2011-09-13 18:03:48 -07:00
Raymes Khoury
4460ea782f Disable locale-gen temporarily to workaround bug
Works around:
http://code.google.com/p/chromium-os/issues/detail?id=20378

which is breaking the tree.

BUG=chromium-os:20378
TEST=cros_sdk --enter

Change-Id: I0d5dc2dd6466efe9baafcc404648d2c53309a3b7
Reviewed-on: http://gerrit.chromium.org/gerrit/7679
Reviewed-by: David James <davidjames@chromium.org>
Reviewed-by: Thieu Le <thieule@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-09-13 17:10:30 -07:00
ttuttle
4b7838e54d Add cros_run_cell_tests.sh
BUG=chromium-os:20349
TEST=working on it

Change-Id: I04d86a270dc33abeec67f797a7612218e585a917
Reviewed-on: http://gerrit.chromium.org/gerrit/7626
Tested-by: ttuttle <ttuttle@chromium.org>
Reviewed-by: ttuttle <ttuttle@chromium.org>
Reviewed-by: David Rochberg <rochberg@chromium.org>
2011-09-13 14:11:09 -07:00
Elly Jones
f4e4833a9e build_kernel_image: use a salt for verity.
BUG=chromium-os:12138
TEST=Adhoc
TESTED_ON=Kaen
Build an image. Look for "Generating root fs hash tree (salt <foo>)." in the
output. Boot the image, grep for 'salt=' in dmesg. All should be well.

Change-Id: Ia7d8504033ff1e62ba451129be5796702809cc7c
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/7335
2011-09-13 12:14:54 -07:00
Nick Sanders
0126898d23 Cache sudo in make_factory_package
BUG=chrome-os-partner:2827
TEST=observe sudo cache

Change-Id: Idf0d63ab07ee2e60f028e44ace4cd9707c2e4d6c
Reviewed-on: http://gerrit.chromium.org/gerrit/7580
Tested-by: Nick Sanders <nsanders@chromium.org>
Reviewed-by: Chinyue Chen <chinyue@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
2011-09-12 22:43:12 -07:00
David James
83848ede89 Remove old versions of dev-libs/gmp to ease chroot upgrades.
BUG=none
TEST=Run it, make sure old versions are unmerged.

Change-Id: I2a281284fc17e221985c9c2b184a5a23ba90d773
2011-09-12 10:45:18 -07:00
Chinyue Chen
67594116c0 Read script parameters from config file.
make_factory_package.sh needs many parameters and to support multi-board
installation it has to be run twice (without and with --subfolder). This
change adds the functionality to read sections of parameters from a
config file and executes them in order.

BUG=chrome-os-partner:5850
TEST=1. Create a config file (mp_factory.conf) with the following
        content:
       [x86-zgb]
        --factory ~/trunk/src/build/images/${BOARD}/latest/chromiumos_factory_image.bin
        --release ~/trunk/src/build/images/${BOARD}/latest/chromiumos_image.bin
        --hwid_updater ~/trunk/src/platform/chromeos-hwid/hwid_bundle_zgb.sh
        --firmware_updater none
        --detect_release_image

       [x86-zgb-he]
        --subfolder x86-zgb-he
        --factory ~/trunk/src/build/images/${BOARD}/latest/chromiumos_factory_image.bin
        --release ~/trunk/src/build/images/${BOARD}/latest/chromiumos_image.bin
        --hwid_updater ~/trunk/src/platform/chromeos-hwid/hwid_bundle_zgb.sh
        --firmware_updater none
        --detect_release_image

     2. BOARD=x86-zgb ./make_factory_package.sh --config mp_factory.conf

Change-Id: I3435f3afbf40ec3b8bd02a3514f599a7a535510e
Reviewed-on: http://gerrit.chromium.org/gerrit/7473
Tested-by: Chinyue Chen <chinyue@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
2011-09-09 09:23:26 -07:00
Stefan Reinauer
b392153c54 rename /u-boot --> /firmware
Also drop find_*_component, because it is not used anywhere anymore.

BUG=chrome-os-partner:5459
TEST=none

Change-Id: Iea49a81538c22fee4b91c8b3e5582b5174282151
Reviewed-on: http://gerrit.chromium.org/gerrit/7425
Tested-by: Stefan Reinauer <reinauer@chromium.org>
Reviewed-by: Anton Staaf <robotboy@chromium.org>
2011-09-08 17:05:10 -07:00
Hung-Te Lin
920482f7f4 crosutils: add HWID folder when building factory test image
To always have HWID bundles in build output, this CL copies the HWID bundle
files from board temporary space into build output folder.

The HWID folder in make_netboot is also removed.

BUG=chrome-os-partner:5796
TEST=./build_image --factory
     # see "hwid" folder in build output

Change-Id: I8c2cd32c257b117261fba3654b52929c71310c82
Reviewed-on: http://gerrit.chromium.org/gerrit/7331
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2011-09-07 21:11:41 -07:00
Raymes Khoury
d37eb67163 Print friendly error message if the libc tarball is missing.
Advises the user to try running setup_board.

BUG=chromium-os:18934
TEST=Ran with missing tarball and checked message.

Change-Id: Ice444139188756932734f5f89ef22de4cd12ab41
Reviewed-on: http://gerrit.chromium.org/gerrit/6709
Tested-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Raymes Khoury <raymes@chromium.org>
2011-09-07 18:19:30 -07:00
David James
d1679ab864 Don't hardcode output directories in boot.desc.
Right now, output directories are hardcoded in boot.desc. This means that
cros_make_image_bootable will still write to the directories mentioned in
build_image, even if it's working on a different directory. This can
mess with the buildbot, because it can cause cros_make_image_bootable
to mess with the files created for existing images.

This change fixes flaky failures where the buildbot fails with warnings
stating that vmlinuz_hd.vblock does not exist. See bug 19956.

BUG=chromium-os:19956
TEST=Run full canary trybot run and verify cros_make_image_bootable uses
     files from the right directory now.

Change-Id: Ib390aa84570b077cbc8b69b757998056acc091ea
Reviewed-on: http://gerrit.chromium.org/gerrit/7282
Reviewed-by: Will Drewry <wad@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-09-06 13:50:40 -07:00
Zdenek Behan
9a995bb37c gcc-update-hook: regenerate latest gcc prior to calling gcc-config
BUG=chromium-os:19613
TEST=below
1) Manually create a chroot with old gcc
2) Let the script update&select new one, unmerge old

Change-Id: I71732da315ed4effc6df48afab797718e44d7a06
Reviewed-on: http://gerrit.chromium.org/gerrit/7117
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2011-09-02 16:30:23 -07:00
Darin Petkov
4b964ca646 Revert "build_kernel_image: use a salt for verity."
Seems to break internal PFQ.

This reverts commit e13e4d94753a585234189f0f34c1723cdbdb36a4

Change-Id: Ia232dc7a50feb87b2af8e945c6940eaa71390525
Reviewed-on: http://gerrit.chromium.org/gerrit/7195
Tested-by: Elly Jones <ellyjones@chromium.org>
Reviewed-by: Elly Jones <ellyjones@chromium.org>
Reviewed-by: Darin Petkov <petkov@chromium.org>
Tested-by: Darin Petkov <petkov@chromium.org>
2011-09-02 13:39:34 -07:00
Elly Jones
6f568e6a94 build_kernel_image: use a salt for verity.
BUG=chromium-os:12138
TEST=Adhoc
TESTED_ON=Kaen
Build an image. Look for "Generating root fs hash tree (salt <foo>)." in the
output. Boot the image, grep for 'salt=' in dmesg. All should be well.

Change-Id: If9dbefbd8a875d06ff45cd54704f166c2511c3b7
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/7174
2011-09-02 12:06:27 -07:00
Chris Sosa
c2a60e2e03 Another copyright fix to test the Commit Queue.
BUG=None
TEST=Visual inspection

Change-Id: I5e3911fca26386a8eea4cc77dd6603b3631a02a4
Reviewed-on: http://gerrit.chromium.org/gerrit/7143
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-09-02 03:24:51 -07:00
Chris Sosa
2eb73c0be1 Quick test.
BUG=None
TEST=Change of copyright.

Change-Id: I9254f9456960a417e8584376d98388d3d8fd4467
Reviewed-on: http://gerrit.chromium.org/gerrit/7140
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-09-01 19:03:39 -07:00
Chris Sosa
c1bd111ef0 Trivial change to test the commit queue by cleaning up Copyright date in bash_history.
BUG=None
TEST=Visual

Change-Id: I6ceb84e47983ac38ba0df7c6176b42c2613f9c58
Reviewed-on: http://gerrit.chromium.org/gerrit/6790
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Darin Petkov <petkov@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
2011-09-01 18:48:39 -07:00
Thieu Le
73293cd578 New script to generate stack traces from BVT test failures.
BUG=chromium-os:19565
TEST=Manually ran script

Change-Id: I7f565ddd2e06f4b25aa5d47614cedf7bec786580
Reviewed-on: http://gerrit.chromium.org/gerrit/7093
Reviewed-by: Daniel Erat <derat@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Thieu Le <thieule@chromium.org>
2011-09-01 17:18:12 -07:00
J. Richard Barnette
8e48095cb3 Delete some unused options from build_image and mod_image_for_recovery.sh
BUG=None
TEST=run both scripts, boot the images

Change-Id: I7209421d3b438d6daf37f210d5e73c0b9f91eb1d
Reviewed-on: http://gerrit.chromium.org/gerrit/6941
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
2011-09-01 09:18:26 -07:00
Hung-Te Lin
d7a3503914 crosutils/make_universal_factory_shim.sh: allow creating images in arbitrary layout
We need more flexible image creation tool for various layout, like "RMA with
multiple release images" which needs 4 images (install, test, release A,
release B).

To make the creation of image more flexible, this CL changes
make_universal_factory_shim to using a "layout file" that can describe any disk
image format we want.

Image creation speed is also improved by aligning partition offsets and
restricting oflag=dsync applied only when being used for block devices so that
make_universal_factory_shim (outputs to a normal temporary file) becomes faster
by utilizing system cache.

For a system with every images already cached:
 - Time for building 3 in 1 factory instal shim:
   Before = 8 seconds, After = 5 seconds.
 - Time for full RMA shim creation:
   Before = 1m25s, After = 40-50 seconds.

BUG=chrome-os-partner:4108
TEST=./make_universal_factory_shim.sh image1 image2 image3 image4
     # boots correctly after being imaged to SD card.
     ./make_factory_package.sh --usbimg RMA ....
     # RMA image can install images and works correctly

Change-Id: I645196d6d6e0a24d3dfa4c413a338279df4c0d5b
Reviewed-on: http://gerrit.chromium.org/gerrit/6893
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
2011-08-31 23:37:55 -07:00
Hung-Te Lin
662588686a crosutils: clean up factory test image modification scripts
This is the counterpart of http://gerrit.chromium.org/gerrit/6992 ,
which moves init modification from scripts folder into ebuild file.
The 'customizeRelease' is also already deprecated.

BUG=chromium-os:9596
TEST=build_images --factory # pass

Change-Id: I8087f627f4b13d9f6b1d4bf6003630298a87c627
Reviewed-on: http://gerrit.chromium.org/gerrit/7040
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2011-08-31 23:37:12 -07:00