109 Commits

Author SHA1 Message Date
David James
d2b72f7c2f Correctly initialize tracking branch in preflight rinse.
Currently cros/master is used for the tracking branch in the preflight
rinse step. This is wrong if --tracking_branch is specified
differently in the command-line options to cbuildbot.py.

BUG=none
TEST=Ran unit tests

Change-Id: Id4fb989863ae11f57889faf6a8d8b043b81da58e

Review URL: http://codereview.chromium.org/4104005
2010-10-26 12:35:37 -07:00
Chris Sosa
f38f249eb4 Update config for cros_au_test_harness.
Change-Id: Iccc5d28f4520418c58089cdb53b6e644271b2eb3

BUG=
TEST=n/a

Review URL: http://codereview.chromium.org/4051003
2010-10-24 02:13:27 -07:00
Chris Sosa
b00c32c335 Fix the tree by stating to right path.
Change-Id: I33e861066e8a585ce839ebc7a8c23243fb18cd1c

BUG=8083
TEST=Ran unit tests

Review URL: http://codereview.chromium.org/4053007
2010-10-23 23:38:48 -07:00
Chris Sosa
4a6e482b31 Gzip using the fastest option.
Change-Id: Id75e54879e896e42ef69659fc602f625858fd33f

BUG=
TEST=

Review URL: http://codereview.chromium.org/3989004
2010-10-22 17:30:21 -07:00
Chris Sosa
7e493cb4d3 Archive test results and image after running vm suite.
Change-Id: Ic6d4a79a7ab58b8043805a6b826f251faa9f3a6c

BUG=
TEST=Running now (adding unit tests as well)

Review URL: http://codereview.chromium.org/4062003
2010-10-22 16:32:30 -07:00
Chris Sosa
6b6f35ec65 Always pass board to cros_mark_as_stable.
Change-Id: I9a724e4e20b8945fdaa1dc53afdddf2477a5bbab

BUG=
TEST=Ran unit tests.

Review URL: http://codereview.chromium.org/3956007
2010-10-22 16:14:36 -07:00
Paul Stewart
f66c6d614e Complain (and more importantly don't blindly run) if AP or Client IP is unset
Add small if statement in cros_run_wifi_tests to balk if we didn't get
valid cell info from the server.

BUG=none
TEST=reboot everything and rerun while testbed server hasn't caught up yet

Review URL: http://codereview.chromium.org/4069001
2010-10-22 10:31:27 -07:00
Anush Elangovan
a6b1390fc5 Add branch support to preflight
Change-Id: Ia2592c6384e94e4e8e5400eddbf2e03aac150eef

BUG=7905
TEST=run cbuildbot on my workstation.

Review URL: http://codereview.chromium.org/4040002
2010-10-22 09:24:43 -07:00
Chris Sosa
dcd1de0bb9 Change preflight cleanup process to always be run before any run and add lazy unmount before every run.
Change-Id: If7db175f828fccdd4caa342a0370e842503f16dc

BUG=8015
TEST=Ran it with cbuildbot with/without --clobber with/without old state.

Review URL: http://codereview.chromium.org/4007005
2010-10-21 18:22:50 -07:00
David James
6be0f08169 Robustify and speed up cros_mark_all_as_stable.
What's new?
 - cros_mark_all_as_stable is now in Python and has unit tests.
 - We now detect and report coding errors that can cause incorrect behavior.
   E.g., if 9999 ebuild or stable ebuild can't be found, we report a hard
   failure so that developers will know about the problem.
 - We now check that git hashes we report actually match up with the
   right repository.
 - Unified diff of changes are now printed to stdout, so that developers
   can see a list of the changes and debug any problems.
 - cros_mark_all_as_stable now takes 2.5 seconds to run.

BUG=chromium-os:7795
TEST=Manually examined diff output from run of cros_mark_all_as_stable
     on full repository.

Change-Id: I762597c9b94e5f8e8171b83c966ad54e21a65c1b

Review URL: http://codereview.chromium.org/3798003
2010-10-21 16:11:00 -07:00
Chris Sosa
aa4670315b Fix equals in RunCommand's in cbuildbot.
Change-Id: If8cdde42a470c3d07cfd41552cf95dea7567688b

BUG=
TEST=Ran cbuildbot.

Review URL: http://codereview.chromium.org/3967002
2010-10-21 10:54:02 -07:00
Scott Zawalski
9310bb6bd3 Fix logic for debug and uprev
BUG=chromium-os:8000
TEST=NA

Review URL: http://codereview.chromium.org/4007003
2010-10-20 22:36:37 -07:00
Will Drewry
55b42c94ca cros_make_image_bootable, mount_gpt_image, common.sh: root filesystem changes: ext2, ro by default
This change makes more of the root filesyste metadata static across builds, but
more can be done there.  It also changes the root filesystem to use ext2 as
we don't need journaling in normal mode.  Optionally we could use ext3 for
non-verified if desired (it's an easy change).

In particular, this change cleans up the following:
- clears the rootfs uuid
- labels it C-ROOT (instead of C-KEYFOB)
- removes reserved inodes and blocks

The major feature of this change, however, is that it adds two simple
helpers to common.sh: disable_rw_mount and enable_rw_mount.  They will
set high order byte (le) in the ro compat field to be 0xff.  This will
tell the kernel that the filesystem uses features R24-R31 which are safe
for use on the running kernel iff the filesystem is mounted read-only.

These functions are called in cros_make_image_bootable and
mount_gpt_image, respectively.  mount_gpt_image will always
enable_rw_mount and cros_make_image_bootable will disable_rw_mount if
--enable_rootfs_verification is true.

The approach is ugly but reasonably well contained.  If ext2 ever gets a
new revision and new features in the same range are introduced, then we
would be getting inconsistent behavior.  That said, it is unlikely that
that churmn will happen and if the impact is negative, it will ideally
show up during testing.

N.B., this will likely result in changes needing to be made to the
signing scripts in vboot_reference to ensure that rw mounting is
enabled/disabled in the same way (E.g., during stamping).

BUG=chromium-os:7468
TEST=- built x86-generic, imaged to usb stick, attempted to mount rw /dev/sdc3 on the host and was properly bounced.
     - booted to the image just fine on a dogfood device.
     - mod'd for recovery, then installed and booted.
     - mod_image_for_test runs with no errors; booted the resulting image as well
     - booted a factory_install with the pending dm changes
     - BVT passed with build_image x86-generic (vboot enabled)
     - [in progress] autotest that checks if the rootdev = /dev/dm-0 and
       then does a dumpe2fs | grep -q FEATURE_R31

Review URL: http://codereview.chromium.org/3916002

Change-Id: If4dcba7568a110f4e32627c916d9e5741e5e5414
2010-10-20 15:44:11 -05:00
Chris Sosa
debca51332 Add --debug option to cbuildbot for developers to use cbuildbot without pushing changes.
Change-Id: Idda6c7287cdb1863eb4abe8c56da2e763e9fe777

BUG=7926
TEST=Ran with --debug and verified I didn't push a change.

Review URL: http://codereview.chromium.org/3880002
2010-10-20 11:00:44 -07:00
Chris Sosa
5911f2003b Clean up any previous kvm state.
Change-Id: I01fecde5e687992ccf36889803a4cb84b71a38b2

BUG=7818
TEST=Ran with/without previous kvm instance

Review URL: http://codereview.chromium.org/3760012
2010-10-19 12:58:24 -07:00
Chris Sosa
17414a6b24 Propagate board correctly for latest image and remote_tests.
Change-Id: I42741280656e9ee300dbc264ef7bf3345efa5223

BUG=
TEST=Ran it with test builder on own machine.

Review URL: http://codereview.chromium.org/3826015
2010-10-18 17:08:53 -07:00
David James
ef9f04e2fd Update cbuildbot to use git.chromium.org instead of src.chromium.org.
src.chromium.org is no longer supported for git operations.

BUG=chromium-os:7830
TEST=none

Change-Id: Ia7cd9ff76625441f16d20d2f9a8e0f752dca0586

Review URL: http://codereview.chromium.org/3782010
2010-10-15 16:18:29 -07:00
Anush Elangovan
189d85d20e Revert "Fix issue with stale cache from bad builds."
This reverts commit 365d4b0cc8b21204e6e093bf910d53d3673f113c.

Fix preflight breakage, this seems like the most possible cause. Will recommit if not.

BUG=
TEST=

Review URL: http://codereview.chromium.org/3767012

Change-Id: I93fe463fec77c445e7ff0ec86db25d5ef2997ad7
2010-10-15 11:36:53 -07:00
Chris Sosa
365d4b0cc8 Fix issue with stale cache from bad builds.
We always want to re-build packages we are revving, however, if the last
build succeeds in building but fails tests, we have a bad local cache that
has revved packages.  This avoids this issue by working on all packages we
are revving.

Change-Id: I3bd7463a4090b1f007d09be81bdf65657bd9c3f3

BUG=7589
TEST=Ran it with cbuildbot.

Review URL: http://codereview.chromium.org/3745006
2010-10-15 10:37:04 -07:00
Chris Sosa
6f1b9bd06d Fix update script to use 127.0.0.1 as hostname usage w/ ssh is broken on builders.
Change-Id: I88524dbe726fc20b8dbc54c712e63b9293479029

BUG=
TEST=Ran it on builder.

Review URL: http://codereview.chromium.org/3809003
2010-10-14 15:46:14 -07:00
Chris Sosa
af62e73dce Add ability to download using newest image from zip url.
This changes the behavior of ctest to try to grab the latest zip from the zip web pages if it can't use the latest link (due to IOError or none provided).  It assumes they have an Apache server and they reference images using version->ChromeOS-version-.*.zip) for each board/channel.

Change-Id: I4ee075db7afd58c8d08f59aca2e69b5bab5ff5e9

BUG=7675
TEST=Ran it with -l "http://Doesnotexist" and also without a -l option.  Added new unittests to test and ran them as well.

Review URL: http://codereview.chromium.org/3659002
2010-10-12 15:07:54 -07:00
Paul Stewart
e45157fe07 Temporarily clear noexec flag on /var if set
The "emerge" process sometimes copies files to /var/tmp
and then run it.  Recent changes to images set the
"noexec" flag on /var, thus preventing such activities.
Temporarily clear this flag from within cros_package_to_live.

BUG=none
TEST=Manually set noexec on /var and run cros_package_to_live

Review URL: http://codereview.chromium.org/3739001
2010-10-12 14:25:02 -07:00
Chris Sosa
e695541bc6 Hide errors until builder succeeds.
Change-Id: Ifb40e4931c1b0a1d32051d1d4851bca572ef3acf

BUG=7649
TEST=Ran it with fake urls

Review URL: http://codereview.chromium.org/3678004
2010-10-12 11:07:28 -07:00
Will Drewry
32d8c11747 build_image & make_bootable: allow additional kernel cmdline args
In order to more easily test kernel commandline tweaks, ensure that
--boot_args is propagated to all kernel cmdline sinks.

This is useful for changing default schedulers and any other kernel argument
tweaks we may want to quickly test out.

TEST=build and tested x86-generic
BUG=none

Change-Id: I5138755aa8c5e32e7f117f18291d2ae197e95bef

Review URL: http://codereview.chromium.org/3644004
2010-10-11 15:37:17 -05:00
Chris Sosa
020aa69f37 Fixes for cbuild to work with ctest.
1)  image_to_vm aborts because e2fschk requires a terminal to
be connected in order to check whether or not you want to repair the fs.  Since
we always want this to be true, set -p

2)  Clean up test harness to only verify the number of tests that passed on the
base image are the same on the update back to.  This is because older images
have many tests that fail.  I leave 10% as a reasonable number to pass.

3)  Redirect output from autotest in run_remote_tests to stderr so it gets
logged in a calling script that captures stdout.

Change-Id: If412274353683add20d136747113eb9c2bd41330

BUG=4690, 5533, 7287
TEST=Ran with internal tools for builders.

Review URL: http://codereview.chromium.org/3536018
2010-10-08 16:36:20 -07:00
Chris Sosa
d76a917edb Add a simple wrapper script around vm lib to start a vm.
Change-Id: Ic3238b3063d0e889c424ff6a9792d9c0f6bebca1

BUG=
TEST=Ran it

Review URL: http://codereview.chromium.org/3628002
2010-10-07 23:20:20 -07:00
Chris Sosa
4befe39060 Revert "Revert "Close tree when smoke bvt fails.""
This reverts commit 224f7548b5067f28c33f3a6607748810cc83eb94.

BUG=
TEST=

Review URL: http://codereview.chromium.org/3525022

Change-Id: Ib4e136b96a1d9dac7345598b281b053a47421e77
2010-10-07 19:04:05 -07:00
Chris Sosa
33146728ef Add ability to vm library to retry until we can ssh into it.
Change-Id: I4de388bb1af6cd4568cdc154971927941eb73800

BUG=7503
TEST=Ran test by closing vm's as they start up to see retry work.

Review URL: http://codereview.chromium.org/3598019
2010-10-07 15:58:24 -07:00
Chris Sosa
d8f50a931f Propagate return code correctly from tests.
Change-Id: I28c5f3e48646ec24813ef86f4cd16aac120cba5b

BUG=
TEST=Introduced artificial bug and checked return code
from test.

Review URL: http://codereview.chromium.org/3593010
2010-10-07 11:24:28 -07:00
Chris Sosa
224f7548b5 Revert "Close tree when smoke bvt fails."
This reverts commit d9bbe8792f412954384de1809e307c508ae37724.

TBR=Open tree.

Change-Id: I2d81711e4f995dbe025b7ed5f99fb4e4f7aba1c8
2010-10-06 17:40:51 -07:00
Chris Sosa
24680f0059 Add ability to test machines with au test harness.
Change-Id: I913acd95c49b1cf0fd666b49fb5ec1e24f0f83d1

BUG=7285
TEST=Ran through test against with two test images.

Review URL: http://codereview.chromium.org/3619009
2010-10-06 16:56:56 -07:00
Chris Sosa
2cc5cd4155 Add ctest that calls the au test harness.
Change-Id: Ief14528b7e98d792da4be39976ff0fe48219f93b

BUG=6913
TEST=Ran image_to_vm, cros_au_test_harness, and ctest using internal urls.

Review URL: http://codereview.chromium.org/3606008
2010-10-06 14:29:45 -07:00
Chris Sosa
d9bbe8792f Close tree when smoke bvt fails.
Change-Id: I01d613527777098616e8d5bd2b4b27bce240c02f

BUG=
TEST=Ran cbuildbot and prematurely killed autotest to show test failure.  CBuildbot fails saying that cros_run_vm_test smoke_Suite failed.

Review URL: http://codereview.chromium.org/3517010
2010-10-06 10:58:08 -07:00
Chris Sosa
c1b14b52d0 Add defaults for cros_make_image_bootable.
This will allow dev who didn't build an image to still use cros_make_image_bootable
to update their image -- including mod_image_for_test / image_to_vm.

Change-Id: I1ba4292223b28b21d5416745e7744641100fa4ad

BUG=7370
TEST=Ran image_to_vm with other image and image I built myself.  Also ran
cros_make_image_bootable directly on a the same two images.

Review URL: http://codereview.chromium.org/3569015
2010-10-05 13:23:50 -07:00
Chris Sosa
af33f8f8ae Add board option to au test harness so that this works for others without a default board and add ability to reference images without relative paths.
Change-Id: I4afa86fa7a6b772124de56114910a1ca5e3d4511

BUG=
TEST=Ran through update harness code and ran loman.

Review URL: http://codereview.chromium.org/3612004
2010-10-04 18:19:29 -07:00
Chris Sosa
3ede50ed4e Add enabling of local account. This should get the suite_Smoke to go green on the pfq.
Change-Id: Id480f45836a81d7d994b92d7a45d16208e1dd11c

BUG=
TEST=Ran locally.  Saw all tests pass.

Review URL: http://codereview.chromium.org/3567010
2010-10-01 14:12:26 -07:00
Paul Stewart
3da3113c01 Remove cros_copy_upgrade_server.sh
This script had a short painful life, constantly being brittle
and broken.  It is time for it to go.  Please use
cros_image_to_target.py instead.

BUG=none
TEST=none

Review URL: http://codereview.chromium.org/3536002
2010-10-01 11:22:23 -07:00
Paul Stewart
48d7d8bb7f Add sha256 to devserver clone updater output
BUG=none
TEST=none

Review URL: http://codereview.chromium.org/3572006
2010-09-30 18:16:21 -07:00
Chris Sosa
cb9e4fe449 Change hostname to localhost ip
TBR

Change-Id: I1a911da2df7cddda3eb35f6df23e203cd94c1ffe

BUG=
TEST=Locally.

Review URL: http://codereview.chromium.org/3536003
2010-09-30 18:02:09 -07:00
Chris Sosa
bccc7c1d03 Initial auto update engine test harness.
This harness has support for test on vm machines.  It takes a base image and target image path, creates an update-enabled vm image from the base image and performs updates using cros_run_vm_update (vm wrapper around image_to_live).  It uses --snapshot for updating vm's and keeps them on (like a real machine would) so that the VM image is clean on the next test.

The current check in has two test cases -- updates without clobbering the stateful partition and updates with clobbering the stateful partition.

Change-Id: I5406eadb39fa36601d17f8dd15625bd6a8b38c15

BUG=6911
TEST=Ran through tests.

Review URL: http://codereview.chromium.org/3554002
2010-09-30 15:06:15 -07:00
Chris Sosa
c2db26c796 Add methods to run vm tests on x86-pre-flight-queue.
Right now we're leaving error_ok=True until the tests are shown to work on builder.

Change-Id: I6b8c690a0da30948389fd4312032c78d87115364

BUG=6906
TEST=Ran through it locally.

Review URL: http://codereview.chromium.org/3591002
2010-09-30 14:25:25 -07:00
Kenneth Waters
e3049de00f build_image: Merge x86 and ARM where possible.
- I looked at all of the x86 and ARM paths through out build image scripts,
  these changes clean up stale comments, stale code, and unforks some small
  things.

BUG=none
TEST=Built images for x86-generic, arm-generic and tegra2-seaboard, booted tegra2-seaboard image.

Review URL: http://codereview.chromium.org/3448022

Change-Id: Ibad2774ff2cbf5f15528454506542b87e43e24a2
2010-09-30 14:20:34 -07:00
Chris Sosa
45bec936f6 Move vm constants into their own file. This will make it easier to share these constants.
Also fixed get latest image logic in image_to_vm and allowed for using the most recent image in cros_run_vm_test to follow other shell convention of using latest.

Change-Id: I60ed4c03d609500da7f6ae34ef57ba2e32f4b0bb

BUG=
TEST=Tested by running image_to_vm with --full and cros_run_vm_test with suite_Smoke

Review URL: http://codereview.chromium.org/3597001
2010-09-29 15:38:53 -07:00
Chris Sosa
3b65e8dd6f Add verify option to image_to_live.
Change-Id: I63f7ce38179b713bb26bed70d6fd392425595a81

BUG=
TEST=Ran with update vm script.

Review URL: http://codereview.chromium.org/3393019
2010-09-28 18:41:56 -07:00
Chris Sosa
f485c86416 Fix unit tests for cbuildbot.
Change-Id: I86b3eb5d7b421aaade85bd702b9089986c82cc76

BUG=
TEST=Re-ran unit tests with cbuildbot TOT.

Review URL: http://codereview.chromium.org/3441027
2010-09-28 13:08:40 -07:00
David James
70b297a263 Update cbuildbot.py to print manifests when we sync.
If cbuildbot.py printed out a manifest when it syncs, it would make it possible
for us to reproduce the build later by looking at what revisions the buildbot
was using. This is useful for debugging.

Change-Id: I186ad01eef0090b6c618cb7ad583085197474018

BUG=chromium-os:7069
TEST=Ran test suite with 3441027 patched in (Sosa's patch that fixes an unrelated issue with the unit tests)

Review URL: http://codereview.chromium.org/3461028
2010-09-25 23:25:18 -07:00
David James
7bf8d2715e Switch off repo sync --jobs, and turn on repo --trace.
repo sync --jobs doesn't check for errors, so we shouldn't use it in
automated scripts. repo --trace is needed so that we can diagnose
whether git is hanging.

BUG=chromium-os:7048, chromium-os:6774
TEST=Ran test suite

Change-Id: I9e68cfffe841a231f5fabef951ea2d45b81c1d5e

Review URL: http://codereview.chromium.org/3419024
2010-09-25 23:23:00 -07:00
Mandeep Singh Baines
ee49ac0ec5 loman: Add support for adding elements from default.xml
This is needed in order to do the right thing when adding projects
which require a revision attribute.

BUG=6811
TEST=Updated unittests and verified they pass.
Verified that I can add chromeos-kernel-next and revision gets set.

Change-Id: I12818dad464094cd50098170da3fcf38b1e32f17

Review URL: http://codereview.chromium.org/3400019
2010-09-24 08:27:50 -07:00
Kenneth Waters
9bc78b3acb make_image: Fake vmlinuz_hd.vblock on ARM.
- This unbreaks the devserver's update server for ARM.
- This also unforks some of build_image.

BUG=None
TEST=Installed to tegra2_seaboard from USB without --skip_vblock,
     updated tegra2_seaboard.

Review URL: http://codereview.chromium.org/3493004

Change-Id: I6e66344de51609393407934f78aa20f49974efef
2010-09-22 13:54:44 -07:00
Chris Sosa
8f6ee51c21 Add persist option to vm library.
BUG=
TEST=Tested by persisting, ctrl+c, continuing and using same vm.
Also without --persist

Review URL: http://codereview.chromium.org/3419014

Change-Id: I49e26eb86101e8b8b50e58a9f277d62d5fadc664
2010-09-22 11:51:52 -07:00