348 Commits

Author SHA1 Message Date
Chris Sosa
564373521e Reconfigure to use ssh connection rather than use different manifest.
This also removes the rw_checkout var as it's not longer changes the affect of whether we use the ssh url or not.

Note that this replaces the other CL from last night because the other CL didn't work.  The other CL still pulled in the same manifest that has http: url's for each git repo.  I reverted that change last night after I noticed this.

Change-Id: I3d4ad2be6887ac2cf4ed2009bad9cae6dfdf5bbf

BUG=chromium-os:9509
TEST=Ran with cbuildbot --clobber and incremental

Review URL: http://codereview.chromium.org/5278010
2010-11-29 10:42:49 -08:00
Chris Sosa
c58667b8c4 This CL adds the ability to Rev and Build Chrome using cros_mark_chrome_as_stable.
Other things:
Removing commit|clean|push command from cros_mark_chrome because it's unnecessary ... I'm just gonna use the same stabilizing branch as the other stable marker, so I can use the same clean / push from the other script.  This makes this change fit in much more nicely to cbuildbot.

Other changes:
Add ability for cros_mark_chrome_as_stable to communicate to calling script through stdout.
Make STABLE_BRANCH in cros_mark_as_stable public for above reason.
Removed push_options from cros_mark and accompanying change to cbuildbot as it isn't used anymore.
Made it easier to debug with cbuildbot (I ran cbuildbot A LOT of times to test this change so I felt the pain)...this includes:
  - only build when syncing
  - allow one to explicitly disable tests
  - use dryrun is options.debug is set

Change-Id: I413a2e81a99cdde2e4d9139561cd518245b9b346

BUG=chromium-os:8693
TEST=Ran cbuildbot with --notest --debug --nosync x86-pre-flight and
saw it go through correctly and dryrun push.  Ran cbuildbot with same and --chrome_rev=tot.  Ran with syncing and running tests too.

Review URL: http://codereview.chromium.org/5154008
2010-11-24 13:18:39 -08:00
Gaurav Shah
d02793a5d7 cros_make_image_bootable: Do a final sanity check on the rootfs by running e2fsck
This is a complement to the proposed fix for http://crosbug.com/9447, and I am getting it out first since it is a relatively simple change.

This change ensures that the rootfs on the image output by cros_make_image_bootable wasn't corrupted accidentally (for example, no boot.desc was provided and the defaults were incorrect for the image). This should catch rootfs errors that we end up discovering at delta AU time.

BUG=chromium-os:9578
TEST= Ran cros_make_image_bootable manually on an existing image with and without boot.desc. As expected, without boot.desc the output image rootfs had errors.

Also ran ./build_image - which worked fine too.

Change-Id: I4ed3a56634f37ab7d5ff2dc052ad607740356a40

Review URL: http://codereview.chromium.org/5216003
2010-11-24 12:48:14 -08:00
Thieu Le
ac13647eda Change _ArchiveTestResults to upload to Google Storage
BUG=chromium-os:8364
TEST=

Change-Id: Icecc8268fa2c12c8413caa2879785f00e4be5a0a

Review URL: http://codereview.chromium.org/4864001
2010-11-24 10:42:32 -08:00
Chris Sosa
444e3a6de0 Revert "Change url of manifest to use ssh one as opposed to mirrored repo."
This reverts commit 4847031faad12e0f7b2fd3f047c77cad7b0a4003.

TBR=Required for pushes to work.

Change-Id: I4f07407e518045ea77fa61942af58f57e68b5f8f
2010-11-24 00:54:48 -08:00
Chris Sosa
4847031faa Change url of manifest to use ssh one as opposed to mirrored repo.
Change-Id: I680e28bcafbfeb4f19e2404e2b28761e26111f7f

BUG=chromium-os:9509
TEST=Ran cbuildbot on my local builder checkout with both clobber and without.
It sync'd and built.

Review URL: http://codereview.chromium.org/5312002
2010-11-23 16:29:01 -08:00
Simon Glass
d73ec0e2a8 cros_image_to_target handles test images, less verbose
added --test option
removed superfluous output (e.g. from dd) unless --verbose given
added --debug option for when you really want that output
added use of pv for a nice progress bar

TEST=(run script several times)
BUG=chromium-os:9563

Change-Id: Ib9d89ad5649b6cb6d5db35f66f6a3fda1ada0c53

Review URL: http://codereview.chromium.org/5176002
2010-11-23 15:30:50 -08:00
Chris Sosa
e4836188a8 Add ability for cbuildbot to only run unit tests for revved packages.
Change-Id: I6744aac09fb075f91f710f16343fab45b8dc8af9

BUG=chromium-os:7283
TEST=Ran it with cbuildbot through the unit tests phase and looked
at the file and saw it get parsed correctly.

Review URL: http://codereview.chromium.org/5124006
2010-11-22 13:44:51 -08:00
Chris Sosa
ed210b40ff Add new chrome revving module to crosutils.
Have subsequent review for cbuildbot that calls this for the pfq.

This module  has 3 modes ... TOT, latest_release, sticky_release that describe the three different types of revs for chrome we will be supporting.

For tot, we grab the latest svn revision # from the chrome src tree, and rev to the corresponding ebuild i.e. 9.0.553.0_alpha-r1 with CROS_SVN_COMMIT=svn revision number.

For latest_release, we grab the latest release from the releases page, and create an ebuild (or rev a previously existing on with same version) and use the release candidate suffix _rc.

For sticky_release, we use much of the same logic as latest_release, however, we specifically look to rev _rc's in the sticky branch.

This change depends on http://codereview.chromium.org/5172003 for the cros_mark_as_stable changes.

Change-Id: Idc1f4cd905cc1dcb4c44422f6104760077a24b8d

BUG=chromium-os:8693
TEST=Ran cros_mark_as_stable with --packages="chromeos-base/chromeos-chrome" --chrome_version=0.8.65.0 --chrome_revision=67890 commit.  Re-ran unit-tests as well (though thinking about adding another unit test for chrome revving)

Review URL: http://codereview.chromium.org/4798001
2010-11-19 14:01:11 -08:00
Don Garrett
d0473d1e9f Don't pass "" to cros_run_vm_test if vm_graphics_flag isn't set.
This would not have been a problem if "--no_graphics" is used.

Change-Id: I3645391d25d4bb8618109ff5be0ca2fbe8e96a60

BUG=chromium-os:9373
TEST=Ran by hand

Review URL: http://codereview.chromium.org/5104006
2010-11-18 16:11:13 -08:00
Paul Stewart
f7597abb17 Use new stateful image builder
Also change download filenames

BUG=chromium-os:9308
TEST=Run script

Review URL: http://codereview.chromium.org/5149002

Change-Id: Id9ed150184559ab31274dc3b4f58bf9fe0a0f76c
2010-11-17 16:22:28 -08:00
David James
77f281a75e Move sanity checks for missing directories until after checkout happens.
Preflight clean needs to wait until after the checkout happens before
it asserts that all of our overlays exist.

BUG=chromium-os:9197
TEST=Ran cbuildbot.py with missing chromiumos-overlay dir

Change-Id: If66574f01d1e85741e971919a9fa2da34f85872d

Review URL: http://codereview.chromium.org/5069001
2010-11-15 20:39:00 -08:00
Chris Sosa
4e75efa930 Add option to check version of Chrome on target.
Change-Id: I34e9d64656e04d4f5c477914512219d9b8563b78

BUG=chromium-os:9137
TEST=Ran with both wrong and right versions against a Google Chrome and
Chromium build

Review URL: http://codereview.chromium.org/4942001
2010-11-15 15:12:06 -08:00
David James
b49c6df754 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=795bd30

Review URL: http://codereview.chromium.org/4442001
2010-11-15 13:06:42 -08:00
David James
02b3e4f3d5 Revert "Add more error checking to preflight queue."
This reverts commit 795bd30f069bc64da6f68d8c3e23ed15fa62e072.

BUG=
TEST=

Review URL: http://codereview.chromium.org/4904003

Change-Id: I9148026ecd0df75c253cfcf0c67af3d26771dd21
2010-11-12 14:47:57 -08:00
David James
795bd30f06 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-12 14:33:14 -08:00
David James
4beb5cb58f Revert "Add more error checking to preflight queue."
This reverts commit b4258c8bd48f89d87f9d1aa2fc23b0556ffc611b.

BUG=br0ken build
TBR=rspangler
TEST=

Review URL: http://codereview.chromium.org/4697007

Change-Id: I839ffd08d33fc5943696cfe31f2978fe32ce5fec
2010-11-11 10:51:07 -08:00
David James
f9b84d61a1 Define buildroot properly.
Followup to "Add more error checking to preflight queue."
BUG=Br0ken build
TBR=kliegs@chromium.org
TEST=Ran unit tests

Review URL: http://codereview.chromium.org/4742004

Change-Id: I63e52265ae404f0193cb1f9ab35f3f1ec266d2c1
2010-11-11 10:45:12 -08:00
David James
b4258c8bd4 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-11 10:37:04 -08:00
Chris Masone
f8513d84fe [crosutils] Propagate use_emerged flag up from run_remote_tests
BUG=8544
TEST=build chrome with build_tests, cros_workon start autotest, and then run desktopui_BrowserTest with cros_run_parallel_vm_tests

Change-Id: I19e181238fd75450d41f6debb3a8f7d0c17d15dc

Review URL: http://codereview.chromium.org/4669007
2010-11-09 14:23:29 -08:00
Ken Mixter
08145697a5 crosutils: Add vnc view of tests running multiple tests for VM tests
Change-Id: I87c41e60797848192cee1f0c165215fcec8f5bd2

BUG=8850
TEST=Ran a few tests including suite_Smoke

Review URL: http://codereview.chromium.org/4647001
2010-11-08 18:10:08 -08:00
Darin Petkov
4b029a27f8 VM: Use -serial none in no-graphics kvm mode...
...so that the console doesn't get messed up due to the serial port being
redirected from stdin.

BUG=8729
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke; checked console

Change-Id: I5a0cedba2b0ac7f453b4574666c645e84b30b379

Review URL: http://codereview.chromium.org/4501001
2010-11-04 14:06:57 -07:00
Darin Petkov
c0644eb90a VM: Implement --order_output option in cros_run_parallel_vm_tests.
If the option is specified, the utility will pipe the test stdout/stderr
to separate files and dump them at the end.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke --order_output

Change-Id: I6c4fcb02456441b4e4fc3f1717cdb5d607d733b9

Review URL: http://codereview.chromium.org/4422001
2010-11-03 14:53:18 -07:00
Darin Petkov
40508146b3 VM: Add board, image_path and base_ssh_port options.
BUG=8585
TEST=manual, tried different options

Change-Id: I66be3930cefdf4ea382e1ed8aba54a66f4bfb36e

Review URL: http://codereview.chromium.org/4403001
2010-11-03 12:47:15 -07:00
Darin Petkov
3bab032123 VM: Add --results_dir_root option to cros_run_parallel_vm_tests
This allows the user to specify a custom results directory root. cbuildbot can
be switched to use cros_run_parallel_vm_tests if necessary without any further
changes.

Also create parents directories for user-specified results directory roots in
run_remote_tests.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke \
  --results_dir_root=/tmp/foo

Change-Id: I7314c1d9e74ca139eaa4bd95290866a43a3606ff

Review URL: http://codereview.chromium.org/4297003
2010-11-03 11:20:53 -07:00
David James
1f79aafb89 Fix broken quoting in cbuildbot.py, allowing for correct revving of ebuilds.
When you pass '--foo="bar baz"' to a script, the argument is literally
passed as "bar baz", with the quotes. So we need to not add unnecessary
quotes in there.

When enter_chroot.sh parses arguments, it actually removes the quotes
for you, and splits up arguments. As a result, it may actually
be necessary to (1) use the above broken syntax when you call scripts
using enter_chroot.sh; but (2) don't use that syntax when calling
scripts without enter_chroot.sh. That's a broken situation, so I've
added a TODO for that. I've also added more warnings to
cros_mark_as_stable.py so that it's easier to debug when it doesn't
work.

BUG=chromium-os:8633
TEST=Ran unit tests. Planning to watch TOT buildbot post-commit to make sure behavior is right.

Change-Id: Ia1a0e60153fec60cb7ed4db2da128ffd9ae81e23

Review URL: http://codereview.chromium.org/4385002
2010-11-03 11:15:11 -07:00
Darin Petkov
a424d94ab3 A basic utility to allow running of tests on VMs in parallel.
Current usage is simple: just specify a list of tests and each one will be run
on a separate VM using the latest image on a default board. Exit code is
non-zero if any of the tests fail. Output from separate tests is interleaved.

More CLs to follow -- specifying a result directory, non-interleaved output,
base ssh port, board, image path, etc.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke

Change-Id: Ifa3396768050905ec02e4cad8b8f065a6d129154

Review URL: http://codereview.chromium.org/4337003
2010-11-03 10:09:39 -07:00
Chris Sosa
a2a597d18b Add ability to fall back to fulls for all updates.
Currently works around issue till we get a good fix in for 9.x.

Change-Id: I9c7225508fa8f46a5fa5e0b41abc4e373cc80085

BUG=8606
TEST=Tested by sending SEGV to a delta update during update and saw it successfully fall back to a full update and succeed.

Review URL: http://codereview.chromium.org/4258002
2010-11-02 18:06:34 -07:00
Scott Zawalski
cb8c73b7fe Adding both for branch preflight
BUG=NA
TEST=NA

Review URL: http://codereview.chromium.org/4280002
2010-11-01 22:43:19 -07:00
Scott Zawalski
92a4d1c10b Add branch mario preflight with both public and private overlays
BUG=NA
TEST=NA

Review URL: http://codereview.chromium.org/4240002
2010-11-01 22:23:25 -07:00
David James
3d57ef8139 Update cros_mark_as_stable to support working on only private overlays.
BUG=chromium-os:8493
TEST=Ran unit tests. Verified that cros_mark_as_stable.py supports
running on just the private or public overlay.

Change-Id: I5e2e8fc5e63b3c93bb688a5ce87233315c0c2d1c

Review URL: http://codereview.chromium.org/4250001
2010-11-01 16:22:35 -07:00
Chris Sosa
b4b09d40a7 Fix versioning logic in ctest to get the correct latest version.
Change-Id: I2721bcd2da38b764ebe21dfb8fe7a39dd947075c

BUG=8500
TEST=Ran it against all boards

Review URL: http://codereview.chromium.org/4235001
2010-11-01 14:36:38 -07:00
Chris Sosa
113739c109 Add components to pass vm flag to dev server.
BUG=8306
TEST=Ran au_test_harness with CL for adding vm option for the devserver on the last channel release and my TOT.  All tests passed.

Review URL: http://codereview.chromium.org/4165009

Change-Id: Ia2ce305dff2911dc4008d5f6e383535e4bbd4ab0
2010-10-28 23:46:20 -07:00
Chris Sosa
42df54df1c Revert to fulls for boths tests if delta fails.
Change-Id: Iabc931f2eb1751ca9d05e92e64e5361755791648

BUG=
TEST=Copied and pasted from other try/except

Review URL: http://codereview.chromium.org/4109006
2010-10-28 14:39:11 -07:00
Chris Sosa
f8b650ac1a Re-try with full updates if first delta fails.
Change-Id: Ica7360dc4b2a62384874ba1b802039289747c6e3

BUG=
TEST=Ran it

Review URL: http://codereview.chromium.org/4137010
2010-10-28 04:12:23 -07:00
Chris Sosa
9b76b421e4 Use the right base path for vm updates.
Change-Id: I98276b26b85e48ae4b65e8f3fb65540b24665f9a

BUG=Will file
TEST=Ran it

Review URL: http://codereview.chromium.org/4185008
2010-10-28 03:41:14 -07:00
Chris Sosa
604a3f3168 Adding tracking branch broke repo sync on full builders.
Change-Id: Iff1a416a1f7c37870fbcd42ed4aaf6f96f1ea11d

BUG=8245
TEST=Ran with --clobber

Review URL: http://codereview.chromium.org/4128007
2010-10-27 11:51:02 -07:00
Chris Sosa
54f3b0bbd6 Add delta support to the update test harness.
Change-Id: I94159ac5468dd486123f7cb00da29526b5857d8f

BUG=8192
TEST=Tested with other CL's.

Review URL: http://codereview.chromium.org/4170004
2010-10-26 21:42:47 -07:00
Chris Sosa
8e3e834e4a Add delta update support to imaging tools.
Change-Id: I3ad6d418a1d47f47b78ba4e011c83bc569041475

BUG=6912
TEST=Ran with http://codereview.chromium.org/4129005/show with and without src_image

Review URL: http://codereview.chromium.org/4147002
2010-10-26 21:40:25 -07:00
David James
d2b72f7c2f Correctly initialize tracking branch in preflight rinse.
Currently cros/master is used for the tracking branch in the preflight
rinse step. This is wrong if --tracking_branch is specified
differently in the command-line options to cbuildbot.py.

BUG=none
TEST=Ran unit tests

Change-Id: Id4fb989863ae11f57889faf6a8d8b043b81da58e

Review URL: http://codereview.chromium.org/4104005
2010-10-26 12:35:37 -07:00
Chris Sosa
f38f249eb4 Update config for cros_au_test_harness.
Change-Id: Iccc5d28f4520418c58089cdb53b6e644271b2eb3

BUG=
TEST=n/a

Review URL: http://codereview.chromium.org/4051003
2010-10-24 02:13:27 -07:00
Chris Sosa
b00c32c335 Fix the tree by stating to right path.
Change-Id: I33e861066e8a585ce839ebc7a8c23243fb18cd1c

BUG=8083
TEST=Ran unit tests

Review URL: http://codereview.chromium.org/4053007
2010-10-23 23:38:48 -07:00
Chris Sosa
4a6e482b31 Gzip using the fastest option.
Change-Id: Id75e54879e896e42ef69659fc602f625858fd33f

BUG=
TEST=

Review URL: http://codereview.chromium.org/3989004
2010-10-22 17:30:21 -07:00
Chris Sosa
7e493cb4d3 Archive test results and image after running vm suite.
Change-Id: Ic6d4a79a7ab58b8043805a6b826f251faa9f3a6c

BUG=
TEST=Running now (adding unit tests as well)

Review URL: http://codereview.chromium.org/4062003
2010-10-22 16:32:30 -07:00
Chris Sosa
6b6f35ec65 Always pass board to cros_mark_as_stable.
Change-Id: I9a724e4e20b8945fdaa1dc53afdddf2477a5bbab

BUG=
TEST=Ran unit tests.

Review URL: http://codereview.chromium.org/3956007
2010-10-22 16:14:36 -07:00
Paul Stewart
f66c6d614e Complain (and more importantly don't blindly run) if AP or Client IP is unset
Add small if statement in cros_run_wifi_tests to balk if we didn't get
valid cell info from the server.

BUG=none
TEST=reboot everything and rerun while testbed server hasn't caught up yet

Review URL: http://codereview.chromium.org/4069001
2010-10-22 10:31:27 -07:00
Anush Elangovan
a6b1390fc5 Add branch support to preflight
Change-Id: Ia2592c6384e94e4e8e5400eddbf2e03aac150eef

BUG=7905
TEST=run cbuildbot on my workstation.

Review URL: http://codereview.chromium.org/4040002
2010-10-22 09:24:43 -07:00
Chris Sosa
dcd1de0bb9 Change preflight cleanup process to always be run before any run and add lazy unmount before every run.
Change-Id: If7db175f828fccdd4caa342a0370e842503f16dc

BUG=8015
TEST=Ran it with cbuildbot with/without --clobber with/without old state.

Review URL: http://codereview.chromium.org/4007005
2010-10-21 18:22:50 -07:00
David James
6be0f08169 Robustify and speed up cros_mark_all_as_stable.
What's new?
 - cros_mark_all_as_stable is now in Python and has unit tests.
 - We now detect and report coding errors that can cause incorrect behavior.
   E.g., if 9999 ebuild or stable ebuild can't be found, we report a hard
   failure so that developers will know about the problem.
 - We now check that git hashes we report actually match up with the
   right repository.
 - Unified diff of changes are now printed to stdout, so that developers
   can see a list of the changes and debug any problems.
 - cros_mark_all_as_stable now takes 2.5 seconds to run.

BUG=chromium-os:7795
TEST=Manually examined diff output from run of cros_mark_all_as_stable
     on full repository.

Change-Id: I762597c9b94e5f8e8171b83c966ad54e21a65c1b

Review URL: http://codereview.chromium.org/3798003
2010-10-21 16:11:00 -07:00
Chris Sosa
aa4670315b Fix equals in RunCommand's in cbuildbot.
Change-Id: If8cdde42a470c3d07cfd41552cf95dea7567688b

BUG=
TEST=Ran cbuildbot.

Review URL: http://codereview.chromium.org/3967002
2010-10-21 10:54:02 -07:00