Commit Graph

57 Commits

Author SHA1 Message Date
Chris Sosa
c58667b8c4 This CL adds the ability to Rev and Build Chrome using cros_mark_chrome_as_stable.
Other things:
Removing commit|clean|push command from cros_mark_chrome because it's unnecessary ... I'm just gonna use the same stabilizing branch as the other stable marker, so I can use the same clean / push from the other script.  This makes this change fit in much more nicely to cbuildbot.

Other changes:
Add ability for cros_mark_chrome_as_stable to communicate to calling script through stdout.
Make STABLE_BRANCH in cros_mark_as_stable public for above reason.
Removed push_options from cros_mark and accompanying change to cbuildbot as it isn't used anymore.
Made it easier to debug with cbuildbot (I ran cbuildbot A LOT of times to test this change so I felt the pain)...this includes:
  - only build when syncing
  - allow one to explicitly disable tests
  - use dryrun is options.debug is set

Change-Id: I413a2e81a99cdde2e4d9139561cd518245b9b346

BUG=chromium-os:8693
TEST=Ran cbuildbot with --notest --debug --nosync x86-pre-flight and
saw it go through correctly and dryrun push.  Ran cbuildbot with same and --chrome_rev=tot.  Ran with syncing and running tests too.

Review URL: http://codereview.chromium.org/5154008
2010-11-24 13:18:39 -08:00
Thieu Le
ac13647eda Change _ArchiveTestResults to upload to Google Storage
BUG=chromium-os:8364
TEST=

Change-Id: Icecc8268fa2c12c8413caa2879785f00e4be5a0a

Review URL: http://codereview.chromium.org/4864001
2010-11-24 10:42:32 -08:00
Chris Sosa
444e3a6de0 Revert "Change url of manifest to use ssh one as opposed to mirrored repo."
This reverts commit 4847031faa.

TBR=Required for pushes to work.

Change-Id: I4f07407e518045ea77fa61942af58f57e68b5f8f
2010-11-24 00:54:48 -08:00
Chris Sosa
4847031faa Change url of manifest to use ssh one as opposed to mirrored repo.
Change-Id: I680e28bcafbfeb4f19e2404e2b28761e26111f7f

BUG=chromium-os:9509
TEST=Ran cbuildbot on my local builder checkout with both clobber and without.
It sync'd and built.

Review URL: http://codereview.chromium.org/5312002
2010-11-23 16:29:01 -08:00
Chris Sosa
e4836188a8 Add ability for cbuildbot to only run unit tests for revved packages.
Change-Id: I6744aac09fb075f91f710f16343fab45b8dc8af9

BUG=chromium-os:7283
TEST=Ran it with cbuildbot through the unit tests phase and looked
at the file and saw it get parsed correctly.

Review URL: http://codereview.chromium.org/5124006
2010-11-22 13:44:51 -08:00
David James
77f281a75e Move sanity checks for missing directories until after checkout happens.
Preflight clean needs to wait until after the checkout happens before
it asserts that all of our overlays exist.

BUG=chromium-os:9197
TEST=Ran cbuildbot.py with missing chromiumos-overlay dir

Change-Id: If66574f01d1e85741e971919a9fa2da34f85872d

Review URL: http://codereview.chromium.org/5069001
2010-11-15 20:39:00 -08:00
David James
b49c6df754 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=795bd30

Review URL: http://codereview.chromium.org/4442001
2010-11-15 13:06:42 -08:00
David James
02b3e4f3d5 Revert "Add more error checking to preflight queue."
This reverts commit 795bd30f06.

BUG=
TEST=

Review URL: http://codereview.chromium.org/4904003

Change-Id: I9148026ecd0df75c253cfcf0c67af3d26771dd21
2010-11-12 14:47:57 -08:00
David James
795bd30f06 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-12 14:33:14 -08:00
David James
4beb5cb58f Revert "Add more error checking to preflight queue."
This reverts commit b4258c8bd4.

BUG=br0ken build
TBR=rspangler
TEST=

Review URL: http://codereview.chromium.org/4697007

Change-Id: I839ffd08d33fc5943696cfe31f2978fe32ce5fec
2010-11-11 10:51:07 -08:00
David James
f9b84d61a1 Define buildroot properly.
Followup to "Add more error checking to preflight queue."
BUG=Br0ken build
TBR=kliegs@chromium.org
TEST=Ran unit tests

Review URL: http://codereview.chromium.org/4742004

Change-Id: I63e52265ae404f0193cb1f9ab35f3f1ec266d2c1
2010-11-11 10:45:12 -08:00
David James
b4258c8bd4 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-11 10:37:04 -08:00
David James
1f79aafb89 Fix broken quoting in cbuildbot.py, allowing for correct revving of ebuilds.
When you pass '--foo="bar baz"' to a script, the argument is literally
passed as "bar baz", with the quotes. So we need to not add unnecessary
quotes in there.

When enter_chroot.sh parses arguments, it actually removes the quotes
for you, and splits up arguments. As a result, it may actually
be necessary to (1) use the above broken syntax when you call scripts
using enter_chroot.sh; but (2) don't use that syntax when calling
scripts without enter_chroot.sh. That's a broken situation, so I've
added a TODO for that. I've also added more warnings to
cros_mark_as_stable.py so that it's easier to debug when it doesn't
work.

BUG=chromium-os:8633
TEST=Ran unit tests. Planning to watch TOT buildbot post-commit to make sure behavior is right.

Change-Id: Ia1a0e60153fec60cb7ed4db2da128ffd9ae81e23

Review URL: http://codereview.chromium.org/4385002
2010-11-03 11:15:11 -07:00
David James
3d57ef8139 Update cros_mark_as_stable to support working on only private overlays.
BUG=chromium-os:8493
TEST=Ran unit tests. Verified that cros_mark_as_stable.py supports
running on just the private or public overlay.

Change-Id: I5e2e8fc5e63b3c93bb688a5ce87233315c0c2d1c

Review URL: http://codereview.chromium.org/4250001
2010-11-01 16:22:35 -07:00
David James
d2b72f7c2f Correctly initialize tracking branch in preflight rinse.
Currently cros/master is used for the tracking branch in the preflight
rinse step. This is wrong if --tracking_branch is specified
differently in the command-line options to cbuildbot.py.

BUG=none
TEST=Ran unit tests

Change-Id: Id4fb989863ae11f57889faf6a8d8b043b81da58e

Review URL: http://codereview.chromium.org/4104005
2010-10-26 12:35:37 -07:00
Chris Sosa
b00c32c335 Fix the tree by stating to right path.
Change-Id: I33e861066e8a585ce839ebc7a8c23243fb18cd1c

BUG=8083
TEST=Ran unit tests

Review URL: http://codereview.chromium.org/4053007
2010-10-23 23:38:48 -07:00
Chris Sosa
4a6e482b31 Gzip using the fastest option.
Change-Id: Id75e54879e896e42ef69659fc602f625858fd33f

BUG=
TEST=

Review URL: http://codereview.chromium.org/3989004
2010-10-22 17:30:21 -07:00
Chris Sosa
7e493cb4d3 Archive test results and image after running vm suite.
Change-Id: Ic6d4a79a7ab58b8043805a6b826f251faa9f3a6c

BUG=
TEST=Running now (adding unit tests as well)

Review URL: http://codereview.chromium.org/4062003
2010-10-22 16:32:30 -07:00
Chris Sosa
6b6f35ec65 Always pass board to cros_mark_as_stable.
Change-Id: I9a724e4e20b8945fdaa1dc53afdddf2477a5bbab

BUG=
TEST=Ran unit tests.

Review URL: http://codereview.chromium.org/3956007
2010-10-22 16:14:36 -07:00
Anush Elangovan
a6b1390fc5 Add branch support to preflight
Change-Id: Ia2592c6384e94e4e8e5400eddbf2e03aac150eef

BUG=7905
TEST=run cbuildbot on my workstation.

Review URL: http://codereview.chromium.org/4040002
2010-10-22 09:24:43 -07:00
Chris Sosa
dcd1de0bb9 Change preflight cleanup process to always be run before any run and add lazy unmount before every run.
Change-Id: If7db175f828fccdd4caa342a0370e842503f16dc

BUG=8015
TEST=Ran it with cbuildbot with/without --clobber with/without old state.

Review URL: http://codereview.chromium.org/4007005
2010-10-21 18:22:50 -07:00
David James
6be0f08169 Robustify and speed up cros_mark_all_as_stable.
What's new?
 - cros_mark_all_as_stable is now in Python and has unit tests.
 - We now detect and report coding errors that can cause incorrect behavior.
   E.g., if 9999 ebuild or stable ebuild can't be found, we report a hard
   failure so that developers will know about the problem.
 - We now check that git hashes we report actually match up with the
   right repository.
 - Unified diff of changes are now printed to stdout, so that developers
   can see a list of the changes and debug any problems.
 - cros_mark_all_as_stable now takes 2.5 seconds to run.

BUG=chromium-os:7795
TEST=Manually examined diff output from run of cros_mark_all_as_stable
     on full repository.

Change-Id: I762597c9b94e5f8e8171b83c966ad54e21a65c1b

Review URL: http://codereview.chromium.org/3798003
2010-10-21 16:11:00 -07:00
Chris Sosa
aa4670315b Fix equals in RunCommand's in cbuildbot.
Change-Id: If8cdde42a470c3d07cfd41552cf95dea7567688b

BUG=
TEST=Ran cbuildbot.

Review URL: http://codereview.chromium.org/3967002
2010-10-21 10:54:02 -07:00
Scott Zawalski
9310bb6bd3 Fix logic for debug and uprev
BUG=chromium-os:8000
TEST=NA

Review URL: http://codereview.chromium.org/4007003
2010-10-20 22:36:37 -07:00
Chris Sosa
debca51332 Add --debug option to cbuildbot for developers to use cbuildbot without pushing changes.
Change-Id: Idda6c7287cdb1863eb4abe8c56da2e763e9fe777

BUG=7926
TEST=Ran with --debug and verified I didn't push a change.

Review URL: http://codereview.chromium.org/3880002
2010-10-20 11:00:44 -07:00
Chris Sosa
5911f2003b Clean up any previous kvm state.
Change-Id: I01fecde5e687992ccf36889803a4cb84b71a38b2

BUG=7818
TEST=Ran with/without previous kvm instance

Review URL: http://codereview.chromium.org/3760012
2010-10-19 12:58:24 -07:00
David James
ef9f04e2fd Update cbuildbot to use git.chromium.org instead of src.chromium.org.
src.chromium.org is no longer supported for git operations.

BUG=chromium-os:7830
TEST=none

Change-Id: Ia7cd9ff76625441f16d20d2f9a8e0f752dca0586

Review URL: http://codereview.chromium.org/3782010
2010-10-15 16:18:29 -07:00
Anush Elangovan
189d85d20e Revert "Fix issue with stale cache from bad builds."
This reverts commit 365d4b0cc8.

Fix preflight breakage, this seems like the most possible cause. Will recommit if not.

BUG=
TEST=

Review URL: http://codereview.chromium.org/3767012

Change-Id: I93fe463fec77c445e7ff0ec86db25d5ef2997ad7
2010-10-15 11:36:53 -07:00
Chris Sosa
365d4b0cc8 Fix issue with stale cache from bad builds.
We always want to re-build packages we are revving, however, if the last
build succeeds in building but fails tests, we have a bad local cache that
has revved packages.  This avoids this issue by working on all packages we
are revving.

Change-Id: I3bd7463a4090b1f007d09be81bdf65657bd9c3f3

BUG=7589
TEST=Ran it with cbuildbot.

Review URL: http://codereview.chromium.org/3745006
2010-10-15 10:37:04 -07:00
Chris Sosa
4befe39060 Revert "Revert "Close tree when smoke bvt fails.""
This reverts commit 224f7548b5.

BUG=
TEST=

Review URL: http://codereview.chromium.org/3525022

Change-Id: Ib4e136b96a1d9dac7345598b281b053a47421e77
2010-10-07 19:04:05 -07:00
Chris Sosa
3ede50ed4e Add enabling of local account. This should get the suite_Smoke to go green on the pfq.
Change-Id: Id480f45836a81d7d994b92d7a45d16208e1dd11c

BUG=
TEST=Ran locally.  Saw all tests pass.

Review URL: http://codereview.chromium.org/3567010
2010-10-01 14:12:26 -07:00
Chris Sosa
c2db26c796 Add methods to run vm tests on x86-pre-flight-queue.
Right now we're leaving error_ok=True until the tests are shown to work on builder.

Change-Id: I6b8c690a0da30948389fd4312032c78d87115364

BUG=6906
TEST=Ran through it locally.

Review URL: http://codereview.chromium.org/3591002
2010-09-30 14:25:25 -07:00
David James
70b297a263 Update cbuildbot.py to print manifests when we sync.
If cbuildbot.py printed out a manifest when it syncs, it would make it possible
for us to reproduce the build later by looking at what revisions the buildbot
was using. This is useful for debugging.

Change-Id: I186ad01eef0090b6c618cb7ad583085197474018

BUG=chromium-os:7069
TEST=Ran test suite with 3441027 patched in (Sosa's patch that fixes an unrelated issue with the unit tests)

Review URL: http://codereview.chromium.org/3461028
2010-09-25 23:25:18 -07:00
David James
7bf8d2715e Switch off repo sync --jobs, and turn on repo --trace.
repo sync --jobs doesn't check for errors, so we shouldn't use it in
automated scripts. repo --trace is needed so that we can diagnose
whether git is hanging.

BUG=chromium-os:7048, chromium-os:6774
TEST=Ran test suite

Change-Id: I9e68cfffe841a231f5fabef951ea2d45b81c1d5e

Review URL: http://codereview.chromium.org/3419024
2010-09-25 23:23:00 -07:00
Chris Sosa
7de6531962 Faster checkout and cleanup of old builds.
Change-Id: Id81fe4e7bd9cae7a51a45b6ea8a535c4ea9430f8

BUG=
TEST=Ran locally on top of previous build area.

Review URL: http://codereview.chromium.org/3337007
2010-09-02 15:10:48 -07:00
Chris Sosa
e13960240b Fix cleanup so we don't have to clobber as much when failures occur
Change-Id: I120c49a38d2a17a2216296a68378fee7345f616a

TBR=petkov

Review URL: http://codereview.chromium.org/3255004
2010-08-30 22:37:28 -07:00
Chris Sosa
aaba25d817 Fix clobber. Strip any extra args and treat last arg as config.
Change-Id: I2bf951342c12d954bd1abbc8b878437c648b4ac7

TBR=Scottz
2010-08-30 20:28:11 -07:00
Chris Sosa
70e2c9e72a Move RunCommand, and Info/Warning/Die into common pylib
TEST=Ran all commands that used RunCommand or Die with
their associated unittests.

Review URL: http://codereview.chromium.org/3266004
2010-08-30 11:05:50 -07:00
Chris Sosa
1ec70345d9 Change clobber logic to be explicit
TEST=Tested with --clobber and without.

Change-Id: I6a938b3da673d97186cec79f25b22ea6176f1fdf

Review URL: http://codereview.chromium.org/3238012
2010-08-30 10:48:08 -07:00
Chris Sosa
0c13b46899 Move back to marking all until we have individual tree closing ready.
Review URL: http://codereview.chromium.org/3246003
2010-08-27 17:01:38 -07:00
Chris Sosa
0c156f9ccb TBR: Add replace to build_image 2010-08-27 13:11:56 -07:00
Chris Sosa
151a896d4b TBR: Fix missing enter_chroot's in pfq 2010-08-27 11:48:08 -07:00
Chris Sosa
6d58d427e3 Add all pfq configurations and add build_image / run_unit_tests.
BUG=5468
TEST=Ran default and ran x86-generic.  Took board values from master.cfg in buildbot

Review URL: http://codereview.chromium.org/3200006

Change-Id: I1d3e4216dc9ca9af0d9e8c1bfffabffa87f89b04
2010-08-27 10:04:31 -07:00
Chris Sosa
4b9baf9fa1 Add ability for cbuildbot master to synchronize with cbuildbot slaves.
TEST=Ran several non-checked in unit tests (will check in later) and tested
with cbuildbot as it is now.
BUG=5989

Review URL: http://codereview.chromium.org/3165052

Change-Id: I867a597b34bbf63261b9d19b57cef9631100c339
2010-08-26 14:06:56 -07:00
Chris Sosa
6f7d6cef0e Fix case where there is nothing to uprev.
TEST=Ran unit tests.

Change-Id: Iafd9702c5364511d79667ecc296eaf68fc89a5fb

Review URL: http://codereview.chromium.org/3159047
2010-08-26 12:34:20 -07:00
Chris Sosa
5140c15674 Fix bug with git packages with no matching ebuild.
Change-Id: I45454c837807fbe6e5996b76087bbfe25d7155ce

Review URL: http://codereview.chromium.org/3171032
2010-08-26 11:18:25 -07:00
Chris Sosa
e9b94eefd0 CBuildbot - Adds ability to only rev packages given by the sourcestamp.
The buildbot drops a source stamp (repo1.git@commit1 repo2.git@commit2 ...).
If CBuildbot finds the source stamp, it uses it and revs only those packages.
If this file is either missing or set to "None", it assumes a Force Build
has been pressed and keeps the old behavior (attempts to mark all).

TEST=Ran with 3 states of revisions file.
BUG=5006

Review URL: http://codereview.chromium.org/3163030

Change-Id: I226fd3bec642224b31ce51eee34d028043964943
2010-08-26 10:55:27 -07:00
Mandeep Singh Baines
a87699b198 cbuildbot: fix to point to new url
BUG=6102
TEST=none

Change-Id: I69312fed2d0eb4934a964792b4af06160b8d8eb5

Review URL: http://codereview.chromium.org/3135049
2010-08-25 14:39:18 -07:00
Chris Sosa
a8a4803f4f Fix cbuildbot. Mistakenly used str.open instead of open
TEST=Test underway.

Review URL: http://codereview.chromium.org/3116029
2010-08-20 10:55:48 -07:00
Mandeep Singh Baines
2e9e668dc0 cbuildbot: fix typo
BUG=5937
TEST=none

Change-Id: I5b4a79fff4f3d263812e1dd0392e21ec3bede69d

Review URL: http://codereview.chromium.org/3175030
2010-08-20 08:08:43 -07:00