Commit Graph

48 Commits

Author SHA1 Message Date
Chris Sosa
c58667b8c4 This CL adds the ability to Rev and Build Chrome using cros_mark_chrome_as_stable.
Other things:
Removing commit|clean|push command from cros_mark_chrome because it's unnecessary ... I'm just gonna use the same stabilizing branch as the other stable marker, so I can use the same clean / push from the other script.  This makes this change fit in much more nicely to cbuildbot.

Other changes:
Add ability for cros_mark_chrome_as_stable to communicate to calling script through stdout.
Make STABLE_BRANCH in cros_mark_as_stable public for above reason.
Removed push_options from cros_mark and accompanying change to cbuildbot as it isn't used anymore.
Made it easier to debug with cbuildbot (I ran cbuildbot A LOT of times to test this change so I felt the pain)...this includes:
  - only build when syncing
  - allow one to explicitly disable tests
  - use dryrun is options.debug is set

Change-Id: I413a2e81a99cdde2e4d9139561cd518245b9b346

BUG=chromium-os:8693
TEST=Ran cbuildbot with --notest --debug --nosync x86-pre-flight and
saw it go through correctly and dryrun push.  Ran cbuildbot with same and --chrome_rev=tot.  Ran with syncing and running tests too.

Review URL: http://codereview.chromium.org/5154008
2010-11-24 13:18:39 -08:00
Chris Sosa
e4836188a8 Add ability for cbuildbot to only run unit tests for revved packages.
Change-Id: I6744aac09fb075f91f710f16343fab45b8dc8af9

BUG=chromium-os:7283
TEST=Ran it with cbuildbot through the unit tests phase and looked
at the file and saw it get parsed correctly.

Review URL: http://codereview.chromium.org/5124006
2010-11-22 13:44:51 -08:00
Chris Sosa
3e689730e4 The major change is refactoring to make functions more accessible for other modules.
Besides changing a lot of _ to not _, I added lots of docstrings, moved things around,
factored out the Marking component of the EBuildStableMarker (for use with other CL),
and fixed the version logic.

The versioning logic was too hacktastic.  So I took a note from David's changes to
take the information from pkgsplit rather than doing the ugly parsing I was doing before.
This is much more robust in the case of no _r*'s and _rc or _alpha*, etc.

This CL also adds the ability to test Push changes in 2 ways.  One using --dryrun which allows someone to call it by not actually push to the remote server and two, a unit test for the Push method.

Used in http://codereview.chromium.org/4798001/

BUG=chromiumos:8693
TEST=Ran unit tests and chrome mark as stable.  Will run some more tests.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=be485ce

Review URL: http://codereview.chromium.org/5172003

Change-Id: I6d1231c6f46d8cc2e5fb57d04f2d3417cfbfb4f8
2010-11-19 07:29:10 -08:00
Chris Sosa
c8d611583a Revert "The major change is refactoring to make functions more accessible for other modules."
This reverts commit be485ced43.

TBR=

Change-Id: I43d753d7e5171bba81ee39fab81a7b831b4711ae
2010-11-18 18:10:51 -08:00
Chris Sosa
be485ced43 The major change is refactoring to make functions more accessible for other modules.
Besides changing a lot of _ to not _, I added lots of docstrings, moved things around,
factored out the Marking component of the EBuildStableMarker (for use with other CL),
and fixed the version logic.

The versioning logic was too hacktastic.  So I took a note from David's changes to
take the information from pkgsplit rather than doing the ugly parsing I was doing before.
This is much more robust in the case of no _r*'s and _rc or _alpha*, etc.

This CL also adds the ability to test Push changes in 2 ways.  One using --dryrun which allows someone to call it by not actually push to the remote server and two, a unit test for the Push method.

Used in http://codereview.chromium.org/4798001/

BUG=chromiumos:8693
TEST=Ran unit tests and chrome mark as stable.  Will run some more tests.

Review URL: http://codereview.chromium.org/5172003

Change-Id: I4319dcd3a4235474ecfd61c680a5242bab5bd00b
2010-11-18 17:33:07 -08:00
Chris Sosa
9b054f0f88 Revert "The major change is refactoring to make functions more accessible for other modules."
This reverts commit 118629dae8.

Revert "Remove _ to fix tree."

This reverts commit a74ee71048.

Revert "Missed tracking branch to re-open tree."

This reverts commit 0b9f24e9be.

TBR=davidjames

Change-Id: Ic7057cd2a01e192ec78dd729a1f3eac7835b96c7
2010-11-18 15:57:20 -08:00
Chris Sosa
0b9f24e9be Missed tracking branch to re-open tree.
Change-Id: I2331cb6eda8c49111f30b62c42946952e450741f

TBR=davidjames
2010-11-18 15:31:52 -08:00
Chris Sosa
a74ee71048 Remove _ to fix tree.
Change-Id: I62a94f8d66ce9b133eda6c085da5e4957ff74796

BUG=
TEST=

Review URL: http://codereview.chromium.org/5138006
2010-11-18 15:05:45 -08:00
Chris Sosa
118629dae8 The major change is refactoring to make functions more accessible for other modules.
Besides changing a lot of _ to not _, I added lots of docstrings, moved things around,
factored out the Marking component of the EBuildStableMarker (for use with other CL),
and fixed the version logic.

The versioning logic was too hacktastic.  So I took a note from David's changes to
take the information from pkgsplit rather than doing the ugly parsing I was doing before.
This is much more robust in the case of no _r*'s and _rc or _alpha*, etc.

Used in http://codereview.chromium.org/4798001/

BUG=chromiumos:8693
TEST=Ran unit tests and chrome mark as stable.  Will run some more tests.

Review URL: http://codereview.chromium.org/5172003

Change-Id: Id6ef9a480591d1a28352f360c6d076564b43e318
2010-11-18 14:35:59 -08:00
Chris Sosa
684d5dbccf Add retries to pfq push.
Change-Id: Ie4632acf22bedb0c94de0cb3fe67b549fc2d0871

BUG=chromium-os:9323
TEST=Ran it and inserted exception for push_try < 3 and
mocked out push.

Review URL: http://codereview.chromium.org/5142003
2010-11-17 17:48:26 -08:00
David James
d4993a7311 Don't use output argument for CalledProcessError (Python 2.7 only)
Also add _Print statement so that stdout is printed.

BUG=chromium-os:9257
TEST=Run cros_mark_as_stable.py clean / commit

Change-Id: I9a01d6369d4b14ca0be812b8b1509c8375b1d575

Review URL: http://codereview.chromium.org/5063004
2010-11-16 15:47:25 -08:00
David James
f7502a6e03 Special-case preflight clean to allow for missing directories.
If there's no directory, I guess things are really, really clean.

BUG=chromium-os:9197
TEST=Ran cros_mark_as_stable.py clean with missing overlay directories.

Change-Id: I3f4811fe01bca56e850f823a973476884292e278

Review URL: http://codereview.chromium.org/4988004
2010-11-15 18:09:10 -08:00
David James
b49c6df754 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=795bd30

Review URL: http://codereview.chromium.org/4442001
2010-11-15 13:06:42 -08:00
David James
02b3e4f3d5 Revert "Add more error checking to preflight queue."
This reverts commit 795bd30f06.

BUG=
TEST=

Review URL: http://codereview.chromium.org/4904003

Change-Id: I9148026ecd0df75c253cfcf0c67af3d26771dd21
2010-11-12 14:47:57 -08:00
David James
795bd30f06 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-12 14:33:14 -08:00
David James
4beb5cb58f Revert "Add more error checking to preflight queue."
This reverts commit b4258c8bd4.

BUG=br0ken build
TBR=rspangler
TEST=

Review URL: http://codereview.chromium.org/4697007

Change-Id: I839ffd08d33fc5943696cfe31f2978fe32ce5fec
2010-11-11 10:51:07 -08:00
David James
b4258c8bd4 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-11 10:37:04 -08:00
Chris Sosa
de285e0238 Add 0.0.1 logic to new cros_mark.
Change-Id: I413d21dd60ce822af65d5c7c22fd2dba67858d43

BUG=6113
TEST=Ran unit tests and also created my own 9999 ebuild and ran with --all and
saw 0.0.1-r1.ebuild get created in stabilizing_branch

Review URL: http://codereview.chromium.org/4449001
2010-11-04 13:03:33 -07:00
David James
1f79aafb89 Fix broken quoting in cbuildbot.py, allowing for correct revving of ebuilds.
When you pass '--foo="bar baz"' to a script, the argument is literally
passed as "bar baz", with the quotes. So we need to not add unnecessary
quotes in there.

When enter_chroot.sh parses arguments, it actually removes the quotes
for you, and splits up arguments. As a result, it may actually
be necessary to (1) use the above broken syntax when you call scripts
using enter_chroot.sh; but (2) don't use that syntax when calling
scripts without enter_chroot.sh. That's a broken situation, so I've
added a TODO for that. I've also added more warnings to
cros_mark_as_stable.py so that it's easier to debug when it doesn't
work.

BUG=chromium-os:8633
TEST=Ran unit tests. Planning to watch TOT buildbot post-commit to make sure behavior is right.

Change-Id: Ia1a0e60153fec60cb7ed4db2da128ffd9ae81e23

Review URL: http://codereview.chromium.org/4385002
2010-11-03 11:15:11 -07:00
David James
3d57ef8139 Update cros_mark_as_stable to support working on only private overlays.
BUG=chromium-os:8493
TEST=Ran unit tests. Verified that cros_mark_as_stable.py supports
running on just the private or public overlay.

Change-Id: I5e2e8fc5e63b3c93bb688a5ce87233315c0c2d1c

Review URL: http://codereview.chromium.org/4250001
2010-11-01 16:22:35 -07:00
David James
af35e5c5f0 Update srcroot to be an absolute path.
This is required so that paths still work after we change directories.
Currently our branch builder uses this broken recipe :(

BUG=none
TEST=Check that "./cros_mark_as_stable.py -r .. clean" cleans both the private overlay and chromiumos-overlay

Change-Id: Ie4047fb091bf7af36dc28482ea73b085ec12893c

Review URL: http://codereview.chromium.org/4175004
2010-10-26 15:17:48 -07:00
Chris Sosa
6b6f35ec65 Always pass board to cros_mark_as_stable.
Change-Id: I9a724e4e20b8945fdaa1dc53afdddf2477a5bbab

BUG=
TEST=Ran unit tests.

Review URL: http://codereview.chromium.org/3956007
2010-10-22 16:14:36 -07:00
Chris Sosa
a851fe004a Remove env logic from cros_mark_as.
TBR

Change-Id: Ib720c2169762b240a3312472fc592ba69ba3494c

BUG=
TEST=

Review URL: http://codereview.chromium.org/4062004
2010-10-22 14:41:43 -07:00
Chris Sosa
804ad7df27 Clean package cache for packages you are building.
Change-Id: If8e3f2cfb649f138e48ab17539b2e02079ff1bed

BUG=
TEST=Ran with cros_mark --all clean / commit and with/without --verbose

Review URL: http://codereview.chromium.org/4019005
2010-10-22 14:29:48 -07:00
David James
676dff53cf Import portage when it's used to fix broken tree.
This is just a temporary workaround until buildbots have portage installed
outside the chroot. We should roll this back when that's fixed.

TBR=sosa@chromium.org
BUG=b0rked tree
TEST=ran unit tests

Change-Id: I004c1476e929212774218d50d10f371806cf6c7f

Review URL: http://codereview.chromium.org/4070002
2010-10-21 16:47:00 -07:00
David James
6be0f08169 Robustify and speed up cros_mark_all_as_stable.
What's new?
 - cros_mark_all_as_stable is now in Python and has unit tests.
 - We now detect and report coding errors that can cause incorrect behavior.
   E.g., if 9999 ebuild or stable ebuild can't be found, we report a hard
   failure so that developers will know about the problem.
 - We now check that git hashes we report actually match up with the
   right repository.
 - Unified diff of changes are now printed to stdout, so that developers
   can see a list of the changes and debug any problems.
 - cros_mark_all_as_stable now takes 2.5 seconds to run.

BUG=chromium-os:7795
TEST=Manually examined diff output from run of cros_mark_all_as_stable
     on full repository.

Change-Id: I762597c9b94e5f8e8171b83c966ad54e21a65c1b

Review URL: http://codereview.chromium.org/3798003
2010-10-21 16:11:00 -07:00
Anush Elangovan
189d85d20e Revert "Fix issue with stale cache from bad builds."
This reverts commit 365d4b0cc8.

Fix preflight breakage, this seems like the most possible cause. Will recommit if not.

BUG=
TEST=

Review URL: http://codereview.chromium.org/3767012

Change-Id: I93fe463fec77c445e7ff0ec86db25d5ef2997ad7
2010-10-15 11:36:53 -07:00
Chris Sosa
365d4b0cc8 Fix issue with stale cache from bad builds.
We always want to re-build packages we are revving, however, if the last
build succeeds in building but fails tests, we have a bad local cache that
has revved packages.  This avoids this issue by working on all packages we
are revving.

Change-Id: I3bd7463a4090b1f007d09be81bdf65657bd9c3f3

BUG=7589
TEST=Ran it with cbuildbot.

Review URL: http://codereview.chromium.org/3745006
2010-10-15 10:37:04 -07:00
David James
4a71ea31cb Update cros_mark_as_stable.py to also update the private overlay
BUG=chromium-os:7218
TEST=./cros_mark_all_as_stable --tracking_branch=cros/master

Change-Id: I0f7c4e2d2aa4fc6325721901a710bfc33fa39a91

Review URL: http://codereview.chromium.org/3516025
2010-10-14 14:03:18 -07:00
Chris Sosa
57e9808254 Fix keywords issue
Change-Id: If89d91624d4e83d8f6923e873bfb57b95fa3b666

BUG=
TEST=Ran with failing packages from buildbot.

Review URL: http://codereview.chromium.org/3330013
2010-09-08 15:03:06 -07:00
Chris Sosa
70e2c9e72a Move RunCommand, and Info/Warning/Die into common pylib
TEST=Ran all commands that used RunCommand or Die with
their associated unittests.

Review URL: http://codereview.chromium.org/3266004
2010-08-30 11:05:50 -07:00
Mandeep Singh Baines
fafb895a4c Fix prefligh buildbot
BUG=none
TEST=Ran it by hand and verified that the error message goes away.

Before this change there is an error about merge_branch already existing.

Change-Id: I8393da46d6bebf1f40d253863cfe4c28b76c1b8e

Review URL: http://codereview.chromium.org/3127036
2010-08-26 10:32:16 -07:00
Mandeep Singh Baines
f30a06ab91 cros_mark_as_stable: delete merge_branch if it already exists
Fixes issue with pfbb where commits aren't getting squashed

BUG=6102
TEST=Ran the code.

Change-Id: Ibbcb8882ee13abf3a7c283bb0d248eef8db31cb8

Review URL: http://codereview.chromium.org/3141045
2010-08-25 19:05:45 -07:00
Chris Sosa
a33a445a8d Don't mask out errors in cros_mark_as_stable
Change-Id: Ib6722b99008f8515aecc3f27896963291d197947

Review URL: http://codereview.chromium.org/3130031
2010-08-18 18:03:46 -07:00
Chris Sosa
b34604c49a Change default tracking branch for new workflow
As I was writing a doc to describe how to manually uprev I decided
that this was a good time to change the default to be consistent
with the new workflow.

Change-Id: I18932f66004161303f22835ab3b4a8436995300a

Review URL: http://codereview.chromium.org/3125024
2010-08-18 15:12:37 -07:00
Chris Sosa
bb1d3b0076 Don't throw an error if not on stabilizing branch and give more information on exit.
TEST=Ran "cros_mark_as_stable push" with no stabilizing_branch

Review URL: http://codereview.chromium.org/3159010

Change-Id: I31dadc354e224f0f19f0878581204678391ac108
2010-08-12 10:33:53 -07:00
Mandeep Singh Baines
91a5215c0b cros_mark_as_stable: use git push instead of git-cl push
Using git-cl push requires setting up Rietveld authentication
cookies on the bots. This is non-trivial. So instead just use
git. This removes a point of failure.

BUG=5258
TEST=Verified by successfully pushing.

Change-Id: Ic3cd68f2d2a9cc59bbe69270896172ee4a418226

Review URL: http://codereview.chromium.org/3015061
2010-08-06 10:14:03 -07:00
Mandeep Singh Baines
6d523e1a56 cros_mark_as_stable: modify push to use tracking branch
BUG=5258
TEST=Successfully pushed.

Change-Id: I985da7577e88f4a734f8281cae4aafce8dda5933

Review URL: http://codereview.chromium.org/3046051
2010-08-05 11:39:03 -07:00
Mandeep Singh Baines
642382095d cros_mark_as_stable: make it work better in repo
Instead of using master, use the tracking branch.

BUG=5258
TEST=Verified that cleanup now works with repo.

Change-Id: I6e415d9068d54b602238b89c77ba4dbe2f9ff4c6

Review URL: http://codereview.chromium.org/3076035
2010-08-05 11:01:36 -07:00
Mandeep Singh Baines
52ed3f969a cros_mark_as_stable: add a --srcroot command-line option
This is required to enable running cros_mark_as_stable from outside
the chroot.

BUG=5258
TEST=Successfully ran outside the chroot and pushed a change.
All unit tests continue to pass.

Change-Id: Ibd23ace6326b8453c132c416d6db6e42c8c2c239

Review URL: http://codereview.chromium.org/2884060
2010-08-02 11:48:52 -07:00
David James
265588e4f4 Add newline after CROS_WORKON_COMMIT="" line.
Currently, cros_mark_as_stable.py does not add a newline after its
CROS_WORKON_COMMIT="" line. This works fine for 9999 ebuilds that have a newline
after the EAPI="" line, but doesn't work for ebuilds that don't do this.

This should fix the preflight queue build.

TEST=Ran cros_mark_as_stable.py on the update engine and verified that the
     CROS_WORKON_COMMIT="" line was not joined together with the next line.
BUG=none

Review URL: http://codereview.chromium.org/3036029
2010-07-30 10:58:40 -07:00
Chris Sosa
ffff7d76cc Add attempt to update remote and rebase before pushing
TEST=Ran cros_mark_all

Review URL: http://codereview.chromium.org/3066010
2010-07-28 13:21:32 -07:00
Chris Sosa
80a3b2810a Add flag to cros_mark_as_stable to specify tracking_branch.
TEST=Ran using default option and then tried -t chromiumorg/master and
saw a correct failure since I don't have that branch to track against.

Review URL: http://codereview.chromium.org/3032019
2010-07-23 17:00:34 -07:00
Chris Sosa
2566c3fc3c Adding --desc_from_logs option from git cl upload
TEST=Tested git cl upload --desc_from_logs -m "Message" in http://codereview.chromium.org/3027010/show

Review URL: http://codereview.chromium.org/3048005
2010-07-21 12:19:08 -07:00
Chris Sosa
db51f31139 Add wrapper script to get list of updates to pass to stablizing script without other changes
TEST=Ran script

Review URL: http://codereview.chromium.org/3034011
2010-07-19 11:57:31 -07:00
Chris Sosa
92d3779dff Return error when error occurs by raising exception
Review URL: http://codereview.chromium.org/2819052
2010-07-19 11:12:20 -07:00
Chris Sosa
16caaf0c7d Adds remaining functions to cros_mark_as_stable and cleaned up error messaging.
TEST=Tested with following workflow:
cros_mark_as_stable -p 'crash-reporter' -i 'boogabooga' commit
cros_mark_as_stable push
cros_mark_as_stable clean

Also passed in some real options for push_options in a branch that had nothing in it.

Review URL: http://codereview.chromium.org/3017002
2010-07-16 15:04:26 -07:00
Chris Sosa
bebf3798e5 First cut at stable script
TEST=Tested by using crash-reporter and crash_ids set to "12345"

Review URL: http://codereview.chromium.org/2873016
2010-07-13 15:34:11 -07:00