Commit Graph

282 Commits

Author SHA1 Message Date
Don Garrett
2d08a983b2 I'm still breaking the build. Reverting again until I figure out why.
TBR

BUG=None
TEST=None

Review URL: http://codereview.chromium.org/6507034

Change-Id: I21283ed2b1f81086ccc95b0436e66c53584f4583
2011-02-16 18:39:35 -08:00
Don Garrett
e47eee3bc8 The original --quiet change failed (and was reverted) because cros_au_test_harness.py wasn't updated when the argument was changed from --quiet to --hide_info.
So... revert the reversion, and fix cros_au_test_harness.

BUG=chromium-os:11680
TEST=Manual

Review URL: http://codereview.chromium.org/6537005

Change-Id: I1bdf66a5735d4178ee1e2eaa838e7b4b7fb966d5
2011-02-16 15:49:14 -08:00
Don Garrett
8dcd981bbf Revert "Add --quiet to run_remote_tests, and use it"
This reverts commit 5b7e1d5da4.

Reverting to fix the build. TBR

BUG=None
TEST=None

Review URL: http://codereview.chromium.org/6480108

Change-Id: Idc9e60f97dd545225538420402c71b611233ce1e
2011-02-15 21:01:40 -08:00
Don Garrett
5b7e1d5da4 Add --quiet to run_remote_tests, and use it
Change-Id: I91f490ba4c4a834ba01b671c56c63ba230561ada

BUG=chromium-os:11680
TEST=Manual

Review URL: http://codereview.chromium.org/6529010
2011-02-15 17:40:56 -08:00
Chris Sosa
69cdce233e Fix tree closer. Deal with none case for PFQ.
Change-Id: Ia7883c196c688236051a4637561c9a01be5358cb

BUG=TreeCloser
TEST=Ran with simple prefix for x86-generic and no keys

Review URL: http://codereview.chromium.org/6480085
2011-02-15 12:51:06 -08:00
Chris Sosa
8dad50d97f Update test harness to take in optional public and private keys to sign payloads.
In addition some refactoring to make this cleaner and easier.

Change-Id: I1607700d065c71aff2b2833b10acbd3ebace68ce

BUG=chromium-os:8212
TEST=Ran with Simple ... running now with full

Review URL: http://codereview.chromium.org/6482017
2011-02-14 15:29:32 -08:00
Don Garrett
6620630c34 Switch ctest to array/appends for cros_au_test_harness call.
I was getting the error about passing along an empty string argument, and this will fix that.

Change-Id: Iffd05fe570f3a6092bbeee05abf8a1884decd644

BUG=N0ne
TEST=remote testing in progress (no VM testing)

Review URL: http://codereview.chromium.org/6465014
2011-02-09 13:51:43 -08:00
Chris Sosa
99768e8e12 Robustify starting of ssh connection to vm updater.
Change-Id: Ia5e0042250da22cebadfd48d143f21597bcada5d

BUG=chromium-os:11716
TEST=Ran vm test and vm update scripts.

Review URL: http://codereview.chromium.org/6413019
2011-02-07 09:02:22 -08:00
Mandeep Singh Baines
93da99be03 cros_workon: fix comments/messages to be consistent with new location
cros_workon has move to /usr/bin/

BUG=11507
TEST=none

Change-Id: I7301f659900713c432104ccceb3e658218e0ecac

Review URL: http://codereview.chromium.org/6334137
2011-02-05 11:42:19 -08:00
Anton Staaf
fd135cd9d0 Move cros_overlay_list to src/platform/dev.
This is step 3/3, after cros_overlay_list is added to dev, and
after the ebuilds are updated to install from dev and not
crosutils, then this can go in.

Change-Id: I8ae26938fb14ccf6737acd9ca99b58aaf65a1aca

BUG=chromium-os:11507
TEST=None

Review URL: http://codereview.chromium.org/6392034
2011-02-02 15:49:23 -08:00
Chris Sosa
79088f496e Remove buildbot code that has been added to chromite.
Change-Id: I7eb27ac81704de8689a65203a440303a52a01dc2

BUG=chromium-os:11171
TEST=we'll see if things explode

Review URL: http://codereview.chromium.org/6286040
2011-02-02 15:39:58 -08:00
Greg Spencer
798d75f3be This starts to fix the scripts so that they load from /usr/lib/crosutils
from within the chroot.

It also fixes a number of style issues.

It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.

It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.

BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.

Review URL: http://codereview.chromium.org/6240018

Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
2011-02-01 22:04:49 -08:00
Chris Sosa
558940dddb Make git config optional to allow multiple calls to enter_chroot at the same time.
It's very rarely needed anyway but enter_chroot does everything under the sun.

Avoids:

error: could not lock config file ~/.gitconfig

Change-Id: Ide72b14fd434b182a88d2fc636559b3515905509

BUG=chromium-os:11523
TEST=Ran it ... 10x

Review URL: http://codereview.chromium.org/6312069
2011-02-01 13:07:55 -08:00
Chris Sosa
19ef17732c Do not pregenerate update payloads in parallel.
TBR dgarret

Change-Id: I71ef4b02108f43717e4bc404290952df99112654

BUG=n0ne
TEST=NA

Review URL: http://codereview.chromium.org/6349021
2011-02-01 09:57:29 -08:00
Chris Sosa
39ed0d0ebe Clear cache for runs of ctest if --cache not set.
Change-Id: I013c838c83dd23cbfe04f77f87df74778d8e874e

BUG=chromium-os:10723
TEST=Ran it

Review URL: http://codereview.chromium.org/6389004
2011-01-31 12:12:41 -08:00
Chris Sosa
b8a84b2cb7 Change uprev to False for Chrome pfq.
Change-Id: I8c635cf228c9bf1c77d97656d6b38de5289ff9e6

BUG=
TEST=

Review URL: http://codereview.chromium.org/6347027
2011-01-28 22:48:46 -08:00
Chris Sosa
25272888e8 add cwd
TBR dgarret

Change-Id: I8d7f86d716f1da76f4c85259f401c3ccc9a031ff

BUG=
TEST=

Review URL: http://codereview.chromium.org/6392010
2011-01-28 20:53:08 -08:00
Chris Sosa
d7beed79b6 Rebuild chroot for full builders.
TBR dgarrett

Change-Id: Ibed5341626d498adeeb259cac54ebb05ca1b4421

BUG=
TEST=

Review URL: http://codereview.chromium.org/6407001
2011-01-28 18:43:46 -08:00
Chris Sosa
b62c4860f8 Fix issue where full builders end prematurely.
TBR dgarrett

Change-Id: I08b7c19b8010003857f9791ca45ec9cadb972d47

BUG=N0ne
TEST=Ran

Review URL: http://codereview.chromium.org/6352027
2011-01-28 18:30:02 -08:00
Don Garrett
7b8075ac6d Added arm-tegra2-seaboard-full, should have added in previous change.
Change-Id: Iecb40121b4bc8ee652fe72c4a8c07e3d231293bf

BUG=chromium-os:11229
TEST=Passed unittests (shows correct syntax)

Review URL: http://codereview.chromium.org/6379015
2011-01-28 14:52:17 -08:00
Chris Sosa
436209ec5b Have the ability for the PFQ to both rev Chrome and other packages.
This issue seems bigger than it first appears because we had put extra logic to "Delete" the branch if we didn't rev anything.  This CL basically takes out all the Deletes and relies and cros_mark_as_stable clean to clean up any old dirs correctly.  To do this, I had to fix a bug in clean so that we actually delete the stable branch as part of it (if it exists).

EOM

Change-Id: Ic1020451dc1d492d4b6e2a233d1d49cf8c1c16b3

BUG=chromium-os:11329
TEST=Ran x86-generic-pfq w/ dev options + --chrome_rev=latest_release ...built chrome,
and revved packages without error.  Doing more testing still.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=a8ac0ec

Review URL: http://codereview.chromium.org/6291013
2011-01-28 14:27:38 -08:00
Don Garrett
e7b9a4f558 Add new configs to cbuildbot_conf to match master.cfg
Change-Id: I4a28ac0f8e1a90313a9373e213e8a30003d1f8ad

BUG=chromium-os:11229
TEST=Unittests

Review URL: http://codereview.chromium.org/6268020
2011-01-28 14:14:26 -08:00
Vincent Palatin
3eb3543e9a Revert "Have the ability for the PFQ to both rev Chrome and ..."
This reverts the commit a8ac0ec8c3.

TBR the x86 PFQ buildbot fails

BUG=chromium-os:11447
TEST=
TBR=sosa

Change-Id: I019de669db13b753365e1208acc9923c4bb5c993

Review URL: http://codereview.chromium.org/6339018
2011-01-28 15:55:47 -05:00
Chris Sosa
a8ac0ec8c3 Have the ability for the PFQ to both rev Chrome and other packages.
This issue seems bigger than it first appears because we had put extra logic to "Delete" the branch if we didn't rev anything.  This CL basically takes out all the Deletes and relies and cros_mark_as_stable clean to clean up any old dirs correctly.  To do this, I had to fix a bug in clean so that we actually delete the stable branch as part of it (if it exists).

EOM

Change-Id: Ic1020451dc1d492d4b6e2a233d1d49cf8c1c16b3

BUG=chromium-os:11329
TEST=Ran x86-generic-pfq w/ dev options + --chrome_rev=latest_release ...built chrome,
and revved packages without error.  Doing more testing still.

Review URL: http://codereview.chromium.org/6291013
2011-01-28 11:17:57 -08:00
Don Garrett
38a66574b6 Add more cbuild options, revise configs
BUG=chromium-os:11229
TEST=Local, unittests

Review URL: http://codereview.chromium.org/6245013

Change-Id: I103d693a2e6ce033c1f495f9f48450156ad72e94
2011-01-27 14:04:15 -08:00
Chris Sosa
6e79ebeaa1 Actually die if a parallel test fails.
Change-Id: I24b498fffb7e7ce54517945730c91fecd1d57586

BUG=chromium-os:10723
TEST=Ran it and killed a test

Review URL: http://codereview.chromium.org/6260023
2011-01-27 11:18:25 -08:00
Anton Staaf
95e56d1ca2 Update cros_overlay_list to work from either src/scripts or /usr/bin.
Change-Id: If4c2852568454c2dccf09b0fc1e180ed228a1bb7

BUG=chromium-os:4230
TEST=run cros_overlay_list from both locations in the chroot and src/scripts outside the chroot.

Review URL: http://codereview.chromium.org/6293021
2011-01-27 09:38:46 -08:00
Chris Sosa
d6ca361c09 Add Chrome PFQ config that is a copy of current pfq.
Change-Id: Ie1ee6554ec0973a85a11c264346295fd27f59ebf

BUG=chromium-os:11329
TEST=Ran with cbuildbot on local system.

Review URL: http://codereview.chromium.org/6384017
2011-01-26 22:01:33 -08:00
Chris Sosa
fb58ea5d96 This change squashes output while pregenerationg updates.
It also caches sudo credentials before running jobs (for devs) and
exits if any updater returns an error code != 0 with the output
of pregenerate update

BUG=chromium-os:10723
TEST=Ran it ... killed one pregeneration to make sure it did print + exit
correctly

Review URL: http://codereview.chromium.org/6381013

Change-Id: I04842211e469b3a0022cb53b78efd17fbebe7010
2011-01-26 18:44:59 -08:00
Don Garrett
91c39378e0 Add buildbot functionality to cbuildbot.
Change-Id: I1291d2f52122fe345c5927a7ef27e20d07e917d7

BUG=chromium-os:11229
TEST=(building locally, still in progress)

Review URL: http://codereview.chromium.org/6317014
2011-01-26 11:53:26 -08:00
Chris Sosa
f2ee5f6871 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=be787f3

Review URL: http://codereview.chromium.org/6277015
2011-01-26 11:38:14 -08:00
Chris Sosa
404bfdf073 Revert "Passes cache location to tests and runs the tests in parallel."
This reverts commit be787f3525.

Revert "Fix sudo again."

This reverts commit 0f411ec7d9.

TBR dgarrett

BUG=
TEST=

Review URL: http://codereview.chromium.org/6348022

Change-Id: I2fb9ca4967ff173c17840b96eb68b02ff2b7774a
2011-01-25 18:16:55 -08:00
Chris Sosa
0f411ec7d9 Fix sudo again.
TBR:dgarrett

Change-Id: Idf2c15b873c4ee6c7865978d62f0b134c149b66f

BUG=chromium-os:10723
TEST=NA

Review URL: http://codereview.chromium.org/6384012
2011-01-25 17:59:46 -08:00
Chris Sosa
be787f3525 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f

Review URL: http://codereview.chromium.org/6277015
2011-01-25 17:16:56 -08:00
Chris Sosa
27729c7dbf Revert "Passes cache location to tests and runs the tests in parallel."
This reverts commit c418a8fce7.

TBR=open tree

Change-Id: I471d186fca9084fd18858998b4bcf8cb1de19f14
2011-01-25 15:12:39 -08:00
Chris Sosa
c418a8fce7 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Review URL: http://codereview.chromium.org/6277015
2011-01-25 13:49:29 -08:00
Chris Sosa
b6d82e8bb7 Pre-generate updates with root user.
image_to_live runs dev_server with sudo so old cached entries are
owned by root rather than $USER.  Change to pre-generate also with
sudo.

Change-Id: I587f15953b37d4de439977e7dbac9fced175c173

BUG=chromium-os:10723
TEST=Ran it

Review URL: http://codereview.chromium.org/6380008
2011-01-20 13:00:08 -08:00
Chris Sosa
a4cc3cd3ab Pre-generate updates for test harness in parallel.
First CL of many.  Thought I'd commit this before the others since
it sets stuff up for the other CL's (i.e. Thread Pool).

With this CL, all updates will be pre-generated before the test run.
This should improve some speeds since the two updates will be produced
at the same time rather than at the time of the sequential tests.

Change-Id: Iaf0a06f0d99c31d3d749e478fd2fb6efa0763a28

BUG=chromium-os:10723
TEST=Ran it with -q

Review URL: http://codereview.chromium.org/6264005
2011-01-20 10:23:54 -08:00
Don Garrett
e70ccb91b1 Use different proxy ports, and log port conflicts. Hopefully works around
build problems, and will give better diagnostics if not.

Change-Id: I2f7a5d9b2e0c63a8189ad212125c7a5c0178c3c4

BUG=10567
TEST=

Review URL: http://codereview.chromium.org/6331007
2011-01-19 13:41:23 -08:00
Chris Sosa
f867de71d0 Disable tests that don't work with verbose and set verbose as default.
Change-Id: I25066c75797ec7ce481ee096a2dc777d48634581

BUG=Get tree 0pen
TEST=Ran it

Review URL: http://codereview.chromium.org/6279002
2011-01-14 10:38:26 -08:00
Chris Sosa
4db7ac8f98 Print out SVN revision number for TOT chrome pfq builds.
Change-Id: I046d49c7002586671dc7c6707afeeeb0a0360cd5

BUG=10934
TEST=Ran it with bin/cros_mark_chrome_as_stable tot. Verified output and that it still did the right thing

Review URL: http://codereview.chromium.org/6327002
2011-01-13 14:11:18 -08:00
Chris Sosa
8617623516 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario.  Re-tested with new changes after build break.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=2b2d7a1

Review URL: http://codereview.chromium.org/6015013
2011-01-10 13:37:49 -08:00
Chris Sosa
f58bc389d9 Create package keywords directory if it doesn't exist.
Clobbering the builder uncovered a bug that was lurking.

Change-Id: Ieb487d2112a7048438943aa96fba23e9c8412a13

BUG=chromium-os:8693
TEST=Running now

Review URL: http://codereview.chromium.org/6050004
2011-01-10 10:58:58 -08:00
Chris Sosa
efbce1e261 Add simple update test to pfq.
This CL also fixes the outstanding issue where to won't report progress in
image_to_live on updates.

Change-Id: I1efaf17f0fd5ebb367ae0872377b4d5d0bf5dbf6

BUG=chromium-os:8680,chromium-os:8901
TEST=Ran it with --debug on the x86 pfq locally.

Review URL: http://codereview.chromium.org/6135002
2011-01-10 09:56:34 -08:00
J. Richard Barnette
e7205e84be Revert "Major cleanup of cros_au_test_harness code."
This reverts commit 2b2d7a1426.

The internal x86 buildbots have been reporting failures from
cros_au_test_harness with every build, starting with the build
that included the change being reverted.

BUG=chromium-os:8901
TEST=None

Review URL: http://codereview.chromium.org/5962019

Change-Id: Iac8c0a13d659f927b3ff61a9a894c10191432e2d
2011-01-05 18:38:49 -08:00
Chris Sosa
2b2d7a1426 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario

Review URL: http://codereview.chromium.org/6015013
2011-01-04 18:05:59 -08:00
Chris Sosa
50744ba57c Remove try/catch so that ctest blocks builds.
All blocking reasons for au test harness flakiness have been resolved.  Removing try/catch from ctest so that we propagate the correct error code upon exit.

Change-Id: Iee1f09ade37ac7a590c2dbf4ebb96985818f1882

BUG=chromium-os:10434
TEST=Ran ctest unittests.

Review URL: http://codereview.chromium.org/5971009
2011-01-04 12:17:17 -08:00
Chris Sosa
70311cdfdd Change logic to auto-rev for latest_release and sticky branch.
Also some cleanup and fixing so we don't create unnecessary ebuilds.
I noticed that both the latest release and stable release code
were trying to rev the same branch whenever we moved to a new stable
version.  It was harmless before but now that we're pushing this as
stable we want exactly to have the minimum number of revs that are
correct

Change-Id: I9f834d35be8e85c27ab0505ec81637f9a3c27b88

BUG=
TEST=Unittests and mocked in my own directory by creating new stable
branch.

Review URL: http://codereview.chromium.org/6025012
2010-12-29 15:23:15 -08:00
Chris Sosa
b1d17e24eb Move git config till after we sync.
This deals with the condition where on a clobber we don't actually
re-config to use ssh:// until the second run (because git config is
called before we first sync the code).

Change-Id: Icf1a46aced633b570db9d280b10ae6866af04709

BUG=chromium-os:10545
TEST=Unit tests

Review URL: http://codereview.chromium.org/6027006
2010-12-22 14:10:10 -08:00
Chris Sosa
4e32890c9b Add new script to stop a kvm using vm library and use in harness.
Change-Id: Ie80843a7b81a37b41ae19fc33244b5c0b8152282

BUG=chromium-os:10434
TEST=Ran it with full au test harness (still running but already showing promise).

Review URL: http://codereview.chromium.org/5988006
2010-12-21 14:56:32 -08:00
Chris Sosa
6b69ba7121 Always run build_packages regardless of sync for chrome builder.
Change-Id: If08f3a0c1212e3e7d4ae8dc9f8aa92e2454d4d74

BUG=chromium-os:8693
TEST=Ran it

Review URL: http://codereview.chromium.org/6016005
2010-12-20 11:40:29 -08:00
Chris Sosa
40fa0ded27 Fixes to get ctest up and running again.
We had a major issue because the latest dev channel release pre-dates the virtio change.  Because of this, ctest was connecting to update that channel using virtio (which is unsupported for that).  This change drops a file telling the vm lib to use e1000 as before.

Also, fixed stateful change logic.

Change-Id: I394a8cece71bdd0d55efa21ba5b6d24804432c6b

BUG=chromium-os:10434
TEST=Ran it with ctest and cros_au_test_harness on images with/without the file.

Review URL: http://codereview.chromium.org/5928005
2010-12-17 13:15:05 -08:00
Chris Sosa
d523d23f2e Remove explicit call to build Chrome.
Since the package.keywords change, it's already built as part of build_packages.
With this change we don't have to needlessly build chrome twice as part of the
pfq.

Change-Id: I8dae745496a5d6f6213663b034ffe8d279d4efd3

BUG=chromium-os:8693
TEST=Ran unittests

Review URL: http://codereview.chromium.org/5973003
2010-12-17 12:35:34 -08:00
Chris Sosa
78bc7d7415 Fix regression introduced by proxy port.
Change-Id: I42a6110d41e79bb0ad50817e2e76b06f7973f6c2

BUG=chromium-os:10434
TEST=Ran it with --test_prefix for delay payload test and it succeeded.

Review URL: http://codereview.chromium.org/5877007
2010-12-16 19:37:28 -08:00
Scott Zawalski
4228f22c97 Change tegra2 binary builder to use board=tegra2_dev-board instead of tegra2.
This aligns the binary builder with the full builder.

BUG=NA
TEST=NA

Review URL: http://codereview.chromium.org/5839004
2010-12-16 13:44:32 -08:00
Chris Sosa
f19dea58e9 Fix discovering of stable candidate with unstable ebuilds.
Also add some more information output when we detect that there
is nothing to do.

Change-Id: Ia3dcf0d3c276e10d3368d6bd508f3a0d72be79cd

BUG=chromium-os:8693
TEST=Ran it with all 3 paramaters and current overlay.

Review URL: http://codereview.chromium.org/5884004
2010-12-16 12:36:45 -08:00
Chris Sosa
7c3fdc13b0 Push prebuilts by default.
Change-Id: I0d03b037253f299e36c3621ff6e5970e9af2bb11

BUG=chromium-os:10375
TEST=NA

Review URL: http://codereview.chromium.org/5876002
2010-12-15 17:42:44 -08:00
Chris Sosa
d75d3b43c8 Undo all the manifest logic from cbuildbot.
We also print to /dev/stderr directly to avoid flushing issues which have been occassionally given us broken manifests printed out

Change-Id: I97819bbc31142432aa5ef926f99f2c4f90fd50ef

BUG=chromium-os:9714
TEST=Ran cbuildbot through sync

Review URL: http://codereview.chromium.org/5888003
2010-12-15 17:42:07 -08:00
Doug Anderson
9b0127c6f5 Fixed build failure w/ new sudo
Change-Id: I2760e33e1473ce04f7adeda39d0db5725eaa81ae

BUG=chromium-os:7072
TEST=Replaced ./build_packages with bash and ran command; valided env var set right.

Review URL: http://codereview.chromium.org/5930002
2010-12-15 16:51:49 -08:00
Chris Sosa
cd2196d6c5 Add package.keywords logic to build version of chrome we specify.
Change-Id: I32c17e7353b420d7a895acffab4579c9b861095c

BUG=chromium-os:8693
TEST=Ran it with chrome_rev=tot, latest_release and stable_release

Review URL: http://codereview.chromium.org/5854001
2010-12-14 17:57:02 -08:00
David McMahon
48db5d2068 Moving chromeos_version.sh to chromiumos_overlay.
BUG=chromium-os:9714
TEST=

Review URL: http://codereview.chromium.org/5513009

Change-Id: Ib569fe59f4e14551b198568b24014208b5d65647
2010-12-14 16:19:24 -08:00
Don Garrett
fdefe8664b Fix cros_run_vm_update --proxy_port support
BUG=chromium-os:10318
TEST=Manual

Review URL: http://codereview.chromium.org/5801001

Change-Id: I26f0cad29e2b36b045883f0e153e8994e6b436cd
2010-12-14 11:09:19 -08:00
Chris Sosa
97432e7e9e Don't push manifests for chrome pfq.
Change-Id: I252616725936b2c471026d39d06c3fbb2fa1ed65

BUG=chromium-os:8693
TEST=Ran with chrome_rev=tot

Review URL: http://codereview.chromium.org/5744003
2010-12-13 16:20:51 -08:00
Chris Sosa
d8a0a57c30 Add support to pushing unstable changes for the chrome pfq.
This specifically gets the pfq one step closer to being completely ready to rev.  We add --noprebuilts for the chrome pfq as its not ready to host prebuilts and instead we push unstable ebuilds.  Anush noticed an Anush today where we are revving even when we don't change, and I've addressed this also in this CL.

Specifically:

1)  Sort in cbuildbot long options by long format
2)  Add --noprebuilts so that the Chrome PFQ can use this for now.
3)  Fixed a unittest that had broken in a previous CL for cros_mark_chrome_as_stable.
4)  Add ability for chrome to only rev if something really has changed:

 -- This means, if the new ebuild either:
  -- doesn't have the exact same chrome_version
  -- or diff -Bu (actual diff) between last ebuild and new ebuild isn't the same (i.e. detecting a 9999 change or different CROS_SVN_COMMIT)

Change-Id: I9b289c2168d6868299573f5d7c7a676380ee497e

BUG=chromium-os:8693
TEST=Ran unittests (cbuildbot, cros_mark_*) and ran cbuildbot with new chrome pfq items.

Review URL: http://codereview.chromium.org/5783001
2010-12-13 11:27:26 -08:00
Chris Sosa
c663f209f7 Add ability to store the manifest in the manifest directory.
Change-Id: I1ac3ed12df00424ba5ba2d1c61fd6fe9716e13b9

BUG=chromium-os:9714
TEST=Ran new unittests as well as ran cbuildbot on local b/cbuild

Review URL: http://codereview.chromium.org/5689003
2010-12-13 11:11:35 -08:00
Don Garrett
8be02f96b6 Create a transparent proxy for test programs to use, with the
ability to filter or manipulate content as it's passed through.

Create two example tests using this utility that cause
updates to close early to test resumes, and to add delays
to make sure the update still completes correctly.

What other tests should be created for this?

BUG=chromium-os:8207
TEST=Run by hand

Review URL: http://codereview.chromium.org/5632002

Change-Id: Iefb8c8e223fb2ba6bad2c551f7c4403a0bec6ecf
2010-12-09 15:54:11 -08:00
Don Garrett
86965c9baf Save off --remote flag
Change-Id: I5621ff8ab81408ac3f7daed970a065641495c4eb

BUG=chromium-os:10179
TEST=none

Review URL: http://codereview.chromium.org/5676002
2010-12-09 15:47:38 -08:00
David James
ab5c6917c6 Update cbuildbot.py to use emptytree option instead of deleting board.
This allows us to avoid deleting the board and still ensure our packages are
in sync with the full buildbot. This allows us to detect conflicts between
old and new packages.

BUG=chromium-os:10127
TEST=Run cbuildbot.py with emptytree feature forced to on and verify that
     build_packages rebuilds all packages. Also run unit tests.

Change-Id: I83e186937e28eef44285724697895a4583910b58

Review URL: http://codereview.chromium.org/5652007
2010-12-09 14:52:46 -08:00
Chris Sosa
53ac954a36 Add some desired options to the au test harness.
Adds verbose flag to print output while updates are running rather than on error.
Adds test_prefix to specify a prefix of tests you want to run
Flushes stderr more reliably for builders and debugging purposes
Dies when stateful_updates fail

Change-Id: I9e4289deec3688c52b92763eee9334a8868f948e

BUG=chromium-os:6910
TEST=Ran with test_prefix and verbose on and off

Review URL: http://codereview.chromium.org/5573007
2010-12-09 11:29:31 -08:00
Chris Sosa
b5ed3128ee Change config to use stable vs. sticky.
Change-Id: I0dda2cfb6e11cb69a6a1a6a4bae478cab78b8947

BUG=chromium-os:8693
TEST=Ran it with cros_mark_chrome with stable_release
TBR=Config change to get chrome builder up.
2010-12-07 17:40:34 -08:00
Don Garrett
e55c89ab44 Create the latest_download dir if it doesn't exist.
I started to use /tmp, but it doesn't exist both inside and outside of the chroot. Using a consistent name like this has the advantage that the files will be cached and not redownloaded every time. Well, except on the build servers that wipe the related directories between builds.

Change-Id: I58b42d9f1bd18cc4eb44043ed2436d4ffa482767

BUG=chromium-os:10012
TEST=

Review URL: http://codereview.chromium.org/5597004
2010-12-06 15:59:14 -08:00
Anush Elangovan
dcd5b59d4a Disable chrome version check in vm_test since it is in a ebuild
Change-Id: I1a3f46cf37341255dbb01866cf68e5308bb36d83

BUG=10024
TEST=adhoc

Review URL: http://codereview.chromium.org/5603007
2010-12-04 17:21:38 -08:00
Chris Sosa
1a4a9665d4 Have upload_prebuilts respect the debug flag.
Chrome PFQ right now is uploading prebuilts which wasn't meant to happen :(

Change-Id: I502a0621a6c9f655e5a3f890dd79e00652925902

BUG=chromium-os:8693
TEST=Ran cbuildbot with --debug

Review URL: http://codereview.chromium.org/5595004
2010-12-03 16:28:33 -08:00
Mandeep Singh Baines
46b5e1d286 cros_workon_make: actually inplace when doing an install
Also, fixed to honour external FEATURES for the install case.

BUG=n0ne
TEST=Built a kernel with this CL.

Change-Id: I17d50cc382b297766761f12d47a555630fedc016

Review URL: http://codereview.chromium.org/5530004
2010-12-03 16:18:30 -08:00
Chris Sosa
4f09b96e70 Use release candidates as a backup in case we fail to find the latest release.
Change-Id: Ia94f1c983aec533fc2813f9c300266d394b8ee7c

BUG=chromium-os:9946
TEST=Ran ctest and saw through the download and start of the test harness

Review URL: http://codereview.chromium.org/5628002
2010-12-02 20:07:48 -08:00
Don Garrett
969a673045 Add testPartialUpdate, testCorruptedUpdate.
Add two new tests that test update-engine handling of a corrupted image,
or a truncated image.

BUG=chromium-os:9502
TEST=Ran against Dell L13, Mario

Review URL: http://codereview.chromium.org/5373008

Change-Id: Ie1043004f01131ea837798c06327c486e1901c15
2010-12-02 17:54:40 -08:00
Chris Sosa
55430beaef Add ability to push to subset of overlays we rev.
We also fix some of the configs for existing configurations.

Change-Id: I34baa9664ff9e91268c8bac976ab2ab0aee51bfb

BUG=chromium-os:9714
TEST=Running now with cbuildbot x86-generic-pre-flight-queue.

Review URL: http://codereview.chromium.org/5531002
2010-12-02 15:49:00 -08:00
David James
59a6600edd Update scripts to avoid using deprecated --test_case syntax.
TEST=Used cbuildbot.py to run tests
BUG=chromium-os:9893

Change-Id: I3c3856fee6fa09c4e2c829f3c8f23d098b4b6bb5

Review URL: http://codereview.chromium.org/5555003
2010-12-02 13:54:48 -08:00
David Rochberg
3b9107014b restart_in_chroot_if_needed can run scripts from bin/. cros_workon_now does this
BUG=chromium-os:9877
TEST=ran bin/cros_workon_now, cros_workon, set_shared_user_password

Note---this introduces another bashism to common.sh.  However, despite
comments to the contrary, common.sh does not actually parse under dash
without this change

Change-Id: I1e6b9751afa8341c100f3b8e0b96284835a53d17

Review URL: http://codereview.chromium.org/5444002
2010-12-02 10:45:21 -05:00
Dave Parker
7efdebf35f Added support for boards with only private overlays. (crosutils)
If a board has a public overlay it will be used as the primary overlay. However, with this change, if a board only has a private overlay it will be used as the primary.

BUG=7339
TEST=Ran setup_board, build_packages, and build_image on a board with only a private overlay (locally created). Ran setup_board, build_packages, and build_image on a board with both public and private overlays.

Change-Id: Ic56e4951272ddb80511a213e3a2e54419267a49a

Related Issues:
http://codereview.chromium.org/3544009
http://codereview.chromium.org/3571015

Review URL: http://codereview.chromium.org/3622003
2010-12-01 11:34:32 -08:00
David James
63c51c4d3c Update cbuildbot.py and prebuilt.py to deduplicate preflight prebuilts.
Preflight prebuilts reference the last full build, so they should always be complete as long as the last full build is still there.

Also add host prebuilts to the preflight prebuilts.

BUG=chromium-os:5311
TEST=Run unit tests for cbuildbot and prebuilt.py. Test runs of cbuildbot.py with --dryrun.

Review URL: http://codereview.chromium.org/5344002

Change-Id: Id95f94c02cc2f6cbd70a029d4f8b94617b7cc071
2010-11-30 11:45:11 -08:00
David James
c509a906da Update cbuildbot.py to upload prebuilts from preflight buildbot.
This change updates cbuildbot.py to upload prebuilts and reference them via PREFLIGHT_BINHOST.

Also ensure make.conf ends in a newline, as it looks nicer.

BUG=chromium-os:5311
TEST=Test that prebuilt.py updates PREFLIGHT_BINHOST in right make.conf
     file. Test that cbuildbot.py runs prebuilt.py with right
     arguments. Run unit tests for cbuildbot and prebuilt.py. Test runs of cbuildbot.py with --dryrun. Test upload of Packages file to local web server and check that emerge works with them.

Change-Id: Iad03c6c469e05b9ee1cceff69cbe2bdd51225e25

Review URL: http://codereview.chromium.org/4969003
2010-11-30 11:36:57 -08:00
Chris Masone
330375f692 [crosutils] Add confirmation warning to cros_workon_make --scrub
BUG=9717
TEST=run cros_workon_make --scrub and ensure that it only scrubs if the user types y or yes (case insensitive)

Change-Id: Ic1f8020fcca8bd57924391fcbf075d3e2adc5e4a

Review URL: http://codereview.chromium.org/5265009
2010-11-29 15:06:53 -08:00
Chris Masone
5c5a95e865 [crosutils] Add script to help build incrementally
cros_workon_make provides a make-style interface to incrementally building
    cros_workon-enabled packages, appropriately setting up the toolchain for
    the target board beforehand.

    To incrementally build your package:
      cros_workon_make --board <board> <package>

    To re-run your package's configure steps and compile:
      cros_workon_make --board <board> <package> --reconf

    To incrementally build your package and tests, and run the tests:
      cros_workon_make --board <board> <package> --tests

    To incrementally build and install your package:
      cros_workon_make --board <board> <package> --install

    To remove ALL files in your package repo not known by git:
      cros_workon_make --board <board> <package> --scrub

    With the exception of --scrub, these flags all compose

BUG=6843
TEST=build workon packages in place

Change-Id: Ie219074127549e1e29adad6d6a03142ab74e0172

Review URL: http://codereview.chromium.org/5244001
2010-11-29 12:49:10 -08:00
Chris Sosa
564373521e Reconfigure to use ssh connection rather than use different manifest.
This also removes the rw_checkout var as it's not longer changes the affect of whether we use the ssh url or not.

Note that this replaces the other CL from last night because the other CL didn't work.  The other CL still pulled in the same manifest that has http: url's for each git repo.  I reverted that change last night after I noticed this.

Change-Id: I3d4ad2be6887ac2cf4ed2009bad9cae6dfdf5bbf

BUG=chromium-os:9509
TEST=Ran with cbuildbot --clobber and incremental

Review URL: http://codereview.chromium.org/5278010
2010-11-29 10:42:49 -08:00
Chris Sosa
c58667b8c4 This CL adds the ability to Rev and Build Chrome using cros_mark_chrome_as_stable.
Other things:
Removing commit|clean|push command from cros_mark_chrome because it's unnecessary ... I'm just gonna use the same stabilizing branch as the other stable marker, so I can use the same clean / push from the other script.  This makes this change fit in much more nicely to cbuildbot.

Other changes:
Add ability for cros_mark_chrome_as_stable to communicate to calling script through stdout.
Make STABLE_BRANCH in cros_mark_as_stable public for above reason.
Removed push_options from cros_mark and accompanying change to cbuildbot as it isn't used anymore.
Made it easier to debug with cbuildbot (I ran cbuildbot A LOT of times to test this change so I felt the pain)...this includes:
  - only build when syncing
  - allow one to explicitly disable tests
  - use dryrun is options.debug is set

Change-Id: I413a2e81a99cdde2e4d9139561cd518245b9b346

BUG=chromium-os:8693
TEST=Ran cbuildbot with --notest --debug --nosync x86-pre-flight and
saw it go through correctly and dryrun push.  Ran cbuildbot with same and --chrome_rev=tot.  Ran with syncing and running tests too.

Review URL: http://codereview.chromium.org/5154008
2010-11-24 13:18:39 -08:00
Gaurav Shah
d02793a5d7 cros_make_image_bootable: Do a final sanity check on the rootfs by running e2fsck
This is a complement to the proposed fix for http://crosbug.com/9447, and I am getting it out first since it is a relatively simple change.

This change ensures that the rootfs on the image output by cros_make_image_bootable wasn't corrupted accidentally (for example, no boot.desc was provided and the defaults were incorrect for the image). This should catch rootfs errors that we end up discovering at delta AU time.

BUG=chromium-os:9578
TEST= Ran cros_make_image_bootable manually on an existing image with and without boot.desc. As expected, without boot.desc the output image rootfs had errors.

Also ran ./build_image - which worked fine too.

Change-Id: I4ed3a56634f37ab7d5ff2dc052ad607740356a40

Review URL: http://codereview.chromium.org/5216003
2010-11-24 12:48:14 -08:00
Thieu Le
ac13647eda Change _ArchiveTestResults to upload to Google Storage
BUG=chromium-os:8364
TEST=

Change-Id: Icecc8268fa2c12c8413caa2879785f00e4be5a0a

Review URL: http://codereview.chromium.org/4864001
2010-11-24 10:42:32 -08:00
Chris Sosa
444e3a6de0 Revert "Change url of manifest to use ssh one as opposed to mirrored repo."
This reverts commit 4847031faa.

TBR=Required for pushes to work.

Change-Id: I4f07407e518045ea77fa61942af58f57e68b5f8f
2010-11-24 00:54:48 -08:00
Chris Sosa
4847031faa Change url of manifest to use ssh one as opposed to mirrored repo.
Change-Id: I680e28bcafbfeb4f19e2404e2b28761e26111f7f

BUG=chromium-os:9509
TEST=Ran cbuildbot on my local builder checkout with both clobber and without.
It sync'd and built.

Review URL: http://codereview.chromium.org/5312002
2010-11-23 16:29:01 -08:00
Simon Glass
d73ec0e2a8 cros_image_to_target handles test images, less verbose
added --test option
removed superfluous output (e.g. from dd) unless --verbose given
added --debug option for when you really want that output
added use of pv for a nice progress bar

TEST=(run script several times)
BUG=chromium-os:9563

Change-Id: Ib9d89ad5649b6cb6d5db35f66f6a3fda1ada0c53

Review URL: http://codereview.chromium.org/5176002
2010-11-23 15:30:50 -08:00
Chris Sosa
e4836188a8 Add ability for cbuildbot to only run unit tests for revved packages.
Change-Id: I6744aac09fb075f91f710f16343fab45b8dc8af9

BUG=chromium-os:7283
TEST=Ran it with cbuildbot through the unit tests phase and looked
at the file and saw it get parsed correctly.

Review URL: http://codereview.chromium.org/5124006
2010-11-22 13:44:51 -08:00
Chris Sosa
ed210b40ff Add new chrome revving module to crosutils.
Have subsequent review for cbuildbot that calls this for the pfq.

This module  has 3 modes ... TOT, latest_release, sticky_release that describe the three different types of revs for chrome we will be supporting.

For tot, we grab the latest svn revision # from the chrome src tree, and rev to the corresponding ebuild i.e. 9.0.553.0_alpha-r1 with CROS_SVN_COMMIT=svn revision number.

For latest_release, we grab the latest release from the releases page, and create an ebuild (or rev a previously existing on with same version) and use the release candidate suffix _rc.

For sticky_release, we use much of the same logic as latest_release, however, we specifically look to rev _rc's in the sticky branch.

This change depends on http://codereview.chromium.org/5172003 for the cros_mark_as_stable changes.

Change-Id: Idc1f4cd905cc1dcb4c44422f6104760077a24b8d

BUG=chromium-os:8693
TEST=Ran cros_mark_as_stable with --packages="chromeos-base/chromeos-chrome" --chrome_version=0.8.65.0 --chrome_revision=67890 commit.  Re-ran unit-tests as well (though thinking about adding another unit test for chrome revving)

Review URL: http://codereview.chromium.org/4798001
2010-11-19 14:01:11 -08:00
Don Garrett
d0473d1e9f Don't pass "" to cros_run_vm_test if vm_graphics_flag isn't set.
This would not have been a problem if "--no_graphics" is used.

Change-Id: I3645391d25d4bb8618109ff5be0ca2fbe8e96a60

BUG=chromium-os:9373
TEST=Ran by hand

Review URL: http://codereview.chromium.org/5104006
2010-11-18 16:11:13 -08:00
Paul Stewart
f7597abb17 Use new stateful image builder
Also change download filenames

BUG=chromium-os:9308
TEST=Run script

Review URL: http://codereview.chromium.org/5149002

Change-Id: Id9ed150184559ab31274dc3b4f58bf9fe0a0f76c
2010-11-17 16:22:28 -08:00
David James
77f281a75e Move sanity checks for missing directories until after checkout happens.
Preflight clean needs to wait until after the checkout happens before
it asserts that all of our overlays exist.

BUG=chromium-os:9197
TEST=Ran cbuildbot.py with missing chromiumos-overlay dir

Change-Id: If66574f01d1e85741e971919a9fa2da34f85872d

Review URL: http://codereview.chromium.org/5069001
2010-11-15 20:39:00 -08:00
Chris Sosa
4e75efa930 Add option to check version of Chrome on target.
Change-Id: I34e9d64656e04d4f5c477914512219d9b8563b78

BUG=chromium-os:9137
TEST=Ran with both wrong and right versions against a Google Chrome and
Chromium build

Review URL: http://codereview.chromium.org/4942001
2010-11-15 15:12:06 -08:00
David James
b49c6df754 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=795bd30

Review URL: http://codereview.chromium.org/4442001
2010-11-15 13:06:42 -08:00
David James
02b3e4f3d5 Revert "Add more error checking to preflight queue."
This reverts commit 795bd30f06.

BUG=
TEST=

Review URL: http://codereview.chromium.org/4904003

Change-Id: I9148026ecd0df75c253cfcf0c67af3d26771dd21
2010-11-12 14:47:57 -08:00
David James
795bd30f06 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-12 14:33:14 -08:00