193 Commits

Author SHA1 Message Date
Chris Sosa
f867de71d0 Disable tests that don't work with verbose and set verbose as default.
Change-Id: I25066c75797ec7ce481ee096a2dc777d48634581

BUG=Get tree 0pen
TEST=Ran it

Review URL: http://codereview.chromium.org/6279002
2011-01-14 10:38:26 -08:00
Chris Sosa
4db7ac8f98 Print out SVN revision number for TOT chrome pfq builds.
Change-Id: I046d49c7002586671dc7c6707afeeeb0a0360cd5

BUG=10934
TEST=Ran it with bin/cros_mark_chrome_as_stable tot. Verified output and that it still did the right thing

Review URL: http://codereview.chromium.org/6327002
2011-01-13 14:11:18 -08:00
Chris Sosa
8617623516 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario.  Re-tested with new changes after build break.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=2b2d7a1

Review URL: http://codereview.chromium.org/6015013
2011-01-10 13:37:49 -08:00
Chris Sosa
f58bc389d9 Create package keywords directory if it doesn't exist.
Clobbering the builder uncovered a bug that was lurking.

Change-Id: Ieb487d2112a7048438943aa96fba23e9c8412a13

BUG=chromium-os:8693
TEST=Running now

Review URL: http://codereview.chromium.org/6050004
2011-01-10 10:58:58 -08:00
Chris Sosa
efbce1e261 Add simple update test to pfq.
This CL also fixes the outstanding issue where to won't report progress in
image_to_live on updates.

Change-Id: I1efaf17f0fd5ebb367ae0872377b4d5d0bf5dbf6

BUG=chromium-os:8680,chromium-os:8901
TEST=Ran it with --debug on the x86 pfq locally.

Review URL: http://codereview.chromium.org/6135002
2011-01-10 09:56:34 -08:00
J. Richard Barnette
e7205e84be Revert "Major cleanup of cros_au_test_harness code."
This reverts commit 2b2d7a1426f485bc2c6fb02399c52618db6dd0f2.

The internal x86 buildbots have been reporting failures from
cros_au_test_harness with every build, starting with the build
that included the change being reverted.

BUG=chromium-os:8901
TEST=None

Review URL: http://codereview.chromium.org/5962019

Change-Id: Iac8c0a13d659f927b3ff61a9a894c10191432e2d
2011-01-05 18:38:49 -08:00
Chris Sosa
2b2d7a1426 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario

Review URL: http://codereview.chromium.org/6015013
2011-01-04 18:05:59 -08:00
Chris Sosa
50744ba57c Remove try/catch so that ctest blocks builds.
All blocking reasons for au test harness flakiness have been resolved.  Removing try/catch from ctest so that we propagate the correct error code upon exit.

Change-Id: Iee1f09ade37ac7a590c2dbf4ebb96985818f1882

BUG=chromium-os:10434
TEST=Ran ctest unittests.

Review URL: http://codereview.chromium.org/5971009
2011-01-04 12:17:17 -08:00
Chris Sosa
70311cdfdd Change logic to auto-rev for latest_release and sticky branch.
Also some cleanup and fixing so we don't create unnecessary ebuilds.
I noticed that both the latest release and stable release code
were trying to rev the same branch whenever we moved to a new stable
version.  It was harmless before but now that we're pushing this as
stable we want exactly to have the minimum number of revs that are
correct

Change-Id: I9f834d35be8e85c27ab0505ec81637f9a3c27b88

BUG=
TEST=Unittests and mocked in my own directory by creating new stable
branch.

Review URL: http://codereview.chromium.org/6025012
2010-12-29 15:23:15 -08:00
Chris Sosa
b1d17e24eb Move git config till after we sync.
This deals with the condition where on a clobber we don't actually
re-config to use ssh:// until the second run (because git config is
called before we first sync the code).

Change-Id: Icf1a46aced633b570db9d280b10ae6866af04709

BUG=chromium-os:10545
TEST=Unit tests

Review URL: http://codereview.chromium.org/6027006
2010-12-22 14:10:10 -08:00
Chris Sosa
4e32890c9b Add new script to stop a kvm using vm library and use in harness.
Change-Id: Ie80843a7b81a37b41ae19fc33244b5c0b8152282

BUG=chromium-os:10434
TEST=Ran it with full au test harness (still running but already showing promise).

Review URL: http://codereview.chromium.org/5988006
2010-12-21 14:56:32 -08:00
Chris Sosa
6b69ba7121 Always run build_packages regardless of sync for chrome builder.
Change-Id: If08f3a0c1212e3e7d4ae8dc9f8aa92e2454d4d74

BUG=chromium-os:8693
TEST=Ran it

Review URL: http://codereview.chromium.org/6016005
2010-12-20 11:40:29 -08:00
Chris Sosa
40fa0ded27 Fixes to get ctest up and running again.
We had a major issue because the latest dev channel release pre-dates the virtio change.  Because of this, ctest was connecting to update that channel using virtio (which is unsupported for that).  This change drops a file telling the vm lib to use e1000 as before.

Also, fixed stateful change logic.

Change-Id: I394a8cece71bdd0d55efa21ba5b6d24804432c6b

BUG=chromium-os:10434
TEST=Ran it with ctest and cros_au_test_harness on images with/without the file.

Review URL: http://codereview.chromium.org/5928005
2010-12-17 13:15:05 -08:00
Chris Sosa
d523d23f2e Remove explicit call to build Chrome.
Since the package.keywords change, it's already built as part of build_packages.
With this change we don't have to needlessly build chrome twice as part of the
pfq.

Change-Id: I8dae745496a5d6f6213663b034ffe8d279d4efd3

BUG=chromium-os:8693
TEST=Ran unittests

Review URL: http://codereview.chromium.org/5973003
2010-12-17 12:35:34 -08:00
Chris Sosa
78bc7d7415 Fix regression introduced by proxy port.
Change-Id: I42a6110d41e79bb0ad50817e2e76b06f7973f6c2

BUG=chromium-os:10434
TEST=Ran it with --test_prefix for delay payload test and it succeeded.

Review URL: http://codereview.chromium.org/5877007
2010-12-16 19:37:28 -08:00
Scott Zawalski
4228f22c97 Change tegra2 binary builder to use board=tegra2_dev-board instead of tegra2.
This aligns the binary builder with the full builder.

BUG=NA
TEST=NA

Review URL: http://codereview.chromium.org/5839004
2010-12-16 13:44:32 -08:00
Chris Sosa
f19dea58e9 Fix discovering of stable candidate with unstable ebuilds.
Also add some more information output when we detect that there
is nothing to do.

Change-Id: Ia3dcf0d3c276e10d3368d6bd508f3a0d72be79cd

BUG=chromium-os:8693
TEST=Ran it with all 3 paramaters and current overlay.

Review URL: http://codereview.chromium.org/5884004
2010-12-16 12:36:45 -08:00
Chris Sosa
7c3fdc13b0 Push prebuilts by default.
Change-Id: I0d03b037253f299e36c3621ff6e5970e9af2bb11

BUG=chromium-os:10375
TEST=NA

Review URL: http://codereview.chromium.org/5876002
2010-12-15 17:42:44 -08:00
Chris Sosa
d75d3b43c8 Undo all the manifest logic from cbuildbot.
We also print to /dev/stderr directly to avoid flushing issues which have been occassionally given us broken manifests printed out

Change-Id: I97819bbc31142432aa5ef926f99f2c4f90fd50ef

BUG=chromium-os:9714
TEST=Ran cbuildbot through sync

Review URL: http://codereview.chromium.org/5888003
2010-12-15 17:42:07 -08:00
Doug Anderson
9b0127c6f5 Fixed build failure w/ new sudo
Change-Id: I2760e33e1473ce04f7adeda39d0db5725eaa81ae

BUG=chromium-os:7072
TEST=Replaced ./build_packages with bash and ran command; valided env var set right.

Review URL: http://codereview.chromium.org/5930002
2010-12-15 16:51:49 -08:00
Chris Sosa
cd2196d6c5 Add package.keywords logic to build version of chrome we specify.
Change-Id: I32c17e7353b420d7a895acffab4579c9b861095c

BUG=chromium-os:8693
TEST=Ran it with chrome_rev=tot, latest_release and stable_release

Review URL: http://codereview.chromium.org/5854001
2010-12-14 17:57:02 -08:00
David McMahon
48db5d2068 Moving chromeos_version.sh to chromiumos_overlay.
BUG=chromium-os:9714
TEST=

Review URL: http://codereview.chromium.org/5513009

Change-Id: Ib569fe59f4e14551b198568b24014208b5d65647
2010-12-14 16:19:24 -08:00
Don Garrett
fdefe8664b Fix cros_run_vm_update --proxy_port support
BUG=chromium-os:10318
TEST=Manual

Review URL: http://codereview.chromium.org/5801001

Change-Id: I26f0cad29e2b36b045883f0e153e8994e6b436cd
2010-12-14 11:09:19 -08:00
Chris Sosa
97432e7e9e Don't push manifests for chrome pfq.
Change-Id: I252616725936b2c471026d39d06c3fbb2fa1ed65

BUG=chromium-os:8693
TEST=Ran with chrome_rev=tot

Review URL: http://codereview.chromium.org/5744003
2010-12-13 16:20:51 -08:00
Chris Sosa
d8a0a57c30 Add support to pushing unstable changes for the chrome pfq.
This specifically gets the pfq one step closer to being completely ready to rev.  We add --noprebuilts for the chrome pfq as its not ready to host prebuilts and instead we push unstable ebuilds.  Anush noticed an Anush today where we are revving even when we don't change, and I've addressed this also in this CL.

Specifically:

1)  Sort in cbuildbot long options by long format
2)  Add --noprebuilts so that the Chrome PFQ can use this for now.
3)  Fixed a unittest that had broken in a previous CL for cros_mark_chrome_as_stable.
4)  Add ability for chrome to only rev if something really has changed:

 -- This means, if the new ebuild either:
  -- doesn't have the exact same chrome_version
  -- or diff -Bu (actual diff) between last ebuild and new ebuild isn't the same (i.e. detecting a 9999 change or different CROS_SVN_COMMIT)

Change-Id: I9b289c2168d6868299573f5d7c7a676380ee497e

BUG=chromium-os:8693
TEST=Ran unittests (cbuildbot, cros_mark_*) and ran cbuildbot with new chrome pfq items.

Review URL: http://codereview.chromium.org/5783001
2010-12-13 11:27:26 -08:00
Chris Sosa
c663f209f7 Add ability to store the manifest in the manifest directory.
Change-Id: I1ac3ed12df00424ba5ba2d1c61fd6fe9716e13b9

BUG=chromium-os:9714
TEST=Ran new unittests as well as ran cbuildbot on local b/cbuild

Review URL: http://codereview.chromium.org/5689003
2010-12-13 11:11:35 -08:00
Don Garrett
8be02f96b6 Create a transparent proxy for test programs to use, with the
ability to filter or manipulate content as it's passed through.

Create two example tests using this utility that cause
updates to close early to test resumes, and to add delays
to make sure the update still completes correctly.

What other tests should be created for this?

BUG=chromium-os:8207
TEST=Run by hand

Review URL: http://codereview.chromium.org/5632002

Change-Id: Iefb8c8e223fb2ba6bad2c551f7c4403a0bec6ecf
2010-12-09 15:54:11 -08:00
Don Garrett
86965c9baf Save off --remote flag
Change-Id: I5621ff8ab81408ac3f7daed970a065641495c4eb

BUG=chromium-os:10179
TEST=none

Review URL: http://codereview.chromium.org/5676002
2010-12-09 15:47:38 -08:00
David James
ab5c6917c6 Update cbuildbot.py to use emptytree option instead of deleting board.
This allows us to avoid deleting the board and still ensure our packages are
in sync with the full buildbot. This allows us to detect conflicts between
old and new packages.

BUG=chromium-os:10127
TEST=Run cbuildbot.py with emptytree feature forced to on and verify that
     build_packages rebuilds all packages. Also run unit tests.

Change-Id: I83e186937e28eef44285724697895a4583910b58

Review URL: http://codereview.chromium.org/5652007
2010-12-09 14:52:46 -08:00
Chris Sosa
53ac954a36 Add some desired options to the au test harness.
Adds verbose flag to print output while updates are running rather than on error.
Adds test_prefix to specify a prefix of tests you want to run
Flushes stderr more reliably for builders and debugging purposes
Dies when stateful_updates fail

Change-Id: I9e4289deec3688c52b92763eee9334a8868f948e

BUG=chromium-os:6910
TEST=Ran with test_prefix and verbose on and off

Review URL: http://codereview.chromium.org/5573007
2010-12-09 11:29:31 -08:00
Chris Sosa
b5ed3128ee Change config to use stable vs. sticky.
Change-Id: I0dda2cfb6e11cb69a6a1a6a4bae478cab78b8947

BUG=chromium-os:8693
TEST=Ran it with cros_mark_chrome with stable_release
TBR=Config change to get chrome builder up.
2010-12-07 17:40:34 -08:00
Don Garrett
e55c89ab44 Create the latest_download dir if it doesn't exist.
I started to use /tmp, but it doesn't exist both inside and outside of the chroot. Using a consistent name like this has the advantage that the files will be cached and not redownloaded every time. Well, except on the build servers that wipe the related directories between builds.

Change-Id: I58b42d9f1bd18cc4eb44043ed2436d4ffa482767

BUG=chromium-os:10012
TEST=

Review URL: http://codereview.chromium.org/5597004
2010-12-06 15:59:14 -08:00
Anush Elangovan
dcd5b59d4a Disable chrome version check in vm_test since it is in a ebuild
Change-Id: I1a3f46cf37341255dbb01866cf68e5308bb36d83

BUG=10024
TEST=adhoc

Review URL: http://codereview.chromium.org/5603007
2010-12-04 17:21:38 -08:00
Chris Sosa
1a4a9665d4 Have upload_prebuilts respect the debug flag.
Chrome PFQ right now is uploading prebuilts which wasn't meant to happen :(

Change-Id: I502a0621a6c9f655e5a3f890dd79e00652925902

BUG=chromium-os:8693
TEST=Ran cbuildbot with --debug

Review URL: http://codereview.chromium.org/5595004
2010-12-03 16:28:33 -08:00
Mandeep Singh Baines
46b5e1d286 cros_workon_make: actually inplace when doing an install
Also, fixed to honour external FEATURES for the install case.

BUG=n0ne
TEST=Built a kernel with this CL.

Change-Id: I17d50cc382b297766761f12d47a555630fedc016

Review URL: http://codereview.chromium.org/5530004
2010-12-03 16:18:30 -08:00
Chris Sosa
4f09b96e70 Use release candidates as a backup in case we fail to find the latest release.
Change-Id: Ia94f1c983aec533fc2813f9c300266d394b8ee7c

BUG=chromium-os:9946
TEST=Ran ctest and saw through the download and start of the test harness

Review URL: http://codereview.chromium.org/5628002
2010-12-02 20:07:48 -08:00
Don Garrett
969a673045 Add testPartialUpdate, testCorruptedUpdate.
Add two new tests that test update-engine handling of a corrupted image,
or a truncated image.

BUG=chromium-os:9502
TEST=Ran against Dell L13, Mario

Review URL: http://codereview.chromium.org/5373008

Change-Id: Ie1043004f01131ea837798c06327c486e1901c15
2010-12-02 17:54:40 -08:00
Chris Sosa
55430beaef Add ability to push to subset of overlays we rev.
We also fix some of the configs for existing configurations.

Change-Id: I34baa9664ff9e91268c8bac976ab2ab0aee51bfb

BUG=chromium-os:9714
TEST=Running now with cbuildbot x86-generic-pre-flight-queue.

Review URL: http://codereview.chromium.org/5531002
2010-12-02 15:49:00 -08:00
David James
59a6600edd Update scripts to avoid using deprecated --test_case syntax.
TEST=Used cbuildbot.py to run tests
BUG=chromium-os:9893

Change-Id: I3c3856fee6fa09c4e2c829f3c8f23d098b4b6bb5

Review URL: http://codereview.chromium.org/5555003
2010-12-02 13:54:48 -08:00
David Rochberg
3b9107014b restart_in_chroot_if_needed can run scripts from bin/. cros_workon_now does this
BUG=chromium-os:9877
TEST=ran bin/cros_workon_now, cros_workon, set_shared_user_password

Note---this introduces another bashism to common.sh.  However, despite
comments to the contrary, common.sh does not actually parse under dash
without this change

Change-Id: I1e6b9751afa8341c100f3b8e0b96284835a53d17

Review URL: http://codereview.chromium.org/5444002
2010-12-02 10:45:21 -05:00
Dave Parker
7efdebf35f Added support for boards with only private overlays. (crosutils)
If a board has a public overlay it will be used as the primary overlay. However, with this change, if a board only has a private overlay it will be used as the primary.

BUG=7339
TEST=Ran setup_board, build_packages, and build_image on a board with only a private overlay (locally created). Ran setup_board, build_packages, and build_image on a board with both public and private overlays.

Change-Id: Ic56e4951272ddb80511a213e3a2e54419267a49a

Related Issues:
http://codereview.chromium.org/3544009
http://codereview.chromium.org/3571015

Review URL: http://codereview.chromium.org/3622003
2010-12-01 11:34:32 -08:00
David James
63c51c4d3c Update cbuildbot.py and prebuilt.py to deduplicate preflight prebuilts.
Preflight prebuilts reference the last full build, so they should always be complete as long as the last full build is still there.

Also add host prebuilts to the preflight prebuilts.

BUG=chromium-os:5311
TEST=Run unit tests for cbuildbot and prebuilt.py. Test runs of cbuildbot.py with --dryrun.

Review URL: http://codereview.chromium.org/5344002

Change-Id: Id95f94c02cc2f6cbd70a029d4f8b94617b7cc071
2010-11-30 11:45:11 -08:00
David James
c509a906da Update cbuildbot.py to upload prebuilts from preflight buildbot.
This change updates cbuildbot.py to upload prebuilts and reference them via PREFLIGHT_BINHOST.

Also ensure make.conf ends in a newline, as it looks nicer.

BUG=chromium-os:5311
TEST=Test that prebuilt.py updates PREFLIGHT_BINHOST in right make.conf
     file. Test that cbuildbot.py runs prebuilt.py with right
     arguments. Run unit tests for cbuildbot and prebuilt.py. Test runs of cbuildbot.py with --dryrun. Test upload of Packages file to local web server and check that emerge works with them.

Change-Id: Iad03c6c469e05b9ee1cceff69cbe2bdd51225e25

Review URL: http://codereview.chromium.org/4969003
2010-11-30 11:36:57 -08:00
Chris Masone
330375f692 [crosutils] Add confirmation warning to cros_workon_make --scrub
BUG=9717
TEST=run cros_workon_make --scrub and ensure that it only scrubs if the user types y or yes (case insensitive)

Change-Id: Ic1f8020fcca8bd57924391fcbf075d3e2adc5e4a

Review URL: http://codereview.chromium.org/5265009
2010-11-29 15:06:53 -08:00
Chris Masone
5c5a95e865 [crosutils] Add script to help build incrementally
cros_workon_make provides a make-style interface to incrementally building
    cros_workon-enabled packages, appropriately setting up the toolchain for
    the target board beforehand.

    To incrementally build your package:
      cros_workon_make --board <board> <package>

    To re-run your package's configure steps and compile:
      cros_workon_make --board <board> <package> --reconf

    To incrementally build your package and tests, and run the tests:
      cros_workon_make --board <board> <package> --tests

    To incrementally build and install your package:
      cros_workon_make --board <board> <package> --install

    To remove ALL files in your package repo not known by git:
      cros_workon_make --board <board> <package> --scrub

    With the exception of --scrub, these flags all compose

BUG=6843
TEST=build workon packages in place

Change-Id: Ie219074127549e1e29adad6d6a03142ab74e0172

Review URL: http://codereview.chromium.org/5244001
2010-11-29 12:49:10 -08:00
Chris Sosa
564373521e Reconfigure to use ssh connection rather than use different manifest.
This also removes the rw_checkout var as it's not longer changes the affect of whether we use the ssh url or not.

Note that this replaces the other CL from last night because the other CL didn't work.  The other CL still pulled in the same manifest that has http: url's for each git repo.  I reverted that change last night after I noticed this.

Change-Id: I3d4ad2be6887ac2cf4ed2009bad9cae6dfdf5bbf

BUG=chromium-os:9509
TEST=Ran with cbuildbot --clobber and incremental

Review URL: http://codereview.chromium.org/5278010
2010-11-29 10:42:49 -08:00
Chris Sosa
c58667b8c4 This CL adds the ability to Rev and Build Chrome using cros_mark_chrome_as_stable.
Other things:
Removing commit|clean|push command from cros_mark_chrome because it's unnecessary ... I'm just gonna use the same stabilizing branch as the other stable marker, so I can use the same clean / push from the other script.  This makes this change fit in much more nicely to cbuildbot.

Other changes:
Add ability for cros_mark_chrome_as_stable to communicate to calling script through stdout.
Make STABLE_BRANCH in cros_mark_as_stable public for above reason.
Removed push_options from cros_mark and accompanying change to cbuildbot as it isn't used anymore.
Made it easier to debug with cbuildbot (I ran cbuildbot A LOT of times to test this change so I felt the pain)...this includes:
  - only build when syncing
  - allow one to explicitly disable tests
  - use dryrun is options.debug is set

Change-Id: I413a2e81a99cdde2e4d9139561cd518245b9b346

BUG=chromium-os:8693
TEST=Ran cbuildbot with --notest --debug --nosync x86-pre-flight and
saw it go through correctly and dryrun push.  Ran cbuildbot with same and --chrome_rev=tot.  Ran with syncing and running tests too.

Review URL: http://codereview.chromium.org/5154008
2010-11-24 13:18:39 -08:00
Gaurav Shah
d02793a5d7 cros_make_image_bootable: Do a final sanity check on the rootfs by running e2fsck
This is a complement to the proposed fix for http://crosbug.com/9447, and I am getting it out first since it is a relatively simple change.

This change ensures that the rootfs on the image output by cros_make_image_bootable wasn't corrupted accidentally (for example, no boot.desc was provided and the defaults were incorrect for the image). This should catch rootfs errors that we end up discovering at delta AU time.

BUG=chromium-os:9578
TEST= Ran cros_make_image_bootable manually on an existing image with and without boot.desc. As expected, without boot.desc the output image rootfs had errors.

Also ran ./build_image - which worked fine too.

Change-Id: I4ed3a56634f37ab7d5ff2dc052ad607740356a40

Review URL: http://codereview.chromium.org/5216003
2010-11-24 12:48:14 -08:00
Thieu Le
ac13647eda Change _ArchiveTestResults to upload to Google Storage
BUG=chromium-os:8364
TEST=

Change-Id: Icecc8268fa2c12c8413caa2879785f00e4be5a0a

Review URL: http://codereview.chromium.org/4864001
2010-11-24 10:42:32 -08:00
Chris Sosa
444e3a6de0 Revert "Change url of manifest to use ssh one as opposed to mirrored repo."
This reverts commit 4847031faad12e0f7b2fd3f047c77cad7b0a4003.

TBR=Required for pushes to work.

Change-Id: I4f07407e518045ea77fa61942af58f57e68b5f8f
2010-11-24 00:54:48 -08:00