Commit Graph

40 Commits

Author SHA1 Message Date
Chris Sosa
8dad50d97f Update test harness to take in optional public and private keys to sign payloads.
In addition some refactoring to make this cleaner and easier.

Change-Id: I1607700d065c71aff2b2833b10acbd3ebace68ce

BUG=chromium-os:8212
TEST=Ran with Simple ... running now with full

Review URL: http://codereview.chromium.org/6482017
2011-02-14 15:29:32 -08:00
Chris Sosa
558940dddb Make git config optional to allow multiple calls to enter_chroot at the same time.
It's very rarely needed anyway but enter_chroot does everything under the sun.

Avoids:

error: could not lock config file ~/.gitconfig

Change-Id: Ide72b14fd434b182a88d2fc636559b3515905509

BUG=chromium-os:11523
TEST=Ran it ... 10x

Review URL: http://codereview.chromium.org/6312069
2011-02-01 13:07:55 -08:00
Chris Sosa
19ef17732c Do not pregenerate update payloads in parallel.
TBR dgarret

Change-Id: I71ef4b02108f43717e4bc404290952df99112654

BUG=n0ne
TEST=NA

Review URL: http://codereview.chromium.org/6349021
2011-02-01 09:57:29 -08:00
Chris Sosa
6e79ebeaa1 Actually die if a parallel test fails.
Change-Id: I24b498fffb7e7ce54517945730c91fecd1d57586

BUG=chromium-os:10723
TEST=Ran it and killed a test

Review URL: http://codereview.chromium.org/6260023
2011-01-27 11:18:25 -08:00
Chris Sosa
fb58ea5d96 This change squashes output while pregenerationg updates.
It also caches sudo credentials before running jobs (for devs) and
exits if any updater returns an error code != 0 with the output
of pregenerate update

BUG=chromium-os:10723
TEST=Ran it ... killed one pregeneration to make sure it did print + exit
correctly

Review URL: http://codereview.chromium.org/6381013

Change-Id: I04842211e469b3a0022cb53b78efd17fbebe7010
2011-01-26 18:44:59 -08:00
Chris Sosa
f2ee5f6871 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=be787f3

Review URL: http://codereview.chromium.org/6277015
2011-01-26 11:38:14 -08:00
Chris Sosa
404bfdf073 Revert "Passes cache location to tests and runs the tests in parallel."
This reverts commit be787f3525.

Revert "Fix sudo again."

This reverts commit 0f411ec7d9.

TBR dgarrett

BUG=
TEST=

Review URL: http://codereview.chromium.org/6348022

Change-Id: I2fb9ca4967ff173c17840b96eb68b02ff2b7774a
2011-01-25 18:16:55 -08:00
Chris Sosa
0f411ec7d9 Fix sudo again.
TBR:dgarrett

Change-Id: Idf2c15b873c4ee6c7865978d62f0b134c149b66f

BUG=chromium-os:10723
TEST=NA

Review URL: http://codereview.chromium.org/6384012
2011-01-25 17:59:46 -08:00
Chris Sosa
be787f3525 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f

Review URL: http://codereview.chromium.org/6277015
2011-01-25 17:16:56 -08:00
Chris Sosa
27729c7dbf Revert "Passes cache location to tests and runs the tests in parallel."
This reverts commit c418a8fce7.

TBR=open tree

Change-Id: I471d186fca9084fd18858998b4bcf8cb1de19f14
2011-01-25 15:12:39 -08:00
Chris Sosa
c418a8fce7 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Review URL: http://codereview.chromium.org/6277015
2011-01-25 13:49:29 -08:00
Chris Sosa
b6d82e8bb7 Pre-generate updates with root user.
image_to_live runs dev_server with sudo so old cached entries are
owned by root rather than $USER.  Change to pre-generate also with
sudo.

Change-Id: I587f15953b37d4de439977e7dbac9fced175c173

BUG=chromium-os:10723
TEST=Ran it

Review URL: http://codereview.chromium.org/6380008
2011-01-20 13:00:08 -08:00
Chris Sosa
a4cc3cd3ab Pre-generate updates for test harness in parallel.
First CL of many.  Thought I'd commit this before the others since
it sets stuff up for the other CL's (i.e. Thread Pool).

With this CL, all updates will be pre-generated before the test run.
This should improve some speeds since the two updates will be produced
at the same time rather than at the time of the sequential tests.

Change-Id: Iaf0a06f0d99c31d3d749e478fd2fb6efa0763a28

BUG=chromium-os:10723
TEST=Ran it with -q

Review URL: http://codereview.chromium.org/6264005
2011-01-20 10:23:54 -08:00
Don Garrett
e70ccb91b1 Use different proxy ports, and log port conflicts. Hopefully works around
build problems, and will give better diagnostics if not.

Change-Id: I2f7a5d9b2e0c63a8189ad212125c7a5c0178c3c4

BUG=10567
TEST=

Review URL: http://codereview.chromium.org/6331007
2011-01-19 13:41:23 -08:00
Chris Sosa
f867de71d0 Disable tests that don't work with verbose and set verbose as default.
Change-Id: I25066c75797ec7ce481ee096a2dc777d48634581

BUG=Get tree 0pen
TEST=Ran it

Review URL: http://codereview.chromium.org/6279002
2011-01-14 10:38:26 -08:00
Chris Sosa
8617623516 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario.  Re-tested with new changes after build break.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=2b2d7a1

Review URL: http://codereview.chromium.org/6015013
2011-01-10 13:37:49 -08:00
Chris Sosa
efbce1e261 Add simple update test to pfq.
This CL also fixes the outstanding issue where to won't report progress in
image_to_live on updates.

Change-Id: I1efaf17f0fd5ebb367ae0872377b4d5d0bf5dbf6

BUG=chromium-os:8680,chromium-os:8901
TEST=Ran it with --debug on the x86 pfq locally.

Review URL: http://codereview.chromium.org/6135002
2011-01-10 09:56:34 -08:00
J. Richard Barnette
e7205e84be Revert "Major cleanup of cros_au_test_harness code."
This reverts commit 2b2d7a1426.

The internal x86 buildbots have been reporting failures from
cros_au_test_harness with every build, starting with the build
that included the change being reverted.

BUG=chromium-os:8901
TEST=None

Review URL: http://codereview.chromium.org/5962019

Change-Id: Iac8c0a13d659f927b3ff61a9a894c10191432e2d
2011-01-05 18:38:49 -08:00
Chris Sosa
2b2d7a1426 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario

Review URL: http://codereview.chromium.org/6015013
2011-01-04 18:05:59 -08:00
Chris Sosa
4e32890c9b Add new script to stop a kvm using vm library and use in harness.
Change-Id: Ie80843a7b81a37b41ae19fc33244b5c0b8152282

BUG=chromium-os:10434
TEST=Ran it with full au test harness (still running but already showing promise).

Review URL: http://codereview.chromium.org/5988006
2010-12-21 14:56:32 -08:00
Chris Sosa
40fa0ded27 Fixes to get ctest up and running again.
We had a major issue because the latest dev channel release pre-dates the virtio change.  Because of this, ctest was connecting to update that channel using virtio (which is unsupported for that).  This change drops a file telling the vm lib to use e1000 as before.

Also, fixed stateful change logic.

Change-Id: I394a8cece71bdd0d55efa21ba5b6d24804432c6b

BUG=chromium-os:10434
TEST=Ran it with ctest and cros_au_test_harness on images with/without the file.

Review URL: http://codereview.chromium.org/5928005
2010-12-17 13:15:05 -08:00
Don Garrett
8be02f96b6 Create a transparent proxy for test programs to use, with the
ability to filter or manipulate content as it's passed through.

Create two example tests using this utility that cause
updates to close early to test resumes, and to add delays
to make sure the update still completes correctly.

What other tests should be created for this?

BUG=chromium-os:8207
TEST=Run by hand

Review URL: http://codereview.chromium.org/5632002

Change-Id: Iefb8c8e223fb2ba6bad2c551f7c4403a0bec6ecf
2010-12-09 15:54:11 -08:00
Don Garrett
86965c9baf Save off --remote flag
Change-Id: I5621ff8ab81408ac3f7daed970a065641495c4eb

BUG=chromium-os:10179
TEST=none

Review URL: http://codereview.chromium.org/5676002
2010-12-09 15:47:38 -08:00
Chris Sosa
53ac954a36 Add some desired options to the au test harness.
Adds verbose flag to print output while updates are running rather than on error.
Adds test_prefix to specify a prefix of tests you want to run
Flushes stderr more reliably for builders and debugging purposes
Dies when stateful_updates fail

Change-Id: I9e4289deec3688c52b92763eee9334a8868f948e

BUG=chromium-os:6910
TEST=Ran with test_prefix and verbose on and off

Review URL: http://codereview.chromium.org/5573007
2010-12-09 11:29:31 -08:00
Don Garrett
e55c89ab44 Create the latest_download dir if it doesn't exist.
I started to use /tmp, but it doesn't exist both inside and outside of the chroot. Using a consistent name like this has the advantage that the files will be cached and not redownloaded every time. Well, except on the build servers that wipe the related directories between builds.

Change-Id: I58b42d9f1bd18cc4eb44043ed2436d4ffa482767

BUG=chromium-os:10012
TEST=

Review URL: http://codereview.chromium.org/5597004
2010-12-06 15:59:14 -08:00
Don Garrett
969a673045 Add testPartialUpdate, testCorruptedUpdate.
Add two new tests that test update-engine handling of a corrupted image,
or a truncated image.

BUG=chromium-os:9502
TEST=Ran against Dell L13, Mario

Review URL: http://codereview.chromium.org/5373008

Change-Id: Ie1043004f01131ea837798c06327c486e1901c15
2010-12-02 17:54:40 -08:00
Don Garrett
d0473d1e9f Don't pass "" to cros_run_vm_test if vm_graphics_flag isn't set.
This would not have been a problem if "--no_graphics" is used.

Change-Id: I3645391d25d4bb8618109ff5be0ca2fbe8e96a60

BUG=chromium-os:9373
TEST=Ran by hand

Review URL: http://codereview.chromium.org/5104006
2010-11-18 16:11:13 -08:00
Chris Sosa
a2a597d18b Add ability to fall back to fulls for all updates.
Currently works around issue till we get a good fix in for 9.x.

Change-Id: I9c7225508fa8f46a5fa5e0b41abc4e373cc80085

BUG=8606
TEST=Tested by sending SEGV to a delta update during update and saw it successfully fall back to a full update and succeed.

Review URL: http://codereview.chromium.org/4258002
2010-11-02 18:06:34 -07:00
Chris Sosa
113739c109 Add components to pass vm flag to dev server.
BUG=8306
TEST=Ran au_test_harness with CL for adding vm option for the devserver on the last channel release and my TOT.  All tests passed.

Review URL: http://codereview.chromium.org/4165009

Change-Id: Ia2ce305dff2911dc4008d5f6e383535e4bbd4ab0
2010-10-28 23:46:20 -07:00
Chris Sosa
42df54df1c Revert to fulls for boths tests if delta fails.
Change-Id: Iabc931f2eb1751ca9d05e92e64e5361755791648

BUG=
TEST=Copied and pasted from other try/except

Review URL: http://codereview.chromium.org/4109006
2010-10-28 14:39:11 -07:00
Chris Sosa
f8b650ac1a Re-try with full updates if first delta fails.
Change-Id: Ica7360dc4b2a62384874ba1b802039289747c6e3

BUG=
TEST=Ran it

Review URL: http://codereview.chromium.org/4137010
2010-10-28 04:12:23 -07:00
Chris Sosa
9b76b421e4 Use the right base path for vm updates.
Change-Id: I98276b26b85e48ae4b65e8f3fb65540b24665f9a

BUG=Will file
TEST=Ran it

Review URL: http://codereview.chromium.org/4185008
2010-10-28 03:41:14 -07:00
Chris Sosa
54f3b0bbd6 Add delta support to the update test harness.
Change-Id: I94159ac5468dd486123f7cb00da29526b5857d8f

BUG=8192
TEST=Tested with other CL's.

Review URL: http://codereview.chromium.org/4170004
2010-10-26 21:42:47 -07:00
Chris Sosa
f38f249eb4 Update config for cros_au_test_harness.
Change-Id: Iccc5d28f4520418c58089cdb53b6e644271b2eb3

BUG=
TEST=n/a

Review URL: http://codereview.chromium.org/4051003
2010-10-24 02:13:27 -07:00
Chris Sosa
aa4670315b Fix equals in RunCommand's in cbuildbot.
Change-Id: If8cdde42a470c3d07cfd41552cf95dea7567688b

BUG=
TEST=Ran cbuildbot.

Review URL: http://codereview.chromium.org/3967002
2010-10-21 10:54:02 -07:00
Chris Sosa
020aa69f37 Fixes for cbuild to work with ctest.
1)  image_to_vm aborts because e2fschk requires a terminal to
be connected in order to check whether or not you want to repair the fs.  Since
we always want this to be true, set -p

2)  Clean up test harness to only verify the number of tests that passed on the
base image are the same on the update back to.  This is because older images
have many tests that fail.  I leave 10% as a reasonable number to pass.

3)  Redirect output from autotest in run_remote_tests to stderr so it gets
logged in a calling script that captures stdout.

Change-Id: If412274353683add20d136747113eb9c2bd41330

BUG=4690, 5533, 7287
TEST=Ran with internal tools for builders.

Review URL: http://codereview.chromium.org/3536018
2010-10-08 16:36:20 -07:00
Chris Sosa
d8f50a931f Propagate return code correctly from tests.
Change-Id: I28c5f3e48646ec24813ef86f4cd16aac120cba5b

BUG=
TEST=Introduced artificial bug and checked return code
from test.

Review URL: http://codereview.chromium.org/3593010
2010-10-07 11:24:28 -07:00
Chris Sosa
24680f0059 Add ability to test machines with au test harness.
Change-Id: I913acd95c49b1cf0fd666b49fb5ec1e24f0f83d1

BUG=7285
TEST=Ran through test against with two test images.

Review URL: http://codereview.chromium.org/3619009
2010-10-06 16:56:56 -07:00
Chris Sosa
af33f8f8ae Add board option to au test harness so that this works for others without a default board and add ability to reference images without relative paths.
Change-Id: I4afa86fa7a6b772124de56114910a1ca5e3d4511

BUG=
TEST=Ran through update harness code and ran loman.

Review URL: http://codereview.chromium.org/3612004
2010-10-04 18:19:29 -07:00
Chris Sosa
bccc7c1d03 Initial auto update engine test harness.
This harness has support for test on vm machines.  It takes a base image and target image path, creates an update-enabled vm image from the base image and performs updates using cros_run_vm_update (vm wrapper around image_to_live).  It uses --snapshot for updating vm's and keeps them on (like a real machine would) so that the VM image is clean on the next test.

The current check in has two test cases -- updates without clobbering the stateful partition and updates with clobbering the stateful partition.

Change-Id: I5406eadb39fa36601d17f8dd15625bd6a8b38c15

BUG=6911
TEST=Ran through tests.

Review URL: http://codereview.chromium.org/3554002
2010-09-30 15:06:15 -07:00