Commit Graph

96 Commits

Author SHA1 Message Date
Don Garrett
01c8423582 RunCommand was catching and rethrowing an exception, which seemed to generate
extra noise in the final exception output.

This change stops catching and rethrowing the same exception, and generally
makes the RunCommand slightly more readable (if longer).

Some unit tests are added, but they only test RunCommand, and not all options
to RunCommand.

BUG=chromium-os:11717
TEST=Manual, and new lib/cros_build_lib_unittest

Review URL: http://codereview.chromium.org/6576016

Change-Id: Ibe9b13ff302621e93196008f86914cb65a232d61
2011-02-23 17:36:52 -08:00
Zdenek Behan
18d5703fd7 scripts: use /sbin/ifconfig where appropriate
BUG=n0ne
TEST=cros_au_test_harness

Change-Id: Ic26cc191fdb995b091d910f1c8a46e37085fea72

Review URL: http://codereview.chromium.org/6538067
2011-02-19 00:36:23 +01:00
Zdenek Behan
c264a45ba8 au_test_harness: fix a function call, add sudo in front of ifconfig
Note: sudo before ifconfig is for running on distros which do not
have /sbin or /usr/sbin in common user's paths, like gentoo.

Change-Id: I85bd379ad059d6ecaa8c11f3167fae27987479dd

BUG=5246
TEST=run cros_au_test_harness and see it not fail

Review URL: http://codereview.chromium.org/6541008
2011-02-18 06:06:55 +01:00
Chris Sosa
99768e8e12 Robustify starting of ssh connection to vm updater.
Change-Id: Ia5e0042250da22cebadfd48d143f21597bcada5d

BUG=chromium-os:11716
TEST=Ran vm test and vm update scripts.

Review URL: http://codereview.chromium.org/6413019
2011-02-07 09:02:22 -08:00
Greg Spencer
798d75f3be This starts to fix the scripts so that they load from /usr/lib/crosutils
from within the chroot.

It also fixes a number of style issues.

It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.

It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.

BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.

Review URL: http://codereview.chromium.org/6240018

Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
2011-02-01 22:04:49 -08:00
Mandeep Singh Baines
f9ea0036fa cros_workon: collapse cros_workon_common.sh
cros_workon is the only user of cros_workon_common.sh

This is pre-requisite to moving cros_workon into devutils.git

BUG=11507
TEST=Ran ./cros_workon --board x86-generic --all list

Change-Id: I1eab551ab33646360e507328932c151a0d36f50a

Review URL: http://codereview.chromium.org/6347052
2011-01-31 16:13:30 -08:00
Chris Sosa
f2ee5f6871 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=be787f3

Review URL: http://codereview.chromium.org/6277015
2011-01-26 11:38:14 -08:00
Chris Sosa
404bfdf073 Revert "Passes cache location to tests and runs the tests in parallel."
This reverts commit be787f3525.

Revert "Fix sudo again."

This reverts commit 0f411ec7d9.

TBR dgarrett

BUG=
TEST=

Review URL: http://codereview.chromium.org/6348022

Change-Id: I2fb9ca4967ff173c17840b96eb68b02ff2b7774a
2011-01-25 18:16:55 -08:00
Chris Sosa
be787f3525 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f

Review URL: http://codereview.chromium.org/6277015
2011-01-25 17:16:56 -08:00
Chris Sosa
27729c7dbf Revert "Passes cache location to tests and runs the tests in parallel."
This reverts commit c418a8fce7.

TBR=open tree

Change-Id: I471d186fca9084fd18858998b4bcf8cb1de19f14
2011-01-25 15:12:39 -08:00
Chris Sosa
c418a8fce7 Passes cache location to tests and runs the tests in parallel.
As a warning, this is a pretty big change.  At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness.  Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).

This change also has the tests run in parallel.  Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.

Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4

BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).

Review URL: http://codereview.chromium.org/6277015
2011-01-25 13:49:29 -08:00
Don Garrett
3343f56b77 Add a verbose option to RunCommandCaptureOutput. Not used yet, does not
dump output until after the command has finished, so not useful for
following 'live'.

Change-Id: Ia8562f5e6f8d7ed8af7e188faa68f5cdbba487c6

BUG=chromium-os:11164
TEST=None

Review URL: http://codereview.chromium.org/6296013
2011-01-21 16:06:51 -08:00
Don Garrett
83d063b670 Pass along RunCommandException correctly. This is a minor
fix, but it leads to better error messages from ctest.

Change-Id: I624ad805ef037b022e6e67b56f0ed10ca05fe325

BUG=11096
TEST=Test currently in progress.

Review URL: http://codereview.chromium.org/6257007
2011-01-20 11:44:41 -08:00
Don Garrett
e70ccb91b1 Use different proxy ports, and log port conflicts. Hopefully works around
build problems, and will give better diagnostics if not.

Change-Id: I2f7a5d9b2e0c63a8189ad212125c7a5c0178c3c4

BUG=10567
TEST=

Review URL: http://codereview.chromium.org/6331007
2011-01-19 13:41:23 -08:00
Hung-Te Lin
4653cbd0e5 crosutils/make_factory_package.sh: support full SSD image as target outside chroot
When running outside chroot, we should use locate_cgpt + "$GPT" to invoke cgpt;
otherwise it may be not in system PATH.

For dealing with a true SSD image (over USB, for example) this CL also improved
data writing speed from <2M/s to 8M/s+ (over USB).

BUG=chromium-os:10531
TEST=./make_factory_package.sh \
  --factory=../build/images/x86-mario/latest/chromiumos_factory_image.bin \
  --release=../build/images/x86-mario/latest//chromiumos_image.bin \
  --diskimg=/dev/sdb

Also tested:
  --diskimg=blah # local file
  (chroot) --diskimg=/dev/sdd
  (chroot) --diskimg=blah

Change-Id: I647d97fe388f5b6a6223fa8cacfecdf00265f60c

Review URL: http://codereview.chromium.org/6261003
2011-01-13 16:59:42 +08:00
Nick Sanders
607c0780c0 Allow disk image target for copy machine.
BUG=10531
TEST=build and imaged a mario

Change-Id: I60790f178159d707f610ba1668c29179a6c32e61

Review URL: http://codereview.chromium.org/6050006
2010-12-28 22:27:11 -08:00
Chris Sosa
4e32890c9b Add new script to stop a kvm using vm library and use in harness.
Change-Id: Ie80843a7b81a37b41ae19fc33244b5c0b8152282

BUG=chromium-os:10434
TEST=Ran it with full au test harness (still running but already showing promise).

Review URL: http://codereview.chromium.org/5988006
2010-12-21 14:56:32 -08:00
Chris Sosa
40fa0ded27 Fixes to get ctest up and running again.
We had a major issue because the latest dev channel release pre-dates the virtio change.  Because of this, ctest was connecting to update that channel using virtio (which is unsupported for that).  This change drops a file telling the vm lib to use e1000 as before.

Also, fixed stateful change logic.

Change-Id: I394a8cece71bdd0d55efa21ba5b6d24804432c6b

BUG=chromium-os:10434
TEST=Ran it with ctest and cros_au_test_harness on images with/without the file.

Review URL: http://codereview.chromium.org/5928005
2010-12-17 13:15:05 -08:00
Don Garrett
8be02f96b6 Create a transparent proxy for test programs to use, with the
ability to filter or manipulate content as it's passed through.

Create two example tests using this utility that cause
updates to close early to test resumes, and to add delays
to make sure the update still completes correctly.

What other tests should be created for this?

BUG=chromium-os:8207
TEST=Run by hand

Review URL: http://codereview.chromium.org/5632002

Change-Id: Iefb8c8e223fb2ba6bad2c551f7c4403a0bec6ecf
2010-12-09 15:54:11 -08:00
Chris Sosa
4a5d7d9858 Wait for child to exit after killing it.
Use general purpose wait in case your connecting to a vm
that you didn't start.

Change-Id: Ie67e2d3235882d372d04e4a44d0781ddfc8d4645

BUG=chromium-os:10121
TEST=Ran it, caused some crashes.  Still ran test after retries.

Review URL: http://codereview.chromium.org/5757001
2010-12-09 15:46:09 -08:00
Chris Sosa
53ac954a36 Add some desired options to the au test harness.
Adds verbose flag to print output while updates are running rather than on error.
Adds test_prefix to specify a prefix of tests you want to run
Flushes stderr more reliably for builders and debugging purposes
Dies when stateful_updates fail

Change-Id: I9e4289deec3688c52b92763eee9334a8868f948e

BUG=chromium-os:6910
TEST=Ran with test_prefix and verbose on and off

Review URL: http://codereview.chromium.org/5573007
2010-12-09 11:29:31 -08:00
Anush Elangovan
0de6de66db Enable virtio for tests
Change-Id: I1d0e546f6ea837928ba426e4690007ef2649f68e

BUG=10102
TEST=test KVM with -net virtio and ensure it boots and can copy files in etc

Review URL: http://codereview.chromium.org/5543006
2010-12-07 17:31:48 -08:00
Don Garrett
cd33866c89 Add RunCommandCaptureOutput as a helper method.
It's like RunCommand, but you can run the command, capture it's error code and output together.

This is to support tests that need to capture output of image_to_live, and to know if it succeeded.

Change-Id: If674b6d79697c0f0b5c96be9fc83adbed9b9893e

BUG=chromium-os:9502
TEST=Ran by hand

Review URL: http://codereview.chromium.org/5339006
2010-11-29 13:30:42 -08:00
Thieu Le
ac13647eda Change _ArchiveTestResults to upload to Google Storage
BUG=chromium-os:8364
TEST=

Change-Id: Icecc8268fa2c12c8413caa2879785f00e4be5a0a

Review URL: http://codereview.chromium.org/4864001
2010-11-24 10:42:32 -08:00
Hung-Te Lin
062947926e cros_utils: refine factory and imaging related scripts
Major changes are:
 - prefix functions in cros_image_common with 'image' to avoid naming conflicts
 - use awk to replace grep+sed+cut
 - use case to handle board name list
 - factory_setup.sh should return error if applying patch failed
 - refine for shell script coding style guide:
   * replace `` with $()
   * quote variables
   * replace "! -z" by "-n"
   * no space between redirection symbol (>) and target name (eg, >>filename)
   * 1> should be simply >
   * no trailing \ for && and ||

BUG=chrome-os-partner:1583
TEST=manually verified:
 ./image_to_usb.sh --factory # calls mod_image_for_test --factory, and works fine
 ./make_factory_package.sh --factory PATH_TO_IMAGE --release PATH_TO_IMAGE ...
  # factory payloads were created successfully

Change-Id: I6bb10bdfb12cbdb14e9816b3ad72dfe4b7b0472f

Review URL: http://codereview.chromium.org/5168001
2010-11-18 14:47:38 +08:00
David James
d87170bb7b Fix infinite loop when FindRepoDir argument is a relative path.
This bug doesn't affect production but affects the unit tests.

BUG=chromium-os:9201
TEST=Confirmed this fixes an infinite loop in the unit tests, and that
    cbuildbot.py still works.

Change-Id: I1a56b92d8229110c84cebbe877e55fb99f053212

Review URL: http://codereview.chromium.org/5025002
2010-11-15 18:57:19 -08:00
David James
b49c6df754 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=795bd30

Review URL: http://codereview.chromium.org/4442001
2010-11-15 13:06:42 -08:00
Hung-Te Lin
bcc60b625d crosutils: move image_common.sh to lib/cros_image_common.sh
The team is moving utility/library scripts into 'lib' folder.
image_common.sh should follow this policy.

Also refined the parameter check of mk_memento_images.sh

BUG=chromium-os:5208
TEST=./make_factory_package.sh --factory PATH_TO_FACTORY --release PATH_TO_RELEASE;
     # factory bundle created successfully
     ./mk_memento_images.sh PATH_TO_PART2 PATH_TO_PART3
     # update.gz created successfully
     ./mk_memento_images.sh PATH_TO_IMGE 2 3
     # update.gz created successfully

Change-Id: I3afecf05da2832986723f28b595045d0540ea9e9

Review URL: http://codereview.chromium.org/4825004
2010-11-13 05:36:20 +08:00
Ken Mixter
08145697a5 crosutils: Add vnc view of tests running multiple tests for VM tests
Change-Id: I87c41e60797848192cee1f0c165215fcec8f5bd2

BUG=8850
TEST=Ran a few tests including suite_Smoke

Review URL: http://codereview.chromium.org/4647001
2010-11-08 18:10:08 -08:00
Darin Petkov
4b029a27f8 VM: Use -serial none in no-graphics kvm mode...
...so that the console doesn't get messed up due to the serial port being
redirected from stdin.

BUG=8729
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke; checked console

Change-Id: I5a0cedba2b0ac7f453b4574666c645e84b30b379

Review URL: http://codereview.chromium.org/4501001
2010-11-04 14:06:57 -07:00
Chris Sosa
aa4670315b Fix equals in RunCommand's in cbuildbot.
Change-Id: If8cdde42a470c3d07cfd41552cf95dea7567688b

BUG=
TEST=Ran cbuildbot.

Review URL: http://codereview.chromium.org/3967002
2010-10-21 10:54:02 -07:00
Chris Sosa
dfcb89b7d5 Increase statefulfs_size to accomadate larger stateful partitions.
Change-Id: I0713b31b0393fc696d1112ee4cad4c20d86e602d

BUG=
TEST=Config change

Review URL: http://codereview.chromium.org/3860002
2010-10-18 10:26:16 -07:00
Olof Johansson
1688a6d416 Change KVM to use e1000
add -net nic,model=e1000 in start_kvm and in the suggested command as
output by image_to_vm.sh

Signed-off-by: Olof Johansson <olofj@chromium.org>

BUG=none
TEST=Boot an image, run lspci in a terminal, see intel nic

Review URL: http://codereview.chromium.org/3779007

Change-Id: Id7f614adec0ae69b8f4de152832538a13ea4cbad
2010-10-14 19:08:44 -05:00
Chris Sosa
d76a917edb Add a simple wrapper script around vm lib to start a vm.
Change-Id: Ic3238b3063d0e889c424ff6a9792d9c0f6bebca1

BUG=
TEST=Ran it

Review URL: http://codereview.chromium.org/3628002
2010-10-07 23:20:20 -07:00
Chris Sosa
33146728ef Add ability to vm library to retry until we can ssh into it.
Change-Id: I4de388bb1af6cd4568cdc154971927941eb73800

BUG=7503
TEST=Ran test by closing vm's as they start up to see retry work.

Review URL: http://codereview.chromium.org/3598019
2010-10-07 15:58:24 -07:00
Chris Sosa
af33f8f8ae Add board option to au test harness so that this works for others without a default board and add ability to reference images without relative paths.
Change-Id: I4afa86fa7a6b772124de56114910a1ca5e3d4511

BUG=
TEST=Ran through update harness code and ran loman.

Review URL: http://codereview.chromium.org/3612004
2010-10-04 18:19:29 -07:00
Chris Sosa
c2db26c796 Add methods to run vm tests on x86-pre-flight-queue.
Right now we're leaving error_ok=True until the tests are shown to work on builder.

Change-Id: I6b8c690a0da30948389fd4312032c78d87115364

BUG=6906
TEST=Ran through it locally.

Review URL: http://codereview.chromium.org/3591002
2010-09-30 14:25:25 -07:00
Chris Sosa
45bec936f6 Move vm constants into their own file. This will make it easier to share these constants.
Also fixed get latest image logic in image_to_vm and allowed for using the most recent image in cros_run_vm_test to follow other shell convention of using latest.

Change-Id: I60ed4c03d609500da7f6ae34ef57ba2e32f4b0bb

BUG=
TEST=Tested by running image_to_vm with --full and cros_run_vm_test with suite_Smoke

Review URL: http://codereview.chromium.org/3597001
2010-09-29 15:38:53 -07:00
Chris Sosa
8f6ee51c21 Add persist option to vm library.
BUG=
TEST=Tested by persisting, ctrl+c, continuing and using same vm.
Also without --persist

Review URL: http://codereview.chromium.org/3419014

Change-Id: I49e26eb86101e8b8b50e58a9f277d62d5fadc664
2010-09-22 11:51:52 -07:00
Zdenek Behan
e9474900c6 cros_workon: redefine the concept of a "workon" package list to depend on the board
* Modified all workon listing functions to also look for keyword

* Added a fallback to list all workon ebuilds if keyword is not specified, which
is needed for cros_mark_all_as_stable, which does not differentiate between boards.

This, amongst other potential issues, resolves the case when it was possible to
start working on a package not keyworded for the given board, and making
build_packages fail unconditionally.

TEST=below
$ ./cros_workon list --all --board=x86-generic |wc -l
73
$ ./cros_workon list --all --host |wc -l
57
Looking at the lists rather than "|wc -l" looks correct
$ ./cros_mark_all_as_stable
^ Produces satisfactory result

BUG=6700

Change-Id: Ieee92a39febcef5fb95e59cf97b6e63281a7c750

Review URL: http://codereview.chromium.org/3400001
2010-09-15 11:33:54 -07:00
Chris Sosa
e13960240b Fix cleanup so we don't have to clobber as much when failures occur
Change-Id: I120c49a38d2a17a2216296a68378fee7345f616a

TBR=petkov

Review URL: http://codereview.chromium.org/3255004
2010-08-30 22:37:28 -07:00
Chris Sosa
70e2c9e72a Move RunCommand, and Info/Warning/Die into common pylib
TEST=Ran all commands that used RunCommand or Die with
their associated unittests.

Review URL: http://codereview.chromium.org/3266004
2010-08-30 11:05:50 -07:00
Zdenek Behan
2654f24e1b cros_workon_common: fix a last minute rename :/
* Let's face it: I broke it in the last second when i renamed the function but not
the place where it's being called, and now cros_workon is dead.

	modified:   lib/cros_workon_common.sh

Review URL: http://codereview.chromium.org/3052025
2010-07-29 14:23:36 -07:00
Zdenek Behan
e6a925894c cros-workon: extend show_workon_ebuilds to give full paths, provide wrappers
This will allow replacement of equery which in cros-workon

Also other ebuild-related operations like grouping workon ebuilds with the same repo

* Find for all ebuilds takes almost no time when cached and up to ~2 secs when not
* equery takes ages just to load

TEST=cros_workon_ebuilds before and after, and diff:
--- list1       2010-07-27 21:22:23.000000000 -0700
+++ list2       2010-07-27 21:22:32.000000000 -0700
@@ -1,6 +1,5 @@
 app-crypt/tpm-emulator
 app-crypt/trousers
-app-i18n/ibus
 app-i18n/ibus-chewing
 app-i18n/ibus-hangul
 app-i18n/ibus-m17n
(ibus is a mistake, not a workon package, i'll fix that in a separate CL)

	modified:   lib/cros_workon_common.sh

Review URL: http://codereview.chromium.org/2808072
2010-07-29 13:26:53 -07:00
Chris Sosa
a5e394704d Create common script file for workon tools. Planning on using in another CL.
Review URL: http://codereview.chromium.org/3022010
2010-07-19 11:50:11 -07:00
Anush Elangovan
2c7d19693e Move shflags into crosutils.git
Change-Id: I3aa6275362910addfaa25db9a1578a5496019c9e

Review URL: http://codereview.chromium.org/2983005
2010-07-13 15:24:54 -07:00