Commit Graph

58 Commits

Author SHA1 Message Date
Chris Sosa
fd2cdec118 Remove testing utilities and fix symlinks.
For run_remote_tests this required "fixing" restart_in_chroot if required.  Right now that script ->only<- works if its run on a scripts being called from src/scripts.  This change makes it more generic by doing this for any script called from within our source tree.

I borrow reinterpret_path_for_chroot from image_to_live.  Other CL to remove it from there.

Change-Id: If717beccd777ac178366d58b91521b9a62d55d85

BUG=chromium-os:11172
TEST=Ran them

Review URL: http://codereview.chromium.org/6730012
2011-03-24 16:06:59 -07:00
Mandeep Singh Baines
7b86ec098e run_remote_tests: fix bug where new symlink got created instead of update
The one symlink should be update instead of new links being created.

BUG=12233
TEST=Ran run_remote_test.sh many time and verified that the link is updated.

Change-Id: I63f325e03439e5c922e9fe2300444193146182f9

Review URL: http://codereview.chromium.org/6576051
2011-02-25 10:18:41 -08:00
Benson Leung
393c17b0cc Allow run_remote_tests to learn board instead of default board
If no board is specified for run remote tests,
learn board from device under test instead of using
default board.

BUG=none
TEST= ./run_remote_tests.sh without --board. Verify detects board and
deploys appropriate version for that board.
./run_remote_test.sh with --board x86-mario. Verify that it deploys
that specific board, regardless of target.

Change-Id: I6e71f0f1bf4b2bcfcd3430b29adc16531c6a8c81
Signed-off-by: Benson Leung <bleung@chromium.org>
Signed-off-by: Chris Sosa <sosa@chromium.org>

Review URL: http://codereview.chromium.org/6580017
2011-02-24 01:46:14 -08:00
Mandeep Singh Baines
0cb1d1def8 run_remote_tests: create a symlink to the last test that completed
Having this symlink is extremely useful if your running the same
test over and over again.

BUG=n0ne
TEST=Verified that the symlink is there.

Change-Id: I625a48ce4348082db05951ffd67929687ad461cc

Review URL: http://codereview.chromium.org/6543021
2011-02-18 14:04:18 -08:00
Don Garrett
2d08a983b2 I'm still breaking the build. Reverting again until I figure out why.
TBR

BUG=None
TEST=None

Review URL: http://codereview.chromium.org/6507034

Change-Id: I21283ed2b1f81086ccc95b0436e66c53584f4583
2011-02-16 18:39:35 -08:00
Don Garrett
e47eee3bc8 The original --quiet change failed (and was reverted) because cros_au_test_harness.py wasn't updated when the argument was changed from --quiet to --hide_info.
So... revert the reversion, and fix cros_au_test_harness.

BUG=chromium-os:11680
TEST=Manual

Review URL: http://codereview.chromium.org/6537005

Change-Id: I1bdf66a5735d4178ee1e2eaa838e7b4b7fb966d5
2011-02-16 15:49:14 -08:00
Don Garrett
8dcd981bbf Revert "Add --quiet to run_remote_tests, and use it"
This reverts commit 5b7e1d5da4.

Reverting to fix the build. TBR

BUG=None
TEST=None

Review URL: http://codereview.chromium.org/6480108

Change-Id: Idc9e60f97dd545225538420402c71b611233ce1e
2011-02-15 21:01:40 -08:00
Don Garrett
5b7e1d5da4 Add --quiet to run_remote_tests, and use it
Change-Id: I91f490ba4c4a834ba01b671c56c63ba230561ada

BUG=chromium-os:11680
TEST=Manual

Review URL: http://codereview.chromium.org/6529010
2011-02-15 17:40:56 -08:00
Mandeep Singh Baines
fa42b03ffe autotest: use installed version of cros_workon
BUG=11507
TEST=Ran run_remote_test.sh and verified that the following is printed:

INFO   : Detected cros_workon autotests. Building and running your autotests from /home/msb/trunk/src/third_party/autotest/files. To use emerged autotest, pass --use_emerged.

Change-Id: Ifc4305088e4edd059fad250d074347bc81a451cf

Review URL: http://codereview.chromium.org/6246141
2011-02-07 11:08:03 -08:00
Greg Spencer
798d75f3be This starts to fix the scripts so that they load from /usr/lib/crosutils
from within the chroot.

It also fixes a number of style issues.

It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.

It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.

BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.

Review URL: http://codereview.chromium.org/6240018

Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
2011-02-01 22:04:49 -08:00
Mandeep Singh Baines
9af6ba1b77 run_remote_test.sh: explicitly start/stop ssh-agent
run_remote_test.sh is implicitly depending on remote_access.sh to setup
the ssh-agent for it. This broke when I tried to remove ssh-agent starting
from remote-access.sh. Implicitly dependencies are ugly.

BUG=n0ne
TEST=Ran a test on my local machine. Tested with and without default board.

Change-Id: I50db1717cafc2565cb8bb648919430d9ce24ce80

Review URL: http://codereview.chromium.org/6333001
2011-01-13 15:45:42 -08:00
Daniel Kurtz
b38b3c81bc run_remote_tests.sh: pass multiple arguments to autoserv
Change-Id: I7f54a0e078d0c7eb70e0a614bbb991cd372dec60

BUG=chromium-os:9841
TEST=
## Test with these 2 scripts
./bin/cros_run_wifi_tests.sh --cell 3 MatFunc --args "test_pat=000*"
## --and--
./run_remote_tests.sh --remote=${DUT} --args="test_pat=000* config_file=wifi_testbed_chromelab server_addr=${SERVER} router_addr=${ROUTER}" MatFunc

Review URL: http://codereview.chromium.org/5821001
2010-12-15 10:31:11 -08:00
Eric Li
b2b446631b Remove extra double quotes while passing args into autoserv.
Change-Id: Iaee4c276b8f03546cc32b30849e424c84c729aef

BUG=9841
TEST=run with sam's test script.

cd chromeos/src/scripts
echo chromelab > .default_wifi_test_lab
sudo echo 172.22.18.5   chromelab >>/etc/hosts
bin/cros_run_wifi_tests.sh --cell=4 MatFunc

Review URL: http://codereview.chromium.org/5523005
2010-12-07 17:35:47 -08:00
Eric Li
6fbd3d123c Refactor run_remote_tests script.
What I did:
1. enter_chroot once and only once if necessary. The previous version would enter/exit chroot 3 times in worst case. And the entire logic happens inside chroot only, this has greatly simplified the overall workflow, and reduced the number of variables used. Shell script variables could be tricky.

2. Change variable type to test_type since type is a bash builtin.

3. get rid of {$TMP}/run_test.sh script, since it is not necessary any more.

4. get rid of rsync step from third_party/autotest/files to ${BUILD_DIR}, since it is not neccessary either.

overall, reduced ~40 lines of code.

All the change should be transparent to end users and there should be no regression changes at all.

w/wo emerge autotest
w/wo cros_workon
in/outside of chroot.

and all its combinations.

Change-Id: I9c1532e9cb6cc0e724d4b6d870723df3e2a147ec

BUG=9291
TEST=Run storageFio test since it need prebuild test and deps.

Review URL: http://codereview.chromium.org/5176009
2010-11-29 14:13:15 -08:00
Eric Li
6f209d1333 Remove the dependency to autotest_run.sh so I can get rid of it in next CL.
Change-Id: Ieaa2553612aefd6161785185350d495cbf0dad81

BUG=9291
TEST=Run storageFio test since it need prebuild test and deps.

w/wo emerge autotest
w/wo cros_workon
in/outside of chroot.
w/wo --used_emerged
w/wo --results_dir_root

and all its combinations.

Review URL: http://codereview.chromium.org/5212003
2010-11-19 10:43:47 -08:00
Chris Sosa
2798f3c3cc Revert "Remove the dependency to autotest_run.sh so I can get rid of it in next CL."
This reverts commit 983fdb5a2d.  TBR to ericli

BUG=
TEST=

Review URL: http://codereview.chromium.org/5090006

Change-Id: Ibf9f94f814abacb4cdaaeb815f944bd943a0dd9a
2010-11-18 11:10:08 -08:00
Eric Li
983fdb5a2d Remove the dependency to autotest_run.sh so I can get rid of it in next CL.
Change-Id: I1032932b696929fccf97f911c555ec5a24f589d4

BUG=9291
TEST=Run storageFio test since it need prebuild test and deps.

w/wo emerge autotest
w/wo cros_workon
in/outside of chroot.

and all its combinations.

Review URL: http://codereview.chromium.org/5188002
2010-11-18 10:28:10 -08:00
Ken Mixter
f010d9365f crosutils: Make running tests easier for developers
Add "fast" mode to run_remote_tests which causes the user to no longer need to emerge autotests before running them (even ones that require compilation.)

Change-Id: I60c1e8bfe562a787075b4f65b714e221e51934f7

BUG=8784
TEST=Ran bvts in and out of fast mode, ran UserCrash, SanAngeles, TPM, and backlight tests (which require compilation) in fast mode.

Review URL: http://codereview.chromium.org/4020004
2010-11-05 10:21:08 -07:00
Darin Petkov
3bab032123 VM: Add --results_dir_root option to cros_run_parallel_vm_tests
This allows the user to specify a custom results directory root. cbuildbot can
be switched to use cros_run_parallel_vm_tests if necessary without any further
changes.

Also create parents directories for user-specified results directory roots in
run_remote_tests.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke \
  --results_dir_root=/tmp/foo

Change-Id: I7314c1d9e74ca139eaa4bd95290866a43a3606ff

Review URL: http://codereview.chromium.org/4297003
2010-11-03 11:20:53 -07:00
Ken Mixter
b8a72348e4 crosutil: No longer use autotest-0.0.1 in run_remote_tests even outside chroot
Change-Id: If83bf22fde6259b64fdd007066b5c46c7925d120

BUG=
TEST=Run a few tests inside and outside chroot

Review URL: http://codereview.chromium.org/3790005
2010-10-18 13:34:29 -07:00
Dale Curtis
15e49571bf Fixed incorrect option comment in run_remote_tests.
Autotest will not split comma separated arguments. The correct calling method for multiple arguments is to separate them by spaces and quote the whole thing. E.g., instead of:

./run_remote_tests --args arg1,arg2

It should be:

./run_remote_tests --args "arg1 arg2"

See line 140 in src/third_party/autotest/files/server/autoserv_parser.py for
confirmation.

Change-Id: I88d65c0bd144942a856c2f95371d5a55b0a3a172

BUG=
TEST=No code changes. Ran --help and --args to verify nothing dumb happened and the correctness of comment.

Review URL: http://codereview.chromium.org/3772002
2010-10-13 12:38:04 -07:00
Chris Sosa
020aa69f37 Fixes for cbuild to work with ctest.
1)  image_to_vm aborts because e2fschk requires a terminal to
be connected in order to check whether or not you want to repair the fs.  Since
we always want this to be true, set -p

2)  Clean up test harness to only verify the number of tests that passed on the
base image are the same on the update back to.  This is because older images
have many tests that fail.  I leave 10% as a reasonable number to pass.

3)  Redirect output from autotest in run_remote_tests to stderr so it gets
logged in a calling script that captures stdout.

Change-Id: If412274353683add20d136747113eb9c2bd41330

BUG=4690, 5533, 7287
TEST=Ran with internal tools for builders.

Review URL: http://codereview.chromium.org/3536018
2010-10-08 16:36:20 -07:00
Eric Li
47518898af Support run multiple control file under the same test directory.
Change-Id: I1ad0aaec6517758812bf4ba0460ab0c81a6f84cb

BUG=7357
TEST=run_remote_tests.sh ... bvt regression and then observer both bvt and regression test result

Review URL: http://codereview.chromium.org/3603012
2010-10-06 10:12:27 -07:00
Ken Mixter
aa5504fdec crosutils: Make run_remote_tests' output for ambiguous case more clear.
BUG=5817
TEST=run_remote_tests --remote=$IP control shows ^...$ on all lines.

Change-Id: Ie0c4ac16493dc97044ec0bd0fd6d6e53a69c9041

Review URL: http://codereview.chromium.org/3473015
2010-09-23 17:12:06 -07:00
Zdenek Behan
1b82990585 run_remote_tests: destroy the with_spaces hack, use the --args instead of -a (which doesn't work)
Change-Id: I8560a4400463d04b7a2441ce6fa9dc9cec834242

BUG=
TEST=run_remote_tests.sh --args='-n 10', and make sure it gets all the way into autotest_run.sh

Review URL: http://codereview.chromium.org/3402007
2010-09-16 14:31:34 -07:00
Zdenek Behan
18d3476a41 run_remote_tests: add quote marks around argument passing
* stops the second-pass parsing of arguments into IFS-delimited elements
which unbreaks things like --args='-n 10'

Change-Id: If8424b5fc06e6dd5a935421a76539e7e91006c37

BUG=
TEST=run it and see the argument being passed correctly

Review URL: http://codereview.chromium.org/3422005
2010-09-15 16:48:59 -07:00
Ken Mixter
ef45f16aa2 Allow prepackaged to still be passed and make learn_board work again
Change-Id: I664858ff9b931479fe2fac89a24aa3e7fce8493b

BUG=
TEST=ran rrt without ==board or default board set

Review URL: http://codereview.chromium.org/3431004
2010-09-14 17:19:33 -07:00
Zdenek Behan
82e95c43e7 run_remote_tests: for cros_workon, always use prepackaged tests
* Fixes running tests different than from the main autotest repo

* Also provides a better fix for chrome tests

Change-Id: I803ba7caa5561e3001d5627219718f471e59bb19

BUG=
TEST=

Review URL: http://codereview.chromium.org/3448002
2010-09-14 15:11:39 -07:00
Zelidrag Hornung
90a850ac11 Fixed run_remote_test.sh to work properly with the new location of chrome tests
BUG=none
TEST=none

Review URL: http://codereview.chromium.org/3341028

Change-Id: I81a8d57b16e7789b0fd6673d9e3b208ce369a575
2010-09-10 14:25:59 -07:00
Ken Mixter
cc4f1dd14e Script to generate post mortem of all crashes on device
BUG=4887

TEST=tested run_remote_tests and image_to_live still work.  Ran script with machine with no crashes, as well as on a machine with a slew of powerd CHECK(false)'s.

Change-Id: Iffb6571d30d99d876f41972f92a7149a716035ee

Review URL: http://codereview.chromium.org/3276002
2010-08-31 12:07:11 -07:00
Ryan Cairns
1703423a77 Runs our BVT suite in a VM
Change-Id: I758c93596d5cbdd6b52b9acc82f4d6e19a326c9f

BUG=5518
TEST=Tested using all the options. All tests listed in the file pass.

Review URL: http://codereview.chromium.org/3107039
2010-08-30 17:03:50 -07:00
Ken Mixter
6cb2462132 Use generate_test_report to generate the run_remote_test report.
BUG=5359

Review URL: http://codereview.chromium.org/3164008
2010-08-11 17:23:40 -07:00
Zdenek Behan
8817081418 autotest_run: kill the destructive stuff that's pwning the autotest installation
* Also changes the way how "new workflow" is detected in run_remote_tests.sh

	modified:   autotest_run.sh
	modified:   run_remote_tests.sh

Review URL: http://codereview.chromium.org/3048051
2010-08-06 11:26:23 -07:00
Zdenek Behan
ddb0f2c8e3 autotest: create a candidate script for replacement ebuild test (running the tests)
* This script should replace the call to ebuild in autotest wrapper, and essentially
duplicates all the test running functions from autotest-0.0.1.ebuild

* duplicate autotest wrapper into autotest_workon to separate conversion and old functionality

* Add a hack into run_remote_tests to allow using autotest_workon instead

	new file:   autotest_run.sh
        new file:   autotest_workon
        modified:   run_remote_tests.sh

Review URL: http://codereview.chromium.org/2854062
2010-07-26 18:02:29 -07:00
Zelidrag Hornung
61d9768584 added support for an alternative ssh port for autotest (needed for running autotest on Vibrant VM)
Review URL: http://codereview.chromium.org/2845002
2010-06-15 11:55:21 -07:00
Daniel Erat
0db85ef360 Make run_remote_tests.sh support --results_dir_root.
Patch author: Andy Chan <andyc@codeaurora.org>
Original issue: http://codereview.chromium.org/2128016/show
Reviewed by sosa@chromium.org.

BUG=none
TEST=none

Review URL: http://codereview.chromium.org/2728006
2010-06-09 15:55:03 -07:00
Anton (whywhat) Vayvod
9ffc3b32a3 Added --test_args parameter that passes its value as -a flag to autoserv.
Review URL: http://codereview.chromium.org/1794004
2010-04-28 21:51:16 +04:00
Eric Li
36e192758f Pass chroot flag into enter_chroot.
This is something missing from the original run_remote_tests script.

Review URL: http://codereview.chromium.org/1589022
2010-04-10 11:58:43 -07:00
Eric Li
7a4c081107 remove build_autotest.sh and change run_remote_tests.sh to use new autotest script.
This change should go with

http://codereview.chromium.org/1545014

together.

Review URL: http://codereview.chromium.org/1603004
2010-04-07 12:51:05 -07:00
Chris Sosa
aa1a7fd63d Adds gmergefs. A method of remoting to a target and pushing new
packages using emerge from the host

Review URL: http://codereview.chromium.org/1515011
2010-04-02 14:06:29 -07:00
Ken Mixter
d99d5e3181 Add -b option to run_remote_tests to build tests before running
Review URL: http://codereview.chromium.org/1548002
2010-03-29 18:59:48 -07:00
Ken Mixter
cad6ceb186 only consider control files of control or control.* form
Review URL: http://codereview.chromium.org/1345003
2010-03-27 16:34:11 -07:00
Darin Petkov
9cff73ea8c Look for ERROR too. Not sure if BAD actually ever shows up.
Review URL: http://codereview.chromium.org/1216001
2010-03-23 15:04:34 -07:00
Ken Mixter
3c6a73ba69 Enable run_remote_tests to run in chroot and clean up output
Make this script's output better formatted.  Better describe how
to disambiguate control files when multiple match.  Get rid of
the output_file concept since it's not needed.  Don't output
a results directory if there's an ambiguity.

Review URL: http://codereview.chromium.org/1090005
2010-03-22 15:45:52 -07:00
Ken Mixter
8142fd6b35 Add restrictions to files considered control files
Review URL: http://codereview.chromium.org/1155003
2010-03-19 18:05:46 -07:00
Ken Mixter
8400a40350 Minor bug fixes for run_remote_tests
By default keep around result directories.
Also find control* named control files.

Review URL: http://codereview.chromium.org/1097003
2010-03-18 16:57:07 -07:00
Ken Mixter
a2c6844e2c learn board and print better status messages
Review URL: http://codereview.chromium.org/668036
2010-03-03 18:57:45 -08:00
Ken Mixter
eb31903982 build_autotest runs as part of build_packages given a flag.
sync_build_test runs build_autotest that way.
archive_build bzips up the autotest artifacts.
run_remote_tests can run autotest artifacts from a prepackaged directory.  Note that because build_platform is in a different repository than these other files, it's unfortunately forced to be reviewed separately in http://codereview.chromium.org/661197 and committed independently.

Review URL: http://codereview.chromium.org/660189
2010-02-26 17:33:51 -08:00
Eric Li
cbbb239570 Change run_remote_tests from portage build env.
Review URL: http://codereview.chromium.org/606054
2010-02-16 08:49:16 -08:00
Chris Sosa
431a7904ef Repatch
Review URL: http://codereview.chromium.org/596051
2010-02-10 17:19:24 -08:00