Commit Graph

28179 Commits

Author SHA1 Message Date
Raja Aluri
d44bcf9a45 Adding a script to generate au-geneate.zip file
Change-Id: Ifc47ed28dc3efc0e7ebd018f6703b36913ffd39c

BUG=8716
TEST=Ran the script inside the chroot to makesure it is generating the package.

Review URL: http://codereview.chromium.org/4425004
2010-11-11 17:46:53 -08:00
David James
4beb5cb58f Revert "Add more error checking to preflight queue."
This reverts commit b4258c8bd4.

BUG=br0ken build
TBR=rspangler
TEST=

Review URL: http://codereview.chromium.org/4697007

Change-Id: I839ffd08d33fc5943696cfe31f2978fe32ce5fec
2010-11-11 10:51:07 -08:00
David James
f9b84d61a1 Define buildroot properly.
Followup to "Add more error checking to preflight queue."
BUG=Br0ken build
TBR=kliegs@chromium.org
TEST=Ran unit tests

Review URL: http://codereview.chromium.org/4742004

Change-Id: I63e52265ae404f0193cb1f9ab35f3f1ec266d2c1
2010-11-11 10:45:12 -08:00
David James
b4258c8bd4 Add more error checking to preflight queue.
What's new?
  - cros_mark_as_stable now exits with errors if directories are specified
    that don't exist.
  - cbuildbot.py always explicitly specifies overlay directories so
    cros_mark_as_stable can rely on them existing.
  - Package names and paths are now separated with colons instead of
    spaces, so as to allow for us using the same syntax with
    enter_chroot.sh as we use without the same script. (enter_chroot.sh
    mucks with command-lines that contain spaces or quotes.)
  - All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.

BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable.

Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7

Review URL: http://codereview.chromium.org/4442001
2010-11-11 10:37:04 -08:00
Chris Sosa
9a40915f00 Add new stateful upload payload generator to crosutils.
Change-Id: I53a24d6188711a248a1dc7c869aa9258b10970d2

BUG=chromium-os:8885
TEST=Ran by itself with latest image and current directory as my
output dir as well as with new dev server changes.

Review URL: http://codereview.chromium.org/4688006
2010-11-10 17:31:51 -08:00
Ken Mixter
5bedbc9ad7 crosutils: Require cros_show_stacks to run inside chroot
Change-Id: I28e01fb8ddb542eddf3750def7d49cd40822d6d9

BUG=9026
TEST=Try outside chroot

Review URL: http://codereview.chromium.org/4782003
2010-11-10 16:42:43 -08:00
Chris Masone
f8513d84fe [crosutils] Propagate use_emerged flag up from run_remote_tests
BUG=8544
TEST=build chrome with build_tests, cros_workon start autotest, and then run desktopui_BrowserTest with cros_run_parallel_vm_tests

Change-Id: I19e181238fd75450d41f6debb3a8f7d0c17d15dc

Review URL: http://codereview.chromium.org/4669007
2010-11-09 14:23:29 -08:00
Chris Masone
d11ce17167 [crosutils] Mask out /usr/local/autotest-pkgs on install
BUG=8544
TEST=build image with --factory_install, ensure that image does not have data in /usr/local/autotest-pkgs

Change-Id: I5ed17d794392ec654bc122fa15f7ab8a7abbba2d

Review URL: http://codereview.chromium.org/4756001
2010-11-09 14:22:08 -08:00
Chris Masone
7d04c7aacb [crosutils] bump default size of VM disk to allow for bigger stateful partition
BUG=8544
TEST=create an VM image with image_to_vm.sh; the stateful partition should be 2GB in size.

Change-Id: I6c0ee9e2cb513e42c25e4f0d1616835a41f378d7

Review URL: http://codereview.chromium.org/4737001
2010-11-09 08:24:13 -08:00
J. Richard Barnette
4800e04fe8 Try and get the internal buildbot smoke tests to pass
BUG=chromium-os:8910
TEST=None

Review URL: http://codereview.chromium.org/4683007

Change-Id: I3ff1ccdd038255d8c3594f853ced614ccafc2af3
2010-11-08 19:29:26 -08:00
Ken Mixter
08145697a5 crosutils: Add vnc view of tests running multiple tests for VM tests
Change-Id: I87c41e60797848192cee1f0c165215fcec8f5bd2

BUG=8850
TEST=Ran a few tests including suite_Smoke

Review URL: http://codereview.chromium.org/4647001
2010-11-08 18:10:08 -08:00
Doug Anderson
ec5d7f0d2d Code cleanup: comments, docstrings, and gpylint fixes.
Most of this is from this code review:
  http://codereview.chromium.org/4175007/diff/1/2

I also ran the code through gpylint and fixed some of the stuff,
like making quoting consistent.

Change-Id: Icb3896dbd4e975c7ea4687d58efd6372adfcc3c9

BUG=chromium-os:8205
TEST=Ran ./cros_changelog 0.7.48.0 and saw error; also ran ./cros_changelog 0.9.98.3

Review URL: http://codereview.chromium.org/4263001
2010-11-08 14:46:03 -08:00
Ken Mixter
47fd7b5c51 crosutils: Always pass an image to devserver so it can pregenerate
Change-Id: Ib366064df269bd0d4aee30e5b1628938bd8fedf9

BUG=8313
TEST=Running it once without passing an image

Review URL: http://codereview.chromium.org/4208003
2010-11-08 13:35:05 -08:00
Will Drewry
226522a011 build_image: rollback default error behavior until kernel-next merge
kernel-next lacks error mode 3.  Rolling back until merged.

TEST=none: rolling back to old value
BUG=chromium-os:8442

Change-Id: I0c1a89dc3e00fe5f315b3c15c5070209b0fd79d0

Review URL: http://codereview.chromium.org/4512004
2010-11-05 20:20:47 -05:00
J. Richard Barnette
2c002821ac Create bootperf-bin tools for running boot time tests and reporting results
BUG=chromium-os:8655
TEST=run bootperf as described by 'bootperf --help'
TEST=run showbootstats as described by 'showbootstats --help'

Review URL: http://codereview.chromium.org/4148011

Change-Id: Idcfe206f7708fe5290960149952239219d1a96f6
2010-11-05 17:43:04 -07:00
Huyen Nguyen
1330edb593 Fix bug where square bracket is not removed before appending new rule.
Change-Id: I5618b211abd69b4e3774fa6e4009183ef996a8d8

BUG=8815
TEST=run locally

Review URL: http://codereview.chromium.org/4498004
2010-11-05 16:15:46 -07:00
David James
139ad9dca0 Update parallel_emerge to correct use flags, fixing a regression.
BUG=chromium-os:8814
TEST=Verify packages with old use flags are updated, and that binary packages
     with wrong use flags aren't used.

Change-Id: I7064a10796f1541ff9ae36f2ba6ef1d65dbe1aa3

Review URL: http://codereview.chromium.org/4552007
2010-11-05 14:39:02 -07:00
Anush Elangovan
6c4a20ba37 export CHROME_VERSION by default
Change-Id: I788ec487f7c63e71a8a7012c1c7c19c02df103e4

BUG=8786
TEST=eval the script to see it is set right

Review URL: http://codereview.chromium.org/4521003
2010-11-05 11:37:14 -07:00
Ken Mixter
f010d9365f crosutils: Make running tests easier for developers
Add "fast" mode to run_remote_tests which causes the user to no longer need to emerge autotests before running them (even ones that require compilation.)

Change-Id: I60c1e8bfe562a787075b4f65b714e221e51934f7

BUG=8784
TEST=Ran bvts in and out of fast mode, ran UserCrash, SanAngeles, TPM, and backlight tests (which require compilation) in fast mode.

Review URL: http://codereview.chromium.org/4020004
2010-11-05 10:21:08 -07:00
Will Drewry
16106bade8 mod_image_for_recovery: keep the stateful partition around.
Developer images may have data on stateful they want to keep. In that case, just swap the kernels.

BUG=chromium-os:7451
TEST=tested with a --withdev build and stateful was intact and the recovery install worked.

Change-Id: I59d7f2e99892448d1eaf964e6292611accb2035c

Review URL: http://codereview.chromium.org/4418002
2010-11-05 10:10:22 -05:00
David James
6418c50450 Update parallel_emerge to support --force-remote-binary.
This feature is useful if you want to stick to an old binary package of a package you're not working on.

BUG=chromium-os:8769
TEST=./parallel_emerge --board=x86-mario -uDNvg chromeos --force-remote-binary=chromeos-chrome --force-remote-binary=libcros

Change-Id: I7d3011fa64134158ed848f136bc75e09b0af438e

Review URL: http://codereview.chromium.org/4555002
2010-11-04 23:48:52 -07:00
Chris Sosa
170bde0fd4 Fix for bug 8764 that breaks prebuilts for files with more than just keyword pairs.
I've also modified the unit test so it actually tests for this case.

Change-Id: I6688d3be7bf1d8962031fcf1d684ddcaab43ba9f

BUG=8764
TEST=Unittests

Review URL: http://codereview.chromium.org/4566001
2010-11-04 20:49:30 -07:00
Will Drewry
137eab6129 build_image: enable new cros specific error handling in dm-verity
A new error mode for dm-verity exists which can be called as
"cros" or "3".  This change enables that mode in builds by default.

TEST=build_image had the correct argument; functionality was there [in progress]
BUG=chromium-os:8442

Change-Id: I3eb3f4a662c5d36011542af4e50718db50ab63bd

Review URL: http://codereview.chromium.org/4530003
2010-11-04 22:06:41 -05:00
Will Drewry
59752f1667 create_legacy_bootloader_templates.sh: ensure grub escapes quotes
Evidently no one has tested the grub.cfg since verified boot landed. Grub eats
double quotes which breaks verified boot. This changes adds the escapes.  No
other code changes should be needed in build and install code since they
replace DMTABLEA and DMTABLEB and not the quotes.

TEST=nsanders escaped the quotes in his build :); I am doing a new build to ensure the output is escaped now.
BUG=chrome-os-partner:1603

Change-Id: I55e823347c91493caf3b5b2e4577a4e4616df7f0

Review URL: http://codereview.chromium.org/4529002
2010-11-04 21:58:02 -05:00
Chris Sosa
5494da3107 Pass board to devserver for latest image.
Change-Id: I339e1ce3d8f73e15aa554e1111aced163819ec23

BUG=8732
TEST=Ran with other CL and using latest image.

Review URL: http://codereview.chromium.org/4527001
2010-11-04 17:33:46 -07:00
Darin Petkov
4b029a27f8 VM: Use -serial none in no-graphics kvm mode...
...so that the console doesn't get messed up due to the serial port being
redirected from stdin.

BUG=8729
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke; checked console

Change-Id: I5a0cedba2b0ac7f453b4574666c645e84b30b379

Review URL: http://codereview.chromium.org/4501001
2010-11-04 14:06:57 -07:00
Chris Sosa
de285e0238 Add 0.0.1 logic to new cros_mark.
Change-Id: I413d21dd60ce822af65d5c7c22fd2dba67858d43

BUG=6113
TEST=Ran unit tests and also created my own 9999 ebuild and ran with --all and
saw 0.0.1-r1.ebuild get created in stabilizing_branch

Review URL: http://codereview.chromium.org/4449001
2010-11-04 13:03:33 -07:00
Huyen Nguyen
ca3029e0d2 Run devserver with --factory_config. Add --start_devserver flag.
Change-Id: I9ed0a10033ce608c7ef35465debfc874b55d748c

BUG=7474
TEST=run locally

Review URL: http://codereview.chromium.org/4452001
2010-11-04 12:56:02 -07:00
Huyen Nguyen
41a2b700a2 Fix bug where subfolder is not add to path of firmware.
Change-Id: I5521baa3a2b176167f827034166ffcd443a6a793

BUG=8711
TEST=run locally

Review URL: http://codereview.chromium.org/4413004
2010-11-04 12:40:28 -07:00
Scott Zawalski
6c74d5f8f3 Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2010-11-03 23:18:18 -07:00
Darin Petkov
c0644eb90a VM: Implement --order_output option in cros_run_parallel_vm_tests.
If the option is specified, the utility will pipe the test stdout/stderr
to separate files and dump them at the end.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke --order_output

Change-Id: I6c4fcb02456441b4e4fc3f1717cdb5d607d733b9

Review URL: http://codereview.chromium.org/4422001
2010-11-03 14:53:18 -07:00
Will Drewry
cef56b7dea make_developer_script_runner.sh: fix using full-sized kernels
The developer script runner did not pad for the trailing
GPT header and using a full-sized kernel image extracted
from a build image resulted in failure.  This change
accomodates for that and warns when an error occurs.

TEST=used a dd'd over kernel image and it worked
BUG=chromium-os:7451

Change-Id: I8efd7fbb92fe4d5a3c715580f8c21e52c21957c9

Review URL: http://codereview.chromium.org/4418001
(cherry picked from commit 5f928bef4eb551b94fcc45821eee71aa206cef86)

Review URL: http://codereview.chromium.org/4417002
2010-11-03 16:45:54 -05:00
Darin Petkov
40508146b3 VM: Add board, image_path and base_ssh_port options.
BUG=8585
TEST=manual, tried different options

Change-Id: I66be3930cefdf4ea382e1ed8aba54a66f4bfb36e

Review URL: http://codereview.chromium.org/4403001
2010-11-03 12:47:15 -07:00
Darin Petkov
3bab032123 VM: Add --results_dir_root option to cros_run_parallel_vm_tests
This allows the user to specify a custom results directory root. cbuildbot can
be switched to use cros_run_parallel_vm_tests if necessary without any further
changes.

Also create parents directories for user-specified results directory roots in
run_remote_tests.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke \
  --results_dir_root=/tmp/foo

Change-Id: I7314c1d9e74ca139eaa4bd95290866a43a3606ff

Review URL: http://codereview.chromium.org/4297003
2010-11-03 11:20:53 -07:00
David James
1f79aafb89 Fix broken quoting in cbuildbot.py, allowing for correct revving of ebuilds.
When you pass '--foo="bar baz"' to a script, the argument is literally
passed as "bar baz", with the quotes. So we need to not add unnecessary
quotes in there.

When enter_chroot.sh parses arguments, it actually removes the quotes
for you, and splits up arguments. As a result, it may actually
be necessary to (1) use the above broken syntax when you call scripts
using enter_chroot.sh; but (2) don't use that syntax when calling
scripts without enter_chroot.sh. That's a broken situation, so I've
added a TODO for that. I've also added more warnings to
cros_mark_as_stable.py so that it's easier to debug when it doesn't
work.

BUG=chromium-os:8633
TEST=Ran unit tests. Planning to watch TOT buildbot post-commit to make sure behavior is right.

Change-Id: Ia1a0e60153fec60cb7ed4db2da128ffd9ae81e23

Review URL: http://codereview.chromium.org/4385002
2010-11-03 11:15:11 -07:00
Darin Petkov
a424d94ab3 A basic utility to allow running of tests on VMs in parallel.
Current usage is simple: just specify a list of tests and each one will be run
on a separate VM using the latest image on a default board. Exit code is
non-zero if any of the tests fail. Output from separate tests is interleaved.

More CLs to follow -- specifying a result directory, non-interleaved output,
base ssh port, board, image path, etc.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke

Change-Id: Ifa3396768050905ec02e4cad8b8f065a6d129154

Review URL: http://codereview.chromium.org/4337003
2010-11-03 10:09:39 -07:00
Chris Sosa
a2a597d18b Add ability to fall back to fulls for all updates.
Currently works around issue till we get a good fix in for 9.x.

Change-Id: I9c7225508fa8f46a5fa5e0b41abc4e373cc80085

BUG=8606
TEST=Tested by sending SEGV to a delta update during update and saw it successfully fall back to a full update and succeed.

Review URL: http://codereview.chromium.org/4258002
2010-11-02 18:06:34 -07:00
Scott Zawalski
7333e2b4c4 Update prebuilt.py call to use the _proper_ variable for finding the build root.
BUG=8603
TEST=Tested where GCLIENT_ROOT points to on rm and x86

Review URL: http://codereview.chromium.org/4363001
2010-11-02 18:01:14 -07:00
Scott Zawalski
667ad56014 Update prebuilt.py call to use the _proper_ variable for finding the
build root.
2010-11-02 17:50:58 -07:00
Scott Zawalski
b8751abcb1 Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2010-11-02 17:46:12 -07:00
Raja Aluri
c6e23cdb38 1. Not to assert inside chroot, if so desired.
2. Source shflags, if they are in the current directory

Change-Id: Id22e597a6fb71905f2d0ca5c4a1d94ce5a8a931f

Review URL: http://codereview.chromium.org/4177005
2010-11-02 15:55:57 -07:00
Huyen Nguyen
7f55e42020 Make --firmware_updater an optional flag.
Make --firmware_updater an optional flag.

Change-Id: If7ec165a3ca123f1e0e5782f203d63e45a82582e

BUG=7474
TEST=run locally

Review URL: http://codereview.chromium.org/4237002
2010-11-02 15:24:50 -07:00
David James
0addea30fd Print status messages in parallel_emerge about jobs that are longer than 2 mins.
Currently, if a build is slow, you only get debug output after an hour.
This is to allow for uncluttered output. If output is too cluttered, it's hard
to distinguish regular output from errors.

The problem with this approach is that it's often hard to debug why the build
is slow. Now that Chrome builds by default, it takes over an hour to build,
and people see little indication as to why. You can show the output with
build_packages --showoutput, but that is often too verbose and clutters the
output too much.

Here's an example log snippet that is hard to debug:

Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 5m20.5s Load 3.69 3.04 2.66]
Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 5m25.5s Load 3.40 2.99 2.65]
... yada yada yada ...
Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 45m32.9s Load 1.00 1.18 2.95]

From the above output, we see that a package is building for a long time, but
we don't know what package. We should output the package name every two minutes
at least so people know what package is taking so long. That's what this change
implements.

BUG=chromium-os:8575
TEST=Confirmed new status appear for regular build_packages. Confirmed
     build_packages --showoutput is unchanged.

Change-Id: Ie18b23ac7a8a6e2c24b43ec3691606c7da5e43cb

Review URL: http://codereview.chromium.org/4318003
2010-11-02 14:52:23 -07:00
Anton Staaf
abb9144d97 Change "cros_workon list" to show packages for all boards.
Change-Id: Id4880b423eaabbdefc91060a04629a9018295b4d

BUG=None
TEST="cros_workon list" works, so does "cros_workon --board tegra2_seaboard start chromeos-u-boot"

Review URL: http://codereview.chromium.org/4131008
2010-11-02 11:23:00 -07:00
David James
c3874d1106 Add chromeos-chrome to rebuild blacklist for faster builds.
chromeos-chrome has mostly self-contained dependencies, so there usually
isn't need to rebuild it just because a dependency changed. This allows for
us to use the binaries for chromeos-chrome more often.

BUG=chromium-os:8394
TEST=Check that chrome doesn't rebuild with
./parallel_emerge -gp --workon=libcros --board=x86-generic
chromeos-chrome

Change-Id: Ifa14c890917991a8d11f1f0e757f28686d611a72

Review URL: http://codereview.chromium.org/4243003
2010-11-02 09:01:47 -07:00
Scott Zawalski
27dbab06f2 Update prebuilt.py call to reference proper external path
BUG=NA
TEST=Ran on a full buildbot run to verify paths

Review URL: http://codereview.chromium.org/4282002
2010-11-02 00:46:37 -07:00
Scott Zawalski
5273a3d9d7 Update prebuilt.py call to reference proper external path 2010-11-02 00:00:29 -07:00
Scott Zawalski
b699df8fe4 Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2010-11-01 22:43:27 -07:00
Scott Zawalski
cb8c73b7fe Adding both for branch preflight
BUG=NA
TEST=NA

Review URL: http://codereview.chromium.org/4280002
2010-11-01 22:43:19 -07:00
Scott Zawalski
65fbd618f6 Add comma 2010-11-01 22:42:54 -07:00