Commit Graph

1136 Commits

Author SHA1 Message Date
David James
1f79aafb89 Fix broken quoting in cbuildbot.py, allowing for correct revving of ebuilds.
When you pass '--foo="bar baz"' to a script, the argument is literally
passed as "bar baz", with the quotes. So we need to not add unnecessary
quotes in there.

When enter_chroot.sh parses arguments, it actually removes the quotes
for you, and splits up arguments. As a result, it may actually
be necessary to (1) use the above broken syntax when you call scripts
using enter_chroot.sh; but (2) don't use that syntax when calling
scripts without enter_chroot.sh. That's a broken situation, so I've
added a TODO for that. I've also added more warnings to
cros_mark_as_stable.py so that it's easier to debug when it doesn't
work.

BUG=chromium-os:8633
TEST=Ran unit tests. Planning to watch TOT buildbot post-commit to make sure behavior is right.

Change-Id: Ia1a0e60153fec60cb7ed4db2da128ffd9ae81e23

Review URL: http://codereview.chromium.org/4385002
2010-11-03 11:15:11 -07:00
Darin Petkov
a424d94ab3 A basic utility to allow running of tests on VMs in parallel.
Current usage is simple: just specify a list of tests and each one will be run
on a separate VM using the latest image on a default board. Exit code is
non-zero if any of the tests fail. Output from separate tests is interleaved.

More CLs to follow -- specifying a result directory, non-interleaved output,
base ssh port, board, image path, etc.

BUG=8585
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke

Change-Id: Ifa3396768050905ec02e4cad8b8f065a6d129154

Review URL: http://codereview.chromium.org/4337003
2010-11-03 10:09:39 -07:00
Chris Sosa
a2a597d18b Add ability to fall back to fulls for all updates.
Currently works around issue till we get a good fix in for 9.x.

Change-Id: I9c7225508fa8f46a5fa5e0b41abc4e373cc80085

BUG=8606
TEST=Tested by sending SEGV to a delta update during update and saw it successfully fall back to a full update and succeed.

Review URL: http://codereview.chromium.org/4258002
2010-11-02 18:06:34 -07:00
Scott Zawalski
7333e2b4c4 Update prebuilt.py call to use the _proper_ variable for finding the build root.
BUG=8603
TEST=Tested where GCLIENT_ROOT points to on rm and x86

Review URL: http://codereview.chromium.org/4363001
2010-11-02 18:01:14 -07:00
Raja Aluri
c6e23cdb38 1. Not to assert inside chroot, if so desired.
2. Source shflags, if they are in the current directory

Change-Id: Id22e597a6fb71905f2d0ca5c4a1d94ce5a8a931f

Review URL: http://codereview.chromium.org/4177005
2010-11-02 15:55:57 -07:00
Huyen Nguyen
7f55e42020 Make --firmware_updater an optional flag.
Make --firmware_updater an optional flag.

Change-Id: If7ec165a3ca123f1e0e5782f203d63e45a82582e

BUG=7474
TEST=run locally

Review URL: http://codereview.chromium.org/4237002
2010-11-02 15:24:50 -07:00
David James
0addea30fd Print status messages in parallel_emerge about jobs that are longer than 2 mins.
Currently, if a build is slow, you only get debug output after an hour.
This is to allow for uncluttered output. If output is too cluttered, it's hard
to distinguish regular output from errors.

The problem with this approach is that it's often hard to debug why the build
is slow. Now that Chrome builds by default, it takes over an hour to build,
and people see little indication as to why. You can show the output with
build_packages --showoutput, but that is often too verbose and clutters the
output too much.

Here's an example log snippet that is hard to debug:

Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 5m20.5s Load 3.69 3.04 2.66]
Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 5m25.5s Load 3.40 2.99 2.65]
... yada yada yada ...
Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 45m32.9s Load 1.00 1.18 2.95]

From the above output, we see that a package is building for a long time, but
we don't know what package. We should output the package name every two minutes
at least so people know what package is taking so long. That's what this change
implements.

BUG=chromium-os:8575
TEST=Confirmed new status appear for regular build_packages. Confirmed
     build_packages --showoutput is unchanged.

Change-Id: Ie18b23ac7a8a6e2c24b43ec3691606c7da5e43cb

Review URL: http://codereview.chromium.org/4318003
2010-11-02 14:52:23 -07:00
Anton Staaf
abb9144d97 Change "cros_workon list" to show packages for all boards.
Change-Id: Id4880b423eaabbdefc91060a04629a9018295b4d

BUG=None
TEST="cros_workon list" works, so does "cros_workon --board tegra2_seaboard start chromeos-u-boot"

Review URL: http://codereview.chromium.org/4131008
2010-11-02 11:23:00 -07:00
David James
c3874d1106 Add chromeos-chrome to rebuild blacklist for faster builds.
chromeos-chrome has mostly self-contained dependencies, so there usually
isn't need to rebuild it just because a dependency changed. This allows for
us to use the binaries for chromeos-chrome more often.

BUG=chromium-os:8394
TEST=Check that chrome doesn't rebuild with
./parallel_emerge -gp --workon=libcros --board=x86-generic
chromeos-chrome

Change-Id: Ifa14c890917991a8d11f1f0e757f28686d611a72

Review URL: http://codereview.chromium.org/4243003
2010-11-02 09:01:47 -07:00
Scott Zawalski
27dbab06f2 Update prebuilt.py call to reference proper external path
BUG=NA
TEST=Ran on a full buildbot run to verify paths

Review URL: http://codereview.chromium.org/4282002
2010-11-02 00:46:37 -07:00
Scott Zawalski
cb8c73b7fe Adding both for branch preflight
BUG=NA
TEST=NA

Review URL: http://codereview.chromium.org/4280002
2010-11-01 22:43:19 -07:00
Scott Zawalski
92a4d1c10b Add branch mario preflight with both public and private overlays
BUG=NA
TEST=NA

Review URL: http://codereview.chromium.org/4240002
2010-11-01 22:23:25 -07:00
David McMahon
5a1a13f83d Created branch 0.9.110.B. Update CHROMEOS_VERSION_BRANCH=111
Change-Id: I86be6097c566a69e54737639e304a4482f439363
2010-11-01 18:54:55 -07:00
Andrew de los Reyes
0eb6d764ad Rootfs: increase size to 850MiB.
BUG=not sure, but this is CL 2/2 to fix broken L13 build
TEST=tested usb intall/auto update

Change-Id: I14cdd1ec941a0653ac25ec7ef72f69f2ec8f64c9

Review URL: http://codereview.chromium.org/4270002
2010-11-01 18:27:53 -07:00
David James
3d57ef8139 Update cros_mark_as_stable to support working on only private overlays.
BUG=chromium-os:8493
TEST=Ran unit tests. Verified that cros_mark_as_stable.py supports
running on just the private or public overlay.

Change-Id: I5e2e8fc5e63b3c93bb688a5ce87233315c0c2d1c

Review URL: http://codereview.chromium.org/4250001
2010-11-01 16:22:35 -07:00
Will Drewry
53320366b5 mod_image_for_recovery: supprt new recovery model
With a change for chromeos-installer and http://codereview.chromium.org/4120005,
switched to a kernel image based approach.

BUG=chromium-os:7451
TEST= See http://codereview.chromium.org/4120005

Review URL: http://codereview.chromium.org/4107003

Change-Id: I84ac794dae652578682c04fdfdbfaa3f0c5c8b33
2010-11-01 17:34:14 -05:00
Chris Sosa
b4b09d40a7 Fix versioning logic in ctest to get the correct latest version.
Change-Id: I2721bcd2da38b764ebe21dfb8fe7a39dd947075c

BUG=8500
TEST=Ran it against all boards

Review URL: http://codereview.chromium.org/4235001
2010-11-01 14:36:38 -07:00
David McMahon
6203601529 Created branch 0.9.108.B. Update CHROMEOS_VERSION_BRANCH=109
Change-Id: Ib3e34a3cd4a1af6f7faa94ed20bd35dd0c7949d9
2010-10-31 10:52:44 -07:00
Doug Anderson
48849e2dea Added option for including priority/milestone to cros_changelog.
You can access this by passing:
   --tracker-user="user@chromium.org"
   --tracker-passfile="fileContainingPassword"

...to access anonymously, just set --tracker-user=""

...if you don't include the --tracker-user option, we won't try to
fetch priority/milestone.

To use this feature, you need the GData library.  Until we get that
put in hard-host-depends, the script will simply print instructions
for installing GData if it detects that you don't have it.

At the moment, I believe that logging in isn't giving you any extra
access.  Therefore, any bugs that don't allow anonymous access will
not show their priority/milestone.  I am working on figuring out what
the problem is there.

Change-Id: If388c20c43ee2fb0c1ab8f748ffea65e354eeb1e

BUG=chromium-os:8205
TEST=Ran ./cros_changelog 0.9.104.0 --tracker-user="" and verified that some bugs got priority/milestone.

Review URL: http://codereview.chromium.org/4102013
2010-10-29 14:50:15 -07:00
Chris Sosa
142e452d37 Fixing of the symlinks isn't being called because rootfs is being remounted ro.
This is being remounted ro in cros_make_image_bootable so I should be checking
for the stateful mount pt not the rootfs mountpt

Change-Id: I1ee64489516fae10a6246c5d79236c8b5df090ee

BUG=8116
TEST=Ran it and ran bin/cros_start vm and inspected /usr/local

Review URL: http://codereview.chromium.org/4148013
2010-10-29 11:58:15 -07:00
Huyen Nguyen
b66af7db79 Adding script to build and serve factory packages.
Add script to build and serve factory packages.

Change-Id: I90180ebd7cf6c9e9dfe22f77c66166d23ad3c287

BUG=7474
TEST=run locally

Review URL: http://codereview.chromium.org/4185009
2010-10-29 11:38:06 -07:00
Scott Zawalski
2eb6dfb872 Fix flag boolean 2010-10-29 11:11:41 -07:00
Scott Zawalski
3b0c6d8f83 Since archive_build is also called by cbuild we need to hide the
prebuilt call behind a flag.
2010-10-29 11:07:42 -07:00
Scott Zawalski
720364bf56 Update relative path to be 4 directories deep since archive_build's cwd
is abused throughout the script and this is where end up at when prebuilt
is called

BUG=8389
TEST=Ran from the directory archive_build actually runs from.

Review URL: http://codereview.chromium.org/4119017
2010-10-29 10:40:57 -07:00
Chris Sosa
cf51905a08 Add direct flush calls to stderr / stdout.
Change-Id: I272661e276f4968d464706c802e24a0600b9af46

BUG=8393
TEST=N/A

Review URL: http://codereview.chromium.org/4088007
2010-10-29 09:33:44 -07:00
Olof Johansson
99b3f551d8 prebuilt.py: fix variant detection regex
Without this, I get:

Traceback (most recent call last):
  File "./prebuilt.py", line 495, in <module>
    main()
  File "./prebuilt.py", line 491, in main
    git_sync=options.git_sync)
  File "./prebuilt.py", line 408, in UploadPrebuilt
    git_file = os.path.join(build_path, DetermineMakeConfFile(board))
  File "./prebuilt.py", line 378, in DetermineMakeConfFile
    raise UnknownBoardFormat('Unknown format: %s' % target)
__main__.UnknownBoardFormat: Unknown format: tegra2_seaboard

Change-Id: I1d595b7e1c596139acca344f7a145143686dace1

BUG=chromium-os:8388
TEST=./prebuilt.py  -p ~/chromiumos -b tegra2_seaboard  -u user@workstation:/tmp

Review URL: http://codereview.chromium.org/4184007
2010-10-29 09:50:23 -05:00
David McMahon
51914da137 Created branch 0.9.106.B. Update CHROMEOS_VERSION_BRANCH=107
Change-Id: I69b45412f4e47941723f4029aad89fdcd7e7f544
2010-10-29 00:29:44 -07:00
Chris Sosa
113739c109 Add components to pass vm flag to dev server.
BUG=8306
TEST=Ran au_test_harness with CL for adding vm option for the devserver on the last channel release and my TOT.  All tests passed.

Review URL: http://codereview.chromium.org/4165009

Change-Id: Ia2ce305dff2911dc4008d5f6e383535e4bbd4ab0
2010-10-28 23:46:20 -07:00
Scott Zawalski
695adf144a Update archive_build to push prebuilts to Google storage land.
Enabled host prebuilt upload on the x86-generic target.

BUG=NA
TEST=Tested on an x86-generic build to ensure it uploads as expected

Review URL: http://codereview.chromium.org/4128014
2010-10-28 16:56:00 -07:00
Chris Sosa
42df54df1c Revert to fulls for boths tests if delta fails.
Change-Id: Iabc931f2eb1751ca9d05e92e64e5361755791648

BUG=
TEST=Copied and pasted from other try/except

Review URL: http://codereview.chromium.org/4109006
2010-10-28 14:39:11 -07:00
David James
6ceddf089f Update cros_changelog to support sort-by-date option.
Other changes:
  1. Clean up revision handling to be more forgiving if people want to use
     things that don't start with "cros/". Instead, we only enforce that
     people use tags if they want to use the one-argument version of
     cros_changelog.
  2. Tags are quoted in the command-line so that people can, for instance,
     diff 'cros/master{2 days ago}' vs 'cros/master{1 day ago}'.

Change-Id: I28c7d77ae83256d4c28411ef589f3e726cc81d80

BUG=chromium-os:8205
TEST=Ran ( ./cros_changelog 0.9.104.0 cros/master > /tmp/test1.html &&
          ./cros_changelog --sort-by-date 0.9.104.0 cros/master > /tmp/test2.html )
    Both commands print changes between 0.9.104.0 and master, but the second command prints results sorted by date.

Review URL: http://codereview.chromium.org/4130009
2010-10-28 13:59:57 -07:00
Chris Sosa
f8b650ac1a Re-try with full updates if first delta fails.
Change-Id: Ica7360dc4b2a62384874ba1b802039289747c6e3

BUG=
TEST=Ran it

Review URL: http://codereview.chromium.org/4137010
2010-10-28 04:12:23 -07:00
Chris Sosa
9b76b421e4 Use the right base path for vm updates.
Change-Id: I98276b26b85e48ae4b65e8f3fb65540b24665f9a

BUG=Will file
TEST=Ran it

Review URL: http://codereview.chromium.org/4185008
2010-10-28 03:41:14 -07:00
David McMahon
0e84847681 Created branch 0.9.104.B. Update CHROMEOS_VERSION_BRANCH=105
Change-Id: I2b51eb4052cc17f73a282e0a6cb6291d45e17305
2010-10-27 22:43:19 -07:00
David James
eb57914ebc Tool for printing changelog descriptions.
This is similar to gencl but supports more features and is written in Python.

BUG=chromium-os:8205
TEST=Ran ( ./cros_changelog 0.9.102.0 cros/master > /tmp/test1.html &&
           ./cros_changelog 0.9.102.0 > /tmp/test2.html )
     First command prints changes between 0.9.102.0 and master. Second
     command prints changes between 0.9.102.0 and previous release.

Review URL: http://codereview.chromium.org/4175007

Change-Id: I1a81f7d02e4164a4c61aeaad0c01903658571260
2010-10-27 19:42:28 -07:00
Chris Sosa
b38c7e637e Use pregenerate_update flag for devserver.
Change-Id: I4a6fc66db7b336ae07bdfa837dd5e0d806880e52

BUG=8255
TEST=Ran it with devserver with option.

Review URL: http://codereview.chromium.org/4169006
2010-10-27 16:05:43 -07:00
Chris Sosa
604a3f3168 Adding tracking branch broke repo sync on full builders.
Change-Id: Iff1a416a1f7c37870fbcd42ed4aaf6f96f1ea11d

BUG=8245
TEST=Ran with --clobber

Review URL: http://codereview.chromium.org/4128007
2010-10-27 11:51:02 -07:00
David James
0a4d1665fd Strip newlines from each line in make.conf.
This prevents us from having double newlines.

Change-Id: I9e9f9e9164872419bf263b15d2a7d048e98a0098

BUG=chromium-os:4843
TEST=Ran unit tests.

Review URL: http://codereview.chromium.org/4179002
2010-10-27 11:39:11 -07:00
Chris Sosa
54f3b0bbd6 Add delta support to the update test harness.
Change-Id: I94159ac5468dd486123f7cb00da29526b5857d8f

BUG=8192
TEST=Tested with other CL's.

Review URL: http://codereview.chromium.org/4170004
2010-10-26 21:42:47 -07:00
Chris Sosa
8e3e834e4a Add delta update support to imaging tools.
Change-Id: I3ad6d418a1d47f47b78ba4e011c83bc569041475

BUG=6912
TEST=Ran with http://codereview.chromium.org/4129005/show with and without src_image

Review URL: http://codereview.chromium.org/4147002
2010-10-26 21:40:25 -07:00
David McMahon
a0ff51c60d Created branch 0.9.102.B. Update CHROMEOS_VERSION_BRANCH=103
Change-Id: I786a1bbcdc57f383a0619e51e5af734cf56d5681
2010-10-26 16:59:32 -07:00
David James
db37501dbd Use delta_generator from chroot instead of from update_engine directory.
BUG=chromium-os:8185
TEST=Verify delta_generator is in path after emerging hard-host-depends.
     Generate image using delta_generator from chroot.

Change-Id: I03f6260960f472d4ebfc8305945da3cbfc67ff6e

Review URL: http://codereview.chromium.org/4170003
2010-10-26 16:42:04 -07:00
David James
af35e5c5f0 Update srcroot to be an absolute path.
This is required so that paths still work after we change directories.
Currently our branch builder uses this broken recipe :(

BUG=none
TEST=Check that "./cros_mark_as_stable.py -r .. clean" cleans both the private overlay and chromiumos-overlay

Change-Id: Ie4047fb091bf7af36dc28482ea73b085ec12893c

Review URL: http://codereview.chromium.org/4175004
2010-10-26 15:17:48 -07:00
Chris Sosa
a8860e4572 Exclude fake crash reporter directories from report.
Change-Id: I11357f7196b66f3e879e68efa5366af115f3992b

BUG=8192
TEST=Ran it with results from a test with crash reports in it.

Review URL: http://codereview.chromium.org/4156003
2010-10-26 14:36:40 -07:00
Chris Sosa
3b39a54ea3 Print last 200 lines of update_engine.log from failed update.
Change-Id: I0dbc760007038086b94d534d497a7f72b957f365

BUG=
TEST=Ran it on a failed update.

Review URL: http://codereview.chromium.org/4137003
2010-10-26 14:04:46 -07:00
David James
d2b72f7c2f Correctly initialize tracking branch in preflight rinse.
Currently cros/master is used for the tracking branch in the preflight
rinse step. This is wrong if --tracking_branch is specified
differently in the command-line options to cbuildbot.py.

BUG=none
TEST=Ran unit tests

Change-Id: Id4fb989863ae11f57889faf6a8d8b043b81da58e

Review URL: http://codereview.chromium.org/4104005
2010-10-26 12:35:37 -07:00
J. Richard Barnette
e1010d8cb2 Fix two very broken tests for null strings in archive_build.sh
Change-Id: I7c5d251ab1ae020c3ae8a10ca017c34a02eff893

BUG=None
TEST=./archive_build.sh --test_mod
TEST=./archive_build.sh --test_mod --from=../build/images/x86-generic
TEST=./archive_build.sh --board=x86-generic --gsutil_archive=/tmp --gsutil=echo
TEST=./archive_build.sh --board=x86-generic --gsutil_archive=/tmp --gsutil=echo --gsd_gen_index=echo

Review URL: http://codereview.chromium.org/4153001
2010-10-26 11:21:47 -07:00
Nick Sanders
ada8a1fdcc Notes:
This just matches H2C and H2O

Changes are:
-earlyprintk: serial isn't supported, so remove
-console=ttyS0: allow /dev/console to function, after serial console support is removed.

+quiet: sets loglevel=2, quiets early boot
+console=tty2: this will output console onto vt2.
+loglevel=1: this reduces output to vt2 past what quiet specifies.

BUG=chromium-os:8084
TEST="boot factory install shim, regular image. Note /dev/console works and outputs to tty2"

Change-Id: I2d77bf1de5870c7e610859f063d5a587acd56051

Review URL: http://codereview.chromium.org/4167001
2010-10-26 02:56:51 -07:00
David McMahon
eee7fdf432 Created branch 0.9.100.B. Update CHROMEOS_VERSION_BRANCH=101
Change-Id: I7e36f53c2ec7e0452a6fb80995bb4d9705677a27
2010-10-25 19:32:51 -07:00
David James
59b403d950 Add support for prebuilt uploading using rsync.
BUG=not yet
TEST=Try uploading prebuilts to codf30.jail. Also run unit tests.

Change-Id: Ib218e705633febeddbe5c104f67fa580e1fc7de4

Review URL: http://codereview.chromium.org/4102001
2010-10-25 16:41:50 -07:00