Commit Graph

2074 Commits

Author SHA1 Message Date
Scott Zawalski
e90029493e Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2011-01-18 17:08:54 -08:00
Scott Zawalski
11bc4c7213 Merge branch 'prebuilt' 2011-01-18 17:08:32 -08:00
Scott Zawalski
61a543d901 Start using prebuilt.conf instead of make.conf this is an initial step
to eventually moving entirely out of the overlay directory. The reason
to do this now is to avoid devs having to remerge there make.conf changes
while they are in flight.

Add private support to prebuilt.py so we will update private overlays if
we are internal and external if we are external.
Update prebuilt_unittest to ensure we are returning the proper path if
the private overlay exists. Also ensure we are returning the proper host
path regardless.

BUG=9919
TEST=unittest added for private case
2011-01-18 17:06:56 -08:00
J. Richard Barnette
3d0a18e676 Fix showbootdata handling when invoked with options but no arguments
Change-Id: Ieacacae968f45325b7a164d1a38b8cd27272d367

BUG=chromium-os:10519
TEST=see bug description

Review URL: http://codereview.chromium.org/6085003
2011-01-18 16:57:52 -08:00
Todd Broch
93300cd754 Revert "build_kernel_image: enable nmi watchdog"
This reverts commit ea621903e9.

TEST=grep dmesg for clock & nmi to make sure things look ok.
Run example program to make sure result of clock_getres call tv_nsec = 1 again
BUG=chromium-os:10720

Change-Id: Ife23fc7d08420788d4c165957d739ec2ce974969

Review URL: http://codereview.chromium.org/6274006
2011-01-18 10:05:21 -08:00
Scott Zawalski
cc4a3f5b28 Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2011-01-14 14:42:23 -08:00
Mandeep Singh Baines
2f3b5fc537 update_kernel.sh: modify to work inside chroot
BUG=n0ne
TEST=Successfully updated a kernel.

Change-Id: If168c47cca4b5b794d95c9b839b17bff3f96ed9c

Review URL: http://codereview.chromium.org/6249004
2011-01-14 14:20:12 -08:00
Mandeep Singh Baines
aef91ad564 remote_access.sh: remove dependence on ssh-agent
This is take 2 of this CL. Modified run_remote_test.sh to
explicitly start an ssh-agent if necessary instead of
implicitly relying on remote_access.sh to do it.

BUG=n0ne
TEST=Verified I could successfully ssh.

Change-Id: I4e70b8574af1119dcf36768f6602ecc0cabedc31

Review URL: http://codereview.chromium.org/6288004
2011-01-14 14:17:25 -08:00
Chris Sosa
f867de71d0 Disable tests that don't work with verbose and set verbose as default.
Change-Id: I25066c75797ec7ce481ee096a2dc777d48634581

BUG=Get tree 0pen
TEST=Ran it

Review URL: http://codereview.chromium.org/6279002
2011-01-14 10:38:26 -08:00
Mandeep Singh Baines
9af6ba1b77 run_remote_test.sh: explicitly start/stop ssh-agent
run_remote_test.sh is implicitly depending on remote_access.sh to setup
the ssh-agent for it. This broke when I tried to remove ssh-agent starting
from remote-access.sh. Implicitly dependencies are ugly.

BUG=n0ne
TEST=Ran a test on my local machine. Tested with and without default board.

Change-Id: I50db1717cafc2565cb8bb648919430d9ce24ce80

Review URL: http://codereview.chromium.org/6333001
2011-01-13 15:45:42 -08:00
Scott Zawalski
d1f0db8c7d Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2011-01-13 15:39:37 -08:00
Chris Sosa
4db7ac8f98 Print out SVN revision number for TOT chrome pfq builds.
Change-Id: I046d49c7002586671dc7c6707afeeeb0a0360cd5

BUG=10934
TEST=Ran it with bin/cros_mark_chrome_as_stable tot. Verified output and that it still did the right thing

Review URL: http://codereview.chromium.org/6327002
2011-01-13 14:11:18 -08:00
Chris Sosa
5c9571a926 Revert "remote_access.sh: remove dependence on ssh-agent"
This reverts commit 78476aba5f.

BUG=
TEST=Reverted on bot and re-ran test suite

Review URL: http://codereview.chromium.org/6311002

Change-Id: Ib9405dac19b8b19d89e5d2346f5e1fd130a7280f
2011-01-13 13:12:55 -08:00
Chris Sosa
bb4661dee7 Put wait around remote_reboot core code itself rather than around the ping.
The last CL had a race condition where we could ping but not run the reboot_check code.
This CL moves to watch the reboot_check code rather than a ping.

Change-Id: I8bdda97850ac085be9cf5dfb0b80fb7e1c8dd212

BUG=chromium-os:10867
TEST=Ran it 100 times without error.

Review URL: http://codereview.chromium.org/6287001
2011-01-13 11:19:02 -08:00
Mandeep Singh Baines
78476aba5f remote_access.sh: remove dependence on ssh-agent
Change-Id: I68d02f5630bd885894ca69728254f2a5ba5cc2aa

BUG=n0ne
TEST=Verified I could successfully ssh.

Review URL: http://codereview.chromium.org/6314001
2011-01-13 11:16:51 -08:00
Hung-Te Lin
4653cbd0e5 crosutils/make_factory_package.sh: support full SSD image as target outside chroot
When running outside chroot, we should use locate_cgpt + "$GPT" to invoke cgpt;
otherwise it may be not in system PATH.

For dealing with a true SSD image (over USB, for example) this CL also improved
data writing speed from <2M/s to 8M/s+ (over USB).

BUG=chromium-os:10531
TEST=./make_factory_package.sh \
  --factory=../build/images/x86-mario/latest/chromiumos_factory_image.bin \
  --release=../build/images/x86-mario/latest//chromiumos_image.bin \
  --diskimg=/dev/sdb

Also tested:
  --diskimg=blah # local file
  (chroot) --diskimg=/dev/sdd
  (chroot) --diskimg=blah

Change-Id: I647d97fe388f5b6a6223fa8cacfecdf00265f60c

Review URL: http://codereview.chromium.org/6261003
2011-01-13 16:59:42 +08:00
Hung-Te Lin
c584cf0e95 crosutils/make_factory_package.sh: Support relative path
Since there's a pushd call when building images, we have to first retrieve
the complete (full/absolute) path of target disk image.

BUG=chromium-os:10706
TEST=make_factory_package.sh \
     --factory ../build/images/x86-mario/latest/factory_image.bin \
     --release ../build/images/x86-mario/latest/chromiumos_image.bin \
     --diskimg blah_relative.bin

Change-Id: I02e2edb0f76b724337a82dd400297877681bc338

Review URL: http://codereview.chromium.org/6244003
2011-01-13 15:34:59 +08:00
Chris Sosa
60587527c1 Add more robust ssh ping to be run on reboot for updates.
Upon investigation it seems there's a race condition when ssh
starts up the first time.  This adds a much more robust way to
ping a machine when it's up then the previous method.

Change-Id: I092744b259c169975d1c4f283f01a556fce24723

BUG=chromium-os:10867
TEST=Ran it 30 times with image_to_live --noupdate --nostateful_update.

Review URL: http://codereview.chromium.org/6115009
2011-01-12 15:10:35 -08:00
Antoine Labour
3866923b00 Revert "This change removes the autotest mount-specific change from the mod_for_test scripts as we no longer need executable /tmp or /home for autotest."
This reverts commit 4092ff5651.

TBR=fes

Change-Id: I9baa8836fce86963af97a618d75aeb90325fbc42
2011-01-12 14:36:51 -08:00
Frank Swiderski
4092ff5651 This change removes the autotest mount-specific change from the mod_for_test scripts as we no longer need executable /tmp or /home for autotest.
Change-Id: I03c563b11d5df043683e904c7794faa34748b063

BUG=chromium-os:7286
TEST=bvt autotest

Review URL: http://codereview.chromium.org/6080007
2011-01-12 13:50:48 -08:00
Will Drewry
d07f61d812 build_image: enable Chrome OS partition/hardware error behavior in dm-verity
ARM (kernel-next) should now be in sync with kernel.git so that we get similar
error handling behavior on all targeted platforms (with our partition layout).

This change makes that the default.

BUG=chromium-os:9697
TEST=manual build_image with kernel-next profile; booted and validated failover error behavior

Change-Id: I281302adb3cbc96aa5770630898f103b0d3639ca

Review URL: http://codereview.chromium.org/6220002
2011-01-12 10:39:54 -06:00
Chris Sosa
8617623516 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario.  Re-tested with new changes after build break.

Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=2b2d7a1

Review URL: http://codereview.chromium.org/6015013
2011-01-10 13:37:49 -08:00
Dale Curtis
86213e6795 Fix path for upcoming Autotest relocation.
Additional change for the relocation of Autotest from /home/autotest to /usr/local/autotest. See http://codereview.chromium.org/6090004/

Change-Id: Id6f7ac6e93124414b0b6c6bed48efccfd982a49d

BUG=chromium-os:10799
TEST=Will test as whole with rest of /usr/local changes.

Review URL: http://codereview.chromium.org/5972012
2011-01-10 11:50:01 -08:00
Chris Sosa
f58bc389d9 Create package keywords directory if it doesn't exist.
Clobbering the builder uncovered a bug that was lurking.

Change-Id: Ieb487d2112a7048438943aa96fba23e9c8412a13

BUG=chromium-os:8693
TEST=Running now

Review URL: http://codereview.chromium.org/6050004
2011-01-10 10:58:58 -08:00
Chris Sosa
efbce1e261 Add simple update test to pfq.
This CL also fixes the outstanding issue where to won't report progress in
image_to_live on updates.

Change-Id: I1efaf17f0fd5ebb367ae0872377b4d5d0bf5dbf6

BUG=chromium-os:8680,chromium-os:8901
TEST=Ran it with --debug on the x86 pfq locally.

Review URL: http://codereview.chromium.org/6135002
2011-01-10 09:56:34 -08:00
Scott Zawalski
53f189e92f Update gsutil path as it has changed 2011-01-06 00:44:36 -08:00
J. Richard Barnette
e7205e84be Revert "Major cleanup of cros_au_test_harness code."
This reverts commit 2b2d7a1426.

The internal x86 buildbots have been reporting failures from
cros_au_test_harness with every build, starting with the build
that included the change being reverted.

BUG=chromium-os:8901
TEST=None

Review URL: http://codereview.chromium.org/5962019

Change-Id: Iac8c0a13d659f927b3ff61a9a894c10191432e2d
2011-01-05 18:38:49 -08:00
Scott Zawalski
a25cede07d Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2011-01-05 17:40:10 -08:00
Chris Sosa
2b2d7a1426 Major cleanup of cros_au_test_harness code.
This cleanup code consolidates parsing of options, removes
globals, cleans up private vs. public functions, removes the backup full update option, and cleans up
docstrings.

Change-Id: I3eba26b3634fd87f4a36f195ab42ec7bd11e963f

BUG=chromium-os:8901
TEST=Ran with prefix SimpleTest and ran whole suite for a vm using my latest built image.  Also ran normal update test using latest image against latest dev channel image for mario

Review URL: http://codereview.chromium.org/6015013
2011-01-04 18:05:59 -08:00
Zdenek Behan
75805716f5 enter_chroot: copy over ssh configuration
TEST=clean (first) enter_chroot.sh; repo sync

The developer instructions for setting up chroot include steps to set up
~/.ssh/config to correctly set up ports and options. And while several
attempts are made in enter_chroot to forward ssh-agent and ssh
authentication to chroot as well, the ssh config stays outside of
chroot, which prevents all attempts to contact the git servers if rw URL
is used.

This makes a copy of config on entering the chroot, allowing people to
use repo from within the chroot. That is not bulletproof and does not
reflect changes of config while inside the chroot, but that should be a
borderline case.

Change-Id: I1fcbf00c413c7af8ef14fc1e846192ac95de106d

BUG=n0ne

Review URL: http://codereview.chromium.org/6065011
2011-01-05 00:56:32 +01:00
David James
3f898563fb Prebuilts should not gate normal builds if the server responds with an error.
Recently the tegra2 prebuilts were uploaded with an internal only address and caused external builds to fail.

If a user cannot get the Packages file their build should continue as normal as if the server responded without a packages file in general.

BUG=chromium-os:9917
TEST=Set PORTAGE_BINHOST to invalid value and run build.

Change-Id: I95cab67da7a539afe8c48c831788eca29cc5dc69

Review URL: http://codereview.chromium.org/6085001
2011-01-04 15:36:47 -08:00
David James
8ce81f32dc Update parallel_emerge to experimentally support Portage 2.1.9.
Portage 2.1.9 changes a few APIs, so parallel_emerge needs to change with it.
With this patch, parallel_emerge basically seems to work with Portage 2.1.9.

Note #1: All of our patches against Portage 2.1.7 will need to be forward
ported to Portage 2.1.9 before we'll have complete support for it. See
http://codereview.chromium.org/6093002/ for that.

Note #2: This includes http://codereview.chromium.org/6055004/ . That patch
can be submitted separately, though, and that'll make this patch smaller.

BUG=chromium-os:10574
TEST=Ran ./build_packages  --board=x86-generic --oldchromebinary with and without new portage.
It completes successfully in both cases.

Review URL: http://codereview.chromium.org/6094001

Change-Id: I0f851b040e266d98933962b4ccc1c85f7851c8ad
2011-01-04 15:33:53 -08:00
David James
999e09f5c4 Fix bug 10466: parallel_emerge use flag calculation favors local binhost.
When we merge a package, we need to keep track of the use flags associated with
the package, so that we don't merge a package with the wrong use flags.

In some cases, there might be two copies of the same binary package: One local
copy, and one remote copy. Our current logic for grabbing the use flags favors
the local copy. But Portage itself favors the remote copy. To reconcile this
difference, we just reuse the logic from Portage here.

BUG=chromium-os:10466
TEST=
  1. Run build_packages with a correct binhost
  2. Switch to a binhost with different use flags for some of the packages, but the same version number and package name.
  3. Run EXTRA_BOARD_FLAGS=--emptytree build_packages
  4. Run build_image

Also test #3 with --oldchromebinary.

Change-Id: I9b917d8b8d902e0581d5a5d23ad20940930b114a

Review URL: http://codereview.chromium.org/6055004
2011-01-04 15:15:02 -08:00
Scott Zawalski
3dfb64fd55 Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2011-01-04 13:21:33 -08:00
Chris Sosa
50744ba57c Remove try/catch so that ctest blocks builds.
All blocking reasons for au test harness flakiness have been resolved.  Removing try/catch from ctest so that we propagate the correct error code upon exit.

Change-Id: Iee1f09ade37ac7a590c2dbf4ebb96985818f1882

BUG=chromium-os:10434
TEST=Ran ctest unittests.

Review URL: http://codereview.chromium.org/5971009
2011-01-04 12:17:17 -08:00
Chris Sosa
70311cdfdd Change logic to auto-rev for latest_release and sticky branch.
Also some cleanup and fixing so we don't create unnecessary ebuilds.
I noticed that both the latest release and stable release code
were trying to rev the same branch whenever we moved to a new stable
version.  It was harmless before but now that we're pushing this as
stable we want exactly to have the minimum number of revs that are
correct

Change-Id: I9f834d35be8e85c27ab0505ec81637f9a3c27b88

BUG=
TEST=Unittests and mocked in my own directory by creating new stable
branch.

Review URL: http://codereview.chromium.org/6025012
2010-12-29 15:23:15 -08:00
Nick Sanders
607c0780c0 Allow disk image target for copy machine.
BUG=10531
TEST=build and imaged a mario

Change-Id: I60790f178159d707f610ba1668c29179a6c32e61

Review URL: http://codereview.chromium.org/6050006
2010-12-28 22:27:11 -08:00
Chris Sosa
b1d17e24eb Move git config till after we sync.
This deals with the condition where on a clobber we don't actually
re-config to use ssh:// until the second run (because git config is
called before we first sync the code).

Change-Id: Icf1a46aced633b570db9d280b10ae6866af04709

BUG=chromium-os:10545
TEST=Unit tests

Review URL: http://codereview.chromium.org/6027006
2010-12-22 14:10:10 -08:00
Olof Johansson
5a46bfb321 update_kernel: add support for modules and firmware
Change-Id: I7b38b5f9656bd21f6c88c5385c1aa0c526f48294
Signed-off-by: Olof Johansson <olofj@chromium.org>

BUG=n0ne
TEST=build kernel and firmware, update a mario

Review URL: http://codereview.chromium.org/5974003
2010-12-22 12:14:21 -08:00
Chris Sosa
4e32890c9b Add new script to stop a kvm using vm library and use in harness.
Change-Id: Ie80843a7b81a37b41ae19fc33244b5c0b8152282

BUG=chromium-os:10434
TEST=Ran it with full au test harness (still running but already showing promise).

Review URL: http://codereview.chromium.org/5988006
2010-12-21 14:56:32 -08:00
Don Garrett
c6c925d8ef image_to_live verifies root_dev changed.
Without this, image_to_live can sometimes think that an update from the same version to same version worked when it failed.

BUG=chromium-os:9002
TEST=By hand (hardware && success only)

Review URL: http://codereview.chromium.org/6045004

Change-Id: Ic0835372a45a45785e21ba1527d478f447bcf42a
2010-12-21 13:41:57 -08:00
Daniel Erat
e82f07c262 crosutils: Exclude Xvfb and Xnest from images.
These are used to run virtual or nested X servers, which we
don't do.  Removing the binaries saves 3.4 MB.

Change-Id: I528231a781f8a7a0202ca109f8064c3b11ff172b

BUG=chromium-os:8073
TEST=built an image and checked that the files were gone

Review URL: http://codereview.chromium.org/5990003
2010-12-21 13:39:22 -08:00
David James
892e6d8f3c Fix failing unit test for cros_mark_as_stable.
Change-Id: I683cdbec489395bbce53ae4d4ba45ec3538c2aac

BUG=chromium-os:8693
TEST=Run unit tests.

Review URL: http://codereview.chromium.org/6089002
2010-12-21 12:50:21 -08:00
David James
702f0e31ca Be more selective about what packages we remove so that we can detect conflicts.
Right now, the preflight queue unmerges all changed packages prior to
upgrading them. This is done to ensure that any lingering state from
failed package uprevs is cleaned up.

This change updates cros_mark_as_stable to be more selective. This both improves
speed and allows for more conflicts to be detected.

Change-Id: I3134918eb16b41e8882c8af1f1bdf9052bafd9ad

BUG=chromium-os:10127
TEST=Test uprev. Run unit tests.

Review URL: http://codereview.chromium.org/5513012
2010-12-21 12:46:46 -08:00
Chris Sosa
6b69ba7121 Always run build_packages regardless of sync for chrome builder.
Change-Id: If08f3a0c1212e3e7d4ae8dc9f8aa92e2454d4d74

BUG=chromium-os:8693
TEST=Ran it

Review URL: http://codereview.chromium.org/6016005
2010-12-20 11:40:29 -08:00
David James
fa98729856 parallel_emerge now uninstalls packages correctly when use flags change.
When I added the --force-remote-binary flag, I needed to find a way to
get emerge to install binary packages whose use flags were not fully up
to date. To do this, I removed the --newuse flag from the call to the
emerge depgraph calculation code. This seemed safe at the time because
parallel_emerge already checks the use flags of every package it
installs. Unfortunately, things are a little more complex. Sometimes,
emerge decides to install a different version of a package because of a
use flag change. To implement this, Portage needs to know about the
--newuse flag.

To fix this bug, I've updated parallel_emerge to force in the remote
binary packages using a less intrusive method, that only impacts the
package that is being forced. This method should have less side effects
and allow us to install packages correctly.

TEST=./setup_board --board=tegra2_dev-board --force &&
     ./parallel_emerge --board=tegra2_dev-board -uDNvg opengles &&
     ./parallel_emerge --board=tegra2_dev-board -puDNvg board-devices
     ./parallel_emerge --board=tegra2_dev-board -uDNvg chromeos
--force-remote-binary=chromeos-chrome
BUG=chromium-os:10423

Change-Id: I2057d2992abcb08f184cfaa6c099bd6eb8c21a99

Review URL: http://codereview.chromium.org/6033001
2010-12-18 07:18:33 -08:00
Daniel Erat
3a13f91195 scripts: Update factory mod diff for boot-complete job.
I'm changing the boot-complete init script to start on
login-prompt-visible instead of login-prompt-ready in
http://codereview.chromium.org/5868004/.

This also adds a missing description and fixes an outdated
email address.

BUG=chromium-os:10349
TEST=ran mod_image_for_test.sh with --factory and with --factory_install and checked that resulting image contained boot-complete.conf change

Review URL: http://codereview.chromium.org/5975002

Change-Id: I709bc2b4eaaccd222b31388633522c32514206ac
2010-12-17 16:36:02 -08:00
Chris Sosa
40fa0ded27 Fixes to get ctest up and running again.
We had a major issue because the latest dev channel release pre-dates the virtio change.  Because of this, ctest was connecting to update that channel using virtio (which is unsupported for that).  This change drops a file telling the vm lib to use e1000 as before.

Also, fixed stateful change logic.

Change-Id: I394a8cece71bdd0d55efa21ba5b6d24804432c6b

BUG=chromium-os:10434
TEST=Ran it with ctest and cros_au_test_harness on images with/without the file.

Review URL: http://codereview.chromium.org/5928005
2010-12-17 13:15:05 -08:00
Chris Sosa
d523d23f2e Remove explicit call to build Chrome.
Since the package.keywords change, it's already built as part of build_packages.
With this change we don't have to needlessly build chrome twice as part of the
pfq.

Change-Id: I8dae745496a5d6f6213663b034ffe8d279d4efd3

BUG=chromium-os:8693
TEST=Ran unittests

Review URL: http://codereview.chromium.org/5973003
2010-12-17 12:35:34 -08:00
David Rochberg
b4f6b3c6a1 Make restart_in_chroot_if_needed run against scripts that are symlinks
This reverts the main change of
84e72fab23

and undoes the breakage of scripts that are symlinks to
...chromeos/scripts..., but at the cost of re-breaking scripts under
scripts/bin.  The correct fix is a combination of removing all the
symlinks to ...chromeos/scripts... and a more robust way of finding
the chroot path of the executable.  But that will have to wait until
someone else does it next week or I return from vacation

TEST=Ran all of these outside chroot.  None seemed more broken than when run in the chroot.  /setup_board  --board=x86-generic, ./cros_workon list --all --board=x86-mario, ./start_devserver , ./build_image --help, ./build_packages --help, ./cros_upgrade_chroot --help, ./enter_chroot.sh , ./customize_rootfs --help, ./run_remote_tests.sh , ./verify_rootfs_chksum.sh , ./set_shared_user_password.sh
BUG=n0ne

Change-Id: Ic2b7484e53e6a977ed6ae675fe99e109b385020d

Review URL: http://codereview.chromium.org/5946003
2010-12-17 11:55:40 -05:00