Commit Graph

5265 Commits

Author SHA1 Message Date
Chris Sosa
cb9e4fe449 Change hostname to localhost ip
TBR

Change-Id: I1a911da2df7cddda3eb35f6df23e203cd94c1ffe

BUG=
TEST=Locally.

Review URL: http://codereview.chromium.org/3536003
2010-09-30 18:02:09 -07:00
Chris Sosa
bccc7c1d03 Initial auto update engine test harness.
This harness has support for test on vm machines.  It takes a base image and target image path, creates an update-enabled vm image from the base image and performs updates using cros_run_vm_update (vm wrapper around image_to_live).  It uses --snapshot for updating vm's and keeps them on (like a real machine would) so that the VM image is clean on the next test.

The current check in has two test cases -- updates without clobbering the stateful partition and updates with clobbering the stateful partition.

Change-Id: I5406eadb39fa36601d17f8dd15625bd6a8b38c15

BUG=6911
TEST=Ran through tests.

Review URL: http://codereview.chromium.org/3554002
2010-09-30 15:06:15 -07:00
Chris Sosa
c2db26c796 Add methods to run vm tests on x86-pre-flight-queue.
Right now we're leaving error_ok=True until the tests are shown to work on builder.

Change-Id: I6b8c690a0da30948389fd4312032c78d87115364

BUG=6906
TEST=Ran through it locally.

Review URL: http://codereview.chromium.org/3591002
2010-09-30 14:25:25 -07:00
Kenneth Waters
e3049de00f build_image: Merge x86 and ARM where possible.
- I looked at all of the x86 and ARM paths through out build image scripts,
  these changes clean up stale comments, stale code, and unforks some small
  things.

BUG=none
TEST=Built images for x86-generic, arm-generic and tegra2-seaboard, booted tegra2-seaboard image.

Review URL: http://codereview.chromium.org/3448022

Change-Id: Ibad2774ff2cbf5f15528454506542b87e43e24a2
2010-09-30 14:20:34 -07:00
Ken Mixter
b5ab343a0d Fix grab-buildbot in sync_build_test by gsdizing it
Review URL: http://codereview.chromium.org/3109023
2010-09-30 13:05:56 -07:00
Tan Gao
29d962035e remove second call to free up loop device
Change-Id: Ibf52597f01b7868f8472d2042e4d335442b82597

TEST=manually build a dev recovery image and verified no loop device warning
was outputted by losetup call

Review URL: http://codereview.chromium.org/3439024
2010-09-30 10:41:09 -07:00
Scott Zawalski
6332272fa7 A few more style changes and logic changes fro code review 2010-09-30 08:26:04 -07:00
David James
5137a74e90 Exit explicitly at end of parallel_emerge, so child processes exit as well.
So, we've been seeing parallel_emerge hang after printing "Done". It's hard to
know for sure why this hang is happening, but my theory right now is that the
Python garbage collector doesn't always know that it needs to kill the child
processes when we exit normally at the bottom of the script. Adding sys.exit(0)
tells Python that yes, we really want to exit.

I haven't tested my hypothesis above, because the hang we've been seeing is
pretty rare and is hard to reproduce. So I'm planning on testing by just
putting in this workaround and seeing if it fixes the problem.

TEST=Run ./parallel_emerge world
BUG=chromium-os:5976

Change-Id: I7b4f2ec4ccba9b00f22f3739dfd3eff51ceed425

Review URL: http://codereview.chromium.org/3448030
2010-09-29 15:50:08 -07:00
Chris Sosa
45bec936f6 Move vm constants into their own file. This will make it easier to share these constants.
Also fixed get latest image logic in image_to_vm and allowed for using the most recent image in cros_run_vm_test to follow other shell convention of using latest.

Change-Id: I60ed4c03d609500da7f6ae34ef57ba2e32f4b0bb

BUG=
TEST=Tested by running image_to_vm with --full and cros_run_vm_test with suite_Smoke

Review URL: http://codereview.chromium.org/3597001
2010-09-29 15:38:53 -07:00
Chris Sosa
3b65e8dd6f Add verify option to image_to_live.
Change-Id: I63f7ce38179b713bb26bed70d6fd392425595a81

BUG=
TEST=Ran with update vm script.

Review URL: http://codereview.chromium.org/3393019
2010-09-28 18:41:56 -07:00
Chris Sosa
e7f1a2b6bb Add read only flag to mount_gpt_image
Change-Id: I33ea8b2148a3e49a6c5de25f2957306130ff092e

BUG=
TEST=Ran and verified with and without flag.  Checked mount and tried writing
file to rootfs (to verify it was ro).  Also re-ran cros_make_image_bootable
to make sure it worked fine after changes.

Review URL: http://codereview.chromium.org/3479007
2010-09-28 18:24:13 -07:00
David McMahon
e86529b046 Created branch 0.9.74.B. Update CHROMEOS_VERSION_BRANCH=75
Change-Id: Ib47af14a6a102e32cd8bdd66d6f62a8a679f7792
2010-09-28 18:23:28 -07:00
Scott Zawalski
6f0584cd2d Update prebuilt and prebuilt_unittest to address comments in code
review.
2010-09-28 14:55:36 -07:00
Chris Sosa
f485c86416 Fix unit tests for cbuildbot.
Change-Id: I86b3eb5d7b421aaade85bd702b9089986c82cc76

BUG=
TEST=Re-ran unit tests with cbuildbot TOT.

Review URL: http://codereview.chromium.org/3441027
2010-09-28 13:08:40 -07:00
Scott Zawalski
e773e514bf Add in __init__.py files in both chromite and chromite/lib for importing 2010-09-28 09:46:52 -07:00
Scott Zawalski
4f33955508 First submission of prebuilt upload script.
This introduces a way to upload prebuilts from the command line.

The following features are available

  * Multiprocess upload (cuts time down by a 1/5th for uploading)
  * Versioned uploading with git file commital
  * Host prebuilt uploading
  * Board prebuilt uploading

BUG=chromium-os:4843,chromium-os:5855

TEST=./prebuilt_unittest.py
{'/b/cbuild/build/chroot/build/x86-dogfood/packages/x11-misc/util-macros-1.5.0.tbz2':
'gs://chromeos-prebuilt/host/version/packages/x11-misc/util-macros-1.5.0.tbz2',
'/b/cbuild/build/chroot/build/x86-dogf}
.FILTERING /usr/local/package/oob
FILTERING /var/tmp/bibby.file
.....Updating stage 20100309/stage3-amd64-20100309.tar.bz2 to stage
test_update
.
----------------------------------------------------------------------
Ran 7 tests in 0.002s

OK
2010-09-28 09:38:03 -07:00
Olof Johansson
3ed8d12f48 image_to_vm: print launch command for qemu images
Change-Id: Id62c28eac319caf15a2bf83a69b4f727f9e9d56b

BUG=none
TEST=sudo emerge qemu-kvm ; image_to_vm <...> ; cut and paste of output

Review URL: http://codereview.chromium.org/3418031
2010-09-27 13:29:40 -05:00
David James
0fd8af4a9b Configure git author name and git author email in .gitconfig.
If git author name is different in the chroot from outside the chroot, repo
may auto-delete your changes when you run repo sync. These changes can be
recovered, but their deletion is inconvenient.

BUG=chromium-os:5934
TEST=Check that git var GIT_COMMITTER_IDENT is now correct inside chroot.

Change-Id: I622cd37caa404a7482c55c4b3984877b32b3807c

Review URL: http://codereview.chromium.org/3352008
2010-09-27 10:04:57 -07:00
David James
70b297a263 Update cbuildbot.py to print manifests when we sync.
If cbuildbot.py printed out a manifest when it syncs, it would make it possible
for us to reproduce the build later by looking at what revisions the buildbot
was using. This is useful for debugging.

Change-Id: I186ad01eef0090b6c618cb7ad583085197474018

BUG=chromium-os:7069
TEST=Ran test suite with 3441027 patched in (Sosa's patch that fixes an unrelated issue with the unit tests)

Review URL: http://codereview.chromium.org/3461028
2010-09-25 23:25:18 -07:00
David James
7bf8d2715e Switch off repo sync --jobs, and turn on repo --trace.
repo sync --jobs doesn't check for errors, so we shouldn't use it in
automated scripts. repo --trace is needed so that we can diagnose
whether git is hanging.

BUG=chromium-os:7048, chromium-os:6774
TEST=Ran test suite

Change-Id: I9e68cfffe841a231f5fabef951ea2d45b81c1d5e

Review URL: http://codereview.chromium.org/3419024
2010-09-25 23:23:00 -07:00
David James
1a6b812cf0 Update parallel_emerge to retry when we encounter DNS errors.
BUG=chromium-os:7049
TEST=Tested that DNS errors are retried. Also tested that 404 errors are not retried.

Change-Id: I0f38764e7af822f7db554697b9fa84af033bf111

Review URL: http://codereview.chromium.org/3473017
2010-09-24 21:55:33 -07:00
Zdenek Behan
358b738f63 upgrade_chroot: create an upgrade script
Change-Id: Ief6360b636884b02bdf74baeb6c4103eb180d2b7

BUG=6151
TEST=run it

Review URL: http://codereview.chromium.org/3331011
2010-09-24 16:23:29 -07:00
Mandeep Singh Baines
ee49ac0ec5 loman: Add support for adding elements from default.xml
This is needed in order to do the right thing when adding projects
which require a revision attribute.

BUG=6811
TEST=Updated unittests and verified they pass.
Verified that I can add chromeos-kernel-next and revision gets set.

Change-Id: I12818dad464094cd50098170da3fcf38b1e32f17

Review URL: http://codereview.chromium.org/3400019
2010-09-24 08:27:50 -07:00
Ken Mixter
aa5504fdec crosutils: Make run_remote_tests' output for ambiguous case more clear.
BUG=5817
TEST=run_remote_tests --remote=$IP control shows ^...$ on all lines.

Change-Id: Ie0c4ac16493dc97044ec0bd0fd6d6e53a69c9041

Review URL: http://codereview.chromium.org/3473015
2010-09-23 17:12:06 -07:00
Tan Gao
853d70bae1 clean up excessive output printed by mod_image_for_dev_recovery
Change-Id: I8d04a04ad75f8cd5b040572ac6bb16e594d59e0e

TEST=manually built a dev recovery image and tested in agz device to verify
it installed payload image onto the target and the target is able to boot from HD

Review URL: http://codereview.chromium.org/3432017
2010-09-23 14:40:11 -07:00
Kenneth Waters
9bc78b3acb make_image: Fake vmlinuz_hd.vblock on ARM.
- This unbreaks the devserver's update server for ARM.
- This also unforks some of build_image.

BUG=None
TEST=Installed to tegra2_seaboard from USB without --skip_vblock,
     updated tegra2_seaboard.

Review URL: http://codereview.chromium.org/3493004

Change-Id: I6e66344de51609393407934f78aa20f49974efef
2010-09-22 13:54:44 -07:00
Zdenek Behan
bc9981cc90 cros_mark_all_as_stable: revert commit 868b777116
TBR=sosa,kwaters

Change-Id: Ieef77cd7f08116b76d84df028be04139c2b10e48

BUG=
TEST=

Review URL: http://codereview.chromium.org/3473012
2010-09-22 12:59:43 -07:00
Chris Sosa
8f6ee51c21 Add persist option to vm library.
BUG=
TEST=Tested by persisting, ctrl+c, continuing and using same vm.
Also without --persist

Review URL: http://codereview.chromium.org/3419014

Change-Id: I49e26eb86101e8b8b50e58a9f277d62d5fadc664
2010-09-22 11:51:52 -07:00
Tan Gao
e5ba918044 TBR: davidjames
Change-Id: I0b8d4277b142185e69abcd8e8bb2dce6395abc56
2010-09-22 09:34:27 -07:00
Chris Sosa
e8c3cdc91c cros_make_image_bootable is not compatible with restart_inside_chroot
restart_inside_chroot first assumes that the calling script is in
src/scripts.  Next any paths are not converted when passed so if
someone passes the most recent image, this will also not be interpreted
correctly.

Change-Id: Icb2dfc661eb84e134b4d8c5cf0354a6c1ab161f0

BUG=
TEST=Manually tested running outside chroot (getting error) and inside chroot.

Review URL: http://codereview.chromium.org/3421025
2010-09-21 17:45:15 -07:00
Chris Sosa
cc09f84b5e Update scripts for vm's
Change-Id: I63bdcaa3630c179d407310f6544c102f5353fa4d

BUG=5530
TEST=Ran both cros_run_vm_test and new script and had the latter update successfully.

Review URL: http://codereview.chromium.org/3427011
2010-09-21 17:09:51 -07:00
Paul Stewart
b0d5cf28e5 New application cros_image_to_target.py for ssh_tunnel upgrade
The RF testbed has hosts behind a one-way firewall, so the
conventional "image_to_live" script does not work.  My initial
attempt at solving this, 'cros_copy_upgrade_server', while
sometimes functional, was always quite brittle since it had
widely ranging dependencies which often changed in ways that
the authors could not predict would break our setup.

I've written from scratch a new script in python which uses
ssh tunnels to get around the problem of the firewall issue.
It also has retlatively minimal dependencies (stateful_update,
cros_generate_update_paylod, cgpt, get_latest_image.sh and
the testing keys) so hopefully it won't get blown away too
often.

BUG=none
TEST=Ran under chroot -- Need help testing

Review URL: http://codereview.chromium.org/3391008
2010-09-21 15:13:59 -07:00
Paul Stewart
fae37db122 Add "--mount" option to package_to_live
Optionally disable mount/remount during package install for
targets that are already mounted read-write.

BUG=none
TEST=rerun

Review URL: http://codereview.chromium.org/3439012
2010-09-21 13:19:48 -07:00
Mandeep Singh Baines
fd76ba692d cros_workon: only touch local_manifest for minilayout users
Otherwise, you'll duplicate an entry already in the developers
manifest and cause repo to get confused.

BUG=6898
TEST=Verified that local_manifest gets updated when using minilayout.xml
Verified that the local_manifset does not get updated when not.

Change-Id: I67ffc7004a2267d0d5aaa31570ac2bbeaa8f4f96

Review URL: http://codereview.chromium.org/3468009
2010-09-21 12:05:10 -07:00
Kenneth Waters
ed54d93e2c build_image: Add verity support for ARM.
- Build a "kernel image" which contains a uboot script and a uboot kernel
  image.
- Fix some sd* assumptions.
- Remove cruft that has never done anything usefull from update_bootloaders

BUG=none
TEST=Built, booted, and updated on tegra2_dev-board

Review URL: http://codereview.chromium.org/3396011

Change-Id: I00ecf57faa5fe64c8e33dd4c042f1dbed806c10a
2010-09-21 10:29:54 -07:00
Tan Gao
bf9178b36e Issue 6821: script to verify rootfs integrity against value stored in kernel
Change-Id: I09b851e5b9e971c281305802f287fe93e9309fe8

BUG=chromium-os:6821
TEST=manually ran script inside chroot for both a USB device and a dev build
of Chrome OS image. USB image has corrupted rootfs and dev build does not. The
script was able to detect both scenarios correctly.

Review URL: http://codereview.chromium.org/3452013
2010-09-21 10:04:32 -07:00
David McMahon
df93272461 Created branch 0.8.72.B. Update CHROMEOS_VERSION_BRANCH=73
Change-Id: Ib9004f5dc0645389dc1934c65826000acca8b908
2010-09-20 18:07:05 -07:00
Tan Gao
db7d4e77d4 Issue 6637: remove implicit dependency on unpack_partition.sh from mod_image_for_dev_recovery.sh
Change-Id: Ibc260c442f6219c000560f7d665d36543c936611

BUG=chromium-os:6637
TEST=manually created a developer recovery image and tested on agz device with developer switch ON that payload image was installed onto the device successfully and the device is able to boot from HD upon completion of recovery process

Review URL: http://codereview.chromium.org/3423014
2010-09-20 14:44:03 -07:00
David James
aee3041603 Print better error messages when the dependency graph is broken.
TEST=Add impossible-to-satisfy constraints into the dependency graph
     that require packages to be reinstalled due to use flags, and run
     ./parallel_emerge -p chromeos
BUG=chromium-os:6625

Change-Id: I389f7e5ba647a4be0413d1b0b6ea079df5b56433

Review URL: http://codereview.chromium.org/3459006
2010-09-19 14:24:16 -07:00
Paul Stewart
30d46e3c0d Re-do argument passing to match zbehan's change
zbehan's change: "run_remote_tests: destroy the with_spaces hack,
use the --args instead of -a (which doesn't work)", despite apparently
not working is in active use by others, both by folks using it on the
command line and by the script cros_run_wifi_tests.sh.  Fix the script
to match the new argument.

BUG=none
TEST=rerun

Review URL: http://codereview.chromium.org/3444010
2010-09-17 16:09:02 -07:00
Zdenek Behan
868b777116 cros_mark_all_as_stable: implement tracking of eclass changes
Change-Id: I4cf6f81aa9ce461c551c43fe456b936da70e8a58

BUG=
TEST=

Review URL: http://codereview.chromium.org/3426001
2010-09-17 13:19:58 -07:00
Chris Sosa
c5a2038009 Add ability for image_to_live to control most of dev_servers abilities.
Before this change, image_to_live could only be used by either passing a custom URL
or using the latest built image.  I've added the following:

- Ability to update from an image.zip
- Ability to update from a specific image.
- Ability to clobber the stateful partition after an update
  (excluding /usr/local and /var)
- Ability to customize and capture both the update and devserver logs.
- Ability to see the progress of an update.

I've also modified the start logic.  It seems the update_engine can get in a state
where if you fail a previous update by killing the devserver, the update engine
will hang for a long time (retries a lot).  So I've changed the behavior of
startup and detecting that the system isn't ready to update, to be, reboot and
then try the update.

BUG=
TEST=Tested with all options.

Review URL: http://codereview.chromium.org/3389009

Change-Id: I9bb42edbf5338385f55db242665c94cd552ab277
2010-09-17 10:55:39 -07:00
Zdenek Behan
3f40a62288 chromite: initial version of image_hacks.sh
Change-Id: Id0f833b5d077e5e90a4f38cb5397aa87ac4dc3bc

BUG=
TEST=

Review URL: http://codereview.chromium.org/3401001
2010-09-16 19:14:40 -07:00
Zdenek Behan
1b82990585 run_remote_tests: destroy the with_spaces hack, use the --args instead of -a (which doesn't work)
Change-Id: I8560a4400463d04b7a2441ce6fa9dc9cec834242

BUG=
TEST=run_remote_tests.sh --args='-n 10', and make sure it gets all the way into autotest_run.sh

Review URL: http://codereview.chromium.org/3402007
2010-09-16 14:31:34 -07:00
Mandeep Singh Baines
30b64cab3f cros_workon: fix bug where cros_workon start doesn't work for first start
If the checkout dir doesn't already exist cros_workon start will print
the wrong dir. Fixed by using readlink -m instead.

BUG=none
TEST=Verified that first start is incorrect without this patch
but is fixed with this patch.

Change-Id: I5db29a8c1737dea1dbe4866e18576f67b9355f84

Review URL: http://codereview.chromium.org/3434008
2010-09-16 14:28:31 -07:00
David James
a48cb7e19d Prototype cros_build_packages, which builds and extracts build tarballs.
Usage: cros_build_packages [options]

Options:
  -h, --help       show this help message and exit
  --board=BOARD    The board to build packages for.
  --debug          Include debug symbols.
  --nowithdev      Don't build useful developer friendly utilities.
  --nowithtest     Build packages required for testing.
  --nowithfactory  Build factory installer
  --nousepkg       Don't use binary packages.
  --nousetarball   Don't use tarball.
  --nofast         Don't merge packages in parallel.

BUG=chromium-os:6604
TEST=Ran cros_build_packages and ./cros_build_packages --nousetarball

Review URL: http://codereview.chromium.org/3315020

Change-Id: Ida9ed6375c2ce4acde304bd3f2796b4419f1d376
2010-09-16 13:05:06 -07:00
Mandeep Singh Baines
a4e793268b cros_workon: modify regen_manifest_and_sync to use loman
This fixes a bug where user added local_manifest.xml entries
get clobbered.

BUG=5787
TEST=Verified start of various packages works correctly.

Change-Id: I2348dfb1be098b81ede5928426192c655160564d

Review URL: http://codereview.chromium.org/3389013
2010-09-16 10:57:54 -07:00
Andrew de los Reyes
396306edea AU: when generating payload, only delta generation needs to be in chroot
BUG=chromium-os:6517
TEST=generated full payload outside chroot

Review URL: http://codereview.chromium.org/3359014
2010-09-15 19:19:04 -07:00
Zdenek Behan
18d3476a41 run_remote_tests: add quote marks around argument passing
* stops the second-pass parsing of arguments into IFS-delimited elements
which unbreaks things like --args='-n 10'

Change-Id: If8424b5fc06e6dd5a935421a76539e7e91006c37

BUG=
TEST=run it and see the argument being passed correctly

Review URL: http://codereview.chromium.org/3422005
2010-09-15 16:48:59 -07:00
David James
0f0402900a Temporary workaround to fix make_chroot --fast
Three changes here:
  1) parallel_emerge now only disables PORTAGE_LOCKS on board builds. This
     slows down make_chroot --fast, but works around a bug where make_chroot
     --fast sometimes merges broken packages.
  2) Only set PORTAGE_LOCKS if it's not already set. This allows users to
     override PORTAGE_LOCKS and request that it be enabled using
     PORTAGE_LOCKS=true
  3) Only add the no-env-update feature if PORTAGE_LOCKS=false. This feature
     is only needed for that case.

Long term fix is to patch PORTAGE_LOCKS feature to lock postinst. That'll be
tracked in a separate bug.

BUG=chromium-os:6750
TEST=Run make_chroot --fast. It works every time now, but is slower.
     build_packages && build_image run at same speed as before.

Change-Id: I12ab40671034e10cd2ffbba45281ca44718d1d2c

Review URL: http://codereview.chromium.org/3446006
2010-09-15 16:19:38 -07:00