Commit Graph

23028 Commits

Author SHA1 Message Date
Scott Zawalski
fba0975883 Remove a line 2010-10-04 15:18:54 -07:00
Scott Zawalski
f5948251b6 Change prebuilt filters to use the private ebuilds to generate a
definitive list.
Add unittests to test an empty filter list, loading filters, and
filtering specific packages based off of mock filters.
2010-10-04 15:15:44 -07:00
Scott Zawalski
11f13514b5 Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2010-10-04 15:15:09 -07:00
Chris Sosa
bd61304ebe Pass --board to get_latest_image.sh.
Change-Id: Iaa388bc6dd3ef8b07199ee42de6693cad2f4368f

BUG=
TEST=rm src/scripts/default_board and ran image_to_vm --board x86-generic.

Review URL: http://codereview.chromium.org/3530007
2010-10-04 13:29:23 -07:00
Scott Zawalski
e394d421fa Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2010-10-04 13:00:24 -07:00
David Rochberg
c1a3e56e23 Fix enter_chroot.sh so that it can be run from a cwd that is not in a git repository
BUG=none
TEST=run from within and without a repo

Change-Id: I00eb999974656450f6af24583ca2247805b5799e

Review URL: http://codereview.chromium.org/3529005
2010-10-04 09:44:57 -04:00
Hung-Te Lin
4732d5d0c3 crosutils: Improve dd in image_to_usb.sh
1. add oflags=sync to allow interrupting dd without hanging system I/O for a long time
   (since bs=4M, the speed is almost the same)
2. use pv to provide progress report if available.
   (add pv into host-depends in http://codereview.chromium.org/3608004)

BUG=none
TEST=manually:
1. install pv in chroot, then execute image_to_usb.sh --to /dev/sdX and verified there's progress bar
2. ctrl-c to break the dd process and verified it can stop immediately
3. remove pv from chroot, then execute image_to_usb.sh and verify that it still works

Change-Id: I62fc373a4feee6d7e61897325d9e1e6d84a74d63

Review URL: http://codereview.chromium.org/3581007
2010-10-04 11:01:11 +08:00
Chris Sosa
3ede50ed4e Add enabling of local account. This should get the suite_Smoke to go green on the pfq.
Change-Id: Id480f45836a81d7d994b92d7a45d16208e1dd11c

BUG=
TEST=Ran locally.  Saw all tests pass.

Review URL: http://codereview.chromium.org/3567010
2010-10-01 14:12:26 -07:00
Scott Zawalski
9d892dd99f First submission of prebuilt upload script.
This introduces a way to upload prebuilts from the command line.

The following features are available

  * Multiprocess upload (cuts time down by a 1/5th for uploading)
  * Versioned uploading with git file commital
  * Host prebuilt uploading
  * Board prebuilt uploading

BUG=chromium-os:4843,chromium-os:5855

TEST=./prebuilt_unittest.py
{'/b/cbuild/build/chroot/build/x86-dogfood/packages/x11-misc/util-macros-1.5.0.tbz2':
'gs://chromeos-prebuilt/host/version/packages/x11-misc/util-macros-1.5.0.tbz2',
'/b/cbuild/build/chroot/build/x86-dogf}
.FILTERING /usr/local/package/oob
FILTERING /var/tmp/bibby.file
.....Updating stage 20100309/stage3-amd64-20100309.tar.bz2 to stage
test_update
.
----------------------------------------------------------------------
Ran 7 tests in 0.002s

OK

Review URL: http://codereview.chromium.org/3452032
2010-10-01 11:28:44 -07:00
Scott Zawalski
da17d44127 Merge branch 'master' of ssh://gitrw.chromium.org:9222/crosutils 2010-10-01 11:28:21 -07:00
Paul Stewart
3da3113c01 Remove cros_copy_upgrade_server.sh
This script had a short painful life, constantly being brittle
and broken.  It is time for it to go.  Please use
cros_image_to_target.py instead.

BUG=none
TEST=none

Review URL: http://codereview.chromium.org/3536002
2010-10-01 11:22:23 -07:00
Scott Zawalski
f9923b0164 Changed FILTER_PACKAGES to _FILTER_PACKAGES
Fixed a missed TODO item without username
2010-10-01 10:58:55 -07:00
Paul Stewart
48d7d8bb7f Add sha256 to devserver clone updater output
BUG=none
TEST=none

Review URL: http://codereview.chromium.org/3572006
2010-09-30 18:16:21 -07:00
Chris Sosa
cb9e4fe449 Change hostname to localhost ip
TBR

Change-Id: I1a911da2df7cddda3eb35f6df23e203cd94c1ffe

BUG=
TEST=Locally.

Review URL: http://codereview.chromium.org/3536003
2010-09-30 18:02:09 -07:00
Chris Sosa
bccc7c1d03 Initial auto update engine test harness.
This harness has support for test on vm machines.  It takes a base image and target image path, creates an update-enabled vm image from the base image and performs updates using cros_run_vm_update (vm wrapper around image_to_live).  It uses --snapshot for updating vm's and keeps them on (like a real machine would) so that the VM image is clean on the next test.

The current check in has two test cases -- updates without clobbering the stateful partition and updates with clobbering the stateful partition.

Change-Id: I5406eadb39fa36601d17f8dd15625bd6a8b38c15

BUG=6911
TEST=Ran through tests.

Review URL: http://codereview.chromium.org/3554002
2010-09-30 15:06:15 -07:00
Chris Sosa
c2db26c796 Add methods to run vm tests on x86-pre-flight-queue.
Right now we're leaving error_ok=True until the tests are shown to work on builder.

Change-Id: I6b8c690a0da30948389fd4312032c78d87115364

BUG=6906
TEST=Ran through it locally.

Review URL: http://codereview.chromium.org/3591002
2010-09-30 14:25:25 -07:00
Kenneth Waters
e3049de00f build_image: Merge x86 and ARM where possible.
- I looked at all of the x86 and ARM paths through out build image scripts,
  these changes clean up stale comments, stale code, and unforks some small
  things.

BUG=none
TEST=Built images for x86-generic, arm-generic and tegra2-seaboard, booted tegra2-seaboard image.

Review URL: http://codereview.chromium.org/3448022

Change-Id: Ibad2774ff2cbf5f15528454506542b87e43e24a2
2010-09-30 14:20:34 -07:00
Ken Mixter
b5ab343a0d Fix grab-buildbot in sync_build_test by gsdizing it
Review URL: http://codereview.chromium.org/3109023
2010-09-30 13:05:56 -07:00
Tan Gao
29d962035e remove second call to free up loop device
Change-Id: Ibf52597f01b7868f8472d2042e4d335442b82597

TEST=manually build a dev recovery image and verified no loop device warning
was outputted by losetup call

Review URL: http://codereview.chromium.org/3439024
2010-09-30 10:41:09 -07:00
Scott Zawalski
6332272fa7 A few more style changes and logic changes fro code review 2010-09-30 08:26:04 -07:00
David James
5137a74e90 Exit explicitly at end of parallel_emerge, so child processes exit as well.
So, we've been seeing parallel_emerge hang after printing "Done". It's hard to
know for sure why this hang is happening, but my theory right now is that the
Python garbage collector doesn't always know that it needs to kill the child
processes when we exit normally at the bottom of the script. Adding sys.exit(0)
tells Python that yes, we really want to exit.

I haven't tested my hypothesis above, because the hang we've been seeing is
pretty rare and is hard to reproduce. So I'm planning on testing by just
putting in this workaround and seeing if it fixes the problem.

TEST=Run ./parallel_emerge world
BUG=chromium-os:5976

Change-Id: I7b4f2ec4ccba9b00f22f3739dfd3eff51ceed425

Review URL: http://codereview.chromium.org/3448030
2010-09-29 15:50:08 -07:00
Chris Sosa
45bec936f6 Move vm constants into their own file. This will make it easier to share these constants.
Also fixed get latest image logic in image_to_vm and allowed for using the most recent image in cros_run_vm_test to follow other shell convention of using latest.

Change-Id: I60ed4c03d609500da7f6ae34ef57ba2e32f4b0bb

BUG=
TEST=Tested by running image_to_vm with --full and cros_run_vm_test with suite_Smoke

Review URL: http://codereview.chromium.org/3597001
2010-09-29 15:38:53 -07:00
Chris Sosa
3b65e8dd6f Add verify option to image_to_live.
Change-Id: I63f7ce38179b713bb26bed70d6fd392425595a81

BUG=
TEST=Ran with update vm script.

Review URL: http://codereview.chromium.org/3393019
2010-09-28 18:41:56 -07:00
Chris Sosa
e7f1a2b6bb Add read only flag to mount_gpt_image
Change-Id: I33ea8b2148a3e49a6c5de25f2957306130ff092e

BUG=
TEST=Ran and verified with and without flag.  Checked mount and tried writing
file to rootfs (to verify it was ro).  Also re-ran cros_make_image_bootable
to make sure it worked fine after changes.

Review URL: http://codereview.chromium.org/3479007
2010-09-28 18:24:13 -07:00
David McMahon
e86529b046 Created branch 0.9.74.B. Update CHROMEOS_VERSION_BRANCH=75
Change-Id: Ib47af14a6a102e32cd8bdd66d6f62a8a679f7792
2010-09-28 18:23:28 -07:00
Scott Zawalski
6f0584cd2d Update prebuilt and prebuilt_unittest to address comments in code
review.
2010-09-28 14:55:36 -07:00
Chris Sosa
f485c86416 Fix unit tests for cbuildbot.
Change-Id: I86b3eb5d7b421aaade85bd702b9089986c82cc76

BUG=
TEST=Re-ran unit tests with cbuildbot TOT.

Review URL: http://codereview.chromium.org/3441027
2010-09-28 13:08:40 -07:00
Scott Zawalski
e773e514bf Add in __init__.py files in both chromite and chromite/lib for importing 2010-09-28 09:46:52 -07:00
Scott Zawalski
4f33955508 First submission of prebuilt upload script.
This introduces a way to upload prebuilts from the command line.

The following features are available

  * Multiprocess upload (cuts time down by a 1/5th for uploading)
  * Versioned uploading with git file commital
  * Host prebuilt uploading
  * Board prebuilt uploading

BUG=chromium-os:4843,chromium-os:5855

TEST=./prebuilt_unittest.py
{'/b/cbuild/build/chroot/build/x86-dogfood/packages/x11-misc/util-macros-1.5.0.tbz2':
'gs://chromeos-prebuilt/host/version/packages/x11-misc/util-macros-1.5.0.tbz2',
'/b/cbuild/build/chroot/build/x86-dogf}
.FILTERING /usr/local/package/oob
FILTERING /var/tmp/bibby.file
.....Updating stage 20100309/stage3-amd64-20100309.tar.bz2 to stage
test_update
.
----------------------------------------------------------------------
Ran 7 tests in 0.002s

OK
2010-09-28 09:38:03 -07:00
Olof Johansson
3ed8d12f48 image_to_vm: print launch command for qemu images
Change-Id: Id62c28eac319caf15a2bf83a69b4f727f9e9d56b

BUG=none
TEST=sudo emerge qemu-kvm ; image_to_vm <...> ; cut and paste of output

Review URL: http://codereview.chromium.org/3418031
2010-09-27 13:29:40 -05:00
David James
0fd8af4a9b Configure git author name and git author email in .gitconfig.
If git author name is different in the chroot from outside the chroot, repo
may auto-delete your changes when you run repo sync. These changes can be
recovered, but their deletion is inconvenient.

BUG=chromium-os:5934
TEST=Check that git var GIT_COMMITTER_IDENT is now correct inside chroot.

Change-Id: I622cd37caa404a7482c55c4b3984877b32b3807c

Review URL: http://codereview.chromium.org/3352008
2010-09-27 10:04:57 -07:00
David James
70b297a263 Update cbuildbot.py to print manifests when we sync.
If cbuildbot.py printed out a manifest when it syncs, it would make it possible
for us to reproduce the build later by looking at what revisions the buildbot
was using. This is useful for debugging.

Change-Id: I186ad01eef0090b6c618cb7ad583085197474018

BUG=chromium-os:7069
TEST=Ran test suite with 3441027 patched in (Sosa's patch that fixes an unrelated issue with the unit tests)

Review URL: http://codereview.chromium.org/3461028
2010-09-25 23:25:18 -07:00
David James
7bf8d2715e Switch off repo sync --jobs, and turn on repo --trace.
repo sync --jobs doesn't check for errors, so we shouldn't use it in
automated scripts. repo --trace is needed so that we can diagnose
whether git is hanging.

BUG=chromium-os:7048, chromium-os:6774
TEST=Ran test suite

Change-Id: I9e68cfffe841a231f5fabef951ea2d45b81c1d5e

Review URL: http://codereview.chromium.org/3419024
2010-09-25 23:23:00 -07:00
David James
1a6b812cf0 Update parallel_emerge to retry when we encounter DNS errors.
BUG=chromium-os:7049
TEST=Tested that DNS errors are retried. Also tested that 404 errors are not retried.

Change-Id: I0f38764e7af822f7db554697b9fa84af033bf111

Review URL: http://codereview.chromium.org/3473017
2010-09-24 21:55:33 -07:00
Zdenek Behan
358b738f63 upgrade_chroot: create an upgrade script
Change-Id: Ief6360b636884b02bdf74baeb6c4103eb180d2b7

BUG=6151
TEST=run it

Review URL: http://codereview.chromium.org/3331011
2010-09-24 16:23:29 -07:00
Mandeep Singh Baines
ee49ac0ec5 loman: Add support for adding elements from default.xml
This is needed in order to do the right thing when adding projects
which require a revision attribute.

BUG=6811
TEST=Updated unittests and verified they pass.
Verified that I can add chromeos-kernel-next and revision gets set.

Change-Id: I12818dad464094cd50098170da3fcf38b1e32f17

Review URL: http://codereview.chromium.org/3400019
2010-09-24 08:27:50 -07:00
Ken Mixter
aa5504fdec crosutils: Make run_remote_tests' output for ambiguous case more clear.
BUG=5817
TEST=run_remote_tests --remote=$IP control shows ^...$ on all lines.

Change-Id: Ie0c4ac16493dc97044ec0bd0fd6d6e53a69c9041

Review URL: http://codereview.chromium.org/3473015
2010-09-23 17:12:06 -07:00
Tan Gao
853d70bae1 clean up excessive output printed by mod_image_for_dev_recovery
Change-Id: I8d04a04ad75f8cd5b040572ac6bb16e594d59e0e

TEST=manually built a dev recovery image and tested in agz device to verify
it installed payload image onto the target and the target is able to boot from HD

Review URL: http://codereview.chromium.org/3432017
2010-09-23 14:40:11 -07:00
Kenneth Waters
9bc78b3acb make_image: Fake vmlinuz_hd.vblock on ARM.
- This unbreaks the devserver's update server for ARM.
- This also unforks some of build_image.

BUG=None
TEST=Installed to tegra2_seaboard from USB without --skip_vblock,
     updated tegra2_seaboard.

Review URL: http://codereview.chromium.org/3493004

Change-Id: I6e66344de51609393407934f78aa20f49974efef
2010-09-22 13:54:44 -07:00
Zdenek Behan
bc9981cc90 cros_mark_all_as_stable: revert commit 868b777116
TBR=sosa,kwaters

Change-Id: Ieef77cd7f08116b76d84df028be04139c2b10e48

BUG=
TEST=

Review URL: http://codereview.chromium.org/3473012
2010-09-22 12:59:43 -07:00
Chris Sosa
8f6ee51c21 Add persist option to vm library.
BUG=
TEST=Tested by persisting, ctrl+c, continuing and using same vm.
Also without --persist

Review URL: http://codereview.chromium.org/3419014

Change-Id: I49e26eb86101e8b8b50e58a9f277d62d5fadc664
2010-09-22 11:51:52 -07:00
Tan Gao
e5ba918044 TBR: davidjames
Change-Id: I0b8d4277b142185e69abcd8e8bb2dce6395abc56
2010-09-22 09:34:27 -07:00
Chris Sosa
e8c3cdc91c cros_make_image_bootable is not compatible with restart_inside_chroot
restart_inside_chroot first assumes that the calling script is in
src/scripts.  Next any paths are not converted when passed so if
someone passes the most recent image, this will also not be interpreted
correctly.

Change-Id: Icb2dfc661eb84e134b4d8c5cf0354a6c1ab161f0

BUG=
TEST=Manually tested running outside chroot (getting error) and inside chroot.

Review URL: http://codereview.chromium.org/3421025
2010-09-21 17:45:15 -07:00
Chris Sosa
cc09f84b5e Update scripts for vm's
Change-Id: I63bdcaa3630c179d407310f6544c102f5353fa4d

BUG=5530
TEST=Ran both cros_run_vm_test and new script and had the latter update successfully.

Review URL: http://codereview.chromium.org/3427011
2010-09-21 17:09:51 -07:00
Paul Stewart
b0d5cf28e5 New application cros_image_to_target.py for ssh_tunnel upgrade
The RF testbed has hosts behind a one-way firewall, so the
conventional "image_to_live" script does not work.  My initial
attempt at solving this, 'cros_copy_upgrade_server', while
sometimes functional, was always quite brittle since it had
widely ranging dependencies which often changed in ways that
the authors could not predict would break our setup.

I've written from scratch a new script in python which uses
ssh tunnels to get around the problem of the firewall issue.
It also has retlatively minimal dependencies (stateful_update,
cros_generate_update_paylod, cgpt, get_latest_image.sh and
the testing keys) so hopefully it won't get blown away too
often.

BUG=none
TEST=Ran under chroot -- Need help testing

Review URL: http://codereview.chromium.org/3391008
2010-09-21 15:13:59 -07:00
Paul Stewart
fae37db122 Add "--mount" option to package_to_live
Optionally disable mount/remount during package install for
targets that are already mounted read-write.

BUG=none
TEST=rerun

Review URL: http://codereview.chromium.org/3439012
2010-09-21 13:19:48 -07:00
Mandeep Singh Baines
fd76ba692d cros_workon: only touch local_manifest for minilayout users
Otherwise, you'll duplicate an entry already in the developers
manifest and cause repo to get confused.

BUG=6898
TEST=Verified that local_manifest gets updated when using minilayout.xml
Verified that the local_manifset does not get updated when not.

Change-Id: I67ffc7004a2267d0d5aaa31570ac2bbeaa8f4f96

Review URL: http://codereview.chromium.org/3468009
2010-09-21 12:05:10 -07:00
Kenneth Waters
ed54d93e2c build_image: Add verity support for ARM.
- Build a "kernel image" which contains a uboot script and a uboot kernel
  image.
- Fix some sd* assumptions.
- Remove cruft that has never done anything usefull from update_bootloaders

BUG=none
TEST=Built, booted, and updated on tegra2_dev-board

Review URL: http://codereview.chromium.org/3396011

Change-Id: I00ecf57faa5fe64c8e33dd4c042f1dbed806c10a
2010-09-21 10:29:54 -07:00
Tan Gao
bf9178b36e Issue 6821: script to verify rootfs integrity against value stored in kernel
Change-Id: I09b851e5b9e971c281305802f287fe93e9309fe8

BUG=chromium-os:6821
TEST=manually ran script inside chroot for both a USB device and a dev build
of Chrome OS image. USB image has corrupted rootfs and dev build does not. The
script was able to detect both scenarios correctly.

Review URL: http://codereview.chromium.org/3452013
2010-09-21 10:04:32 -07:00
David McMahon
df93272461 Created branch 0.8.72.B. Update CHROMEOS_VERSION_BRANCH=73
Change-Id: Ib9004f5dc0645389dc1934c65826000acca8b908
2010-09-20 18:07:05 -07:00