Commit Graph

28179 Commits

Author SHA1 Message Date
David McMahon
df93272461 Created branch 0.8.72.B. Update CHROMEOS_VERSION_BRANCH=73
Change-Id: Ib9004f5dc0645389dc1934c65826000acca8b908
2010-09-20 18:07:05 -07:00
Tan Gao
db7d4e77d4 Issue 6637: remove implicit dependency on unpack_partition.sh from mod_image_for_dev_recovery.sh
Change-Id: Ibc260c442f6219c000560f7d665d36543c936611

BUG=chromium-os:6637
TEST=manually created a developer recovery image and tested on agz device with developer switch ON that payload image was installed onto the device successfully and the device is able to boot from HD upon completion of recovery process

Review URL: http://codereview.chromium.org/3423014
2010-09-20 14:44:03 -07:00
David James
aee3041603 Print better error messages when the dependency graph is broken.
TEST=Add impossible-to-satisfy constraints into the dependency graph
     that require packages to be reinstalled due to use flags, and run
     ./parallel_emerge -p chromeos
BUG=chromium-os:6625

Change-Id: I389f7e5ba647a4be0413d1b0b6ea079df5b56433

Review URL: http://codereview.chromium.org/3459006
2010-09-19 14:24:16 -07:00
Paul Stewart
30d46e3c0d Re-do argument passing to match zbehan's change
zbehan's change: "run_remote_tests: destroy the with_spaces hack,
use the --args instead of -a (which doesn't work)", despite apparently
not working is in active use by others, both by folks using it on the
command line and by the script cros_run_wifi_tests.sh.  Fix the script
to match the new argument.

BUG=none
TEST=rerun

Review URL: http://codereview.chromium.org/3444010
2010-09-17 16:09:02 -07:00
Zdenek Behan
868b777116 cros_mark_all_as_stable: implement tracking of eclass changes
Change-Id: I4cf6f81aa9ce461c551c43fe456b936da70e8a58

BUG=
TEST=

Review URL: http://codereview.chromium.org/3426001
2010-09-17 13:19:58 -07:00
Chris Sosa
c5a2038009 Add ability for image_to_live to control most of dev_servers abilities.
Before this change, image_to_live could only be used by either passing a custom URL
or using the latest built image.  I've added the following:

- Ability to update from an image.zip
- Ability to update from a specific image.
- Ability to clobber the stateful partition after an update
  (excluding /usr/local and /var)
- Ability to customize and capture both the update and devserver logs.
- Ability to see the progress of an update.

I've also modified the start logic.  It seems the update_engine can get in a state
where if you fail a previous update by killing the devserver, the update engine
will hang for a long time (retries a lot).  So I've changed the behavior of
startup and detecting that the system isn't ready to update, to be, reboot and
then try the update.

BUG=
TEST=Tested with all options.

Review URL: http://codereview.chromium.org/3389009

Change-Id: I9bb42edbf5338385f55db242665c94cd552ab277
2010-09-17 10:55:39 -07:00
Zdenek Behan
3f40a62288 chromite: initial version of image_hacks.sh
Change-Id: Id0f833b5d077e5e90a4f38cb5397aa87ac4dc3bc

BUG=
TEST=

Review URL: http://codereview.chromium.org/3401001
2010-09-16 19:14:40 -07:00
Zdenek Behan
1b82990585 run_remote_tests: destroy the with_spaces hack, use the --args instead of -a (which doesn't work)
Change-Id: I8560a4400463d04b7a2441ce6fa9dc9cec834242

BUG=
TEST=run_remote_tests.sh --args='-n 10', and make sure it gets all the way into autotest_run.sh

Review URL: http://codereview.chromium.org/3402007
2010-09-16 14:31:34 -07:00
Mandeep Singh Baines
30b64cab3f cros_workon: fix bug where cros_workon start doesn't work for first start
If the checkout dir doesn't already exist cros_workon start will print
the wrong dir. Fixed by using readlink -m instead.

BUG=none
TEST=Verified that first start is incorrect without this patch
but is fixed with this patch.

Change-Id: I5db29a8c1737dea1dbe4866e18576f67b9355f84

Review URL: http://codereview.chromium.org/3434008
2010-09-16 14:28:31 -07:00
David James
a48cb7e19d Prototype cros_build_packages, which builds and extracts build tarballs.
Usage: cros_build_packages [options]

Options:
  -h, --help       show this help message and exit
  --board=BOARD    The board to build packages for.
  --debug          Include debug symbols.
  --nowithdev      Don't build useful developer friendly utilities.
  --nowithtest     Build packages required for testing.
  --nowithfactory  Build factory installer
  --nousepkg       Don't use binary packages.
  --nousetarball   Don't use tarball.
  --nofast         Don't merge packages in parallel.

BUG=chromium-os:6604
TEST=Ran cros_build_packages and ./cros_build_packages --nousetarball

Review URL: http://codereview.chromium.org/3315020

Change-Id: Ida9ed6375c2ce4acde304bd3f2796b4419f1d376
2010-09-16 13:05:06 -07:00
Mandeep Singh Baines
a4e793268b cros_workon: modify regen_manifest_and_sync to use loman
This fixes a bug where user added local_manifest.xml entries
get clobbered.

BUG=5787
TEST=Verified start of various packages works correctly.

Change-Id: I2348dfb1be098b81ede5928426192c655160564d

Review URL: http://codereview.chromium.org/3389013
2010-09-16 10:57:54 -07:00
Andrew de los Reyes
396306edea AU: when generating payload, only delta generation needs to be in chroot
BUG=chromium-os:6517
TEST=generated full payload outside chroot

Review URL: http://codereview.chromium.org/3359014
2010-09-15 19:19:04 -07:00
Zdenek Behan
18d3476a41 run_remote_tests: add quote marks around argument passing
* stops the second-pass parsing of arguments into IFS-delimited elements
which unbreaks things like --args='-n 10'

Change-Id: If8424b5fc06e6dd5a935421a76539e7e91006c37

BUG=
TEST=run it and see the argument being passed correctly

Review URL: http://codereview.chromium.org/3422005
2010-09-15 16:48:59 -07:00
David James
0f0402900a Temporary workaround to fix make_chroot --fast
Three changes here:
  1) parallel_emerge now only disables PORTAGE_LOCKS on board builds. This
     slows down make_chroot --fast, but works around a bug where make_chroot
     --fast sometimes merges broken packages.
  2) Only set PORTAGE_LOCKS if it's not already set. This allows users to
     override PORTAGE_LOCKS and request that it be enabled using
     PORTAGE_LOCKS=true
  3) Only add the no-env-update feature if PORTAGE_LOCKS=false. This feature
     is only needed for that case.

Long term fix is to patch PORTAGE_LOCKS feature to lock postinst. That'll be
tracked in a separate bug.

BUG=chromium-os:6750
TEST=Run make_chroot --fast. It works every time now, but is slower.
     build_packages && build_image run at same speed as before.

Change-Id: I12ab40671034e10cd2ffbba45281ca44718d1d2c

Review URL: http://codereview.chromium.org/3446006
2010-09-15 16:19:38 -07:00
David James
5ebed121aa Allow parallel_emerge to be used when network is unavailable.
When --getbinpkg is not used, skip looking on the remote server for
packages. This should allow people to use parallel_emerge with no
binary packages when the network is unavailable.

BUG=chromium-os:6684
TEST=Disconnect network and run ./build_packages --nousepkg

Change-Id: Id0e5d03aca30d85cb6990d977983f1e0655726ab

Review URL: http://codereview.chromium.org/3294022
2010-09-15 16:18:48 -07:00
Mandeep Singh Baines
90438ca854 cros_workon: WORKON_FILE should be owned by the user
BUG=none
TEST=Verified that WORKON_FILE is now owned by the user.

Change-Id: I32f05d5de5177756945b6f5bc037a2f738ffffe5

Review URL: http://codereview.chromium.org/3446002
2010-09-15 12:28:42 -07:00
Zdenek Behan
e9474900c6 cros_workon: redefine the concept of a "workon" package list to depend on the board
* Modified all workon listing functions to also look for keyword

* Added a fallback to list all workon ebuilds if keyword is not specified, which
is needed for cros_mark_all_as_stable, which does not differentiate between boards.

This, amongst other potential issues, resolves the case when it was possible to
start working on a package not keyworded for the given board, and making
build_packages fail unconditionally.

TEST=below
$ ./cros_workon list --all --board=x86-generic |wc -l
73
$ ./cros_workon list --all --host |wc -l
57
Looking at the lists rather than "|wc -l" looks correct
$ ./cros_mark_all_as_stable
^ Produces satisfactory result

BUG=6700

Change-Id: Ieee92a39febcef5fb95e59cf97b6e63281a7c750

Review URL: http://codereview.chromium.org/3400001
2010-09-15 11:33:54 -07:00
Ken Mixter
ef45f16aa2 Allow prepackaged to still be passed and make learn_board work again
Change-Id: I664858ff9b931479fe2fac89a24aa3e7fce8493b

BUG=
TEST=ran rrt without ==board or default board set

Review URL: http://codereview.chromium.org/3431004
2010-09-14 17:19:33 -07:00
Zdenek Behan
82e95c43e7 run_remote_tests: for cros_workon, always use prepackaged tests
* Fixes running tests different than from the main autotest repo

* Also provides a better fix for chrome tests

Change-Id: I803ba7caa5561e3001d5627219718f471e59bb19

BUG=
TEST=

Review URL: http://codereview.chromium.org/3448002
2010-09-14 15:11:39 -07:00
Dale Curtis
d0a2e12511 image_to_live was not providing a URL to the stateful_update script executed on the remote machine. Without the URL, stateful_update would default to reading /mnt/stateful_partition/etc/lsb-release for the URL, which is not setup by the script.
I've modified image_to_live to convert the dev server url into the proper form to access the stateful.image.gz and pass it along to stateful_update on the remote system.

Change-Id: I77971c83b7efef5f053fda7fcd2585ca73f3134a

BUG=6705
TEST=Imaged several machines using Dev Server.

Review URL: http://codereview.chromium.org/3417001
2010-09-14 13:03:11 -07:00
Mandeep Singh Baines
6c6000b2c2 loman: don't report error when adding an exact duplicate.
BUG=5787
TEST=Updated and re-ran unittests.

Change-Id: I399e8cacde103f16b749dd83c57d03bcdf2e9475

Review URL: http://codereview.chromium.org/3385001
2010-09-14 10:19:50 -07:00
Paul Stewart
03b6ebf7d4 Add client_prefix (string) and old_prefix (bool) command line options
cros_copy_upgrade_server can now start the devserver with either prefix.
This is to help the migration from Memento to ChromeOSUpdateEngine.

BUG=none
TEST=none (yet)

Review URL: http://codereview.chromium.org/3304013
2010-09-13 13:28:14 -07:00
Tom Wai-Hong Tam
10a980eeb6 Reflect the change of QualDbs location moved.
Reflect the change of https://chromereview.prom.corp.google.com/1638035

TEST=manaul
 - image_to_usb --factory
 - run factory test
   -- match HWID
   -- write GBB
   -- reboot
   -- check HW components

Review URL: http://codereview.chromium.org/3290021

Change-Id: Id97153fe1d06917ac13d1470866c17d5e4acdc9d
2010-09-14 02:52:26 +08:00
Nikita Kostylev
0ae1505d89 Disable OEM partition mounting for test images.
BUG=chromium-os:3828
TEST=Boot with test image. Check that OEM partiotion is not mounted at /mnt/partner_partition

Review URL: http://codereview.chromium.org/3355019

Change-Id: Ib7c15010df7d7f30e49149c42034b26a13d69573
2010-09-13 18:58:31 +04:00
David James
cc19005e15 Fix rebuild blacklist so that icedtea isn't rebuilt unnecessarily.
It turns out that my previous rebuild_blacklist code had a bug where
blacklisted packages would get rebuilt unnecessarily. This would result
in slow builds if icedtea was built.

BUG=chromium-os:6628
TEST=Mock case where prebuilts for icedtea are older than packages that
     icedtea depends on.

Change-Id: I498b6499b78ab065109c1fe0e77969028ceff6cb

Review URL: http://codereview.chromium.org/3308027
2010-09-10 18:07:55 -07:00
Tan Gao
012bff9e81 Add Python tests for build_image script
Change-Id: I95bb4518bbfa2931407a90180686845fd9e928e5

BUG=chromium-os:6574
TEST=manually run test script and verified all 4 test cases passed

Review URL: http://codereview.chromium.org/3341024
2010-09-10 16:31:26 -07:00
Tan Gao
cf4eb5b38c Issue 6600: RunCommand should return an object with various attributes
Change-Id: I6b9333fcf1325dbe141ad387f52d85781de96819

BUG=chromium-os:6600
TEST=added 8 test cases for RunCommand (up from zero :-) Ran all tests and
verified all passed. No regression either

Review URL: http://codereview.chromium.org/3307022
2010-09-10 14:50:47 -07:00
Zelidrag Hornung
90a850ac11 Fixed run_remote_test.sh to work properly with the new location of chrome tests
BUG=none
TEST=none

Review URL: http://codereview.chromium.org/3341028

Change-Id: I81a8d57b16e7789b0fd6673d9e3b208ce369a575
2010-09-10 14:25:59 -07:00
Kenneth Waters
cebafd2ec3 image_to_usb: Add an option to install to the USB stick.
- This is needed for devices which use a USB stick or MMC card as their
  primary storage.

BUG=1150
TEST=Installed an image to a MMC card, and copied an image to a MMC card.

Review URL: http://codereview.chromium.org/3337020

Change-Id: Ieb7f7cf7b8f9c608a23c55176c49b6805e08d382
2010-09-10 11:05:26 -07:00
Mandeep Singh Baines
865a42edee loman: tool for updating local_manifest (initial commit)
BUG=5787
TEST=Ran unittests.

Change-Id: I6b8cc37eb2bf5f81f803f61b886106b6f08f3315

Review URL: http://codereview.chromium.org/3331013
2010-09-10 11:04:17 -07:00
Paul Stewart
12973e562d Install make.globals and create make.profile directory
Some mod_for_test images don't have the files necessary to run
emerge.  Install them.

BUG=none
TEST=Rerun package install

Review URL: http://codereview.chromium.org/3351016
2010-09-09 20:04:11 -07:00
Anush Elangovan
bc2aaf67f8 Make factory emerge the virtual/kernel atom to support profile based kernel
This is dependent on http://codereview.chromium.org/3317012/show

Change-Id: Ifb6e7511619e9f2000448408dd800dd7043a7783

BUG=5847
TEST=new setupboard and build_packages/image for arm/x86-generic with the CL 3317012 included

Review URL: http://codereview.chromium.org/3328016
2010-09-09 19:40:30 -07:00
Anush Elangovan
050c95cab7 Sample chromite spec
Change-Id: I049b31c2fc7369fa58ca43e24c2015e574231d7f

BUG=
TEST=none

Review URL: http://codereview.chromium.org/3369009
2010-09-09 15:45:28 -07:00
Mandeep Singh Baines
4903d4354e ignore: tmp files and .pyc files
BUG=none
TEST=none

Change-Id: I936c60226e75a7f0204331da7ab32b4d9fb0bd3f

Review URL: http://codereview.chromium.org/3316004
2010-09-09 14:25:33 -07:00
Tan Gao
74a07cac15 Issue 6577: remove confirmation to delete output directory
Change-Id: Ib19986121a8988c6cae23527148a7b5d4a58663e

BUG=chromium-os:6577
TEST=(inside chroot) run 4 test cases from src/scripts/, using Ctrl+C to force abort (hence triggering "delete_prompt")
a.) (stdin tty) "./build_image --board=x86-generic < /dev/null", expected = no prompt and delete output dir; actual == expected
b.) (stdout tty) "./build_image --board=x86-generic > foo.txt", expected = no prompt and delete output dir; actual == expected
c.) (normal user case) "./build_image --board=x86-generic",
expected = prompt to delete output dir, if y, output dir is removed; actual == expected
d.) (normal user case) "./build_image --board=x86-generic", expected = prompt to delete output dir, if N, output dir is NOT removed; actual == expected

Review URL: http://codereview.chromium.org/3373003
2010-09-09 11:41:30 -07:00
David James
2e79fe48ce Add shell parameter to allow running commands directly.
For chromite, we often want to run shell commands directly instead of using the
array syntax currently accepted by RunCommand. This makes it easier to run
piped commands, for example.

BUG=chromium-os:6573
TEST=I'm using it in cros_build.

Change-Id: Ic846e257fec4e423b0253d06641eac232545f6f6

Review URL: http://codereview.chromium.org/3323019
2010-09-09 10:49:11 -07:00
Nick Sanders
b3d04a0529 Filter more stuff from shim
Change-Id: I23ee03198acb7a792310799d7a72f8a19ca5445b

Review URL: http://codereview.chromium.org/3331021
2010-09-09 00:01:24 -07:00
Scott Zawalski
57729c3615 Add ListFiles a function to recursively list files in a directory.
Add unittest for ListFiles to cover common use cases.

BUG=
TEST=

Review URL: http://codereview.chromium.org/3325017
2010-09-08 15:59:23 -07:00
Chris Sosa
57e9808254 Fix keywords issue
Change-Id: If89d91624d4e83d8f6923e873bfb57b95fa3b666

BUG=
TEST=Ran with failing packages from buildbot.

Review URL: http://codereview.chromium.org/3330013
2010-09-08 15:03:06 -07:00
Zdenek Behan
ded5074afd cros_mark_all_as_stable: mark packages for all architectures, always
Change-Id: I415f089bc253c1ff9919451cf62c11a27a0bcf6d

BUG=6507
TEST=run cros_mark_all_as_stable and see the three previously ignored packages being uprevved

Review URL: http://codereview.chromium.org/3372002
2010-09-08 14:14:58 -07:00
Scott Zawalski
92097e69a6 Copy cros_build_lib.py to chromite/lib for chromite development
TBR=sosa
BUG=
TEST=

Review URL: http://codereview.chromium.org/3329015
2010-09-08 12:47:28 -07:00
David James
0ac3ac5aa5 Fix cycle cracking once and for all.
While testing parallel_emerge, I found more cases where it would fail
to crack cycles. The issue is that parallel_emerge simply cracks the first
cycle it finds that involves an edge, rather than all cycles that involve an
edge.

Finding all cycles that involve an edge without looping forever is a rather
difficult problem, so I've updated the algorithm to simply keep searching
until it runs out of cycles. Testing this, I haven't found any cases where
the performance of the cycle cracking is a problem. I updated the algorithm
to log its performance stats so we can track this.

TEST=emerge -ep portage
BUG=none

Change-Id: I1cb95ffe9d977b9f8d38626d2d6cdbb766c14669

Review URL: http://codereview.chromium.org/3340010
2010-09-07 16:57:56 -07:00
David James
ffede3b414 Use --usepkgonly in build_image to ensure no accidental builds of packages.
Historically, we used --usepkgonly in build_image to ensure that packages
were not accidentally built during this step. Recently, build_image was
updated to use --usepkg. The benefit of using --usepkg was that it would not
install obsolete binary packages to your image. It was counter-intuitive that
--usepkgonly would install packages to your image that are not actually
installed to your board root. The disadvantage, however, of --usepkg, was that
it sometimes tried to build packages afresh using the image as your build root,
and this failed with strange errors because you're not supposed to do that.

This change fixes build_image to give useful errors by switching back to
--usepkgonly. To fix the problem where --usepkgonly installs packages that are
not installed, we first run eclean-<board> to clean all unused packages from
the board root.

This change is a big improvement because a number of people have run into
strange issues with build_image due to this problem and have had trouble
debugging them.

This change was actually written by Sean Paul, and I am shepherding his change
through for him because he doesn't have a Chromium account yet.

BUG=chromium-os:6437
TEST=
 1) build_packages && build_image should still work
 2) build_packages && ./cros_workon start power_manager && build_image should
    fail with an error that all versions of the power manager are masked. This
    happens because you started working on the power manager, but did not build
    the version of the power manager you were working on before installing it.
 3) Assuming you are working on the power manager, build_packages && build_image
    should succeed because your cros_workon'd version of the package is built.
 4) ( ./cros_workon start power_manager && build_packages &&
      ./cros_workon stop power_manager && build_image ) should fail with the
    same error message as case 2 for similar reasons.
 5) ( ./cros_workon start power_manager && build_packages &&
      ./cros_workon stop power_manager && build_packages && build_image )
    should work because you've built your new version of the stable package.

Change-Id: Ia3858c70997bc6f0ec0b6d1bfaede8d3272a0976

Review URL: http://codereview.chromium.org/3305010
2010-09-07 14:22:38 -07:00
Sameer Nanda
f7cf2ca225 Moved disable_idle_suspend to rootfs from stateful partition for test images.
BUG=chromium-os:6204
TEST=Ran mod_image_for_test.sh, mounted resulting image and verfied
     disable_idle_suspend now shows up under /usr/share/power_manager
     instead of /var/lib/power_manager

Change-Id: Ia34787b364cd5ffa199efc4a36b5d9d691a33c5a

Review URL: http://codereview.chromium.org/3326011
2010-09-07 09:34:12 -07:00
Jon Kliegman
03a4dad734 Whitespace/tab cleanup for cros_workon
Change-Id: I17379dc302fcad0f3a7f59771c48d86df320c0d0

BUG=NONE
TEST=Manual diff inspection.  Ran scripts and confirmed no syntax errors.

Review URL: http://codereview.chromium.org/3335004
2010-09-07 11:12:20 -04:00
vbendeb
68fd512d31 Include saft.conf in the list of scripts modified for test.
This change must accompany http://codereview.chromium.org/3330005/show

Change-Id: If57a726933e4a243e424bdc98ddd577081afe6e2

BUG=chromium-os:6345
TEST=manual
built an image, ran mod_image_for_test on it, then
split the image into partition blobs, mounted part_3 and
checked /etc/init/{openssh-server,iptables,saft}.conf
- they all have the #for_test string removed.
removed.

Review URL: http://codereview.chromium.org/3309006
2010-09-03 18:30:47 -07:00
Nick Sanders
446c2115e4 Don't kill sdcards with boot_tries
USB/SDCard boot devices typicaly don't run software updater,
and won't have success ever set. We never want to try to
fall back to the empty B partition either. The workaround
for this is to set the successful bit regardless of actual
boot success.

bug=6395
Change-Id: I67c625804203b13be9a0c626c404fa38bafb5445

Review URL: http://codereview.chromium.org/3344008
2010-09-03 18:15:35 -07:00
Zdenek Behan
6fb7a3680c cros_workon: print out short summary of what has been done for start/stop
BUG=6325
TEST=cros_workon start and stop for board and host and see it work

Change-Id: Ic0680a273391fdf93b6ebbe0e34497807f31f240

Review URL: http://codereview.chromium.org/3352002
2010-09-03 17:29:09 -07:00
David James
77076b4144 Fix typo in parallel_emerge which was breaking updates.
I put portage when I meant to put portage_pkg, so it was running
code unconditionally that should have only ran when portage was
being upgraded. Oops!

We also should clear the dependency list, since we don't want to
list dependencies other than portage when we're installing portage.

TBR=zbehan
TEST=parallel_emerge
BUG=none

Change-Id: Ic2861d78913f89fc42cedd96ffb125e08287f662

Review URL: http://codereview.chromium.org/3318009
2010-09-02 16:51:03 -07:00
David James
a164ad46dd Update parallel_emerge to restart after upgrading portage in root.
We had a build failure today because portage was upgraded, and
parallel_emerge did not restart after emerging portage. This resulted
in build breakage because parallel_emerge did not pick up all of the
changes (only some of them).

BUG=chromium-os:6394
TEST=Tested that parallel_emerge restarts after upgrading portage in root,
     but not after upgrading portage in board.

Change-Id: I480dc2dc034afb5b9e0468ed6e4d16de608d3d32

Review URL: http://codereview.chromium.org/3304010
2010-09-02 15:13:56 -07:00