Commit Graph

28179 Commits

Author SHA1 Message Date
Raymes Khoury
b1d37f74db Altered sync_build_test so that when --nousepkg is specified, it is also passed to setup_board.
Review URL: http://codereview.chromium.org/3041033
2010-08-02 14:53:03 -07:00
Raymes Khoury
d868edc220 Updated sync_build_test to use new unit test script.
I believe it only makes sense to run this for an x86 target but I might be wrong.

Review URL: http://codereview.chromium.org/3082008
2010-08-02 14:49:05 -07:00
Scott Zawalski
ec6421c21a Currently we pass in the path that is outside of the chroot to a script that runs inside of the chroot. We need to make sure we pass the path that is in the chroot to the mod_image_for_recovery.sh.
Review URL: http://codereview.chromium.org/3014048
2010-08-02 12:31:35 -07:00
Mandeep Singh Baines
52ed3f969a cros_mark_as_stable: add a --srcroot command-line option
This is required to enable running cros_mark_as_stable from outside
the chroot.

BUG=5258
TEST=Successfully ran outside the chroot and pushed a change.
All unit tests continue to pass.

Change-Id: Ibd23ace6326b8453c132c416d6db6e42c8c2c239

Review URL: http://codereview.chromium.org/2884060
2010-08-02 11:48:52 -07:00
Paul Stewart
e9e9ce49f9 Add independence to location of cros common.sh (will it move to bin/?)
BUG=none
TEST=reran script

Review URL: http://codereview.chromium.org/3064025
2010-07-30 17:50:16 -07:00
Mandeep Singh Baines
9f8e53b27b cros_mark_as_stable: fix unittests
Code was changed to add a CR. Fix tests to accomodate.

BUG=5258
TEST=Unittests pass again.

Change-Id: Iacdbffb14d14f60a43883db3ee12b99a5eeb9f65

Review URL: http://codereview.chromium.org/3014045
2010-07-30 16:37:45 -07:00
Mandeep Singh Baines
204f44d557 cros_mark_as_stable: fix unittests
BUG=5258
TEST=Unittests now pass.

Change-Id: Ie29a222a2ebe691e251abe21a1b74bd3d48fbfd8

Review URL: http://codereview.chromium.org/3029045
2010-07-30 16:22:18 -07:00
David James
265588e4f4 Add newline after CROS_WORKON_COMMIT="" line.
Currently, cros_mark_as_stable.py does not add a newline after its
CROS_WORKON_COMMIT="" line. This works fine for 9999 ebuilds that have a newline
after the EAPI="" line, but doesn't work for ebuilds that don't do this.

This should fix the preflight queue build.

TEST=Ran cros_mark_as_stable.py on the update engine and verified that the
     CROS_WORKON_COMMIT="" line was not joined together with the next line.
BUG=none

Review URL: http://codereview.chromium.org/3036029
2010-07-30 10:58:40 -07:00
David James
4a046b6ccb Actually disable collision-protect in parallel_emerge.
I tried to disable parallel_emerge yesterday, but my change didn't work,
so the buildbots are still running into occasional related issues.

Portage has two flags for doing collision protection: collision-protect
and protect-owned. The protect-owned feature is enabled by default and
is quite useful: it checks to make sure that we don't have multiple
packages that own the same file. The collision-protect feature is more
strict, and less useful: it fails if it finds a conflicting file, even
if that file was created by an earlier ebuild that failed to install.

We want to disable collision-protect here because we don't handle
failures during the merge step very well -- we just leave the old
Unfortunately, we haven't quite figure out the best way to handle
these failures in parallel emerge, so for now we disable the flag.

TEST=Created a fake collision, made sure that packages still merge.
BUG=none

Review URL: http://codereview.chromium.org/2825076
2010-07-29 23:26:02 -07:00
David McMahon
7f2951d993 Add do_recovery_mod() to create recovery image
Review URL: http://codereview.chromium.org/3020037
2010-07-29 14:37:06 -07:00
Zdenek Behan
2654f24e1b cros_workon_common: fix a last minute rename :/
* Let's face it: I broke it in the last second when i renamed the function but not
the place where it's being called, and now cros_workon is dead.

	modified:   lib/cros_workon_common.sh

Review URL: http://codereview.chromium.org/3052025
2010-07-29 14:23:36 -07:00
Zdenek Behan
e6a925894c cros-workon: extend show_workon_ebuilds to give full paths, provide wrappers
This will allow replacement of equery which in cros-workon

Also other ebuild-related operations like grouping workon ebuilds with the same repo

* Find for all ebuilds takes almost no time when cached and up to ~2 secs when not
* equery takes ages just to load

TEST=cros_workon_ebuilds before and after, and diff:
--- list1       2010-07-27 21:22:23.000000000 -0700
+++ list2       2010-07-27 21:22:32.000000000 -0700
@@ -1,6 +1,5 @@
 app-crypt/tpm-emulator
 app-crypt/trousers
-app-i18n/ibus
 app-i18n/ibus-chewing
 app-i18n/ibus-hangul
 app-i18n/ibus-m17n
(ibus is a mistake, not a workon package, i'll fix that in a separate CL)

	modified:   lib/cros_workon_common.sh

Review URL: http://codereview.chromium.org/2808072
2010-07-29 13:26:53 -07:00
Zelidrag Hornung
151112ce00 Add checkpoints to archive_build
Review URL: http://codereview.chromium.org/2806088
2010-07-29 10:32:26 -07:00
David James
de456309c8 Disable collision-protect in parallel_emerge.
collision-protect doesn't make sense for parallel_emerge, because
we don't lock around merges. Also, if merge fails part-way through,
it's strange to fail just because some files are lying around.
This fixes an issue with the buildbots.

TEST=Checked that collision-protect feature gets deleted by this.
BUG=none

Review URL: http://codereview.chromium.org/3061029
2010-07-29 01:01:15 -07:00
David James
0366864a98 Enable fast build by default in build_image and mod_image_for_test.sh.
The fast build is stable enough now that we can enable it for everybody.
I ran the build constantly all weekend on two machines and I only ran into
one (rare) error, which I've fixed in a separate patch.

See <http://codereview.chromium.org/2856048/show>

TEST=Ran lots and lots of fast builds
BUG=none

Review URL: http://codereview.chromium.org/3059001
2010-07-28 17:08:29 -07:00
Olof Johansson
8ce8312e43 parallel_emerge: Fix builds when PORTAGE_BINHOST is unset
BUG=none
TEST=Try building for a branch new board overlay that doesn't already have a binhost setup.

Review URL: http://codereview.chromium.org/2870068
2010-07-28 17:20:16 -05:00
Chris Sosa
ffff7d76cc Add attempt to update remote and rebase before pushing
TEST=Ran cros_mark_all

Review URL: http://codereview.chromium.org/3066010
2010-07-28 13:21:32 -07:00
Anush Elangovan
9df55b2055 Add repo local manifest support 2010-07-28 12:07:48 -07:00
Zelidrag Hornung
e6532bc3dd Added output to tar command so it does not cause timeout to kick in.
Review URL: http://codereview.chromium.org/3076012
2010-07-28 11:34:29 -07:00
Nikita Kostylev
c8bba90ae4 Make oem_customization flag optional.
BUG=chromium-os:1888
TEST=Manual. build_image should run fine when oem_customization flag is not specified.

Review URL: http://codereview.chromium.org/3076009
2010-07-28 17:05:26 +04:00
Zelidrag Hornung
e93c60e9a9 Removed chrome ui tests and its dependencies from the factory test image
BUG=4828
TEST=make sure factory image actually builds while chrome ui autotests are included in the build

Review URL: http://codereview.chromium.org/3010035
2010-07-27 16:06:20 -07:00
Elly Jones
c5fe33d940 Revert "Copy multiple qualified DBs to the image based on the board name."
This commit breaks the build for x86-full-fast-generic at least. It causes a
file to be copied to itself during the build archive step:
http://build.chromium.org/buildbot/chromiumos/builders/x86%20generic%20full%20fast/builds/128/steps/archive_build/logs/stdio

TEST=None
BUG=None

This reverts commit 1cf4329681.

Signed-Off-By: Elly Jones <ellyjones@chromium.org>
Signed-Off-By: Nathan Williams <njw@chromium.org>

Review URL: http://codereview.chromium.org/2808069
2010-07-27 10:40:10 -04:00
Tom Wai-Hong Tam
1cf4329681 Copy multiple qualified DBs to the image based on the board name.
The DBs are named in the format of qualified_components_${BOARD}_${HWQUALID}.
If only one hwqual id for the board, named in qualified_components_${BOARD}.

Review URL: http://codereview.chromium.org/3038026
2010-07-27 17:02:35 +08:00
Zdenek Behan
ddb0f2c8e3 autotest: create a candidate script for replacement ebuild test (running the tests)
* This script should replace the call to ebuild in autotest wrapper, and essentially
duplicates all the test running functions from autotest-0.0.1.ebuild

* duplicate autotest wrapper into autotest_workon to separate conversion and old functionality

* Add a hack into run_remote_tests to allow using autotest_workon instead

	new file:   autotest_run.sh
        new file:   autotest_workon
        modified:   run_remote_tests.sh

Review URL: http://codereview.chromium.org/2854062
2010-07-26 18:02:29 -07:00
Will Drewry
efce66880f image_to_vm: convert to calling setimage directly
The last changes to the installer broke image_to_vm.  Since image_to_vm
doesn't need to run postinst, but just change the active image, it
now just calls chromeos-setimage and passes in its needed flags (already
supported by the last installer change).  It will also detect whether
the image was built with verification of the rootfs enabled or not
by looking at the default.cfg file.

Also updates the location of where you should run the command in
build_image.

TEST=built a new image with --enable_rootfs_verification and started with qemu -curses -hda [output]; did the same without verification
BUG=chromium-os:2963

Review URL: http://codereview.chromium.org/3034030

Change-Id: I265d6ad8971f9427b78cc07d784fced9cceb5974
2010-07-23 19:43:27 -05:00
Bill Richardson
8bfa4685df Add "kern_guid=%U" to the secure boot kernel command-line template.
With the newest Chrome OS BIOS and bootstub, this will be expanded to the
booted kernel partition's UniqueGuid, so that the kernel device can be
determined with certainty, since the BIOS and kernel may enumerate drives
differently.

You can identify the booted kernel partition at runtime with something like
this:

  sudo cgpt find -1 -u \
    $(cat /proc/cmdline | sed 's/.*kern_guid=\([0-9a-f-]\+\).*/\1/')

Review URL: http://codereview.chromium.org/3035020
2010-07-23 17:24:15 -07:00
Chris Sosa
80a3b2810a Add flag to cros_mark_as_stable to specify tracking_branch.
TEST=Ran using default option and then tried -t chromiumorg/master and
saw a correct failure since I don't have that branch to track against.

Review URL: http://codereview.chromium.org/3032019
2010-07-23 17:00:34 -07:00
Chris Sosa
b28538692c Need $ on same line to not eval hash as command in bash
TEST=Re-ran script and saw new changes.

Review URL: http://codereview.chromium.org/3032021
2010-07-23 16:59:50 -07:00
Nick Sanders
409703715b The current rekeying command is probably too complicated for ODMs to be happy with. I've added a script that wraps this.
I'd be OK just setting the defaults in the original command as well.

BUG=4246
TEST=ran it and image booted in normal mode.

Review URL: http://codereview.chromium.org/3008021
2010-07-23 16:55:28 -07:00
Will Drewry
4f0a7f6ef1 create_legacy_bootloader_templates: re-enable chromeos-vusb with --enable-rootfs_verification
This change can land once the kernel support for dm device waiting and
the chromeos-installer changes land and their ebuilds are rev'd.

TEST=build_image --enable_rootfs_verification; booted to a verified image!
BUG=chromium-os:2963

Change-Id: Ia68f90a59ab1360da01d5f422c16178af16cbaeb

Review URL: http://codereview.chromium.org/3013028
2010-07-23 12:33:57 -05:00
Olof Johansson
e07324f53c don't use qemu on x86 for mod_image_for_test
BUG=none
TEST=run mod_image_for_test on x86. Watch it fail. D'oh.

Review URL: http://codereview.chromium.org/3048017
2010-07-22 22:38:36 -05:00
Justin Neddo
6b5b16b8fe Generate fake root certs for autotest via qemu
mod_image_for_test.sh now passes the target architecture to the subscripts
mod_for_test_scripts/710enableAuthTesting generates the /etc/fake_root_cert
databases by running nsscertutil under QEMU.

BUG=3310
TEST=Built autotest enabled images for x86 and arm.  On arm, the browser no
longer crashes trying to read the fake root certs.

Review URL: http://codereview.chromium.org/2878033
2010-07-22 10:17:03 -05:00
Antoine Labour
2a58d0e610 fix toolchain location when using variants
BUG=None
TEST=run setup_board with a variant, run build_packages --fast

Review URL: http://codereview.chromium.org/3046009
2010-07-21 15:28:31 -07:00
Tom Wai-Hong Tam
a1feb99412 TBR: Issue 2861071: Fix name changed of init script dump-boot-stats.conf to boot-complete.conf.
A file name changed which breaks the factory test phase on the builders.

This change was LGTM'd but not submitted.
Patch from Tom Wai-Hong Tam <waihong@chromium.org>.

Change-Id: I961515ba4667891d82833035f2d312a653f746fd
2010-07-21 16:37:25 -05:00
David James
2be3f829e9 Fix infinite recursion in PrebuiltsReady when graph is cyclic.
Our package graph is cyclic, so parallel_emerge needs to handle cycles
correctly in all cases. PrebuiltsReady should only need to check each package
once, so we should set cache[pkg] to True if we found the package in the cache.

TEST=Ran build_packages --fast
BUG=none

Review URL: http://codereview.chromium.org/3047009
2010-07-21 14:17:05 -07:00
Will Drewry
9926ee2879 build_image: avoid losetup -c because older kernels seem to balk
Instead of resizing the loop device after adding padding for the
hash tree data.  Just ensure that the mkfs.ext3 call doesn't exceed
the rootfs size.

TEST=reran build_image on my lucid machine and checked the rootfs size; asked someone with hardy machine to test.
BUG=none

Change-Id: I59f95e1d17e35aca265bd44bb863da6069c05bd2

Review URL: http://codereview.chromium.org/3048009
2010-07-21 15:11:10 -05:00
Chris Sosa
a48a37af11 Remove support for old way of building and running unit tests
Review URL: http://codereview.chromium.org/3041012
2010-07-21 12:40:04 -07:00
Chris Sosa
2566c3fc3c Adding --desc_from_logs option from git cl upload
TEST=Tested git cl upload --desc_from_logs -m "Message" in http://codereview.chromium.org/3027010/show

Review URL: http://codereview.chromium.org/3048005
2010-07-21 12:19:08 -07:00
Will Drewry
78992a33f4 build_image, build_kernel_image, update_bootloaders: fix up rootfs_verification
This change adds
- --rootfs_hash_pad to specify the MBs reserved for the pad
- the implementation of the above flag
- check if total fs size + pad size exceeds the partition size
- hash appending in make_image_bootable()

Fixes:
- a style for ROOT_FS_HASH usage
- bad mount|grep
- bad bash subst for root devices in all boot paths
- fixed a typo in the update_bootloaders table creation
- disables verified usb for now

Adding the padding argument ensures that the generated hash tree for the root filesystem is appended to the image.  Assuming the rootfs is _never_ mounted read-write
again, that hash tree will be valid and vboot will be able to proceed.

BUG=chromium-os:2693
TEST=manual build_image

Review URL: http://codereview.chromium.org/3043011

Change-Id: I67d9b0f91cacdefa309c0cc2dd7fed1d2eddd7a7
2010-07-21 14:02:20 -05:00
Rahul Chaturvedi
e79e88eb0e Changes to use the postinst --esp-part-file flag; postinst fixups removed
Added the flags required for mounted images to the postinst call.
Scripts no longer need to fixup postinst hence that code is removed.

Review URL: http://codereview.chromium.org/3027012
2010-07-21 02:04:59 +05:30
David James
dd059edd58 Integrate parallel_emerge with emerge, boosting performance.
Instead of loosely wrapping emerge, parallel_emerge now integrates tightly with emerge. This boosts performance while allowing us to map dependencies more accurately.

With this change, build_image --fast can clock under 4 minutes, and build_packages --fast now clocks as low as 4:30. With the --rebuild option, build_packages takes 5:30, but it has the big benefit of producing actually-correct results.

Note that this change also depends on us updating the various build scripts to prefix
calls to parallel_emerge with sudo -E.

TEST=Ran several parallel_emerge test cases, build_packages --fast, and
     build_image --fast
BUG=none

Review URL: http://codereview.chromium.org/2891013
2010-07-20 11:21:23 -07:00
David James
b44b04d0f9 Update mod_image_for_test to support --fast option.
Added --fast option. Updated the one place that wasn't using sudo when calling
emerge to use sudo like the others. This is safe because we're merging with
--usepkgonly.

TEST=Ran mod_image_for_test.sh with and without the --fast options
BUG=none

Review URL: http://codereview.chromium.org/2834055
2010-07-20 00:36:32 -07:00
David McMahon
21fd22e1b1 Created branch 0.8.60.B. Update CHROMEOS_VERSION_BRANCH=61
Change-Id: I724a2f805f237ef4b7e102b6011603dbe7450462
2010-07-19 17:22:54 -07:00
David McMahon
ce4eccfbc6 Created branch 0.8.58.B. Update CHROMEOS_VERSION_BRANCH=59
Change-Id: Iec03956393563287d742a9ebc65998589c8b3a88
2010-07-19 15:01:56 -07:00
Chris Sosa
db51f31139 Add wrapper script to get list of updates to pass to stablizing script without other changes
TEST=Ran script

Review URL: http://codereview.chromium.org/3034011
2010-07-19 11:57:31 -07:00
Chris Sosa
a5e394704d Create common script file for workon tools. Planning on using in another CL.
Review URL: http://codereview.chromium.org/3022010
2010-07-19 11:50:11 -07:00
Chris Sosa
92d3779dff Return error when error occurs by raising exception
Review URL: http://codereview.chromium.org/2819052
2010-07-19 11:12:20 -07:00
Paul Stewart
90f689c2d3 Pack and unpack scripts switched to bash
The "[[ ... ]]" syntax is bash-specific.  Reflect this by changing shbang to "#!/bin/bash".  Many of our dev systems symlink "/bin/sh" to "/bin/bash" which has masked this problem for a long time, but "real" sh fails on "[[".

BUG=none
TEST=rerun

Review URL: http://codereview.chromium.org/2825050
2010-07-19 10:37:06 -07:00
Tan Gao
7ead6c4fa1 issue 2825046
Change-Id: I8eb21d76e75f97270596907fb309b81e8b6c8d2e

Review URL: http://codereview.chromium.org/3007005
2010-07-19 10:12:37 -07:00
Tom Wai-Hong Tam
ab3682d570 If qualified components not specified, use the one based on board name.
Review URL: http://codereview.chromium.org/2965013
2010-07-19 00:22:42 +08:00