Commit Graph

2152 Commits

Author SHA1 Message Date
Will Drewry
7ab698d713 build_image, mod_image_for_*: break out make_image_bootable
Breaks make_image_bootable out of build_image into a separate
script.  In addition, it make a helper .sh in the OUTPUT_DIR
to let make_image_bootable be re-run on an image with the same
arguments that were passed in via build_image.

This change also removes the use of the boot/ directory in
OUTPUT_DIR.

TEST=build_image verified; image_to_usb --test_image; booted l13
           build_image; image_to_vm; qemu booted
           build_image verified; image_to_usb; booted l13
BUG=chromium-os:5081

Review URL: http://codereview.chromium.org/3066037

Change-Id: I627d678089aa71c353347f387ad5b14063fd4e7b
2010-08-05 13:57:52 -05:00
Mandeep Singh Baines
6d523e1a56 cros_mark_as_stable: modify push to use tracking branch
BUG=5258
TEST=Successfully pushed.

Change-Id: I985da7577e88f4a734f8281cae4aafce8dda5933

Review URL: http://codereview.chromium.org/3046051
2010-08-05 11:39:03 -07:00
J. Richard Barnette
138904ef93 Fix gsdcurl to correctly URL-quote a password passed via GSDCURL_PASSWORD
BUG=none
TEST=use this script with a (correct) password that includes punctuation such as one of these: $&+,/:;=?@

Review URL: http://codereview.chromium.org/3058046
2010-08-05 11:03:37 -07:00
Mandeep Singh Baines
642382095d cros_mark_as_stable: make it work better in repo
Instead of using master, use the tracking branch.

BUG=5258
TEST=Verified that cleanup now works with repo.

Change-Id: I6e415d9068d54b602238b89c77ba4dbe2f9ff4c6

Review URL: http://codereview.chromium.org/3076035
2010-08-05 11:01:36 -07:00
Mandeep Singh Baines
92e674967d Add gflags.py
This is a temporary hack to allow cros_mark_as_stable to work outside
the chroot. Eventually, we'll re-write the code to use optparse.

BUG=5412
TEST=Verified the cros_mark_as_stable now works outside the chroot.

Change-Id: Ife1f7aeadb70ba004d3928a9c71d6e95c042833d

Review URL: http://codereview.chromium.org/3037046
2010-08-05 10:37:17 -07:00
Mandeep Singh Baines
b4b6a260f2 cros_mark_all_as_stable: add tracking branch support
The tracking branch is different between repo and gclient.

BUG=5258
TEST=Verifed the tracking branching option is correctly propagated.

Change-Id: I9e5b8cb68d335c47e47499b4964dcaa83ac26f76

Review URL: http://codereview.chromium.org/3037045
2010-08-05 10:36:52 -07:00
Tom Wai-Hong Tam
2272ae5ce2 Handle Qual DBs based on default board name.
All Qual DBs may be existed in the same directory of hardware_Components (overlaid internal git). So, if QUALDB is not specified, first check the existence of DBs matched the board name. Then remove other DBs not belonging to the board.

TEST=build_packages, build_image, image_to_usb

Review URL: http://codereview.chromium.org/3032036
2010-08-05 14:19:45 +08:00
Olof Johansson
58f25cca72 Create a latest link to point at newest image
build_image creates it after a successful build, and get_latest_image
can use it if it exists. Fall back to old behavior for now when there
are directories without a link.

BUG=none
TEST=Try building an image, check link. Rebuild, check new link.

Review URL: http://codereview.chromium.org/3014051
2010-08-04 21:18:49 -07:00
David James
77e8092a9c Update parallel_emerge to use no-env-update feature which skips env_update.
Currently, our build occasionally fails with errors about about symlinks
already existing. This usually only slows down the build, because we have
retries, but it would be good to fix the problem.

This patch updates parallel_emerge to use the no-env-update feature for cross-
compiles. This feature skips the env_update step, which isn't thread-safe, and
isn't necessary for when we're cross-compiling and don't actually need to run
the binaries. Instead we run the env_update at the end, avoiding a thread-safety
issue.

TEST=Ran full build --nousepkg with env_update occuring at end.
BUG=chromium-os:5382

Review URL: http://codereview.chromium.org/3034051
2010-08-04 17:26:28 -07:00
David James
56e08b2424 Fix --onlydeps flag.
When --onlydeps is specified, we don't want to merge packages that were
specified on the command-line. Previously, it threw an exception in this
case because it couldn't find the command-line packages in the list of
installed packages.

TEST=Run ./parallel_emerge --onlydeps autotest, and make sure it
     doesn't crash.
     Run ./parallel_emerge --board=x86-generic  -po chromeos power_manager and make sure it does emerge power_manager, and doesn't crash.
BUG=none

Review URL: http://codereview.chromium.org/2868094
2010-08-04 17:21:12 -07:00
Ken Mixter
1365cf79ba Don't build_autotests separately since they're built on demand while the image is tested.
Review URL: http://codereview.chromium.org/2856056
2010-08-04 16:53:52 -07:00
bradnelson
2a0d4818f6 Dropping useless comment. Using getuser which is more correct.
Review URL: http://codereview.chromium.org/3005055
2010-08-04 13:15:23 -07:00
Tan Gao
6adaea74ae add script to mod dev install shim to create a dev recovery image
Change-Id: I1333cef94e5157206a464d9505de42707ead10a4

Review URL: http://codereview.chromium.org/3083007
2010-08-04 11:03:35 -07:00
bradnelson
825074abc0 Making default to LOGNAME work correctly.
Modifying gsdcurl to accept username and password from the environment. Switching cros_download_latest_image to use gsdcurl

Merge branch 'master' of ssh://chromiumos-git/crosutils

Switching back to MixedCase. Wrong style guide...

Switching to match external style guide.

Dropping unneeded arg as per review.

Fixing copyright.

Making the prompts go to stderr. Renaming for tab completion.

Adding gsdcurl utility for pulling for Google Storage for Developers.

BUG=None
TEST=None

Review URL: http://codereview.chromium.org/3060043
2010-08-03 17:30:53 -07:00
Tan Gao
97117a9d97 remove obsolete arg kern_img from install_gpt() call
Change-Id: I62a7e85f098327404f52752d85450ff24f2a7e78

Review URL: http://codereview.chromium.org/3014050
2010-08-03 17:01:25 -07:00
David McMahon
9d4984388f Created branch 0.8.62.B. Update CHROMEOS_VERSION_BRANCH=63
Change-Id: I9d9825ea3b414a2fbc93324e6a5f18b5dbbb57be
2010-08-03 16:42:10 -07:00
robotboy
bd06106e80 Add checks that the board and variant names don't contain underscores.
Review URL: http://codereview.chromium.org/2878077
2010-08-03 13:31:52 -07:00
robotboy
465c39ab50 Move overlay_list to bin/cros_overlay_list.
This conforms to the new layout of scripts in which all scripts
other than make_chroot and enter_chroot are installed in the
chroot using an ebuild.

Review URL: http://codereview.chromium.org/3084012
2010-08-03 12:42:58 -07:00
Anush Elangovan
56de1dd852 Check for system shflags
This change checks for and sources shflags from /usr/lib before
depending on the gclient-style path.

TEST=None
BUG=4230

Review URL: http://codereview.chromium.org/3010045

Change-Id: I3085c2356c40c691a2a72bf79c1d9167162d6dca
2010-08-02 23:56:07 -07:00
David James
0faf7eb18d Blacklist icedtea and o3d from automatic rebuilds.
The icedtea and o3d packages take a really long time to rebuild (over 30 mins
each), so we can give our developers a break if we exempt them from automatic
rebuilds.

TEST=Rev the portage ebuild or the fontconfig ebuild and run build_packages.
     Without the change, takes over 45 mins. With the change, it takes 5 mins.
BUG=none

Review URL: http://codereview.chromium.org/3029051
2010-08-02 21:34:12 -07:00
David James
98c52bdea2 Make the homedir check a bit simpler.
Rather than using split, we can just use os.path.basename(path) to get
the username from the homedir name. This is a bit simpler.

I've also added a comment which I missed in my last commit.

TEST=Ran ./parallel_emerge, verified the new method of getting
PORTAGE_USERNAME gets the right username.
BUG=none

Review URL: http://codereview.chromium.org/3056002
2010-08-02 21:21:17 -07:00
robotboy
f968285d63 Add overlay_list script.
This script generates the list of overlays, including variant
overlays, for a particular board/variant combination.  It will
be called from setup_board.

Review URL: http://codereview.chromium.org/3080019
2010-08-02 17:07:36 -07:00
Raymes Khoury
415d42d0de Added required BOARD environment variable when building chrome browser in sync_build_test.
Also minor indentation fix.

Review URL: http://codereview.chromium.org/3023031
2010-08-02 14:55:13 -07:00
Raymes Khoury
b1d37f74db Altered sync_build_test so that when --nousepkg is specified, it is also passed to setup_board.
Review URL: http://codereview.chromium.org/3041033
2010-08-02 14:53:03 -07:00
Raymes Khoury
d868edc220 Updated sync_build_test to use new unit test script.
I believe it only makes sense to run this for an x86 target but I might be wrong.

Review URL: http://codereview.chromium.org/3082008
2010-08-02 14:49:05 -07:00
Scott Zawalski
ec6421c21a Currently we pass in the path that is outside of the chroot to a script that runs inside of the chroot. We need to make sure we pass the path that is in the chroot to the mod_image_for_recovery.sh.
Review URL: http://codereview.chromium.org/3014048
2010-08-02 12:31:35 -07:00
Mandeep Singh Baines
52ed3f969a cros_mark_as_stable: add a --srcroot command-line option
This is required to enable running cros_mark_as_stable from outside
the chroot.

BUG=5258
TEST=Successfully ran outside the chroot and pushed a change.
All unit tests continue to pass.

Change-Id: Ibd23ace6326b8453c132c416d6db6e42c8c2c239

Review URL: http://codereview.chromium.org/2884060
2010-08-02 11:48:52 -07:00
Paul Stewart
e9e9ce49f9 Add independence to location of cros common.sh (will it move to bin/?)
BUG=none
TEST=reran script

Review URL: http://codereview.chromium.org/3064025
2010-07-30 17:50:16 -07:00
Mandeep Singh Baines
9f8e53b27b cros_mark_as_stable: fix unittests
Code was changed to add a CR. Fix tests to accomodate.

BUG=5258
TEST=Unittests pass again.

Change-Id: Iacdbffb14d14f60a43883db3ee12b99a5eeb9f65

Review URL: http://codereview.chromium.org/3014045
2010-07-30 16:37:45 -07:00
Mandeep Singh Baines
204f44d557 cros_mark_as_stable: fix unittests
BUG=5258
TEST=Unittests now pass.

Change-Id: Ie29a222a2ebe691e251abe21a1b74bd3d48fbfd8

Review URL: http://codereview.chromium.org/3029045
2010-07-30 16:22:18 -07:00
David James
265588e4f4 Add newline after CROS_WORKON_COMMIT="" line.
Currently, cros_mark_as_stable.py does not add a newline after its
CROS_WORKON_COMMIT="" line. This works fine for 9999 ebuilds that have a newline
after the EAPI="" line, but doesn't work for ebuilds that don't do this.

This should fix the preflight queue build.

TEST=Ran cros_mark_as_stable.py on the update engine and verified that the
     CROS_WORKON_COMMIT="" line was not joined together with the next line.
BUG=none

Review URL: http://codereview.chromium.org/3036029
2010-07-30 10:58:40 -07:00
David James
4a046b6ccb Actually disable collision-protect in parallel_emerge.
I tried to disable parallel_emerge yesterday, but my change didn't work,
so the buildbots are still running into occasional related issues.

Portage has two flags for doing collision protection: collision-protect
and protect-owned. The protect-owned feature is enabled by default and
is quite useful: it checks to make sure that we don't have multiple
packages that own the same file. The collision-protect feature is more
strict, and less useful: it fails if it finds a conflicting file, even
if that file was created by an earlier ebuild that failed to install.

We want to disable collision-protect here because we don't handle
failures during the merge step very well -- we just leave the old
Unfortunately, we haven't quite figure out the best way to handle
these failures in parallel emerge, so for now we disable the flag.

TEST=Created a fake collision, made sure that packages still merge.
BUG=none

Review URL: http://codereview.chromium.org/2825076
2010-07-29 23:26:02 -07:00
David McMahon
7f2951d993 Add do_recovery_mod() to create recovery image
Review URL: http://codereview.chromium.org/3020037
2010-07-29 14:37:06 -07:00
Zdenek Behan
2654f24e1b cros_workon_common: fix a last minute rename :/
* Let's face it: I broke it in the last second when i renamed the function but not
the place where it's being called, and now cros_workon is dead.

	modified:   lib/cros_workon_common.sh

Review URL: http://codereview.chromium.org/3052025
2010-07-29 14:23:36 -07:00
Zdenek Behan
e6a925894c cros-workon: extend show_workon_ebuilds to give full paths, provide wrappers
This will allow replacement of equery which in cros-workon

Also other ebuild-related operations like grouping workon ebuilds with the same repo

* Find for all ebuilds takes almost no time when cached and up to ~2 secs when not
* equery takes ages just to load

TEST=cros_workon_ebuilds before and after, and diff:
--- list1       2010-07-27 21:22:23.000000000 -0700
+++ list2       2010-07-27 21:22:32.000000000 -0700
@@ -1,6 +1,5 @@
 app-crypt/tpm-emulator
 app-crypt/trousers
-app-i18n/ibus
 app-i18n/ibus-chewing
 app-i18n/ibus-hangul
 app-i18n/ibus-m17n
(ibus is a mistake, not a workon package, i'll fix that in a separate CL)

	modified:   lib/cros_workon_common.sh

Review URL: http://codereview.chromium.org/2808072
2010-07-29 13:26:53 -07:00
Zelidrag Hornung
151112ce00 Add checkpoints to archive_build
Review URL: http://codereview.chromium.org/2806088
2010-07-29 10:32:26 -07:00
David James
de456309c8 Disable collision-protect in parallel_emerge.
collision-protect doesn't make sense for parallel_emerge, because
we don't lock around merges. Also, if merge fails part-way through,
it's strange to fail just because some files are lying around.
This fixes an issue with the buildbots.

TEST=Checked that collision-protect feature gets deleted by this.
BUG=none

Review URL: http://codereview.chromium.org/3061029
2010-07-29 01:01:15 -07:00
David James
0366864a98 Enable fast build by default in build_image and mod_image_for_test.sh.
The fast build is stable enough now that we can enable it for everybody.
I ran the build constantly all weekend on two machines and I only ran into
one (rare) error, which I've fixed in a separate patch.

See <http://codereview.chromium.org/2856048/show>

TEST=Ran lots and lots of fast builds
BUG=none

Review URL: http://codereview.chromium.org/3059001
2010-07-28 17:08:29 -07:00
Olof Johansson
8ce8312e43 parallel_emerge: Fix builds when PORTAGE_BINHOST is unset
BUG=none
TEST=Try building for a branch new board overlay that doesn't already have a binhost setup.

Review URL: http://codereview.chromium.org/2870068
2010-07-28 17:20:16 -05:00
Chris Sosa
ffff7d76cc Add attempt to update remote and rebase before pushing
TEST=Ran cros_mark_all

Review URL: http://codereview.chromium.org/3066010
2010-07-28 13:21:32 -07:00
Anush Elangovan
9df55b2055 Add repo local manifest support 2010-07-28 12:07:48 -07:00
Zelidrag Hornung
e6532bc3dd Added output to tar command so it does not cause timeout to kick in.
Review URL: http://codereview.chromium.org/3076012
2010-07-28 11:34:29 -07:00
Nikita Kostylev
c8bba90ae4 Make oem_customization flag optional.
BUG=chromium-os:1888
TEST=Manual. build_image should run fine when oem_customization flag is not specified.

Review URL: http://codereview.chromium.org/3076009
2010-07-28 17:05:26 +04:00
Zelidrag Hornung
e93c60e9a9 Removed chrome ui tests and its dependencies from the factory test image
BUG=4828
TEST=make sure factory image actually builds while chrome ui autotests are included in the build

Review URL: http://codereview.chromium.org/3010035
2010-07-27 16:06:20 -07:00
Elly Jones
c5fe33d940 Revert "Copy multiple qualified DBs to the image based on the board name."
This commit breaks the build for x86-full-fast-generic at least. It causes a
file to be copied to itself during the build archive step:
http://build.chromium.org/buildbot/chromiumos/builders/x86%20generic%20full%20fast/builds/128/steps/archive_build/logs/stdio

TEST=None
BUG=None

This reverts commit 1cf4329681.

Signed-Off-By: Elly Jones <ellyjones@chromium.org>
Signed-Off-By: Nathan Williams <njw@chromium.org>

Review URL: http://codereview.chromium.org/2808069
2010-07-27 10:40:10 -04:00
Tom Wai-Hong Tam
1cf4329681 Copy multiple qualified DBs to the image based on the board name.
The DBs are named in the format of qualified_components_${BOARD}_${HWQUALID}.
If only one hwqual id for the board, named in qualified_components_${BOARD}.

Review URL: http://codereview.chromium.org/3038026
2010-07-27 17:02:35 +08:00
Zdenek Behan
ddb0f2c8e3 autotest: create a candidate script for replacement ebuild test (running the tests)
* This script should replace the call to ebuild in autotest wrapper, and essentially
duplicates all the test running functions from autotest-0.0.1.ebuild

* duplicate autotest wrapper into autotest_workon to separate conversion and old functionality

* Add a hack into run_remote_tests to allow using autotest_workon instead

	new file:   autotest_run.sh
        new file:   autotest_workon
        modified:   run_remote_tests.sh

Review URL: http://codereview.chromium.org/2854062
2010-07-26 18:02:29 -07:00
Will Drewry
efce66880f image_to_vm: convert to calling setimage directly
The last changes to the installer broke image_to_vm.  Since image_to_vm
doesn't need to run postinst, but just change the active image, it
now just calls chromeos-setimage and passes in its needed flags (already
supported by the last installer change).  It will also detect whether
the image was built with verification of the rootfs enabled or not
by looking at the default.cfg file.

Also updates the location of where you should run the command in
build_image.

TEST=built a new image with --enable_rootfs_verification and started with qemu -curses -hda [output]; did the same without verification
BUG=chromium-os:2963

Review URL: http://codereview.chromium.org/3034030

Change-Id: I265d6ad8971f9427b78cc07d784fced9cceb5974
2010-07-23 19:43:27 -05:00
Bill Richardson
8bfa4685df Add "kern_guid=%U" to the secure boot kernel command-line template.
With the newest Chrome OS BIOS and bootstub, this will be expanded to the
booted kernel partition's UniqueGuid, so that the kernel device can be
determined with certainty, since the BIOS and kernel may enumerate drives
differently.

You can identify the booted kernel partition at runtime with something like
this:

  sudo cgpt find -1 -u \
    $(cat /proc/cmdline | sed 's/.*kern_guid=\([0-9a-f-]\+\).*/\1/')

Review URL: http://codereview.chromium.org/3035020
2010-07-23 17:24:15 -07:00
Chris Sosa
80a3b2810a Add flag to cros_mark_as_stable to specify tracking_branch.
TEST=Ran using default option and then tried -t chromiumorg/master and
saw a correct failure since I don't have that branch to track against.

Review URL: http://codereview.chromium.org/3032019
2010-07-23 17:00:34 -07:00