Commit Graph

1008 Commits

Author SHA1 Message Date
David James
6be0f08169 Robustify and speed up cros_mark_all_as_stable.
What's new?
 - cros_mark_all_as_stable is now in Python and has unit tests.
 - We now detect and report coding errors that can cause incorrect behavior.
   E.g., if 9999 ebuild or stable ebuild can't be found, we report a hard
   failure so that developers will know about the problem.
 - We now check that git hashes we report actually match up with the
   right repository.
 - Unified diff of changes are now printed to stdout, so that developers
   can see a list of the changes and debug any problems.
 - cros_mark_all_as_stable now takes 2.5 seconds to run.

BUG=chromium-os:7795
TEST=Manually examined diff output from run of cros_mark_all_as_stable
     on full repository.

Change-Id: I762597c9b94e5f8e8171b83c966ad54e21a65c1b

Review URL: http://codereview.chromium.org/3798003
2010-10-21 16:11:00 -07:00
Chris Sosa
aa4670315b Fix equals in RunCommand's in cbuildbot.
Change-Id: If8cdde42a470c3d07cfd41552cf95dea7567688b

BUG=
TEST=Ran cbuildbot.

Review URL: http://codereview.chromium.org/3967002
2010-10-21 10:54:02 -07:00
Scott Zawalski
9310bb6bd3 Fix logic for debug and uprev
BUG=chromium-os:8000
TEST=NA

Review URL: http://codereview.chromium.org/4007003
2010-10-20 22:36:37 -07:00
Raja Aluri
2524a6d474 Revert "'Update CHROMEOS_VERSION_MAJOR=0 CHROMEOS_VERSION_MINOR=9 CHROMEOS_VERSION_BRANCH=94 CHROMEOS_VERSION_PATCH=0 CHROME_VERSION=8.0.552.10"
This reverts commit fb6410ca67.
2010-10-20 22:20:01 -07:00
Raja Aluri
fb6410ca67 'Update CHROMEOS_VERSION_MAJOR=0 CHROMEOS_VERSION_MINOR=9 CHROMEOS_VERSION_BRANCH=94 CHROMEOS_VERSION_PATCH=0 CHROME_VERSION=8.0.552.10
Change-Id: Ic0121ebe1d8b25887a37c23a249513b7fc0af280
2010-10-20 22:19:24 -07:00
Luigi Semenzato
195aebeaea Add tpm_tis module configuration options in preparation for linking with kernel
We need to set these options in order to link the TPM driver in the kernel before we fix the PNP table in the firmware.  This is the least disruptive change that makes the TPM available earlier in the boot sequence.

Change-Id: I729cd7c153507200e177895bae01951e97b70968

BUG=none
TEST=rebuilt kernel, reinstalled, booted, verified that tcsd still runs

Review URL: http://codereview.chromium.org/3969001
2010-10-20 17:35:00 -07:00
Will Drewry
ef22d8fe60 kernel_fetcher.sh: use ro for rootfs.image
With new changes to the rootfs, if it is verified, it won't be
mountable rw.  Since this script just copies out a file, making it
ro (instead of calling enable_rw_mount) seems to make sense.

TEST=?
BUG=chromium-os:7468

Change-Id: I0941e981826005cea1c70653c45c5ae2c5f5a9e0

Review URL: http://codereview.chromium.org/4027001
2010-10-20 18:58:05 -05:00
Will Drewry
f929c3012c image_to_vm: fix up the rootfs for mounting rw
Use new helper for accessing a verified rootfs.

TEST=ran image_to_vm and it worked
BUG=tree b0rked

Change-Id: I03c1c55d1e1056081078cbdd8797235636d4845d

Review URL: http://codereview.chromium.org/4023001
2010-10-20 18:48:20 -05:00
Will Drewry
9b7cb51697 common.sh: switch out echo -ne for printf and fix incorrect offset
echo -ne isn't portable but printf with octals is so this
makes the switch.

In addition, the current offset is off by 3 and is updating the UUID space.
This was from style cleanup and all functional test passed - of course.
This change fixes that too.

TEST=built and installed and checked ro enforcement works
BUG=chromium-os:7972

Change-Id: Ifb3cb2c6f3219af819baff5750453439e1ba6edf

Review URL: http://codereview.chromium.org/3997001
2010-10-20 18:11:24 -05:00
Chris Sosa
e38f90f309 Add ability to blacklist packages from chromeos-base/chromeos.
Change-Id: I05370137dbe9f9adc7b18f2ff77d1b7680275c37

BUG=7973
TEST=Ran with/without current blacklist file.  Also, added chromeos-base/chromeos-chrome
to test it actually dies correctly.

Review URL: http://codereview.chromium.org/4005002
2010-10-20 16:04:23 -07:00
David James
4f7f855728 Update prebuilt.py to filter Packages database.
This ensures that the Packages database doesn't contain private packages.
This is necessary so that the database matches up with the actual files
we upload.

BUG=chromium-os:4843
TEST=Ran unit test

Change-Id: I0355b4ab867a0d08396e45b04523a487951475f4

Review URL: http://codereview.chromium.org/3930001
2010-10-20 14:48:04 -07:00
Will Drewry
55b42c94ca cros_make_image_bootable, mount_gpt_image, common.sh: root filesystem changes: ext2, ro by default
This change makes more of the root filesyste metadata static across builds, but
more can be done there.  It also changes the root filesystem to use ext2 as
we don't need journaling in normal mode.  Optionally we could use ext3 for
non-verified if desired (it's an easy change).

In particular, this change cleans up the following:
- clears the rootfs uuid
- labels it C-ROOT (instead of C-KEYFOB)
- removes reserved inodes and blocks

The major feature of this change, however, is that it adds two simple
helpers to common.sh: disable_rw_mount and enable_rw_mount.  They will
set high order byte (le) in the ro compat field to be 0xff.  This will
tell the kernel that the filesystem uses features R24-R31 which are safe
for use on the running kernel iff the filesystem is mounted read-only.

These functions are called in cros_make_image_bootable and
mount_gpt_image, respectively.  mount_gpt_image will always
enable_rw_mount and cros_make_image_bootable will disable_rw_mount if
--enable_rootfs_verification is true.

The approach is ugly but reasonably well contained.  If ext2 ever gets a
new revision and new features in the same range are introduced, then we
would be getting inconsistent behavior.  That said, it is unlikely that
that churmn will happen and if the impact is negative, it will ideally
show up during testing.

N.B., this will likely result in changes needing to be made to the
signing scripts in vboot_reference to ensure that rw mounting is
enabled/disabled in the same way (E.g., during stamping).

BUG=chromium-os:7468
TEST=- built x86-generic, imaged to usb stick, attempted to mount rw /dev/sdc3 on the host and was properly bounced.
     - booted to the image just fine on a dogfood device.
     - mod'd for recovery, then installed and booted.
     - mod_image_for_test runs with no errors; booted the resulting image as well
     - booted a factory_install with the pending dm changes
     - BVT passed with build_image x86-generic (vboot enabled)
     - [in progress] autotest that checks if the rootdev = /dev/dm-0 and
       then does a dumpe2fs | grep -q FEATURE_R31

Review URL: http://codereview.chromium.org/3916002

Change-Id: If4dcba7568a110f4e32627c916d9e5741e5e5414
2010-10-20 15:44:11 -05:00
Chris Sosa
6304adc473 Setup python symlinks for python on stateful partition.
Change-Id: Ifbfbd3b55ae97ea94ea5bba40a001c6a1ddbf7a0

BUG=720
TEST=Ran suite_Smoke in a VM and ran python from command line and checked
to see if I could import packages from /usr/local/site-packages.

Review URL: http://codereview.chromium.org/3972001
2010-10-20 13:40:33 -07:00
Will Drewry
d6435d4746 kernel cmdline: add dm_verity.dev_wait=1
dm_verity will wait for a device to be ready. This is needed in the factory installer so that
dmsetup doesn't fail early before the usb device is visible.

TEST=built images and booted them (x86-generic & factory installer)
            Prior to commit, need to ensure dev_wait doesn't break unverified boot behavior
BUG=chromium-os:7451

Change-Id: I5b838b94e6a17dd0778331121311cdfe180991ce

Review URL: http://codereview.chromium.org/3936001
2010-10-20 15:37:46 -05:00
Chris Sosa
debca51332 Add --debug option to cbuildbot for developers to use cbuildbot without pushing changes.
Change-Id: Idda6c7287cdb1863eb4abe8c56da2e763e9fe777

BUG=7926
TEST=Ran with --debug and verified I didn't push a change.

Review URL: http://codereview.chromium.org/3880002
2010-10-20 11:00:44 -07:00
Scott Zawalski
cd1c749b0e Created branch 0.9.92.B. Update CHROMEOS_VERSION_BRANCH=93
Change-Id: Ie0599d9d0f38502f4b4e35155bbde1a34bf24204
2010-10-19 23:52:46 -07:00
Doug Anderson
0c9e88de6c Added a function to update chroot sudoers automatically.
This function is now called by enter_chroot.sh.  A separate change
will call the function from make_chroot

Change-Id: I4fc07c413e56db3e5e7617428f20f2ffc02790d7

BUG=chromium-os:7072
TEST=Took root out of sudoers and ran enter_chroot.sh script; saw that it was re-added.

Review URL: http://codereview.chromium.org/3909001
2010-10-19 14:49:39 -07:00
Chris Sosa
62d75f0582 Get correct DEBUG logs for generate_test_report.
Change-Id: Ia77ec225abc0e8906662b57ca8e71bbb13b42635

BUG=
TEST=Ran it.

Review URL: http://codereview.chromium.org/3890002
2010-10-19 13:57:37 -07:00
David McMahon
80e71f2407 Moving CHROMEOS_VERSION_BRANCH=91 2010-10-19 13:21:26 -07:00
Chris Sosa
5911f2003b Clean up any previous kvm state.
Change-Id: I01fecde5e687992ccf36889803a4cb84b71a38b2

BUG=7818
TEST=Ran with/without previous kvm instance

Review URL: http://codereview.chromium.org/3760012
2010-10-19 12:58:24 -07:00
Chris Sosa
17414a6b24 Propagate board correctly for latest image and remote_tests.
Change-Id: I42741280656e9ee300dbc264ef7bf3345efa5223

BUG=
TEST=Ran it with test builder on own machine.

Review URL: http://codereview.chromium.org/3826015
2010-10-18 17:08:53 -07:00
Scott Zawalski
fe3805ac16 Update prebuilt to modify make.conf files for the board/host targets.
This changes the format that I was updating from 'key value' to 'key=value'.
I modified my unittests to test this.

BUG=NA
TEST=Branch runs without commit.

Review URL: http://codereview.chromium.org/3858003
2010-10-18 16:36:44 -07:00
Ken Mixter
b8a72348e4 crosutil: No longer use autotest-0.0.1 in run_remote_tests even outside chroot
Change-Id: If83bf22fde6259b64fdd007066b5c46c7925d120

BUG=
TEST=Run a few tests inside and outside chroot

Review URL: http://codereview.chromium.org/3790005
2010-10-18 13:34:29 -07:00
Scott Zawalski
273cdc8a3e Moving trunk to CHROMEOS_VERSION_BRANCH=89 2010-10-18 13:22:01 -07:00
Chris Sosa
dfcb89b7d5 Increase statefulfs_size to accomadate larger stateful partitions.
Change-Id: I0713b31b0393fc696d1112ee4cad4c20d86e602d

BUG=
TEST=Config change

Review URL: http://codereview.chromium.org/3860002
2010-10-18 10:26:16 -07:00
Scott Zawalski
d1ea7efe8b Created branch 0.9.86.B. Update CHROMEOS_VERSION_BRANCH=88
Change-Id: I32c0d6a289276c74dc573131a687a9c170b81a58
2010-10-17 20:11:31 -07:00
Chris Sosa
d4b4cb8bc7 Print out debug of test when it fails.
Change-Id: I9dc535a7d1ee5feeb364456ca8ff367285cdad2b

BUG=
TEST=Ran with with a test suite with both passing / failing tests

Review URL: http://codereview.chromium.org/3814009
2010-10-15 16:36:45 -07:00
David James
ef9f04e2fd Update cbuildbot to use git.chromium.org instead of src.chromium.org.
src.chromium.org is no longer supported for git operations.

BUG=chromium-os:7830
TEST=none

Change-Id: Ia7cd9ff76625441f16d20d2f9a8e0f752dca0586

Review URL: http://codereview.chromium.org/3782010
2010-10-15 16:18:29 -07:00
Anush Elangovan
189d85d20e Revert "Fix issue with stale cache from bad builds."
This reverts commit 365d4b0cc8.

Fix preflight breakage, this seems like the most possible cause. Will recommit if not.

BUG=
TEST=

Review URL: http://codereview.chromium.org/3767012

Change-Id: I93fe463fec77c445e7ff0ec86db25d5ef2997ad7
2010-10-15 11:36:53 -07:00
Chris Sosa
ccfa2ac970 Change hostname to ip address if we can get it from ifconfig.
Change-Id: I6719ffe0c3c9356cf48b391b0a257e268eec94dc

BUG=7780
TEST=Ran it with cros_run_vm_test.

Review URL: http://codereview.chromium.org/3829002
2010-10-15 10:43:23 -07:00
Chris Sosa
365d4b0cc8 Fix issue with stale cache from bad builds.
We always want to re-build packages we are revving, however, if the last
build succeeds in building but fails tests, we have a bad local cache that
has revved packages.  This avoids this issue by working on all packages we
are revving.

Change-Id: I3bd7463a4090b1f007d09be81bdf65657bd9c3f3

BUG=7589
TEST=Ran it with cbuildbot.

Review URL: http://codereview.chromium.org/3745006
2010-10-15 10:37:04 -07:00
Scott Zawalski
f94950fb39 Created branch 0.9.84.B. Update CHROMEOS_VERSION_BRANCH=85
Change-Id: I71d16ea7af9aa1f59279cbb2d70c49e1cc0e5237
2010-10-14 21:17:15 -07:00
Doug Anderson
77b239b86a Fixed partner issue 1414: Failed to create dev_recovery_image.bin using dev install shim
Change-Id: I69291bf29d68f9511d0618e55cabccee9904749b

BUG=chrome-os-partner:1414
TEST=Ran the script

Review URL: http://codereview.chromium.org/3789004
2010-10-14 17:29:19 -07:00
Olof Johansson
1688a6d416 Change KVM to use e1000
add -net nic,model=e1000 in start_kvm and in the suggested command as
output by image_to_vm.sh

Signed-off-by: Olof Johansson <olofj@chromium.org>

BUG=none
TEST=Boot an image, run lspci in a terminal, see intel nic

Review URL: http://codereview.chromium.org/3779007

Change-Id: Id7f614adec0ae69b8f4de152832538a13ea4cbad
2010-10-14 19:08:44 -05:00
Chris Sosa
6f1b9bd06d Fix update script to use 127.0.0.1 as hostname usage w/ ssh is broken on builders.
Change-Id: I88524dbe726fc20b8dbc54c712e63b9293479029

BUG=
TEST=Ran it on builder.

Review URL: http://codereview.chromium.org/3809003
2010-10-14 15:46:14 -07:00
David James
4a71ea31cb Update cros_mark_as_stable.py to also update the private overlay
BUG=chromium-os:7218
TEST=./cros_mark_all_as_stable --tracking_branch=cros/master

Change-Id: I0f7c4e2d2aa4fc6325721901a710bfc33fa39a91

Review URL: http://codereview.chromium.org/3516025
2010-10-14 14:03:18 -07:00
Ken Mixter
9b8fd4e671 crosutils: make cros_run_unit_tests prepare before building tests
Change-Id: I105c8e37b1241e89efda2c7cef35840bdc32e108

BUG=7754
TEST=Ran cros_run_unit_tests and verify patch (during prepare step) was done

Review URL: http://codereview.chromium.org/3801003
2010-10-14 13:23:44 -07:00
Darin Petkov
9bdd04b730 AU: Provide an option for forcing full or delta kernel update.
BUG=7705
TEST=tried generating updates with and without the option

Change-Id: I45fccece0aa37c92e63427ea8f86e208f24253eb

Review URL: http://codereview.chromium.org/3782005
2010-10-14 09:25:21 -07:00
Scott Zawalski
74eee7ad20 Created branch 0.9.82.B. Update CHROMEOS_VERSION_BRANCH=83
Change-Id: Ib6c67a1ac5bc85588123d7e200957c168a0fd586
2010-10-13 22:54:50 -07:00
Dale Curtis
15e49571bf Fixed incorrect option comment in run_remote_tests.
Autotest will not split comma separated arguments. The correct calling method for multiple arguments is to separate them by spaces and quote the whole thing. E.g., instead of:

./run_remote_tests --args arg1,arg2

It should be:

./run_remote_tests --args "arg1 arg2"

See line 140 in src/third_party/autotest/files/server/autoserv_parser.py for
confirmation.

Change-Id: I88d65c0bd144942a856c2f95371d5a55b0a3a172

BUG=
TEST=No code changes. Ran --help and --args to verify nothing dumb happened and the correctness of comment.

Review URL: http://codereview.chromium.org/3772002
2010-10-13 12:38:04 -07:00
David McMahon
f36d37fd67 Created branch 0.9.80.B. Update CHROMEOS_VERSION_BRANCH=81 2010-10-13 11:41:59 -07:00
Chris Sosa
af62e73dce Add ability to download using newest image from zip url.
This changes the behavior of ctest to try to grab the latest zip from the zip web pages if it can't use the latest link (due to IOError or none provided).  It assumes they have an Apache server and they reference images using version->ChromeOS-version-.*.zip) for each board/channel.

Change-Id: I4ee075db7afd58c8d08f59aca2e69b5bab5ff5e9

BUG=7675
TEST=Ran it with -l "http://Doesnotexist" and also without a -l option.  Added new unittests to test and ran them as well.

Review URL: http://codereview.chromium.org/3659002
2010-10-12 15:07:54 -07:00
Paul Stewart
e45157fe07 Temporarily clear noexec flag on /var if set
The "emerge" process sometimes copies files to /var/tmp
and then run it.  Recent changes to images set the
"noexec" flag on /var, thus preventing such activities.
Temporarily clear this flag from within cros_package_to_live.

BUG=none
TEST=Manually set noexec on /var and run cros_package_to_live

Review URL: http://codereview.chromium.org/3739001
2010-10-12 14:25:02 -07:00
Chris Sosa
e695541bc6 Hide errors until builder succeeds.
Change-Id: Ifb40e4931c1b0a1d32051d1d4851bca572ef3acf

BUG=7649
TEST=Ran it with fake urls

Review URL: http://codereview.chromium.org/3678004
2010-10-12 11:07:28 -07:00
Chris Sosa
4bec3be90b Gets image_to_live to work with a custom devserver_port
BUG=
TEST=Ran image_to_live using port 9090

Review URL: http://codereview.chromium.org/3646005

Change-Id: I2a7086ec46f4e1f221ca4dd22f636d53708a88a1
2010-10-11 19:50:10 -07:00
Scott Zawalski
96619e72e1 Update RemoteUpload to return a set Update if failed_uploads to check deeper
in the returned set to see if it is a bonified failure

BUG=NA
TEST=Local run to verify

Review URL: http://codereview.chromium.org/3667005
2010-10-11 15:31:53 -07:00
David James
807d018656 Update all ebuilds to point to a branch or tag.
Mark all ebuilds in chromiumos-overlay and chromeos-overlay as stable and
point them at the current version we have checked out.

This is useful in case we want to verify that all ebuilds are pointing at the
right commit hash when we are doing a build on a branch. It is intended to be
used prior to tagging and can be used instead of a preflight queue. It is an
alternative to cros_mark_all_as_stable that does not require a chroot and
does not increase the revision number on ebuilds.

Unlike cros_mark_all_as_stable, which only updates the latest stable ebuild
for a given board, this script updates all stable ebuilds for all boards,
including unused stable ebuilds. We do this for the sake of simplicity
because it's hard to know in advance which ebuilds are truly unused, and it
shouldn't hurt to update unused ebuilds.

This script does not support cros_mark_as_stable_blacklist. If there are any
packages in the blacklist, this script exits with an error message.

TEST=Ran this script to point power_manager at branch and tested that we
     built power_manager from the branch.
BUG=chromium-os:7218, chromium-os:6429

Change-Id: Ib2308806c5a4db979b1245a90407fd5b6353e4fd

Review URL: http://codereview.chromium.org/3607014
2010-10-11 15:04:08 -07:00
Will Drewry
32d8c11747 build_image & make_bootable: allow additional kernel cmdline args
In order to more easily test kernel commandline tweaks, ensure that
--boot_args is propagated to all kernel cmdline sinks.

This is useful for changing default schedulers and any other kernel argument
tweaks we may want to quickly test out.

TEST=build and tested x86-generic
BUG=none

Change-Id: I5138755aa8c5e32e7f117f18291d2ae197e95bef

Review URL: http://codereview.chromium.org/3644004
2010-10-11 15:37:17 -05:00
David James
3a4a6bbfb9 Update cros_mark_as_stable_unittest.py to pass again.
cros_mark_as_stable.py was updated on Sep 8, but the unit test was not updated
to match. This patch fixes the unit test.

BUG=chromium-os:7586
TEST=Ran unit test.

Change-Id: I283ef33e172a95efc41b4090dc4f67c9d84dd9a3

Review URL: http://codereview.chromium.org/3682003
2010-10-11 09:59:39 -07:00
Chris Sosa
020aa69f37 Fixes for cbuild to work with ctest.
1)  image_to_vm aborts because e2fschk requires a terminal to
be connected in order to check whether or not you want to repair the fs.  Since
we always want this to be true, set -p

2)  Clean up test harness to only verify the number of tests that passed on the
base image are the same on the update back to.  This is because older images
have many tests that fail.  I leave 10% as a reasonable number to pass.

3)  Redirect output from autotest in run_remote_tests to stderr so it gets
logged in a calling script that captures stdout.

Change-Id: If412274353683add20d136747113eb9c2bd41330

BUG=4690, 5533, 7287
TEST=Ran with internal tools for builders.

Review URL: http://codereview.chromium.org/3536018
2010-10-08 16:36:20 -07:00