We need more flexible image creation tool for various layout, like "RMA with
multiple release images" which needs 4 images (install, test, release A,
release B).
To make the creation of image more flexible, this CL changes
make_universal_factory_shim to using a "layout file" that can describe any disk
image format we want.
Image creation speed is also improved by aligning partition offsets and
restricting oflag=dsync applied only when being used for block devices so that
make_universal_factory_shim (outputs to a normal temporary file) becomes faster
by utilizing system cache.
For a system with every images already cached:
- Time for building 3 in 1 factory instal shim:
Before = 8 seconds, After = 5 seconds.
- Time for full RMA shim creation:
Before = 1m25s, After = 40-50 seconds.
BUG=chrome-os-partner:4108
TEST=./make_universal_factory_shim.sh image1 image2 image3 image4
# boots correctly after being imaged to SD card.
./make_factory_package.sh --usbimg RMA ....
# RMA image can install images and works correctly
Change-Id: I645196d6d6e0a24d3dfa4c413a338279df4c0d5b
Reviewed-on: http://gerrit.chromium.org/gerrit/6893
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
"cros_sdk" is a drop-in replacement of enter_chroot.sh. The only
important difference is in the calling path, specifically that
cros_sdk is in path but has to be called from $(pwd) being inside the
repo checkout, while enter_chroot.sh is called by explicit path.
Invariably, "./enter_chroot.sh" can be replaced, as pwd=src/scripts.
Calling by absolute path can be replaced by first changing directory
anywhere into the repo checkout, and then calling cros_sdk.
BUG=chromium-os:18750
TEST=run them
Change-Id: Ieff91a27bb419e1121361d5b3a11e4c87ff7a087
Reviewed-on: http://gerrit.chromium.org/gerrit/6273
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
In order to support building arbitrary image, the partition copying scripts has
been changed to support "copying partitions in same size" and "overwriting
partitions in different size", and "copying partition from external file".
We need these APIs to create disk/usb image with release images that is using
partition with different size (ex, recovery images).
Image copying buffer selection and disk image creation time are also improved.
BUG=chromium-os:15050
TEST=./make_factory_package.sh ... --diskimg preimage.bin
./make_factory_package.sh ... --usbimg rma.bin
./make_factory_package.sh ... # omaha mode
Change-Id: I6a4c820abf59e780985c95dc35f9340b347bd952
Reviewed-on: http://gerrit.chromium.org/gerrit/5981
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
The release image parameter must be an image signed for SSD booting. This CL
adds detection code and allows on-the-fly conversion from recovery to SSD image.
BUG=chromium-os:15050
TEST=./make_factory_package --release RECOVERY --factory FACTORY # success
# Seeing: INFO : Image type is [recovery]:...
./make_factory_package --release USB --factory FACTORY # success
# Seeing: Image type is [usb]:...
./make_factory_package --release SSD --factory FACTORY # success
# Seeing: Image type is [ssd]:...
./make_factory_package --release GARBAGE --factory FACTORY # failure
# Seeing: Image type is [invalid]:...
./make_factory_package --release GARBAGE --factory FACTORY --nodetect_release_image # success
# No image type messages
Change-Id: I8530b3f58574a4298b4d6d904a12bb92636c7365
Reviewed-on: http://gerrit.chromium.org/gerrit/5965
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
When we want to separate kernel and rootfs, there must be a more flexible syntax
to assign input source.
The partition info parsing code is also improved to detect errors.
BUG=chromium-os:15050
TEST=./make_factory_package --release RECOVERY --factory FACTORY
Change-Id: Ie74b3e23117480a7f503488b39dedceadbfb41e3
Reviewed-on: http://gerrit.chromium.org/gerrit/5962
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
RunCommand should only pass in stdin, stdout, and stderr to processes,
not other file handles. Fixing this prevents issues where child
processes can cause the parent to hang by forgetting to close the
file handle when launching daemons.
BUG=chromium-os:18104
TEST=Buildbot run with smoke suite. Unit tests.
Change-Id: Ib85095e1fb592ef05a972a5412348363049e6d86
Reviewed-on: http://gerrit.chromium.org/gerrit/4673
Reviewed-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: David James <davidjames@chromium.org>
It seems like we are seeing whacky errors with sys.stderr
dying when trying to print out a process and its exception.
I think controlling how the output is printed in RunCommand
is better than passing it with the exception.
Change-Id: I3711575e5d9e710c3aea6ff19d43d149fd83f44c
BUG=chromium-os:14398
TEST=Ran it with au_test_harness
Review URL: http://codereview.chromium.org/6902129
Change-Id: If46fd1558a00498ce4721dd756532e916f49bc19
BUG=chromium-os:12950
TEST=Ran with autestharness and ran unittests.
Review URL: http://codereview.chromium.org/6877015
Cleaned up _InsertKeyIntoImage to be more general (able to remove key if no key set). Also made it much more intelligent ... only doing things when it needs to.
Moved mount / unmount code to cros_lib.
BUG=chromium-os:12684
TEST=Ran it a lot ... inspected images after and saw tests pass.
Review URL: http://codereview.chromium.org/6698017
Change-Id: Icd96d1178eeabf45a2d5916fbcab8bf7ffba7e21
Change-Id: Ib5617612be38bb6d7dafcb2edab887c998a676ab
BUG=chromium-os:12951
TEST=Ran it with Simple. Saw logs generated (1_update) and in root/dev_server.log. Ran unittests for cros_build_lib.
Review URL: http://codereview.chromium.org/6672007
This CL is a large refactoring that moves the test_harness from using sub-classed classes of au_test into a one where we have the same AU_Test for all test flows but different test workers that operate on different types of machines. Specifically we move the VM / Real-image specific logic into real_au_worker and vm_au_worker. There isn't anything functionally different in this change.
Also, we move the classes into their own modules and clean up use of cros_build_lib to be more stylistically correct.
Change-Id: I3e25141174c3d5ba22962bf94365815e69e5bedf
BUG=chromium-os:11172
TEST=Ran with simple and full w/ w/out delta update payloads and with unittest and without unittest keys. Ran real test using full suite on test device using full update payloads.
Review URL: http://codereview.chromium.org/6597122
This reintroduces the change after commit 3635aaa55e, which reverted 01c8423582.
BUG=chromium-os:11717
TEST=Ran new unit tests, and did full build.
Review URL: http://codereview.chromium.org/6579048
Change-Id: I598e459ddaa156af47d47d0482cac16ce44f07a4
extra noise in the final exception output.
This change stops catching and rethrowing the same exception, and generally
makes the RunCommand slightly more readable (if longer).
Some unit tests are added, but they only test RunCommand, and not all options
to RunCommand.
BUG=chromium-os:11717
TEST=Manual, and new lib/cros_build_lib_unittest
Review URL: http://codereview.chromium.org/6576016
Change-Id: Ibe9b13ff302621e93196008f86914cb65a232d61
Note: sudo before ifconfig is for running on distros which do not
have /sbin or /usr/sbin in common user's paths, like gentoo.
Change-Id: I85bd379ad059d6ecaa8c11f3167fae27987479dd
BUG=5246
TEST=run cros_au_test_harness and see it not fail
Review URL: http://codereview.chromium.org/6541008
Change-Id: Ia5e0042250da22cebadfd48d143f21597bcada5d
BUG=chromium-os:11716
TEST=Ran vm test and vm update scripts.
Review URL: http://codereview.chromium.org/6413019
from within the chroot.
It also fixes a number of style issues.
It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.
It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.
BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.
Review URL: http://codereview.chromium.org/6240018
Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
cros_workon is the only user of cros_workon_common.sh
This is pre-requisite to moving cros_workon into devutils.git
BUG=11507
TEST=Ran ./cros_workon --board x86-generic --all list
Change-Id: I1eab551ab33646360e507328932c151a0d36f50a
Review URL: http://codereview.chromium.org/6347052
As a warning, this is a pretty big change. At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness. Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).
This change also has the tests run in parallel. Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.
Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4
BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).
Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f
Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=be787f3
Review URL: http://codereview.chromium.org/6277015
As a warning, this is a pretty big change. At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness. Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).
This change also has the tests run in parallel. Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.
Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4
BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).
Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=c418a8f
Review URL: http://codereview.chromium.org/6277015
As a warning, this is a pretty big change. At a high-level,
this changes the harness to move the managing of the devserver from
image_to_live into the actual test harness. Paths of the cache locations (for
archive_url) are taken when pre-generating the updates and stored
in a dictionary (maps "path_to_base->path_to_target" (or path for full updates)->
cache paths).
This change also has the tests run in parallel. Because we now start
X number of VM's at once, each VM needs it's own pid file and ssh_port.
This logic was added as well as running the actual tests in parallel.
Change-Id: I1275d79740c50c2a8028489b43dcbbcf5bbd56c4
BUG=chromium-os:10723
TEST=Ran it ... a lot with -q but without a test_prefix (so full test suite).
Review URL: http://codereview.chromium.org/6277015
dump output until after the command has finished, so not useful for
following 'live'.
Change-Id: Ia8562f5e6f8d7ed8af7e188faa68f5cdbba487c6
BUG=chromium-os:11164
TEST=None
Review URL: http://codereview.chromium.org/6296013
fix, but it leads to better error messages from ctest.
Change-Id: I624ad805ef037b022e6e67b56f0ed10ca05fe325
BUG=11096
TEST=Test currently in progress.
Review URL: http://codereview.chromium.org/6257007
build problems, and will give better diagnostics if not.
Change-Id: I2f7a5d9b2e0c63a8189ad212125c7a5c0178c3c4
BUG=10567
TEST=
Review URL: http://codereview.chromium.org/6331007
When running outside chroot, we should use locate_cgpt + "$GPT" to invoke cgpt;
otherwise it may be not in system PATH.
For dealing with a true SSD image (over USB, for example) this CL also improved
data writing speed from <2M/s to 8M/s+ (over USB).
BUG=chromium-os:10531
TEST=./make_factory_package.sh \
--factory=../build/images/x86-mario/latest/chromiumos_factory_image.bin \
--release=../build/images/x86-mario/latest//chromiumos_image.bin \
--diskimg=/dev/sdb
Also tested:
--diskimg=blah # local file
(chroot) --diskimg=/dev/sdd
(chroot) --diskimg=blah
Change-Id: I647d97fe388f5b6a6223fa8cacfecdf00265f60c
Review URL: http://codereview.chromium.org/6261003
Change-Id: Ie80843a7b81a37b41ae19fc33244b5c0b8152282
BUG=chromium-os:10434
TEST=Ran it with full au test harness (still running but already showing promise).
Review URL: http://codereview.chromium.org/5988006
We had a major issue because the latest dev channel release pre-dates the virtio change. Because of this, ctest was connecting to update that channel using virtio (which is unsupported for that). This change drops a file telling the vm lib to use e1000 as before.
Also, fixed stateful change logic.
Change-Id: I394a8cece71bdd0d55efa21ba5b6d24804432c6b
BUG=chromium-os:10434
TEST=Ran it with ctest and cros_au_test_harness on images with/without the file.
Review URL: http://codereview.chromium.org/5928005
ability to filter or manipulate content as it's passed through.
Create two example tests using this utility that cause
updates to close early to test resumes, and to add delays
to make sure the update still completes correctly.
What other tests should be created for this?
BUG=chromium-os:8207
TEST=Run by hand
Review URL: http://codereview.chromium.org/5632002
Change-Id: Iefb8c8e223fb2ba6bad2c551f7c4403a0bec6ecf
Use general purpose wait in case your connecting to a vm
that you didn't start.
Change-Id: Ie67e2d3235882d372d04e4a44d0781ddfc8d4645
BUG=chromium-os:10121
TEST=Ran it, caused some crashes. Still ran test after retries.
Review URL: http://codereview.chromium.org/5757001
Adds verbose flag to print output while updates are running rather than on error.
Adds test_prefix to specify a prefix of tests you want to run
Flushes stderr more reliably for builders and debugging purposes
Dies when stateful_updates fail
Change-Id: I9e4289deec3688c52b92763eee9334a8868f948e
BUG=chromium-os:6910
TEST=Ran with test_prefix and verbose on and off
Review URL: http://codereview.chromium.org/5573007
Change-Id: I1d0e546f6ea837928ba426e4690007ef2649f68e
BUG=10102
TEST=test KVM with -net virtio and ensure it boots and can copy files in etc
Review URL: http://codereview.chromium.org/5543006
It's like RunCommand, but you can run the command, capture it's error code and output together.
This is to support tests that need to capture output of image_to_live, and to know if it succeeded.
Change-Id: If674b6d79697c0f0b5c96be9fc83adbed9b9893e
BUG=chromium-os:9502
TEST=Ran by hand
Review URL: http://codereview.chromium.org/5339006
Major changes are:
- prefix functions in cros_image_common with 'image' to avoid naming conflicts
- use awk to replace grep+sed+cut
- use case to handle board name list
- factory_setup.sh should return error if applying patch failed
- refine for shell script coding style guide:
* replace `` with $()
* quote variables
* replace "! -z" by "-n"
* no space between redirection symbol (>) and target name (eg, >>filename)
* 1> should be simply >
* no trailing \ for && and ||
BUG=chrome-os-partner:1583
TEST=manually verified:
./image_to_usb.sh --factory # calls mod_image_for_test --factory, and works fine
./make_factory_package.sh --factory PATH_TO_IMAGE --release PATH_TO_IMAGE ...
# factory payloads were created successfully
Change-Id: I6bb10bdfb12cbdb14e9816b3ad72dfe4b7b0472f
Review URL: http://codereview.chromium.org/5168001
This bug doesn't affect production but affects the unit tests.
BUG=chromium-os:9201
TEST=Confirmed this fixes an infinite loop in the unit tests, and that
cbuildbot.py still works.
Change-Id: I1a56b92d8229110c84cebbe877e55fb99f053212
Review URL: http://codereview.chromium.org/5025002
What's new?
- cros_mark_as_stable now exits with errors if directories are specified
that don't exist.
- cbuildbot.py always explicitly specifies overlay directories so
cros_mark_as_stable can rely on them existing.
- Package names and paths are now separated with colons instead of
spaces, so as to allow for us using the same syntax with
enter_chroot.sh as we use without the same script. (enter_chroot.sh
mucks with command-lines that contain spaces or quotes.)
- cbuildbot.py now ensures its build path is a absolute path. This ensures we don't kill the wrong processes, if, for instance, the buildpath is '../..'
- All buildbots now explicitly specify what overlays they want to rev. Public buildbots only rev public ebuilds and private buildbots now only rev private ebuilds.
BUG=chromium-os:8647
TEST=Ran unit tests. Manually marked packages as stable. Ran cbuildbot.py test run.
Change-Id: I1df6d428973d91329c4f5159e2886889a3ebb7c7
Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=795bd30
Review URL: http://codereview.chromium.org/4442001
The team is moving utility/library scripts into 'lib' folder.
image_common.sh should follow this policy.
Also refined the parameter check of mk_memento_images.sh
BUG=chromium-os:5208
TEST=./make_factory_package.sh --factory PATH_TO_FACTORY --release PATH_TO_RELEASE;
# factory bundle created successfully
./mk_memento_images.sh PATH_TO_PART2 PATH_TO_PART3
# update.gz created successfully
./mk_memento_images.sh PATH_TO_IMGE 2 3
# update.gz created successfully
Change-Id: I3afecf05da2832986723f28b595045d0540ea9e9
Review URL: http://codereview.chromium.org/4825004
Change-Id: I87c41e60797848192cee1f0c165215fcec8f5bd2
BUG=8850
TEST=Ran a few tests including suite_Smoke
Review URL: http://codereview.chromium.org/4647001
...so that the console doesn't get messed up due to the serial port being
redirected from stdin.
BUG=8729
TEST=./bin/cros_run_parallel_vm_tests suite_Smoke suite_Smoke; checked console
Change-Id: I5a0cedba2b0ac7f453b4574666c645e84b30b379
Review URL: http://codereview.chromium.org/4501001
