Commit Graph

1728 Commits

Author SHA1 Message Date
Hung-Te Lin
7d164d20fc crosutils: clean up make_factory_package
This is part of clean-up in factory related scripts.

Calling mk_memento_images does not need sudo, and some pushd can be removed
because most operations take full path now.

BUG=chromium-os:15050
TEST=./make_factory_package --release RECOVERY --factory FACTORY

Change-Id: I323c9abbc4a98b22736c755669f8ecd18b05cfd5
Reviewed-on: http://gerrit.chromium.org/gerrit/5964
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2011-08-14 23:33:17 -07:00
Hung-Te Lin
6e885f2e84 make_factory_package: always require partition tools (cgpt or parted)
When there're more and more advanced features landed into make_factory_package,
supporting the "unpacked mode" (use unpack_partition.sh to extract all
partitions) becomes a problem.

cgpt is mandatory in chroot, and parted is installed by default for most Linux
distributions, so I think it may be safe now to force using partition tools,
which is also more robust than unpack_partition.

BUG=chromium-os:15050
TEST=./make_factory_package --release RECOVERY --factory FACTORY

Change-Id: I865aebdf3ce2cbd21b0fe22fdce8612810ee78f7
Reviewed-on: http://gerrit.chromium.org/gerrit/5963
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2011-08-14 23:33:04 -07:00
Hung-Te Lin
40b5b87d98 crosutils: change mk_memento_images.sh to support arbitrary input source
When we want to separate kernel and rootfs, there must be a more flexible syntax
to assign input source.

The partition info parsing code is also improved to detect errors.

BUG=chromium-os:15050
TEST=./make_factory_package --release RECOVERY --factory FACTORY

Change-Id: Ie74b3e23117480a7f503488b39dedceadbfb41e3
Reviewed-on: http://gerrit.chromium.org/gerrit/5962
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
2011-08-14 22:56:48 -07:00
Chris Sosa
e8ff2e848e Fix bug where we crash if our non -9 kill fails.
BUG=chromium-os:19154
TEST=Ran cros_stop_vm

Change-Id: I7150481649086e535469009c207c6fee7bb1990c
Reviewed-on: http://gerrit.chromium.org/gerrit/5972
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
2011-08-14 17:47:35 -07:00
Zdenek Behan
892e6acdc0 crosutils: move enter_chroot.sh to sdk_lib, code unchanged
* Removed boilerplate, simplified search code.
* Fixed one too long line

This will unfortunately kill all outstanding CLs into enter_chroot.

BUG=chromium-os:18750
TEST=run it

Change-Id: I39c45fa8163d92487b512e7e8d298ce9231f4bd2
Reviewed-on: http://gerrit.chromium.org/gerrit/5830
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Anush Elangovan <anush@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2011-08-12 16:35:07 -07:00
David James
27d00f3c9e Update build_image to not use -uDN for developer packages.
The main benefit of this change is that it is no longer necessary to specify
custom use flags to build_image because it just merges what you already have
installed in your buildroot. It does not second guess the packages you already
have installed and just installs them, as it should.

1. Packages in the developer image should already be up to date, so there
   is no need to specify '-u' for --update or '-D' for --deep.
2. The developer image should have identical use flags to the base image,
   so there is no need to specify -N for '--newuse'.
3. The --verbose flag is generally useful, so I've updated all emerges to
   use them so you can see what the use flags are used for the emerges.

BUG=chromium-os:19078
TEST=Verify build_image builds same image and installs same packages
     with and without change. Verified specifying USE= is not necessary
     if a few dev packages were customized.

Change-Id: I4c205c961cca84ca3161b49f59cdd37a5a4ed5b1
Reviewed-on: http://gerrit.chromium.org/gerrit/5816
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-08-12 10:43:08 -07:00
Mike Frysinger
a1a06abeb7 common.sh: add quotes where appropriate
Much of this file quotes paths which could possibly contain breaking
chars (e.g. spaces), but a few are missed.  In general, this probably
doesn't break as no one is adventurous enough to use spaces, but might
as well do it for the fun of it.  And to be consistent.  One of those.

BUG=None
TEST=ran make_chroot/enter_chroot/build_packages; seemed to be OK

Change-Id: Ic64d60790f50bb812c87a4672e2a24fafbccd749
Reviewed-on: http://gerrit.chromium.org/gerrit/5759
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
2011-08-12 08:16:14 -07:00
David James
e29e0448e2 Ensure builders work with cached output image.
If test image is present already, mod_image_for_test should
succeed, even if the input image is missing.

BUG=chromium-os:19087
TEST=Run x86-mario-release builder and verify it works now.

Change-Id: If9729328120c7bbd9c1fffca26d6b1cddf2e980e
Reviewed-on: http://gerrit.chromium.org/gerrit/5831
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-08-11 21:28:34 -07:00
Mike Frysinger
c2474ae596 common.sh: tweak color order for set -x love
If you trace any script with `set -x` enabled, bash will dump the color
variables and all of your output will be colored.  So move the "reset"
color to last so that the bleeding is localized to a few lines rather
than a few hundred.  There should be no functional difference.

BUG=None
TEST=ran setup_board with and w/out -x to create x86-generic to check result

Change-Id: I1e598cc4a38e9f217f2f670a00bd676bf70c2338
Reviewed-on: http://gerrit.chromium.org/gerrit/5804
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-08-11 19:40:59 -07:00
Mike Frysinger
5df0eab4fa enter_chroot: drop LANG=C hardcode
The source of this LANG=C seems to date back to when apt was used as
the package manager [1].  It was further migrated to enter_chroot to
keep noise in the logs buildbot down to a minimal [2].  I've seen the
excess spam apt can spew in a default deb chroot when the active env
locale is not available (in reality it is perl that complains) because
the locale data has not yet been generated.

None of this should apply to Gentoo though as we don't execute perl
anywhere during the build.  So hopefully we can simply drop this now
as an artifact of days gone by.

The reason for not hardcoding this in the first place is that it can
easily wreck havoc when the host locale is multibyte (e.g. en_US.UTF8)
but the chroot is forced to narrow (e.g. C).  Attempts to use multibyte
chars will usually render correctly, but the apps in the chroot will
just get immensely confused as they will be using narrow functions to
process input.

[1] the code says "Warn less when apt-get installing packqages"
[2] http://codereview.chromium.org/521053

BUG=None
TEST=entered chroot and saw LANG preserved from host env, and ran cbuildbot

Change-Id: I0490b992d5e2d7bfba945f787e65a59943b8d35c
Reviewed-on: http://gerrit.chromium.org/gerrit/5767
Reviewed-by: Anush Elangovan <anush@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
2011-08-11 19:39:23 -07:00
Elly Jones
dfd369431d update_bootloaders: use uuids for legacy boot.
BUG=chromium-os:14127,chromium-os:17184
TEST=Adhoc
Build a regular image (for Alex) and boot it on a Dogfood machine (which uses
legacy boot). Check dmesg for PARTUUID. Unpack partitions, mount partition 12,
and check for root=PARTUUID= in /efi/grub/grub.cfg (it should be there instead
of root=/dev/$linuxpart).

Change-Id: Ie403632f227d514a5876f8338afa4ad80708ed55
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/5668
Reviewed-by: Will Drewry <wad@chromium.org>
2011-08-11 13:57:49 -07:00
David James
4dd4c54ccb Fix race condition by using different symlinks for factory images.
Both the tests and archive_build read and write the latest symlink.
This CL fixes a race condition in archive_build by moving away from
using the latest symlink in archive_build.

BUG=chromium-os:18967
TEST=Run release cbuildbot archive_build on x86-mario-release.
     Verify it completes successfully and that latest symlink
     is unchanged now.

Change-Id: Ia32d20903f3ef74e360944fbabdd9834c0edb99a
Reviewed-on: http://gerrit.chromium.org/gerrit/5746
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-08-11 12:44:02 -07:00
Andrew de los Reyes
f984f20a3f sha256_partitions: script to easily sha256 partitions of a .bin file.
This script is short and should run on any Ubuntu system, outside the
chroot. This will help testers and release-engineers see the checksums
that .bin files contain.

BUG=chromium-os:18940
TEST=manually ran outside chroot

Change-Id: Ib742f946d0724c57d67e7bd1236f4a490d996313
Reviewed-on: http://gerrit.chromium.org/gerrit/5613
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
Tested-by: Andrew de los Reyes <adlr@chromium.org>
2011-08-11 10:00:12 -07:00
J. Richard Barnette
e4e3decfc0 Extract some common code for sharing with build_image, et al.
BUG=None
TEST=build regular, test, and recovery images

Change-Id: I2d7d073c27d14fb88be6a63953dcc77fc76a0807
Reviewed-on: http://gerrit.chromium.org/gerrit/5512
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2011-08-11 09:51:31 -07:00
Nick Sanders
2027ca8dfb Fix build break on factory install kernel
BUG=chrome-os-partner:6384
TEST=it can netboot

Change-Id: I5bc3a64f2a70b6740791a68e25cb44536236c41b
Reviewed-on: http://gerrit.chromium.org/gerrit/5538
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
Tested-by: Nick Sanders <nsanders@chromium.org>
2011-08-09 01:06:05 -07:00
Chris Masone
1b27797710 Make nssdb creation in /etc/fake_root_ca work on ARM
This command was failing before (and the error getting swamped in log output)
resulting in a later command auto-creating the database and failing to
set the trust settings of the fake SSL cert we use in tests correctly.

BUG=chrome-os-partner:4923
TEST=build arm-generic test image, see that there are no error messages after "Generating a 1024 bit RSA private key" in the output log

Change-Id: I5b0c05d90e528aa5047cb12aa338cef176d233aa
Reviewed-on: http://gerrit.chromium.org/gerrit/5519
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
2011-08-08 21:14:05 -07:00
Kris Rambish
de42359b74 Use vboot for signing
TEST=Ran it
BUG=chromium-os:18449
STATUS=Fixed

Change-Id: Ic09abe2b409f7da3b16eec0efb233063c9c5c178
Reviewed-on: http://gerrit.chromium.org/gerrit/5367
Tested-by: Kris Rambish <krisr@chromium.org>
Reviewed-by: Kris Rambish <krisr@chromium.org>
Reviewed-by: Nirnimesh <nirnimesh@chromium.org>
2011-08-08 15:19:34 -07:00
Elly Jones
377939c760 build_image: keep quiet about the autotest file list
This one rsync command is responsible for over 50% of the length of a typical
buildbot log file.

BUG=chromium-os:18830
TEST=Adhoc
Build a test image, both with build_image and mod_image_for_test.sh.

Change-Id: I8d35b605b0d3f1d0c97698bad815ed3dab36cba0
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/5460
2011-08-08 08:09:24 -07:00
David James
27064994fa Use pbzip2 in archive_build, if it is available.
BUG=chromium-os:16502
TEST=Run cbuildbot run of archive stage with official build.

Change-Id: I03898404dd27d6930c4a01bd9826796ddb07a4cd
Reviewed-on: http://gerrit.chromium.org/gerrit/5445
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-08-06 18:58:44 -07:00
J. Richard Barnette
46657a3d09 Delete support for build_image --preserve
A survey of users on chromium-os-dev@chromium.org found no-one uses
the option.  Furthermore, the option was meant merely for
performance improvements, but parallel emerge is more effective for
that purpose.

BUG=None
TEST=build and boot image

Change-Id: I42ab4a9cb5d70f1181915e957a5cc9fc4e3dbed7
Reviewed-on: http://gerrit.chromium.org/gerrit/5356
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-08-05 13:45:06 -07:00
David James
0ea96e4fd6 Install recovery kernel directly to image, not to build dir.
The recovery kernel is different from the regular kernel, and
should be kept sandboxed so that it does not mess with our
build directory.

We also need to give the recovery kernel a different PKGDIR so
that it does not overwrite our prebuilts for the ordinary
kernel.

BUG=chromium-os:18691
TEST=build_image --factory_install, mod_image_for_recovery.sh

Change-Id: I678a94305d5f30bc2c95bf4e53cfe81b2ae9d8ef
Reviewed-on: http://gerrit.chromium.org/gerrit/5305
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-08-05 12:33:15 -07:00
Chris Wolfe
d13775f60c Add /usr/src to masked install paths.
This makes it possible for chromeos-kernel to include a source tree,
without the tree being installed to CrOS images.

BUG=None
TEST=Manual with modified chromeos-kernel

Change-Id: I9f1193277638e828ffa116c5a397609a3246c80b
Reviewed-on: http://gerrit.chromium.org/gerrit/4760
Reviewed-by: Chris Wolfe <cwolfe@chromium.org>
Tested-by: Chris Wolfe <cwolfe@chromium.org>
2011-08-05 06:26:18 -07:00
Nick Sanders
daebdee972 Add HWID bundle into buildbot output
Add bundle to factory_shim/hwid

BUG=chromium-os:16734
TEST=archive_build, with and without hwid present

Change-Id: I70eeca5ce9cd99d229081a92af5bc0c3c997dce7
Reviewed-on: http://gerrit.chromium.org/gerrit/5189
Reviewed-by: Rong Chang <rongchang@chromium.org>
Tested-by: Nick Sanders <nsanders@chromium.org>
2011-08-03 23:16:41 -07:00
Gabe Black
83d8b824a3 Add some functions which will help transition to a single firmware directory.
This is the first in three stages of changes to transition from separate
"u-boot" and "coreboot" output directories to a single "firmware"
directory. This change/stage will give scripts a facility to easily be
compatible with both schemes. The second stage will transition all the
scripts to use the mechanism, and then finally the third stage will
switch the directories over. Some additional work may be needed if there
are file name clashes, although those sorts of problems would likely
already be present. These new functions could also optionally be taken back
out after the transition.

This change adds find_fw_component, find_u_boot_component, and
find_coreboot_component functions to common.sh. The first function provides
the bulk of the functionality, while the second and third wrap the first and
pass it parameters appropriate for u-boot or coreboot. The first function
accepts an override which it returns if set, and if not it checks for a file
in each of the directories it's passed in the second parameter one by one and
returns the first one it finds. If it doesn't find any it defaults to the last
one.

BUG=chromium-os:18107
TEST=Modified some u-boot related ebuilds to put some files in the firmware
directory. Ran the ebuilds and verified the files showed up where expected.
Modified the cros_write_firmware.sh script to use these new functions.  Ran
cros_write_firmware and saw it find the files in their new location.
Also verified that the firmware on the target was updated as expected.

Change-Id: Ic664e0e1c4a5b27efc9fb892ff5bd91cf1b5be56
Signed-off-by: Gabe Black <gabeblack@google.com>
Reviewed-on: http://gerrit.chromium.org/gerrit/5170
Tested-by: Gabe Black <gabeblack@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
2011-08-02 19:54:17 -07:00
J. Richard Barnette
9fb286ea02 Skip filling in kernel partition until cros_make_image_bootable
When first creating an image, vmlinuz was extracted from /boot, and
installed in its place in the kernel partition.  However, none of
that was necessary, because cros_make_image_bootable walks over the
same ground later in the build process.

BUG=chromium-os:17390
TEST=build_image, boot base and dev images on ZGB and legacy device
TEST=inspect build output directory, to confirm no stray artifacts

Change-Id: Iaf332b6603e0bcb17585adbc95a7b65bb8bfe790
Reviewed-on: http://gerrit.chromium.org/gerrit/5107
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
2011-08-01 16:09:00 -07:00
Richard Barnette
c6818be0da Revert "Skip filling in kernel partition until cros_make_image_bootable"
This reverts commit a966c2bad686f787268f01a7529851a1a2a6ad9b

Change-Id: I0990252c34853f7e1884da19b2649ed32e52e078
Reviewed-on: http://gerrit.chromium.org/gerrit/5091
Reviewed-by: Elly Jones <ellyjones@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-08-01 13:53:34 -07:00
Brian Harring
44da798e21 punt watchlists; no longer used
BUG=N/A
TEST=N/A

Change-Id: I86f1d29423659823ca177bbfa2dc287f36f01c33
Reviewed-on: http://gerrit.chromium.org/gerrit/4321
Reviewed-by: Anush Elangovan <anush@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2011-08-01 12:57:58 -07:00
J. Richard Barnette
77ec32a128 Skip filling in kernel partition until cros_make_image_bootable
When first creating an image, vmlinuz was extracted from /boot, and
installed in its place in the kernel partition.  However, none of
that was necessary, because cros_make_image_bootable walks over the
same ground later in the build process.

BUG=chromium-os:17390
TEST=build_image, boot base and dev images on ZGB and legacy device
TEST=inspect build output directory, to confirm no stray artifacts

Change-Id: Icd4902f5a823241f24eb64f68f80c8e5e5198341
Reviewed-on: http://gerrit.chromium.org/gerrit/4928
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-08-01 11:41:17 -07:00
Hung-Te Lin
1926874a19 make_factory_package: support --firmware in RMA (--usbimg) mode
Support using --firmware in RMA (usbimg) mode.
The firmware updater will be copied into partition 1 (statefu-partition) on RMA
shim.

BUG=chrome-os-partner:5000
TEST=./make_factory_package.sh --release RELEASE.bin --factory FACTORY.bin \
    --install FACTORY_INSTALLER.bin --firmware FIRMWAREUPDATER \
    --usbimg rma_image.bin
    # Then, mount partition 1 in rma_image.bin, find "chromeos-firmwareupdate"
    # file. Also a # "FACTORY_INSTALL_FIRMWARE=/mnt/stateful_partition/chromeos-firmwareupdate"
    # entry is found in dev_image/etc/lsb-factory file.

Change-Id: Id77ab7c8faa5f8e646a07c319d496947949ffa63
Reviewed-on: http://gerrit.chromium.org/gerrit/4637
Tested-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
2011-07-31 19:28:52 -07:00
David James
65d88bf903 Update cros_download_latest_image to work again.
cros_download_latest_image was using cros_gsdcurl, which has stopped working,
even when you use application-specific passwords. I've updated it to use
gsutil instead, which is more supported.

BUG=chromium-os:17512
TEST=Try downloading the latest x86-generic image.

Change-Id: Ia7b0704a8c7ff6897508c44e882f87bc44ccbe28
Reviewed-on: http://gerrit.chromium.org/gerrit/5031
Reviewed-by: Rahul Chaturvedi <rkc@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-07-29 15:23:49 -07:00
J. Richard Barnette
bb783007ce Delete dead code in build_image relating to ESP_LOOP_DEV
BUG=None
TEST=build_image

Change-Id: I10aba40538e9c6aeaf803ffc50bb029dcf6d4be5
Reviewed-on: http://gerrit.chromium.org/gerrit/5009
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-07-29 11:55:04 -07:00
David James
5ee38905e5 Don't rm $OUTDIR during archive_build.sh.
BUG=chromium-os:18382
TEST=Run sample buildbot run.

Change-Id: Ib6e81e9ec4b90b8f089da6de50bee1accafec9dd
Reviewed-on: http://gerrit.chromium.org/gerrit/4978
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-07-29 09:41:46 -07:00
Taylor Hutt
18594a87ba chromium-os:17680 : Establish Syncer PID file invariants
Details

  This change updates the use of '-s' to have the proper polarity to
  see if the syncer PID file is zero bytes in size.

  As a bit of other cleanup, if a file of size zero is found, it's
  removed.  This allows the second size test of the syncer PID file to
  be removed.

Testing

  o Created zero-length PID file: saw warning message on console.  Saw
    that the file contains a valid pin while the chroot is entered,
    and verified the PID file is deleted after exiting chroot.

       INFO    : You may have suffered from chromium-os:17680 and
       INFO    : could have stray 'enter_chroot.sh' processes running.
       INFO    : Erroneous PID file will be cleaned up.

  o Entered chroot with multiple terminals, saw that the PID file is
    removed & the syncer process is killed when the last client exits
    the chroot.

  o Executed './enter_chroot.sh --verbose', changed ownership of PID file to
    'root'.

    Exited and did not see the following diagnostic message:

       INFO    : Unable to clean up syncer process.

    The message should not be printed because the file is removed
    under 'sudo'.

BUG=chromium-os:17680
TEST=See above.
Signed-off-by: Taylor Hutt <thutt@chromium.org>

Change-Id: I055fe72b9322acee8c774401b3452ef502b30970
Reviewed-on: http://gerrit.chromium.org/gerrit/4936
Tested-by: Taylor Hutt <thutt@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
2011-07-28 13:46:52 -07:00
Kris Rambish
09e87bca4f Add a new script to add pyauto to test images
BUG=chromium-os:16876
TEST=Ran it
STATUS=Fixed

Change-Id: I4b9391371686d304d84528d1959adfff1a1d8e6c
Reviewed-on: http://gerrit.chromium.org/gerrit/4868
Tested-by: Kris Rambish <krisr@chromium.org>
Reviewed-by: Kris Rambish <krisr@chromium.org>
2011-07-28 11:48:55 -07:00
J. Richard Barnette
0fad3d7171 Make build_gpt.sh into a shell function library.
Converted build_library/build_gpt.sh from a shell script to a shell
library defining a function to replace the script.

Converted build_library/emit_gpt_scripts.sh into a shell function
that is defined in build_library/build_gpt.sh.

BUG=chromium-os:17390
TEST=build_image
TEST=inspect and run pack_ and unpack_partitions.sh on new image

Change-Id: Ifdcb58f492f871e120cbec9c67bdeab94d1a4d3f
Reviewed-on: http://gerrit.chromium.org/gerrit/4866
Reviewed-by: Vince Laviano <vlaviano@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-07-28 08:11:28 -07:00
Taylor Hutt
25bf949c63 chromium-os:17680 : Establish Syncer PID file invariants
Details

  I have a set of scripts which run from outside the chroot; they
  execute things inside the chroot, and therefore enter & exit the
  chroot frequently.

  This morning I noticed this morning that I was getting

    kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]

  on the console each time I exited the chroot.

  I tracked this down to the 'sync process' PID file; in my case, it
  was zero bytes in size.

  A little more detective work led to
  412b902b0e and I think I pieced
  together that I had suffered from chromium-os:17680, which resulted
  in an empty file.

  This change is my attempt at repairing the damage if
  chromium-os:17680 has been hit, and preventing other types of
  similar issues with the PID file.

  412b902b0e addressed the main cause of
  issue chromium-os:17680, and this change  addresses mysterious
  behavior in chroots that have suffered from it.

  To allow people to know that they've possibly got stray syncer
  porcesses running, I also output a message to the console which
  indicates such.

Testing

  o Created zero-length PID file: saw warning message on console.  Saw
    that the file contains a valid pin while the chroot is entered,
    and verified the PID file is deleted after exiting chroot.

       INFO    : You may have suffered from chromium-os:17680 and
       INFO    : could have stray 'enter_chroot.sh' processes running.
       INFO    : Erroneous PID file will be cleaned up.

  o Entered chroot with multiple terminals, saw that the PID file is
    removed & the syncer process is killed when the last client exits
    the chroot.

  o Executed './enter_chroot.sh --verbose', changed ownership of PID file to
    'root'.

    Exited and saw the following diagnostic message:

       INFO    : Unable to clean up syncer process.

BUG=chromium-os:17680
TEST=See above
Signed-off-by: Taylor Hutt <thutt@chromium.org>

Change-Id: Idf9059509bbc6e0a926efd549922d05b9b742fa0
Signed-off-by: Taylor Hutt <thutt@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/4837
2011-07-28 07:13:53 -07:00
Gaurav Shah
786ceeb389 Only use colored output if the terminal supports it
This also removes all the tput noise we are seeing in buildbot logs.
("tput: No value for $TERM and no -T specified", "tput: unknown terminal
"unknown", etc.)

BUG=none
TEST=manual below
src/platform/scripts$ TERM=unknown; ./enter_chroot (No more tput errors)

Change-Id: Ic74a0bd66d7e0cedcf7b65ec3a5f43b61b8276f9
Reviewed-on: http://gerrit.chromium.org/gerrit/4763
Tested-by: Gaurav Shah <gauravsh@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
2011-07-27 15:08:29 -07:00
Puneet Kumar
0fffd2bc75 Reenable crosbug12352_arm_kernel_signing flag as a dummy
This is being done so that the image parser run by the bots
can still parse images built with this flag

BUG=chromium-os:18299
TEST=Wait for bots to cycle green with old images

Change-Id: Ibebe62f0db60ab3a57e04964f872398028ee48c8
Reviewed-on: http://gerrit.chromium.org/gerrit/4824
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Puneet Kumar <puneetster@chromium.org>
Reviewed-by: Puneet Kumar <puneetster@chromium.org>
2011-07-27 11:37:23 -07:00
J. Richard Barnette
f2206118f3 Isolate scripts and files that are used only in build_image
A modest number of different scripts and files have a usage
restricted to build_image.  Move those files to build_library,
to prevent them accumulating unwanted clients.

BUG=chromium-os:17390
TEST=build_image
TEST=grep relevant sources for references to all the files

Change-Id: I3e9f6447ec34c09ea2d61f29ac343386a1e122b9
Reviewed-on: http://gerrit.chromium.org/gerrit/4762
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-07-27 08:46:01 -07:00
Che-Liang Chiou
611d5b7b0a Add a dummy flag to fix buildbot failure
BUG=chrome-os-partner:5177
TEST=./build_image -h | grep crosbug12352_arm_kernel_signing

Change-Id: Id61889f4e16ed7e4e774c00c3cb886811c5ccfd7
Reviewed-on: http://gerrit.chromium.org/gerrit/4805
Reviewed-by: Che-Liang Chiou <clchiou@chromium.org>
Tested-by: Che-Liang Chiou <clchiou@chromium.org>
2011-07-27 01:05:37 -07:00
Che-Liang Chiou
e51bdf284c Obsolete --crosbug12352_arm_kernel_signing flag
This reverts commit 451f36e4a8.

Last time I removed the --crosbug12352_arm_kernel_signing flag, buildbot
failed. The reason seemed to be that buildbot still passing this flag to
build_image. However, I cannot find anywhere in the log that indicates
buildbot did pass this flag to build_image. So I think the last failure
should be transient and it is good to obsolete this flag.

BUG=chromium-os:12352
TEST=build_image
TEST=load_kernel_test -b 2 /path/to/image /path/to/recovery_key.vbpubk

Change-Id: Ic757eb2dc4304e7205b483063335f8816b536433
Reviewed-on: http://gerrit.chromium.org/gerrit/4794
Reviewed-by: Che-Liang Chiou <clchiou@chromium.org>
Tested-by: Che-Liang Chiou <clchiou@chromium.org>
2011-07-26 21:40:13 -07:00
J. Richard Barnette
5b80a0b22c Incorporate the contents of create_esp.sh inline in build_image
Stripped of its boilerplate, create_esp.sh was a one-line
script called from only one place in the source.  Replace the
script with the one command it represented.

BUG=chromium-os:17390
TEST=build_image
TEST=grep all relevant sources for "create_esp"

Change-Id: Iaa4be285066b524ff169f5185c455e2566177648
Reviewed-on: http://gerrit.chromium.org/gerrit/4756
Reviewed-by: Will Drewry <wad@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-07-26 16:16:47 -07:00
J. Richard Barnette
8447777820 Simplify standard boilerplate for selected scripts
This shortens the standard boilerplate for finding and sourcing
shell function libraries for build_image, mod_image_for_test.sh and
mod_image_for_recovery.sh.

As a side effect of the change, both mod_image_for_test.sh and
mod_image_for_recovery.sh will now restart inside the chroot if
invoked from outside; this is consistent with the pre-existing
behavior of build_image.

BUG=None
TEST=run the three scripts, from both inside and outside the chroot

Change-Id: Idd91cbee323346a871b49deea31a76875f5ee3c4
Reviewed-on: http://gerrit.chromium.org/gerrit/4675
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Vince Laviano <vlaviano@chromium.org>
2011-07-26 14:05:43 -07:00
J. Richard Barnette
1fe3c57d18 scripts: Delete administrative files obviated by gerrit
BUG=None
TEST=None

Change-Id: I5b707c1fff048d8b6e64bfc668b3e7d3ccd3e7a4
Reviewed-on: http://gerrit.chromium.org/gerrit/4746
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Darin Petkov <petkov@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
2011-07-26 11:30:13 -07:00
J. Richard Barnette
f16e1028d6 Delete resize_stateful_partition.sh
The file was written as a script, but in fact was merely a shell
library of mostly unused functions.  Remove the file, and move the
one function actually used to the one place it was used in
mod_image_for_recovery.sh.

BUG=None
TEST=run mod_image_for_recovery.sh
TEST=grep all relevant scripts for references to the deleted file

Change-Id: I424f6986089467d2b8cd95bb033851171c1d1e3a
Reviewed-on: http://gerrit.chromium.org/gerrit/4681
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
2011-07-26 10:17:12 -07:00
David James
4e098db36f Close fds in RunCommand to prevent children from inheriting file handles.
RunCommand should only pass in stdin, stdout, and stderr to processes,
not other file handles. Fixing this prevents issues where child
processes can cause the parent to hang by forgetting to close the
file handle when launching daemons.

BUG=chromium-os:18104
TEST=Buildbot run with smoke suite. Unit tests.

Change-Id: Ib85095e1fb592ef05a972a5412348363049e6d86
Reviewed-on: http://gerrit.chromium.org/gerrit/4673
Reviewed-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: David James <davidjames@chromium.org>
2011-07-25 13:31:30 -07:00
J. Richard Barnette
f5029789d6 Remove unsued code from common.sh
BUG=None
TEST=setup_board, build_packages, build_image
TEST=grep all sources for the unused functions

Change-Id: Iaac5589aaba6f6f48aa00cdab76a7765f8d931ce
Reviewed-on: http://gerrit.chromium.org/gerrit/4580
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
2011-07-22 14:39:39 -07:00
Brian Harring
d5d5dbffa1 Fix/standardize exiting if common.sh can't be found
The problem here is that most were doing their exiting w/in a subshell;
exit within a subshell kills the subshell, not the parent.  Not all scripts
were using set -e (which would pick up the failing subshell); as such
just rewriting them to remove the potential via eliminateing the subshelling.

Beyond that, removed a couple of custom (working, although non-standard)
approaches, and removed a duplicate common.sh sourc'ing w/in mk_memento_images.sh

TEST=force 'find_common_sh' to fail, note the scripts fails to exit
BUG=none

Change-Id: Ia1108a091a6399ad6aedd3cade4a107f4411686c
Reviewed-on: http://gerrit.chromium.org/gerrit/3905
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2011-07-22 12:06:59 -07:00
Brian
242212311d file_copy is no longer used, thus remove it
Last user of it was removed via 4364a2

TEST=N/A
BUG=None

Change-Id: Id43c5571a823d6d56ca2f20c138bc0ed9de3d1d0
Reviewed-on: http://gerrit.chromium.org/gerrit/3893
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
2011-07-22 11:59:32 -07:00
David James
8b9643f7d8 Fix losetup race condition.
The following pattern creates a race condition:

   LOOP_DEV=$(sudo losetup -f)
   ...
   sudo losetup "${LOOP_DEV}" "${ROOT_FS_IMG}"

If two steps similar to the above run in parallel, and both steps pick up
the same free loop device, they may try to mount different images with the
same loop device and one of the two stages will get a "device is busy" error.

To fix this, we should switch to the following pattern:
   LOOP_DEV=$(sudo losetup --show -f "${ROOT_FS_IMG}")

This CL implements the above.`

BUG=chromium-os:18046
TEST=Run buildbot run. Test case where we run out of loop devices and verify
     logic still works.

Change-Id: Ie457701fda61e5fc3b9112c1bfef9fb9713ea265
Reviewed-on: http://gerrit.chromium.org/gerrit/4555
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Daniel Kurtz <djkurtz@chromium.org>
2011-07-22 00:02:53 -07:00