This also removes all the tput noise we are seeing in buildbot logs.
("tput: No value for $TERM and no -T specified", "tput: unknown terminal
"unknown", etc.)
BUG=none
TEST=manual below
src/platform/scripts$ TERM=unknown; ./enter_chroot (No more tput errors)
Change-Id: Ic74a0bd66d7e0cedcf7b65ec3a5f43b61b8276f9
Reviewed-on: http://gerrit.chromium.org/gerrit/4763
Tested-by: Gaurav Shah <gauravsh@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
This is being done so that the image parser run by the bots
can still parse images built with this flag
BUG=chromium-os:18299
TEST=Wait for bots to cycle green with old images
Change-Id: Ibebe62f0db60ab3a57e04964f872398028ee48c8
Reviewed-on: http://gerrit.chromium.org/gerrit/4824
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Puneet Kumar <puneetster@chromium.org>
Reviewed-by: Puneet Kumar <puneetster@chromium.org>
A modest number of different scripts and files have a usage
restricted to build_image. Move those files to build_library,
to prevent them accumulating unwanted clients.
BUG=chromium-os:17390
TEST=build_image
TEST=grep relevant sources for references to all the files
Change-Id: I3e9f6447ec34c09ea2d61f29ac343386a1e122b9
Reviewed-on: http://gerrit.chromium.org/gerrit/4762
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
This reverts commit 451f36e4a8.
Last time I removed the --crosbug12352_arm_kernel_signing flag, buildbot
failed. The reason seemed to be that buildbot still passing this flag to
build_image. However, I cannot find anywhere in the log that indicates
buildbot did pass this flag to build_image. So I think the last failure
should be transient and it is good to obsolete this flag.
BUG=chromium-os:12352
TEST=build_image
TEST=load_kernel_test -b 2 /path/to/image /path/to/recovery_key.vbpubk
Change-Id: Ic757eb2dc4304e7205b483063335f8816b536433
Reviewed-on: http://gerrit.chromium.org/gerrit/4794
Reviewed-by: Che-Liang Chiou <clchiou@chromium.org>
Tested-by: Che-Liang Chiou <clchiou@chromium.org>
Stripped of its boilerplate, create_esp.sh was a one-line
script called from only one place in the source. Replace the
script with the one command it represented.
BUG=chromium-os:17390
TEST=build_image
TEST=grep all relevant sources for "create_esp"
Change-Id: Iaa4be285066b524ff169f5185c455e2566177648
Reviewed-on: http://gerrit.chromium.org/gerrit/4756
Reviewed-by: Will Drewry <wad@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
This shortens the standard boilerplate for finding and sourcing
shell function libraries for build_image, mod_image_for_test.sh and
mod_image_for_recovery.sh.
As a side effect of the change, both mod_image_for_test.sh and
mod_image_for_recovery.sh will now restart inside the chroot if
invoked from outside; this is consistent with the pre-existing
behavior of build_image.
BUG=None
TEST=run the three scripts, from both inside and outside the chroot
Change-Id: Idd91cbee323346a871b49deea31a76875f5ee3c4
Reviewed-on: http://gerrit.chromium.org/gerrit/4675
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Vince Laviano <vlaviano@chromium.org>
The file was written as a script, but in fact was merely a shell
library of mostly unused functions. Remove the file, and move the
one function actually used to the one place it was used in
mod_image_for_recovery.sh.
BUG=None
TEST=run mod_image_for_recovery.sh
TEST=grep all relevant scripts for references to the deleted file
Change-Id: I424f6986089467d2b8cd95bb033851171c1d1e3a
Reviewed-on: http://gerrit.chromium.org/gerrit/4681
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
RunCommand should only pass in stdin, stdout, and stderr to processes,
not other file handles. Fixing this prevents issues where child
processes can cause the parent to hang by forgetting to close the
file handle when launching daemons.
BUG=chromium-os:18104
TEST=Buildbot run with smoke suite. Unit tests.
Change-Id: Ib85095e1fb592ef05a972a5412348363049e6d86
Reviewed-on: http://gerrit.chromium.org/gerrit/4673
Reviewed-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: David James <davidjames@chromium.org>
BUG=None
TEST=setup_board, build_packages, build_image
TEST=grep all sources for the unused functions
Change-Id: Iaac5589aaba6f6f48aa00cdab76a7765f8d931ce
Reviewed-on: http://gerrit.chromium.org/gerrit/4580
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
The problem here is that most were doing their exiting w/in a subshell;
exit within a subshell kills the subshell, not the parent. Not all scripts
were using set -e (which would pick up the failing subshell); as such
just rewriting them to remove the potential via eliminateing the subshelling.
Beyond that, removed a couple of custom (working, although non-standard)
approaches, and removed a duplicate common.sh sourc'ing w/in mk_memento_images.sh
TEST=force 'find_common_sh' to fail, note the scripts fails to exit
BUG=none
Change-Id: Ia1108a091a6399ad6aedd3cade4a107f4411686c
Reviewed-on: http://gerrit.chromium.org/gerrit/3905
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Last user of it was removed via 4364a2
TEST=N/A
BUG=None
Change-Id: Id43c5571a823d6d56ca2f20c138bc0ed9de3d1d0
Reviewed-on: http://gerrit.chromium.org/gerrit/3893
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
The following pattern creates a race condition:
LOOP_DEV=$(sudo losetup -f)
...
sudo losetup "${LOOP_DEV}" "${ROOT_FS_IMG}"
If two steps similar to the above run in parallel, and both steps pick up
the same free loop device, they may try to mount different images with the
same loop device and one of the two stages will get a "device is busy" error.
To fix this, we should switch to the following pattern:
LOOP_DEV=$(sudo losetup --show -f "${ROOT_FS_IMG}")
This CL implements the above.`
BUG=chromium-os:18046
TEST=Run buildbot run. Test case where we run out of loop devices and verify
logic still works.
Change-Id: Ie457701fda61e5fc3b9112c1bfef9fb9713ea265
Reviewed-on: http://gerrit.chromium.org/gerrit/4555
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Daniel Kurtz <djkurtz@chromium.org>
This allows the buildbot to calculate the change identifier and pass it into
archive_build. This makes it easier for us to parallelize the archive_build
step.
BUG=chromium-os:12220
TEST=Verify this argument works. Verify archive_build works without the argument
too.
Change-Id: I6b757cc7795fb6f2f24a400502fe6f70416ab44f
Reviewed-on: http://gerrit.chromium.org/gerrit/4440
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: David James <davidjames@chromium.org>
BUG=chromium-os:17722
TEST=Run sample archive_build with --debug mode and verify location of gsutil.
Change-Id: Idf39bfc9a80cdcd5f25d63f4839e9168f593adfd
Reviewed-on: http://gerrit.chromium.org/gerrit/4264
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Sometimes, two processes might start the env_syncer at the same time.
If this occurs, then the pid file will get overwritten and one of the
processes won't be killed.
I've also fixed the env_syncer to properly daemonize, such that
cbuildbot won't wait on the env_syncer to be killed before exiting.
This requires closing stdin, stderr, and stdout.
BUG=chromium-os:17680
TEST=Reproduce hang, and verify that this fixes the hang. Verify that
multiple chroots can be entered at the same time and that the
env_syncer is killed when the last chroot exits. Verify that
cbuildbot is not kept hanging even if the env_syncer doesn't
exit.
Change-Id: I3cb9fbdf47a406e818e12cf91593b8e3481aa578
Reviewed-on: http://gerrit.chromium.org/gerrit/4232
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: David James <davidjames@chromium.org>
enter_chroot is often called from multiple processes and we expect that
it doesn't modify anything unless it's within a lock. This CL moves
chroot modifications up to occur during the lock so that we don't have
multiple modifications to .gitconfig occurring at the same time.
This prevents the following warning:
error: could not lock config file
/b/cbuild/chroot/home/chrome-bot/.gitconfig: File exists
See http://chromeos-botmaster.mtv.corp.google.com:8026/builders/TOT%20Pre-Flight%20Queue/builds/3771/steps/Test/logs/stdio
BUG=chromium-os:17661
TEST=Run enter_chroot.sh and verify gitconfig is still setup.
Change-Id: I73a7755d62cce895c76b8e0f35838b3874e5db33
Reviewed-on: http://gerrit.chromium.org/gerrit/4208
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: David James <davidjames@chromium.org>
This change addresses a defect which prevents simultaneous use of
'cros_image_to_target.py'.
The issue is that the default invocation will always use the same port
number of HTTP communication. This change, with default parameters,
will now start at the same port number, but will now instead iterate
through a number of ports (the number of ports is set to 10).
If no available port can be found, the script will fail with an
appropriate message.
If the user specifies a specific port, then only that port number will
be used; if the port is not available, the script will fail with an
approrpriate message.
BUG=chromium-os:17558
TEST=Uploaded two images simultaneously to different boards.
Change-Id: Icbbea12468fd766897daeecb0b7f1d3929f52a1c
Signed-off-by: Taylor Hutt <thutt@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/4110
Reviewed-by: Paul Stewart <pstew@chromium.org>
Reviewed-by: Terry Lambert <tlambert@chromium.org>
To make "USB installation image" (useful for RMA images), make_factory_package
now supports --usbimg and --install_shim for the new type of image.
Some internal refactoring is also included in this CL to ease adding new
generation methods.
BUG=chrome-os-partner:4108
TEST=make_factory_package --usbimg rma.bin --factory PATH_TO_FACTORY \
--release PATH_TO_RELEASE --install PATH_TO_INSTALL_SHIM
# then turn on developer switch, hold recovery button to boot by the
# rma.bin. Seeing installation from USB works fine.
Change-Id: I65789bfea92cf04e89ea2b3319fc9beb59a17bd8
Reviewed-on: http://gerrit.chromium.org/gerrit/4026
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
Some images (especially RMA) need to use stateful or other lagacy stuff from
other image files.
This CL adds "-m" to make_universal_factory_shim so that we can change the
master image (the one for all legacy / stateful partition).
BUG=chrome-os-partner:4108
TEST=./make_universal_factory_shim.sh -m factory_test.bin \
factory_install.bin factory_test.bin release_image.bin
Change-Id: I6c8e096fd7ae921a4fb157c035f575a7c3b33834
Reviewed-on: http://gerrit.chromium.org/gerrit/4020
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
by mobile-providers.
BUG=none
TEST=./build_packages
Change-Id: I07a2fc91f0782d6a03f535333a8925f9fe6283c1
Reviewed-on: http://gerrit.chromium.org/gerrit/3973
Tested-by: Nathan J. Williams <njw@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Currently, enter_chroot.sh uses a hack to ensure that "--" is preserved in
commands. This is only necessary if you want to run commands that start with
hyphens. E.g. if you wanted to run a command called "--foo", enter_chroot
would break. But we don't have any existing commands that start with hyphens,
so this logic seems unnecessary.
This also makes enter_chroot more flexible, i.e. we now support case #6 below,
which was not supported previously.
Here are use cases:
1. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2 -- cmd arg1 arg2
Set env vars and run cmd w/ args
2. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2
Set env vars and run shell
3. ./enter_chroot [chroot_flags] -- cmd arg1 arg2
Run cmd w/ args
4. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2 cmd arg1 arg2
Like #1 _if_ args aren't flags (if they are, enter_chroot will claim them)
5. ./enter_chroot [chroot_flags] cmd arg1 arg2
Like #3 _if_ args aren't flags (if they are, enter_chroot will claim them)
6. ./enter_chroot [chroot_flags] -- VAR1=val1 VAR2=val2 cmd arg1 arg2
Set env vars and run cmd w/ args
7. ./enter_chroot
Just enter the chroot.
BUG=chromium-os:17468
TEST=Test above cases
Change-Id: I1801ac3927aacddd6d556c5939d3a42b31252675
Reviewed-on: http://gerrit.chromium.org/gerrit/3910
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: David James <davidjames@chromium.org>
When building a VM or USB-run image the boot args come from a preset template
which does not include the cros_debug flag. If this flag is present we maintain
it in this new boot configuration. The way we do it is by replacing the string
"cros_legacy" (which originates with the template) with
"cros_legacy cros_debug".
TEST=Build VM from a developer and test image and check grep "cros_debug"
/proc/cmdline.
BUG=chromium-os:17392
Change-Id: I2cb578274cf9596bf863ec835b1c921fff252a04
Reviewed-on: http://gerrit.chromium.org/gerrit/3808
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
Tested-by: Arkaitz Ruiz Alvarez <arkaitzr@chromium.org>
By default, "cros_debug" should be included in the kernel commandline for
developer and test images. This change adds "cros_debug" to the kernel
commandline for test images when using the deprecated mod_image_for_test.sh
and syncs this file with previous change to build_image:
http://gerrit.chromium.org/gerrit/#change,3790
TEST=Use mod_image_for_test.sh to create a test image, boot it and check
/proc/cmdline for "cros_debug".
BUG=chromium-os:17393
Change-Id: I7774dec80707f702036eff0f6277837858b64182
Reviewed-on: http://gerrit.chromium.org/gerrit/3897
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Arkaitz Ruiz Alvarez <arkaitzr@chromium.org>
By default, "cros_debug" should be included in the kernel commandline for
developer images. This change adds "cros_debug" to the kernel commandline
for test images, which are based on developer images but overwrite the boot
arguments.
TEST=Build test image, install on machine and grep "cros_debug" /proc/cmdline
BUG=chromium-os:17393
Change-Id: Ie0de11baf60a3a69a7fef0639247e2edae455ffb
Reviewed-on: http://gerrit.chromium.org/gerrit/3790
Tested-by: Arkaitz Ruiz Alvarez <arkaitzr@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
David James removed docgen from the chroot, but missed this script that
invokes it during Release builds. This is causing all Canary builds to
fail during archive_build.
TEST=None (not sure how to test this outside of the release build process)
BUG=None
Change-Id: I922ce30722ef67601a07ea75604850eb2c8fed28
Reviewed-on: http://gerrit.chromium.org/gerrit/3737
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
BUG=chromium-os:15989
TEST=build an image and mod it for test. See that there's no complaint about an 'Invalid Password'. Run suite_Smoke
STATUS=Fixed
Change-Id: I34bb47576ab8d5df11e79489f4996897ebf2ca3f
Reviewed-on: http://gerrit.chromium.org/gerrit/3694
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott James Remnant <keybuk@chromium.org>
BUG=chromium-os:14756
TEST=ran it on cr48 and Latitiude 13
Change-Id: I3c46337f0e4741786ec980100cdebc24fdc02ef9
Reviewed-on: http://gerrit.chromium.org/gerrit/3093
Reviewed-by: Paul Taysom <taysom@google.com>
Tested-by: Paul Taysom <taysom@google.com>
Since now the arm firmware can parse %U as x86 bios, and kernel can
parse PARTNROFF=%d, we are able to generate kernel command line with
such construct.
BUG=chromium-os:14022,15683
TEST=manual
1. Build image with root filesystem verification turns off
2. Boot successfully
3. Run 'cat /proc/cmdline' and validate its output
4. Run 'rootdev' and validate its output
Change-Id: I11de0a30928efe9d9b0149feb3389a2f30063516
Reviewed-on: http://gerrit.chromium.org/gerrit/1104
Tested-by: Che-Liang Chiou <clchiou@chromium.org>
Reviewed-by: <nsanders@google.com>
Reviewed-by: Che-Liang Chiou <clchiou@chromium.org>
BUG=chromium-os:11744
TEST=Built test images with and without the standard backdoor.
Could log in with it.
Could not without.
Change-Id: I23fa55cbb1287f9385b0589b04f4c006903b9eef
Reviewed-on: http://gerrit.chromium.org/gerrit/3530
Tested-by: David Rochberg <rochberg@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
The '500populateQualDbs' script is now deprecated by the new chromeos-hwid
ebuild package.
HWQual database are merged into images automatically, no more need for
post-processing.
QUALDB and --qualdb are also removed, with typo in comments fixed.
BUG=chrome-os-partner:4276
TEST=./build_image --factory
Change-Id: I76d9a72943567444e26200fccc6fe5dff95b2687
Reviewed-on: http://gerrit.chromium.org/gerrit/3431
Reviewed-by: Vince Laviano <vlaviano@chromium.org>
Tested-by: Hung-Te Lin <hungte@chromium.org>
The default is --standard_backdoor, which installs well-known ssh keys and sets
a well-known root password. Passing --nostandard_backdoor will cause
mod_image_for_test to use ssh keys from ~/.ssh/*.pub instead of the test keys
and not set the root password.
BUG=chromium-os:11744
TEST=Adhoc
Build an image with --standard_backdoor.
ssh -i ${SRC}/src/scripts/mod_for_test_scripts/ssh_keys/testing_rsa root@${DUT}
ssh root@${DUT} with 'test0000'
cat /root/.ssh/authorized_keys # check for the test key
Build an image with --nostandard_backdoor.
ssh -o PubkeyAuthentication=no root@${DUT} # this will fail
ssh root@${DUT} # this should work
cat /root/.ssh/authorized_keys # check for just your keys
Change-Id: Ie92fbc9d3815f478698c8c94d938daca2b5cd53e
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/3449
Reviewed-by: David Rochberg <rochberg@chromium.org>
With Portage 2.1.10, the number of spaces in the output changed. This broke
the get_package_list and caused failures during unit tests. To fix this,
I've updated the script to handle this.
BUG=chromium-os:17151
TEST=Verify that get_package_list works now.
Change-Id: I2a334451807abdbeb9fb5cbb99a54ab75f234330
Reviewed-on: http://gerrit.chromium.org/gerrit/3415
Reviewed-by: Anush Elangovan <anush@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Every developer image (and test images based on the developer image) should
include "cros_debug" in the kernel commandline. This flag is used by the
crossystem application to determine if the image being run is a developer
image. cros_make_image_bootable receives a new flag, --force_developer_mode
that appends "cros_debug" to the boot args.
BUG=chromium-os:16951
TEST=build images. Flash a base image and confirm the absence of "cros_debug"
in /proc/cmdline. Flash the developer image and confirm the existence of the
string.
Change-Id: I9f748638b5dac384be991908239e8b447ddf3b5e
Reviewed-on: http://gerrit.chromium.org/gerrit/3169
Tested-by: Arkaitz Ruiz Alvarez <arkaitzr@chromium.org>
Reviewed-by: Gaurav Shah <gauravsh@chromium.org>
libc debug symbols (libpthread in particular) are needed to debug threaded software with gdb. This CL copies
debug symbols for glibc when building a dev image.
This CL also removes an old hack related to selecting the correct path to the glibc tarball.
BUG=chromium-os:16847
TEST=Build dev image (--withdev is the default). Checked that debug info is copied. Booted image and checked
the debugging threaded programs works with gdb. Build non-dev image (--nowithdev) checked that debug info is
not copied.
Change-Id: Id908e8260b1431ed1256a13dfb86b05ac242f263
Reviewed-on: http://gerrit.chromium.org/gerrit/3091
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Raymes Khoury <raymes@chromium.org>
This will prevent nasty surprises when it fails silently and one ends up with a
non-bootable recovery image.
BUG=chromium-os:16366
TEST=Adhoc
Hack /usr/lib/crosutils/build_kernel_image.sh to exit 1, run
mod_image_for_recovery
Change-Id: If23d181959aecf75044cb4581e25f8851d1b9840
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/3097
Reviewed-by: Will Drewry <wad@chromium.org>
BUG=None
TEST=run with a "--to" path that is the default location where the image is generated.
Verify the final image is not 0 bytes.
Change-Id: Ib9a7feceb17054362f00a813c369564e753cd3ab
Reviewed-on: http://gerrit.chromium.org/gerrit/2988
Tested-by: Dave Parker <dparker@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
Additional fix needed on top of nsanders commit from yesterday:
http://gerrit.chromium.org/gerrit/#change,2028
ARM generic doesn't build chromeos-u-boot and thus has no u-boot
directory in the $SYSROOT.
BUG=chromium-os:16833
TEST=archive_build.sh
Change-Id: Ic8422419779dca1965c6832a4a0bd73078945e69
Signed-off-by: Grant Grundler <grundler@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/3001
Reviewed-by: Anton Staaf <robotboy@chromium.org>
Reviewed-by: Nick Sanders <nsanders@chromium.org>
