This change removes /usr/sbin/write_gpt.sh from images which we have no
use for. This allows us to drop the indirection of writing partition
tables by first writing out a script to call. Now cgpt.py can call cgpt
directly to initialize the partition layout. This opens the way for
further improvements to how disk images are created.
Trying to include version info by adding the directory name to VM image
names didn't work and a better solution is for build_image to write out
a version.txt file. This should also fix an issue where uploading from
image_to_vm.sh didn't always go to the same location as the images
uploaded from build_image did.
The old script was heading towards spaghetti code realm. This breaks up
all the image variations such as hybrid MBR, OEM packages, etc into
configuration options and small functions that actually do the work.
All this is in the new vm_image_util.sh library but the command line
parsing and overall procedure remains in image_to_vm.sh
As part of this we gain support for putting some qemu options in a
config file as well as Xen virtual machines using pygrub and pvgrub.
Lots of generally unused options have been removed to simplify things
and keep output file names consistent.
Make use of the new partition UUIDs for ROOT-A and ROOT-B in the root=
kernel parameters provided by the legacy (non-kexec) bootloaders. This
makes all of our images bootable as-is without having to pass them
through image_to_vm.sh. :-D
As outlined here we need a new partition layout, this patch makes the
necessary changes:
https://groups.google.com/forum/#!topic/coreos-dev/bA7gwGGoTng
The first big change is making all of the scripts obey partition numbers
based on labels in the disk_layout.json. This makes it much easier to
change later on.
The second big change is in the layout itself. The json file was updated
to reflect the document above.
And finally the grub boot configuration needed for pv-grub and pygrub
were added to the create_legacy_bootloader_templates.sh library utlity.
Everything seems to work and boot now.
xen requires a /boot/grub/menu.lst for pygrub and pvgrub on partition 1.
Put it on the stateful partition for now and come back around and fix
this up when we redo the partition layout.
mod_image_for_test.sh doesn't work anymore so nobody should be using it.
There are a few places where scripts try to use mod_image_for_test.sh,
and these are timebombs because they fail if a test image needs to actually
be produced.
BUG=chromium-os:31183
TEST=Tested that this script doesn't produce any images anymore, so
it should be fine to delete it.
Change-Id: If80337407023d62f76117dc44cadfa46801ca236
Reviewed-on: https://gerrit.chromium.org/gerrit/40955
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Moving to the cirrus driver (instead of std) so we can support
cirrus-fb, in preparation for nonroot-X support.
BUG=chromium-os:388
TEST=local build, run with x86-generic
Change-Id: Ic7d5776f673ac3c15680d62331345d55985f1d73
Reviewed-on: https://gerrit.chromium.org/gerrit/39900
Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: John Sheu <sheu@chromium.org>
Commit-Queue: John Sheu <sheu@chromium.org>
This allows you to build a VM image using the 4gb-rootfs layout
BUG=none
TEST=Build iamge using 4gb-rootfs and build VM image
Change-Id: I1efa45d1448b22a9905dae764f0e4cdc0538be68
Reviewed-on: https://gerrit.chromium.org/gerrit/37489
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
This change adds support for building the disk layout from a
configuration file. It also cleans up much of the image creation
code.
install_gpt no longer exists, and has been replaced by cgpt.py's
write action. This spits out a file that has two functions that can
be called to write a partition layout to a disk/file. This gets rid
of the gigantic nest of calculations that built the layout previously.
All instances of partition/filesystem sizes in build scripts should now
be gone in favour of calls to the cgpt.py tool.
create_boot_desc has moved inside the base image creation, in an effort
to simplify build_image.
load_kernel_test is gone since it's apparently not supposed to be called
here anyway (asked wfrichar/rspangler about this one).
Base image creation now uses files rather than loop devices when
building an image. This means we can simply umount them once we're
done and not worry about cleaning up the loop device, since it's
been done for us.
Hash pad calculation has been removed. This is now set manually inside
the partition config file.
Hybrid MBR creation is gone, since it's now possible to do that in a board
specific hook (see overlay-beaglebone/scripts/board_specific_setup.sh).
OEM partition now has a filesystem, which is mounted at /usr/share/oem
during emerge so that packages can stash files here.
root_fs_dir and friends are still globals, but the long-term idea
is to make this not the case.
BUG=chromium-os:33817
TEST=All types of images and their respective flows
(VM, recovery, test, factory etc)
Change-Id: I8a596728a4d1845c930e837bea627f5b6a11c098
Reviewed-on: https://gerrit.chromium.org/gerrit/29931
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
Reviewed-by: Liam McLoughlin <lmcloughlin@chromium.org>
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
Specifically, detect if the umount failed due to current access, if so,
give it up to 9 more runs (w/ 1s pauses) continuing only if it's still
failing due to currently open files.
Via this, it should suppress the race of gvfs/trashd looking at
quick mounted/umounted pathways.
This CL is a two parter; this adds the script, and converts common.sh
consumers over to using the override.
The next CL will modify the chroot itself to ensure our script gets
picked up/used.
BUG=chromium-os:23443
TEST=trybot, manul validation.
Change-Id: I92dedd91d6133c2063b1e5dbbc1a68366844801d
Reviewed-on: https://gerrit.chromium.org/gerrit/32087
Commit-Ready: Brian Harring <ferringb@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
fixup_image_for_qemu.py exists only to add an xorg.conf; however, Xorg's
autodetection has handled qemu/kvm just fine for quite a while now, with
or without -vga std. Drop the script and the call to it from
image_to_vm.sh.
BUG=None
TEST=Build an image, confirm that it still works fine in kvm and
autodetects an appropriate resolution.
Change-Id: I9f550fef1eebe7387d232795105ec47bfa908f22
Reviewed-on: https://gerrit.chromium.org/gerrit/31181
Tested-by: Josh Triplett <josh@joshtriplett.org>
Reviewed-by: Rahul Chaturvedi <rkc@chromium.org>
Commit-Ready: Josh Triplett <josh@joshtriplett.org>
There is a bug in bash where BASH_COMMAND is not correct when an error
is hit in a subshell (it retains the command run *before* the subshell).
Since this has confused a few people using image_to_vm.sh so far, drop
the use of subshells in this file. They weren't necessary in the first
place so it isn't like we're losing anything.
URL: http://lists.gnu.org/archive/html/bug-bash/2012-08/msg00052.html
BUG=None
TEST=`./image_to_vm.sh --board=x86-alex --from=../build/images/x86-alex/latest` still works
TEST=`rm ../build/images/x86-alex/latest/chromiumos_image.bin && ./image_to_vm.sh --board=x86-alex --from=../build/images/x86-alex/latest/` reports correct line
Change-Id: Iaa7560b4868128119a4f6223b54020611f203635
Reviewed-on: https://gerrit.chromium.org/gerrit/31135
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Rather than forcing all consumers of DEFAULT_BOARD to remember to call
get_default_board, just do it for them automatically.
BUG=None
TEST=`cbuildbot {arm,amd64,x86}-generic-full` works
TEST=`./build_packages --help` shows correct default
Change-Id: I8d6ccb83babb2764a50692318eb9193c45fb3b39
Reviewed-on: https://gerrit.chromium.org/gerrit/17868
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Currently, the scripts in src/scripts have multiple implementations
for handling when common.sh fails to load, some of which are buggy.
To simplify the boilerplate, these scripts now just exit if common.sh
fails to load. The shell itself will print the following message if
common.sh is not found:
/usr/lib/crosutils/common.sh: No such file or directory
BUG=chromium-os:32442
TEST=Run these scripts with and without common.sh installed.
Change-Id: Ie54420b6c649774f9cb039c14c80f4cf6c6ebc07
Reviewed-on: https://gerrit.chromium.org/gerrit/27058
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why. This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.
Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down. This includes
outputing the relevant stack trace, or at least what we can get of
it.
The 'die' function is now enhanced to automatically dump the trace
that lead to it. For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise. For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.
Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).
Before:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
<no output at all, just exit code 1>
With this CL:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
ERROR : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR : Backtrace: (most recent call is last)
ERROR : file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR : file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR :
ERROR : Command failed:
ERROR : Command '/bin/false' exited with nonzero code: 1
BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote
Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
The -d option to umount is unnecessary in cases where the original
mount was created with '-o loop' (it says so in the man page).
Moreover, there's a demonstrable bug when using 'umount -d' in this
case: umount complains that it's unable to delete the loop device
in this case. This change silences those messages.
The demonstrated bug in umount is a plausible explanation for build
failures characterized by unexpected failures of 'mount -o loop';
see the BUG, below.
BUG=chromium-os:24975
TEST=run the changed commands, see no loopback devices leaked
Change-Id: I1df9f60b40eabd279164b79ee66994ff9f644ee9
Reviewed-on: https://gerrit.chromium.org/gerrit/15998
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Change helper invocation to use discovered script path so image_to_vm.sh
and mount_gpt_image.sh can be run outside of ~/trunk/src/scripts.
Additionally, minimize SCRIPT_ROOT boilerplate to not potentially mix
execution locations, as suggested by David James.
BUG=None
TEST=Built images from outside of ~/trunk/src/scripts
Change-Id: I4851cc74e7da8e08b4e7be79651fb41ce8868aed
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/10241
Reviewed-by: David James <davidjames@chromium.org>
To resize an image, there's no need to use a loop device. We can just operate
on the image directly. This is simpler and avoids doing a sync which can
noticeably delay the build.
BUG=chromium-os:19150
TEST=Run image_to_vm.sh --board=x86-mario
Change-Id: Idbfc99cee9fd890aaad6379fbde511b273cc1d41
Reviewed-on: http://gerrit.chromium.org/gerrit/6036
Reviewed-by: Anush Elangovan <anush@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Calling sync in build scripts pauses the build unnecessarily, particularly
when other steps are running in parallel.
BUG=chromium-os:19150
TEST=Run cbuildbot release build and watch faster performance.
Change-Id: Ia2469e3be68fdd38474ab4e6f67b06339c04822f
Reviewed-on: http://gerrit.chromium.org/gerrit/5966
Reviewed-by: Thieu Le <thieule@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: <taysom@google.com>
Tested-by: David James <davidjames@chromium.org>
The problem here is that most were doing their exiting w/in a subshell;
exit within a subshell kills the subshell, not the parent. Not all scripts
were using set -e (which would pick up the failing subshell); as such
just rewriting them to remove the potential via eliminateing the subshelling.
Beyond that, removed a couple of custom (working, although non-standard)
approaches, and removed a duplicate common.sh sourc'ing w/in mk_memento_images.sh
TEST=force 'find_common_sh' to fail, note the scripts fails to exit
BUG=none
Change-Id: Ia1108a091a6399ad6aedd3cade4a107f4411686c
Reviewed-on: http://gerrit.chromium.org/gerrit/3905
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
The following pattern creates a race condition:
LOOP_DEV=$(sudo losetup -f)
...
sudo losetup "${LOOP_DEV}" "${ROOT_FS_IMG}"
If two steps similar to the above run in parallel, and both steps pick up
the same free loop device, they may try to mount different images with the
same loop device and one of the two stages will get a "device is busy" error.
To fix this, we should switch to the following pattern:
LOOP_DEV=$(sudo losetup --show -f "${ROOT_FS_IMG}")
This CL implements the above.`
BUG=chromium-os:18046
TEST=Run buildbot run. Test case where we run out of loop devices and verify
logic still works.
Change-Id: Ie457701fda61e5fc3b9112c1bfef9fb9713ea265
Reviewed-on: http://gerrit.chromium.org/gerrit/4555
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Daniel Kurtz <djkurtz@chromium.org>
By default, "cros_debug" should be included in the kernel commandline for
developer images. This change adds "cros_debug" to the kernel commandline
for test images, which are based on developer images but overwrite the boot
arguments.
TEST=Build test image, install on machine and grep "cros_debug" /proc/cmdline
BUG=chromium-os:17393
Change-Id: Ie0de11baf60a3a69a7fef0639247e2edae455ffb
Reviewed-on: http://gerrit.chromium.org/gerrit/3790
Tested-by: Arkaitz Ruiz Alvarez <arkaitzr@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
BUG=none
TEST=Run ChromeOS under qemu, the screen should be 1280x1024 not 800x600.
Review URL: http://codereview.chromium.org/6768009
Change-Id: I9a73025244a85d3b07e98136b373b00bb40cd9dd
Change-Id: I52901180e962c463bdaacbc573f93700cf9ab4bd
BUG=None
TEST=Run Chrome OS in qemu, check that the mouse is back.
Review URL: http://codereview.chromium.org/6749020
In addition some refactoring to make this cleaner and easier.
Change-Id: I1607700d065c71aff2b2833b10acbd3ebace68ce
BUG=chromium-os:8212
TEST=Ran with Simple ... running now with full
Review URL: http://codereview.chromium.org/6482017
into mod_image_for_test.sh rather than slightly different versions
of the same in image_to_usb.sh and image_to_vm.sh
Added a function to get a test image into common.sh
Added --inplace option to mod_image_for_test, which is the default,
and preserves the original behaviour. But using --noinplace it will
now do the copy for you.
Found that chromiumos_image.bin appears throughout the scripts, so added it and the test variant to common.sh
BUG=chromiumos-10126
TEST=run mod_image_for_test.sh with and without --noinplace
run image_to_usb.sh and image_to_vm.sh with both options
test on Seaboard that correct image is provided
Really we should have automated testing for these scripts
Change-Id: I5cfa91792c7fded35e7f4ca8f8f27c6b270817fb
Committed: http://chrome-svn/viewvc/chromeos?view=rev&revision=4fc5227
Review URL: http://codereview.chromium.org/5271010
from within the chroot.
It also fixes a number of style issues.
It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.
It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.
BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.
Review URL: http://codereview.chromium.org/6240018
Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
Change-Id: I1d0e546f6ea837928ba426e4690007ef2649f68e
BUG=10102
TEST=test KVM with -net virtio and ensure it boots and can copy files in etc
Review URL: http://codereview.chromium.org/5543006
BUG=8544
TEST=create an VM image with image_to_vm.sh; the stateful partition should be 2GB in size.
Change-Id: I6c0ee9e2cb513e42c25e4f0d1616835a41f378d7
Review URL: http://codereview.chromium.org/4737001
Use new helper for accessing a verified rootfs.
TEST=ran image_to_vm and it worked
BUG=tree b0rked
Change-Id: I03c1c55d1e1056081078cbdd8797235636d4845d
Review URL: http://codereview.chromium.org/4023001
add -net nic,model=e1000 in start_kvm and in the suggested command as
output by image_to_vm.sh
Signed-off-by: Olof Johansson <olofj@chromium.org>
BUG=none
TEST=Boot an image, run lspci in a terminal, see intel nic
Review URL: http://codereview.chromium.org/3779007
Change-Id: Id7f614adec0ae69b8f4de152832538a13ea4cbad
1) image_to_vm aborts because e2fschk requires a terminal to
be connected in order to check whether or not you want to repair the fs. Since
we always want this to be true, set -p
2) Clean up test harness to only verify the number of tests that passed on the
base image are the same on the update back to. This is because older images
have many tests that fail. I leave 10% as a reasonable number to pass.
3) Redirect output from autotest in run_remote_tests to stderr so it gets
logged in a calling script that captures stdout.
Change-Id: If412274353683add20d136747113eb9c2bd41330
BUG=4690, 5533, 7287
TEST=Ran with internal tools for builders.
Review URL: http://codereview.chromium.org/3536018
Also fixed get latest image logic in image_to_vm and allowed for using the most recent image in cros_run_vm_test to follow other shell convention of using latest.
Change-Id: I60ed4c03d609500da7f6ae34ef57ba2e32f4b0bb
BUG=
TEST=Tested by running image_to_vm with --full and cros_run_vm_test with suite_Smoke
Review URL: http://codereview.chromium.org/3597001
