PROD_IMAGE is a flag that indicates a production image should be
built, and will be set for dev builds if the user specifies that
both dev and prod images should be built. build_image was
incorrectly using the PROD_IMAGE variable to conditionaly do some
setup depending on the image type.
Add a new variable IMAGE_BUILD_TYPE that can be tested for the type
of image currently being built and replace the PROD_IMAGE usage.
Signed-off-by: Geoff Levand <geoff@infradead.org>
We need to ship some PCR measurements alongside images in order to make it
easier for admins to provide an appropriate policy. Add some tooling to
generate the appropriate hashes during build, pack those into a zip file
and upload it.
Allows build_image to be used without first running build_packages.
Note: setup_board --force is required before build_packages will work
properly after doing this since baselayout won't be installed otherwise.
- May be sourced early, so explicitly die if source fails.
- Add a function for getting the latest version of a package.
- Read PROVIDES metadata using portageq, enabling data to be read from
binary packages in addition to installed packages. The performance
issue is not an issue here and needed to support empty build roots.
This variable was semi-deprecated ages ago so `version.txt` could follow
a similar variable naming pattern to `os-release`. Finally drop usage of
it here in favor of `$COREOS_VERSION`.
Add the necessary variables in grub.cfg and populate the EFI
partition with arm64 efi executable and modules.
Signed-off-by: Andrej Rosano <andrej@inversepath.com>
Failing to explicitly set the selinux policy store to operate on may
result in semodule installing the policy in an incorrect location. Pass
it on the command line in order to avoid this.
ldconfig does not work for non-native arches. Create a new
build_image routine run_ldconfig that uses qemu user emulation
to run the board ldconfig on the board rootfs when the board and
SDK arches are different.
See: http://code.google.com/p/chromium/issues/detail?id=378377
Prior to calling run_ldconfig the board rootfs must have ldconfig
installed. To arrange this move the call of run_ldconfig to after
the base package install.
Fixes build_image errors like these when building for arm64:
/sbin/ldconfig: /lib64/libXXX is for unknown machine 183.
Signed-off-by: Geoff Levand <geoff@infradead.org>
The new python script check_root uses data that portage already
maintains on what shared libraries packages need or provide instead of
re-scanning whatever ELF files that can be found. This is much more
comprehensive but there is a bit of a transition issue for folks with
long-lived SDKs: packages built with portage older than 2.2.18 do not
include this data. As such for now the check is non-fatal and provides a
command you can use to refresh locally installed packages.
The code checking for conflicts between top level directories and /usr
has also been rewritten. Both tests now are considerably faster.
Once we're signing the root filesystem, we're not going to be able to boot
the kernel from there. Copy the kernel out to the EFI System Partition and
sign it.
There isn't a sane way for users to know the licenses of individual
packages in CoreOS images in built images. The information is hidden
away back in the original ebuilds. This extends our existing package
list with a new file that also includes licenses:
```
app-admin/flannel-0.3.0-r3::coreos Apache-2.0
app-admin/fleet-0.9.1::coreos Apache-2.0
app-admin/locksmith-0.2.3::coreos Apache-2.0
app-admin/sdnotify-proxy-0.1.0::coreos Apache-2.0
app-admin/sudo-1.8.10_p2::portage-stable ISC BSD
app-admin/toolbox-0.0.0-r4::coreos Apache-2.0
app-arch/bzip2-1.0.6-r6::portage-stable BZIP2
app-arch/gzip-1.5::portage-stable GPL-3
app-arch/tar-1.27.1-r2::portage-stable GPL-3+
...
```
This uses our new GRUB2 features to handle GPT priority partition
selection, terminal selection, OEM tweaks, etc. The old SYSLINUX and
PV-GRUB configs are now unused except for maintaining compatibility
with older installs. Of the old configs only the ones that
coreos-postinst copies are needed. The new setup supports using GRUB2
under Xen, giving us automatic fallback support on all of our platforms
for the very first time!
Since grub.cfg is copied into place instead of generated, build_image's
--boot_args option is no longer supported. It could be re-added later
with some sed goo but for now it is easy enough to just edit grub.cfg.
The new grub install script must be called after the image is unmounted
and the old bootloaders script doesn't need to touch grub at all. For
now we will continue to use the existing syslinux configs but
interpreted by grub. Beyond the grub menu flashing by during boot
everything should still be functionally equivalent.
The passing ROOT= as an environment variable to board wrapper scripts
doesn't work, the script unconditionally overrides it. This means so far
our packages.txt files have listed the contents of /build/amd64-usr
instead of the image. Fix this by calling equery directly instead.
We have long since stopped installing anything to the /boot directory of
the root filesystem. Mount the ESP partition to /boot for consistancy
with the discoverable partition spec.
Create profile as a real directory instead of a symlink to the board
root's configuration. Normally the board root does not modify this but
it is useful for build_image to use it to modify package.provided.
Instead of gluing in a special PROD_INSTALL_MASK for all images use
profiles to configure the differences between the base build root,
production images, and developer images. This offers much more
flexibility and is needed for providing a full dev environment in
developer images.
Using parallel_emerge has been disabled by default for all commands
except build_image for quite a while now, build_image kept it just
because it was still a bit faster than normal emerge. Keeping
parallel_emerge complicates future changes to build_image so it needs to
drop it entirely. Since that means nothing uses it by default we might
as well just rip out support for it entirely.
Missed this in 7231b95a, the update zip should still be built when the
usr partition is extracted for generating updates but build_image itself
is not generating and signing the update.
The current generate_update function is now less useful, the important
part that we need is just the partition image now. Also by defaulting to
extracting the partition the old cors_generate_update which is still in
use by devserver can be removed entirely, devserver will just expect the
extracted partition image instead.
Evaluating this as a user config causes it to block on
coreos-environment-setup.service which will wait on networking. This
makes it hard to add extra tricks for testing/debugging situations where
networking is failing. For example, to trigger dhcpcd if networkd dies:
#cloud-config
write_files:
- path: /etc/systemd/system/systemd-networkd.service.d/dhcpcd.conf
content: |
[Unit]
OnFailure=dhcpcd.service
[Service]
Restart=no
The new --developer_data option can be used to specify a path to a cloud
config to bundle into the image. If none is provided but a shared user
password (for core) is set then generate a config to set that password.
This lets us use the same mechanism for setting the default password for
both disk and PXE images.
To make it possible to plop a CoreOS install into a simple
single-filesystem image for use as a container some things like
configuring bootloaders need to be skipped.
Use what was the base image build function as setup/finalize steps in
the dev and prod build functions. This eliminates duplicate code
that mounted and unmounted the filesystem images.
We need some more control over exactly what lands in dev vs prod images
which will require letting them diverge in what is currently the common
base image step. There isn't any real need for the base image in the
first place other than to speed up building both dev and prod images at
the same time but that isn't common enough to worry about.
As part of this cleanup also remove references to CHROMEOS_* variables
and the recovery image that never actually existed in CoreOS.
For generating images for groups other than the one given to build_image
run this script along with the usual image_to_vm.sh commands. To avoid
ambiguity with the 'latest' symlink, this script creates $group-latest
symlinks instead. build_image creates the new symlink too.
Nothing from chromeos-common.sh is needed for image building now. Also
kill off build_common.sh which was just a weird way of sourcing
common.sh. The two piddly functions it provided fit better in
build_image_util.sh
cros_make_image_bootable now only is relevant for prod images, so move
the remaining code to prod_image_util in a similar scheme that base and
dev images use.
Lots of things are either unused or meaningless. A particularly creative
one is the fact that there are command line flags for mount point
locations that are then overwritten.
This makes it possible to toggle parallel_emerge just as other scripts
do. In other scripts update the help string to be more specific, the
--jobs option can be used to control parallelism.
Right now there is some funky logic to either use a previous build as a
seed or the current SDK tarball if it happens to have been downloaded.
This is a bit confusing and doesn't work reliably since it is reasonable
for there to be neither a previous build or the current SDK available if
the SDK chroot was created some time ago. Fix this by using the new SDK
library and always use the latest SDK, downloading it if needed.
Remove --verity_*: Unused, we don't support verity
Remove --usb_disk: Unused, we use PARTUUID now.
Remove --enable_serial: Unused, and serial is enabled for syslinux
A few things here:
- Source manifests/version.txt directly instead of coreos-version.sh
- Remove Chrome branch from target image directory names.
- Use proper version instead of timestap for catalyst builds.
- Move lsb_release script from coreos-overlay to build_library.
As outlined here we need a new partition layout, this patch makes the
necessary changes:
https://groups.google.com/forum/#!topic/coreos-dev/bA7gwGGoTng
The first big change is making all of the scripts obey partition numbers
based on labels in the disk_layout.json. This makes it much easier to
change later on.
The second big change is in the layout itself. The json file was updated
to reflect the document above.
And finally the grub boot configuration needed for pv-grub and pygrub
were added to the create_legacy_bootloader_templates.sh library utlity.
Everything seems to work and boot now.
Enable the serial port from the build_image command line.
Currenly, you have to edit build_kernel_image.sh to enable
the serial port for kernel debugging. Now:
./build_image --board=${BOARD} --enable_serial=ttyS0
Will enable sending printks to the specified serial port.
BUG=chromium-os:38026
TEST=built images with and without serial enabled
Change-Id: I9ef4f2a20f0d451e132371339c4eba1faf4c94de
Reviewed-on: https://gerrit.chromium.org/gerrit/41638
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Paul Taysom <taysom@chromium.org>
Tested-by: Paul Taysom <taysom@chromium.org>
Now that we have an ebuild for factory install shim, let's move to that
instead of base image.
BUG=chrome-os-partner:16712
TEST=Test network boot, install shim, RMA shim on Link.
Test install shim on Snow.
CQ-DEPEND=CL:39952
Change-Id: Ib6a7a598087647b44a4aeca970e293ea72023890
Signed-off-by: Vic Yang <victoryang@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39955
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
BUG=chromium-os:36862
TEST=./build_image factory_install --board=parrot
Change-Id: If1e59d09eec45340e49b2711b4799eb1f83ea007
Reviewed-on: https://gerrit.chromium.org/gerrit/39133
Tested-by: Paul Taysom <taysom@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Paul Taysom <taysom@chromium.org>
Changes to build_image to allow enabling of bootcache.
./build_image --board=$BOARD --enable_bootcache
A board can be configured to use the bootcache by
the following lines in private-overlays/overlay-<board>-private/scripts
if [[ ${FLAGS_bootcache_use_board_default} -eq ${FLAGS_TRUE} ]]; then
FLAGS_enable_bootcache=${FLAGS_TRUE}
fi
Setting --noenable_bootcache or --enable_bootcache on the comand
line will override the default.
BUG=chromium-os:25441
TEST=built and installed snow, amd-64(latitude), stumpy
Change-Id: Ie081ef94f4799b0071b53e0587d89f1247b4a11f
Reviewed-on: https://gerrit.chromium.org/gerrit/38414
Tested-by: Paul Taysom <taysom@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: Paul Taysom <taysom@chromium.org>
This reverts commit acff376525
This broke the signing process due to changed kernel params.
Please update ensure_secure_kernelparams.config under the
cros-signing/ tree before relanding this.
Change-Id: I3be62e16299eb69bbfef9f1530d92200a2e309d7
Reviewed-on: https://gerrit.chromium.org/gerrit/34320
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Added a new flag for enabling the boot cache.
BUG=chromium-os:25441
TEST=built and ran amd64 and arm
Change-Id: Ia151d40c4b02f4353981affd321763521d972ee6
Reviewed-on: https://gerrit.chromium.org/gerrit/33617
Tested-by: Paul Taysom <taysom@chromium.org>
Reviewed-by: Olof Johansson <olofj@chromium.org>
Commit-Ready: Paul Taysom <taysom@chromium.org>
This change adds support for building the disk layout from a
configuration file. It also cleans up much of the image creation
code.
install_gpt no longer exists, and has been replaced by cgpt.py's
write action. This spits out a file that has two functions that can
be called to write a partition layout to a disk/file. This gets rid
of the gigantic nest of calculations that built the layout previously.
All instances of partition/filesystem sizes in build scripts should now
be gone in favour of calls to the cgpt.py tool.
create_boot_desc has moved inside the base image creation, in an effort
to simplify build_image.
load_kernel_test is gone since it's apparently not supposed to be called
here anyway (asked wfrichar/rspangler about this one).
Base image creation now uses files rather than loop devices when
building an image. This means we can simply umount them once we're
done and not worry about cleaning up the loop device, since it's
been done for us.
Hash pad calculation has been removed. This is now set manually inside
the partition config file.
Hybrid MBR creation is gone, since it's now possible to do that in a board
specific hook (see overlay-beaglebone/scripts/board_specific_setup.sh).
OEM partition now has a filesystem, which is mounted at /usr/share/oem
during emerge so that packages can stash files here.
root_fs_dir and friends are still globals, but the long-term idea
is to make this not the case.
BUG=chromium-os:33817
TEST=All types of images and their respective flows
(VM, recovery, test, factory etc)
Change-Id: I8a596728a4d1845c930e837bea627f5b6a11c098
Reviewed-on: https://gerrit.chromium.org/gerrit/29931
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
Reviewed-by: Liam McLoughlin <lmcloughlin@chromium.org>
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
Currently, if set -e spots a nonzero exit we basically have
no real debug information- it just stops immediately without stating
where or why. This forces our scripts to be stupidly verbose so
we can track roughly where they were, thus when they fail we can
use that information to localize the rough exit point.
Instead we should be traping that set -e induced exit and
outputing necessary debug information to run it down. This includes
outputing the relevant stack trace, or at least what we can get of
it.
The 'die' function is now enhanced to automatically dump the trace
that lead to it. For most consumers this is desired- however for
commandline parsing induced dies ("--board is missing" for example),
the trace is noise. For those cases, a 'die_notrace' function was
added that retains the original non-backtrace behaviour.
Example output via instrumenting cros_generate_breakpad_symbols
w/ the failing command '/bin/false' (nonzero exit code).
Before:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
<no output at all, just exit code 1>
With this CL:
./cros_generate_breakpad_symbols monkeys --board=x86-alex
ERROR : script called: ./cros_generate_breakpad_symbols 'monkeys' '--board=x86-alex'
ERROR : Backtrace: (most recent call is last)
ERROR : file cros_generate_breakpad_symbols, line 207, called: main 'monkeys' '--board=x86-alex'
ERROR : file cros_generate_breakpad_symbols, line 163, called: die_err_trap '/bin/false' '1'
ERROR :
ERROR : Command failed:
ERROR : Command '/bin/false' exited with nonzero code: 1
BUG=chromium-os:30598
TEST=inject a failing command into a script, verify the output.
TEST=inject a 'command not found', verify the output
TEST=cbuildbot x86-generic-full --remote
TEST=cbuildbot arm-tegra2-full --remote
TEST=cbuildbot chromiumos-sdk --remote
Change-Id: I517ffde4d1bb7e2310a74f5a6455b53ba2dea86c
Reviewed-on: https://gerrit.chromium.org/gerrit/17225
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Brian Harring <ferringb@chromium.org>
This change adds the ability for a caller to pass a specific version name
to build_image rather than rely on parsing of chromeos_version.sh + attempt
number.
BUG=chromium-os:29077
TEST=Ran build_image w/ w/out option
Change-Id: I69b76ae4bfc148325686902606476a0aae293e56
Reviewed-on: https://gerrit.chromium.org/gerrit/19861
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
Using these flags directly was deprecated a while back and is causing confusion
which way is the right way. This CL removes all these FLAGS from build_image
(though keeps support of them in mod_image_for_test/image_to_usb invocations
in common.sh). In this change I've also defaulted build_image to build the
developer image (and only the developer image).
BUG=chromium-os:27362
TEST=Been busy testing. Have built the following combinations and verified them:
build_image base
build_image
build_image base dev test
build_image dev
build_image dev factory_test
build_image factory_install
Change-Id: Ie534c276a9ec571926964320ac176daa91b12a81
Reviewed-on: https://gerrit.chromium.org/gerrit/17386
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
This comes from not correctly negating all FLAGS and also the weird
way we set PRISTINE_IMAGE_NAME. Addressed in both cases and
simplified FLAGS_* logic in build_image_util.sh
BUG=chromium-os:24627
TEST=build_image base so far.
Change-Id: I92e8550db3ea713cda1f997b702777035145d8d5
Reviewed-on: https://gerrit.chromium.org/gerrit/13636
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Mandeep Singh Baines <msb@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
Fixed all callers in my big CL to no longer have to wrap poorly using
\ and start from the beginning of the next line but rather pass in
an array of args i.e.
info "tacos" " are" " delicious".
BUG=None
TEST=Ran to see the errors in parse_build_image and manual eyeing.
Change-Id: I5eac8a5ae7a8d314dbc4e821ee33cf88213711d0
Reviewed-on: https://gerrit.chromium.org/gerrit/10823
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
This CL does two things.
First it introduces an argument interface that works with all the following
combinations:
factory_install | [base|dev[test|factory_test]] rather than use --test etc.
This does not build extra images. If you just want a test image you can
run build_image --board=<board> test and all that's in your latest_dir is
chromiumos_test_image.bin.
Second, we only build what we ask and only finalize what we ask so on
an invocation of build_image test we only actually run cros_make_image_bootable
once saving 2 minutes on my machine (from 6:50->4:50 on multiple iterations).
BUG=chromium-os:22048
TEST=build_image test, all possible combinations of args through to start of
build. Built factory_install using new and old methods and dev test with both
methods. Also verified mod_image_for_test still works as advertises stand-alone.
Change-Id: I9fe2feb50a941c007214decd9ba1627012c050af
Reviewed-on: https://gerrit.chromium.org/gerrit/10621
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
This will prevent the recovery kernel from having a different salt from the
rootfs it corresponds to.
BUG=chromium-os:20766
TESTED_ON=kaen
TEST=Adhoc
Build a recovery image and do recovery from it.
Change-Id: I96f735e527d807247e09e17aac1ed5b51367f0ef
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/8288
Right now, output directories are hardcoded in boot.desc. This means that
cros_make_image_bootable will still write to the directories mentioned in
build_image, even if it's working on a different directory. This can
mess with the buildbot, because it can cause cros_make_image_bootable
to mess with the files created for existing images.
This change fixes flaky failures where the buildbot fails with warnings
stating that vmlinuz_hd.vblock does not exist. See bug 19956.
BUG=chromium-os:19956
TEST=Run full canary trybot run and verify cros_make_image_bootable uses
files from the right directory now.
Change-Id: Ib390aa84570b077cbc8b69b757998056acc091ea
Reviewed-on: http://gerrit.chromium.org/gerrit/7282
Reviewed-by: Will Drewry <wad@chromium.org>
Tested-by: David James <davidjames@chromium.org>
BUG=None
TEST=run the commands in various combinations.
Change-Id: I94fb167d8312a90818910085adebfb1d0396cdbe
Reviewed-on: http://gerrit.chromium.org/gerrit/6866
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Vince Laviano <vlaviano@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
