We need some more control over exactly what lands in dev vs prod images
which will require letting them diverge in what is currently the common
base image step. There isn't any real need for the base image in the
first place other than to speed up building both dev and prod images at
the same time but that isn't common enough to worry about.
As part of this cleanup also remove references to CHROMEOS_* variables
and the recovery image that never actually existed in CoreOS.
The existing version.txt is kinda annoying. The common case of referring
to the current version requires joining three values and the names of
those values only make sense in ChromeOS. Instead just use version as a
string, using VERSION, VERSION_ID, and BUILD_ID just as they appear in
os-release. It is up to the few scripts that need the individual parts
to break the version apart.
The old values remain for the sake of compatibility.
I would like to phase out parallel_emerge so disable it for all commands
other than build_image which is the only one that shows a noticeable
benefit from it (~2 min with --fast, ~3 min with --nofast).
Make it possible for other scripts to share the same value for our
release repository and equally easy to override with a custom value.
Also allow setting the root from the command line in addition to the
environment. Usually --upload_root is better to use than --upload_path.
For multi-file uploads we should explicitly declare what the name of the
.DIGESTS file should be instead of using the first file name. Relying on
the ordering was subtle and easy to break.
A number of places refer to these paths and that number is going to
grow. Since the standard pattern is to use environment variables for
commonly used paths it is time to add ones for these:
REPO_CACHE_DIR
REPO_MANIFESTS_DIR
We don't have any particular reason for the weird hackery required to
install packages into /usr/local instead of root. The rootfs image is
already being modified a little might as well modify it a lot. :)
When running from a au zip or other strange situation assume the version
in version.txt should be used as-is. This avoid the need to set
COREOS_OFFICIAL=1 in these situations which can be lead to surprises.
We don't need the default root filesystem fsck and remount targets
provided by systemd since root is read only. The only default one what
was included in this way was tmp.mount but that is now covered by
a dependency in the coreos-init package.
Its single use is in build_common and even then having a little progress
bar for copying images isn't that interesting, they just get lost in the
noise of the emerge output. Keep it simple, use cp.
/mnt/stateful_partition was already a little unruly with
/mnt/stateful_partition/home and /mnt/stateful_partition/var_overlay
serving similar functional purposes.
Then we needed to also add /opt and /srv overlays.
I also have wanted to get rid of the ugly and weird
/mnt/stateful_partition name so lets just have one big move.
/mnt/stateful_partition -> /media/state
/mnt/stateful_partition/var_overlay -> /media/overlays/var
/mnt/stateful_partition/home -> /media/overlays/home
From there we add /media/overlays/srv and /media/overlays/opt
The old script was heading towards spaghetti code realm. This breaks up
all the image variations such as hybrid MBR, OEM packages, etc into
configuration options and small functions that actually do the work.
All this is in the new vm_image_util.sh library but the command line
parsing and overall procedure remains in image_to_vm.sh
As part of this we gain support for putting some qemu options in a
config file as well as Xen virtual machines using pygrub and pvgrub.
Lots of generally unused options have been removed to simplify things
and keep output file names consistent.
As-is safe_umount is extremely dangerous. When passed multiple mount
points and any one of them fail with a "not mounted" or "doesn't exist"
error then any others that fail with a more serious error will be
silently ignored. This can cause untold sadness when running deleting a
chroot with cros_sdk if /mnt/host/source is left mounted, all your code
will be gone. To avoid this situation remove *ALL* this extra logic and
die very loudly when umount fails. Due to the way bind mounts interact
with this code "not mounted" so when unmounting a full tree we need to
still need to gracefully retry when the first umount fails.
If ~/.subversion doesn't exist, the code didn't run, but if it existed
there is no reason to re-create it, nor is it necessary to change its
permissions since they are inherited by the bind mount source.
However user_mkdir was trying to run chown as root which does not work
over NFS with root_squash or krb-nfs.
Therefore, the un-needed call to user_mkdir is removed.
(this is an issue because cros_sdk --replace does call this code path
multiple times).
BUG=None
TEST=Built the chroot, and the permission denied on 'install' went away.
Change-Id: I01e9a7baf51a99a96d790c9613e26e652379e6df
Reviewed-on: https://gerrit.chromium.org/gerrit/44880
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
Drop zero padding and format versions as described by the semver spec.
The terminology is a little awkward because we inherited the backwards
meaning of 'BUILD' and 'BRANCH' version identifiers but that the version
strings themselves conform to semver.
(This doesn't change the current version, that'll happen with our next
branch cut)
A few things here:
- Source manifests/version.txt directly instead of coreos-version.sh
- Remove Chrome branch from target image directory names.
- Use proper version instead of timestap for catalyst builds.
- Move lsb_release script from coreos-overlay to build_library.
on Fedora 18 on Gnome 3.0 something is making the first attempt at
unmounting return busy. Unfortunatly, the return code is 32 everytime
so we have to parse the output of umount :( :( :(
Change-Id: I7f94bf6c2059c7e7cb4fb173d9ffbabd59f2b24f
This directory contains profiling/coverage data that is generated by some
packages during the build process. It should not be installed onto the device,
but we want the data to live on in /build to allow developers to consult it.
BUG=chromium-os:37854
TEST=None, this change will be tested in CL 42339 since it currently has not
effect
Change-Id: I882fa7a7496db52a72bb8c5409b3715490d95879
Reviewed-on: https://gerrit.chromium.org/gerrit/42499
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Ryan Harrison <rharrison@chromium.org>
Tested-by: Ryan Harrison <rharrison@chromium.org>
We don't need to support POSIX shell, so convert to bash.
BUG=None
TEST=`cbuildbot {amd64,x86,arm}-generic-full chromiumos-sdk daisy-release` worked
Change-Id: I33ad25d2310c593f3e346d955e3aa27da41091fc
Reviewed-on: https://gerrit.chromium.org/gerrit/41271
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Issues flagged during unittest run; bit unsure how this got missed
since trybots were involved (Presuming human error).
BUG=chromium-os:37347
TEST=trybot run
Change-Id: Ib6e49d28f9427197a0abd0e4da68d2f628dc2803
Reviewed-on: https://gerrit.chromium.org/gerrit/41203
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Rather than having to find /home/${SUDO_USER:-${USER}}/trunk, instead
just look for /mnt/host/trunk (defined by common.sh as $CHROOT_TRUNK_DIR).
This simplifies code flow, and is a requirement for shoving chromite
into PYTHONPATH globally w/in the chroot.
BUG=chromium-os:37347
TEST=cros_sdk --replace; cros_sdk w/ chroot upgrade.
Change-Id: I9ee3e6556541a91193f49cbf74ffc5a8e090537f
Reviewed-on: https://gerrit.chromium.org/gerrit/39921
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
By default, pv automatically infers and uses the full width of the
terminal. This generally makes sense for a console application, but in
the case of pv it just causes the progress bar to be ridiculously wide
when run on wide terminals, to the point it's hard to read. This CL is
setting it to 80 characters, the widely accepted standard width for
a terminal, in cases where the terminal appears to be larger than
80 columns. Note that:
* Even with -w, pv appears to be resizing the progress bar as the
terminal width changes midway through the run. This means that if
a user widens the window, then the progress bar will go wide again and
there's nothing to be done about it.
* Theoretically, in very rare cases this may lead to a progress bar the
exceeds the width of the terminal (i.e. set to 80 columns on
a terminal that has just shrunk to fewer columns). The odds for such
timing are close to nil and even then the damage is minimal.
* This will work for non-terminal runs, or otherwise runs where stty
does not produce any output.
* To avoid the initialization overhead for all common.sh inclusion,
replacing the variable with a function that prints the pv/cat command.
BUG=None
TEST=Ran ./image_to_usb on wide and narrow terminal windows, it works.
Change-Id: I549df1dd29e93909ea646ae9b9e09d9a588ad382
Reviewed-on: https://gerrit.chromium.org/gerrit/40937
Commit-Queue: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
mod_image_for_test.sh doesn't work anymore so nobody should be using it.
There are a few places where scripts try to use mod_image_for_test.sh,
and these are timebombs because they fail if a test image needs to actually
be produced.
BUG=chromium-os:31183
TEST=Tested that this script doesn't produce any images anymore, so
it should be fine to delete it.
Change-Id: If80337407023d62f76117dc44cadfa46801ca236
Reviewed-on: https://gerrit.chromium.org/gerrit/40955
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Similar to the uinput module, install the testcases/ tree of modules
only on test images.
BUG=chromium-os:37353
TEST=daisy test image build has modules, production does not.
CQ-DEPEND=I16eee5afd1664f0ab4a9ab48b6cb2beaa6f30017
Change-Id: I5e7a48936b3fb45f49ff7ca4d592d46150c5610c
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39920
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
This reduces tens of MB of install shim.
BUG=chrome-os-partner:16712
TEST=Build install shim and install with it.
Change-Id: Ic7fab3c6e26d9b78f5181c4f10181217c1c2697a
Signed-off-by: Vic Yang <victoryang@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39534
Reviewed-by: Tom Wai-Hong Tam <waihong@chromium.org>
When running on NFS, the root user may not be able to access ~/.ssh and
~/.gitconfig, so it is necessary to fallback to SUDO_USER to access these
files.
To discourage users from using NFS homedirs, print warnings every time
cros_sdk is run with an NFS homedir.
BUG=chromium-os:36783
TEST=Try cros_sdk --replace and cros_sdk with and without NFS homedirs.
Change-Id: I4cdbceca485d3491656d6f743814da4ebcdd75ad
Reviewed-on: https://gerrit.chromium.org/gerrit/38953
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
