A number of places refer to these paths and that number is going to
grow. Since the standard pattern is to use environment variables for
commonly used paths it is time to add ones for these:
REPO_CACHE_DIR
REPO_MANIFESTS_DIR
We don't have any particular reason for the weird hackery required to
install packages into /usr/local instead of root. The rootfs image is
already being modified a little might as well modify it a lot. :)
When running from a au zip or other strange situation assume the version
in version.txt should be used as-is. This avoid the need to set
COREOS_OFFICIAL=1 in these situations which can be lead to surprises.
We don't need the default root filesystem fsck and remount targets
provided by systemd since root is read only. The only default one what
was included in this way was tmp.mount but that is now covered by
a dependency in the coreos-init package.
Its single use is in build_common and even then having a little progress
bar for copying images isn't that interesting, they just get lost in the
noise of the emerge output. Keep it simple, use cp.
/mnt/stateful_partition was already a little unruly with
/mnt/stateful_partition/home and /mnt/stateful_partition/var_overlay
serving similar functional purposes.
Then we needed to also add /opt and /srv overlays.
I also have wanted to get rid of the ugly and weird
/mnt/stateful_partition name so lets just have one big move.
/mnt/stateful_partition -> /media/state
/mnt/stateful_partition/var_overlay -> /media/overlays/var
/mnt/stateful_partition/home -> /media/overlays/home
From there we add /media/overlays/srv and /media/overlays/opt
The old script was heading towards spaghetti code realm. This breaks up
all the image variations such as hybrid MBR, OEM packages, etc into
configuration options and small functions that actually do the work.
All this is in the new vm_image_util.sh library but the command line
parsing and overall procedure remains in image_to_vm.sh
As part of this we gain support for putting some qemu options in a
config file as well as Xen virtual machines using pygrub and pvgrub.
Lots of generally unused options have been removed to simplify things
and keep output file names consistent.
As-is safe_umount is extremely dangerous. When passed multiple mount
points and any one of them fail with a "not mounted" or "doesn't exist"
error then any others that fail with a more serious error will be
silently ignored. This can cause untold sadness when running deleting a
chroot with cros_sdk if /mnt/host/source is left mounted, all your code
will be gone. To avoid this situation remove *ALL* this extra logic and
die very loudly when umount fails. Due to the way bind mounts interact
with this code "not mounted" so when unmounting a full tree we need to
still need to gracefully retry when the first umount fails.
If ~/.subversion doesn't exist, the code didn't run, but if it existed
there is no reason to re-create it, nor is it necessary to change its
permissions since they are inherited by the bind mount source.
However user_mkdir was trying to run chown as root which does not work
over NFS with root_squash or krb-nfs.
Therefore, the un-needed call to user_mkdir is removed.
(this is an issue because cros_sdk --replace does call this code path
multiple times).
BUG=None
TEST=Built the chroot, and the permission denied on 'install' went away.
Change-Id: I01e9a7baf51a99a96d790c9613e26e652379e6df
Reviewed-on: https://gerrit.chromium.org/gerrit/44880
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Marc MERLIN <merlin@chromium.org>
Tested-by: Marc MERLIN <merlin@chromium.org>
Drop zero padding and format versions as described by the semver spec.
The terminology is a little awkward because we inherited the backwards
meaning of 'BUILD' and 'BRANCH' version identifiers but that the version
strings themselves conform to semver.
(This doesn't change the current version, that'll happen with our next
branch cut)
A few things here:
- Source manifests/version.txt directly instead of coreos-version.sh
- Remove Chrome branch from target image directory names.
- Use proper version instead of timestap for catalyst builds.
- Move lsb_release script from coreos-overlay to build_library.
on Fedora 18 on Gnome 3.0 something is making the first attempt at
unmounting return busy. Unfortunatly, the return code is 32 everytime
so we have to parse the output of umount :( :( :(
Change-Id: I7f94bf6c2059c7e7cb4fb173d9ffbabd59f2b24f
This directory contains profiling/coverage data that is generated by some
packages during the build process. It should not be installed onto the device,
but we want the data to live on in /build to allow developers to consult it.
BUG=chromium-os:37854
TEST=None, this change will be tested in CL 42339 since it currently has not
effect
Change-Id: I882fa7a7496db52a72bb8c5409b3715490d95879
Reviewed-on: https://gerrit.chromium.org/gerrit/42499
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Ryan Harrison <rharrison@chromium.org>
Tested-by: Ryan Harrison <rharrison@chromium.org>
We don't need to support POSIX shell, so convert to bash.
BUG=None
TEST=`cbuildbot {amd64,x86,arm}-generic-full chromiumos-sdk daisy-release` worked
Change-Id: I33ad25d2310c593f3e346d955e3aa27da41091fc
Reviewed-on: https://gerrit.chromium.org/gerrit/41271
Reviewed-by: David James <davidjames@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Issues flagged during unittest run; bit unsure how this got missed
since trybots were involved (Presuming human error).
BUG=chromium-os:37347
TEST=trybot run
Change-Id: Ib6e49d28f9427197a0abd0e4da68d2f628dc2803
Reviewed-on: https://gerrit.chromium.org/gerrit/41203
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Rather than having to find /home/${SUDO_USER:-${USER}}/trunk, instead
just look for /mnt/host/trunk (defined by common.sh as $CHROOT_TRUNK_DIR).
This simplifies code flow, and is a requirement for shoving chromite
into PYTHONPATH globally w/in the chroot.
BUG=chromium-os:37347
TEST=cros_sdk --replace; cros_sdk w/ chroot upgrade.
Change-Id: I9ee3e6556541a91193f49cbf74ffc5a8e090537f
Reviewed-on: https://gerrit.chromium.org/gerrit/39921
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
By default, pv automatically infers and uses the full width of the
terminal. This generally makes sense for a console application, but in
the case of pv it just causes the progress bar to be ridiculously wide
when run on wide terminals, to the point it's hard to read. This CL is
setting it to 80 characters, the widely accepted standard width for
a terminal, in cases where the terminal appears to be larger than
80 columns. Note that:
* Even with -w, pv appears to be resizing the progress bar as the
terminal width changes midway through the run. This means that if
a user widens the window, then the progress bar will go wide again and
there's nothing to be done about it.
* Theoretically, in very rare cases this may lead to a progress bar the
exceeds the width of the terminal (i.e. set to 80 columns on
a terminal that has just shrunk to fewer columns). The odds for such
timing are close to nil and even then the damage is minimal.
* This will work for non-terminal runs, or otherwise runs where stty
does not produce any output.
* To avoid the initialization overhead for all common.sh inclusion,
replacing the variable with a function that prints the pv/cat command.
BUG=None
TEST=Ran ./image_to_usb on wide and narrow terminal windows, it works.
Change-Id: I549df1dd29e93909ea646ae9b9e09d9a588ad382
Reviewed-on: https://gerrit.chromium.org/gerrit/40937
Commit-Queue: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
mod_image_for_test.sh doesn't work anymore so nobody should be using it.
There are a few places where scripts try to use mod_image_for_test.sh,
and these are timebombs because they fail if a test image needs to actually
be produced.
BUG=chromium-os:31183
TEST=Tested that this script doesn't produce any images anymore, so
it should be fine to delete it.
Change-Id: If80337407023d62f76117dc44cadfa46801ca236
Reviewed-on: https://gerrit.chromium.org/gerrit/40955
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Similar to the uinput module, install the testcases/ tree of modules
only on test images.
BUG=chromium-os:37353
TEST=daisy test image build has modules, production does not.
CQ-DEPEND=I16eee5afd1664f0ab4a9ab48b6cb2beaa6f30017
Change-Id: I5e7a48936b3fb45f49ff7ca4d592d46150c5610c
Signed-off-by: Kees Cook <keescook@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39920
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Grant Grundler <grundler@chromium.org>
This reduces tens of MB of install shim.
BUG=chrome-os-partner:16712
TEST=Build install shim and install with it.
Change-Id: Ic7fab3c6e26d9b78f5181c4f10181217c1c2697a
Signed-off-by: Vic Yang <victoryang@chromium.org>
Reviewed-on: https://gerrit.chromium.org/gerrit/39534
Reviewed-by: Tom Wai-Hong Tam <waihong@chromium.org>
When running on NFS, the root user may not be able to access ~/.ssh and
~/.gitconfig, so it is necessary to fallback to SUDO_USER to access these
files.
To discourage users from using NFS homedirs, print warnings every time
cros_sdk is run with an NFS homedir.
BUG=chromium-os:36783
TEST=Try cros_sdk --replace and cros_sdk with and without NFS homedirs.
Change-Id: I4cdbceca485d3491656d6f743814da4ebcdd75ad
Reviewed-on: https://gerrit.chromium.org/gerrit/38953
Commit-Ready: David James <davidjames@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: David James <davidjames@chromium.org>
sudo takes 150ms per invocation on Goobuntu, and with 10 invocations in
enter_chroot.sh, this means that we're wasting a lot of time, every time
cros_sdk is invoked. Cutting these unnecessary invocations reduces the time
required to run enter_chroot.sh from 2.3s to 0.8s.
CL:36618 is the companion change that updates cros_sdk to invoke
sudo unshare -m prior to calling enter_chroot.sh.
Summary of changes:
1. Remove all calls to sudo and just run the commands directly.
- Remove the mount queue and any sudo_multi optimizations.
- Rename sudo_chroot -> bare_chroot because we don't run sudo anymore there.
- Remove code for validating sudo timestamp.
2. Allow the scripts to work as root:
- Ensure that files created by cros_sdk that previously were owned by the
user still are owned by the user (either using chown or cp -p).
- Use $SUDO_USER to find the user's account.
- Use $SUDO_HOME instead of $HOME to find the user's home dir.
- Remove outdated code for disabling automount on Lucid, which doesn't work
when run as root.
- Update code for calculating the user's git username to use sudo to switch
to the user. Also move it to make_chroot.sh so that this change doesn't
impact performance.
3. Cleanup
- Remove environment syncer process in favor of just syncing once when chroot
is entered.
- Remove teardown and instead rely on unshare to unmount the mounts. To make
sure that outside processes never notice the mounts, we use mount -n. This
also ensures that /etc/mtab never contains stale mounts.
- Remove path-overrides, since it is no longer needed.
BUG=chromium-os:35714, chromium-os:35679
TEST=Trybot runs.
CQ-DEPEND=CL:36618
Change-Id: I919a8aadb08fafde97348e8511573c28fdd47186
Reviewed-on: https://gerrit.chromium.org/gerrit/36619
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
Mounts can sometimes be unmounted from under our feet. I'm not sure
what's causing this problem, but I can reproduce it both on gPrecise
and on our builders (chromiumos-sdk). I've confirmed this patch fixes
the problem.
BUG=chromium-os:35679
TEST=20 chromiumos-sdk trybot runs. Confirmed this code is executing
and catching a race condition in the trybot runs.
Change-Id: Iff019e672e9124053ac62cf3017dba485a4989cb
Reviewed-on: https://gerrit.chromium.org/gerrit/36698
Tested-by: David James <davidjames@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Commit-Ready: David James <davidjames@chromium.org>
BUG=None
TEST=build_image
Change-Id: Ib1ff8ac57b48402d2353d33413b936e5a6627176
Reviewed-on: https://gerrit.chromium.org/gerrit/28120
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
This change adds support for building the disk layout from a
configuration file. It also cleans up much of the image creation
code.
install_gpt no longer exists, and has been replaced by cgpt.py's
write action. This spits out a file that has two functions that can
be called to write a partition layout to a disk/file. This gets rid
of the gigantic nest of calculations that built the layout previously.
All instances of partition/filesystem sizes in build scripts should now
be gone in favour of calls to the cgpt.py tool.
create_boot_desc has moved inside the base image creation, in an effort
to simplify build_image.
load_kernel_test is gone since it's apparently not supposed to be called
here anyway (asked wfrichar/rspangler about this one).
Base image creation now uses files rather than loop devices when
building an image. This means we can simply umount them once we're
done and not worry about cleaning up the loop device, since it's
been done for us.
Hash pad calculation has been removed. This is now set manually inside
the partition config file.
Hybrid MBR creation is gone, since it's now possible to do that in a board
specific hook (see overlay-beaglebone/scripts/board_specific_setup.sh).
OEM partition now has a filesystem, which is mounted at /usr/share/oem
during emerge so that packages can stash files here.
root_fs_dir and friends are still globals, but the long-term idea
is to make this not the case.
BUG=chromium-os:33817
TEST=All types of images and their respective flows
(VM, recovery, test, factory etc)
Change-Id: I8a596728a4d1845c930e837bea627f5b6a11c098
Reviewed-on: https://gerrit.chromium.org/gerrit/29931
Commit-Ready: Liam McLoughlin <lmcloughlin@chromium.org>
Reviewed-by: Liam McLoughlin <lmcloughlin@chromium.org>
Tested-by: Liam McLoughlin <lmcloughlin@chromium.org>
Do this via ensuring that any common.sh invoker
of raw umount (say a root script) sees our umount
path.
Additionally, inject into default profiles our override,
and via an upgrade scriptlet.
This is round two; originally appeared as CL:32088, was
reverted due to:
https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos%20sdk/builds/2314/steps/BuildBoard/logs/stdio
The fix however is just adding a single sudo mkdir. :/
BUG=chromium-os:23443
TEST=cros_sdk --replace --bootstrap
TEST=cros_sdk --replace
Change-Id: I0dc7522a9c623f40081d4f138cea0c2c45171fea
Reviewed-on: https://gerrit.chromium.org/gerrit/32365
Commit-Ready: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
