* This CL deprecates the use of enter_chroot and make_chroot completely,
leaving the functionality exposed only through cros_sdk.
BUG=chromium-os:18750
TEST=run them
Change-Id: I864960b4e25245341431c3a3950638fa569820ed
Reviewed-on: http://gerrit.chromium.org/gerrit/6358
Reviewed-by: Anush Elangovan <anush@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>
* Removed boilerplate, simplified search code.
* Fixed one too long line
This will unfortunately kill all outstanding CLs into enter_chroot.
BUG=chromium-os:18750
TEST=run it
Change-Id: I39c45fa8163d92487b512e7e8d298ce9231f4bd2
Reviewed-on: http://gerrit.chromium.org/gerrit/5830
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Anush Elangovan <anush@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
The source of this LANG=C seems to date back to when apt was used as
the package manager [1]. It was further migrated to enter_chroot to
keep noise in the logs buildbot down to a minimal [2]. I've seen the
excess spam apt can spew in a default deb chroot when the active env
locale is not available (in reality it is perl that complains) because
the locale data has not yet been generated.
None of this should apply to Gentoo though as we don't execute perl
anywhere during the build. So hopefully we can simply drop this now
as an artifact of days gone by.
The reason for not hardcoding this in the first place is that it can
easily wreck havoc when the host locale is multibyte (e.g. en_US.UTF8)
but the chroot is forced to narrow (e.g. C). Attempts to use multibyte
chars will usually render correctly, but the apps in the chroot will
just get immensely confused as they will be using narrow functions to
process input.
[1] the code says "Warn less when apt-get installing packqages"
[2] http://codereview.chromium.org/521053
BUG=None
TEST=entered chroot and saw LANG preserved from host env, and ran cbuildbot
Change-Id: I0490b992d5e2d7bfba945f787e65a59943b8d35c
Reviewed-on: http://gerrit.chromium.org/gerrit/5767
Reviewed-by: Anush Elangovan <anush@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Details
This change updates the use of '-s' to have the proper polarity to
see if the syncer PID file is zero bytes in size.
As a bit of other cleanup, if a file of size zero is found, it's
removed. This allows the second size test of the syncer PID file to
be removed.
Testing
o Created zero-length PID file: saw warning message on console. Saw
that the file contains a valid pin while the chroot is entered,
and verified the PID file is deleted after exiting chroot.
INFO : You may have suffered from chromium-os:17680 and
INFO : could have stray 'enter_chroot.sh' processes running.
INFO : Erroneous PID file will be cleaned up.
o Entered chroot with multiple terminals, saw that the PID file is
removed & the syncer process is killed when the last client exits
the chroot.
o Executed './enter_chroot.sh --verbose', changed ownership of PID file to
'root'.
Exited and did not see the following diagnostic message:
INFO : Unable to clean up syncer process.
The message should not be printed because the file is removed
under 'sudo'.
BUG=chromium-os:17680
TEST=See above.
Signed-off-by: Taylor Hutt <thutt@chromium.org>
Change-Id: I055fe72b9322acee8c774401b3452ef502b30970
Reviewed-on: http://gerrit.chromium.org/gerrit/4936
Tested-by: Taylor Hutt <thutt@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Details
I have a set of scripts which run from outside the chroot; they
execute things inside the chroot, and therefore enter & exit the
chroot frequently.
This morning I noticed this morning that I was getting
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
on the console each time I exited the chroot.
I tracked this down to the 'sync process' PID file; in my case, it
was zero bytes in size.
A little more detective work led to
412b902b0e and I think I pieced
together that I had suffered from chromium-os:17680, which resulted
in an empty file.
This change is my attempt at repairing the damage if
chromium-os:17680 has been hit, and preventing other types of
similar issues with the PID file.
412b902b0e addressed the main cause of
issue chromium-os:17680, and this change addresses mysterious
behavior in chroots that have suffered from it.
To allow people to know that they've possibly got stray syncer
porcesses running, I also output a message to the console which
indicates such.
Testing
o Created zero-length PID file: saw warning message on console. Saw
that the file contains a valid pin while the chroot is entered,
and verified the PID file is deleted after exiting chroot.
INFO : You may have suffered from chromium-os:17680 and
INFO : could have stray 'enter_chroot.sh' processes running.
INFO : Erroneous PID file will be cleaned up.
o Entered chroot with multiple terminals, saw that the PID file is
removed & the syncer process is killed when the last client exits
the chroot.
o Executed './enter_chroot.sh --verbose', changed ownership of PID file to
'root'.
Exited and saw the following diagnostic message:
INFO : Unable to clean up syncer process.
BUG=chromium-os:17680
TEST=See above
Signed-off-by: Taylor Hutt <thutt@chromium.org>
Change-Id: Idf9059509bbc6e0a926efd549922d05b9b742fa0
Signed-off-by: Taylor Hutt <thutt@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/4837
The problem here is that most were doing their exiting w/in a subshell;
exit within a subshell kills the subshell, not the parent. Not all scripts
were using set -e (which would pick up the failing subshell); as such
just rewriting them to remove the potential via eliminateing the subshelling.
Beyond that, removed a couple of custom (working, although non-standard)
approaches, and removed a duplicate common.sh sourc'ing w/in mk_memento_images.sh
TEST=force 'find_common_sh' to fail, note the scripts fails to exit
BUG=none
Change-Id: Ia1108a091a6399ad6aedd3cade4a107f4411686c
Reviewed-on: http://gerrit.chromium.org/gerrit/3905
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Brian Harring <ferringb@chromium.org>
Sometimes, two processes might start the env_syncer at the same time.
If this occurs, then the pid file will get overwritten and one of the
processes won't be killed.
I've also fixed the env_syncer to properly daemonize, such that
cbuildbot won't wait on the env_syncer to be killed before exiting.
This requires closing stdin, stderr, and stdout.
BUG=chromium-os:17680
TEST=Reproduce hang, and verify that this fixes the hang. Verify that
multiple chroots can be entered at the same time and that the
env_syncer is killed when the last chroot exits. Verify that
cbuildbot is not kept hanging even if the env_syncer doesn't
exit.
Change-Id: I3cb9fbdf47a406e818e12cf91593b8e3481aa578
Reviewed-on: http://gerrit.chromium.org/gerrit/4232
Reviewed-by: Ryan Cui <rcui@chromium.org>
Tested-by: David James <davidjames@chromium.org>
enter_chroot is often called from multiple processes and we expect that
it doesn't modify anything unless it's within a lock. This CL moves
chroot modifications up to occur during the lock so that we don't have
multiple modifications to .gitconfig occurring at the same time.
This prevents the following warning:
error: could not lock config file
/b/cbuild/chroot/home/chrome-bot/.gitconfig: File exists
See http://chromeos-botmaster.mtv.corp.google.com:8026/builders/TOT%20Pre-Flight%20Queue/builds/3771/steps/Test/logs/stdio
BUG=chromium-os:17661
TEST=Run enter_chroot.sh and verify gitconfig is still setup.
Change-Id: I73a7755d62cce895c76b8e0f35838b3874e5db33
Reviewed-on: http://gerrit.chromium.org/gerrit/4208
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: David James <davidjames@chromium.org>
Currently, enter_chroot.sh uses a hack to ensure that "--" is preserved in
commands. This is only necessary if you want to run commands that start with
hyphens. E.g. if you wanted to run a command called "--foo", enter_chroot
would break. But we don't have any existing commands that start with hyphens,
so this logic seems unnecessary.
This also makes enter_chroot more flexible, i.e. we now support case #6 below,
which was not supported previously.
Here are use cases:
1. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2 -- cmd arg1 arg2
Set env vars and run cmd w/ args
2. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2
Set env vars and run shell
3. ./enter_chroot [chroot_flags] -- cmd arg1 arg2
Run cmd w/ args
4. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2 cmd arg1 arg2
Like #1 _if_ args aren't flags (if they are, enter_chroot will claim them)
5. ./enter_chroot [chroot_flags] cmd arg1 arg2
Like #3 _if_ args aren't flags (if they are, enter_chroot will claim them)
6. ./enter_chroot [chroot_flags] -- VAR1=val1 VAR2=val2 cmd arg1 arg2
Set env vars and run cmd w/ args
7. ./enter_chroot
Just enter the chroot.
BUG=chromium-os:17468
TEST=Test above cases
Change-Id: I1801ac3927aacddd6d556c5939d3a42b31252675
Reviewed-on: http://gerrit.chromium.org/gerrit/3910
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: David James <davidjames@chromium.org>
The default is --standard_backdoor, which installs well-known ssh keys and sets
a well-known root password. Passing --nostandard_backdoor will cause
mod_image_for_test to use ssh keys from ~/.ssh/*.pub instead of the test keys
and not set the root password.
BUG=chromium-os:11744
TEST=Adhoc
Build an image with --standard_backdoor.
ssh -i ${SRC}/src/scripts/mod_for_test_scripts/ssh_keys/testing_rsa root@${DUT}
ssh root@${DUT} with 'test0000'
cat /root/.ssh/authorized_keys # check for the test key
Build an image with --nostandard_backdoor.
ssh -o PubkeyAuthentication=no root@${DUT} # this will fail
ssh root@${DUT} # this should work
cat /root/.ssh/authorized_keys # check for just your keys
Change-Id: Ie92fbc9d3815f478698c8c94d938daca2b5cd53e
Signed-off-by: Elly Jones <ellyjones@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/3449
Reviewed-by: David Rochberg <rochberg@chromium.org>
Some ssh configuration options are not accepted by the ssh
version running inside chroot. Those options need to be filtered
out when the configuration is copied while executing
enter_chroot.sh. A new function is being added to do that. The
list of substrings to be filtered out is defined as an array and
can be extended as required.
BUG=chromium-os:16441
TEST=manual:
scripts 78 $ egrep '(UseProxyIf=|GSSAPIAuthentication no)' ~/.ssh/config
UseProxyIf=false
scripts 79 $ ./enter_chroot.sh
(Grepo1) vbendeb@eskimo ~/trunk/src/scripts $ egrep '(UseProxyIf=|GSSAPIAuthentication no)' ~/.ssh/config
(Grepo1) vbendeb@eskimo ~/trunk/src/scripts $
Change-Id: Ic52ef1ba7d015d76558efc39e178156f3d81bf78
Signed-off-by: Vadim Bendebury <vbendeb@chromium.org>
Reviewed-on: http://gerrit.chromium.org/gerrit/2515
Reviewed-by: Chris Sosa <sosa%chromium.org@gtempaccount.com>
These hacks are all at least three months old, and our official policy is to
remove hacks after two weeks. For future hacks, the plan is to use the chroot
upgrader script so that old chroots can be supported indefinitely.
Thanks to the Gerrit transition, there are probably few old chroots around
anyway that aren't going to be recreated anyway.
BUG=chromium-os:16151
TEST=Verify enter_chroot.sh still works.
Change-Id: Ie0c777aabbd70d2a0c88b59ddd71e9b67965ca94
Reviewed-on: http://gerrit.chromium.org/gerrit/2158
Reviewed-by: Raymes Khoury <raymes@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Tested-by: David James <davidjames@chromium.org>
BUG=chromium-os:10048
TEST=Enter chroot with http_proxy set and check export.
Change-Id: I590bd38d22ca1be8164d2a2b408230163d9e5777
Reviewed-on: http://gerrit.chromium.org/gerrit/344
Reviewed-by: David James <davidjames@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
Tested-by: Doug Anderson <dianders@chromium.org>
Currently used for resolv.conf and hosts, because these files can
change during the lifetime of a chroot, for example on computers
with more dynamic network (laptops).
While this creates a persistent process in the background for the
sole purpose of syncing files, the performance impact is negligible.
BUG=12316
TEST=below
1) enter_chroot once+quit, many times + quit, verify correct exit
behaviour
2) enter_chroot, modify host resolv.conf, see chroot being updated
Change-Id: I26573570c027acc2c214a00838a6f982a7585b13
R=robotboy@chromium.org,dparker@chromium.org,sosa@chromium.org
Review URL: http://codereview.chromium.org/6720005
git config doesn't behave well if there are multiple callers
calling it at the same time. Since this is best effort anyway,
I'm adding an || true so that multithreads calls to this don't
fail.
I removed my previous work around to this since it wasn't generic enough.
Change-Id: I12f2d3faaa745c1ff675a297bb09c567a88aa185
BUG=chromium-os:13070
TEST=Ran it a lot
Review URL: http://codereview.chromium.org/6693001
BUG=chromium-os:11944
TEST=the correct test for this is a complete rebuild; I have this running, but it's going to take a little while. I have tested this on the specific cases of ssh_auth_sock and .subversion
Change-Id: I61723356c58bfb7c2090e950208b8a6ab8fa2fc9
Review URL: http://codereview.chromium.org/6519022
Also refactors some cut-and-paste code
Fix: makes mountpoint for subversion directory
BUG=chromium-os:11944 chromium-os:12058
TEST=enter_chroot.sh with and without another enter_chroot running. Note that there are 0 or 1 mounts of the /tmp/ssh-.... directory. Check that /proc,/sys,/dev,$SSH_AUTH_SOCK,/dev/pts,/home/${USER}/trunk,/home/${USER}/.subversion are all mounted. Test that subversion mountpoint is created
Change-Id: I9dada6f7f98d263345af29a5734c1c70709f6a1e
Review URL: http://codereview.chromium.org/6498001
Review URL: http://codereview.chromium.org/6525020
Also refactors some cut-and-paste code
BUG=11944
TEST=enter_chroot.sh with and without another enter_chroot running. Note that there are 0 or 1 mounts of the /tmp/ssh-.... directory. Check that /proc,/sys,/dev,$SSH_AUTH_SOCK,/dev/pts,/home/${USER}/trunk,/home/${USER}/.subversion are all mounted
Change-Id: I9dada6f7f98d263345af29a5734c1c70709f6a1e
Review URL: http://codereview.chromium.org/6498001
Change-Id: I6593336a52b7298bd358f3e55531b2e6a9e6dff1
BUG=chromium-os:11883
TEST=Tried deleting the symlink and deleting the line from bashrc and things got updated; ran again w/ no problems.
Review URL: http://codereview.chromium.org/6480005
Change-Id: I5f76f0f27b5789d88740b36fa6ce696ef7e68e8a
BUG=chromium-os:11602
TEST=Tried steps in bug 11602 (but hacked a sudo -K to the start of teardown_env so I didn't need to wait 15 mins).
Review URL: http://codereview.chromium.org/6286069
Change-Id: Ibd32a4be12e88d183dd0acbcb156582ad946c92f
BUG=chromium-os:11602
TEST=Added a 'sudo -K' as the first line of teardown_env to simulate sudo timeout, then performed steps in bug report (without 15 minute wait).
Review URL: http://codereview.chromium.org/6250116
from within the chroot.
It also fixes a number of style issues.
It changes the meaning of cros_workon "list-all" to list all available
packages, and adds "list-live" to list all live packages.
It changes things that load chromeos-common.sh from the installer to
load it from /usr/lib/installer.
BUG=chromium-os:4230
TEST=synced, rebuilt chroot, made packages, made images, built chrome
from source, and wrote an image to a USB stick.
Review URL: http://codereview.chromium.org/6240018
Change-Id: I90c34420af1a64020402bafef8e9e77f56837c02
It's very rarely needed anyway but enter_chroot does everything under the sun.
Avoids:
error: could not lock config file ~/.gitconfig
Change-Id: Ide72b14fd434b182a88d2fc636559b3515905509
BUG=chromium-os:11523
TEST=Ran it ... 10x
Review URL: http://codereview.chromium.org/6312069
TEST=clean (first) enter_chroot.sh; repo sync
The developer instructions for setting up chroot include steps to set up
~/.ssh/config to correctly set up ports and options. And while several
attempts are made in enter_chroot to forward ssh-agent and ssh
authentication to chroot as well, the ssh config stays outside of
chroot, which prevents all attempts to contact the git servers if rw URL
is used.
This makes a copy of config on entering the chroot, allowing people to
use repo from within the chroot. That is not bulletproof and does not
reflect changes of config while inside the chroot, but that should be a
borderline case.
Change-Id: I1fcbf00c413c7af8ef14fc1e846192ac95de106d
BUG=n0ne
Review URL: http://codereview.chromium.org/6065011
Examples of how people might be using enter_chroot:
1. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2 -- cmd arg1 arg2
Set env vars and run cmd w/ args
2. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2
Set env vars and run shell
3. ./enter_chroot [chroot_flags] -- cmd arg1 arg2
Run cmd w/ args
4. ./enter_chroot [chroot_flags] VAR1=val1 VAR2=val2 cmd arg1 arg2
Like #1 _if_ args aren't flags (if they are, enter_chroot will claim them)
5. ./enter_chroot [chroot_flags] cmd arg1 arg2
Like #3 _if_ args aren't flags (if they are, enter_chroot will claim them)
I also updated the help to indicate that whole-command quoting is no longer
supported. If you really need whole-command quoting (maybe you want in-
chroot redirection), you'll need to use sh -c like:
./enter_chroot.sh -- sh -c "echo \"Save me\" > /tmp/save.txt"
You should avoid single quotes in the command and arguments. This isn't
a new limitation: it's shflags related.
Change-Id: I0452a8730ac9b8197834edc753b9eece69896135
BUG=chromium-os:7072
TEST=Tried a whole bunch of these commands.
Review URL: http://codereview.chromium.org/5840003
Change-Id: I906f1959769ac9d7a2abc04827cad8f5201984de
BUG=chromeos:9168
TEST=Made sure that setting and unsetting this value and enter_chroot is honoring the pass thru setting.
Review URL: http://codereview.chromium.org/5008006
This function is now called by enter_chroot.sh. A separate change
will call the function from make_chroot
Change-Id: I4fc07c413e56db3e5e7617428f20f2ffc02790d7
BUG=chromium-os:7072
TEST=Took root out of sudoers and ran enter_chroot.sh script; saw that it was re-added.
Review URL: http://codereview.chromium.org/3909001
BUG=none
TEST=run from within and without a repo
Change-Id: I00eb999974656450f6af24583ca2247805b5799e
Review URL: http://codereview.chromium.org/3529005
If git author name is different in the chroot from outside the chroot, repo
may auto-delete your changes when you run repo sync. These changes can be
recovered, but their deletion is inconvenient.
BUG=chromium-os:5934
TEST=Check that git var GIT_COMMITTER_IDENT is now correct inside chroot.
Change-Id: I622cd37caa404a7482c55c4b3984877b32b3807c
Review URL: http://codereview.chromium.org/3352008
This also mounts the path to our ssh-agent socket (usually in /tmp) inside the
chroot so we can use our external agent.
TEST=None
BUG=None
Change-Id: I543e8b2527be9958c1158234f39ecc34fc9dd0df
Signed-Off-By: Elly Jones <ellyjones@chromium.org>
Signed-Off-By: sosa <sosa@chromium.org>
Review URL: http://codereview.chromium.org/3277006
so that the original users' subversion access permissions are preserved inside the chroot
Change-Id: I486070b3c1a2dda169ae0a95982ba693574e001b
BUG=
TEST=
Review URL: http://codereview.chromium.org/3249008
It's sometimes useful to run a command in chroot and redirect/pipe stdout. Any objections to having enter_chroot send its chatter to stderr?
Review URL: http://codereview.chromium.org/3160024
This fix separates out any additional command line arguments given to
the enter_chroot.sh script when passed to the interior chroot. This
allows one to pass environment variable settings to the interior
chroot shell such as:
./enter_chroot.sh -- CHROME_ORIGIN=LOCAL_SOURCE emerge-x86-generic ...
Before this fix any of these variable settings will be sucked up by
the shell launching the chroot, not the interior shell.
BUG=chromium-os:2457
TEST=ran test cases detailed in bug report
Review URL: http://codereview.chromium.org/2855034
* This also adds reverse sort into the umount list, because of cascading mounts, so that /foo/anything is always before /foo, and as such is always unmounted first. If the list were not sorted, "mount point busy" errors might occur, depending on various conditions.
TEST=
1) With a previously created chroot, I ran enter_chroot.sh, and got the correct prompt. After logging out of that prompt, everything else was correctly unmounted, without any error messages. When entering chroot twice from two different shells, everything was correctly unmounted after the last shell exited.
2) Repeat 1 but with cleanly created chroot just for this purpose.
3) In the chroot, make packages for one board (x86-generic) using build_packages
4) The messages "openpty failed: 'out of pty devices'" whenever trying to emerge anything (or other commands looking up pts) have disappeared.
Review URL: http://codereview.chromium.org/2714010
Added chrome_root_mount option to specify the mount point to work around a permissions issue.
BUG=3464
TEST=none
Review URL: http://codereview.chromium.org/2126011
Unfortunately, bind mounting /media inside the chroot did not allow us
to programmatically unmount automounted USB sticks from within the chroot.
Instead, I disable automounting on our corporate desktops when you enter the
chroot. Your normal setting is restored when you exit the chroot again.
TEST=run gconftool-2 -g /apps/nautilus/preferences/media_automount to print the current value of your automount pref; run this before entering the chroot, outside the chroot while you've got some other terminal that's inside the chroot, and then after exiting the chroot.
Review URL: http://codereview.chromium.org/2029009
Talked to kmixter, and we felt that bindmounting the directory in
which the host os creates mount points for automounted external
devices is the cleanest solution to the inside/outside the chroot problem.
By doing this, image_to_usb can be run from inside the chroot without concern.
That means that (as far as I understand) the factory install flow can all be
done inside the chroot, as well as modding images for test and everything else.
Review URL: http://codereview.chromium.org/1991006
