If a board has a public overlay it will be used as the primary overlay. However, with this change, if a board only has a private overlay it will be used as the primary.
BUG=7339
TEST=Ran setup_board, build_packages, and build_image on a board with only a private overlay (locally created). Ran setup_board, build_packages, and build_image on a board with both public and private overlays.
Change-Id: Ic56e4951272ddb80511a213e3a2e54419267a49a
Related Issues:
http://codereview.chromium.org/3544009http://codereview.chromium.org/3571015
Review URL: http://codereview.chromium.org/3622003
The --fast support for build_packages is stable now, and should significantly
improve the speed of our official builders. We should turn it on so that
builders can finish their builds in less than 8 hours.
BUG=chromium-os:6706
TEST=Run ./enter_chroot.sh --official_build and verify that --fast is on by
default now
Change-Id: I6ad126b9b6ce16ffc9887a7af22c2e3f85afbf42
Review URL: http://codereview.chromium.org/3418001
When the server returns a 5xx error code, we should retry. Such errors are
often due to flakiness, so retries resolve the issue.
BUG=chromium-os:9217
TEST=Tested parallel_emerge with fake URL to verify it retries.
Change-Id: Ie5cf003562b5261005be9929624769394b4802a1
Review URL: http://codereview.chromium.org/5056003
BUG=chromium-os:8814
TEST=Verify packages with old use flags are updated, and that binary packages
with wrong use flags aren't used.
Change-Id: I7064a10796f1541ff9ae36f2ba6ef1d65dbe1aa3
Review URL: http://codereview.chromium.org/4552007
This feature is useful if you want to stick to an old binary package of a package you're not working on.
BUG=chromium-os:8769
TEST=./parallel_emerge --board=x86-mario -uDNvg chromeos --force-remote-binary=chromeos-chrome --force-remote-binary=libcros
Change-Id: I7d3011fa64134158ed848f136bc75e09b0af438e
Review URL: http://codereview.chromium.org/4555002
Currently, if a build is slow, you only get debug output after an hour.
This is to allow for uncluttered output. If output is too cluttered, it's hard
to distinguish regular output from errors.
The problem with this approach is that it's often hard to debug why the build
is slow. Now that Chrome builds by default, it takes over an hour to build,
and people see little indication as to why. You can show the output with
build_packages --showoutput, but that is often too verbose and clutters the
output too much.
Here's an example log snippet that is hard to debug:
Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 5m20.5s Load 3.69 3.04 2.66]
Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 5m25.5s Load 3.40 2.99 2.65]
... yada yada yada ...
Pending 2, Ready 0, Running 1, Retrying 0, Total 22 [Time 45m32.9s Load 1.00 1.18 2.95]
From the above output, we see that a package is building for a long time, but
we don't know what package. We should output the package name every two minutes
at least so people know what package is taking so long. That's what this change
implements.
BUG=chromium-os:8575
TEST=Confirmed new status appear for regular build_packages. Confirmed
build_packages --showoutput is unchanged.
Change-Id: Ie18b23ac7a8a6e2c24b43ec3691606c7da5e43cb
Review URL: http://codereview.chromium.org/4318003
chromeos-chrome has mostly self-contained dependencies, so there usually
isn't need to rebuild it just because a dependency changed. This allows for
us to use the binaries for chromeos-chrome more often.
BUG=chromium-os:8394
TEST=Check that chrome doesn't rebuild with
./parallel_emerge -gp --workon=libcros --board=x86-generic
chromeos-chrome
Change-Id: Ifa14c890917991a8d11f1f0e757f28686d611a72
Review URL: http://codereview.chromium.org/4243003
I've cleaned up parallel_emerge to send explicit signals to children to tell
them when they need to exit. I also cleaned up the CTRL-C handling to
correctly print data when interrupted by CTRL-C (previously, the print
thread exited first, so the data we wanted on the failure wasn't
actually printed.)
BUG=chromium-os:5976
TEST=Test several board emerges, including exiting early with CTRL-C.
Change-Id: Iab6efc8e1bf868106244a6210bd02e9e8283af37
Review URL: http://codereview.chromium.org/3534006
So, we've been seeing parallel_emerge hang after printing "Done". It's hard to
know for sure why this hang is happening, but my theory right now is that the
Python garbage collector doesn't always know that it needs to kill the child
processes when we exit normally at the bottom of the script. Adding sys.exit(0)
tells Python that yes, we really want to exit.
I haven't tested my hypothesis above, because the hang we've been seeing is
pretty rare and is hard to reproduce. So I'm planning on testing by just
putting in this workaround and seeing if it fixes the problem.
TEST=Run ./parallel_emerge world
BUG=chromium-os:5976
Change-Id: I7b4f2ec4ccba9b00f22f3739dfd3eff51ceed425
Review URL: http://codereview.chromium.org/3448030
BUG=chromium-os:7049
TEST=Tested that DNS errors are retried. Also tested that 404 errors are not retried.
Change-Id: I0f38764e7af822f7db554697b9fa84af033bf111
Review URL: http://codereview.chromium.org/3473017
TEST=Add impossible-to-satisfy constraints into the dependency graph
that require packages to be reinstalled due to use flags, and run
./parallel_emerge -p chromeos
BUG=chromium-os:6625
Change-Id: I389f7e5ba647a4be0413d1b0b6ea079df5b56433
Review URL: http://codereview.chromium.org/3459006
Three changes here:
1) parallel_emerge now only disables PORTAGE_LOCKS on board builds. This
slows down make_chroot --fast, but works around a bug where make_chroot
--fast sometimes merges broken packages.
2) Only set PORTAGE_LOCKS if it's not already set. This allows users to
override PORTAGE_LOCKS and request that it be enabled using
PORTAGE_LOCKS=true
3) Only add the no-env-update feature if PORTAGE_LOCKS=false. This feature
is only needed for that case.
Long term fix is to patch PORTAGE_LOCKS feature to lock postinst. That'll be
tracked in a separate bug.
BUG=chromium-os:6750
TEST=Run make_chroot --fast. It works every time now, but is slower.
build_packages && build_image run at same speed as before.
Change-Id: I12ab40671034e10cd2ffbba45281ca44718d1d2c
Review URL: http://codereview.chromium.org/3446006
When --getbinpkg is not used, skip looking on the remote server for
packages. This should allow people to use parallel_emerge with no
binary packages when the network is unavailable.
BUG=chromium-os:6684
TEST=Disconnect network and run ./build_packages --nousepkg
Change-Id: Id0e5d03aca30d85cb6990d977983f1e0655726ab
Review URL: http://codereview.chromium.org/3294022
It turns out that my previous rebuild_blacklist code had a bug where
blacklisted packages would get rebuilt unnecessarily. This would result
in slow builds if icedtea was built.
BUG=chromium-os:6628
TEST=Mock case where prebuilts for icedtea are older than packages that
icedtea depends on.
Change-Id: I498b6499b78ab065109c1fe0e77969028ceff6cb
Review URL: http://codereview.chromium.org/3308027
While testing parallel_emerge, I found more cases where it would fail
to crack cycles. The issue is that parallel_emerge simply cracks the first
cycle it finds that involves an edge, rather than all cycles that involve an
edge.
Finding all cycles that involve an edge without looping forever is a rather
difficult problem, so I've updated the algorithm to simply keep searching
until it runs out of cycles. Testing this, I haven't found any cases where
the performance of the cycle cracking is a problem. I updated the algorithm
to log its performance stats so we can track this.
TEST=emerge -ep portage
BUG=none
Change-Id: I1cb95ffe9d977b9f8d38626d2d6cdbb766c14669
Review URL: http://codereview.chromium.org/3340010
I put portage when I meant to put portage_pkg, so it was running
code unconditionally that should have only ran when portage was
being upgraded. Oops!
We also should clear the dependency list, since we don't want to
list dependencies other than portage when we're installing portage.
TBR=zbehan
TEST=parallel_emerge
BUG=none
Change-Id: Ic2861d78913f89fc42cedd96ffb125e08287f662
Review URL: http://codereview.chromium.org/3318009
We had a build failure today because portage was upgraded, and
parallel_emerge did not restart after emerging portage. This resulted
in build breakage because parallel_emerge did not pick up all of the
changes (only some of them).
BUG=chromium-os:6394
TEST=Tested that parallel_emerge restarts after upgrading portage in root,
but not after upgrading portage in board.
Change-Id: I480dc2dc034afb5b9e0468ed6e4d16de608d3d32
Review URL: http://codereview.chromium.org/3304010
parallel_emerge now checks the use flags of installed packages against new
packages, and checks the use flags of binary packages against source packages,
so that, when the appropriate flags are used, we never install packages with
the wrong use flags.
I've also fix related bugs in rebuild_blacklist. Previously, rebuild_blacklist
was too aggressive -- it even blacklisted packages from being rebuilt if they
are explicit workon packages. This bug also causes problems for the use flag
detection code because packages in this list would not get rebuilt when
the use flags changed. So I fixed this too.
Also relatedly, I've fixed our code for checking whether --selective is
enabled to look at the --newuse and --reinstall=changed-use flags. These
flags actually enable --selective implicitly, so we need to honor that for
correct behavior.
BUG=chromium-os:6359
TEST=Made sure that packages get rebuilt / reinstalled when use flags change.
Change-Id: I2d945b552863410ec7b286c3ed55ae4a64336ed5
Review URL: http://codereview.chromium.org/3361002
I've cleaned up the cycle cracking in parallel_emerge to hopefully be
significantly easier to understand. This also fixes some bugs.
Most significantly, this patch restores the dependency cracking so that it
actually cracks cycles that are against the emerge ordering, instead of
cracking everything but those dependencies. This was only broken temporarily
due to my fix for Bug 5795. Oops. Fortunately, this bug doesn't affect many
people, because make_chroot --fast is deprecated anyway.
TEST=make_chroot --fast, parallel_emerge -pe hard-host-depends
BUG=chromium-os:5795
Review URL: http://codereview.chromium.org/3184011
Previously, parallel_emerge only broke counter-plan dependencies if they were
mutually cyclic. This doesn't work, because these deps might be involved in
an unrelated cycle.
To fix this, we break all counter-plan dependencies that are involved in any
cycle. This fixes make_chroot --fast, which currently fails with a deadlock.
Besides the above, I've also added additional cycle-checking to the install
plan generation code so that cyclic dependencies are caught earlier. I also
cleaned up FindCyclesAtNode to be a bit more understandable -- I wasn't 100% sure
that it was actually correct so I cleaned it up so that it was easier for me to
understand that it is in fact correct. Not sure that this part fixes any bugs,
but it's easier for me to analyze the behavior of FindCyclesAtNode now.
TEST=make_chroot --fast (Note that make_chroot --fast is deprecated, but it's
good to fix the cycle cracking anyway.)
BUG=chromium-os:5795
Review URL: http://codereview.chromium.org/3156018
When we pass on --jobs to individual emerge processes, it causes them
to hide their output. This makes it harder to debug why they are
failing or hanging. Example: http://goo.gl/C6Ru
TEST=Made sure that output is shown by emerge processes when --jobs is used.
BUG=chromium-os:5684
Review URL: http://codereview.chromium.org/3168012
This should cut several minutes off the time for build_packages --nousepkg
--showoutput.
TEST=Ran build_packages --showoutput, verified output.
BUG=chromium-os:5647
Review URL: http://codereview.chromium.org/3106008
By default, emerge will pause and beep when it has messages that it wants the
user to see. This beeping is not useful during a parallel merge, so we turn it
off.
TEST=Ran parallel_emerge udev. It stopped beeping and completed faster.
BUG=none
Review URL: http://codereview.chromium.org/3080005
BUG=none
TEST=run build_packages on --board=tegra2_dev-board-opengl right now, since binhost hasn't been populated yet.
Review URL: http://codereview.chromium.org/3035058
Also update parallel_emerge to print output in general if you hit
CTRL-C or if a job is running for more than an hour. Also cleanup exit
handling so that emerge exits a bit more cleanly, and so that we don't
hang if emerge throws an Exception.
BUG=none
TEST=Ran ./parallel_emerge --show-output -uDNve chromeos
Review URL: http://codereview.chromium.org/3010056
Currently, our build occasionally fails with errors about about symlinks
already existing. This usually only slows down the build, because we have
retries, but it would be good to fix the problem.
This patch updates parallel_emerge to use the no-env-update feature for cross-
compiles. This feature skips the env_update step, which isn't thread-safe, and
isn't necessary for when we're cross-compiling and don't actually need to run
the binaries. Instead we run the env_update at the end, avoiding a thread-safety
issue.
TEST=Ran full build --nousepkg with env_update occuring at end.
BUG=chromium-os:5382
Review URL: http://codereview.chromium.org/3034051
When --onlydeps is specified, we don't want to merge packages that were
specified on the command-line. Previously, it threw an exception in this
case because it couldn't find the command-line packages in the list of
installed packages.
TEST=Run ./parallel_emerge --onlydeps autotest, and make sure it
doesn't crash.
Run ./parallel_emerge --board=x86-generic -po chromeos power_manager and make sure it does emerge power_manager, and doesn't crash.
BUG=none
Review URL: http://codereview.chromium.org/2868094
The icedtea and o3d packages take a really long time to rebuild (over 30 mins
each), so we can give our developers a break if we exempt them from automatic
rebuilds.
TEST=Rev the portage ebuild or the fontconfig ebuild and run build_packages.
Without the change, takes over 45 mins. With the change, it takes 5 mins.
BUG=none
Review URL: http://codereview.chromium.org/3029051
Rather than using split, we can just use os.path.basename(path) to get
the username from the homedir name. This is a bit simpler.
I've also added a comment which I missed in my last commit.
TEST=Ran ./parallel_emerge, verified the new method of getting
PORTAGE_USERNAME gets the right username.
BUG=none
Review URL: http://codereview.chromium.org/3056002
I tried to disable parallel_emerge yesterday, but my change didn't work,
so the buildbots are still running into occasional related issues.
Portage has two flags for doing collision protection: collision-protect
and protect-owned. The protect-owned feature is enabled by default and
is quite useful: it checks to make sure that we don't have multiple
packages that own the same file. The collision-protect feature is more
strict, and less useful: it fails if it finds a conflicting file, even
if that file was created by an earlier ebuild that failed to install.
We want to disable collision-protect here because we don't handle
failures during the merge step very well -- we just leave the old
Unfortunately, we haven't quite figure out the best way to handle
these failures in parallel emerge, so for now we disable the flag.
TEST=Created a fake collision, made sure that packages still merge.
BUG=none
Review URL: http://codereview.chromium.org/2825076
collision-protect doesn't make sense for parallel_emerge, because
we don't lock around merges. Also, if merge fails part-way through,
it's strange to fail just because some files are lying around.
This fixes an issue with the buildbots.
TEST=Checked that collision-protect feature gets deleted by this.
BUG=none
Review URL: http://codereview.chromium.org/3061029
Our package graph is cyclic, so parallel_emerge needs to handle cycles
correctly in all cases. PrebuiltsReady should only need to check each package
once, so we should set cache[pkg] to True if we found the package in the cache.
TEST=Ran build_packages --fast
BUG=none
Review URL: http://codereview.chromium.org/3047009
Instead of loosely wrapping emerge, parallel_emerge now integrates tightly with emerge. This boosts performance while allowing us to map dependencies more accurately.
With this change, build_image --fast can clock under 4 minutes, and build_packages --fast now clocks as low as 4:30. With the --rebuild option, build_packages takes 5:30, but it has the big benefit of producing actually-correct results.
Note that this change also depends on us updating the various build scripts to prefix
calls to parallel_emerge with sudo -E.
TEST=Ran several parallel_emerge test cases, build_packages --fast, and
build_image --fast
BUG=none
Review URL: http://codereview.chromium.org/2891013
When we run custom actions in parallel_emerge, we should pass the list of
packages on to emerge. Otherwise emerge won't know what packages to unmerge.
Besides the above, also fix an issue where --workon doesn't override
--getbinpkgonly or --usepkgonly.
TEST=Ran parallel_emerge --unmerge dhcpd and parallel_emerge --usepkgonly --board=x86-generic
--workon=power_manager power_manager
BUG=none
Review URL: http://codereview.chromium.org/2927013
Currently, parallel_emerge doesn't know how to handle uninstall actions, so it
actually installs packages when it should be uninstalling them. This patch
fixes that.
TEST=sudo ./parallel_emerge -uDNv hard-host-depends, and make sure it
uninstalls nih-dbus-tool, which is blocked by hard-host-depends
BUG=none
Review URL: http://codereview.chromium.org/2990001
Packages specified as workon packages are always built from source.
Dependencies of workon packages are also built from source.
Dependencies are calculated with --selective=n so that workon packages will be included in the install list. Packages that are not being worked on and are being unnecessarily replaced because of --selective=n are filtered out later.
This patch also fixes a bug with world file updating -- previously the world file was never updated because the packages were getting removed by RemoveInstalledPackages. We now keep the appropriate world file updates, and instead update SanitizeTree to handle world file updates without trouble.
I also optimized the cycle cracking algorithm so that it only visits each node once. This seems to improve run time significantly on graphs that have cycles.
TEST=Ran ./parallel_emerge -uDNvpg --board=x86-generic chromeos --workon=chromeos-chrome &&
./parallel_emerge -uDNvpg --board=x86-generic chromeos --workon=libcros &&
./parallel_emerge -uDNvpg world --workon=libxml2 &&
./parallel_emerge -uDNv -p world hard-host-depends --workon='bcel'
BUG=none
Review URL: http://codereview.chromium.org/2959006
- Add support for --jobs flag.
- Add support for --depclean with no arguments.
- Cleanup comments.
- Print out error details the first time a package fails.
TEST=Ran full build_packages --fast
BUG=none
Review URL: http://codereview.chromium.org/2853031
- Unmerge appropriate packages during upgrades and downgrades.
- Calculate time spent in dependency generation to the tenth of a second.
- Only track dependencies of packages that are actually being installed.
- Ignore PDEPEND, as it has no impact on dependency ordering.
- Only break dependency chains that go against Portage's install order.
- Rename Failed -> Retrying.
- Print emerge command lines as they are run.
TEST=Emerged hard-host-depends and ran build_packages with parallel_emerge
BUG=none
Review URL: http://codereview.chromium.org/2886010
Include checked in parallel emerge,
with an optional (default false) argument
in build_image to turn it on.
Review URL: http://codereview.chromium.org/2827037
