This is intended to be called after branch_release and all official
builds from that new branch are complete. Then updating master's
version.txt will switch things to start using that new release as the
source for binary packages.
Complete documentation for this whole process coming soon. :)
A few things here:
- Source manifests/version.txt directly instead of coreos-version.sh
- Remove Chrome branch from target image directory names.
- Use proper version instead of timestap for catalyst builds.
- Move lsb_release script from coreos-overlay to build_library.
Nothing in build_packages references the variables from
coreos_version.sh and nothing in our ebuilds depends on it being in the
environments so just remove it.
The big here document in the middle of a if statement confuses the code
layout some and I put the git commit outside of the else clause instead
of inside. Ooops. The joy of shell.
When running bootstrap_sdk for the first time on a host the default of
using a previous catalyst run as a seed won't work but this may be a
fresh SDK install so that tarball is probably around somewhere and will
work as a default seed for most things.
gsutil can be hard to follow when parallel upload/downloads are enabled.
"I see it is transferring something, but what?" So this provides an
option to disable that for debugging purposes.
The name "coreos-sdk-amd64-..." makes much more sense for general
distribution than "stage4..." so after catalyst is done rename the final
tarball and fixup the DIGESTS file to refer to the new name.
Temporary workaround until we have a version of emirrordist that can
operate directly on remote storage. Download the entire mirror (if it
isn't already local) so emirrordist can make an incremental update.
Not great but it will do for now.
This script wraps emirrordist to scan all ebuilds and fetch every
SRC_URI to construct a distfiles mirror. It can then optionally upload
to storage.core-os.net via gsutil.
Note: This is a work-in-progress, emirrordist expects to operate on a
complete mirror on the local filesystem but maintaining a complete local
copy of everything is burdensome. Before this is truly practical a
modified version of emirrordist will be needed to either accept a list
of already-uploaded files it can assume are OK or operate on the cloud
storage system directly.
This file mostly just defined BINHOSTs but I'd like to move that
completely into coreos-overlay as the SDK BINHOSTs have always been. It
also sourced src/overlays/overlay-amd64-generic/make.conf but I want to
move the few things it defined into coreos-overlay as well.
The one interesting thing this file did was to optionally define
ACCEPT_LICENSE but that can go into /etc/make.conf.board_setup instead.
Overall this takes a chunk out of the make.conf spaghetti. :)
By default emerge will not install build dependencies if it doesn't need
them (i.e. installing a binary package) but we want to make sure
everything gets included in stage4 no matter how it was installed.
This uses Gentoo's catalyst for very thoroughly building images from
scratch. Using images based on this will eliminate some of the hackery
in make_chroot.sh for building up the sdk from a stock stage3 tarball.
For reference the procedure it performs is this:
1. snapshot: Grab a snapshot of portage-stable. Note that overalys are
not snapshotted.
2. stage1: Using a "seed" tarball as a build environment, build a
minimal root file system into a clean directory using ROOT=...
and USE=-* The restricted USE flags are key be small and avoid
circular dependencies.
3. stage2: Run portage-stable/scripts/bootstrap.sh
This rebuilds the toolchain. Probably not strictly necessary most of
the time but does super-duper-promise that the toolchain isn't linked
to or otherwise influenced by whatever was in the "seed" tarball.
4. stage3: Run emerge -e system to rebuild everything using the fresh
toolchain using the normal USE flags provided by the profile. This
will also pull in assorted base system packages that weren't included
in the minimal environment stage1 created.
5. stage4: Install any extra packages or other desired tweaks. For the
sdk we just install all the packages normally make_chroot.sh does.
Meant to add this last week... It can either pull from Gentoo CVS or a
local directory (in case you rsynced the whole portage tree). Just name
a package by pkg-cat/name and it will update portage-stable.
As-is all of the various emerge wrapping scripts default to using
--getbinpkg whenever --usepkg is enabled. This means every single emerge
command made makes multiple synchronous HTTP requests to the upstream
binary package repository to get the latest package list. This gets
really frustrating when working remotely with limited network
connectivity. Using --usepkg with --nogetbinpkg will use locally cached
packages without making remote requests.
This script will boot the build target root via systemd-nspawn which is
a mighty bit faster than building and booting vm images. :)
Note: systemd-nspwan doesn't do anything special for networking so port
conflicts for things like sshd are to be expected. Works though. :)
During builds var_overlay is always mounted over /var. We want to do the
same at run time but we also want to ensure everything expected to be
there always does. After emerge completes gen_tmpfiles.py will scan /var
for any .keep files that were installed and records their parent
directories' permissions and ownership to /usr/lib/tmpfiles.d. On each
boot systemd will automatically recreate anything that goes missing.
This also means that going forward any ebuild that needs a directory in
/var (or anywhere else the stateful partition is bound) can simply rely
on the 'keepdir' ebuild function instead of adding things to
coreos_startup.
