When running bootstrap_sdk for the first time on a host the default of
using a previous catalyst run as a seed won't work but this may be a
fresh SDK install so that tarball is probably around somewhere and will
work as a default seed for most things.
gsutil can be hard to follow when parallel upload/downloads are enabled.
"I see it is transferring something, but what?" So this provides an
option to disable that for debugging purposes.
The name "coreos-sdk-amd64-..." makes much more sense for general
distribution than "stage4..." so after catalyst is done rename the final
tarball and fixup the DIGESTS file to refer to the new name.
Temporary workaround until we have a version of emirrordist that can
operate directly on remote storage. Download the entire mirror (if it
isn't already local) so emirrordist can make an incremental update.
Not great but it will do for now.
This script wraps emirrordist to scan all ebuilds and fetch every
SRC_URI to construct a distfiles mirror. It can then optionally upload
to storage.core-os.net via gsutil.
Note: This is a work-in-progress, emirrordist expects to operate on a
complete mirror on the local filesystem but maintaining a complete local
copy of everything is burdensome. Before this is truly practical a
modified version of emirrordist will be needed to either accept a list
of already-uploaded files it can assume are OK or operate on the cloud
storage system directly.
This file mostly just defined BINHOSTs but I'd like to move that
completely into coreos-overlay as the SDK BINHOSTs have always been. It
also sourced src/overlays/overlay-amd64-generic/make.conf but I want to
move the few things it defined into coreos-overlay as well.
The one interesting thing this file did was to optionally define
ACCEPT_LICENSE but that can go into /etc/make.conf.board_setup instead.
Overall this takes a chunk out of the make.conf spaghetti. :)
By default emerge will not install build dependencies if it doesn't need
them (i.e. installing a binary package) but we want to make sure
everything gets included in stage4 no matter how it was installed.
This uses Gentoo's catalyst for very thoroughly building images from
scratch. Using images based on this will eliminate some of the hackery
in make_chroot.sh for building up the sdk from a stock stage3 tarball.
For reference the procedure it performs is this:
1. snapshot: Grab a snapshot of portage-stable. Note that overalys are
not snapshotted.
2. stage1: Using a "seed" tarball as a build environment, build a
minimal root file system into a clean directory using ROOT=...
and USE=-* The restricted USE flags are key be small and avoid
circular dependencies.
3. stage2: Run portage-stable/scripts/bootstrap.sh
This rebuilds the toolchain. Probably not strictly necessary most of
the time but does super-duper-promise that the toolchain isn't linked
to or otherwise influenced by whatever was in the "seed" tarball.
4. stage3: Run emerge -e system to rebuild everything using the fresh
toolchain using the normal USE flags provided by the profile. This
will also pull in assorted base system packages that weren't included
in the minimal environment stage1 created.
5. stage4: Install any extra packages or other desired tweaks. For the
sdk we just install all the packages normally make_chroot.sh does.
Meant to add this last week... It can either pull from Gentoo CVS or a
local directory (in case you rsynced the whole portage tree). Just name
a package by pkg-cat/name and it will update portage-stable.
As-is all of the various emerge wrapping scripts default to using
--getbinpkg whenever --usepkg is enabled. This means every single emerge
command made makes multiple synchronous HTTP requests to the upstream
binary package repository to get the latest package list. This gets
really frustrating when working remotely with limited network
connectivity. Using --usepkg with --nogetbinpkg will use locally cached
packages without making remote requests.
This script will boot the build target root via systemd-nspawn which is
a mighty bit faster than building and booting vm images. :)
Note: systemd-nspwan doesn't do anything special for networking so port
conflicts for things like sshd are to be expected. Works though. :)
During builds var_overlay is always mounted over /var. We want to do the
same at run time but we also want to ensure everything expected to be
there always does. After emerge completes gen_tmpfiles.py will scan /var
for any .keep files that were installed and records their parent
directories' permissions and ownership to /usr/lib/tmpfiles.d. On each
boot systemd will automatically recreate anything that goes missing.
This also means that going forward any ebuild that needs a directory in
/var (or anywhere else the stateful partition is bound) can simply rely
on the 'keepdir' ebuild function instead of adding things to
coreos_startup.
The UUID changed for CoreOS. Make sure we use our own shiny UUID.
Add a doc note that the update should be generated from the "template
image" not from one of the vm images.
As outlined here we need a new partition layout, this patch makes the
necessary changes:
https://groups.google.com/forum/#!topic/coreos-dev/bA7gwGGoTng
The first big change is making all of the scripts obey partition numbers
based on labels in the disk_layout.json. This makes it much easier to
change later on.
The second big change is in the layout itself. The json file was updated
to reflect the document above.
And finally the grub boot configuration needed for pv-grub and pygrub
were added to the create_legacy_bootloader_templates.sh library utlity.
Everything seems to work and boot now.
xen requires a /boot/grub/menu.lst for pygrub and pvgrub on partition 1.
Put it on the stateful partition for now and come back around and fix
this up when we redo the partition layout.
this is a bit of a hack but I wanted to see if it had any utility during
development before making it all pretty. Essentially this is a copy of
build_image but instead of building up an entire image it simply puts
the files into directories on disk to be ran with systemd-nspawn/lxc/etc
so it is a bit complicated but essentially gtest pulls in python which
pulls in pyton-updater which wants portage so portage gets installed in
teh real root not the dev one. Just leave it for now.
use the efunctions package for the /etc/init.d/functions.sh script
instead of backing up the old function.sh which doesn't work with the
new baselayout
we remove openrc which provides /etc/init.d/functions.sh. Unfortunatly
other things rely on this file. Stash it away in /tmp/ then restore it
for now.
Change-Id: I18a59e05ecdf08cc8a560b29049c8d25ac1bf5a3
