It isn't worth renaming the directory used on the mirror, and the
Portage configuration still points to the old name.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Catalyst 4 has totally changed the way repositories are handled. It only
works when the name of the directory containing the repository matches
the configured name of that repository. This was not the case for us,
with the coreos repository residing in the coreos-overlay directory. We
wanted to move and rename our repositories anyway, but this is a big
change, so we'll do separately. For now, this just renames coreos to
coreos-overlay.
Catalyst 4 also ingests the main repository snapshot as a squashfs
rather than a tarball. It features a utility to generate such a
snapshot, but it doesn't fit Flatcar well, particularly because it
expects each ebuild repository to reside at the top level of its own git
repository. It was very easy to call tar2sqfs manually though.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
We have never actually deleted anything from our mirror since gsutil cp
didn't really have a way to do that and the for some unknown reason
emirrordist never tried to delete anything anyway. Additionally we
actually don't want to delete anything from our mirror to ensure that
the source remains available for old releases.
The distfiles cache is always under .cache in the repo tree but there is
a lot of extra logic to make that configurable along with compatibility
symlinks for previous locations. Just yank it all out.
gsutil can be hard to follow when parallel upload/downloads are enabled.
"I see it is transferring something, but what?" So this provides an
option to disable that for debugging purposes.
Temporary workaround until we have a version of emirrordist that can
operate directly on remote storage. Download the entire mirror (if it
isn't already local) so emirrordist can make an incremental update.
Not great but it will do for now.
This script wraps emirrordist to scan all ebuilds and fetch every
SRC_URI to construct a distfiles mirror. It can then optionally upload
to storage.core-os.net via gsutil.
Note: This is a work-in-progress, emirrordist expects to operate on a
complete mirror on the local filesystem but maintaining a complete local
copy of everything is burdensome. Before this is truly practical a
modified version of emirrordist will be needed to either accept a list
of already-uploaded files it can assume are OK or operate on the cloud
storage system directly.
