We have never actually deleted anything from our mirror since gsutil cp
didn't really have a way to do that and the for some unknown reason
emirrordist never tried to delete anything anyway. Additionally we
actually don't want to delete anything from our mirror to ensure that
the source remains available for old releases.
The distfiles cache is always under .cache in the repo tree but there is
a lot of extra logic to make that configurable along with compatibility
symlinks for previous locations. Just yank it all out.
gsutil can be hard to follow when parallel upload/downloads are enabled.
"I see it is transferring something, but what?" So this provides an
option to disable that for debugging purposes.
Temporary workaround until we have a version of emirrordist that can
operate directly on remote storage. Download the entire mirror (if it
isn't already local) so emirrordist can make an incremental update.
Not great but it will do for now.
This script wraps emirrordist to scan all ebuilds and fetch every
SRC_URI to construct a distfiles mirror. It can then optionally upload
to storage.core-os.net via gsutil.
Note: This is a work-in-progress, emirrordist expects to operate on a
complete mirror on the local filesystem but maintaining a complete local
copy of everything is burdensome. Before this is truly practical a
modified version of emirrordist will be needed to either accept a list
of already-uploaded files it can assume are OK or operate on the cloud
storage system directly.
