This is a breaking change. This commit separates syncstorage and tokenserver settings into separate structs that are contained by a parent `Settings` struct. This means that any env vars that hold settings specific to syncstorage (e.g. `SYNC_DATABASE_URL`) have been renamed to `SYNC_SYNCSTORAGE__DATABASE_URL`. Any settings that were moved from the top-level `Settings` struct to the lower level, syncstorage-specific struct will now have a `SYNC_SYNCSTORAGE__` prefix instead of a `SYNC_` prefix.
Closes#1276
* Support a mode option in purge_ttl
* Support an expiry mode option in purge_ttl
* Support serially deleting prefixes by regex
Also changes the collection IDs option from a JSON list to an args list
(i.e. [item1,item2,item3]).
Closes#735Closes#743
Added `--uid_starts` option which will limit purge_ttl bso deletes to
fxa_uids that begin with specified characters.
(e.g.
`purge_ttl.py --uid_starts="AA" --collection_ids=[7,8]`
will limit to scanning bso tables to fxa_uid values that begin with "AA"
and only the 7 & 8 collections
Closes#713
* feat: add conditions, args to purge_ttl script
attempt to try and provide a way to allow the purge_ttl script to
complete.
* Adds arguments (ENV VARS):
--instance_id (INSTANCE_ID) Spanner instance id
--database_id (DATABASE_ID) Spanner database id
--sync_database_url (SYNC_DATABASE_URL) Spanner DSN
`spanner://instance/database`
--collection_ids (COLLECTION_IDS)
JSON formatted list of collections to limit deletions
e.g. `--collection_ids=123` limits to just collection 123
`--collection_ids=[123,456]` limits to both 123 & 456
default is all collections
Issue #631
Co-authored-by: Philip Jenvey <pjenvey@underboss.org>
* Update vendor libs to use protobuf 2_11_0
* removed unused vendored subcomponents (bigtable, pubsub)
* updated README.md to include better instructions on how to do this
again in the future
* Temporarily disable cargo audit due to sudden spike in package
maintenance triggering deployment warnings
* updated generate.sh script to only generated needed components, and
remind about mod.rs
Closes#537
- always prefix primary key names with their table name (id -> bso_id,
collection_id). a best practice in spanner, as when tables
interleave, their shared id names must always match
- bso -> bsos (every other table name's plural)
- give batches an expiry index
Closes#313
