Commit Graph

5 Commits

Author SHA1 Message Date
jrconlin
08a646a36e
feat: add --user_percent option
The `--user_percent` option will divvy up the users into blocks and
move the specified block. It takes an option formatted as
"block#:percentage". Block numbers are 1 based. For example,
--user_percent=2:33 will divide the total distinct users into
non-overlapping blocks of approximately 33%, and then move the second
block (e.g. the 33-65th users in the list). Extra users that may not be
evenly divided into percentage blocks will be appended to the last
block. (e.g. for `--user_percent=3:33`, users 66-99 would be copied
over, a total of 34 users)

Issue #407
2020-04-02 09:38:54 -07:00
jrconlin
0a9cf9c650
f pjenvey fix 2020-03-18 15:02:33 -07:00
jrconlin
be3b18f879
f add rust_migration WIP
* make user sorting optional
* formatting tweaks to dump_mysql.py and sync.avsc
2020-03-17 16:53:07 -07:00
jrconlin
a65123bcf2
feat: Add --abort and --user_range flags
* --abort stops copying BSO records after N instances.
* --user_range limits copy to offset:limit users.
* sorts users by fxa_uid
2020-03-16 13:32:38 -07:00
JR Conlin
ecfca9fdf5
feat: more user_migration stuff (#450)
* feat: more user_migration stuff

* create script to move users by node directly
* moved old scripts to `old` directory (for historic reasons, as well as
possible future use)
* cleaned up README
* try to solve the `parent row` error
an intermittent error may be responsible from one of two things:
1) a transaction failure resulted in a premature add of the unique key
to the UC filter.
2) an internal spanner update error resulting from trying to write the
bso before the user_collection row was written.
* Added "fix_collections.sql" script to update collections table to add
well known collections for future rectification.
* returned collection name lookup
* add "--user" arg to set bso and user id
* add `--dryrun` mode
2020-03-02 20:26:07 -08:00