36 Commits

Author SHA1 Message Date
Taddes
adbb0a65d9
feat: tokenserver script postgres support (#1857)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
feat: tokenserver script postgres support
2025-10-23 18:01:07 -04:00
dependabot[bot]
207a010ed0
chore(deps): bump authlib in /tools/tokenserver/loadtests
Bumps [authlib](https://github.com/authlib/authlib) from 1.6.4 to 1.6.5.
- [Release notes](https://github.com/authlib/authlib/releases)
- [Changelog](https://github.com/authlib/authlib/blob/main/docs/changelog.rst)
- [Commits](https://github.com/authlib/authlib/compare/v1.6.4...v1.6.5)

---
updated-dependencies:
- dependency-name: authlib
  dependency-version: 1.6.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-10 20:50:19 +00:00
dependabot[bot]
f60189d9a0
chore(deps): bump authlib in /tools/tokenserver/loadtests (#1816)
Bumps [authlib](https://github.com/authlib/authlib) from 1.6.1 to 1.6.4.
- [Release notes](https://github.com/authlib/authlib/releases)
- [Changelog](https://github.com/authlib/authlib/blob/main/docs/changelog.rst)
- [Commits](https://github.com/authlib/authlib/compare/v1.6.1...v1.6.4)

---
updated-dependencies:
- dependency-name: authlib
  dependency-version: 1.6.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-22 11:54:09 -04:00
Taddes
8995db6268
feat: ruff for python lint and format (#1742)
Some checks are pending
Glean probe-scraper / glean-probe-scraper (push) Waiting to run
feat: ruff for python lint and format
2025-08-12 20:26:55 -04:00
Taddes
8055e7429d
test: resolve test deprecations (#1732)
test: resolve test deprecations
2025-08-12 17:47:55 -04:00
Taddes
77254b4a6e
refactor: python imports (#1730)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
refactor: unified python imports for consistency in local, production, and test environments
2025-08-04 15:37:46 -04:00
Taddes
f8715d4e91
feat: use poetry for dependency management (#1706)
Some checks are pending
Glean probe-scraper / glean-probe-scraper (push) Waiting to run
feat: use poetry for dependency management
2025-07-28 21:48:46 -04:00
Nick Shirley
aeedcf1e19
chore: upload test artifacts to gcs
* chore: upload test results to gcs

* chore: use specific ete variables for gcp upload

* chore: override google_project_number to blank

* chore: use actual env var

* style: fixes flake8 E302 error

* docs: add documentation for new circleci vars

Closes SYNC-4584
2025-03-25 16:33:49 -06:00
Nick Shirley
15840c5ecf
chore: migrate tokenserver tests to pytest with junit output
* add pytest and script to make for running tests with pytest

* add conftest for pytest path handling

* move store test results to capture integration results

* remove unneeded run_tests and fix warnings about assert_()

Closes SYNC-4612
2025-03-17 22:36:42 -06:00
Philip Jenvey
1b13123a2b
feat: Revert "feat: Revert "fix: revert the python3.10 match statement (for now) (#1592)""
This reverts commit f3bdda91660a6777b715b59253234c4d7ba4a520.

Issue SYNC-4363
2024-10-21 16:54:42 -07:00
Philip Jenvey
3675c9387b
fix: correctly read the SYNC_STATSD_HOST/PORT settings (#1601)
Closes SYNC-4417
2024-09-23 17:31:51 -07:00
Philip Jenvey
f3bdda9166
feat: Revert "fix: revert the python3.10 match statement (for now) (#1592)"
This reverts commit dc0d571c055741297a77dd47c70b7ef55b552530.
(minus the added RUSTSEC ignore)

utilize the match statement now that we're back on debian
bookworm/python 3.10

Issue SYNC-4259
2024-09-19 11:27:02 -07:00
Philip Jenvey
b52e44ab52
feat: pickup the syncserver metrics settings (#1598)
and remove syncstorage dbg! statements

Issue SYNC-4259
2024-09-17 13:29:07 -07:00
Philip Jenvey
dc0d571c05
fix: revert the python3.10 match statement (for now) (#1592)
so the script can operate under bullseye/python3.9

and ignore RUSTSEC-2024-0365

Issue SYNC-4259
2024-08-29 17:55:47 -07:00
Philip Jenvey
6c9b771bc5
chore: bump crytography/pyramid to quiet a number of security alerts (#1574)
Closes SYNC-4301
2024-06-14 17:05:43 -07:00
JR Conlin
dbbdd1dfc3
feat: Remove support for BrowserID (#1531)
* feat: Remove support for BrowserID
* mark test only functions as such. I think we can drop MapAndThenTrait for more recent implementations of rust.

Closes: SYNC-3684
2024-06-14 12:51:02 -07:00
JR Conlin
6537783a9c
feat: Add metrics, gcp logging to tokenserver scripts (#1555)
* feat: Add metrics, gcp logging to tokenserver scripts

This adds GCP logging JSON output formatting for logs as well as DataDog
style metric reporting.

New Environs:
* `METRIC_API_KEY` - Optional Datadog API key
* `METRIC_APP_KEY` - Optional Datadog APP key
* `METRIC_HOST` - Collector Host address
* `METRIC_PORT` - Collector Host port

This also adds the `--human` arg if you want to use the older, human
readable logs instead.


---------

Co-authored-by: Philip Jenvey <pjenvey@underboss.org>
2024-05-22 18:19:50 -07:00
JR Conlin
581c250739
feat: Ignore non-spanner nodes for scripts (#1557)
Closes SYNC-4264
2024-05-08 14:54:19 -07:00
Philip Jenvey
4a145dd18b
feat: optionally force the spanner node via get_best_node (#1553)
Issue SYNC-4181
2024-05-02 09:09:53 -07:00
JR Conlin
5dc53f2282
Feat/range2 (#1550)
* bug: validate val names

* not sure why we look for two args when the secret should always be last
* fix names of the range vals to match opts
* normalize `range` to `uid_range` everywhere to prevent possible
problems.
2024-04-30 14:33:08 -07:00
JR Conlin
cc16082241
feat: Allow uid range for purge function (SYNC-4246) (#1547)
* feat: Allow uid range for purge function

In an attempt to parallelize this script after a very long delay,
specify a range so that multiple scripts can try to process
different ranges of the database.

Closes #1548
2024-04-30 08:53:27 -07:00
Philip Jenvey
f68fb607fe
fix: take keys_changed_at into account w/ migrated records' special case (#1545)
and make the comparison more clear/explicit as the goal is to check if the
`fxa_kid`s match

Issue SYNC-4225
2024-04-25 14:05:14 -07:00
Philip Jenvey
13e53eba13
feat: special case purging of users previously migrated to Spanner (#1543)
* feat: special case purging of users previously migrated to Spanner

don't issue deletes to their current data on syncstorage if their old
"migration records" point to it

and update per RUSTSEC-2024-0336

Closes SYNC-4225
2024-04-24 10:46:47 -07:00
JR Conlin
b777fa0d96
bug: Add try/except handler for force (#1535)
* bug: Add try/except handler for force

This will wrap the force `delete_service_data` with a try/except handler
so as not to prevent the tokenserver record from being removed if the
delete_service_data fails.

* Try/except will capture any errors from trying to call the delete on a
given node, this could fail if the auth or node aren't correct.
* Override will override the attempted node. This will allow us to force
the delete message, even if the original data was copied over and the
node was incorrect.
2024-04-16 10:07:01 -07:00
JR Conlin
06ecb78e24
bug: Copy modified purge script from old tokenserver repo (#1512)
* bug: Copy modified purge script from old tokenserver  repo
2024-02-27 12:46:47 -08:00
Philip Jenvey
31192d52c9
fix: connect to the db once instead of every loop iteration
SYNC-3638
2023-03-06 16:43:34 -08:00
Philip Jenvey
7e298c2dd0
fix: handle nullable (None) keys_changed_at values (#1464)
a regression from the Python 3 conversion (max's changes)

SVCSE-3633
2023-03-03 14:11:35 -08:00
Philip Jenvey
d2f6cf65ff
chore: add missing util.py to docker for process_account_events.py (#1455) (#1457)
and add PyBrowserID for it and tokenlib for purge_old_records.py

SVCSE-474
Issue #1086
2023-02-06 13:17:12 -08:00
Ethan Donowitz
ebf425fe26
bug: fix JWT library for Tokenserver load tests (#1373)
Closes #1372
2022-08-09 15:30:18 -04:00
Ethan Donowitz
8c59bb4f80
bug: rework Tokenserver load tests for local OAuth verification (#1357) 2022-07-25 12:24:27 -04:00
Ethan Donowitz
b6d87b7214
test: Add BrowserId support to Tokenserver load tests (#1219)
Closes #1213
2022-04-14 12:58:33 -04:00
Ethan Donowitz
38d6a27b02
feat: add BrowserID support for Tokenserver (#1216)
Closes #1215
2022-03-31 15:13:04 -04:00
Ethan Donowitz
ae6597022c
refactor: remove static service IDs (#1199)
Note: this commit contains a breaking change. Previously, there was a migration that included two services with static IDs by default, but this commit adds a new migration that removes those two services and adjusts the code to account for the change.

Closes #1144
2022-01-07 10:41:28 -05:00
Ethan Donowitz
46d4a9ea43
test: add Tokenserver load tests (#1184)
Closes #1107
2021-12-17 18:14:30 -05:00
Ethan Donowitz
0c05e999d1
fix: resolve intermittent Tokenserver test failure (#1171)
Closes #1170
2021-11-22 17:19:47 -05:00
Ethan Donowitz
0ac30958de
feat: add Tokenserver admin scripts (#1168)
Closes #1086
2021-11-18 09:37:02 -05:00