85 Commits

Author SHA1 Message Date
Barry Chen
66d9d11e27
feat: log when Sentry is not configured (#2073)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncserver-postgres (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs-spanner-python-utils (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncserver-postgres-python-utils (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncserver-mysql (push) Has been cancelled
MySQL Build and Test / build-and-test-mysql (push) Has been cancelled
Postgres Build and Test / build-and-test-postgres (push) Has been cancelled
Publish Sync docs to pages / build-mdbook (push) Has been cancelled
Publish Sync docs to pages / build-openapi (push) Has been cancelled
Spanner Build, Test, and Push / build-and-test-spanner (push) Has been cancelled
MySQL Build and Test / build-mysql-image (push) Has been cancelled
MySQL Build and Test / mysql-e2e-tests (push) Has been cancelled
Postgres Build and Test / build-postgres-image (push) Has been cancelled
Postgres Build and Test / postgres-e2e-tests (push) Has been cancelled
Publish Sync docs to pages / combine-and-prepare (push) Has been cancelled
Publish Sync docs to pages / deploy (push) Has been cancelled
Spanner Build, Test, and Push / build-spanner-image (push) Has been cancelled
Spanner Build, Test, and Push / spanner-e2e-tests (push) Has been cancelled
* feat: log when Sentry is not configured

* Update syncserver-common/src/middleware/sentry.rs

Co-authored-by: Philip Jenvey <pjenvey@underboss.org>

* warn and optionally print backtrace

* Use error!

* panic!

* Cleanup and comment

* Cargo.lock

---------

Co-authored-by: Philip Jenvey <pjenvey@underboss.org>
2026-02-25 12:46:58 -08:00
Barry Chen
b54129a334
chore: cargo upgrade deps (#2061)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncserver-postgres (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs-spanner-python-utils (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncserver-postgres-python-utils (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncserver-mysql (push) Has been cancelled
MySQL Build and Test / build-and-test-mysql (push) Has been cancelled
MySQL Build and Test / build-mysql-image (push) Has been cancelled
MySQL Build and Test / mysql-e2e-tests (push) Has been cancelled
Postgres Build and Test / build-and-test-postgres (push) Has been cancelled
Postgres Build and Test / build-postgres-image (push) Has been cancelled
Postgres Build and Test / postgres-e2e-tests (push) Has been cancelled
Publish Sync docs to pages / build-mdbook (push) Has been cancelled
Publish Sync docs to pages / build-openapi (push) Has been cancelled
Publish Sync docs to pages / combine-and-prepare (push) Has been cancelled
Publish Sync docs to pages / deploy (push) Has been cancelled
Spanner Build, Test, and Push / build-and-test-spanner (push) Has been cancelled
Spanner Build, Test, and Push / build-spanner-image (push) Has been cancelled
Spanner Build, Test, and Push / spanner-e2e-tests (push) Has been cancelled
2026-02-17 19:03:57 -08:00
Barry Chen
0ddc3b0af1
chore: upgrade to Rust 2024 edition (#2048) 2026-02-11 11:00:29 -08:00
Taddes
62bd7d2c6a
docs: open api docs utoipa (#2023)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs-postgres (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs-spanner-python-utils (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs-postgres-python-utils (push) Has been cancelled
Build, Tag and Push Container Images to GAR Repository / build-and-push-syncstorage-rs-mysql (push) Has been cancelled
MySQL Build and Test / build-and-test-mysql (push) Has been cancelled
Postgres Build and Test / build-and-test-postgres (push) Has been cancelled
Publish Sync docs to pages / build-mdbook (push) Has been cancelled
Publish Sync docs to pages / build-openapi (push) Has been cancelled
Spanner Build, Test, and Push / build-and-test-spanner (push) Has been cancelled
MySQL Build and Test / build-mysql-image (push) Has been cancelled
MySQL Build and Test / mysql-e2e-tests (push) Has been cancelled
MySQL Build and Test / deploy-mysql-dockerhub (push) Has been cancelled
Postgres Build and Test / build-postgres-image (push) Has been cancelled
Postgres Build and Test / postgres-e2e-tests (push) Has been cancelled
Postgres Build and Test / deploy-postgres-dockerhub (push) Has been cancelled
Publish Sync docs to pages / combine-and-prepare (push) Has been cancelled
Publish Sync docs to pages / deploy (push) Has been cancelled
Spanner Build, Test, and Push / build-spanner-image (push) Has been cancelled
Spanner Build, Test, and Push / spanner-e2e-tests (push) Has been cancelled
Spanner Build, Test, and Push / deploy-spanner-dockerhub (push) Has been cancelled
docs: open api docs utoipa
2026-02-06 17:05:36 -05:00
Philip Jenvey
e8a392b0a3
fix: move version.json to the root (#2041)
as the build-and-push action automatically creates one for us here,
otherwise we lack a populated syncserver/version.json during this
action

Closes STOR-474
2026-02-04 11:07:04 -08:00
Philip Jenvey
ab1887d283
feat: kill long removed browserid references
Closes STOR-458
2026-01-15 13:33:45 -08:00
Taddes
d05cf81858
fix: point to tokenserver api (#1970)
fix: point to tokenserver api
2025-12-31 17:24:35 -05:00
Philip Jenvey
afaafc5704
fix: initializing the app once suffices (#1937)
otherwise the postgres Pool init deadlocks on pending DDL statements
from the migration

Closes STOR-426
2025-12-16 10:42:38 -08:00
Philip Jenvey
1edfd870b3
feat: add the remaining postgres get methods (#1912)
* feat: add the remaining postgres get methods

- add a FromSql<Timestamp, _> for SyncTimestamp
- utilize Selectable for GetBso
- simplify bsos_query to rely on type inference of the result

Closes STOR-337
2025-12-08 11:53:41 -08:00
Barry Chen
ec42126299
Merge branch 'master' into refactor/u64-DEFAULT_MAX_QUOTA_LIMIT-STOR-294 2025-12-04 14:45:51 -06:00
Fritz Elfert
7ba0b4338f
feat: support systemd-journal logging (#1858)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
2025-12-02 14:01:09 -08:00
Barry Chen
f8d63999f4
refactor: use u64 for DEFAULT_MAX_QUOTA_LIMIT 2025-12-02 13:31:56 -06:00
Barry Chen
9e0d369851
refactor(metrics): make hostname in metrics optional (#1880)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
The hostname in the metrics could significantly increase the
cardinality.  This patch make the inclusion of the hostname optional,
with a default of false.

Closes STOR-393
2025-11-10 10:42:12 -08:00
Philip Jenvey
bcc5990eea
feat: add syncstorage-postgres
Closes STOR-330
2025-10-17 16:13:05 -07:00
Philip Jenvey
21bcc42feb
refactor: String -> &str
Issue STOR-330
2025-10-17 14:03:19 -07:00
Taddes
8e3f367095
refactor: rmv last insert id, simplify queries, add check (#1841)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
refactor: rmv last insert id, simplify queries, add check
2025-10-17 09:39:58 -04:00
Barry Chen
3404150792
chore(syncserver): break up extractors.rs 2025-10-06 21:41:24 -04:00
Barry Chen
730d6b8173
refactor(db): return only a SyncTimestamp from post_bsos db fns
Co-authored-by: Philip Jenvey <pjenvey@underboss.org>
2025-09-29 14:36:09 -05:00
Philip Jenvey
192d64c7c3
feat: switch syncstorage to diesel-async
Closes STOR-364
2025-09-23 18:32:51 -07:00
Philip Jenvey
0400c2dd15
feat: adapt the test suite to the generic dyn DbPool (#1808)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
and moved the couple generic Db methods into the trait itself

Closes STOR-360
2025-09-19 13:20:49 -07:00
Barry Chen
204c13e75d
chore: use environment value from syncserver settings for Sentry
Co-authored-by: Philip Jenvey <pjenvey@underboss.org>
2025-09-18 08:14:38 -05:00
Philip Jenvey
f2698a4251
feat: switch tokenserver to diesel-async (#1790)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
Closes STOR-326
2025-09-08 13:08:29 -07:00
Philip Jenvey
b89b06d24f
feat: kill Db's impl of Clone (#1789)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
by utilizing async closures' improved borrowing semantics

Closes STOR-350
2025-09-04 14:58:11 -07:00
Philip Jenvey
38cb38fbd9
refactor: switch syncstorage Db methods to &mut self
Closes STOR-327
2025-09-02 13:50:20 -07:00
Philip Jenvey
614e3902d0
refactor: switch tokenserver Db methods to &mut self
Closes STOR-332
2025-09-02 13:50:19 -07:00
Taddes
f7197fefb8
chore: update syncstorage rust (#1749)
chore: update syncstorage rust
2025-08-13 15:14:34 -04:00
Philip Jenvey
2fb6b84ad4
fix: make pyo3 usage optional per the feature flag (#1731)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
and cleanup feature flag usage (only specifying defaults in syncserver's
Cargo.toml)

Closes STOR-295
2025-07-29 17:08:29 -07:00
Philip Jenvey
dd546f028e
chore: strip actix-web features (#1713)
Some checks are pending
Glean probe-scraper / glean-probe-scraper (push) Waiting to run
while providing an optional actix-compress feature

also disables actix-web's cookies and unicode features

Closes STOR-256
2025-07-02 14:04:43 -07:00
Taddes
0e3f7d162c
refactor: quiet dbg calls (#1701)
refactor: quiet dbg calls
2025-06-04 17:25:16 -04:00
Taddes
0768d4975d
chore: remove extraneous dependencies (#1700)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
chore: remove extraneous dependencies
2025-05-22 19:06:58 -04:00
Philip Jenvey
0148e04dd2
chore: bump to latest rust
for cargo nextest/backtrace
2025-05-07 16:49:09 -07:00
Philip Jenvey
624eced1e9
feat: emit oauth verification timeouts as metrics (not sentry) (#1694)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
and fix the sentry middleware to use ReportableError::tags

Closes STOR-201
2025-05-02 13:51:52 -07:00
Philip Jenvey
0ca435fb1a
refactor: simplify metric_label to return a &str
Issue STOR-64
2025-04-23 16:01:07 -07:00
Taddes
7c9bc0089d
docs: document pruning scripts (#1645)
* docs for tokenserver process_account_events

* add docs for spanner purge ttl

* add kubernetes workload references and status

* cargo update, update validator to 0.19, audit ignore due to diesel limitation
2025-01-07 13:23:09 -08:00
Taddes
c01021b87d
Feat: add task to release unused db conns (#1640)
Feat: add task to release unused db conns
2024-12-05 17:10:14 -05:00
Taddes
7f7cfb7897
move context of glean metrics into statsd get_collections metric (#1638)
Co-authored-by: JR Conlin <jconlin+git@mozilla.com>
2024-12-04 14:24:02 -08:00
Taddes
9e9869ee06
feat: glean metrics logic (#1626)
feat: glean metrics logic
2024-11-26 17:04:38 -06:00
Philip Jenvey
9b033edcb0
fix: don't add extra prefixes to middleware emitted metrics (#1630)
Closes SYNC-4478
2024-11-13 12:04:18 -08:00
Taddes
98ccc95482
feat: add metric values to get_collections (#1616)
feat: add metric values to get_collections
2024-11-06 10:33:42 -05:00
Taddes
cc6dd13749
feat: add hashed_device_id to HawkIdentifier (#1615)
feat: add hashed_device_id to HawkIdentifier and UserIdentifier
2024-10-25 10:24:57 -04:00
Taddes
715cf950ba
feat: add hashed_fxa_uid to HawkPayload (#1613)
add hashed_fxa_uid to HawkPayload, HawkIdentifier, UserIdentifier
2024-10-21 19:41:43 -04:00
Taddes
7f2ef062fc
feat: user agent parsing (#1607)
user agent parsing module for syncserver
2024-10-21 17:09:42 -04:00
Philip Jenvey
c535e5ae52
feat: wire MysqlError's ReportableError impl into TokenserverError (#1611)
* feat: wire MysqlError's ReportableError impl into TokenserverError

and quiet Pool errors

Closes SYNC-4424

* kluge: allow(clippy::result_large_err) for TokenserverError

(for now) we can get rid of this in a later refactoring
2024-10-17 14:23:45 -07:00
Philip Jenvey
1f0e28d7af
feat: track the pool's queued vs actually active tasks (#1605)
and consider idle threads as a "max" value as the underlying tokio threadpool
will exit threads idle for too long

Closes SYNC-4424
2024-09-30 16:13:51 -07:00
Philip Jenvey
5d9d203c62
refactor: move sentry middlware and Taggable to syncserver-common
Issue SYNC-4407
2024-09-18 13:26:45 -07:00
Philip Jenvey
2b8b1f5dde
fix: downcast to tokenserver's actual error type (#1596)
which its handlers (except hearbeat) return vs ApiError

otherwise we fail to take ReportableError::is_sentry_event into
account and flood sentry with all errors

also:
- revert:
"feat: debug "Invalid OAuth token" (verifier returns None) error cases (#1595)"

This reverts commit 1443b31e5af1f10f8a52bf1bb91dc817ce0b75f2.

- update quinn-proto per RUSTSEC-2024-0373

Issue SYNC-4262
2024-09-12 11:42:13 -07:00
Philip Jenvey
1edce04154
fix: don't hide TokenserverPool initialization errors on startup (#1584)
Closes SYNC-4346
2024-07-11 16:25:50 -07:00
JR Conlin
1713962c6a
Chore/update 2406 (#1576)
chore: Updates for Jun-2024

* f normalize diesel to 1.4

We're currenlty locked to diesel 1.4 due to some significant changes
in the `diesel_logger` crate and how it expects `Connections` to be
defined.
2024-06-25 13:29:42 -07:00
JR Conlin
dbbdd1dfc3
feat: Remove support for BrowserID (#1531)
* feat: Remove support for BrowserID
* mark test only functions as such. I think we can drop MapAndThenTrait for more recent implementations of rust.

Closes: SYNC-3684
2024-06-14 12:51:02 -07:00
JR Conlin
2584b977b8
feat: Add timeouts for tokenserver database calls. (#1561)
Closes: SYNC-4270
2024-05-31 13:53:51 -07:00