73 Commits

Author SHA1 Message Date
Barry Chen
30478687d7
chore(ci): configure CircleCI for postgres builds
This commit enables Postgres builds and tests in CircleCI.  Only
Tokenserver supports Postgres at the moment, and the Python based
integration tests have an implicit dependency on MySQL, so the CI jobs only
use Postgres where possible.

The jobs have been split up and named more explicitly.  The hope is to
simply delete the mysql jobs in the future.
2025-10-27 13:23:48 -05:00
Barry Chen
31e5e2ba2f
chore(CI): fetch MySQL public key from MIT 2025-10-23 12:49:07 -05:00
Taddes
19b6176d5b
chore: update sync python version (#1774)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
chore: update sync python version
2025-08-28 20:27:58 -04:00
Taddes
f7197fefb8
chore: update syncstorage rust (#1749)
chore: update syncstorage rust
2025-08-13 15:14:34 -04:00
Taddes
f8715d4e91
feat: use poetry for dependency management (#1706)
Some checks are pending
Glean probe-scraper / glean-probe-scraper (push) Waiting to run
feat: use poetry for dependency management
2025-07-28 21:48:46 -04:00
Taddes
31c3b866a4
refactor: remove purge_ttl.rs (#1702)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
refactor: remove purge_ttl.rs
2025-06-04 22:52:10 -04:00
Philip Jenvey
0148e04dd2
chore: bump to latest rust
for cargo nextest/backtrace
2025-05-07 16:49:09 -07:00
Philip Jenvey
e93bb8821c
chore: update python cryptography (#1690)
Some checks failed
Glean probe-scraper / glean-probe-scraper (push) Has been cancelled
2025-04-18 13:33:10 -07:00
Philip Jenvey
e0093a88bf
chore: switch back to libmariadb-dev (#1665)
as it builds on more architectures (such as arm64) and we no longer require its
TLS support

derived from ghagl's and tommie's PRs (thanks!)
https://github.com/mozilla-services/syncstorage-rs/pull/1635
https://github.com/mozilla-services/syncstorage-rs/pull/1644

Closes SYNC-4690
2025-03-21 15:42:17 -07:00
Nick Shirley
8c56cae890
chore: migrate unit tests to nextest and llvm-cov
* add nextest and llvm-cov for running unit tests. 
* test results and coverage output to store_test_results and store_artifacts. 
* upgrade build image and Docker Rust version 1.78.0 → 1.81.0

Closes SYNC-4611
2025-03-10 10:34:54 -06:00
Philip Jenvey
e170518c0f
feat: revert "feat: Revert "chore: revert back to bullseye (for now) (#1589)""
This reverts commit bbdfb1933dc557ae23fabcb87eb5a22e4478a069.

Issue SYNC-4363
2024-10-21 16:54:19 -07:00
Philip Jenvey
bbdfb1933d
feat: Revert "chore: revert back to bullseye (for now) (#1589)"
This reverts commit 4a503f8c36fe070e11df43a8ce0b3c71358e983c.

debian bookworm doesn't appear to be any part of the issues on
production caused by SYNC-4262

Issue SYNC-4363
2024-09-19 11:27:02 -07:00
Philip Jenvey
4a503f8c36
chore: revert back to bullseye (for now) (#1589)
this also downgrades Oracle's libmysqlclient-dev from 8.0.37-1debian11 to
8.0.39-1debian12

this is to test if the debian version and or the libmysqlclient-dev version are
the cause of both db pool initialization failures as well as other seemingly
mysterious likely network related issues seen in 0.17.x

Issue SYNC-4363
2024-08-07 15:29:26 -07:00
JR Conlin
0f86587edd
bug: Revert the venv configuration for python (#1571)
use `--system-break-packages` flag to force install to system.

Closes SYNC-4298
2024-06-14 16:25:31 -07:00
JR Conlin
8f9e1c27cf
chore: Update to debian bookworm / Python 3.12 (#1567)
* chore: Update to debian bookworm / Python 3.12
2024-06-10 07:22:57 -07:00
Tarik Eshaq
1b11684648
feat: Puts pyo3 behind feature flag and derives tokens directly in Rust (#1513)
* Removes pyo3 and derives tokens directly in Rust

* Adds tests for JWT verifying

* Adds tests for token generation

* Adds metrics for oauth verify error cases

* Updates jsonwebtoken to not include default features (including pem loading)

* Adds context and logs errors during oauth verify

* Uses ring for cryptographic rng

* Adds back python impl under feature flag

* Uses one cached http client for reqwest
2024-02-12 11:14:15 -05:00
Tarik Eshaq
f87293cc38
Upgrades rustc in CI and Docker to 1.75 2024-02-08 14:08:45 -05:00
JR Conlin
cc8d5847dc
Chore/update 2312 (#1510)
* chore: Update for 12-2023

Closes: #1503, #1502

switch to fetching mysql_pubkey.asc

Closes #1509 SYNC-4055
2024-01-12 15:44:44 -08:00
JR Conlin
1b9ebbf40f
Chore/contributor rollup (#1479)
* These needed to be rolled in due to various dependency changes.

 * Fix the URL people should set to use their Custom Server (https://github.com/mozilla-services/syncstorage-rs/pull/1453) -
[Ekleog](https://github.com/Ekleog)
 * Make docker files more podman friendly (https://github.com/mozilla-services/syncstorage-rs/pull/1431) -
[mb](https://github.com/mb)

* This PR also updates a number of dependencies
2023-10-18 14:24:41 -07:00
Philip Jenvey
95bad0ae70
merge master 2023-09-25 12:23:50 -07:00
Philip Jenvey
0ccaa4ed02
chore: bump the rust version and some crates
per the latest cargo audit requiring rust >= 1.65

bump openssl per multiple RUSTSECs and ignore the h2 and webpki audits
2023-09-21 15:56:29 -07:00
Philip Jenvey
80d21e7143
merge master 2023-03-07 12:02:16 -08:00
Philip Jenvey
6ee39da4a0
chore: add another missing file to docker for process_account_events.py (#1463)
(just add the whole dir this time)

SVCSE-474
Issue #1086
2023-02-24 09:16:44 -08:00
Philip Jenvey
d2f6cf65ff
chore: add missing util.py to docker for process_account_events.py (#1455) (#1457)
and add PyBrowserID for it and tokenlib for purge_old_records.py

SVCSE-474
Issue #1086
2023-02-06 13:17:12 -08:00
Philip Jenvey
489ee051a5
chore: add missing util.py to docker for process_account_events.py (#1455)
and add PyBrowserID for it and tokenlib for purge_old_records.py

SVCSE-474
Issue #1086
2023-02-01 10:15:48 -08:00
Ethan Donowitz
b5b7e57f93
refactor: add database crates (#1407)
Closes #1277
2023-01-10 16:06:03 -05:00
Ethan Donowitz
d11787965c
chore: updates for Rust 1.66 (#1451) 2023-01-09 15:06:52 -05:00
Ethan Donowitz
b9c1f7f67b
chore: remove spanner_config.ini (#1446) 2022-12-14 14:21:21 -05:00
Ethan Donowitz
b95e549acb
chore: upgrade to Rust 1.65 (#1441) 2022-11-18 17:25:39 -05:00
Philip Jenvey
23a3c4ba3a
merge master 2022-11-01 12:55:04 -07:00
Philip Jenvey
b4fe184f51
chore: switch from mariadb libmysqlclient to mysql's (#1435)
as it defaults to TLS connections. also diesel's ssl_mode relies on the mysql
specific MYSQL_OPT_SSL_MODE option

Closes #1434
2022-11-01 11:57:09 -07:00
Ethan Donowitz
0ae5fd2059
refactor: add settings crates (#1306)
This is a breaking change. This commit separates syncstorage and tokenserver settings into separate structs that are contained by a parent `Settings` struct. This means that any env vars that hold settings specific to syncstorage (e.g. `SYNC_DATABASE_URL`) have been renamed to `SYNC_SYNCSTORAGE__DATABASE_URL`. Any settings that were moved from the top-level `Settings` struct to the lower level, syncstorage-specific struct will now have a `SYNC_SYNCSTORAGE__` prefix instead of a `SYNC_` prefix.

Closes #1276
2022-10-19 17:15:56 -04:00
Ethan Donowitz
fca795e3c0
chore: update to Rust 1.64.0 (#1415) 2022-10-05 12:54:51 -04:00
Philip Jenvey
2821b80e1b
fix: write to the new version.json location (#1344)
Closes #1343
2022-06-24 15:24:57 -07:00
Ethan Donowitz
75e5f273ab
chore: add process_account_events.py to Docker image (#1325) 2022-06-14 09:54:23 -04:00
Ethan Donowitz
a52900f6a9
refactor: add common crates (#1281)
Closes #1275
2022-05-06 17:42:50 -04:00
Philip Jenvey
8ff7a40de7
chore: disable grpcio openssl (#1288)
(#1215 follow up) we shouldn't need it enabled since moving to reqwest+rustls

and upgrade the easier dependencies

Issue #1215
2022-04-27 11:09:15 -07:00
Ethan Donowitz
c4bca395f1
chore: update for Rust 1.60.0 (#1280) 2022-04-14 09:44:57 -04:00
Ethan Donowitz
38d6a27b02
feat: add BrowserID support for Tokenserver (#1216)
Closes #1215
2022-03-31 15:13:04 -04:00
Ethan Donowitz
0e9b0f6e6c
chore: update to Rust 1.59 (#1227) 2022-02-28 17:34:51 -05:00
Ethan Donowitz
a7c5f809d0
chore: update to Rust 1.58 and switch to GCP Rust crate (#1201) 2022-01-24 16:56:46 -05:00
Ethan Donowitz
aa18c1a01d
test: add Tokenserver integration tests to CI (#1180)
Closes #1174
2021-12-08 08:56:16 -05:00
Ethan Donowitz
04cf2344d5
chore: update 12/2/21 (#1181)
Updates for the new rust version
2021-12-03 15:17:10 -08:00
JR Conlin
9a82b3cd27
Chore/update 2021 07 (#1124)
chore: Dep update for Jul 2021
2021-08-02 10:42:24 -07:00
Ethan Donowitz
3440177702
refactor: Use PyFxA to verify Tokenserver tokens (#1103)
Closes #1098, #1102
2021-06-25 11:34:56 -04:00
Ethan Donowitz
e6ec1acd87
feat: Integrate Spanner emulator with CI (#1079)
Closes #566
2021-05-19 18:20:25 -04:00
JR Conlin
f795eb0813
bug: Restore hawk error metrics (#1033)
* chore: clippy updates for rust 1.51

Closes: #1032

* bug: Restore hawk error metrics

(Includes clippy changes for rust 1.51)

Hawk errors should be returned as metrics. During the middleware purge, this broke. This PR includes metric reporting into the sentry handler.

Closes #812.
2021-04-05 14:45:39 -07:00
JR Conlin
56cadcb2cd
chore: fmt & clippy fixes for Rust 1.50 (#1004)
* chore: fmt & clippy fixes for Rust 1.50
* update rand-core
* update googleapis (updated grpcio)

Co-authored-by: Philip Jenvey <pjenvey@underboss.org>
2021-02-17 10:33:26 -08:00
JR Conlin
69cf5ec2de
Chore/update (#966)
* chore: update dependencies

* includes grpcio

* grpcio update

* f update libs

* f clippy/fmt

* f update rust to 1.49

* f update dependencies

* Holding back bb8 to 0.4.2 because of upgrade lifetime issues.

* f added "rt" feature for tests

* chore: Jan 2021.a update for RUSTSEC-2021-0003

* includes protobuf update because `cargo update`
2021-01-12 15:38:39 -05:00
Donovan Preston
d6e6c2c0a1
Py3 integration tests (#917)
* feat: Port integration tests to py3, and run them in the same docker container as the rust server.

Fix #911
2020-12-01 21:20:59 -05:00