279 Commits

Author SHA1 Message Date
Vault Automation
92ddb4684c
Vault-31540 : Parallelization support for Sync and Unsync flows within Secret Sync (#10473) (#10749)
* Fix typos

* Use a goroutine around syncSecret

* Lock around map writes and memDB operations

* Add TODO comments

* Add unsync TODO

* adding unsync changes

* initial commit

* moving nil checks in memdb calls

* fixed tests; adjusted mutex locks while setting secret stores

* adding changelog

* addressing review comments: mutex adjustments, nits

* adding mutex to memDBSetStoresForSecret

* fixing data race test failures

* addressing review comments: configurable workerpool limit, nits

* removing debug logs that got missed

* Update changelog/_10473.txt



* addressing review comments: using default when custom woker pool count read fails, nits

* fix: updating worker pool count to address Vercel API rate limits

* Vault 40557/parallelize secret sync test aws gcp (#10645)

* add integratio test case for parallelize secret sync test aws and gcp store types

* resolve PR comments

* resolve PR comments

* add doc comments on TestSecretsSyncBackend_Queue_SecretKey test function

---------



---------

Co-authored-by: Murali <137029787+murali-partha@users.noreply.github.com>
Co-authored-by: robmonte <17119716+robmonte@users.noreply.github.com>
Co-authored-by: Vivek Pandey <vivek.pandey@hashicorp.com>
Co-authored-by: Vivek Pandey <vivekpandey@Viveks-MacBook-Pro.local>
2025-11-14 13:39:01 -05:00
Vault Automation
0c6c13dd38
license: update headers to IBM Corp. (#10229) (#10233)
* license: update headers to IBM Corp.
* `make proto`
* update offset because source file changed

Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
2025-10-21 15:20:20 -06:00
Vault Automation
23fd7533aa
Add root rotation for snowflake database secrets keypair configurations (#9432) (#9851)
* Initial implementation

* Use rotation_statements, handle both password and private_key

* Remove debug prints

* Merge in main

* Remove duplicated error text

* Rename keypair root rotation function

* Use NewRotateRootCredentialsWALPasswordEntry

* Add changelog file

* Move back to original file for now, for review

* put generatePassword into function

* Fix names, call helper for generatePassword

* Generalize the rotation flow and keypair path

* Fix conditional check, remove new file

* Fix changelog

* Add test file

* Fix username check var name

* Fix name variable

* Return an error when both fields are set during rotation, and return an error if somehow walEntry is nil

* Fix test godoc

* Remove print

* change rotated key bits to 4096

Co-authored-by: Robert <17119716+robmonte@users.noreply.github.com>
2025-10-03 21:34:42 +00:00
Vault Automation
bc60502ec9
Add role rotation info to create/update observations (#9254) (#9368)
* Add role rotation info to create/update observations

* observatin enhancements

* observatin enhancements

* remove log

* duration strings instead of seconds

* the stringening

* more times

* credential type

* Add rotation schedule/period to root rotation

* more ttls

* updates

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
2025-09-16 15:36:02 +00:00
Vault Automation
eaf949cb1f
VAULT-37633: Database static role recover operations (#8922) (#8982)
* initial implementation

* fix

* tests

* changelog

* fix vet errors

* pr comments

Co-authored-by: miagilepner <mia.epner@hashicorp.com>
2025-08-29 09:48:18 -05:00
Vault Automation
04451634d9
VAULT-38193 Add database observations to Vault (#8727) (#8802)
* VAULT-38193 database observations (WIP)

* VAULT-38193 database observations

* nil check

* make it consistent

* Clean up

Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
2025-08-21 12:26:58 -04:00
kpcraig
1fafe2f4d5
Log DB Rotations (#31402) 2025-08-08 16:15:02 -04:00
Ellie
1e7f22aeec
Add DB type consts (#31295)
* add necessary consts

* add other db plugins

* correct ES

* Fix consts in test
2025-07-17 12:08:27 -05:00
Ellie
23e04c2409
Add MetricsReporter interface so that databaseBackend's can share their connection counts with CensusManager (#31269)
* add interface and impl

* add tests

* fix comments

* Update builtin/logical/database/backend.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-14 14:52:55 -05:00
Ellie
6360705f0a
Require rotation_schedule cron style strings to be defined in UTC (VAULT-35616) (#30606)
* remove local time logic, and force cron to be UTC

* add test comment

* update docs

* add changelog

* change mesasge

* add utc clarification to docs

* remove utc reference in root token docs

* remove doc from partial
2025-06-16 12:51:07 -05:00
helenfufu
146c032600
CE changes for plugin download (#30927)
* ce changes for https://github.com/hashicorp/vault-enterprise/pull/8193

* lower case enterprise only errors

---------

Co-authored-by: Ben Ash <bash@hashicorp.com>
2025-06-10 10:31:24 -04:00
Ellie
294c304947
db: consider possibility of NextVaultRotation being unset on queue population (VAULT-35639) (#30320)
* consider possibility of NextVaultRotation being nil on queue population

* move test

* add changelog

* fix reference to nil, and improve debug log

* use helper function to write static roles to storage

* add password check in test

* fix godoc

* fix changelog and add remediation debug line

* force ticker to run, and make sure credential doesnt rotate

* add another edge case

* fix godoc

* check ttl is less in test

* check error case and if resp is nil

* make check on ttl more robust
2025-04-28 16:11:54 -05:00
Robert
bf339bc50d
Add snowflake DB API warning (#30327)
* Add API warning based on DB type

* Add deprecation notice

* Add warning to the top of the docs pages

* Update capabilities table

* Filter SQLConnectionProducer fields from unrecognized parameters warning

* Add test case
2025-04-28 13:05:55 -05:00
vinay-gopalan
d16b0beee3
Forward Performance Standby requests when configuring root credentials for AWS, LDAP and DB engines (#30039) 2025-03-27 14:32:49 -07:00
vinay-gopalan
e8c07ec68e
Small fixes on UX of Automated Root Rotation parameters (#29685) 2025-02-25 09:14:38 -08:00
John-Michael Faircloth
e2f09cb2ab
database: fix reload to not fail early (#29519)
* database: fix reload to not fail early

* return logical.ErrorRresponse; add tests

* do not return noop warnings; add logs

* changelog

* use name for log; remove event doc
2025-02-20 14:53:58 +00:00
vinay-gopalan
6a9de17ac4
move logs into if block (#29634) 2025-02-13 22:56:22 +00:00
vinay-gopalan
9e38a88883
Add automated root rotation support to DB Secrets (#29557) 2025-02-11 12:09:26 -08:00
John-Michael Faircloth
8d0443fd48
db: honor static role TTL across restarts when skip import rotation i… (#29537)
* db: honor static role TTL across restarts when skip import rotation is enabled

* changelog
2025-02-10 15:28:19 -06:00
John-Michael Faircloth
28b2746545
db: return success response on static role create/update (#29407) 2025-01-24 11:02:38 -08:00
John-Michael Faircloth
c39aa51916
test: fix ce/ent diff (#29307) 2025-01-07 09:19:32 -08:00
John-Michael Faircloth
6110ee084f
db: allow updates to self_managed_password (#29283) 2025-01-06 12:05:41 -06:00
John-Michael Faircloth
9a830736c8
fix db test data race for queue tick interval (#29276) 2025-01-03 09:27:10 -06:00
John-Michael Faircloth
f5191bd06e
db: fix skip-import-rotation/rootless integration (#29202)
* db: fix skip-import-rotation/rootless integration

* prevent setting both password and self_managed_password

* move func call and add comment
2024-12-17 11:17:02 -06:00
Mike Palmiotto
bf1741e123
make fmt (#29196) 2024-12-16 13:07:28 -05:00
John-Michael Faircloth
d411a44c18
secrets/db: enable skip auto import rotation of static roles (#29093)
* secrets/db: enable skip auto import rotation of static roles

* fix panic due to empty role name causing role to not be stored

* fix role upgrade test

* Apply suggestions from code review

Co-authored-by: vinay-gopalan <86625824+vinay-gopalan@users.noreply.github.com>
Co-authored-by: kpcraig <3031348+kpcraig@users.noreply.github.com>

* use password in favor of self_managed_password

* add deprecated to self_managed_password field

* fix bug with allowing updates to password

---------

Co-authored-by: vinay-gopalan <86625824+vinay-gopalan@users.noreply.github.com>
Co-authored-by: kpcraig <3031348+kpcraig@users.noreply.github.com>
2024-12-12 01:39:09 +00:00
Scott Miller
86ba0dbdeb
Use go-secure-stdlib's RSA key generator backed by a DRBG (#29020)
* Use DRBG based RSA key generation everywhere

* switch to the conditional generator

* Use DRBG based RSA key generation everywhere

* switch to the conditional generator

* Add an ENV var to disable the DRBG in a pinch

* update go.mod

* Use DRBG based RSA key generation everywhere

* switch to the conditional generator

* Add an ENV var to disable the DRBG in a pinch

* Use DRBG based RSA key generation everywhere

* update go.mod

* fix import

* Remove rsa2 alias, remove test code

* move cryptoutil/rsa.go to sdk

* move imports too

* remove makefile change

* rsa2->rsa

* more rsa2->rsa, remove test code

* fix some overzelous search/replace

* Update to a real tag

* changelog

* copyright

* work around copyright check

* work around copyright check pt2

* bunch of dupe imports

* missing import

* wrong license

* fix go.mod conflict

* missed a spot

* dupe import
2024-12-05 15:39:16 -06:00
vinay-gopalan
93f5777f6f
Update DB Static role rotation logic to generate new password if retried password fails (#28989) 2024-12-03 11:29:13 -08:00
Luis (LT) Carbonell
b861d8b03f
Fix Issue with Lost Timezone in Metadata for Database Secret Engines (#28509)
* Set cron schedule location after pulling from storage

* Add changelog
2024-09-25 18:40:50 -04:00
vinay-gopalan
ec9b675f70
Add OSS stub functions for Self-Managed Static Roles (#28199) 2024-08-29 10:01:01 -07:00
davidadeleon
fe44e55943
VAULT-29784: Skip connection verification on DB config read (#28139)
* skip connection verification on config read

* ensure appropriate default on config update call that results in a creation

* changelog

* leave verify_connection in config read response

* update test to handle output of verify_connection parameter

* fix remaining tests
2024-08-21 16:43:37 -04:00
John-Michael Faircloth
1b1f22192a
postgres: sanitize private_key from READ config endpoint (#28070) 2024-08-13 13:29:57 -07:00
John-Michael Faircloth
3fcb1a67c5
database/postgres: add inline certificate authentication fields (#28024)
* add inline cert auth to postres db plugin

* handle both sslinline and new TLS plugin fields

* refactor PrepareTestContainerWithSSL

* add tests for postgres inline TLS fields

* changelog

* revert back to errwrap since the middleware sanitizing depends on it

* enable only setting sslrootcert
2024-08-09 14:20:19 -05:00
John-Michael Faircloth
899ebd4aff
db/postgres: add feature flag protected sslinline configuration (#27871)
* adds sslinline option to postgres conn string
* for database secrets type postgres, inspects the connection string for sslinline and generates a tlsconfig from the connection string.

* support fallback hosts

* remove broken multihost test

* bootstrap container with cert material

* overwrite pg config and set key file perms

* add feature flag check

* add tests

* add license and comments

* test all ssl modes

* add test cases for dsn (key/value) connection strings

* add fallback test cases

* fix error formatting

* add test for multi-host when using pgx native conn url parsing

---------

Co-authored-by: Branden Horiuchi <Branden.Horiuchi@blackline.com>
2024-08-01 11:43:54 -05:00
Violet Hynes
dbecbcec18
VAULT-27384 Fix faulty assignments and unchecked errors (#27810)
* VAULT-27384 Fix faulty assignments and unchecked errors

* Another missed error

* Small refactor
2024-07-22 16:53:02 -04:00
John-Michael Faircloth
d6a588b8d2
db: refactor postgres test helpers (#27811)
* db: refactor postgres test helpers

* fix references to refactored test helper

* fix references to refactored test helper

* fix failing test
2024-07-19 09:47:34 -05:00
Christopher Swenson
a65d9133a1
database: Avoid race condition in connection creation (#26147)
When creating database connections, there is a race
condition when multiple goroutines try to create the
connection at the same time. This happens, for
example, on leadership changes in a cluster.

Normally, the extra database connections are cleaned
up when this is detected. However, some database
implementations, notably Postgres, do not seem to
clean up in a timely manner, and can leak in these
scenarios.

To fix this, we create a global lock when creating
database connections to prevent multiple connections
from being created at the same time.

We also clean up the logic at the end so that
if (somehow) we ended up creating an additional
connection, we use the existing one rather than
the new one. This by itself would solve our
problem long-term, however, would still involve
many transient database connections being created
and immediately killed on leadership changes.

It's not ideal to have a single global lock for
database connection creation. Some potential
alternatives:

* a map of locks from the connection name to the lock.
  The biggest downside is the we probably will want to
  garbage collect this map so that we don't have an
  unbounded number of locks.
* a small pool of locks, where we hash the connection
  names to pick the lock. Using such a pool generally
  is a good way to introduce deadlock, but since we
  will only use it in a specific case, and the purpose
  is to improve performance for concurrent connection
  creation, this is probably acceptable.

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2024-03-26 16:58:07 +00:00
Josh Black
fa13dbd381
add gosimport to make fmt and run it (#25383)
* add gosimport to make fmt and run it

* move installation to tools.sh

* correct weird spacing issue

* Update Makefile

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* fix a weird issue

---------

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2024-02-13 14:07:02 -08:00
Christopher Swenson
55d2dfb3d0
database: Emit event notifications (#24718)
Including for failures to write credentials and failure to rotate.
2024-02-05 10:30:00 -08:00
Tom Proctor
78ef25e70c
HTTP API for pinning plugin versions (#25105) 2024-01-30 10:24:33 +00:00
Tom Proctor
af27ab3524
Add version pinning to plugin catalog (#24960)
Adds the ability to pin a version for a specific plugin type + name to enable an easier plugin upgrade UX. After pinning and reloading, that version should be the only version in use.

No HTTP API implementation yet for managing pins, so no user-facing effects yet.
2024-01-26 17:21:43 +00:00
Tom Proctor
6e537bb376
Support reloading database plugins across multiple mounts (#24512)
* Support reloading database plugins across multiple mounts
* Add clarifying comment to MountEntry.Path field
* Tests: Replace non-parallelisable t.Setenv with plugin env settings
2024-01-08 12:21:13 +00:00
Violet Hynes
75d0581464
VAULT-8790 Ensure time.NewTicker never gets called with a negative value (#24402)
* Ensure time.NewTicker never gets called with a negative value

* Remove naughty newline

* VAULT-8790 review feedback
2024-01-03 15:34:41 -05:00
Tom Proctor
dc5c3e8d97
New database plugin API to reload by plugin name (#24472) 2023-12-13 10:23:34 +00:00
Tom Proctor
a4180c193b
Refactor plugin catalog and plugin runtime catalog into their own package (#24403)
* Refactor plugin catalog into its own package
* Fix some unnecessarily slow tests due to accidentally running multiple plugin processes
* Clean up MakeTestPluginDir helper
* Move getBackendVersion tests to plugin catalog package
* Use corehelpers.MakeTestPlugin consistently
* Fix semgrep failure: check for nil value from logical.Storage
2023-12-07 12:36:17 +00:00
modrake
eca4b4d801
Relplat 897 copywrite fixes for mutliple licenses (#23722) 2023-10-20 08:40:43 -07:00
vinay-gopalan
8924f9592d
Remove SA Credentials from DB Connection Details on Read (#23256) 2023-09-22 10:49:46 -07:00
John-Michael Faircloth
9569b16114
secrets/db: add rotation error path test (#23182)
* secrets/db: add rotation error path test

We add a test to verify that failed rotations can successfully recover
and that they do not occur outside of a rotation window. Additionally,
we remove registering some external plugins in getCluster() that shaves
off about 5 minutes the database package tests.

* remove dead code and add test comment

* revert to original container helper after refactor
2023-09-20 14:07:17 -05:00
John-Michael Faircloth
1e76ad42ef
secrets/db: add tests for static role config updates (#23153) 2023-09-19 10:12:09 -05:00
John-Michael Faircloth
16f805419f
fix rotation_window bug in error path (#22699) 2023-08-31 15:45:01 -05:00