* Clarify SetLastVaultRotation comment, only set some of the PopulateAutomatedRotationData fields when they are set
* Revert "Clarify SetLastVaultRotation comment, only set some of the PopulateAutomatedRotationData fields when they are set"
This reverts commit f8353cd1c86bfd705dc47d059b8f4f4da2d93677.
* Add deprecation remark and new function
* Force PR to recognize changes, hopefully
* Create a TestRotationSystemView in the sdk for ent testing
* ent directive
* Remove LastVaultRotation from test systemview
* Add MigratedLegacyNextRotationTime for rotation manager migration
* comment
* Set the MountPoint on a rotation job based on the entry regardless of what the plugin claims
* Update fields.go
* Add test coverage
Co-authored-by: Robert <17119716+robmonte@users.noreply.github.com>
* rotation-manager: enable RM to send rotation information to plugins on registration/rotation operations (#11810)
* initial commit for sending NVR to plugins
* add changelog
* add NVR to plugin fields, add RotationInfo to GRPC request handler
* fix tests
* ensure consistent formats on times and ttls
* add translation to allow grpc data transfer
* fix tests and rename fields
* fix missed field renames in tests
* make all methods net-new for backwards compatibility
* update mock plugin and add oss stub back
* remove method with no usages
* Address wrapper comments
* Rebuild proto
* Nil check around SetRotationInfo, return n/a for no last_vault_rotation
* Fix error to match other instances
* Update fields.go
* Return nil if unset for next/last vault rotation times
---------
Co-authored-by: robmonte <17119716+robmonte@users.noreply.github.com>
* Fix return type in stub method
---------
Co-authored-by: vinay-gopalan <86625824+vinay-gopalan@users.noreply.github.com>
Co-authored-by: robmonte <17119716+robmonte@users.noreply.github.com>
* Vault 42177 Add Backend Field (#12092)
* add a new struct for the total number of successful requests for transit and transform
* implement tracking for encrypt path
* implement tracking in encrypt path
* add tracking in rewrap
* add tracking to datakey path
* add tracking to hmac path
* add tracking to sign path
* add tracking to verify path
* unit tests for verify path
* add tracking to cmac path
* reset the global counter in each unit test
* add tracking to hmac verify
* add methods to retrieve and flush transit count
* modify the methods that store and update data protection call counts
* update the methods
* add a helper method to combine replicated and local data call counts
* add tracking to the endpoint
* fix some formatting errors
* add unit tests to path encrypt for tracking
* add unit tests to decrypt path
* fix linter error
* add unit tests to test update and store methods for data protection calls
* stub fix: do not create separate files
* fix the tracking by coordinating replicated and local data, add unit tests
* update all reference to the new data struct
* revert to previous design with just one global counter for all calls for each cluster
* complete external test
* no need to check if current count is greater than 0, remove it
* feedback: remove unnacassary comments about atomic addition, standardize comments
* leave jira id on todo comment, remove unused method
* rename mathods by removing HWM and max in names, update jira id in todo comment, update response field key name
* feedback: remove explicit counter in cmac tests, instead put in the expected number
* feedback: remove explicit tracking in the rest of the tests
* feedback: separate transit testing into its own external test
* Update vault/consumption_billing_util_test.go
Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
* update comment after test name change
* fix comments
* fix comments in test
* another comment fix
* feedback: remove incorrect comment
* fix a CE test
* fix the update method: instead of storing max, increment by the current count value
* update the unit test, remove local prefix as argument to the methods since we store only to non-replicated paths
* update the external test
* Adds a field to backend to track billing data
removed file
* Changed implementation to use a map instead
* Some more comments
* Add more implementation
* Edited grpc server backend
* Refactored a bit
* Fix one more test
* Modified map:
* Revert "Modified map:"
This reverts commit 1730fe1f358b210e6abae43fbdca09e585aaaaa8.
* Removed some other things
* Edited consumption billing files a bit
* Testing function
* Fix transit stuff and make sure tests pass
* Changes
* More changes
* More changes
* Edited external test
* Edited some more tests
* Edited and fixed tests
* One more fix
* Fix some more tests
* Moved some testing structures around and added error checking
* Fixed some nits
* Update builtin/logical/transit/path_sign_verify.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Edited some errors
* Fixed error logs
* Edited one more thing
* Decorate the error
* Update vault/consumption_billing.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
---------
Co-authored-by: Amir Aslamov <amir.aslamov@hashicorp.com>
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Edited stub function
---------
Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
Co-authored-by: Amir Aslamov <amir.aslamov@hashicorp.com>
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Co-authored-by: divyaac <divyaac@berkeley.edu>
* license: update headers to IBM Corp.
* `make proto`
* update offset because source file changed
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* add ce side code and stubs
* add changelog
* style refactor
* try to use APIPath as mount point instead of request field
* fix linter
* return a response struct instead of a pure timestamp
* add issue time to response
* add ttl to GetRotationInformation response
* rename field for clarity
* update ttl to just seconds
* rename next and last rotation time field; describe what they are
* rename function
* catch up to ent PR
* fix patch merge mistake
* Fix "t.Fatal from a non-test goroutine" errors in cache_test.go
- t.Fatal(f) should not be called within a Go routine based on it's documentation and only from the main test's thread.
- In 1.24 this seems to cause build failures
* Address all "non-constant format string errors" from go vet
- Within 1.24 these now cause test builds to fail
…" from go vet
* Add WriteForwardedStorage to sdk's plugin, logical in OSS
This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.
Notably, these semantics are subject to change and shouldn't yet be
relied on.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Collect paths for write forwarding in OSS
This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add forwarded writer implementation to OSS
Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Wire up mount-specific request forwarding in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify that state lock needs to be held to call HAState in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move cluster sentinel constant to sdk/logical
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Expose ClusterID to Plugins via SystemView
This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add GRPC ClusterID implementation
For any external plugins which wish to use it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Move version out of SDK. For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString. That will have to wait until we've removed uses of useragent.String from all builtins.
And return an error instead of panicking.
This situation can occur if a plugin attempts to access the system
view during setup when Vault is checking the plugin metadata.
Fixes#17878.
* Support version selection for database plugins
* Don't consider unversioned plugins for version selection algorithm
* Added version to 'plugin not found' error
* Add PluginFactoryVersion function to avoid changing sdk/ API
* feat: DB plugin multiplexing (#13734)
* WIP: start from main and get a plugin runner from core
* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode
* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient
* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient
* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists
* add ID to dbplugin.DatabasePluginClient struct
* only create one plugin process per plugin type
* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server
* add v6 multiplexing protocol version
* WIP: backwards compat for db plugins
* Ensure locking on plugin catalog access
- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin
* use the New constructor for db plugins
* grpc server: use write lock for Close and rlock for CRUD
* cleanup MultiplexedClients on Close
* remove TODO
* fix multiplexing regression with grpc server connection
* cleanup grpc server instances on close
* embed ClientProtocol in Multiplexer interface
* use PluginClientConfig arg to make NewPluginClient plugin type agnostic
* create a new plugin process for non-muxed plugins
* feat: plugin multiplexing: handle plugin client cleanup (#13896)
* use closure for plugin client cleanup
* log and return errors; add comments
* move rpcClient wrapping to core for ID injection
* refactor core plugin client and sdk
* remove unused ID method
* refactor and only wrap clientConn on multiplexed plugins
* rename structs and do not export types
* Slight refactor of system view interface
* Revert "Slight refactor of system view interface"
This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.
* Revert "Revert "Slight refactor of system view interface""
This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.
* only provide pluginRunner arg to the internal newPluginClient method
* embed ClientProtocol in pluginClient and name logger
* Add back MLock support
* remove enableMlock arg from setupPluginCatalog
* rename plugin util interface to PluginClient
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
* feature: multiplexing: fix unit tests (#14007)
* fix grpc_server tests and add coverage
* update run_config tests
* add happy path test case for grpc_server ID from context
* update test helpers
* feat: multiplexing: handle v5 plugin compiled with new sdk
* add mux supported flag and increase test coverage
* set multiplexingSupport field in plugin server
* remove multiplexingSupport field in sdk
* revert postgres to non-multiplexed
* add comments on grpc server fields
* use pointer receiver on grpc server methods
* add changelog
* use pointer for grpcserver instance
* Use a gRPC server to determine if a plugin should be multiplexed
* Apply suggestions from code review
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* add lock to removePluginClient
* add multiplexingSupport field to externalPlugin struct
* do not send nil to grpc MultiplexingSupport
* check err before logging
* handle locking scenario for cleanupFunc
* allow ServeConfigMultiplex to dispense v5 plugin
* reposition structs, add err check and comments
* add comment on locking for cleanupExternalPlugin
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* Add random string generator with rules engine
This adds a random string generation library that validates random
strings against a set of rules. The library is designed for use as generating
passwords, but can be used to generate any random strings.
* Move SudoPrivilege out of SystemView
We only use this in token store and it literally doesn't work anything
that isn't the token store or system mount, so we should stop exposing
something that doesn't work.
* Reconcile extended system view with sdk/logical a bit and put an explanation for why SudoPrivilege isn't moved over
