* feat: DB plugin multiplexing (#13734) * WIP: start from main and get a plugin runner from core * move MultiplexedClient map to plugin catalog - call sys.NewPluginClient from PluginFactory - updates to getPluginClient - thread through isMetadataMode * use go-plugin ClientProtocol interface - call sys.NewPluginClient from dbplugin.NewPluginClient * move PluginSets to dbplugin package - export dbplugin HandshakeConfig - small refactor of PluginCatalog.getPluginClient * add removeMultiplexedClient; clean up on Close() - call client.Kill from plugin catalog - set rpcClient when muxed client exists * add ID to dbplugin.DatabasePluginClient struct * only create one plugin process per plugin type * update NewPluginClient to return connection ID to sdk - wrap grpc.ClientConn so we can inject the ID into context - get ID from context on grpc server * add v6 multiplexing protocol version * WIP: backwards compat for db plugins * Ensure locking on plugin catalog access - Create public GetPluginClient method for plugin catalog - rename postgres db plugin * use the New constructor for db plugins * grpc server: use write lock for Close and rlock for CRUD * cleanup MultiplexedClients on Close * remove TODO * fix multiplexing regression with grpc server connection * cleanup grpc server instances on close * embed ClientProtocol in Multiplexer interface * use PluginClientConfig arg to make NewPluginClient plugin type agnostic * create a new plugin process for non-muxed plugins * feat: plugin multiplexing: handle plugin client cleanup (#13896) * use closure for plugin client cleanup * log and return errors; add comments * move rpcClient wrapping to core for ID injection * refactor core plugin client and sdk * remove unused ID method * refactor and only wrap clientConn on multiplexed plugins * rename structs and do not export types * Slight refactor of system view interface * Revert "Slight refactor of system view interface" This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6. * Revert "Revert "Slight refactor of system view interface"" This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f. * only provide pluginRunner arg to the internal newPluginClient method * embed ClientProtocol in pluginClient and name logger * Add back MLock support * remove enableMlock arg from setupPluginCatalog * rename plugin util interface to PluginClient Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com> * feature: multiplexing: fix unit tests (#14007) * fix grpc_server tests and add coverage * update run_config tests * add happy path test case for grpc_server ID from context * update test helpers * feat: multiplexing: handle v5 plugin compiled with new sdk * add mux supported flag and increase test coverage * set multiplexingSupport field in plugin server * remove multiplexingSupport field in sdk * revert postgres to non-multiplexed * add comments on grpc server fields * use pointer receiver on grpc server methods * add changelog * use pointer for grpcserver instance * Use a gRPC server to determine if a plugin should be multiplexed * Apply suggestions from code review Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com> * add lock to removePluginClient * add multiplexingSupport field to externalPlugin struct * do not send nil to grpc MultiplexingSupport * check err before logging * handle locking scenario for cleanupFunc * allow ServeConfigMultiplex to dispense v5 plugin * reposition structs, add err check and comments * add comment on locking for cleanupExternalPlugin Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com> Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Combined Database Engine
This package is how database plugins interact with Vault.
Upgrading to Version 5
Background
In Vault 1.6, a new Database interface was created that solved a number of issues with the previous interface:
- It could not use password policies because the database plugins were responsible for generating passwords.
- There were significant inconsistencies between functions in the interface.
- Several functions (
SetCredentialsandRotateRootCredentials) were doing the same operation. - It had a function that was no longer being used as it had been deprecated in a previous version but never removed.
Prior to Vault 1.6, the Database interface is version 4 (with other versions in older versions of Vault). The new version introduced in Vault 1.6 is version 5. This distinction was not exposed in previous iterations of the Database interface as the previous versions were additive to the interface. Since version 5 is an overhaul of the interface, this distinction needed to be made.
We highly recommend that you upgrade any version 4 database plugins to version 5 as version 4 is considered deprecated and support for it will be removed in a future release. Version 5 plugins will not function with Vault prior to Vault 1.6.
The new interface is roughly modeled after a gRPC interface. It has improved future compatibility by not requiring changes to the interface definition to add additional data in the requests or responses. It also simplifies the interface by merging several into a single function call.
Upgrading your custom database
Vault 1.6 supports both version 4 and version 5 database plugins. The support for version 4 plugins will be removed in a future release. Version 5 database plugins will not function with Vault prior to version 1.6. If you upgrade your database plugins, ensure that you are only using Vault 1.6 or later. To determine if a plugin is using version 4 or version 5, the following is a list of changes in no particular order that you can check against your plugin to determine the version:
- The import path for version 4 is
github.com/hashicorp/vault/sdk/database/dbpluginwhereas the import path for version 5 isgithub.com/hashicorp/vault/sdk/database/dbplugin/v5 - Version 4 has the following functions:
Initialize,Init,CreateUser,RenewUser,RevokeUser,SetCredentials,RotateRootCredentials,Type, andClose. You can see the full function signatures insdk/database/dbplugin/plugin.go. - Version 5 has the following functions:
Initialize,NewUser,UpdateUser,DeleteUser,Type, andClose. You can see the full function signatures insdk/database/dbplugin/v5/database.go.
If you are using a version 4 custom database plugin, the following are basic instructions for upgrading to version 5.
-> In version 4, password generation was the responsibility of the plugin. This is no longer
the case with version 5. Vault is responsible for generating passwords and passing them to
the plugin via NewUserRequest.Password and UpdateUserRequest.Password.NewPassword.
- Change the import path from
github.com/hashicorp/vault/sdk/database/dbplugintogithub.com/hashicorp/vault/sdk/database/dbplugin/v5. The package name is the same, so any references todbplugincan remain as long as those symbols exist within the new package (such as theServefunction). - An easy way to see what functions need to be implemented is to put the following as a
global variable within your package:
var _ dbplugin.Database = (*MyDatabase)(nil). This will fail to compile if theMyDatabasetype does not adhere to thedbplugin.Databaseinterface. - Replace
InitandInitializewith the newInitializefunction definition. The fields thatInitwas taking (configandverifyConnection) are now wrapped intoInitializeRequest. The returnedmap[string]interface{}object is now wrapped intoInitializeResponse. OnlyInitializeis needed to adhere to theDatabaseinterface. - Update
CreateUsertoNewUser. TheNewUserRequestobject contains the username and password of the user to be created. It also includes a list of statements for creating the user as well as several other fields that may or may not be applicable. Your custom plugin should use the password provided in the request, not generate one. If you generate a password instead, Vault will not know about it and will give the caller the wrong password. SetCredentials,RotateRootCredentials, andRenewUserare combined intoUpdateUser. The request object,UpdateUserRequestcontains three parts: the username to change, aChangePasswordand aChangeExpirationobject. When one of the objects is not nil, this indicates that particular field (password or expiration) needs to change. For instance, if theChangePasswordfield is not-nil, the user's password should be changed. This is equivalent to callingSetCredentials. If theChangeExpirationfield is not-nil, the user's expiration date should be changed. This is equivalent to callingRenewUser. Many databases don't need to do anything with the updated expiration.- Update
RevokeUsertoDeleteUser. This is the simplest change. The username to be deleted is enclosed in theDeleteUserRequestobject.