Commit Graph

230 Commits

Author SHA1 Message Date
Jeff Mitchell
ff8becca15 Make proto
It appears the only thing that actually change is that the tag got
`proto3` values added.
2018-07-10 20:49:48 -04:00
Jim Kalafut
3a8b404621 Fix interface conversion panic during database creds revoke (#4850) 2018-06-28 09:42:04 -07:00
Jeff Mitchell
df00e62d92
Database updates (#4787)
* Database updates

* Add create/update distinction for connection config
* Add create/update distinction for role config
* Add db name and revocation statements to leases to give revocation a
shot at working if the role has been deleted

Fixes #3544
Fixes #4782

* Add create/update info to docs
2018-06-19 11:24:28 -04:00
Chris Hoffman
3b1ade706c
adding option go_package to protos (#4687)
* adding option go_package to protos

* switching proto output dir to relative paths
2018-06-04 10:19:26 -04:00
Brian Kassouf
aeb29a5048 Update proto files (#4651) 2018-05-29 18:23:51 -04:00
Jeff Mitchell
b61b541581
Revert "proto changes (#4503)" (#4504)
This reverts commit 14594bd76e.
2018-05-03 15:38:53 -04:00
Vishal Nayak
14594bd76e
proto changes (#4503) 2018-05-03 15:23:14 -04:00
Becca Petrin
ab9b638dbb
New proto version (#4501) 2018-05-03 10:19:39 -07:00
Chris Hoffman
295db4718f
Root Credential Rotation Docs (#4312)
* updating root credential docs

* more docs updates

* more docs updates
2018-04-09 12:20:29 -04:00
Chris Hoffman
aac9ee83f4
Fix deadlock in root credential rotation (#4309)
* fix deadlock in root credential rotation

* add more logging of errors

* adding cassandra test
2018-04-08 13:34:59 -04:00
Chris Hoffman
74ec7eb6e8
Prevent returning password in reads of connection config info (#4300)
* prevent returning password in reads of connection config info

* fixing a test

* masking password in connection url on reads

* addressing feedback

* removing extra check
2018-04-07 11:06:04 -04:00
Vishal Nayak
e2bb2ec3b9
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Chris Hoffman
def6e525be
Fix a few missing TTL core changes (#4265)
* Fix missing ttl handling in backends

* fix test
2018-04-04 06:43:21 -04:00
Chris Hoffman
6e3520658f
Core handling of TTLs (#4230)
* govet cleanup in token store

* adding general ttl handling to login requests

* consolidating TTL calculation to system view

* deprecate LeaseExtend

* deprecate LeaseExtend

* set the increment to the correct value

* move calculateTTL out of SystemView

* remove unused value

* add back clearing of lease id

* implement core ttl in some backends

* removing increment and issue time from lease options

* adding ttl tests, fixing some compile issue

* adding ttl tests

* fixing some explicit max TTL logic

* fixing up some tests

* removing unneeded test

* off by one errors...

* adding back some logic for bc

* adding period to return on renewal

* tweaking max ttl capping slightly

* use the appropriate precision for ttl calculation

* deprecate proto fields instead of delete

* addressing feedback

* moving TTL handling for backends to core

* mongo is a secret backend not auth

* adding estimated ttl for backends that also manage the expiration time

* set the estimate values before calling the renew request

* moving calculate TTL to framework, revert removal of increment and issue time from logical

* minor edits

* addressing feedback

* address more feedback
2018-04-03 12:20:20 -04:00
Becca Petrin
792d219aa9 Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Calvin Leung Huang
938b748914
Remove sensitive fields when reading config data (#4216)
* Remove sensitive fields when reading config data

* Do not use structs; build and return map explicitly

* Revert tag in postgresql

* Fix tests
2018-03-30 10:17:39 -04:00
Chris Hoffman
44aa151b78
Database Root Credential Rotation (#3976)
* redoing connection handling

* a little more cleanup

* empty implementation of rotation

* updating rotate signature

* signature update

* updating interfaces again :(

* changing back to interface

* adding templated url support and rotation for postgres

* adding correct username

* return updates

* updating statements to be a list

* adding error sanitizing middleware

* fixing log sanitizier

* adding postgres rotate test

* removing conf from rotate

* adding rotate command

* adding mysql rotate

* finishing up the endpoint in the db backend for rotate

* no more structs, just store raw config

* fixing tests

* adding db instance lock

* adding support for statement list in cassandra

* wip redoing interface to support BC

* adding falllback for Initialize implementation

* adding backwards compat for statements

* fix tests

* fix more tests

* fixing up tests, switching to new fields in statements

* fixing more tests

* adding mssql and mysql

* wrapping all the things in middleware, implementing templating for mongodb

* wrapping all db servers with error santizer

* fixing test

* store the name with the db instance

* adding rotate to cassandra

* adding compatibility translation to both server and plugin

* reordering a few things

* store the name with the db instance

* reordering

* adding a few more tests

* switch secret values from slice to map

* addressing some feedback

* reinstate execute plugin after resetting connection

* set database connection to closed

* switching secret values func to map[string]interface for potential future uses

* addressing feedback
2018-03-21 15:05:56 -04:00
Josh Soref
e43b76ef97 Spelling (#4119) 2018-03-20 14:54:10 -04:00
Jeff Mitchell
f51a7dad65
Honor mount-tuned ttl/max ttl for database credential generatoin (#4053) 2018-03-15 09:24:02 -07:00
Jeff Mitchell
7d73ac4c96 go vet fixes 2018-02-05 14:26:31 -05:00
Calvin Leung Huang
c3c63313f3
Version protocol switch (#3833)
* Use version to determine plugin protocol to use

* Remove field from ServeOpts

* Fix missing assignment, handle errors

* contraint -> constraint

* Inject the version string from the vault side

* Fix the version check

* Add grpc support check to database plugins

* Default to use grpc unless missing env var or fail on contraint check

* Add GRPCSupport test

* Add greater than test case

* Add go-version dep
2018-01-23 17:29:26 -05:00
Brian Kassouf
8142b42d95 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Brian Kassouf
03f6108822
gRPC Backend Plugins (#3808)
* Add grpc plugins

* Add grpc plugins

* Translate wrap info to/from proto

* Add nil checks

* Fix nil marshaling errors

* Provide logging through the go-plugin logger

* handle errors in the messages

* Update the TLS config so bidirectional connections work

* Add connectivity checks

* Restart plugin and add timeouts where context is not availible

* Add the response wrap data into the grpc system implementation

* Add leaseoptions to pb.Auth

* Add an error translator

* Add tests for translating the proto objects

* Fix rename of function

* Add tracing to plugins for easier debugging

* Handle plugin crashes with the go-plugin context

* Add test for grpcStorage

* Add tests for backend and system

* Bump go-plugin for GRPCBroker

* Remove RegisterLicense

* Add casing translations for new proto messages

* Use doneCtx in grpcClient

* Use doneCtx in grpcClient

* s/shutdown/shut down/
2018-01-18 13:49:20 -08:00
Jeff Mitchell
69eca11b62
Fix max_ttl not being honored in database backend when default_ttl is zero (#3814)
Fixes #3812
2018-01-18 01:43:38 -05:00
Chris Hoffman
c7b4fc314b Locking updates in database backend (#3774) 2018-01-17 19:21:59 -05:00
Brian Kassouf
a2b9ce7de8
remove the Initialize wrap and call close explicitly (#3769) 2018-01-10 13:07:55 -08:00
Brian Kassouf
05f20305b2
secret/database: ensure plugins are closed if they cannot be initialized (#3768) 2018-01-09 13:14:50 -08:00
Brian Kassouf
78adac0a24
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Brian Kassouf
6a74c119f3
secret/database: Fix upgrading database backend (#3714) 2017-12-18 19:38:47 -08:00
Brian Kassouf
a401cc7cb5
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Jeff Mitchell
8004f052da
Add some more SealWrap declarations (#3531) 2017-11-03 11:43:31 -04:00
Brian Kassouf
4121791cb9
Add the ability to glob allowed roles in the Database Backend (#3387)
* Add the ability to glob allowed roles in the Database Backend

* Make the error messages better

* Switch to the go-glob repo
2017-10-30 13:24:25 -07:00
Calvin Leung Huang
e0c84d0f9f Direct plugin logs through vault's logger (#3142)
* Direct plugin logs through vault's logger

* Pass in a logger in testConfig
2017-08-15 10:16:48 -04:00
Brian Kassouf
6e6bd6ee5c Bump database plugin protocol version 2017-08-08 17:01:38 -07:00
Jeff Mitchell
efe5a35c4a Clean up plugin tests with CA info 2017-07-31 15:09:19 -04:00
Jeff Mitchell
c6615e1b51 Add a -dev-three-node option for devs. (#3081) 2017-07-31 11:28:06 -04:00
Calvin Leung Huang
dd72c96dc8 Add BackendType to existing backends (#3078) 2017-07-28 14:04:46 -04:00
Calvin Leung Huang
2b0f80b981 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Jeff Mitchell
f75f5b0add Port TestCluster changes from proxy branch 2017-07-03 14:54:01 -04:00
Jeff Mitchell
2daf018361 Add listing to database connections. (#2827)
Fixes #2823
2017-06-07 10:03:17 -04:00
Brian Kassouf
abc900157b Use the role name in the db username (#2812) 2017-06-06 09:49:49 -04:00
Brian Kassouf
c55fd585f3 Add plugin level docs for what statements are supported and how they should be formatted 2017-05-11 11:59:58 -07:00
Brian Kassouf
886f873ffc Update docs and return a better error message 2017-05-04 11:45:27 -07:00
Brian Kassouf
c825362304 PR comments 2017-05-04 10:41:59 -07:00
Brian Kassouf
2af2b855f5 Feedback from PR 2017-05-03 17:37:34 -07:00
Brian Kassouf
a3619c4521 Update databse backend tests to use the APIClientMeta for the plugin conns 2017-05-03 16:34:09 -07:00
Brian Kassouf
60753dcf12 Only wrap in tracing middleware if the logger is set to trace level 2017-05-02 17:19:49 -07:00
Brian Kassouf
1df8ec9ef7 Update the api for serving plugins and provide a utility to pass TLS data for commuinicating with the vault process 2017-05-02 14:40:11 -07:00
Brian Kassouf
6ddfe9aa7f Rename NewPluginServer to just Serve 2017-05-02 02:00:39 -07:00
Brian Kassouf
6ca436cdf5 Don't store an error response as a package variable 2017-05-01 15:30:56 -07:00
Brian Kassouf
b87f8a13ed Update interface name from Wrapper to a more descriptive RunnerUtil 2017-05-01 14:59:55 -07:00
Brian Kassouf
2e2d3827da Add check to ensure we don't overwrite existing connections 2017-04-26 16:43:42 -07:00
Brian Kassouf
f92d6868a0 Add an error check to reset a plugin if it is closed 2017-04-26 15:55:34 -07:00
Brian Kassouf
6b050470fd Update to a RWMutex 2017-04-26 15:23:14 -07:00
Brian Kassouf
d8dbfc6a0c Update the error messages for renew and revoke 2017-04-26 10:29:16 -07:00
Brian Kassouf
37aacba0da Change ttl types to TypeDurationSecond 2017-04-26 10:02:37 -07:00
Brian Kassouf
6131bdd3b9 Default deny when allowed roles is empty 2017-04-25 11:48:24 -07:00
Brian Kassouf
e18757628c Update the connection details data and fix allowedRoles 2017-04-25 11:11:10 -07:00
Brian Kassouf
58b0bbd477 Rename path_role_create to path_creds_create 2017-04-25 10:39:17 -07:00
Brian Kassouf
22612adefc Use TypeCommaStringSlice for allowed_roles 2017-04-25 10:26:23 -07:00
Brian Kassouf
6741811407 Update logging to new structure 2017-04-25 10:24:19 -07:00
Brian Kassouf
194695f1fa Don't uppercase ErrorResponses 2017-04-24 14:03:48 -07:00
Brian Kassouf
f6b96ccfa2 s/DatabaseType/Database/ 2017-04-24 13:59:12 -07:00
Brian Kassouf
f1fa617e03 Calls to builtin plugins now go directly to the implementation instead of go-plugin 2017-04-20 18:46:41 -07:00
Brian Kassouf
b20c17745c Add allowed_roles parameter and checks 2017-04-13 10:33:34 -07:00
Brian Kassouf
4c75326aad Cleanup path files 2017-04-12 17:35:02 -07:00
Brian Kassouf
03e2bcbc79 Update Type() to return an error 2017-04-12 16:41:06 -07:00
Brian Kassouf
8f75c30311 Update help text and comments 2017-04-11 11:50:34 -07:00
Brian Kassouf
da4d9a8b4f Remove unnecessary abstraction 2017-04-10 18:38:34 -07:00
Brian Kassouf
de36d61e5a Mlock the plugin process 2017-04-10 17:12:52 -07:00
Brian Kassouf
f54c4de98a Add a flag to tell plugins to verify the connection was successful 2017-04-10 15:36:59 -07:00
Brian Kassouf
64efc505c8 Update plugin test 2017-04-10 14:12:28 -07:00
Brian Kassouf
73f66f89cd Update the interface for plugins removing functions for creating creds 2017-04-10 12:24:16 -07:00
Brian Kassouf
3c1c388589 Update backend tests 2017-04-10 10:35:16 -07:00
Brian Kassouf
9ae5a2aede Add backend test 2017-04-07 15:50:03 -07:00
Brian Kassouf
8e77bd98d8 Move plugin code into sub directory 2017-04-06 12:20:10 -07:00
Brian Kassouf
8a2e29c607 Refactor to use builtin plugins from an external repo 2017-04-05 16:20:31 -07:00
Calvin Leung Huang
73a2cdf6a5 Do not mark conn as initialized until the end (#2567) 2017-04-04 14:26:59 -07:00
Brian Kassouf
df944f2d92 Don't return strings, always structs 2017-04-04 11:33:58 -07:00
Calvin Leung Huang
8e3cb50bfc Database refactor invalidate (#2566)
* WIP on invalidate function

* cassandraConnectionProducer has Close()

* Delete database from connections map on successful db.Close()

* Move clear connection into its own func

* Use const for database config path
2017-04-04 11:32:42 -07:00
Brian Kassouf
1faa5fc020 On change of configuration rotate the database type 2017-04-03 18:30:38 -07:00
Brian Kassouf
b54e1cd295 Merge branch 'database-refactor' of github.com:hashicorp/vault into database-refactor 2017-04-03 17:52:41 -07:00
Brian Kassouf
ac519abecf Plugin catalog 2017-04-03 17:52:29 -07:00
Calvin Leung Huang
2b08521ab6 Database refactor mssql (#2562)
* WIP on mssql secret backend refactor

* Add RevokeUser test, and use sqlserver driver internally

* Remove debug statements

* Fix code comment
2017-04-03 09:59:30 -07:00
Brian Kassouf
1d3d3b7803 fix for plugin commands that have more than one paramater 2017-03-28 14:37:57 -07:00
Brian Kassouf
8ef78f0610 Add comments to connection and credential producers 2017-03-28 13:08:11 -07:00
Brian Kassouf
947fd66480 Cleanup the db factory code and add comments 2017-03-28 12:57:30 -07:00
Brian Kassouf
0c562fa3d7 Update tests 2017-03-28 12:20:17 -07:00
Brian Kassouf
6de5cfad5e Add functionaility to build db objects from disk so restarts work 2017-03-28 11:30:45 -07:00
Brian Kassouf
d93378bb29 Fix for checking types of database on update 2017-03-28 10:04:42 -07:00
Brian Kassouf
b2c4555c1f Wrap the database calls with tracing information 2017-03-27 15:17:28 -07:00
Brian Kassouf
ca026c6cfd Remove the unused sync.Once object 2017-03-27 11:46:20 -07:00
Brian Kassouf
e870e399a2 More work on getting tests to pass 2017-03-23 15:54:15 -07:00
Brian Kassouf
a1b72465dd Remove unsused code block 2017-03-22 17:09:39 -07:00
Brian Kassouf
cab491f7b7 s/postgres/mysql/ 2017-03-22 16:44:33 -07:00
Brian Kassouf
73e553af95 Add test files for postgres and mysql databases 2017-03-22 16:39:08 -07:00
Brian Kassouf
9aaec25a4e Add a error message for empty creation statement 2017-03-22 12:40:16 -07:00
Brian Kassouf
1be813605f Fix race with deleting the connection 2017-03-22 09:54:19 -07:00
Brian Kassouf
2d6f36df17 Add a delete method 2017-03-21 17:19:30 -07:00
Brian Kassouf
2fdb3422a9 Verify connections regardless of if this connections is already existing 2017-03-21 16:05:59 -07:00
Brian Kassouf
ff6749b198 Comment and fix plugin Type function 2017-03-16 18:24:56 -07:00
Brian Kassouf
404596e261 Change the handshake config from the default 2017-03-16 17:51:25 -07:00
Brian Kassouf
4043f533b8 Add a secure config to verify the checksum of the plugin 2017-03-16 16:20:18 -07:00
Brian Kassouf
2ef1cbf3a6 Comment and slight refactor of the TLS plugin helper 2017-03-16 14:14:49 -07:00
Brian Kassouf
3890f194a4 Break tls code into helper library 2017-03-16 11:55:21 -07:00
Brian Kassouf
5b05f62fa3 Work on TLS communication over plugins 2017-03-15 17:14:48 -07:00
Brian Kassouf
a6ae4bd356 wrap plugin database type with metrics middleware 2017-03-14 13:12:47 -07:00
Brian Kassouf
143166b1ba Add a metrics middleware 2017-03-14 13:11:28 -07:00
Brian Kassouf
c111b02568 Add a way to initalize plugins and builtin databases the same way. 2017-03-13 14:39:55 -07:00
Brian Kassouf
a0d207e254 Add checksum attribute 2017-03-10 14:10:42 -08:00
Brian Kassouf
72a878b180 Rename reset to close 2017-03-09 22:35:45 -08:00
Brian Kassouf
b63147b7c2 Add special path to enforce root on plugin configuration 2017-03-09 21:31:29 -08:00
Brian Kassouf
3766ab14e5 Add plugin file 2017-03-09 17:43:58 -08:00
Brian Kassouf
d4ea6c1768 Add plugin features 2017-03-09 17:43:37 -08:00
Brian Kassouf
00359cdea4 Update secrets fields 2017-03-08 14:46:53 -08:00
Brian Kassouf
cd68899a4a Fix renew and revoke calls 2017-03-07 17:21:44 -08:00
Brian Kassouf
73200db1d9 Add defaults to the cassandra databse type 2017-03-07 17:00:52 -08:00
Brian Kassouf
78fdc2ad24 Pass statements object 2017-03-07 16:48:17 -08:00
Brian Kassouf
01300e026b Remove unused sql object 2017-03-07 15:34:23 -08:00
Brian Kassouf
1d23bbbe28 Remove double lock 2017-03-07 15:33:05 -08:00
Brian Kassouf
c823ad0597 Update locking functionaility 2017-03-07 13:48:29 -08:00
Brian Kassouf
354233f91d rename mysql variable 2017-03-03 15:07:41 -08:00
Brian Kassouf
4d335099de Make db instances immutable and add a reset path to tear down and create a new database instance with an updated config 2017-03-03 14:38:49 -08:00
Brian Kassouf
fa8da4cf91 Fix mysql connections 2017-03-03 14:38:49 -08:00
Brian Kassouf
e442917e26 Add mysql into the factory 2017-03-03 14:38:48 -08:00
Brian Kassouf
5e2cffcdd0 Add max connection lifetime param and set consistancy on cassandra session 2017-03-03 14:38:48 -08:00
Brian Kassouf
cee3dc9b9e s/Statement/Statements/ 2017-03-03 14:38:48 -08:00
Brian Kassouf
bfbb104e19 Add mysql database type 2017-03-03 14:38:48 -08:00
Brian Kassouf
ad17d113c7 More work on refactor and cassandra database 2017-03-03 14:38:48 -08:00
Brian Kassouf
3d77a9a6f4 Begin work on database refactor 2017-03-03 14:38:48 -08:00