Commit Graph

195 Commits

Author SHA1 Message Date
Jeff Mitchell
20423f7082
Add allowed_response_headers (#6115) 2019-02-05 16:02:15 -05:00
Jeff Mitchell
47d339fce1 Revert "Refactor common token fields and operations into a helper (#5953)"
This reverts commit 66c226c593.
2019-02-01 11:23:40 -05:00
Jeff Mitchell
66c226c593
Refactor common token fields and operations into a helper (#5953) 2019-01-30 16:23:28 -05:00
Jim Kalafut
a54603039d Run goimports across the repository (#6010)
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
2019-01-08 16:48:57 -08:00
Jeff Mitchell
ca55573d38
Strip empty strings from database revocation stmts (#5955)
* Strip empty strings from database revocation stmts

It's technically valid to give empty strings as statements to run on
most databases. However, in the case of revocation statements, it's not
only generally inadvisable but can lead to lack of revocations when you
expect them. This strips empty strings from the array of revocation
statements.

It also makes two other changes:

* Return statements on read as empty but valid arrays rather than nulls,
so that typing information is inferred (this is more in line with the
rest of Vault these days)

* Changes field data for TypeStringSlice and TypeCommaStringSlice such
that a client-supplied value of `""` doesn't turn into `[]string{""}`
but rather `[]string{}`.

The latter and the explicit revocation statement changes are related,
and defense in depth.
2018-12-14 09:12:26 -05:00
Becca Petrin
fb89af7cfa
Run all builtins as plugins (#5536) 2018-11-06 17:21:24 -08:00
Vishal Nayak
448eb98987
Remove namespace.TestContext and namespace.TestNamespace (#5682) 2018-11-05 11:11:32 -05:00
Brian Kassouf
3dcb20fdd2
Plugin version negotiation (#5434)
* Plugin version updates

* Update datatbase plugins

* Revert netRPC deletions

* Revert netRPC deletions

* Update plugins to serve both versions

* Update database plugins

* Add Initialize back in

* revert pointer changes

* Add deprecation warning

* Update tests

* Update go-plugin

* Review Feedback
2018-10-19 15:56:17 -07:00
Jeff Mitchell
e7c4158e60 Fix compilation/protobuf 2018-09-22 17:58:39 -04:00
Calvin Leung Huang
494b9a039c
Add ability to provide env vars to plugins (#5359)
* Add ability to provide env vars to plugins

* Update docs

* Update docs with examples

* Refactor TestAddTestPlugin, remove TestAddTestPluginTempDir
2018-09-20 10:50:29 -07:00
Jeff Mitchell
b7d6d55ac1
The big one (#5346) 2018-09-17 23:03:00 -04:00
Jeff Mitchell
7a72866797
Fix nil pointer if the embedded revocation statements from a role are nil (#5268)
Fixes #5262
2018-09-05 11:41:37 -04:00
Jim Kalafut
c4b32ced0e
Fix DB role statement update (#5058)
The backwards compatibility logic was preventing updates to role
statements from taking effect. This change removes persistence of
deprecated statement fields.
2018-08-10 09:00:52 -07:00
Calvin Leung Huang
ece7569aca Clean up container on connection failure, switch to ory/dockertest on package postgresql (#5050) 2018-08-07 09:56:33 -07:00
Jeff Mitchell
dba48d5265
Re-add dockertest and fix up imports and update script (#4909) 2018-07-11 17:49:13 -04:00
Jeff Mitchell
ff8becca15 Make proto
It appears the only thing that actually change is that the tag got
`proto3` values added.
2018-07-10 20:49:48 -04:00
Jim Kalafut
3a8b404621 Fix interface conversion panic during database creds revoke (#4850) 2018-06-28 09:42:04 -07:00
Jeff Mitchell
df00e62d92
Database updates (#4787)
* Database updates

* Add create/update distinction for connection config
* Add create/update distinction for role config
* Add db name and revocation statements to leases to give revocation a
shot at working if the role has been deleted

Fixes #3544
Fixes #4782

* Add create/update info to docs
2018-06-19 11:24:28 -04:00
Chris Hoffman
3b1ade706c
adding option go_package to protos (#4687)
* adding option go_package to protos

* switching proto output dir to relative paths
2018-06-04 10:19:26 -04:00
Brian Kassouf
aeb29a5048 Update proto files (#4651) 2018-05-29 18:23:51 -04:00
Jeff Mitchell
b61b541581
Revert "proto changes (#4503)" (#4504)
This reverts commit 14594bd76e.
2018-05-03 15:38:53 -04:00
Vishal Nayak
14594bd76e
proto changes (#4503) 2018-05-03 15:23:14 -04:00
Becca Petrin
ab9b638dbb
New proto version (#4501) 2018-05-03 10:19:39 -07:00
Chris Hoffman
295db4718f
Root Credential Rotation Docs (#4312)
* updating root credential docs

* more docs updates

* more docs updates
2018-04-09 12:20:29 -04:00
Chris Hoffman
aac9ee83f4
Fix deadlock in root credential rotation (#4309)
* fix deadlock in root credential rotation

* add more logging of errors

* adding cassandra test
2018-04-08 13:34:59 -04:00
Chris Hoffman
74ec7eb6e8
Prevent returning password in reads of connection config info (#4300)
* prevent returning password in reads of connection config info

* fixing a test

* masking password in connection url on reads

* addressing feedback

* removing extra check
2018-04-07 11:06:04 -04:00
Vishal Nayak
e2bb2ec3b9
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Chris Hoffman
def6e525be
Fix a few missing TTL core changes (#4265)
* Fix missing ttl handling in backends

* fix test
2018-04-04 06:43:21 -04:00
Chris Hoffman
6e3520658f
Core handling of TTLs (#4230)
* govet cleanup in token store

* adding general ttl handling to login requests

* consolidating TTL calculation to system view

* deprecate LeaseExtend

* deprecate LeaseExtend

* set the increment to the correct value

* move calculateTTL out of SystemView

* remove unused value

* add back clearing of lease id

* implement core ttl in some backends

* removing increment and issue time from lease options

* adding ttl tests, fixing some compile issue

* adding ttl tests

* fixing some explicit max TTL logic

* fixing up some tests

* removing unneeded test

* off by one errors...

* adding back some logic for bc

* adding period to return on renewal

* tweaking max ttl capping slightly

* use the appropriate precision for ttl calculation

* deprecate proto fields instead of delete

* addressing feedback

* moving TTL handling for backends to core

* mongo is a secret backend not auth

* adding estimated ttl for backends that also manage the expiration time

* set the estimate values before calling the renew request

* moving calculate TTL to framework, revert removal of increment and issue time from logical

* minor edits

* addressing feedback

* address more feedback
2018-04-03 12:20:20 -04:00
Becca Petrin
792d219aa9 Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Calvin Leung Huang
938b748914
Remove sensitive fields when reading config data (#4216)
* Remove sensitive fields when reading config data

* Do not use structs; build and return map explicitly

* Revert tag in postgresql

* Fix tests
2018-03-30 10:17:39 -04:00
Chris Hoffman
44aa151b78
Database Root Credential Rotation (#3976)
* redoing connection handling

* a little more cleanup

* empty implementation of rotation

* updating rotate signature

* signature update

* updating interfaces again :(

* changing back to interface

* adding templated url support and rotation for postgres

* adding correct username

* return updates

* updating statements to be a list

* adding error sanitizing middleware

* fixing log sanitizier

* adding postgres rotate test

* removing conf from rotate

* adding rotate command

* adding mysql rotate

* finishing up the endpoint in the db backend for rotate

* no more structs, just store raw config

* fixing tests

* adding db instance lock

* adding support for statement list in cassandra

* wip redoing interface to support BC

* adding falllback for Initialize implementation

* adding backwards compat for statements

* fix tests

* fix more tests

* fixing up tests, switching to new fields in statements

* fixing more tests

* adding mssql and mysql

* wrapping all the things in middleware, implementing templating for mongodb

* wrapping all db servers with error santizer

* fixing test

* store the name with the db instance

* adding rotate to cassandra

* adding compatibility translation to both server and plugin

* reordering a few things

* store the name with the db instance

* reordering

* adding a few more tests

* switch secret values from slice to map

* addressing some feedback

* reinstate execute plugin after resetting connection

* set database connection to closed

* switching secret values func to map[string]interface for potential future uses

* addressing feedback
2018-03-21 15:05:56 -04:00
Josh Soref
e43b76ef97 Spelling (#4119) 2018-03-20 14:54:10 -04:00
Jeff Mitchell
f51a7dad65
Honor mount-tuned ttl/max ttl for database credential generatoin (#4053) 2018-03-15 09:24:02 -07:00
Jeff Mitchell
7d73ac4c96 go vet fixes 2018-02-05 14:26:31 -05:00
Calvin Leung Huang
c3c63313f3
Version protocol switch (#3833)
* Use version to determine plugin protocol to use

* Remove field from ServeOpts

* Fix missing assignment, handle errors

* contraint -> constraint

* Inject the version string from the vault side

* Fix the version check

* Add grpc support check to database plugins

* Default to use grpc unless missing env var or fail on contraint check

* Add GRPCSupport test

* Add greater than test case

* Add go-version dep
2018-01-23 17:29:26 -05:00
Brian Kassouf
8142b42d95 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Brian Kassouf
03f6108822
gRPC Backend Plugins (#3808)
* Add grpc plugins

* Add grpc plugins

* Translate wrap info to/from proto

* Add nil checks

* Fix nil marshaling errors

* Provide logging through the go-plugin logger

* handle errors in the messages

* Update the TLS config so bidirectional connections work

* Add connectivity checks

* Restart plugin and add timeouts where context is not availible

* Add the response wrap data into the grpc system implementation

* Add leaseoptions to pb.Auth

* Add an error translator

* Add tests for translating the proto objects

* Fix rename of function

* Add tracing to plugins for easier debugging

* Handle plugin crashes with the go-plugin context

* Add test for grpcStorage

* Add tests for backend and system

* Bump go-plugin for GRPCBroker

* Remove RegisterLicense

* Add casing translations for new proto messages

* Use doneCtx in grpcClient

* Use doneCtx in grpcClient

* s/shutdown/shut down/
2018-01-18 13:49:20 -08:00
Jeff Mitchell
69eca11b62
Fix max_ttl not being honored in database backend when default_ttl is zero (#3814)
Fixes #3812
2018-01-18 01:43:38 -05:00
Chris Hoffman
c7b4fc314b Locking updates in database backend (#3774) 2018-01-17 19:21:59 -05:00
Brian Kassouf
a2b9ce7de8
remove the Initialize wrap and call close explicitly (#3769) 2018-01-10 13:07:55 -08:00
Brian Kassouf
05f20305b2
secret/database: ensure plugins are closed if they cannot be initialized (#3768) 2018-01-09 13:14:50 -08:00
Brian Kassouf
78adac0a24
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Brian Kassouf
6a74c119f3
secret/database: Fix upgrading database backend (#3714) 2017-12-18 19:38:47 -08:00
Brian Kassouf
a401cc7cb5
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Jeff Mitchell
8004f052da
Add some more SealWrap declarations (#3531) 2017-11-03 11:43:31 -04:00
Brian Kassouf
4121791cb9
Add the ability to glob allowed roles in the Database Backend (#3387)
* Add the ability to glob allowed roles in the Database Backend

* Make the error messages better

* Switch to the go-glob repo
2017-10-30 13:24:25 -07:00
Calvin Leung Huang
e0c84d0f9f Direct plugin logs through vault's logger (#3142)
* Direct plugin logs through vault's logger

* Pass in a logger in testConfig
2017-08-15 10:16:48 -04:00
Brian Kassouf
6e6bd6ee5c Bump database plugin protocol version 2017-08-08 17:01:38 -07:00
Jeff Mitchell
efe5a35c4a Clean up plugin tests with CA info 2017-07-31 15:09:19 -04:00
Jeff Mitchell
c6615e1b51 Add a -dev-three-node option for devs. (#3081) 2017-07-31 11:28:06 -04:00
Calvin Leung Huang
dd72c96dc8 Add BackendType to existing backends (#3078) 2017-07-28 14:04:46 -04:00
Calvin Leung Huang
2b0f80b981 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Jeff Mitchell
f75f5b0add Port TestCluster changes from proxy branch 2017-07-03 14:54:01 -04:00
Jeff Mitchell
2daf018361 Add listing to database connections. (#2827)
Fixes #2823
2017-06-07 10:03:17 -04:00
Brian Kassouf
abc900157b Use the role name in the db username (#2812) 2017-06-06 09:49:49 -04:00
Brian Kassouf
c55fd585f3 Add plugin level docs for what statements are supported and how they should be formatted 2017-05-11 11:59:58 -07:00
Brian Kassouf
886f873ffc Update docs and return a better error message 2017-05-04 11:45:27 -07:00
Brian Kassouf
c825362304 PR comments 2017-05-04 10:41:59 -07:00
Brian Kassouf
2af2b855f5 Feedback from PR 2017-05-03 17:37:34 -07:00
Brian Kassouf
a3619c4521 Update databse backend tests to use the APIClientMeta for the plugin conns 2017-05-03 16:34:09 -07:00
Brian Kassouf
60753dcf12 Only wrap in tracing middleware if the logger is set to trace level 2017-05-02 17:19:49 -07:00
Brian Kassouf
1df8ec9ef7 Update the api for serving plugins and provide a utility to pass TLS data for commuinicating with the vault process 2017-05-02 14:40:11 -07:00
Brian Kassouf
6ddfe9aa7f Rename NewPluginServer to just Serve 2017-05-02 02:00:39 -07:00
Brian Kassouf
6ca436cdf5 Don't store an error response as a package variable 2017-05-01 15:30:56 -07:00
Brian Kassouf
b87f8a13ed Update interface name from Wrapper to a more descriptive RunnerUtil 2017-05-01 14:59:55 -07:00
Brian Kassouf
2e2d3827da Add check to ensure we don't overwrite existing connections 2017-04-26 16:43:42 -07:00
Brian Kassouf
f92d6868a0 Add an error check to reset a plugin if it is closed 2017-04-26 15:55:34 -07:00
Brian Kassouf
6b050470fd Update to a RWMutex 2017-04-26 15:23:14 -07:00
Brian Kassouf
d8dbfc6a0c Update the error messages for renew and revoke 2017-04-26 10:29:16 -07:00
Brian Kassouf
37aacba0da Change ttl types to TypeDurationSecond 2017-04-26 10:02:37 -07:00
Brian Kassouf
6131bdd3b9 Default deny when allowed roles is empty 2017-04-25 11:48:24 -07:00
Brian Kassouf
e18757628c Update the connection details data and fix allowedRoles 2017-04-25 11:11:10 -07:00
Brian Kassouf
58b0bbd477 Rename path_role_create to path_creds_create 2017-04-25 10:39:17 -07:00
Brian Kassouf
22612adefc Use TypeCommaStringSlice for allowed_roles 2017-04-25 10:26:23 -07:00
Brian Kassouf
6741811407 Update logging to new structure 2017-04-25 10:24:19 -07:00
Brian Kassouf
194695f1fa Don't uppercase ErrorResponses 2017-04-24 14:03:48 -07:00
Brian Kassouf
f6b96ccfa2 s/DatabaseType/Database/ 2017-04-24 13:59:12 -07:00
Brian Kassouf
f1fa617e03 Calls to builtin plugins now go directly to the implementation instead of go-plugin 2017-04-20 18:46:41 -07:00
Brian Kassouf
b20c17745c Add allowed_roles parameter and checks 2017-04-13 10:33:34 -07:00
Brian Kassouf
4c75326aad Cleanup path files 2017-04-12 17:35:02 -07:00
Brian Kassouf
03e2bcbc79 Update Type() to return an error 2017-04-12 16:41:06 -07:00
Brian Kassouf
8f75c30311 Update help text and comments 2017-04-11 11:50:34 -07:00
Brian Kassouf
da4d9a8b4f Remove unnecessary abstraction 2017-04-10 18:38:34 -07:00
Brian Kassouf
de36d61e5a Mlock the plugin process 2017-04-10 17:12:52 -07:00
Brian Kassouf
f54c4de98a Add a flag to tell plugins to verify the connection was successful 2017-04-10 15:36:59 -07:00
Brian Kassouf
64efc505c8 Update plugin test 2017-04-10 14:12:28 -07:00
Brian Kassouf
73f66f89cd Update the interface for plugins removing functions for creating creds 2017-04-10 12:24:16 -07:00
Brian Kassouf
3c1c388589 Update backend tests 2017-04-10 10:35:16 -07:00
Brian Kassouf
9ae5a2aede Add backend test 2017-04-07 15:50:03 -07:00
Brian Kassouf
8e77bd98d8 Move plugin code into sub directory 2017-04-06 12:20:10 -07:00
Brian Kassouf
8a2e29c607 Refactor to use builtin plugins from an external repo 2017-04-05 16:20:31 -07:00
Calvin Leung Huang
73a2cdf6a5 Do not mark conn as initialized until the end (#2567) 2017-04-04 14:26:59 -07:00
Brian Kassouf
df944f2d92 Don't return strings, always structs 2017-04-04 11:33:58 -07:00
Calvin Leung Huang
8e3cb50bfc Database refactor invalidate (#2566)
* WIP on invalidate function

* cassandraConnectionProducer has Close()

* Delete database from connections map on successful db.Close()

* Move clear connection into its own func

* Use const for database config path
2017-04-04 11:32:42 -07:00
Brian Kassouf
1faa5fc020 On change of configuration rotate the database type 2017-04-03 18:30:38 -07:00
Brian Kassouf
b54e1cd295 Merge branch 'database-refactor' of github.com:hashicorp/vault into database-refactor 2017-04-03 17:52:41 -07:00
Brian Kassouf
ac519abecf Plugin catalog 2017-04-03 17:52:29 -07:00
Calvin Leung Huang
2b08521ab6 Database refactor mssql (#2562)
* WIP on mssql secret backend refactor

* Add RevokeUser test, and use sqlserver driver internally

* Remove debug statements

* Fix code comment
2017-04-03 09:59:30 -07:00
Brian Kassouf
1d3d3b7803 fix for plugin commands that have more than one paramater 2017-03-28 14:37:57 -07:00