Commit Graph

327 Commits

Author SHA1 Message Date
Brian Kassouf
c0815bd2b0
Add context to the NewSalt function (#4102) 2018-03-08 11:21:11 -08:00
Jeff Mitchell
65bd8dc8b0
Make grpc plugin client use an atomic server value to fix a data race. (#4089)
Also add some coordination to ensure we don't try to clean up the grpc
server before it's created/started
2018-03-07 09:09:37 -05:00
Brian Kassouf
83a7d20b81
logical/framework: add a field data type for parsing integer arrays (#4064)
* logical/framework: add a field data type for parsing integer arrays

* Fix comment

* Add zero value
2018-03-02 15:01:13 -08:00
Brian Kassouf
cfa758c80b
http/logical: Add http GET parameters to the data map (#4012)
* Add get parameters to the data object

* Add test for get params
2018-02-21 14:36:53 -08:00
Brian Kassouf
9c91fd6ea8
http: Add a method for returning a 404 with data (#3994)
* Add a method for returning a 404 with data

* Pass the full resp object through to respond raw

* Add comment

* Refactor so it works across plugin gRPC

* Handle some review comments

* Pass request object instead of request ID
2018-02-21 14:22:21 -08:00
Brian Kassouf
d9be0e0707
plugin/grpc: Add RemoteAddr to the request object (#3997) 2018-02-16 17:41:01 -08:00
Jeff Mitchell
a43a854740
Support other names in SANs (#3889) 2018-02-16 17:19:34 -05:00
Brian Kassouf
e8c5d89ec0
plugin/gRPC: Add more documentation to the proto file (#3988)
* plugin/gRPC: Add more documentation to the proto file

* Fix typo

* Update comments
2018-02-15 14:20:50 -08:00
Jeff Mitchell
5f776f660a Convert logical.InmemStorage to a wrapper around physical/inmem.
The original reason for the split was physical's dependencies, but those
haven't been onerous for a long time. Meanwhile it's a totally separate
implementation so we could be getting faulty results from tests. Get rid
of it and use the unified physical/inmem.
2018-02-12 11:16:16 -05:00
Vishal Nayak
e57aff9e69 Make mount view read only until after mount persist (#3910) 2018-02-09 14:04:25 -05:00
Brian Kassouf
5ccb34744e
plugin/gRPC: fix panic when special paths are not set (#3946)
* plugin/gRPC: fix panic when special paths are not set

* Remove comment
2018-02-08 16:51:26 -08:00
Jeff Mitchell
85c7b528e2
Change grpc's max sent/recv size to a very large value. (#3912) 2018-02-06 13:52:35 -05:00
Jeff Mitchell
5d6c4ae368 Fix compilation of test against new go-plugin 2018-02-06 13:33:29 -05:00
Jeff Mitchell
63efb0e111 Some vet fixes 2018-02-04 20:37:57 -05:00
Jeff Mitchell
3d7d2a01dd
Add a sysview call to determine if a mount is local. (#3899)
This is useful for deciding when to run upgrade logic, e.g. if on a
performance secondary but local it's fine to run.
2018-02-02 18:17:12 -05:00
Brian Kassouf
91dffedc39
plugins/gRPC: fix issues with reserved keywords in response data (#3881)
* plugins/gRPC: fix issues with reserved keywords in response data

* Add the path raw file for mock plugin

* Fix panic when special paths is nil

* Add tests for Listing and raw requests from plugins

* Add json.Number case when decoding the status

* Bump the version required for gRPC defaults

* Fix test for gRPC version check
2018-02-01 14:30:17 -08:00
Vishal Nayak
909b9ff574
Remove logical.Initialize() method (#3848)
* Remove logical.Initialize() method

* More cleanup

* Fix test
2018-01-25 20:19:27 -05:00
Calvin Leung Huang
c3c63313f3
Version protocol switch (#3833)
* Use version to determine plugin protocol to use

* Remove field from ServeOpts

* Fix missing assignment, handle errors

* contraint -> constraint

* Inject the version string from the vault side

* Fix the version check

* Add grpc support check to database plugins

* Default to use grpc unless missing env var or fail on contraint check

* Add GRPCSupport test

* Add greater than test case

* Add go-version dep
2018-01-23 17:29:26 -05:00
Jeff Mitchell
a109e2a11e Sync some bits over 2018-01-22 21:44:49 -05:00
Brian Kassouf
3627fc7c9a
Update data values from byte arrays to strings in proto definition (#3829)
* Update data values from byte arrays to strings in proto definition

* Update comments
2018-01-22 17:56:34 -08:00
Brian Kassouf
8142b42d95 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Brian Kassouf
03f6108822
gRPC Backend Plugins (#3808)
* Add grpc plugins

* Add grpc plugins

* Translate wrap info to/from proto

* Add nil checks

* Fix nil marshaling errors

* Provide logging through the go-plugin logger

* handle errors in the messages

* Update the TLS config so bidirectional connections work

* Add connectivity checks

* Restart plugin and add timeouts where context is not availible

* Add the response wrap data into the grpc system implementation

* Add leaseoptions to pb.Auth

* Add an error translator

* Add tests for translating the proto objects

* Fix rename of function

* Add tracing to plugins for easier debugging

* Handle plugin crashes with the go-plugin context

* Add test for grpcStorage

* Add tests for backend and system

* Bump go-plugin for GRPCBroker

* Remove RegisterLicense

* Add casing translations for new proto messages

* Use doneCtx in grpcClient

* Use doneCtx in grpcClient

* s/shutdown/shut down/
2018-01-18 13:49:20 -08:00
Jeff Mitchell
f7ae903371 Remove RegisterLicense from logical.Backend
It's almost certainly the wrong signature and nothing uses it currently
anyways.
2018-01-18 13:44:29 -05:00
Vishal Nayak
d552102457
SHA2-256 salting for AppID (#3806)
* Use SHA2-256 hash with prefix to upgrade the paths

* test the SHA1 upgrade to SHA256

* Remove hash identifier and the delimiter; use 's' instead

* Added API test to verify the correctness of the fix

* Fix broken test

* remove unneeded test
2018-01-17 19:48:32 -05:00
Brian Kassouf
2a3243546a
Update plugin deps to include context changes (#3765)
* Update plugin deps to include context changes

* Fix tests
2018-01-08 12:26:13 -08:00
Brian Kassouf
78adac0a24
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Calvin Leung Huang
895cffa4cf
Add logic for using Auth.Period when handling auth login/renew requests (#3677)
* Add logic for using Auth.Period when handling auth login/renew requests

* Set auth.TTL if not set in handleLoginRequest

* Always set auth.TTL = te.TTL on handleLoginRequest, check TTL and period against sys values on RenewToken

* Get sysView from le.Path, revert tests

* Add back auth.Policies

* Fix TokenStore tests, add resp warning when capping values

* Use switch for ttl/period check on RenewToken

* Move comments around
2017-12-15 13:30:05 -05:00
Brian Kassouf
a401cc7cb5
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Jeff Mitchell
c77196cea2
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Jeff Mitchell
84f80db4bf
Seal wrap requesting in logical.Request wrapinfo (#3559) 2017-11-09 10:32:49 -05:00
Chris Hoffman
92650358d6
Add TypeKVPairs field type (#3535) 2017-11-07 11:11:49 -05:00
Vishal Nayak
2994b26194
Handle 'not supplied' case for field type TypeNameString (#3546)
* Fix panic if value is not supplied for variables of TypeNameString

* Add tests for 'not supplied' case of all field types
2017-11-07 10:59:57 -05:00
Calvin Leung Huang
ca76bc4f44
Return role info for each role on pathRoleList (#3532)
* Return role info for each role on pathRoleList

* Change roles -> key_info, only return key_type

* Do not initialize result map in parseRole, refactor ListResponseWithInfo

* Add role list test
2017-11-03 17:12:03 -04:00
Vishal Nayak
66642a0935
External identity groups (#3447)
* external identity groups

* add local LDAP groups as well to group aliases

* add group aliases for okta credential backend

* Fix panic in tests

* fix build failure

* remove duplicated struct tag

* add test steps to test out removal of group member during renewals

* Add comment for having a prefix check in router

* fix tests

* s/parent_id/canonical_id

* s/parent/canonical in comments and errors
2017-11-02 16:05:48 -04:00
Jeff Mitchell
47e409778a Make compile 2017-10-23 17:15:56 -04:00
Jeff Mitchell
9cedc9aecc Fix some merge/update bugs 2017-10-23 16:49:46 -04:00
Jeff Mitchell
98168dc791 Sync over 2017-10-23 16:43:07 -04:00
Seth Vargo
e768b73403 Allow more complex errors from plugins (no interface change) (#3444)
* Allow more complex errors from plugins

This enables more complex types to be registered and returned from plugins.

*  Register common error types

This is a slightly less drastic change, which keeps the HTTPCodedError
as an interface.

* Remove replication error from list
2017-10-19 13:29:59 -07:00
Vishal Nayak
6b9ce0c8c5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Seth Vargo
fab7f3bd9a Fix tags on structure (#3442)
The comma breaks mapstructure for this actually.
2017-10-10 10:53:46 -04:00
Brian Kassouf
9689e88561 Allow for response errors to be passed back from a plugin (#3412)
* Allow for response errors to be passed back from a plugin

* Fix issue with checking plugin.BasicError == nil
2017-10-06 14:20:57 -07:00
Vishal Nayak
21d83e1251 plugin version bump (#3423) 2017-10-04 20:02:13 -04:00
Calvin Leung Huang
6c1ba24a7d Fix plugin args output (#3424) 2017-10-04 20:00:53 -04:00
Vishal Nayak
8ae19b2686 Rename persona to alias (#3420)
Merging this will break the build. I'll fix it post merge by updating the vendor libs.
2017-10-04 13:35:05 -04:00
Jeff Mitchell
1c2bd5b9a1 Add PersonaLookahead operation 2017-09-14 20:54:09 -04:00
Brian Kassouf
de7f39e064 Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Brian Kassouf
4b80f4b58e Plugin Version Update (#3275)
* Bump plugin version requirement

* Register time.Duration with gob
2017-09-01 00:01:14 -07:00
Calvin Leung Huang
3b8b68097d Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Calvin Leung Huang
c8388a9eba Add plugin auto-reload capability (#3171)
* Add automatic plugin reload

* Refactor builtin/backend

* Remove plugin reload at the core level

* Refactor plugin tests

* Add auto-reload test case

* Change backend to use sync.RWMutex, fix dangling test plugin processes

* Add a canary to plugin backends to avoid reloading many times (#3174)

* Call setupPluginCatalog before mount-related operations in postUnseal

* Don't create multiple system backends since core only holds a reference (#3176)

to one.
2017-08-15 22:10:32 -04:00
vishalnayak
77d70aff85 Added persona to logical auth 2017-08-15 13:55:58 -04:00