305 Commits

Author SHA1 Message Date
Jeff Mitchell
f7ae903371 Remove RegisterLicense from logical.Backend
It's almost certainly the wrong signature and nothing uses it currently
anyways.
2018-01-18 13:44:29 -05:00
Vishal Nayak
d552102457
SHA2-256 salting for AppID (#3806)
* Use SHA2-256 hash with prefix to upgrade the paths

* test the SHA1 upgrade to SHA256

* Remove hash identifier and the delimiter; use 's' instead

* Added API test to verify the correctness of the fix

* Fix broken test

* remove unneeded test
2018-01-17 19:48:32 -05:00
Brian Kassouf
2a3243546a
Update plugin deps to include context changes (#3765)
* Update plugin deps to include context changes

* Fix tests
2018-01-08 12:26:13 -08:00
Brian Kassouf
78adac0a24
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Calvin Leung Huang
895cffa4cf
Add logic for using Auth.Period when handling auth login/renew requests (#3677)
* Add logic for using Auth.Period when handling auth login/renew requests

* Set auth.TTL if not set in handleLoginRequest

* Always set auth.TTL = te.TTL on handleLoginRequest, check TTL and period against sys values on RenewToken

* Get sysView from le.Path, revert tests

* Add back auth.Policies

* Fix TokenStore tests, add resp warning when capping values

* Use switch for ttl/period check on RenewToken

* Move comments around
2017-12-15 13:30:05 -05:00
Brian Kassouf
a401cc7cb5
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Jeff Mitchell
c77196cea2
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Jeff Mitchell
84f80db4bf
Seal wrap requesting in logical.Request wrapinfo (#3559) 2017-11-09 10:32:49 -05:00
Chris Hoffman
92650358d6
Add TypeKVPairs field type (#3535) 2017-11-07 11:11:49 -05:00
Vishal Nayak
2994b26194
Handle 'not supplied' case for field type TypeNameString (#3546)
* Fix panic if value is not supplied for variables of TypeNameString

* Add tests for 'not supplied' case of all field types
2017-11-07 10:59:57 -05:00
Calvin Leung Huang
ca76bc4f44
Return role info for each role on pathRoleList (#3532)
* Return role info for each role on pathRoleList

* Change roles -> key_info, only return key_type

* Do not initialize result map in parseRole, refactor ListResponseWithInfo

* Add role list test
2017-11-03 17:12:03 -04:00
Vishal Nayak
66642a0935
External identity groups (#3447)
* external identity groups

* add local LDAP groups as well to group aliases

* add group aliases for okta credential backend

* Fix panic in tests

* fix build failure

* remove duplicated struct tag

* add test steps to test out removal of group member during renewals

* Add comment for having a prefix check in router

* fix tests

* s/parent_id/canonical_id

* s/parent/canonical in comments and errors
2017-11-02 16:05:48 -04:00
Jeff Mitchell
47e409778a Make compile 2017-10-23 17:15:56 -04:00
Jeff Mitchell
9cedc9aecc Fix some merge/update bugs 2017-10-23 16:49:46 -04:00
Jeff Mitchell
98168dc791 Sync over 2017-10-23 16:43:07 -04:00
Seth Vargo
e768b73403 Allow more complex errors from plugins (no interface change) (#3444)
* Allow more complex errors from plugins

This enables more complex types to be registered and returned from plugins.

*  Register common error types

This is a slightly less drastic change, which keeps the HTTPCodedError
as an interface.

* Remove replication error from list
2017-10-19 13:29:59 -07:00
Vishal Nayak
6b9ce0c8c5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Seth Vargo
fab7f3bd9a Fix tags on structure (#3442)
The comma breaks mapstructure for this actually.
2017-10-10 10:53:46 -04:00
Brian Kassouf
9689e88561 Allow for response errors to be passed back from a plugin (#3412)
* Allow for response errors to be passed back from a plugin

* Fix issue with checking plugin.BasicError == nil
2017-10-06 14:20:57 -07:00
Vishal Nayak
21d83e1251 plugin version bump (#3423) 2017-10-04 20:02:13 -04:00
Calvin Leung Huang
6c1ba24a7d Fix plugin args output (#3424) 2017-10-04 20:00:53 -04:00
Vishal Nayak
8ae19b2686 Rename persona to alias (#3420)
Merging this will break the build. I'll fix it post merge by updating the vendor libs.
2017-10-04 13:35:05 -04:00
Jeff Mitchell
1c2bd5b9a1 Add PersonaLookahead operation 2017-09-14 20:54:09 -04:00
Brian Kassouf
de7f39e064 Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Brian Kassouf
4b80f4b58e Plugin Version Update (#3275)
* Bump plugin version requirement

* Register time.Duration with gob
2017-09-01 00:01:14 -07:00
Calvin Leung Huang
3b8b68097d Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Calvin Leung Huang
c8388a9eba Add plugin auto-reload capability (#3171)
* Add automatic plugin reload

* Refactor builtin/backend

* Remove plugin reload at the core level

* Refactor plugin tests

* Add auto-reload test case

* Change backend to use sync.RWMutex, fix dangling test plugin processes

* Add a canary to plugin backends to avoid reloading many times (#3174)

* Call setupPluginCatalog before mount-related operations in postUnseal

* Don't create multiple system backends since core only holds a reference (#3176)

to one.
2017-08-15 22:10:32 -04:00
vishalnayak
77d70aff85 Added persona to logical auth 2017-08-15 13:55:58 -04:00
Calvin Leung Huang
e0c84d0f9f Direct plugin logs through vault's logger (#3142)
* Direct plugin logs through vault's logger

* Pass in a logger in testConfig
2017-08-15 10:16:48 -04:00
Calvin Leung Huang
01d1c20e4c Add plugin backend reload capability (#3112)
* Add plugin reload capability on all mounts for a specific plugin type

* Comments cleanup

* Add per-mount plugin backend reload, add tests

* Fix typos

* Remove old comment

* Reuse existing storage view in reloadPluginCommon

* Correctly handle reloading auth plugin backends

* Update path to plugin/backend/reload

* Use multierrors on reloadMatchingPluginMounts, attempt to reload all mounts provided

* Use internal value as check to ensure plugin backend reload

* Remove connection state from request for plugins at the moment

* Minor cleanup

* Refactor tests
2017-08-08 00:18:59 -04:00
Calvin Leung Huang
663185c8b3 Update mock-plugin (#3107) 2017-08-04 11:43:36 -04:00
Jeff Mitchell
0ac531d3f4 Migrate physical backends into separate packages (#3106) 2017-08-03 13:24:27 -04:00
Calvin Leung Huang
15634f3b6e Store original request path in WrapInfo (#3100)
* Store original request path in WrapInfo as CreationPath

* Add wrapping_token_creation_path to CLI output

* Add CreationPath to AuditResponseWrapInfo

* Fix tests

* Add and fix tests, update API docs with new sample responses
2017-08-02 18:28:58 -04:00
Jeff Mitchell
d6fc971726 Add more TypeDurationSecond conversions 2017-08-02 18:12:02 -04:00
Calvin Leung Huang
f80de5b38d Default mock-plugin type to logical.TypeLogical on Factory func 2017-07-31 16:20:52 -04:00
Vishal Nayak
9feddee953 Adding logical/identity.go to OSS (#3054) 2017-07-27 11:56:32 -04:00
Jeff Mitchell
823cef4cbf Make logical.InmemStorage standalone (#3066) 2017-07-26 17:59:14 -04:00
Calvin Leung Huang
c78d077e6e Fix gob register issue when using tls certs on plugins (#3060) 2017-07-26 13:44:07 -04:00
Calvin Leung Huang
ce1e883a4f Do not send storage on HandleRequest and HandleExistenceCheck on plugins 2017-07-25 16:57:26 -04:00
Calvin Leung Huang
2b0f80b981 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Chris Hoffman
fab73e1f7a fixing Validate() for field data on TypeNameString (#3030) 2017-07-17 13:44:47 -07:00
Chris Hoffman
508183a1b3 Add field type TypeNameString (#3028) 2017-07-17 11:39:58 -07:00
Jeff Mitchell
8a88e37f2b Add string slice with single value test function 2017-06-14 14:20:37 -04:00
Jeff Mitchell
e5e6f0fc52 Automatically upgrade values to salted on access if salt exists in PathMap (#2831) 2017-06-07 15:22:08 -04:00
Jeff Mitchell
83ecd0f9ad Allow accessing Warnings directly in Response. (#2806)
A change in copystructure has caused some panics due to the custom copy
function. I'm more nervous about production panics than I am about
keeping some bad code wiping out some existing warnings, so remove the
custom copy function and just allow direct setting of Warnings.
2017-06-05 10:52:43 -04:00
Jeff Mitchell
eb0e7cd0d2 Don't write salts in initialization, look up on demand (#2702) 2017-05-09 17:51:09 -04:00
Jeff Mitchell
5815fc2627 Add salt mutex to app-id (#2690) 2017-05-08 16:15:24 -04:00
Brian Kassouf
55f1f5116a Merge remote-tracking branch 'oss/master' into database-refactor 2017-05-04 10:45:18 -07:00
Brian Kassouf
c825362304 PR comments 2017-05-04 10:41:59 -07:00
Brian Kassouf
4c306bd76e Change MlockDisabled to MlockEnabled 2017-04-24 12:21:49 -07:00