547 Commits

Author SHA1 Message Date
Jeff Mitchell
9c68bf2a20
Allow Default for TimeDurationSecond values to be time.Duration (#6934) 2019-06-20 12:28:15 -04:00
Jim Kalafut
76b551f818
Add new structures for OpenAPI/UI enhancements (#6931) 2019-06-19 16:48:58 -07:00
Clint
35667f93a7
Combined Database Backend: Static Accounts (#6834)
* Add priority queue to sdk

* fix issue of storing pointers and now copy

* update to use copy structure

* Remove file, put Item struct def. into other file

* add link

* clean up docs

* refactor internal data structure to hide heap method implementations. Other cleanup after feedback

* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods

* updates after feedback

* refactoring/renaming

* guard against pushing a nil item

* minor updates after feedback

* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC

* Initial Combined database backend implementation of static accounts and automatic rotation

* vendor updates

* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation

* add lock and setup of rotation queue

* vendor the queue

* rebase on new method signature of queue

* remove mongo tests for now

* update default role sql

* gofmt after rebase

* cleanup after rebasing to remove checks for ErrNotFound error

* rebase cdcr-priority-queue

* vendor dependencies with 'go mod vendor'

* website database docs for Static Role support

* document the rotate-role API endpoint

* postgres specific static role docs

* use constants for paths

* updates from review

* remove dead code

* combine and clarify error message for older plugins

* Update builtin/logical/database/backend.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups from feedback

* code and comment cleanups

* move db.RLock higher to protect db.GenerateCredentials call

* Return output with WALID if we failed to delete the WAL

* Update builtin/logical/database/path_creds_create.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* updates after running 'make fmt'

* update after running 'make proto'

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update comment and remove and rearrange some dead code

* Update website/source/api/secret/databases/index.html.md

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups after review

* Update sdk/database/dbplugin/grpc_transport.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* code cleanup after feedback

* remove PasswordLastSet; it's not used

* document GenerateCredentials and SetCredentials

* Update builtin/logical/database/path_rotate_credentials.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* wrap pop and popbykey in backend methods to protect against nil cred rotation queue

* use strings.HasPrefix instead of direct equality check for path

* Forgot to commit this

* updates after feedback

* re-purpose an outdated test to now check that static and dynamic roles cannot share a name

* check for unique name across dynamic and static roles

* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials

* remove commented out code

* refactor to have loadstaticwals filter out wals for roles that no longer exist

* return error if nil input given

* add nil check for input into setStaticAccount

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* add constant for queue tick time in seconds, used for comparrison in updates

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* code cleanup after review

* remove misplaced code comment

* remove commented out code

* create a queue in the Factory method, even if it's never used

* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type

* document new method

* move rotation things into a specific file

* rename test file and consolidate some static account tests

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update code comments, method names, and move more methods into rotation.go

* update comments to be capitalized

* remove the item from the queue before we try to destroy it

* findStaticWAL returns an error

* use lowercase keys when encoding WAL entries

* small cleanups

* remove vestigial static account check

* remove redundant DeleteWAL call in populate queue

* if we error on loading role, push back to queue with 10 second backoff

* poll in initqueue to make sure the backend is setup and can write/delete data

* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false

* add code comments on read-only loop

* code comment updates

* re-push if error returned from find static wal

* add locksutil and acquire locks when pop'ing from the queue

* grab exclusive locks for updating static roles

* Add SetCredentials and GenerateCredentials stubs to mockPlugin

* add a switch in initQueue to listen for cancelation

* remove guard on zero time, it should have no affect

* create a new context in Factory to pass on and use for closing the backend queue

* restore master copy of vendor dir
2019-06-19 14:45:39 -05:00
Jeff Mitchell
f990eb1626 Remove gogo proto from where it snuck in 2019-06-18 14:45:42 -04:00
Jeff Mitchell
0d8de0b548 Return integers, not floats, when reading token params 2019-06-18 12:22:12 -04:00
Jeff Mitchell
1ce8868aaf Update go-plugin dep 2019-06-18 10:53:38 -04:00
Jeff Mitchell
263b96ef4e
Tokenhelper v2 (#6662)
This provides an sdk util for common token fields and parsing and plumbs it into token store roles.
2019-06-14 10:17:04 -04:00
Calvin Leung Huang
485b4d173e
core: add ForwardGenericRequest to StaticSystemView to satisfy ExtendedSystemView (#6867) 2019-06-11 14:07:04 -07:00
Calvin Leung Huang
1ac329a9c7
core: add generic request forwarding bits to oss (#6866) 2019-06-11 13:13:03 -07:00
Jeff Mitchell
ab520a9119 Remove data races around error/latency injector rand objects 2019-06-05 01:37:40 -04:00
Lexman
4ed616dacb transit cache is an Interface implemented by wrapped versions of sync… (#6225)
* transit cache is an Interface implemented by wrapped versions of syncmap and golang-lru

* transit cache is an Interface implemented by wrapped versions of syncmap and golang-lru

* changed some import paths to point to sdk

* Apply suggestions from code review

Co-Authored-By: Lexman42 <Lexman42@users.noreply.github.com>

* updates docs with information on transit/cache-config endpoint

* updates vendored files

* fixes policy tests to actually use a cache where expected and renames the struct and storage path used for cache configurations to be more generic

* updates document links

* fixed a typo in a documentation link

* changes cache_size to just size for the cache-config endpoint
2019-06-04 15:40:56 -07:00
tonyd
ed57002711 Allow logical backends access to the disabled state of an entity (#6791)
* Allow logical backends access to the disabled state of an entity via SystemView.EntityInfo().

* Add generated file in vendor directory.
2019-05-28 16:31:50 -05:00
ncabatoff
6c836bcd9b
Allow plugins to submit audit requests/responses via extended SystemView (#6777)
Move audit.LogInput to sdk/logical.  Allow the Data values in audited
logical.Request and Response to implement OptMarshaler, in which case
we delegate hashing/serializing responsibility to them.  Add new
ClientCertificateSerialNumber audit request field.

SystemView can now be cast to ExtendedSystemView to expose the Auditor
interface, which allows submitting requests and responses to the audit
broker.
2019-05-22 18:52:53 -04:00
Clint
593c065216
Add Priority Queue library to sdk (#6664)
* Add priority queue to sdk

* fix issue of storing pointers and now copy

* update to use copy structure

* Remove file, put Item struct def. into other file

* add link

* clean up docs

* refactor internal data structure to hide heap method implementations. Other cleanup after feedback

* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods

* updates after feedback

* refactoring/renaming

* guard against pushing a nil item

* minor updates after feedback

* Add read lock to the Len() method and move the interface check into the test file

* fix a deadlock

* make the mutex a RWMutex, and make it private again

* nil check itemRaw before trying to type cast it
2019-05-16 11:15:57 -05:00
Patrick Hayes
b9f054d8ab Maximum typo in Vault UI (#6743) 2019-05-16 08:44:34 +02:00
ncabatoff
7c193f53c8
Copy LogInput from audit package, add OptMarshaler interface (#6735)
Adds Type field and makes Request and Response interface{}.  

Add OptMarshaler interface for doing JSON marshaling with options.
2019-05-15 09:05:30 -04:00
Jeff Mitchell
eebd582f7f
Update grpc and protos (#6725)
gRPC updated to 1.20.1 which fixes a couple of important bugs.

Updates protos as well.
2019-05-13 12:09:30 -04:00
Jeff Mitchell
c8d6ad43ca
Fix some regressions (#6723)
Multierror is not nil unless you return ErrorOrNil, so this was causing
non-nil errors to be returned when not expected.

Also we need to ensure we only call handleWALRollback if a function
exists.
2019-05-13 11:04:06 -04:00
Clint
43ecb08dfd
Update handleRollback to run both PeriodicFunc and handleWALRollback (#6717) 2019-05-10 14:11:42 -05:00
ncabatoff
1d13290b36
Refactor cert util (#6676)
Break dataBundle into two pieces: inputBundle, which contains data that
is specific to the pki backend, and creationBundle, which is a more
generic bundle of validated inputs given to certificate creation/signing routines.

Move functions that only take creationBundle to certutil and make them public.
2019-05-09 11:43:11 -04:00
Mark Gritter
8fc8af3199
Merge branch 'master' into fix-ou-ordering 2019-05-06 14:41:44 -05:00
Jim Kalafut
8b6ea178b2
Don't show TypeHeader fields as being sent as headers in OpenAPI (#6679)
Fixes #6671
2019-05-03 15:12:24 -07:00
Mark Gritter
f03bcf337a
Merge branch 'master' into fix-ou-ordering 2019-05-02 18:59:17 -05:00
mgritter
6c43d516f7 gofmt fixes. 2019-05-02 16:29:41 -07:00
Jim Kalafut
dc4f938e7c
Apply suggestions from code review
Co-Authored-By: mgritter <mgritter@gmail.com>
2019-05-02 18:02:15 -05:00
mgritter
bb425118da Ensure OU entries are not reordered. 2019-05-02 14:31:29 -07:00
Jim Kalafut
7f5e7818fd
Update SDK vendor (#6669) 2019-05-01 18:48:12 -07:00
Jim Kalafut
c9ac721887
Fix Okta auth to allow group names containing slashes (#6665)
This PR also adds CollectKeysPrefix which allows a more memory efficient
key scan for those cases where the result is immediately filtered by
prefix.
2019-05-01 14:56:18 -07:00
Jeff Mitchell
f7bb5a2e56
Port over some SP v2 bits (#6516)
* Port over some SP v2 bits

Specifically:

* Add too-large handling to Physical (Consul only for now)
* Contextify some identity funcs
* Update SP protos

* Add size limiting to inmem storage
2019-05-01 13:47:41 -04:00
Jeff Mitchell
e0c71cb282 Update deep to pull in default full-level-diff behavior 2019-04-19 19:52:03 -04:00
Jeff Mitchell
6ec1bcfb20 Merge branch '1.1.2' into master-oss 2019-04-18 18:49:49 -04:00
Brian Kassouf
7ec1fe75a9
Move cluster logic out of vault package (#6601)
* Move cluster logic out of vault package

* Dedup heartbeat and fix tests

* Fix test
2019-04-17 13:50:31 -07:00
Jeff Mitchell
71d635293d Move physical/file to sdk 2019-04-15 14:51:33 -04:00
Jeff Mitchell
8060a7b505 More rearranging of API and SDK 2019-04-15 13:38:08 -04:00
Jeff Mitchell
7bff97d829 Move some things around in api/sdk 2019-04-15 12:14:20 -04:00
Jeff Mitchell
05bcacee74 Migrate database plugin methods to sdk 2019-04-15 11:36:10 -04:00
Jeff Mitchell
278bdd1f4e
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell
1d5aeccb60 Update protobufs, sdk changes 2019-04-12 23:13:14 -04:00
Jeff Mitchell
5103b7426a Update physical_util.go with entry.go 2019-04-12 22:56:04 -04:00
Jeff Mitchell
ecf4f58fae Remove mock-plugin binary causing circular dep 2019-04-12 22:22:07 -04:00
Jeff Mitchell
aa67ad63dd Fix more tests 2019-04-12 22:14:50 -04:00
Jeff Mitchell
f95571a361 Move ldaputil and tlsutil over to sdk 2019-04-12 18:26:54 -04:00
Jeff Mitchell
6388efd4f0 Move useragent to sdk 2019-04-12 18:17:49 -04:00
Jeff Mitchell
a920c10be6 Move password to sdk 2019-04-12 18:12:13 -04:00
Jeff Mitchell
7ae6fb56ff Move policyutil to sdk 2019-04-12 18:08:46 -04:00
Jeff Mitchell
cf03949594 Move cidrutil to sdk 2019-04-12 18:03:59 -04:00
Jeff Mitchell
170521481d
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00