1256 Commits

Author SHA1 Message Date
Mike Jarmy
3d02fb4b86
Test pre-1.4 seal migration (#9085)
* enable seal wrap in all seal migration tests

* move adjustForSealMigration to vault package

* fix adjustForSealMigration

* begin working on new seal migration test

* create shamir seal migration test

* refactor testhelpers

* add VerifyRaftConfiguration to testhelpers

* stub out TestTransit

* Revert "refactor testhelpers"

This reverts commit 39593defd0d4c6fd79aedfd37df6298391abb9db.

* get shamir test working again

* stub out transit join

* work on transit join

* Revert "move resuable storage test to avoid creating import cycle"

This reverts commit b3ff2317381a5af12a53117f87d1c6fbb093af6b.

* remove debug code

* initTransit now works with raft join

* runTransit works with inmem

* work on runTransit with raft

* runTransit works with raft

* get rid of dis-used test

* cleanup tests

* TestSealMigration_TransitToShamir_Pre14

* TestSealMigration_ShamirToTransit_Pre14

* split for pre-1.4 testing

* add simple tests for transit and shamir

* fix typo in test suite

* debug wrapper type

* test debug

* test-debug

* refactor core migration

* Revert "refactor core migration"

This reverts commit a776452d32a9dca7a51e3df4a76b9234d8c0c7ce.

* begin refactor of adjustForSealMigration

* fix bug in adjustForSealMigration

* clean up tests

* clean up core refactoring

* fix bug in shamir->transit migration

* remove unnecessary lock from setSealsForMigration()

* rename sealmigration test package

* use ephemeral ports below 30000

* simplify use of numTestCores
2020-06-11 15:07:59 -04:00
Yoan Blanc
3a1a92bb0e
operator: init -status to return JSON (#8773) 2020-06-08 14:35:39 -04:00
Jason O'Donnell
35dda4910f
agent/raft: fix typo in help strings (#9114) 2020-06-02 10:17:08 -04:00
ncabatoff
60c7e22ddf
Extend agent template tests to also validate that updated templates get re-rendered. (#9097) 2020-05-29 13:36:59 -04:00
Thomas L. Kula
f31092e7ad
Allow auto_auth with templates without specifying a sink (#8812)
For situations where you want the Vault agent to handle one or more templates but do not require the acquired credentials elsewhere.

Modify the logic in SyncServer so that if there are no sinks, ignore any new credentials. Since SyncServer is responsible for shutting down the agent, make sure it still properly shuts down in this new situation.

Solves #7988
2020-05-26 13:52:14 -04:00
Alexander Bezobchuk
f425d5ba37
Merge PR #9078: Add go version to server message output 2020-05-26 12:28:51 -04:00
Alexander Bezobchuk
2e1f75a26e
Merge PR #9060: Support reading Raft TLS flags from file 2020-05-23 11:09:55 -04:00
Josh Black
af5338b485
Add a new "vault monitor" command (#8477)
Add a new "vault monitor" command

Co-authored-by: ncabatoff <ncabatoff@hashicorp.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
2020-05-21 13:07:50 -07:00
Jason O'Donnell
dbaa9ec65f
agent/template: update consul-template dep (#9044) 2020-05-20 13:03:33 -04:00
Josh Black
f0c8b4c7a9
Fix JSON encoding adding newlines. (#8928)
Fix JSON encoding adding newlines.

This manifested itself when encoding config values, which all map to
strings. An extra new line would get added by json.Encode, which caused
other things to break with confusing error messagges. Switching to
json.Marshal seems to solve the problem.
2020-05-19 16:13:05 -07:00
ncabatoff
e52f34772a
Don't include username or password of proxy env vars when logging them. (#9022) 2020-05-19 10:07:46 -04:00
Brian Kassouf
b8799b3358
storage/raft: Add committed and applied indexes to the status output (#9011)
* storage/raft: Add committed and applied indexes to the status output

* Update api vendor

* changelog++

* Update http/sys_leader.go

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-05-18 16:07:27 -07:00
Calvin Leung Huang
5894f65138
Refactor service registration (#8976)
* serivceregistration: refactor service registration logic to run later

* move state check to the internal func

* sr/kubernetes: update setInitialStateInternal godoc

* sr/kubernetes: remove return in setInitialState

* core/test: fix mockServiceRegistration

* address review feedback
2020-05-15 11:06:58 -07:00
Jeff Mitchell
91b09c09b5
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Mark Gritter
d5b1d5d3a5
Metrics wrapper that adds the cluster name as a label. (#8961) 2020-05-12 21:00:59 -05:00
Vladimir Dimitrov
52cadd32d9
Added plugin reload function to api (#8777)
* Added plugin reload function to api

* Apply suggestions from code review

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* go mod vendor

* addressing comments

* addressing comments

* add docs

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-05-04 11:14:23 -07:00
ncabatoff
124572bf80
Ensure that the .vault-token file writen by vault login always has the correct permissions and ownership. (#8867) 2020-04-27 19:55:13 -04:00
Calvin Leung Huang
e23cacb33a
seal/gcpkms: fix panic that occurs when the seal config map is nil (#8840) 2020-04-24 13:31:03 -07:00
Calvin Leung Huang
6c9d585fe2
http: ensure return after writing response by respondError (#8796) 2020-04-21 15:30:36 -07:00
Calvin Leung Huang
ed5e1218b9
raft: check for nil on concrete type in SetupCluster (#8784)
* raft: check for nil on concrete type in SetupCluster

* raft: move check to its own func

* raft: func cleanup

* raft: disallow disable_clustering = true when raft storage is used

* docs: update disable_clustering to mention new behavior
2020-04-21 13:45:07 -07:00
Mike Jarmy
1530c05430
Enable transit->shamir seal migration in Enterprise (#8737)
* Enable transit->shamir seal migration in Enterprise

* prove that we can stop the transit cluster after migration is complete
2020-04-14 10:36:46 -04:00
Vishal Nayak
bb4b2b024e
Fix segfault when seal block is removed (#8517) 2020-04-13 22:54:21 -04:00
Becca Petrin
1a16c61ed0
minor docs updates (#8725) 2020-04-13 10:03:05 -07:00
Vishal Nayak
4ee817b474
Support unwrapping tokens that does not contain data (#8714)
* Support unwrapping tokens that does not contain data

* s/token/secret
2020-04-09 14:09:23 -04:00
Calvin Leung Huang
78189b801d
raft: disallow ha_storage stanza when raft storage is used (#8707) 2020-04-08 13:07:42 -07:00
Calvin Leung Huang
ba1871d48e
test: port test changes to oss (#8694) 2020-04-07 11:26:51 -07:00
Mike Jarmy
bf7bcc1ad8
Expand seal migration tests (#8658)
* rename some seal migration tests

* add comments to seal migration tests

* add comments to seal migration tests

* rename sealhelper

* creat TestSealMigration_ShamirToTestSeal

* fix typo in test suite

* create TestSealMigration_TransitToTestSeal
2020-04-01 13:21:32 -04:00
Becca Petrin
7cb59dc326
allow delete-version-after to be reset to 0 (#8635) 2020-03-30 16:19:44 -07:00
Vishal Nayak
3dd3b20619
Raft cli docs (#8548)
* Raft cli docs

* s/raft/Raft

* s/raft/Raft

* Update website/pages/docs/commands/operator/raft.mdx

Co-Authored-By: Meggie <m.ladlow@gmail.com>

* Mention that shamir seal requires unseal keys

Co-authored-by: Meggie <m.ladlow@gmail.com>
2020-03-17 09:46:44 -04:00
Tommy Murphy
45f488bb4a
stackdriver: metric label extraction (#8073)
* stackdriver: use label extraction and add debug config

* go.mod: update go-metrics-stackdriver

* vendor go-metrics-stackdriver
2020-03-13 07:58:45 +01:00
Jeff Mitchell
c53e0c8ebf Remove misleading comment 2020-03-11 09:04:45 -04:00
Brian Kassouf
ccb0dd7c05
Rename raft configuration command to list-peers and make output easier to read (#8484)
* Make the output of raft configuration easier to read

* Rename raft configuration sub command to list-peers

* Update command/operator_raft_listpeers.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-03-06 14:40:50 -08:00
Brian Kassouf
05a754241d
Fix migrating away from raft storage (#8485) 2020-03-06 09:35:17 -08:00
ncabatoff
1ff460597f
Use public seal helper to create a transit server to handle autounseal. (#8395) 2020-02-20 10:05:23 -05:00
Jeff Mitchell
468ed7043a Move kv-builder to internalshared 2020-02-19 11:13:13 -05:00
Jeff Mitchell
5350e670d2 Bump API/SDK and adapt to move from SDK stuff 2020-02-15 14:58:05 -05:00
Gerardo Di Giacomo
0e8c6c2171
enabling TLS 1.3 support for TCP listeners (#8305)
* adding support for TLS 1.3 for TCP listeners

* removed test as CI uses go 1.12

* removed Cassandra support, added deprecation notice

* re-added TestTCPListener_tls13
2020-02-15 11:40:18 -08:00
Jason O'Donnell
7d550e24da
Add OpenLDAP Secret Plugin (#8360)
* Add openldap secret plugin

* go mod vendor

* Revert to go-ldap 3.1.3

* go mod vendor
2020-02-15 13:21:07 -05:00
Calvin Leung Huang
c73a776ba2
command/generate-root: support returning json format on decode and generate otp (#7870) 2020-02-14 18:26:49 -08:00
Joe Dollard
e1c0a1c959
provide vault server flag to exit on core shutdown (#7561)
* provide vault server flag to exit on core shutdown

* Update command/server.go

Co-Authored-By: Jeff Mitchell <jeffrey.mitchell@gmail.com>

Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-14 18:07:31 -08:00
Jeff Mitchell
4bf92f2617
Switch output loggers in hclog instead of going through gatedlogger (#8228)
After flushing we no longer have to take a mutex hit for every log line.
2020-02-14 17:50:47 -08:00
Brian Kassouf
317bf94c7a
Port some changes back to OSS (#8359) 2020-02-14 16:39:13 -08:00
Clint
f0e4c56ed0
Docs enforce autoauth token (#8270)
* rename UseAutoAuthForce to ForceAutoAuth, because I think it reads better

* Document 'ForceAuthAuthToken' option for Agent Cache

* Update website/pages/docs/agent/caching/index.mdx

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* Add additional tests around use_auto_auth=force and add documentation

* remove note, it's no longer correct

Co-authored-by: Jim Kalafut <jim@kalafut.net>
2020-02-14 15:48:12 -06:00
Clint
e87f01845c
Guard against using Raft as a seperate HA Storage (#8239)
* Guard against using Raft as a seperate HA Storage

* Document that Raft cannot be used as a seperate ha_storage backend at this time

* remove duplicate imports from updating with master
2020-02-14 14:25:53 -06:00
Vishal Nayak
9f980ade31
Seal migration with Raft (#8103)
* Seal migration after unsealing

* Refactor migration fields migrationInformation in core

* Perform seal migration as part of postUnseal

* Remove the sleep logic

* Use proper seal in the unseal function

* Fix migration from Auto to Shamir

* Fix the recovery config missing issue

* Address the non-ha migration case

* Fix the multi cluster case

* Avoid re-running seal migration

* Run the post migration code in new leaders

* Fix the issue of wrong recovery being set

* Address review feedback

* Add more complete testing coverage for seal migrations.   (#8247)

* Add more complete testing coverage for seal migrations.  Also remove VAULT_ACC gate from some tests that just depend on docker, cleanup dangling recovery config in storage after migration, and fix a call in adjustCoreForSealMigration that seems broken.

* Fix the issue of wrong recovery key being set

* Adapt tests to work with multiple cores.

* Add missing line to disable raft join.

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* Fix all known issues

* Remove warning

* Review feedback.

* Revert my previous change that broke raft tests.  We'll need to come back and at least comment
this once we better understand why it's needed.

* Don't allow migration between same types for now

* Disable auto to auto tests for now since it uses migration between same types which is not allowed

* Update vault/core.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Add migration logs

* Address review comments

* Add the recovery config check back

* Skip a few steps if migration is already done

* Return from waitForLeadership if migration fails

Co-authored-by: ncabatoff <nick.cabatoff@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-13 16:27:31 -05:00
Becca Petrin
0937a58ad7
Add Kubernetes service registration (#8249) 2020-02-13 09:56:29 -08:00
Jeff Malnick
942dd1ef9e
Add redshift database plugin (#8299)
* feat: add redshift database plugin

* build: update vendored libraries

* docs: add reference doc for redshift variant of the database secrets engine

* feat: set middlewear type name for better metrics naming (#8346)

Co-authored-by: Becca Petrin <beccapetrin@gmail.com>
2020-02-13 09:42:30 -08:00
Jeff Mitchell
6ca61fa265 Bump api/sdk and fix imports 2020-02-13 10:41:16 -05:00
ncabatoff
4d7bfb4927
Add support for telemetry option metrics_prefix (#8340)
When set, instead of starting with `vault_`, metrics will start with `${prefix}_`.
2020-02-13 09:00:32 -05:00
Daniel Spangenberg
75461a6522
Fix CLI namespace commands (#8315)
* Fix namespace commands help

* Fix useless prediction for namespace commands

* Add namespace prediction capability
2020-02-09 19:38:39 +01:00