Commit Graph

162 Commits

Author SHA1 Message Date
Hridoy Roy
0767980550
Port: Telemetry For Lease Expiration Times (#10375)
* port lease metrics

* go mod vendor

* caught a bug
2020-11-13 10:26:58 -08:00
Brian Kassouf
3bc7d15e6b
Backport some OSS changes (#10267)
* Backport some OSS changes

* go mod vendor
2020-10-29 16:47:34 -07:00
Brian Kassouf
f5be0716db
Revert "Migrate internalshared out (#9727)" (#10141)
This reverts commit ee6391b691.
2020-10-13 16:38:21 -07:00
Jeff Mitchell
ee6391b691
Migrate internalshared out (#9727)
* Migrate internalshared out

* fix merge issue

* fix merge issue

* go mod vendor

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2020-10-12 11:56:24 -07:00
ncabatoff
2b3aef242b
Run CI tests in docker instead of a machine. (#8948) 2020-09-15 10:01:26 -04:00
Rodrigo D. L
3eb79c5209
adding new config flag disable_sentinel_trace (#9696) 2020-08-10 06:23:44 -04:00
ncabatoff
9a3744f502
Fix parsing of seal stanzas that have an array for purpose (#9589)
Hexadecimal integers will be converted to decimal, which is unfortunate but shouldn't have any negative effects other than perhaps confusion in the `vault debug` output.
2020-07-27 16:28:52 -04:00
ncabatoff
34144e38ee
Fix 1.5 regression that meant non-string values in the seal stanza would fail config parsing, preventing startup. (#9555) 2020-07-23 13:53:00 -04:00
Brian Kassouf
da472e53d1
Fix UI in dev mode 2020-07-02 15:18:57 -07:00
Scott Miller
2648713fcd
Add new Telemetry config options (#9238)
* Add new Telemetry config options

Add cluster_name, maximum_gauge_cardinality, and usage_gauge_period
configuration options to the config stanza.

Update unit tests.

Document.

Co-authored-by: Mark Gritter <mgritter@hashicorp.com>
2020-06-17 10:07:33 -05:00
Josh Black
f0c8b4c7a9
Fix JSON encoding adding newlines. (#8928)
Fix JSON encoding adding newlines.

This manifested itself when encoding config values, which all map to
strings. An extra new line would get added by json.Encode, which caused
other things to break with confusing error messagges. Switching to
json.Marshal seems to solve the problem.
2020-05-19 16:13:05 -07:00
Jeff Mitchell
91b09c09b5
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Calvin Leung Huang
e23cacb33a
seal/gcpkms: fix panic that occurs when the seal config map is nil (#8840) 2020-04-24 13:31:03 -07:00
Tommy Murphy
45f488bb4a
stackdriver: metric label extraction (#8073)
* stackdriver: use label extraction and add debug config

* go.mod: update go-metrics-stackdriver

* vendor go-metrics-stackdriver
2020-03-13 07:58:45 +01:00
Jeff Mitchell
c53e0c8ebf Remove misleading comment 2020-03-11 09:04:45 -04:00
Jeff Mitchell
5350e670d2 Bump API/SDK and adapt to move from SDK stuff 2020-02-15 14:58:05 -05:00
Gerardo Di Giacomo
0e8c6c2171
enabling TLS 1.3 support for TCP listeners (#8305)
* adding support for TLS 1.3 for TCP listeners

* removed test as CI uses go 1.12

* removed Cassandra support, added deprecation notice

* re-added TestTCPListener_tls13
2020-02-15 11:40:18 -08:00
Brian Kassouf
317bf94c7a
Port some changes back to OSS (#8359) 2020-02-14 16:39:13 -08:00
Vishal Nayak
9f980ade31
Seal migration with Raft (#8103)
* Seal migration after unsealing

* Refactor migration fields migrationInformation in core

* Perform seal migration as part of postUnseal

* Remove the sleep logic

* Use proper seal in the unseal function

* Fix migration from Auto to Shamir

* Fix the recovery config missing issue

* Address the non-ha migration case

* Fix the multi cluster case

* Avoid re-running seal migration

* Run the post migration code in new leaders

* Fix the issue of wrong recovery being set

* Address review feedback

* Add more complete testing coverage for seal migrations.   (#8247)

* Add more complete testing coverage for seal migrations.  Also remove VAULT_ACC gate from some tests that just depend on docker, cleanup dangling recovery config in storage after migration, and fix a call in adjustCoreForSealMigration that seems broken.

* Fix the issue of wrong recovery key being set

* Adapt tests to work with multiple cores.

* Add missing line to disable raft join.

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* Fix all known issues

* Remove warning

* Review feedback.

* Revert my previous change that broke raft tests.  We'll need to come back and at least comment
this once we better understand why it's needed.

* Don't allow migration between same types for now

* Disable auto to auto tests for now since it uses migration between same types which is not allowed

* Update vault/core.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Add migration logs

* Address review comments

* Add the recovery config check back

* Skip a few steps if migration is already done

* Return from waitForLeadership if migration fails

Co-authored-by: ncabatoff <nick.cabatoff@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-13 16:27:31 -05:00
Jeff Mitchell
6ca61fa265 Bump api/sdk and fix imports 2020-02-13 10:41:16 -05:00
ncabatoff
4d7bfb4927
Add support for telemetry option metrics_prefix (#8340)
When set, instead of starting with `vault_`, metrics will start with `${prefix}_`.
2020-02-13 09:00:32 -05:00
Jim Kalafut
060775d9e9
Run goimports (#8251) 2020-01-27 21:11:00 -08:00
Jeff Mitchell
c9de89d87b
Pull wrapping creation to a var (#8137)
* Pull a func out to a var

* Funcouttovarextension (#8153)

* Update test

Co-authored-by: Lexman <Lexman42@users.noreply.github.com>
2020-01-16 13:10:30 -05:00
Vishal Nayak
1a2ec77a7a Raft retry join (#7856)
* Raft retry join

* update

* Make retry join work with shamir seal

* Return upon context completion

* Update vault/raft.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Address some review comments

* send leader information slice as a parameter

* Make retry join work properly with Shamir case. This commit has a blocking issue

* Fix join goroutine exiting before the job is done

* Polishing changes

* Don't return after a successful join during unseal

* Added config parsing test

* Add test and fix bugs

* minor changes

* Address review comments

* Fix build error

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-01-13 17:02:16 -08:00
Jeff Mitchell
157e805b97
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00
Mike Jarmy
df01a4307d
Introduce optional service_registration stanza (#7887)
* move ServiceDiscovery into methods

* add ServiceDiscoveryFactory

* add serviceDiscovery field to vault.Core

* refactor ConsulServiceDiscovery into separate struct

* cleanup

* revert accidental change to go.mod

* cleanup

* get rid of un-needed struct tags in vault.CoreConfig

* add service_discovery parser

* add ServiceDiscovery to config

* cleanup

* cleanup

* add test for ConfigServiceDiscovery to Core

* unit testing for config service_discovery stanza

* cleanup

* get rid of un-needed redirect_addr stuff in service_discovery stanza

* improve test suite

* cleanup

* clean up test a bit

* create docs for service_discovery

* check if service_discovery is configured, but storage does not support HA

* tinker with test

* tinker with test

* tweak docs

* move ServiceDiscovery into its own package

* tweak a variable name

* fix comment

* rename service_discovery to service_registration

* tweak service_registration config

* Revert "tweak service_registration config"

This reverts commit 5509920a8a.

* simplify naming

* refactor into ./serviceregistration/consul
2019-12-06 09:46:39 -05:00
Michel Vocks
f0ff4c9e4f
Add enable_hostname_label option to telementry stanza (#7902) 2019-12-02 09:26:46 +01:00
Jim Kalafut
cb178b7e4f
Run go fmt (#7823) 2019-11-07 08:54:34 -08:00
Lexman
47a54ac8bc
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Calvin Leung Huang
656b113dbd
sys/config: config state endpoint (#7424)
* sys/config: initial work on adding config state endpoint

* server/config: add tests, fix Sanitized method

* thread config through NewTestCluster's config to avoid panic on dev modes

* properly guard endpoint against request forwarding

* add http tests, guard against panics on nil RawConfig

* ensure non-nil rawConfig on NewTestCluster cores

* update non-forwarding logic

* fix imports; use no-forward handler

* add missing config test fixture; update gitignore

* return sanitized config as a map

* fix test, use deep.Equal to check for equality

* fix http test

* minor comment fix

* config: change Sanitized to return snake-cased keys, update tests

* core: hold rlock when reading config; add docstring

* update docstring
2019-10-08 10:57:15 -07:00
Vu Pham
feafd1b388 OCI KMS auto-unseal plugin (#6950) 2019-09-04 12:40:54 -07:00
Tommy Murphy
1c146a1012 telemetry: add stackdriver metrics sink (#6957)
* telemetry: add stackdriver metrics sink

* telemetry: stackdriver go mod tidy
2019-08-20 14:47:08 -07:00
Calvin Leung Huang
d0d17ed40e command/server: fix TestLoadConfigFile_json2 test, fix hcl tags (#7300)
* command/server: fix TestLoadConfigFile_json2 test, fix hcl tags

Fixes test to call the equality check, and add missing values to the expected object. Fixes hcl tags in the Telemetry structs.

* fix PrometheusRetentionTime tag
2019-08-14 10:32:11 -04:00
Jim Kalafut
ec42f5f219
Fix broken server config test (#7153) 2019-07-22 09:39:13 -07:00
Mike Jarmy
be3e2a1288
Add 'log-format' CLI flag, along with associated config flag, for 'vault server' command. (#6840)
* Read config before creating logger when booting vault server

* Allow for specifying log output in JSON format in a config file, via a 'log_level' flag

* Create parser for log format flag

* Allow for specifying log format in a config file, via a 'log_format' flag. Also, get rid of 'log_json' flag.

* Add 'log-format' command line flag

* Update documentation to include description of log_format setting

* Tweak comment for VAULT_LOG_FORMAT environment variable

* add test for ParseEnvLogFormat()

* clarify how log format is set

* fix typos in documentation
2019-07-18 15:59:27 -04:00
Jeff Mitchell
2ac94d5581
Add the ability to use a dev Consul node for dev storage (#6965) 2019-06-24 13:29:47 -04:00
Jeff Mitchell
278bdd1f4e
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell
170521481d
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Vishal Nayak
3c7c593bca Agent: Listener refactoring and socket file system permissions (#6397)
* Listener refactoring and file system permissions

* added listenerutil and move some common code there

* Added test for verifying socket file permissions

* Change default port of agent to 8200

* address review feedback

* Address review feedback

* Read socket options from listener config
2019-03-14 11:53:14 -07:00
Jeff Mitchell
213da13264 Add ability to migrate autoseal to autoseal (#5930)
* Add ability to migrate autoseal to autoseal

This adds the ability to migrate from shamir to autoseal, autoseal to
shamir, or autoseal to autoseal, by allowing multiple seal stanzas. A
disabled stanza will be used as the config being migrated from; this can
also be used to provide an unwrap seal on ent over multiple unseals.

A new test is added to ensure that autoseal to autoseal works as
expected.

* Fix test

* Provide default shamir info if not given in config

* Linting feedback

* Remove context var that isn't used

* Don't run auto unseal watcher when in migration, and move SetCores to SetSealsForMigration func

* Slight logic cleanup

* Fix test build and fix bug

* Updates

* remove GetRecoveryKey function
2019-03-04 14:11:56 -08:00
Michel Vocks
9617832784
Print warning when 'tls_cipher_suites' includes blacklisted cipher suites (#6300)
* Implemented a warning when tls_cipher_suites includes only cipher suites which are not supprted by the HTTP/2 spec

* Added test for cipher suites

* Added hard fail on startup when all defined cipher suites are blacklisted. Added warning when some ciphers are blacklisted.

* Replaced hard failure with warning. Removed bad cipher util function and replaced it by external library.

* Added missing dependency. Fixed renaming of package name.
2019-03-01 16:48:06 +01:00
Chris Hoffman
2d0c3ff335 Transit Autounseal (#5995)
* Adding Transit Autoseal

* adding tests

* adding more tests

* updating seal info

* send a value to test and set current key id

* updating message

* cleanup

* Adding tls config, addressing some feedback

* adding tls testing

* renaming config fields for tls
2019-02-28 13:13:56 -08:00
Chris Hoffman
ec015ec6bb fixing command server tests (#6242) 2019-02-15 09:06:20 -08:00
Vishal Nayak
e39a5f28df
Vault Agent Cache (#6220)
* vault-agent-cache: squashed 250+ commits

* Add proper token revocation validations to the tests

* Add more test cases

* Avoid leaking by not closing request/response bodies; add comments

* Fix revoke orphan use case; update tests

* Add CLI test for making request over unix socket

* agent/cache: remove namespace-related tests

* Strip-off the auto-auth token from the lookup response

* Output listener details along with configuration

* Add scheme to API address output

* leasecache: use IndexNameLease for prefix lease revocations

* Make CLI accept the fully qualified unix address

* export VAULT_AGENT_ADDR=unix://path/to/socket

* unix:/ to unix://
2019-02-14 20:10:36 -05:00
Martin
5dd50ef281 Prometheus support on v1/sys/metrics endpoint (#5308)
* initial commit for prometheus and sys/metrics support

* Throw an error if prometheusRetentionTime is 0,add prometheus in devmode

* return when format=prometheus is used and prom is disable

* parse prometheus_retention_time from string instead of int

* Initialize config.Telemetry if nil

* address PR issues

* add sys/metrics framework.Path in a factory

* Apply requiredMountTable entries's MountConfig to existing core table

* address pr comments

* enable prometheus sink by default

* Move Metric-related code in a separate metricsutil helper
2019-02-14 12:46:59 -08:00
Noel Cower
21e062d9d8 Merge all configuration fields (#6028)
This changes (*Config).Merge to merge all fields of a Config.
Previously, when merging Configs, some configuration fields were
ignored and completely lost, including APIAddr, ClusterAddr, and
a couple boolean fields. This only occurs when using multiple config
files and does not affect single config files (even when loading from
a directory -- Merge is only called after a second file is loaded).

- Fix APIAddr not being merged.
- Fix ClusterAddr not being merged.
- Fix DisablePrintableCheck not being merged.
- Fix DisableClustering not being merged. The DisableClusteringRaw
  value is also preserved so that it can be used in overrides for
  storage fields.
- Use merged top-level config as storage field overrides.
- Update config dir test fixtures to set some fields missed by
  (*Config).Merge previously.
2019-01-23 11:27:21 -05:00
Jim Kalafut
a54603039d Run goimports across the repository (#6010)
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
2019-01-08 16:48:57 -08:00
Chris Hoffman
caf81bc28c
Add -dev-auto-seal option (#5629)
* adding a -dev-auto-seal option

* adding logger to TestSeal
2018-10-29 09:30:24 -04:00
Jeff Mitchell
6b0782989a Fix build 2018-10-23 15:09:35 -04:00
Jeff Mitchell
41deb88d68 Add disable-indexing 2018-10-23 15:03:17 -04:00