Commit Graph

66 Commits

Author SHA1 Message Date
hghaf099
13b8306575
VAULT-1564 report in-flight requests (#13024)
* VAULT-1564 report in-flight requests

* adding a changelog

* Changing some variable names and fixing comments

* minor style change

* adding unauthenticated support for in-flight-req

* adding documentation for the listener.profiling stanza

* adding an atomic counter for the inflight requests
addressing comments

* addressing comments

* logging completed requests

* fixing a test

* providing log_requests_info as a config option to determine at which level requests should be logged

* removing a member and a method from the StatusHeaderResponseWriter struct

* adding api docks

* revert changes in NewHTTPResponseWriter

* Fix logging invalid log_requests_info value

* Addressing comments

* Fixing a test

* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core

* fixing go.sum

* minor refactoring

* protecting InFlightRequests from data race

* another try on fixing a data race

* another try to fix a data race

* addressing comments

* fixing couple of tests

* changing log_requests_info to log_requests_level

* minor style change

* fixing a test

* removing the lock in InFlightRequests

* use single-argument form for interface assertion

* adding doc for the new configuration paramter

* adding the new doc to the nav data file

* minor fix
2021-12-08 17:34:42 -05:00
Scott Miller
886911d8b7
Diagnose partial/missing telemetry configuration (#12802)
* Diagnose partial/missing telemetry configuration

* changelog

* fixup

* not sure which component?
2021-10-20 16:47:59 -05:00
hghaf099
e0bfb73815
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00
contrun
452e47cbf2
remove trailing comma in json file (#12640)
Signed-off-by: YI <uuuuuu@protonmail.com>
2021-10-12 12:03:01 -04:00
Hridoy Roy
46afacb013
Diagnose Language Pass (#11909)
* save

* save

* save

* first round of the diagnose language pass

* capitalization

* first round of feedback

* fix bug in advise

* a few more nouns to verbs
2021-07-11 15:44:19 -07:00
Pratyoy Mukhopadhyay
c7b829165a
[VAULT-2776] Add prefix_filter option to Vault (#12025)
* [VAULT-2776] Add prefix_filter support to vault

* [VAULT-2776] Add filter_default config, update docs

* [VAULT-2776] Add changelog file

* [VAULT-2776] Update telemetry tests and error handling

* [VAULT-2776] Add test fixtures, update test

* [VAULT-2776] Update gitignore hcl filter
2021-07-09 14:49:53 -05:00
hghaf099
41fa51a4cd
diagnose: Add seal transit tls check (#11922)
* diagnose: Add seal transit tls check

* Fixing the path to the config file and the path to the cert files

* Addressing comment

* Addressing seal transit tls check comments
2021-06-24 18:30:42 -04:00
Hridoy Roy
25346e824e
Diagnose Storage Panic Bugfixes (#11923)
* partial

* fix raft panics and ensure checks are skipped if storage isnt initialized

* cleanup directories

* newline

* typo in nil check

* another nil check
2021-06-24 09:56:38 -07:00
Scott Miller
1c31d5ac53
Wire configuration checks into diagnose and fix resulting bugs. (#11854)
* Actually call config.Validate in diagnose

* Wire configuration checks into diagnose and fix resulting bugs.

* go mod vendor

* Merge to vendorless version

* Remove sentinel section to allow diagnose_ok to pass

* Fix unit tests
2021-06-17 13:09:37 -05:00
Hridoy Roy
afc0478a33
Diagnose checks for raft quorum status and file backend permissions (#11771)
* raft file and quorum checks

* raft checks

* backup

* raft file checks test

* address comments and add more raft and file and process checks

* syntax issues

* modularize functions to compile differently on different os

* compile raft checks everywhere

* more build tag issues

* raft-diagnose

* correct file permission checks

* upgrade tests and add a getConfigOffline test that currently does not work

* comment

* update file checks method signature on windows

* Update physical/raft/raft_test.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft tests

* add todo comment for windows root ownership

* voter count message

* raft checks test fixes

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-17 10:04:21 -07:00
Scott Miller
1cf7637400
Diagnose Disk usage checks (#11673)
* Disk usage checks

* Move disk free earlier

* Move logic to helpers

* Bring over test logic from the ulimit PR

* imports

* Report error

* Get unit tests working
2021-05-27 10:17:52 -07:00
Hridoy Roy
4b8ca940c8
Modularize Run Command (#11573)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* fix storage latency test errors

* review comments

* use random uuid for latency checks instead of static id
2021-05-25 15:23:20 -07:00
Nick Cabatoff
55ce931730
config for autoloading license (oss parts) 2021-05-07 08:55:41 -04:00
Scott Miller
2feeb39b85
Expose unknown fields and duplicate sections as diagnose warnings (#11455)
* Expose unknown fields and duplicate sections as diagnose warnings

* section counts not needed, already handled

* Address PR feedback

* Prune more of the new fields before tests call deep.Equals

* Update go.mod
2021-05-04 14:47:56 -05:00
Hridoy Roy
52d70a4683
Diagnoses Consul Direct Access Check (#11505)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* run direct access checks in diagnose command

* review comments

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 16:21:06 -07:00
Hridoy Roy
a20b005e3e
Consul TLS Checks for Diagnose [draft] (#11467)
* consul tls checks

* fix some tests

* complete physical and service registration tls checks
2021-04-28 08:55:18 -07:00
Josh Black
795ce10c6a
Add HTTP response headers for hostname and raft node ID (if applicable) (#11289) 2021-04-20 15:25:04 -07:00
Hridoy Roy
2fd30841cd
Initial Diagnose Command for TLS and Listener [VAULT-1896, VAULT-1899] (#11249)
* sanity checks for tls config in diagnose

* backup

* backup

* backup

* added necessary tests

* remove comment

* remove parallels causing test flakiness

* comments

* small fix

* separate out config hcl test case into new hcl file

* newline

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* reload funcs should be allowed to be nil
2021-04-06 16:40:43 -07:00
Hridoy Roy
41f2a7732e
Diagnose Stub Command Tests (#11180)
* a few tests to the operator diagnose stub command

* a few tests to the operator diagnose stub command

* a few tests to the operator diagnose stub command

* empty commit to fix circle ci permissions issue

* empty commit to fix circle ci permissions issue
2021-03-29 14:22:32 -07:00
Hridoy Roy
0767980550
Port: Telemetry For Lease Expiration Times (#10375)
* port lease metrics

* go mod vendor

* caught a bug
2020-11-13 10:26:58 -08:00
Brian Kassouf
3bc7d15e6b
Backport some OSS changes (#10267)
* Backport some OSS changes

* go mod vendor
2020-10-29 16:47:34 -07:00
Rodrigo D. L
3eb79c5209
adding new config flag disable_sentinel_trace (#9696) 2020-08-10 06:23:44 -04:00
ncabatoff
34144e38ee
Fix 1.5 regression that meant non-string values in the seal stanza would fail config parsing, preventing startup. (#9555) 2020-07-23 13:53:00 -04:00
Scott Miller
2648713fcd
Add new Telemetry config options (#9238)
* Add new Telemetry config options

Add cluster_name, maximum_gauge_cardinality, and usage_gauge_period
configuration options to the config stanza.

Update unit tests.

Document.

Co-authored-by: Mark Gritter <mgritter@hashicorp.com>
2020-06-17 10:07:33 -05:00
Josh Black
f0c8b4c7a9
Fix JSON encoding adding newlines. (#8928)
Fix JSON encoding adding newlines.

This manifested itself when encoding config values, which all map to
strings. An extra new line would get added by json.Encode, which caused
other things to break with confusing error messagges. Switching to
json.Marshal seems to solve the problem.
2020-05-19 16:13:05 -07:00
Jeff Mitchell
91b09c09b5
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
ncabatoff
4d7bfb4927
Add support for telemetry option metrics_prefix (#8340)
When set, instead of starting with `vault_`, metrics will start with `${prefix}_`.
2020-02-13 09:00:32 -05:00
Vishal Nayak
1a2ec77a7a Raft retry join (#7856)
* Raft retry join

* update

* Make retry join work with shamir seal

* Return upon context completion

* Update vault/raft.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Address some review comments

* send leader information slice as a parameter

* Make retry join work properly with Shamir case. This commit has a blocking issue

* Fix join goroutine exiting before the job is done

* Polishing changes

* Don't return after a successful join during unseal

* Added config parsing test

* Add test and fix bugs

* minor changes

* Address review comments

* Fix build error

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-01-13 17:02:16 -08:00
Mike Jarmy
df01a4307d
Introduce optional service_registration stanza (#7887)
* move ServiceDiscovery into methods

* add ServiceDiscoveryFactory

* add serviceDiscovery field to vault.Core

* refactor ConsulServiceDiscovery into separate struct

* cleanup

* revert accidental change to go.mod

* cleanup

* get rid of un-needed struct tags in vault.CoreConfig

* add service_discovery parser

* add ServiceDiscovery to config

* cleanup

* cleanup

* add test for ConfigServiceDiscovery to Core

* unit testing for config service_discovery stanza

* cleanup

* get rid of un-needed redirect_addr stuff in service_discovery stanza

* improve test suite

* cleanup

* clean up test a bit

* create docs for service_discovery

* check if service_discovery is configured, but storage does not support HA

* tinker with test

* tinker with test

* tweak docs

* move ServiceDiscovery into its own package

* tweak a variable name

* fix comment

* rename service_discovery to service_registration

* tweak service_registration config

* Revert "tweak service_registration config"

This reverts commit 5509920a8a.

* simplify naming

* refactor into ./serviceregistration/consul
2019-12-06 09:46:39 -05:00
Lexman
47a54ac8bc
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Calvin Leung Huang
656b113dbd
sys/config: config state endpoint (#7424)
* sys/config: initial work on adding config state endpoint

* server/config: add tests, fix Sanitized method

* thread config through NewTestCluster's config to avoid panic on dev modes

* properly guard endpoint against request forwarding

* add http tests, guard against panics on nil RawConfig

* ensure non-nil rawConfig on NewTestCluster cores

* update non-forwarding logic

* fix imports; use no-forward handler

* add missing config test fixture; update gitignore

* return sanitized config as a map

* fix test, use deep.Equal to check for equality

* fix http test

* minor comment fix

* config: change Sanitized to return snake-cased keys, update tests

* core: hold rlock when reading config; add docstring

* update docstring
2019-10-08 10:57:15 -07:00
Chris Hoffman
ec015ec6bb fixing command server tests (#6242) 2019-02-15 09:06:20 -08:00
Noel Cower
21e062d9d8 Merge all configuration fields (#6028)
This changes (*Config).Merge to merge all fields of a Config.
Previously, when merging Configs, some configuration fields were
ignored and completely lost, including APIAddr, ClusterAddr, and
a couple boolean fields. This only occurs when using multiple config
files and does not affect single config files (even when loading from
a directory -- Merge is only called after a second file is loaded).

- Fix APIAddr not being merged.
- Fix ClusterAddr not being merged.
- Fix DisablePrintableCheck not being merged.
- Fix DisableClustering not being merged. The DisableClusteringRaw
  value is also preserved so that it can be used in overrides for
  storage fields.
- Use merged top-level config as storage field overrides.
- Update config dir test fixtures to set some fields missed by
  (*Config).Merge previously.
2019-01-23 11:27:21 -05:00
Jeff Mitchell
935815d3da Sync over some stuff 2018-08-24 12:09:03 -04:00
Jeff Mitchell
5d706e165d Fix printable check key not being valid 2018-07-12 16:59:07 -04:00
Calvin Leung Huang
faccf38997 Move HA-related config values to top level (#3550)
* Move HA-related config values to top level

* Add config2.hcl test-fixture
2017-11-08 14:19:41 -05:00
Vishal Nayak
de7ac83df6 Add 'pid_file' config option (#3321)
* add pid_file config option

* address review feedback

* address review comments
2017-09-16 17:09:37 -04:00
Chris Hoffman
4a8c33cca3 Disable the sys/raw endpoint by default (#3329)
* disable raw endpoint by default

* adding docs

* config option raw -> raw_storage_endpoint

* docs updates

* adding listing on raw endpoint

* reworking tests for enabled raw endpoints

* root protecting base raw endpoint
2017-09-15 00:21:35 -04:00
Jeff Mitchell
3f31ed733f Add option to set cluster TLS cipher suites. (#3228)
* Add option to set cluster TLS cipher suites.

Fixes #3227
2017-08-30 16:28:23 -04:00
Jeff Mitchell
22e06c05e8 Convert listener arguments to map[string]interface{} (#2905)
This allows people to use more natural constructs, e.g. for tls_disable
it can be a bool, int, or string.
2017-06-22 20:29:53 +01:00
Jeff Mitchell
f5de93f1bf Add DogStatsD metrics output. (#2883)
Fixes #2490
2017-06-16 23:51:46 -04:00
Jeff Mitchell
b1ed578f3d Rename physical backend to storage and alias old value (#2456) 2017-03-08 09:17:00 -05:00
Jeff Mitchell
bfaf9022fa Do some porting to make diffing easier 2017-02-24 10:45:29 -05:00
matt maier
2cd3cfd83e Vendor circonus (#2082) 2016-11-10 16:17:55 -05:00
Jeff Mitchell
0765d8e938 Switch default case of disable cluster. (#1959) 2016-10-02 14:54:01 -04:00
Jeff Mitchell
d40277a18f Plumb through the ability to set the storage read cache size. (#1784)
Plumb through the ability to set the storage read cache size.

Fixes #1772
2016-08-26 10:27:06 -04:00
Jeff Mitchell
68345eb770 Convert to logxi 2016-08-21 18:13:37 -04:00
Jeff Mitchell
645540012f Request forwarding (#1721)
Add request forwarding.
2016-08-15 09:42:42 -04:00
vishalnayak
f76c926b0a Added cluster_name for existing config tests 2016-07-26 11:38:24 -04:00
matt maier
a1b50427f2 Circonus integration for telemetry metrics 2016-07-22 15:49:23 -04:00