Commit Graph

1395 Commits

Author SHA1 Message Date
Hridoy Roy
25346e824e
Diagnose Storage Panic Bugfixes (#11923)
* partial

* fix raft panics and ensure checks are skipped if storage isnt initialized

* cleanup directories

* newline

* typo in nil check

* another nil check
2021-06-24 09:56:38 -07:00
hghaf099
a6d73e3dc2
Fix diagnose panic when configuration file does not exist (#11932)
* Fix diagnose panic when configuration file does not exist

* Addressing comments

* Update command/operator_diagnose.go

Co-authored-by: Hridoy Roy <roy@hashicorp.com>

Co-authored-by: Hridoy Roy <roy@hashicorp.com>
2021-06-23 18:35:52 -04:00
Calvin Leung Huang
523a9c969c
agent: restart template runner on retry for unlimited retries (#11775)
* agent: restart template runner on retry for unlimited retries

* template: log error message early

* template: delegate retries back to template if param is set to true

* agent: add and use the new template config stanza

* agent: fix panic, fix existing tests

* changelog: add changelog entry

* agent: add tests for exit_on_retry_failure

* agent: properly check on agent exit cases, add separate tests for missing key vs missing secrets

* agent: add note on difference between missing key vs missing secret

* docs: add docs for template_config

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* docs: fix exit_on_retry_failure, fix Functionality section

* docs: update interaction title

* template: add internal note on behavior for persist case

* docs: update agent, template, and template-config docs

* docs: update agent docs on retry stanza

* Apply suggestions from code review

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11775.txt

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* agent/test: rename expectExit to expectExitFromError

* agent/test: add check on early exits on the happy path

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-21 16:10:15 -07:00
hghaf099
72c18c60fb
Tls check client certs [VAULT-2117] (#11860)
* add expiration warning to certificate checking for diagnose

* Adding TLS Client CA checks to diagnose
Adding checks for tls_disable_client_certs and tls_require_and_verify_client_cert flags

* Check validity of TLSClientCAFile
Adding related tests

* Addressing comments

* Fixing some tls tests

* Addressing comments

Co-authored-by: HridoyRoy <roy@hashicorp.com>
2021-06-18 19:35:38 -04:00
Brian Kassouf
d62b4e7581
Diagnose: Skip test-access-storage on raft storage (#11889) 2021-06-17 12:15:19 -07:00
Scott Miller
1c31d5ac53
Wire configuration checks into diagnose and fix resulting bugs. (#11854)
* Actually call config.Validate in diagnose

* Wire configuration checks into diagnose and fix resulting bugs.

* go mod vendor

* Merge to vendorless version

* Remove sentinel section to allow diagnose_ok to pass

* Fix unit tests
2021-06-17 13:09:37 -05:00
Hridoy Roy
afc0478a33
Diagnose checks for raft quorum status and file backend permissions (#11771)
* raft file and quorum checks

* raft checks

* backup

* raft file checks test

* address comments and add more raft and file and process checks

* syntax issues

* modularize functions to compile differently on different os

* compile raft checks everywhere

* more build tag issues

* raft-diagnose

* correct file permission checks

* upgrade tests and add a getConfigOffline test that currently does not work

* comment

* update file checks method signature on windows

* Update physical/raft/raft_test.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft tests

* add todo comment for windows root ownership

* voter count message

* raft checks test fixes

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-17 10:04:21 -07:00
Hridoy Roy
cea68aaa68
Add expiration warning to certificate checking for diagnose [VAULT-1018] (#11850)
* add expiration warning to certificate checking for diagnose

* Update serviceregistration/consul/consul_service_registration.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* review comments

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-15 09:53:29 -07:00
Josh Black
38b6271de1
OSS parts of sighup license reload test (#11816) 2021-06-11 09:08:19 -07:00
Hridoy Roy
3ee219e9c4
Diagnose resource creation checks (#11627)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* merge main branch

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* backup while I fix something else

* fix storage latency test errors

* runtime checks

* diagnose with timeout on seal
2021-06-10 12:29:32 -07:00
Shanker Naik
868e6e2246
fix typo in vault operator init cmd (#11792) 2021-06-10 10:39:53 -04:00
Nick Cabatoff
9f38504e06
Pull out license commands, and make the OSS changes needed for the license inspect PR in ent. (#11783) 2021-06-07 14:44:20 -04:00
Scott Miller
d702ee96d8
Add Advice and Advise functions for adding an advice section to any span. (#11760)
* wip

* wip

* Finish implementing advice handling and word wrapping

* Properly word wrap messages and warnings

* Remove debugging

* Remove debugging

* Remove unnecessary test

* unit test bug

* go vendor
2021-06-07 11:29:36 -05:00
Josh Black
f2dd7fe9a7
OSS parts of sighup license reload (#11767) 2021-06-04 10:24:35 -07:00
Scott Miller
001b89c090
Return different exit codes for different diagnose results (#11758)
This allows operators to run diagnose in scripts and detect the difference between success, warning, and failure.

Exit codes are now:
0: Success (no warnings)
1: Failure (some test failed)
2: Warning (some test warned)
3: User input failure such as a bad flag
4: Other error
2021-06-04 11:52:49 -05:00
Hridoy Roy
8ffef9a32a
remove operating system stanza from tests on account of disk usage being different across ci for different runs (#11764) 2021-06-03 14:11:31 -07:00
Scott Miller
02c2f731ec
Edge cases in the skipping gathering and output logic (#11752)
* Edge cases in the skipping gathering and output logic

* Fix unit test
2021-06-03 11:01:14 -05:00
Scott Miller
d6a162bd53
Remove duplicate children test (#11751) 2021-06-02 15:10:43 -05:00
Scott Miller
a0db8d8523
Segment out disk checks to disable on openbsd/arm (#11749)
* Segment out disk checks to disable on openbsd/arm

Also add a spot skipped helper.

* Expected results may be fewer than actual because of variable length tests like disk usage

* Move to os_common and build on windows
2021-06-02 12:17:52 -05:00
Lars Lehtonen
8d515fec2b
command: deprecate errwrap.Wrapf() (#11744) 2021-06-02 09:22:31 -04:00
Scott Miller
6583a5543f
Add ulimit check, and tidy unit test cases (#11678)
* Add ulimit check, and tidy unit test cases to avoid needing to have all results and perfect ordering

* Make order independent check recursive

* Fix unit tests

* Try a 5s request timeout
2021-06-01 12:43:51 -05:00
Nick Cabatoff
e5b6515ee7
Restore server license config lost in #11573 (#11730) 2021-06-01 08:42:30 -04:00
Nick Cabatoff
997143fdb8
OSS parts of #1903: we'd disabled these tests on enterprise temporarily, but now we're ready to make them run there again. (#11729)
Vault cares about the VAULT_LICENSE env var, but we don't want to set that in CI because it would change behaviour of tests that don't intend for it to be set.  Instead, we use VAULT_LICENSE_CI so that only packages/tests that opt-in will use it.
2021-05-31 12:08:10 -04:00
Scott Miller
1cf7637400
Diagnose Disk usage checks (#11673)
* Disk usage checks

* Move disk free earlier

* Move logic to helpers

* Bring over test logic from the ulimit PR

* imports

* Report error

* Get unit tests working
2021-05-27 10:17:52 -07:00
Jason O'Donnell
412db2245e
logical/aw: move sts signing request to awsutil (#11704) 2021-05-26 13:30:46 -04:00
Hridoy Roy
4b8ca940c8
Modularize Run Command (#11573)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* fix storage latency test errors

* review comments

* use random uuid for latency checks instead of static id
2021-05-25 15:23:20 -07:00
Scott Miller
2fe4bda547
Initial Diagnose CLI output (#11583)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* wip

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* wip

* wip

* wip

* Revendor otel

* Fix merge related problems

* imports

* Fix unit tests

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-21 19:21:11 -07:00
Nick Cabatoff
380d34395c
OSS parts of ent PR #1857: license autoloading init changes. (#11623) 2021-05-17 14:10:26 -04:00
Pierce Bartine
c8fe8984eb
Add ServerName to Vault Agent template config (#11288)
* Add ServerName to Vault Agent template config

* Remove newline

* Add changelog for 11288

* Update changelog/11288.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2021-05-13 07:18:15 -07:00
Calvin Leung Huang
e2f47039b5
agent/cert: properly return the cached client on AuthClient (#11576)
* agent/cert: properly return the cached client on AuthClient

* test: pass in nil client config, check on pointer values directly

* test: pass in nil client config

* changelog: add changelog entry
2021-05-12 14:59:07 -07:00
Josh Black
f1c0a7744f
Provide a new API endpoint for retrieving signed licenses (#11543) 2021-05-12 12:19:25 -07:00
Scott Miller
08de9ad978
Add infrastructure and helpers for skipping diagnose checks (#11593)
* Add infrastructure for skipping tests

* Add infrastructure for skipping tests

* Set it

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Implement type alias for test functions

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-12 12:54:40 -05:00
Scott Miller
d21e60eaa9
Move config validation code to it's own, non ENT specific file (#11579)
* Move config validation code to it's own, non ENT specific file

* Fix imports

* import order
2021-05-11 11:59:08 -05:00
Nick Cabatoff
55ce931730
config for autoloading license (oss parts) 2021-05-07 08:55:41 -04:00
Tom Proctor
d60b698c2d
Allow Agent auto auth to read symlinked JWT files (#11502) 2021-05-06 14:11:57 +01:00
Josh Black
ed1727c9c2
Introduce a "vault license get" command (#11526) 2021-05-04 12:55:48 -07:00
Scott Miller
2feeb39b85
Expose unknown fields and duplicate sections as diagnose warnings (#11455)
* Expose unknown fields and duplicate sections as diagnose warnings

* section counts not needed, already handled

* Address PR feedback

* Prune more of the new fields before tests call deep.Equals

* Update go.mod
2021-05-04 14:47:56 -05:00
Scott Miller
03c9933d2b
When running under systemd, send notifications about server startup, shutdown, and config reload (#11517) 2021-05-04 14:47:16 -05:00
Hridoy Roy
52d70a4683
Diagnoses Consul Direct Access Check (#11505)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* run direct access checks in diagnose command

* review comments

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 16:21:06 -07:00
Hridoy Roy
e06b90b7dc
Diagnose V0: Storage End to End Checks (#11468)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* cosmetic changes

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 13:33:13 -07:00
Clint
2a8c36be28
Update Agent Auth with GCP to use new SignJWT endpoint (#11473)
* Update Agent Auth with GCP to use new SignJWT endpoint

* use iamcredentials name instead of renaming the package on import

* add changelog

* Update changelog/11473.txt

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-04-30 15:45:06 -05:00
Scott Miller
d60057b820
Create helpers which integrate with OpenTelemetry for diagnose collection (#11454)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-04-29 13:32:41 -05:00
Hridoy Roy
a20b005e3e
Consul TLS Checks for Diagnose [draft] (#11467)
* consul tls checks

* fix some tests

* complete physical and service registration tls checks
2021-04-28 08:55:18 -07:00
Nick Cabatoff
acbb3b9d98
Update tests to reflect new HC public pgp keys. (#11446) 2021-04-23 12:32:46 -04:00
Josh Black
795ce10c6a
Add HTTP response headers for hostname and raft node ID (if applicable) (#11289) 2021-04-20 15:25:04 -07:00
Nick Cabatoff
c039894baf
Add support for unauthenticated pprof access on a per-listener basis,… (#11324)
* Add support for unauthenticated pprof access on a per-listener basis, as we do for metrics.

* Add missing pprof sub-targets like 'allocs' and 'block'.  Capture the goroutine subtarget a second time in text form.  This is mostly a convenience, but also I think the pprof format might be a bit lossy?
2021-04-19 14:30:59 -04:00
Nick Cabatoff
5a17c4e5ff
Don't cut off stack traces at 32MB. (#11364) 2021-04-16 15:55:05 -04:00
Brian Kassouf
43e4a6172f
Fix flakey TestAgent_Template_Retry test (#11332) 2021-04-09 12:11:01 -07:00
Brian Kassouf
a24653cc5c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Brian Kassouf
2df57a0418
Fix a few static analysis findings (#11307) 2021-04-07 16:48:40 -07:00