Commit Graph

2422 Commits

Author SHA1 Message Date
Alexander Scheel
754e2adc99
Add acme account storage (#19953)
* Enable creation of accounts

 - Refactors many methods to take an acmeContext, which holds the
   storageContext on it.
 - Updates the core ACME Handlers to use *acmeContext, to avoid
   copying structs.
 - Makes JWK exported so the JSON parser can find it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Finish ACME account creation

 - This ensures a Kid is created when one doesn't exist
 - Expands the parsed handler capabilities, to format the response and
   set required headers.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-03 16:08:25 -04:00
Steven Clark
a6ee618590
Rework the ACME test suite to use full Vault cluster to validate behavior (#19874)
- Instead of using tests that just test the plugin storage/interface
   layer, use a full Vault instance to validate that we can send/receive
   the proper headers and responses back to a client.
 - Found an issue with HEAD new-nounce api calls returning 500 errors.
 - Add the /acme/ suffix to the baseUrl in the acme context so we don't
   have to keep adding it a bit everywhere.
2023-04-03 09:38:20 -04:00
Alexander Scheel
32e3cd6972
Merge ACME package back into the PKI package (#19826)
* Squash pki/acme package down to pki folder

Without refactoring most of PKI to export the storage layer, which we
were initially hesitant about, it would be nearly impossible to have the
ACME layer handle its own storage while being in the acme/ subpackage
under the pki package.

Thus, merge the two packages together again.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Properly format errors for missing parameters

When missing required ACME request parameters, don't return Vault-level
errors, but drop into the PKI package to return properly-formatted ACME
error messages.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Error type clarifications

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix GetOk with type conversion calls

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-29 21:08:31 +00:00
Alexander Scheel
71441742fa
Add ACME new account creation handlers (#19820)
* Identify whether JWKs existed or were created, set KIDs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Reclassify ErrAccountDoesNotExist as 400 per spec

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional stub methods for ACME accounts

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Start adding ACME newAccount handlers

This handler supports two pieces of functionality:

 1. Searching for whether an existing account already exists.
 2. Creating a new account.

One side effect of our JWS parsing logic is that we needed a way to
differentiate between whether a JWK existed on disk from an account or
if it was specified in the request. This technically means we're
potentially responding to certain requests with positive results (e.g.,
key search based on kid) versus erring earlier like other
implementations do.

No account storage has been done as part of this commit.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Unify path fields handling, fix newAccount method

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-29 19:06:09 +00:00
Steven Clark
ddb5db1538
Initial ACME new-nonce API (#19822)
* Initial ACME new-nonce API implementation

* Return proper HTTP status codes for ACME new-nonce API handler
2023-03-29 18:22:48 +00:00
Steven Clark
4063d6ed09
PKI: Initial ACME directory API support (#19803)
* PKI: Initial ACME directory API support along with basic tests for error handler and the directory itself across various paths.
2023-03-29 16:29:19 +00:00
Alexander Scheel
c3fce3512c
Tighten JWS algorithm lists (#19797)
* Add new PKI ACME subpackage to test_packages list

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Restrict JWS keys to specified algorithms

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-28 20:49:07 +00:00
Alexander Scheel
a060e6ee7e
Add initial ACME server library to PKI (#19778)
* Add ACME package to provide a nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add JWS parsing helper

Using go-jose v2, we start building a JWS parsing helper, ensuring that
fields are properly validated w.r.t. the ACME spec's intersection with
JWS.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add error context information

Start adding the ability to wrap errors returned by Vault to
ACME-specific errors.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Make ACMEState exported

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-28 17:29:54 +00:00
Anton Averchenkov
f674f0ea32
Fix response structure validation tests for non-2xx responses (#19726) 2023-03-23 16:33:44 -04:00
Mason Foster
cfff8d420e
aws: pass cancelable context with aws calls (#19365)
* auth/aws: use cancelable context with aws calls

* secrets/aws: use cancelable context with aws calls
2023-03-23 12:02:24 -05:00
Alexander Scheel
dcde779450
Add test to test SSH endpoint authentication (#19705)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-23 16:00:21 +00:00
Alexander Scheel
bebdc6551d
Add tests for PKI endpoint authentication (#19704)
* Split (un,)authenticated issuer fetch endpoints

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow unauthed per-issuer unified-crl access

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests to validate endpoint authentication status

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Disable schema validation in TestBackend_IfModifiedSinceHeaders

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-23 15:54:26 +00:00
Steven Clark
94a6dca9d6
Fix for PKI.TestStandby_Operations test to work in ENT (#19647)
* Fix for PKI.TestStandby_Operations test to work in ENT

 - Remove wait call to testhelpers.WaitForActiveNodeAndStandbys and
   leverage testhelpers.WaitForStandbyNode instead.

* Use InmemBackendSetup for a proper HA backend in ENT
2023-03-20 21:11:56 +00:00
Alexander Scheel
89ff5f4844
More wrapped errors - PKI, SSH, Transit (#19631)
* Add missing wrapped errors in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing wrapped errors in SSH

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing wrapped errors in Transit

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-20 16:00:49 +00:00
Steven Clark
69e238771a
Forward PKI revocation requests received by standby nodes to active node (#19624)
* Forward PKI revocation requests received by standby nodes to active node

 - A refactoring that occurred in 1.13 timeframe removed what was
   considered a specific check for standby nodes that wasn't required
   as a writes should be returning ErrReadOnly.
 - That sadly exposed a long standing bug where the errors from the
   storage layer were not being properly wrapped, hiding the ErrReadOnly
   coming from a write and failing the request.

* Add cl

* Add test for basic PKI operations against standby nodes
2023-03-20 14:58:36 +00:00
Hamid Ghaf
e55c18ed12
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
AnPucel
4ecb18b262
PKI Responses Part 4 (#18612) 2023-03-14 15:00:37 -07:00
Anton Averchenkov
ce420de231
Ignore special HTTP fields in response validation tests (#19530) 2023-03-14 13:21:20 -04:00
Alexander Scheel
9e188973d9
Add support for importing RSA-PSS keys into Transit (#19519)
* Add support for importing RSA-PSS keys in Transit

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-13 17:03:01 +00:00
Jakob Beckmann
39f9e5e775
Allow alias dereferencing in LDAP searches (#18230)
* impr(auth/ldap): allow to dereference aliases in searches

* docs: add documentation for LDAP alias dereferencing

* chore(auth/ldap): add changelog entry for PR 18230

* chore: run formatter

* fix: update default LDAP configuration with new default

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>

* docs(ldap): add alias dereferencing to API docs for LDAP

---------

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
2023-02-24 13:49:17 -05:00
davidadeleon
8154be65a9
add nil check for secret id entry on delete via accessor (#19186)
* add nil check for secret id entry on delete via accessor

* add changelog

* add godoc to test

* improve feedback on nil entry

* fix error reporting on invalid secret id accessor

* fix test to expect implemented error
2023-02-24 13:18:08 -05:00
Jakob Beckmann
0d3c0c09c8
feat(auth/ldap): allow passing the LDAP password via an env var (#18225)
* feat(auth/ldap): allow passing the LDAP password via an environment variable when authenticating via the CLI

* chore(auth/ldap): add changelog entry for PR 18225
2023-02-23 11:16:17 -05:00
Raymond Ho
77e80a8030
use github token env var if present when fetching org id (#19244) 2023-02-21 12:17:35 -08:00
AnPucel
5cfb93837a
Adding PKI Responses 3 (#18596) 2023-02-16 17:31:45 -08:00
John-Michael Faircloth
4bfc64992a
plugin/secrets/auth: enable multiplexing (#19215)
* plugin/auth: enable multiplexing

- the plugin will be multiplexed when run as an external plugin
  by vault versions that support secrets/auth plugin multiplexing (> 1.12)
- we continue to set the TLSProviderFunc to maintain backwards
  compatibility with vault versions that don't support AutoMTLS (< 1.12)

* enable multiplexing for secrets engines

* add changelog

* revert call to ServeMultiplex for pki and transit

* Revert "revert call to ServeMultiplex for pki and transit"

This reverts commit 755be28d14.
2023-02-16 22:25:15 +00:00
AnPucel
8651d6de38
PKI Response Structures Part 2 (#18479)
Response structures from intermediate --> manage_keys
2023-02-15 15:09:57 -08:00
AnPucel
d09e02a9a3
Adding Response Structures to PKI Config (#18376) 2023-02-15 14:51:27 -08:00
Raymond Ho
07821f27f5
fix TestBackendAcc_LoginWithCallerIdentity (#19201) 2023-02-15 12:40:49 -08:00
Anton Averchenkov
bc303fee63
Refactor approle response validation tests (#19188) 2023-02-15 12:29:15 -05:00
Steven Clark
42369ac698
Attempt at fixing the memory usage in CI (#19171)
- Do not keep many intervals of the in memory sink metrics
   collector. Otherwise we fill up the CI's memory
2023-02-14 08:40:25 -05:00
Ellie
b3a4ddfb09
add error message when trying to rotate mssql root without password in configuration (#19103)
* add error message when trying to rotate mssql root without password in configuration

* add changelog
2023-02-13 07:31:13 -05:00
Kit Haines
2dd4528ed8
Telemetry Metrics Configuration. (#18186)
* Telemetry Metrics Configuration.

* Err Shadowing Fix (woah, semgrep is cool).

* Fix TestBackend_RevokePlusTidy_Intermediate

* Add Changelog.

* Fix memory leak.  Code cleanup as suggested by Steve.

* Turn off metrics by default, breaking-change.

* Show on tidy-status before start-up.

* Fix tests

* make fmt

* Add emit metrics to periodicFunc

* Test not delivering unavailable metrics + fix.

* Better error message.

* Fixing the false-error bug.

* make fmt.

* Try to fix race issue, remove confusing comments.

* Switch metric counter variables to an atomic.Uint32

 - Switch the metric counter variables to an atomic variable type
   so that we are forced to properly load/store values to it

* Fix race-issue better by trying until the metric is sunk.

* make fmt.

* empty commit to retrigger non-race tests that all pass locally

---------

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2023-02-10 21:31:56 +00:00
Michael Dempsey
0bacc16233
Add default to allowed values for algorithm_signer (#17894)
* Add default to allowed values for algorithm_signer

* Add possible values for algorithm signer in ui
2023-02-09 13:03:53 -05:00
Steven Clark
afd9e82e9b
Add a comment around why we are grabbing a lock to update an atomic boolean (#19087) 2023-02-09 09:12:37 -05:00
Steven Clark
05e742b4ee
Revert URL encoding OCSP GET fix #18938 (#19037)
- This fix was incorrect as now the tests and program are double
   URL encoding the OCSP GET requests, so the base64 + characters
   when using Vault proper are becoming space characters.
2023-02-08 11:24:26 -05:00
miagilepner
13caa0842e
VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
Nick Cabatoff
192baa88db
Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
Steven Clark
576c12e2ec
Use the unified CRL on local CRL paths if UnifiedCRLOnExistingPaths is set (#18989)
* Use the unified CRL on legacy CRL paths if UnifiedCRLOnExistingPaths is set

 - If the crl configuration option unified_crl_on_existing_paths is set
   to true along with the unified_crl feature, provide the unified crl
   on the existing CRL paths.
 - Added some test helpers to help debugging, they are being used by
   the ENT test that validates this feature.

* Rename method to shouldLocalPathsUseUnified
2023-02-03 14:38:36 -05:00
Alexander Scheel
780dcf1316
Use UTC for leaf exceeding CA's notAfter (#18984)
* Use UTC for leaf exceeding CA's notAfter

When generating a leaf which exceeds the CA's validity period, Vault's
error message was confusing as the leaf would use the server's time
zone, but the CA's notAfter date would use UTC. This could cause
user confusion as the leaf's expiry might look before the latter, due
to using different time zones. E.g.:

> cannot satisfy request, as TTL would result in notAfter
> 2023-03-06T16:41:09.757694-08:00 that is beyond the expiration of
> the CA certificate at 2023-03-07T00:29:52Z

Consistently use UTC for this instead.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-03 17:00:42 +00:00
Steven Clark
f030cbc847
Fix race accessing b.crls within cert auth (#18945)
* Fix race accessing b.crls within cert auth

 - Discovered by CircleCI the pathLogin, pathLoginRenew paths access
   and reloads the b.crls member variable without a lock.
 - Also discovered that pathLoginResolveRole never populated an empty
   b.crls before usage within b.verifyCredentials

* Add cl

* Misc cleanup

 - Introduce a login path wrapper instead of repeating in all the
   various login methods the crl reloading
 - Cleanup updatedConfig, never returned an error and nothing looked at
   the error returned
 - Make the test within TestCRLFetch a little less timing sensitive as
   I was able to trigger a failure due to my machine taking more than
   150ms to load the new CRL
2023-02-01 16:23:06 -05:00
Steven Clark
b5f04147cb
Apply URL encoding/unencoding to OCSP Get requests (#18938)
* Apply URL encoding/unencoding to OCSP Get requests

 - Missed this during development and sadly the unit tests were written
   at a level that did not expose this issue originally, there are
   certain combinations of issuer cert + serial that lead to base64
   data containing a '/' which will lead to the OCSP handler not getting
   the full parameter.
 - Do as the spec says, this should be treated as url-encoded data.

* Add cl

* Add higher level PKI OCSP GET/POST tests

* Rename PKI ocsp files to path_ocsp to follow naming conventions

* make fmt
2023-02-01 11:03:43 -05:00
Alexander Scheel
e4766aa974
Allow cleanup ssh dynamic keys host keys (#18939)
* Add ability to clean up host keys for dynamic keys

This adds a new endpoint, tidy/dynamic-keys that removes any stale host
keys still present on the mount. This does not clean up any pending
dynamic key leases and will not remove these keys from systems with
authorized hosts entries created by Vault.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-01 15:09:16 +00:00
Steven Clark
d34a4b6a2a
Refactor an ENT test helper back to OSS to wait for updated CRLs within PKI tests (#18933)
- We had two separate helpers that were very close, one only in ENT, so unify them here in OSS.
2023-02-01 08:47:26 -05:00
Steven Clark
f05c20ba42
PKI Unified CRL/OCSP apis should be ent only (#18913)
* PKI Unified CRL/OCSP apis should be ent only

 - Do not enable any of the unified crl/ocsp related apis on OSS.

* Rollback refactoring of pathFetchCRLViaCertPath

 - As pointed out in the PR, this method isn't actually being used at
   the moment with the <serial> handler, pathFetchValid, matching
   everything under the cert/XXXX path.

* Fix schema for ent/oss diff

 - Define the OSS vs ENT urls we want to see within the schema
   definition even if they aren't really going to be used in the end.
2023-02-01 08:35:14 -05:00
Nick Cabatoff
970ed07b04
Vault test cluster helper refactorings, mostly audit related (#18928)
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package.  Consolidate on a single "noop audit" implementation.
2023-02-01 08:33:16 -05:00
Alexander Scheel
8e7f2076a2
Remove dynamic keys from SSH Secrets Engine (#18874)
* Remove dynamic keys from SSH Secrets Engine

This removes the functionality of Vault creating keys and adding them to
the authorized keys file on hosts.

This functionality has been deprecated since Vault version 0.7.2.

The preferred alternative is to use the SSH CA method, which also allows
key generation but places limits on TTL and doesn't require Vault reach
out to provision each key on the specified host, making it much more
secure.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove dynamic ssh references from documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove dynamic key secret type entirely

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify changelog language

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add removal notice to the website

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-31 16:02:22 -05:00
Nick Cabatoff
efa8c22f17
TestClusterCore's TLSConfig becomes a method and does a Clone. (#18914) 2023-01-31 11:05:16 -05:00
Steven Clark
6f0552c0e3
Address pki::TestAutoRebuild flakiness (#18903)
* Address pki::TestAutoRebuild flakiness

 - Wait for a CRL change before progressing to the next step after
   we change configuration. Prior to this we would be racing against
   the CRL reloading from the configuration change.
2023-01-30 21:38:38 +00:00
akshya96
14c355cd70
Brute forcing unlock user bug (#18890)
* brute forcing unlock user bug

* add changelog

* fix changelog
2023-01-30 13:06:10 -08:00
Matt Schultz
564357f1f9
Add some managed key build tags to support incoming enterprise feature work. (#18902) 2023-01-30 14:57:01 -06:00