This will allow us to refactor the storage functions to take additional
parameters (or backend-inferred values) in the future. In particular, as
we look towards adding a storage cache layer, we'll need to add this to
the backend, which is now accessible from all storage functions.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
When tidy is called without arguments, we kick off a tidy operation with
no targets. This results in nothing being done, though the user might
reasonably expect some results.
Throw a warning in this case, so the user knows not to expect anything.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Add database plugin metrics around connections
This is a replacement for #15923 that takes into account recent lock
cleanup.
I went ahead and added back in the hanging plugin test, which I meant to
add in #15944 but forgot.
I tested this by spinning up a statsd sink in the tests and verifying I
got a stream of metrics:
```
$ nc -u -l 8125 | grep backend
test.swenson-Q9Q0L72D39.secrets.database.backend.connections.count.pgx.5.:1.000000|g
test.swenson-Q9Q0L72D39.secrets.database.backend.connections.count.pgx.5.:0.000000|g
test.swenson-Q9Q0L72D39.secrets.database.backend.connections.count.pgx.5.:1.000000|g
test.swenson-Q9Q0L72D39.secrets.database.backend.connections.count.pgx.5.:0.000000|g
```
We have to rework the shared gauge code to work without a full
`ClusterMetricSink`, since we don't have access to the core metrics from
within a plugin.
This only reports metrics every 10 minutes by default, but it solves
some problems we would have had with the gauge values becoming stale and
needing to be re-sent.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* Return errors on short PEM bundles (keys, issuers)
When users pass the path of the bundle to the API, rather than the
contents of the bundle (say, by omitting the `@` symbol on a Vault CLI
request), give a better error message indicating to the user what the
potential problem might be. While a larger bound for certificates was
given (75 bytes, likely 100 would be fine as well), a smaller bound had
to be chosen for keys as there's less standard DER encoding data around
them.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add signature_bits to sign-intermediate
This endpoint was lacking the signature_bits field like all the other
endpoints. Notably, in #15478, the ability to customize the intermediate
CSR's signature bits was removed without checking for the ability to
customize the final (root-signed) intermediate certificate's value.
This adds in that missing ability, bringing us parity with root
generation and role-based signing.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add signature_bits to sign-verbatim
This endpoint was also lacking the signature_bits field, preventing
other signature hash functions from being utilized here.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test for revocation under intermediate CA
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow revocation of certs with key-less issuers
In Vault 1.11's multiple issuer functionality, we incorrectly fetched
the full CA signing bundle for validating revocation of leaf certs (when
attempting to prohibit revocation of issuers in the mount). When the
issuer lacked a key (such as the root issuer on an intermediate mount),
this signing bundle creation failed.
Instead of fetching the full CA signing bundle, fetch instead the raw
certutil.CertBundle and parse it (to x509.Certificate form) ourselves.
This manifests as the error on revocation:
> URL: PUT http://127.0.0.1:8200/v1/pki_int/revoke
> * could not fetch the CA certificate for issuer id 156e1b99-4f04-5b5e-0036-cc0422c0c0d3: unable to fetch corresponding key for issuer 156e1b99-4f04-5b5e-0036-cc0422c0c0d3; unable to use this issuer for signing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Cleanup and simplify lock usage in database plugin
Following up from discussions in #15923 and #15933, I wanted to split
out a separate PR that drastically reduced the complexity of the use of
the databaseBackend lock. We no longer need it at all for the
`credRotationQueue`, and we can move it to be solely used in a few,
small connections map management functions.
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* ssh: Fix template regex test for defaultExtensions
- The regex to identify if our defaultExtensions contains a template was
a little too greedy, requiring the entire field to be just the regex. Allow
additional text within the value field to be added
* Add cl
* Add cn_validations PKI Role parameter
This new parameter allows disabling all validations on a common name,
enabled by default on sign-verbatim and issuer generation options.
Presently, the default behavior is to allow either an email address
(denoted with an @ in the name) or a hostname to pass validation.
Operators can restrict roles to just a single option (e.g., for email
certs, limit CNs to have strictly email addresses and not hostnames).
By setting the value to `disabled`, CNs of other formats can be accepted
without validating their contents against our minimal correctness checks
for email/hostname/wildcard that we typically apply even when broad
permissions (allow_any_name=true, enforce_hostnames=false, and
allow_wildcard_certificates=true) are granted on the role.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update PKI tests for cn_validation support
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PKI API documentation on cn_validations
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor role issuance tests to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 5.879s
After:
github.com/hashicorp/vault/builtin/logical/pki 1.063s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor role key bit tests to use direct backend
Also removes redundant cases.
Before:
github.com/hashicorp/vault/builtin/logical/pki 136.605s
After:
github.com/hashicorp/vault/builtin/logical/pki 24.713s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor common name test to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.767s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.611s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor device cert tests to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.725s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.402s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor invalid parameter test to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 3.777s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.021s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor Alt Issuer tests to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.560s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.111s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor root idempotency tests to use direct backend
As a result, we've had to import a root cert from elsewhere in the test
suite, rather than using the one off the cluster.
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.399s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.523s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move PKI direct backend helpers to common location
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor OID SANs test to direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 5.284s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.808s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor allowed serial numbers test to direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.789s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.600s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor URI SANs to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.245s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.600s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor Full Chain CA tests to direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 14.503s
After:
github.com/hashicorp/vault/builtin/logical/pki 2.082s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update Allow Past CA tests to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.323s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.322s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Convert existing-key root test to direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.430s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.370s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor CRL enable/disable tests to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 5.738s
After:
github.com/hashicorp/vault/builtin/logical/pki 2.482s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update intermediate existing key tests to use direct backend
Before:
github.com/hashicorp/vault/builtin/logical/pki 4.182s
After:
github.com/hashicorp/vault/builtin/logical/pki 0.416s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor Issuance TTL verification tests to use direct backend
Also shorten sleep duration slightly by precisely calculating it
relative to the actual cert life time.
Before:
github.com/hashicorp/vault/builtin/logical/pki 19.755s
After:
github.com/hashicorp/vault/builtin/logical/pki 11.521s
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow reading Nomad CA/Client cert configuration
In the Nomad secret engine, writing to /nomad/config/access allows users
to specify a CA certificate and client credential pair. However, these
values are not in the read of the endpoint, making it hard for operators
to see if these values were specified and if they need to be rotated.
Add `ca_cert` and `client_cert` parameters to the response, eliding the
`client_key` parameter as it is more sensitive (and should most likely
be replaced at the same time as `client_cert`).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix tests to expect additional fields
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test with existing CA/client cert+key
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Handle func
* Update - check if key_type and key_bits are allowed
* Update - fields
* Generating keys based on provided key_type and key_bits
* Returning signed key
* Refactor
* Refactor update to common logic function
* Descriptions
* Tests added
* Suggested changes and tests added and refactored
* Suggested changes and fmt run
* File refactoring
* Changelog file
* Update changelog/15561.txt
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Suggested changes - consistent returns and additional info to test messages
* ssh issue key pair documentation
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
- Selecting a constant default value exposed a possible edge case
that the migration would fail if a previous migration contained the
same issuer or key name.
* Add parsing for NSS-wrapped Ed25519 keys
NSS wraps Ed25519 using the PKCS#8 standard structure. The Go standard
library as of Go 1.18.x doesn't support parsing this key type with the
OID used by NSS; it requires the 1.3.101.112/RFC 8410 format, rather
than the RFC 5915-esque structure supported here.
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add integration test with NSS-created wrapped key
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
* Support for CPS URLs in Custom Policy Identifiers.
* go fmt
* Add Changelog
* Fix panic in test-cases.
* Update builtin/logical/pki/path_roles.go
Fix intial nil identifiers.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Make valid policy OID so don't break ASN parse in test.
* Add test cases.
* go fmt.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
- Do not set the first issuer we attempt to import as the default issuer unless
it has a corresponding key.
- Add the ability to set a default issuer if none exist and we import it's corresponding key after the fact.
- Add a warning to an end-user if we imported multiple issuers with keys and we
choose one of them as the default value.
Update AWS auth method certificates
Add tests that the `rsa2048` document can also be verified using the
`pkcs7` field for AWS auth.
Due to the use of SHA-1-based signatures for the `identity` and `pkcs7`
methods, we want to encourage moving toward using the RSA 2048 workflow,
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/verify-rsa2048.html
This doesn't require code changes for Vault necessarily, but adding in
the (many) certificates will help end users.
Also adds `rsa2048` option to API to fetch the RSA 2048 signature.
I will make a PR to update to the AWS auth docs to document the RSA 2048
flow soon after this.
* Add integration tests for aliased PKI paths (root/rotate, root/replace)
- Add tests for the two api endpoints
- Also return the issuer_name field within the generate root api response
* Add key_name to generate root api endpoint response and doc updates
- Since we are now returning issuer_name, we should also return key_name
- Update the api-docs for the generate root endpoint responses and add
missing arguments that we accept.
* WIP replacing lib/pq
* change timezome param to be URI format
* add changelog
* add changelog for redshift
* update changelog
* add test for DSN style connection string
* more parseurl and quoteidentify to sdk; include copyright and license
* call dbutil.ParseURL instead, fix import ordering
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Allow role-based sign-verbatim with chosen issuer
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning with missing requested verbatim role
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update builtin/logical/pki/backend.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Fix handling of username_as_alias during LDAP authentication
There is a bug that was introduced in the LDAP authentication method by https://github.com/hashicorp/vault/pull/11000.
It was thought to be backward compatible but has broken a number of users. Later
a new parameter `username_as_alias` was introduced in https://github.com/hashicorp/vault/pull/14324
to make it possible for operators to restore the previous behavior.
The way it is currently working is not completely backward compatible thought
because when username_as_alias is set, a call to GetUserAliasAttributeValue() will
first be made, then this value is completely discarded in pathLogin() and replaced
by the username as expected.
This is an issue because it makes useless calls to the LDAP server and will break
backward compatibility if one of the constraints in GetUserAliasAttributeValue()
is not respected, even though the resulting value will be discarded anyway.
In order to maintain backward compatibility here we have to only call
GetUserAliasAttributeValue() if necessary.
Since this change of behavior was introduced in 1.9, this fix will need to be
backported to the 1.9, 1.10 and 1.11 branches.
* Add changelog
* Add tests
* Format code
* Update builtin/credential/ldap/backend.go
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Format and fix declaration
* Reword changelog
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Add tests for role patching
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Prevent bad issuer names on update
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on PATCH operations
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add a warning when Issuing Certificate set on a role does not resolve.
* Ivanka's requests - add a warning on deleting issuer or changing it's name.
* Fix nil checks; reduce number of roles to iterate through; only verify roles after migration.
* Fix semgrep failure, ignore roles deleted behind our back.
* Patch functionality for roles
* Make Patch Roles work again, add back patch issuers.
* Add changelog.
* Fix nil-reversion on empty response.
* Panics are bad. don't do that.
* Return signed ca as part of ca_chain field within sign-intermediate
- When signing a CA certificate we should include it along with the signing CA's CA chain in the response.
* PKI - Add not_before_duration API parameter to:
- Root CA generation
- Intermediate CA generation
- Intermediate CA signing
* Move not_before_duration to addCACommonFields
This gets applied on both root generation and intermediate signing,
which is the correct place to apply this.
Co-authored-by: guysv <sviryguy@gmail.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Resolves: #10631
Co-authored-by: guysv <sviryguy@gmail.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case for root/generate, sign-intermediate
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update path role description
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new not_before_duration to relevant docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: guysv <sviryguy@gmail.com>
* Add warning on missing AIA info fields
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog:
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add a warning when Issuing Certificate set on a role does not resolve.
* Ivanka's requests - add a warning on deleting issuer or changing it's name.
* reduce number of roles to iterate through; only verify roles after migration. ignore roles deleted behind our back.
* Protect against key and issuer name re-use
- While importing keys and issuers verify that the provided name if any has not been used by another key that we did not match against.
- Validate an assumption within the key import api, that we were provided a single key
- Add additional tests on the new key generation and key import handlers.
* Protect key import api end-users from using "default" as a name
- Do not allow end-users to provide the value of default as a name for key imports
as that would lead to weird and wonderful behaviors to the end-user.
* Add missing api-docs for PKI key import
* Warn on empty Subject field for issuers
When generating a root or signing an intermediate certificate, it is
possible to have Vault generate a certificate with an empty Subject.
These don't validate in most TLS implementations well, so add a warning.
Note that non-Common Name fields could be present to make a non-empty
subject, so simply requiring a CommonName isn't strictly the best.
For example:
$ vault write pki/root/generate/exported common_name=""
WARNING! The following warnings were returned from Vault:
* This issuer certificate was generated without a Subject; this makes
it likely that issuing leaf certs with this certificate will cause TLS
validation libraries to reject this certificate.
....
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove signature_bits on intermediate generate
This extraneous field wasn't respected during intermediate generation
and it isn't clear that it should be. Strictly, this field, if it were
to exist, would control the CSR's internal signature algorithm (certutil
defaults to the sane SHA-256 here). However, there's little value in
changing this as the signing authority can and probably will override
the final certificate's signature bits value, completely ignoring
whatever was in the provided CSR.
Removing this field will now cause warnings for those providing the
parameter (which already wasn't respected), which is the desired
behavior. No breakage should occur as a result of this change.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Always return non-nil CRL configuration
When using the default CRL configuration (as none has been set), return
the default configuration rather than inferring it in buildCRL. This
additionally allows us to return the default configuration on GET
operations to /config/crl.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Always return non-nil URL configuration
When using the default (empty) URL configuration as none has been set,
return the default configuration rather than inferring it inside of
fetchCAInfoByIssuerId or generateCert. This additionally allows us to
return the default configuration on GET operations to /config/urls.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* secret/pki: Return correct algorithm type from key fetch api for managed keys
- fix an issue that key_type field returned from the key fetch api had
the ManagedPrivateKey type instead of the real algorithm of the managed key.
* Remove key_type from key list PKI operation. Partial revert of #15435
- The key_type field should be used solely for the key algorithm but as implemented
we would be returning the value ManagedPrivateKey for managed keys which is not
in sync with the rest of the apis. We also did not want to take the performance
hit if many managed keys existed so we will simply remove the field from the list
operation
- No point in writing any logs if no previous bundle exists
- Only log output and schedule a CRL rebuild is we actually migration something
- Do not log on PKI storage version set/checks.
