* Fix `vault path-help` for selected paths with bad regexps
See the comment being added in `sdk/framework/path.go` for the
explanation of why this change is needed.
* Grammar fix and add changelog
* Also fix hardcoded expectations in a new test
* Add a couple more testcases, and some comments.
* Tweak spelling in comment
Vault API endpoints are defined using regexes in instances of the SDK's
framework.Path structure. However, OpenAPI does not use regexes, so a
translation is performed. It is technically possible that this
translation produces colliding OpenAPI paths from multiple
framework.Path structures. When this happens, there has formerly been no
diagnostic, and one result silently overwrites the other in a map.
As a result of this, several operations are currently accidentally
missing from the Vault OpenAPI, which is also the trigger for
https://github.com/hashicorp/vault-client-go/issues/180.
This PR adds a log message, to help catch such accidents so that they
can be fixed. Much of the PR is propagating a logger to the point where
it is needed, and adjusting tests for the API change.
With current Vault, this will result in the following being logged each
time a request is made which triggers OpenAPI generation:
```
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/totp
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/okta
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/duo
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/pingid
```
I will submit a further PR to fix the issue - this one is just to add
the diagnostic.
* Replace all time.ParseDurations with testutil.ParseDurationSeconds
* Changelog
* Import formatting
* Import formatting
* Import formatting
* Import formatting
* Semgrep rule that runs as part of CI
Add paths for seal config to cache exceptions.
Add barrierSealConfigPath and recoverySealConfigPlaintextPath to
cacheExceptionsPaths in order to avoid a race that causes some
nodes to always see a nil value.
* Allow CSRs with basic constraint extension with IsCA=false
- We previously forbid any CSR with a basic constraint extension within the CSR.
- It was discovered that some ACME clients (Proxmox ACME client) do send us this extension with a value of IsCA to false.
- So allow the extension to be set within the ACME CSR with
a value of IsCA set to false
- Add a test for both the IsCA=true and IsCA=false use-cases and make sure we don't actually set the extension back within the generated certificate.
* PR feedback
- Move basic constraint function to sdk, increase test coverage
- Error out on extra characters being returned from the asn1 unmarshalling.
* make fmt
* Fix non-deterministic ordering of 'required' field in OpenAPI spec
Fixes a minor annoyance I discovered whilst comparing before and after
OpenAPI specs whilst working on hashicorp/vault-client-go#180.
Sort the entries in a JSON array which has set semantics, after we
construct it by iterating a map (non-deterministic ordering).
* changelog
* Fix export of HMAC typed keys
When initially implemented, exporting HMAC keys resulted in returning
the unused, internal HMACKey value rather than the main Key value that
is used for HMAC operations.
This is a breaking change.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Consistently handle HMAC keys in keysutil
When generating HMAC-typed keys, set HMACKey = Key consistently, to
allow users of HMAC-typed keys to use them backwards compatibly.
Notably, this could discard the (unused) HMACKey field set today.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test proving export of HMAC keys work
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add WrapKey capabilities to keysutil
This allows one keysutil to wrap another key, assuming that key has an
type matching one of keysutil's allowed KeyTypes. This allows completing
the BYOK import loop with Transit, allowing imported wrapping keys to
export (wrap) other keys in transit, without having them leave in
plaintext.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /byok-export/:dst/:src[/:version] to Transit
Still respecting exportable, we allow encrypted-only export of transit
keys to another cluster using the BYOK semantics.
In particular, this allows an operator to securely establish key
material between two separate Transit installations. This potentially
allows one cluster to be used as a source cluster (encrypting a large
amount of data) and a second cluster to decrypt this data later. This
might be useful in hybrid or site-specific deployments of Vault for
instance.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing dependency to sdk/, vault/
Also updates to a newer version while we're here.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on BYOK export
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for BYOK export/import
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/transit.mdx
* Update builtin/logical/transit/path_byok.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* poc: client cert credential type
* go mod tidy
* fix typo
* fix newUserReqToProto
* add changelog
* add newline
* fix changelog
* add test for the client cert generator
* Fix formatting
* unset signing bundle URLs
* set BasicConstraintsValidForNonCA to false
* backdate cert by 30s
* remove empty creation params URLs
* check cert BasicConstraintsValid
* set default key bits in newClientCertificateGenerator
* fix client cert gen test with default values
* Add default for key_type
* fix default key_type
* update test with default key type
* update test
* Update changelog/20425.txt
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
* set default key bits and sig bits
* remove the default for key type ad fix the test
* make fmt + add comments for each exported field
* restart test
---------
Co-authored-by: Austin Gebauer <agebauer@hashicorp.com>
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
* Allow public keys to be always exported
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Simplify import key version semantics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for updated ImportVersion semantics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix handling of RSA private key pairs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refine documentation for public_key
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support additional key types in importing version
This originally left off the custom support for Ed25519 and RSA-PSS
formatted keys that we've added manually.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for Ed25519 keys
Here, we prevent importing public-key only keys with derived Ed25519
keys. Notably, we still allow import of derived Ed25519 keys via private
key method, though this is a touch weird: this private key must have
been packaged in an Ed25519 format (and parseable through Go as such),
even though it is (strictly) an HKDF key and isn't ever used for Ed25519.
Outside of this, importing non-derived Ed25519 keys works as expected.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key only export method to Transit
This allows the existing endpoints to retain private-key only, including
empty strings for versions which lack private keys. On the public-key
endpoint, all versions will have key material returned.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tests for exporting via public-key interface
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key export option to docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Build a better nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add internal nonce service for testing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add benchmarks for nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add statistics around how long tidy took
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Replace ACME nonces with shared nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add an initialize method to nonce services
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use the new initialize helper on nonce service in PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional tests for nonces
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Format sdk/helper/nonce
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use default 90s nonce expiry in PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove parallel test case as covered by benchmark
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional commentary to encrypted nonce implementation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add nonce to test_packages
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* pki: add subject key identifier to read key response
This will be helpful for the Terraform Vault Provider to detect
migration of pre-1.11 exported keys (from CA generation) into post-1.11
Vault.
* add changelog
* Update builtin/logical/pki/path_fetch_keys.go
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* check for managed key first
* Validate the SKID matches on root CAs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate SKID matches on int CAs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix formatting of tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Do not set use_csr_values when issuing ACME certs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure CSRs with Basic Constraints are rejected
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test to ensure CA certificates cannot be issued
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update builtin/logical/pkiext/pkiext_binary/acme_test.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Update builtin/logical/pkiext/pkiext_binary/acme_test.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Update acme_test.go to include certutil
* Update acme_test.go - unused imports, reformat
* Update acme_test.go - hex really was used
This is why I can't use the GH web editor. :-)
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add stub ACME billing interfaces
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add initial implementation of client count
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly attribute to mount, namespace
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor adding entities of custom types
This begins to add custom types of events; presently these are counted
as non-entity tokens, but prefixed with a custom ClientID prefix.
In the future, this will be the basis for counting these events
separately (into separate buckets and separate storage segments).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of ACME mounts
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case for billing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better support managed key system view casting
Without an additional parameter, SystemView could be of a different
internal implementation type that cannot be directly casted to in OSS.
Use a separate parameter for the managed key system view to use instead.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of mounts for enterprise
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate mounts in ACME billing tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use a hopefully unique separator for encoded identifiers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use mount accesor, not path
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename AddEventToFragment->AddActivityToFragment
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
* VAULT-15547 First pass at agent/proxy decoupling
* VAULT-15547 Fix some imports
* VAULT-15547 cases instead of string.Title
* VAULT-15547 changelog
* VAULT-15547 Fix some imports
* VAULT-15547 some more dependency updates
* VAULT-15547 More dependency paths
* VAULT-15547 godocs for tests
* VAULT-15547 godocs for tests
* VAULT-15547 test package updates
* VAULT-15547 test packages
* VAULT-15547 add proxy to test packages
* VAULT-15547 gitignore
* VAULT-15547 address comments
* VAULT-15547 Some typos and small fixes
* import rsa and ecdsa public keys
* allow import_version to update public keys - wip
* allow import_version to update public keys
* move check key fields into func
* put private/public keys in same switch cases
* fix method in UpdateKeyVersion
* move asymmetrics keys switch to its own method - WIP
* test import public and update it with private counterpart
* test import public keys
* use public_key to encrypt if RSAKey is not present and failed to decrypt
if key version does not have a private key
* move key to KeyEntry parsing from Policy to KeyEntry method
* move extracting of key from input fields into helper function
* change back policy Import signature to keep backwards compatibility and
add new method to import private or public keys
* test import with imported public rsa and ecdsa keys
* descriptions and error messages
* error messages, remove comments and unused code
* changelog
* documentation - wip
* suggested changes - error messages/typos and unwrap public key passed
* fix unwrap key error
* fail if both key fields have been set
* fix in extractKeyFromFields, passing a PolicyRequest wouldn't not work
* checks for read, sign and verify endpoints so they don't return errors when a private key was not imported and tests
* handle panic on "export key" endpoint if imported key is public
* fmt
* remove 'isPrivateKey' argument from 'UpdateKeyVersion' and
'parseFromKey' methods
also: rename 'UpdateKeyVersion' method to 'ImportPrivateKeyForVersion' and 'IsPublicKeyImported' to 'IsPrivateKeyMissing'
* delete 'RSAPublicKey' when private key is imported
* path_export: return public_key for ecdsa and rsa when there's no private key imported
* allow signed data validation with pss algorithm
* remove NOTE comment
* fix typo in EC public key export where empty derBytes was being used
* export rsa public key in pkcs8 format instead of pkcs1 and improve test
* change logic on how check for is private key missing is calculated
---------
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Export DockerAPI for use by other consumers
As usage of DockerCluster gets more advanced, some users may want to
interact with the container nodes of the cluster. While, if you already
have a DockerAPI instance lying around you can reuse that safely, for
use cases where an existing e.g., docker/testhelpers's runner instance
is not available, reusing the existing cluster's DockerAPI is easiest.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ability to exec commands without runner
When modifying DockerTestCluster's containers manually, we might not
have a Runner instance; instead, expose the ability to run commands via
a DockerAPI instance directly, as they're awfully convenient.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS resolver into ACME tests
This updates the pkiext_binary tests to use an adjacent DNS resolver,
allowing these tests to eventually be extended to solve DNS challenges,
as modifying the /etc/hosts file does not allow this.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix loading DNS resolver onto network
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bug with DNS configuration validation
Both conditionals here were inverted: address being empty means a bad
specification was given, and the parse being nil means that it was not a
valid IP address.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix specifying TXT records, allow removing records
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Handle caching of ACME config
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS resolvers to ACME configuration
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add custom DNS resolver to challenge verification
This required plumbing through the config, reloading it when necessary,
and creating a custom net.Resolver instance.
Not immediately clear is how we'd go about building a custom DNS
validation mechanism that supported multiple resolvers. Likely we'd need
to rely on meikg/dns and handle the resolution separately for each
container and use a custom Dialer that assumes the address is already
pre-resolved.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Improvements to Docker harness
- Expose additional service information, allowing callers to figure out
both the local address and the network-specific address of the
service container, and
- Allow modifying permissions on uploaded container files.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add infrastructure to run Bind9 in a container for tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate DNS-01 challenge works
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* remove intercepting helpText
* add subtext directly to StringList input component
* update tests and add coverage for new openapi-attrs util
* update test
* add warning validation to input
* lol is this right i dont know go
* literally no idea what im doing
* add Description to display attrs struct
* update struct comment
* add descriptions to remaining go fields
* add missing comma
* remaining commas..."
* add description to display attrs
* update tests
* update tests
* add changelog;
* Update ui/app/utils/openapi-to-attrs.js
* update tests following backend changes
* clearly name variable
* format files
* no longer need to test for modified tooltip since coming from backend now
* Add fix for Go x/crypto/ocsp failure case
When calling ocsp.ParseRequest(req, issue) with a non-nil issuer on a
ocsp request which _unknowingly_ contains an entry in the
BasicOCSPResponse's certs field, Go incorrectly assumes that the issuer
is a direct parent of the _first_ certificate in the certs field,
discarding the rest.
As documented in the Go issue, this is not a valid assumption and thus
causes OCSP verification to fail in Vault with an error like:
> bad OCSP signature: crypto/rsa: verification error
which ultimately leads to a cert auth login error of:
> no chain matching all constraints could be found for this login certificate
We address this by using the unsafe issuer=nil argument, taking on the
task of validating the OCSP response's signature as best we can in the
absence of full chain information on either side (both the trusted
certificate whose OCSP response we're verifying and the lack of any
additional certs the OCSP responder may have sent).
See also: https://github.com/golang/go/issues/59641
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case with Vault PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
