* Ignore nonces when encrypting without convergence or with convergence versions > 1
* Honor nonce use warning in non-FIPS modes
* Revert "Honor nonce use warning in non-FIPS modes"
This reverts commit 2aee3dbdc1.
* Add a test func that removes a nonce when not needed
* err out rather than ignore the nonce
* Alter unit test to cover, also cover convergent version 3
* More unit test work
* Fix test 14
* changelog
* tests not already in a nonce present path
* Update unit test to not assume warning when nonce provided incorrectly
* remove unused test field
* Fix auto-squash events experiments
When #22835 was merged, it was auto-squashed, so the `experiments`
import was removed, but the test still referenced it.
This removes the (now unnecessary) experiment from the test.
* Allow nonces for managed keys, because we have no way of knowing if the backing cipher/mode needs one
---------
Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
* Also makes plugin directory optional when registering container plugins
* And threads plugin runtime settings through to plugin execution config
* Add runsc to github runner for plugin container tests
* Fix transit panic with invalid PEM
When an invalid (non-PEM) public key is given to Transit's import, this
fails with a panic in server logs:
2023-09-05T08:11:11.526-0400 [INFO] http: panic serving 127.0.0.1:42414: runtime error: invalid memory address or nil pointer dereference
goroutine 950 [running]:
net/http.(*conn).serve.func1()
/usr/local/go/src/net/http/server.go:1868 +0xb9
panic({0x8371620?, 0x1050b390?})
/usr/local/go/src/runtime/panic.go:920 +0x270
github.com/hashicorp/vault/sdk/helper/keysutil.(*Policy).ImportPublicOrPrivate(0xc003fff440, {0xaf02918, 0xc004509920}, {0xaf03670, 0xc0032e4180}, {0xc004532ea0, 0x188, 0x1a0}, 0x0, {0xae7f5e0, ...})
/home/cipherboy/GitHub/cipherboy/vault/sdk/helper/keysutil/policy.go:1538 +0x687
github.com/hashicorp/vault/sdk/helper/keysutil.(*LockManager).ImportPolicy(0xc001a29410, {0xaf02918, 0xc004509920}, {{0xaf03670, 0xc0032e4180}, {0xc003eb5ab5, 0xb}, 0x3, 0x0, 0x0, ...}, ...)
/home/cipherboy/GitHub/cipherboy/vault/sdk/helper/keysutil/lock_manager.go:517 +0x38a
This is unfortunate and doesn't reveal the cause of the failure: input
was not provided in PEM format, per docs.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix additional PEM decode without error check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Implements running plugins in containers to give them some degree
of isolation from the main Vault process and other plugins. It only
supports running on Linux initially, where it is easiest to manage unix
socket communication across the container boundary.
Additionally
* Adds -env arg to vault plugin register.
* Don't return env from 'vault plugin info'
Historically it's been omitted, and it could conceivably have secret information in
it, so if we want to return it in the response, it should probably only be via explicit
opt-in. Skipping for now though as it's not the main purpose of the commit.
* Remove note
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove duplicate curve check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove CreateOperation from new Transit X.509 endpoints
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test to validate key matching works
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* setup initial boilerplate code for sign csr endpoint
* add function to sign csr
* working version of sign csr endpoint
* improving errors for csr create and sign endpoint
* initial implementation for import leaf certificate endpoint
* check if more than one certificate was provided in the ceritificate chain
* improve validate cert public key matches transit key
* convert provided cert chain from PEM to DER so it can be parsed by
x509.ParseCertificates and fixing other bugs
* fix creation of csr from csrTemplate
* add missing persist of certificate chain after validations in set-certificate endpoint
* allow exporting a certificate-chain
* move function declaration to end of page
* improving variable and function names, removing comments
* fix certificate chain parsing - work in progress
* test for signCsr endpoint
* use Operations instead of Callbacks in framework.Path
* setup test for set-certificate endpoint
fix problems with sign-csr endpoint returning base64
* finish set-certificate endpoint test
* use public key KeyEntry fields instead of retrieving public key from private
* improve error message and make better distinction between client and server error
also moved check of key types before checking if key match to endpoint handler
* check if private key has been imported for key version selected when signing a csr
* improve errors
* add endpoint description and synopsis
* fix functions calls in backend as function names changed
* improve import cert chain test
* trim whitespaces on export certificate chain
* changelog
* pass context from handler function to policy Persist
* make fmt run
* fix: assign returned error from PersistCertificateChain to err so it can be evaluated
* additional validations and improvements to parseCertificateChain function
* add validation to check if there is only one certificate in the certificate chain and it is in the first position
* import cert chain test: move creation of cluster to exported test function
* move check of end-cert pub key algorithm and key transit algorithm match into a separate function
* test export certificate chain
* Update sdk/helper/keysutil/policy.go
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* fix validateLeafCertPosition
* reject certificate actions on policies that allow key derivation and remove derived checks
* return UserError from CreateCSR SDK function as 400 in transit API handler
* add derived check for ED5519 keys on CreateCSR SDK func
* remove unecessary calls of x509.CreateCertificateRequest
* move validate key type match back into SDK ValidateLeafCertMatch function
* add additional validations (ValidateLeafCertKeyMatch, etc) in SDK PersistCertificateChain function
* remove uncessary call of ValidateLeafCertKeyMatch in parseImportCertChainWrite
* store certificate chain as a [][]byte instead of []*x509.Certificate
* include persisted ca chain in import cert-chain response
* remove NOTE comment
* allow exporting cert-chain even if exportable is set as false
* remove NOTE comment
* add certifcate chain to formatKeyPublic if present
also added an additional check to validate if field is added when
certchain is present
---------
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Initial oss-patch apply
* Added changelog
* Renamed changelog txt
* Added the imports to the handler file
* Added a check that no two ports are the same, and modified changelog
* Edited go sum entry
* Tidy up using go mod
* Use strutil instead
* Revert go sum and go mod
* Revert sdk go sum
* Edited go.sum to before
* Edited go.sum again to initial
* Revert changes
* Refactor sign-intermediate API response
- Allow the sign-intermediate response handling code to be shared
across different API calls.
* Add missing cieps.go
* Migrate protobuf generation to Buf
Buf simplifies the generation story and allows us to lean
into other features in the Buf ecosystem, such as dependency
management, linting, breaking change detection, formatting
and remote plugins.
* Format all protobuf files with buf
Also add a CI job to ensure formatting remains consistent
* Add CI job to warn on proto generate diffs
Some files were not regenerated with the latest version
of the protobuf binary. This CI job will ensure we are always
detect if the protobuf files need regenerating.
* Add CI job for linting protobuf files
* OpenAPI: Define default response structure for ListOperations
Almost all Vault ListOperation responses have an identical response
schema. Update the OpenAPI generator to know this, and remove a few
instances where that standard response schema had been manually
copy/pasted into place in individual endpoints.
* changelog
* Only render StandardListResponse schema, if an operation uses it
* Teach the response schema validation test helper about the default list schema too
---------
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* OSS: Add standard CIEPS request/response structs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* OSS: Add support for parsing TLS-related values
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Replace all time.ParseDurations with testutil.ParseDurationSeconds
* Changelog
* Import formatting
* Import formatting
* Import formatting
* Import formatting
* Semgrep rule that runs as part of CI
* Allow CSRs with basic constraint extension with IsCA=false
- We previously forbid any CSR with a basic constraint extension within the CSR.
- It was discovered that some ACME clients (Proxmox ACME client) do send us this extension with a value of IsCA to false.
- So allow the extension to be set within the ACME CSR with
a value of IsCA set to false
- Add a test for both the IsCA=true and IsCA=false use-cases and make sure we don't actually set the extension back within the generated certificate.
* PR feedback
- Move basic constraint function to sdk, increase test coverage
- Error out on extra characters being returned from the asn1 unmarshalling.
* make fmt
* Fix export of HMAC typed keys
When initially implemented, exporting HMAC keys resulted in returning
the unused, internal HMACKey value rather than the main Key value that
is used for HMAC operations.
This is a breaking change.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Consistently handle HMAC keys in keysutil
When generating HMAC-typed keys, set HMACKey = Key consistently, to
allow users of HMAC-typed keys to use them backwards compatibly.
Notably, this could discard the (unused) HMACKey field set today.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test proving export of HMAC keys work
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add WrapKey capabilities to keysutil
This allows one keysutil to wrap another key, assuming that key has an
type matching one of keysutil's allowed KeyTypes. This allows completing
the BYOK import loop with Transit, allowing imported wrapping keys to
export (wrap) other keys in transit, without having them leave in
plaintext.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /byok-export/:dst/:src[/:version] to Transit
Still respecting exportable, we allow encrypted-only export of transit
keys to another cluster using the BYOK semantics.
In particular, this allows an operator to securely establish key
material between two separate Transit installations. This potentially
allows one cluster to be used as a source cluster (encrypting a large
amount of data) and a second cluster to decrypt this data later. This
might be useful in hybrid or site-specific deployments of Vault for
instance.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing dependency to sdk/, vault/
Also updates to a newer version while we're here.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on BYOK export
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for BYOK export/import
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/transit.mdx
* Update builtin/logical/transit/path_byok.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Allow public keys to be always exported
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Simplify import key version semantics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for updated ImportVersion semantics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix handling of RSA private key pairs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refine documentation for public_key
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support additional key types in importing version
This originally left off the custom support for Ed25519 and RSA-PSS
formatted keys that we've added manually.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for Ed25519 keys
Here, we prevent importing public-key only keys with derived Ed25519
keys. Notably, we still allow import of derived Ed25519 keys via private
key method, though this is a touch weird: this private key must have
been packaged in an Ed25519 format (and parseable through Go as such),
even though it is (strictly) an HKDF key and isn't ever used for Ed25519.
Outside of this, importing non-derived Ed25519 keys works as expected.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key only export method to Transit
This allows the existing endpoints to retain private-key only, including
empty strings for versions which lack private keys. On the public-key
endpoint, all versions will have key material returned.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tests for exporting via public-key interface
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key export option to docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Build a better nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add internal nonce service for testing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add benchmarks for nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add statistics around how long tidy took
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Replace ACME nonces with shared nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add an initialize method to nonce services
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use the new initialize helper on nonce service in PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional tests for nonces
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Format sdk/helper/nonce
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use default 90s nonce expiry in PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove parallel test case as covered by benchmark
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional commentary to encrypted nonce implementation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add nonce to test_packages
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* pki: add subject key identifier to read key response
This will be helpful for the Terraform Vault Provider to detect
migration of pre-1.11 exported keys (from CA generation) into post-1.11
Vault.
* add changelog
* Update builtin/logical/pki/path_fetch_keys.go
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* check for managed key first
* Validate the SKID matches on root CAs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate SKID matches on int CAs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix formatting of tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Do not set use_csr_values when issuing ACME certs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure CSRs with Basic Constraints are rejected
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test to ensure CA certificates cannot be issued
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update builtin/logical/pkiext/pkiext_binary/acme_test.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Update builtin/logical/pkiext/pkiext_binary/acme_test.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Update acme_test.go to include certutil
* Update acme_test.go - unused imports, reformat
* Update acme_test.go - hex really was used
This is why I can't use the GH web editor. :-)
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* VAULT-15547 First pass at agent/proxy decoupling
* VAULT-15547 Fix some imports
* VAULT-15547 cases instead of string.Title
* VAULT-15547 changelog
* VAULT-15547 Fix some imports
* VAULT-15547 some more dependency updates
* VAULT-15547 More dependency paths
* VAULT-15547 godocs for tests
* VAULT-15547 godocs for tests
* VAULT-15547 test package updates
* VAULT-15547 test packages
* VAULT-15547 add proxy to test packages
* VAULT-15547 gitignore
* VAULT-15547 address comments
* VAULT-15547 Some typos and small fixes
* import rsa and ecdsa public keys
* allow import_version to update public keys - wip
* allow import_version to update public keys
* move check key fields into func
* put private/public keys in same switch cases
* fix method in UpdateKeyVersion
* move asymmetrics keys switch to its own method - WIP
* test import public and update it with private counterpart
* test import public keys
* use public_key to encrypt if RSAKey is not present and failed to decrypt
if key version does not have a private key
* move key to KeyEntry parsing from Policy to KeyEntry method
* move extracting of key from input fields into helper function
* change back policy Import signature to keep backwards compatibility and
add new method to import private or public keys
* test import with imported public rsa and ecdsa keys
* descriptions and error messages
* error messages, remove comments and unused code
* changelog
* documentation - wip
* suggested changes - error messages/typos and unwrap public key passed
* fix unwrap key error
* fail if both key fields have been set
* fix in extractKeyFromFields, passing a PolicyRequest wouldn't not work
* checks for read, sign and verify endpoints so they don't return errors when a private key was not imported and tests
* handle panic on "export key" endpoint if imported key is public
* fmt
* remove 'isPrivateKey' argument from 'UpdateKeyVersion' and
'parseFromKey' methods
also: rename 'UpdateKeyVersion' method to 'ImportPrivateKeyForVersion' and 'IsPublicKeyImported' to 'IsPrivateKeyMissing'
* delete 'RSAPublicKey' when private key is imported
* path_export: return public_key for ecdsa and rsa when there's no private key imported
* allow signed data validation with pss algorithm
* remove NOTE comment
* fix typo in EC public key export where empty derBytes was being used
* export rsa public key in pkcs8 format instead of pkcs1 and improve test
* change logic on how check for is private key missing is calculated
---------
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Export DockerAPI for use by other consumers
As usage of DockerCluster gets more advanced, some users may want to
interact with the container nodes of the cluster. While, if you already
have a DockerAPI instance lying around you can reuse that safely, for
use cases where an existing e.g., docker/testhelpers's runner instance
is not available, reusing the existing cluster's DockerAPI is easiest.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ability to exec commands without runner
When modifying DockerTestCluster's containers manually, we might not
have a Runner instance; instead, expose the ability to run commands via
a DockerAPI instance directly, as they're awfully convenient.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS resolver into ACME tests
This updates the pkiext_binary tests to use an adjacent DNS resolver,
allowing these tests to eventually be extended to solve DNS challenges,
as modifying the /etc/hosts file does not allow this.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix loading DNS resolver onto network
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bug with DNS configuration validation
Both conditionals here were inverted: address being empty means a bad
specification was given, and the parse being nil means that it was not a
valid IP address.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix specifying TXT records, allow removing records
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
