- The key update API would release the lock a little too early
after it persisted the update so the reference could be updated
when it was preparing the response to the caller across updates
and/or key rotations
- The storage updates were okay, just the response back to the caller
of the update might see a mixture of different updates
* [transit-pkcs1v15] transit support for the pkcs1v15 padding scheme – without UI tests (yet).
* [transit-pkcs1v15] renamed padding_scheme parameter in transit documentation.
* [transit-pkcs1v15] add changelog file.
* [transit-pkcs1v15] remove the algorithm path as padding_scheme is chosen by parameter.
* Update ui/app/templates/components/transit-key-action/datakey.hbs
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
* Update ui/app/templates/components/transit-key-action/datakey.hbs
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
* Update ui/app/templates/components/transit-key-action/datakey.hbs
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
* Add warnings to PKCS1v1.5 usage
* Update transit
* Update transit, including separating encrypt/decrypt paddings for rewrap
* Clean up factory use in the presence of padding
* address review feedback
* remove defaults
* lint
* more lint
* Some fixes for UI issues
- Fix padding scheme dropdown console error by adding values
to the transit-key-actions.hbs
- Populate both padding scheme drop down menus within rewrap,
not just the one padding_scheme
- Do not submit a padding_scheme value through POST for non-rsa keys
* Fix Transit rewrap API to use decrypt_padding_scheme, encrypt_padding_scheme
- Map the appropriate API fields for the RSA padding scheme to the
batch items within the rewrap API
- Add the ability to create RSA keys within the encrypt API endpoint
- Add test case for rewrap api that leverages the padding_scheme fields
* Fix code linting issues
* simply padding scheme enum
* Apply suggestions from code review
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
* Fix padding_scheme processing on data key api
- The data key api was using the incorrect parameter name for
the padding scheme
- Enforce that padding_scheme is only used on RSA keys, we
are punting on supporting it for managed keys at the moment.
* Add tests for parsePaddingSchemeArg
* Add missing copywrite headers
* Some small UI fixes
* Add missing param to datakey in api-docs
* Do not send padding_scheme for non-RSA key types within UI
* add UI tests for transit key actions form
---------
Co-authored-by: Marcel Lanz <marcellanz@n-1.ch>
Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
Co-authored-by: claire bontempo <cbontempo@hashicorp.com>
Use hash_algorithm parameter on Transit's verify HMAC requests.
Parameter 'algorithm' has been deprecated in favour of 'hash_algorithm', so
update the pathHMACVerify() handler to use it when it is present.
* add key types for cmac for transit key creation
* add test for key creation
* fix test logic and add cases
* fix logic for hmac
* add go doc
* fix key size and add check for HMAC key
* add gosimport to make fmt and run it
* move installation to tools.sh
* correct weird spacing issue
* Update Makefile
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* fix a weird issue
---------
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Transit: Release locks using defer statements
- Leverage defer statements to Unlock the fetched policy
to avoid issues with forgetting to manually Unlock during
each return statement
* Add cl
* Address a panic export RSA public keys in transit
- When attempting to export the public key for an RSA key that
we only have a private key for, the export panics with a nil
deference.
- Add additional tests around Transit key exporting
* Add cl
* Forbid setting auto_rotate_period on transit managed keys
- Prevent and guard against auto-rotating managed keys as we
generate an invalid key version without the uuid field set.
- Hook in the datakey generation api into managed key encryption.
* Add cl
* Ignore nonces when encrypting without convergence or with convergence versions > 1
* Honor nonce use warning in non-FIPS modes
* Revert "Honor nonce use warning in non-FIPS modes"
This reverts commit 2aee3dbdc11c4e333ecb20503539c7993b24ee57.
* Add a test func that removes a nonce when not needed
* err out rather than ignore the nonce
* Alter unit test to cover, also cover convergent version 3
* More unit test work
* Fix test 14
* changelog
* tests not already in a nonce present path
* Update unit test to not assume warning when nonce provided incorrectly
* remove unused test field
* Fix auto-squash events experiments
When #22835 was merged, it was auto-squashed, so the `experiments`
import was removed, but the test still referenced it.
This removes the (now unnecessary) experiment from the test.
* Allow nonces for managed keys, because we have no way of knowing if the backing cipher/mode needs one
---------
Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
The recent additions to the transit secret engine have created two new
endpoints which both have the incorrect (and duplicate) operation ID of
just `"update"`.
Amend to unique meaningful values.
* Remove note
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove duplicate curve check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove CreateOperation from new Transit X.509 endpoints
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test to validate key matching works
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* setup initial boilerplate code for sign csr endpoint
* add function to sign csr
* working version of sign csr endpoint
* improving errors for csr create and sign endpoint
* initial implementation for import leaf certificate endpoint
* check if more than one certificate was provided in the ceritificate chain
* improve validate cert public key matches transit key
* convert provided cert chain from PEM to DER so it can be parsed by
x509.ParseCertificates and fixing other bugs
* fix creation of csr from csrTemplate
* add missing persist of certificate chain after validations in set-certificate endpoint
* allow exporting a certificate-chain
* move function declaration to end of page
* improving variable and function names, removing comments
* fix certificate chain parsing - work in progress
* test for signCsr endpoint
* use Operations instead of Callbacks in framework.Path
* setup test for set-certificate endpoint
fix problems with sign-csr endpoint returning base64
* finish set-certificate endpoint test
* use public key KeyEntry fields instead of retrieving public key from private
* improve error message and make better distinction between client and server error
also moved check of key types before checking if key match to endpoint handler
* check if private key has been imported for key version selected when signing a csr
* improve errors
* add endpoint description and synopsis
* fix functions calls in backend as function names changed
* improve import cert chain test
* trim whitespaces on export certificate chain
* changelog
* pass context from handler function to policy Persist
* make fmt run
* fix: assign returned error from PersistCertificateChain to err so it can be evaluated
* additional validations and improvements to parseCertificateChain function
* add validation to check if there is only one certificate in the certificate chain and it is in the first position
* import cert chain test: move creation of cluster to exported test function
* move check of end-cert pub key algorithm and key transit algorithm match into a separate function
* test export certificate chain
* Update sdk/helper/keysutil/policy.go
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* fix validateLeafCertPosition
* reject certificate actions on policies that allow key derivation and remove derived checks
* return UserError from CreateCSR SDK function as 400 in transit API handler
* add derived check for ED5519 keys on CreateCSR SDK func
* remove unecessary calls of x509.CreateCertificateRequest
* move validate key type match back into SDK ValidateLeafCertMatch function
* add additional validations (ValidateLeafCertKeyMatch, etc) in SDK PersistCertificateChain function
* remove uncessary call of ValidateLeafCertKeyMatch in parseImportCertChainWrite
* store certificate chain as a [][]byte instead of []*x509.Certificate
* include persisted ca chain in import cert-chain response
* remove NOTE comment
* allow exporting cert-chain even if exportable is set as false
* remove NOTE comment
* add certifcate chain to formatKeyPublic if present
also added an additional check to validate if field is added when
certchain is present
---------
Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Updating the license from MPL to Business Source License.
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.
* add missing license headers
* Update copyright file headers to BUS-1.1
* Fix test that expected exact offset on hcl file
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
* CreateOperation should only be implemented alongside ExistenceCheck
Closes#12329
Vault treats all POST or PUT HTTP requests equally - they default to
being treated as UpdateOperations, but, if a backend implements an
ExistenceCheck function, CreateOperations can be separated out when the
existence check returns false.
It follows, then, that if a CreateOperation handler is implemented
without an ExistenceCheck function, this is unreachable code - a coding
error. It's a fairly minor error in the grand scheme of things, but it
causes the generated OpenAPI spec to include x-vault-createSupported for
operations on which create can never actually be invoked - and promotes
muddled understanding of the create/update feature.
In this PR:
1) Implement a new test, which checks all builtin auth methods and
secrets engines can be successfully initialized. (This is important
to validate the next part.)
2) Expand upon the existing coding error checks built in to
framework.Backend, adding a check for this misuse of CreateOperation.
3) Fix up instances of improper CreateOperation within the Vault
repository - just two, transit and mock.
Note: At this point, the newly added test will **fail**.
There are improper uses of CreateOperation in all of the following:
vault-plugin-auth-cf
vault-plugin-auth-kerberos
vault-plugin-auth-kubernetes
vault-plugin-secrets-ad
vault-plugin-secrets-gcpkms
vault-plugin-secrets-kubernetes
vault-plugin-secrets-kv
vault-plugin-secrets-openldap
vault-plugin-secrets-terraform
each of which needs to be fixed and updated in go.mod here, before this
new check can be added.
* Add subtests
* Add in testing of KV v2, which otherwise doesn't get tested
This is a surprisingly complicated special case
* The database plugin needs special handling as well, and add in help invocations of the builtin backends too
* Fix extra package prefix
* Add changelog
* Update 6 out of 9 plugins to needed new versions
Note, this IS an upgrade despite the apparent version numbers going
down. (That's a consequence of slightly odd release management occurring
in the plugin repositories.)
* Update to deal with code changes since branch originally created
* Perform necessary update of vault-plugin-secrets-kubernetes so that CI checks on PR can run
* Fix another instance of incorrect CreateOperation, for a test-only endpoint
By being hidden behind a Go build constraint, it had evaded notice until
now.
* Add an opportunistic test of sys/internal/specs/openapi too
* Fix export of HMAC typed keys
When initially implemented, exporting HMAC keys resulted in returning
the unused, internal HMACKey value rather than the main Key value that
is used for HMAC operations.
This is a breaking change.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Consistently handle HMAC keys in keysutil
When generating HMAC-typed keys, set HMACKey = Key consistently, to
allow users of HMAC-typed keys to use them backwards compatibly.
Notably, this could discard the (unused) HMACKey field set today.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test proving export of HMAC keys work
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add WrapKey capabilities to keysutil
This allows one keysutil to wrap another key, assuming that key has an
type matching one of keysutil's allowed KeyTypes. This allows completing
the BYOK import loop with Transit, allowing imported wrapping keys to
export (wrap) other keys in transit, without having them leave in
plaintext.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add /byok-export/:dst/:src[/:version] to Transit
Still respecting exportable, we allow encrypted-only export of transit
keys to another cluster using the BYOK semantics.
In particular, this allows an operator to securely establish key
material between two separate Transit installations. This potentially
allows one cluster to be used as a source cluster (encrypting a large
amount of data) and a second cluster to decrypt this data later. This
might be useful in hybrid or site-specific deployments of Vault for
instance.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing dependency to sdk/, vault/
Also updates to a newer version while we're here.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on BYOK export
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for BYOK export/import
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/transit.mdx
* Update builtin/logical/transit/path_byok.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Allow public keys to be always exported
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Simplify import key version semantics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for updated ImportVersion semantics
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix handling of RSA private key pairs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refine documentation for public_key
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support additional key types in importing version
This originally left off the custom support for Ed25519 and RSA-PSS
formatted keys that we've added manually.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for Ed25519 keys
Here, we prevent importing public-key only keys with derived Ed25519
keys. Notably, we still allow import of derived Ed25519 keys via private
key method, though this is a touch weird: this private key must have
been packaged in an Ed25519 format (and parseable through Go as such),
even though it is (strictly) an HKDF key and isn't ever used for Ed25519.
Outside of this, importing non-derived Ed25519 keys works as expected.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key only export method to Transit
This allows the existing endpoints to retain private-key only, including
empty strings for versions which lack private keys. On the public-key
endpoint, all versions will have key material returned.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tests for exporting via public-key interface
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add public-key export option to docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Respond with cache size on config write
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Respond with key policy on write
This includes creating a key, but also trimming or rotating an
existing key.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly handle locking around policy formatting
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate that responses are non-empty
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* import rsa and ecdsa public keys
* allow import_version to update public keys - wip
* allow import_version to update public keys
* move check key fields into func
* put private/public keys in same switch cases
* fix method in UpdateKeyVersion
* move asymmetrics keys switch to its own method - WIP
* test import public and update it with private counterpart
* test import public keys
* use public_key to encrypt if RSAKey is not present and failed to decrypt
if key version does not have a private key
* move key to KeyEntry parsing from Policy to KeyEntry method
* move extracting of key from input fields into helper function
* change back policy Import signature to keep backwards compatibility and
add new method to import private or public keys
* test import with imported public rsa and ecdsa keys
* descriptions and error messages
* error messages, remove comments and unused code
* changelog
* documentation - wip
* suggested changes - error messages/typos and unwrap public key passed
* fix unwrap key error
* fail if both key fields have been set
* fix in extractKeyFromFields, passing a PolicyRequest wouldn't not work
* checks for read, sign and verify endpoints so they don't return errors when a private key was not imported and tests
* handle panic on "export key" endpoint if imported key is public
* fmt
* remove 'isPrivateKey' argument from 'UpdateKeyVersion' and
'parseFromKey' methods
also: rename 'UpdateKeyVersion' method to 'ImportPrivateKeyForVersion' and 'IsPublicKeyImported' to 'IsPrivateKeyMissing'
* delete 'RSAPublicKey' when private key is imported
* path_export: return public_key for ecdsa and rsa when there's no private key imported
* allow signed data validation with pss algorithm
* remove NOTE comment
* fix typo in EC public key export where empty derBytes was being used
* export rsa public key in pkcs8 format instead of pkcs1 and improve test
* change logic on how check for is private key missing is calculated
---------
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for importing RSA-PSS keys in Transit
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* plugin/auth: enable multiplexing
- the plugin will be multiplexed when run as an external plugin
by vault versions that support secrets/auth plugin multiplexing (> 1.12)
- we continue to set the TLSProviderFunc to maintain backwards
compatibility with vault versions that don't support AutoMTLS (< 1.12)
* enable multiplexing for secrets engines
* add changelog
* revert call to ServeMultiplex for pki and transit
* Revert "revert call to ServeMultiplex for pki and transit"
This reverts commit 755be28d14b4c4c4d884d3cf4d2ec003dda579b9.
* Return the partial success code override for all batch error types
* changelog
* docs
* Lost the actual override logic. :)
* And don't hardcode 400
* gate on success
* Rename path_config -> path_keys_config
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add config/keys to disable upserting
Transit would allow anyone with Create permissions on the encryption
endpoint to automatically create new encryption keys. This becomes hard
to reason about for operators, especially if typos are subtly
introduced (e.g., my-key vs my_key) -- there is no way to merge these
two keys afterwards.
Add the ability to globally disable upserting, so that if the
applications using Transit do not need the capability, it can be
globally disallowed even under permissive policies.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on disabling upsert
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
* Add support for PKCSv1_5_NoOID signatures
This assumes a pre-hashed input has been provided to Vault, but we do
not write the hash's OID into the signature stream. This allows us to
generate the alternative PKCSv1_5_NoOID signature type rather than the
existing PKCSv1_5_DERnull signature type we presently use.
These are specified in RFC 3447 Section 9.2.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Exclude new none type from PSS based tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for PKCS#1v1.5 signatures
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow passing AssociatedData factories in keysutil
This allows the high-level, algorithm-agnostic Encrypt/Decrypt with
Factory to pass in AssociatedData, and potentially take multiple
factories (to allow KMS keys to work). On AEAD ciphers with a relevant
factory, an AssociatedData factory will be used to populate the
AdditionalData field of the SymmetricOpts struct, using it in the AEAD
Seal process.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add associated_data to Transit Encrypt/Decrypt API
This allows passing the associated_data (the last AD in AEAD) to
Transit's encrypt/decrypt when using an AEAD cipher (currently
aes128-gcm96, aes256-gcm96, and chacha20-poly1305). We err if this
parameter is passed on non-AEAD ciphers presently.
This associated data can be safely transited in plaintext, without risk
of modifications. In the event of tampering with either the ciphertext
or the associated data, decryption will fail.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add to documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
