* license: update headers to IBM Corp.
* `make proto`
* update offset because source file changed
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* fix aws auth client cache to use accound ID
* return error if no sts config found
* cache ec2 clients by account ID, region, and role
* add changelog
* fix log syntax
Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
The code that loads the trusted certificate cache for cert-based
authentication ignores any error that occurs while attempting to load
any of the certificates that it finds. Undoubtedly some deployments
have broken certificates or other non-certificate files stored in
their respective back-ends, and so this is important behavior: we
don't want to fail authentication just because `README.md` is not a
valid certificate!
In addition, because listing files and loading certificates is
expensive, the server maintains a cache of trusted certificates. This
cache is populated the first time it's needed, and then used for the
lifetime of the process. If a file fails to load as a certificate,
then it is simply not included in the cache.
These two things lead to a problem when using a backend that might be
subject to transient failures: a hiccough in the certificate loading
process can cause the server to establish a cache that is missing an
otherwise valid certificate. This can then lead to clients failing to
authenticate to the server, until such time as the server is restarted
and the cache reloaded.
This change makes the certificate cache more resilient to loading
failures, by caching partial successes. With this patch, the cache
behavior becomes:
- If the cache exists *and* is either complete or it is not yet time
to attempt to reload the certificates, then the cached results are
used without reservation.
- Otherwise we attempt to load the certificates from storage:
- If the cache does not already exist then a new, empty cache is
created.
- The storage is listed, we attempt to load everything in storage,
skipping things that we have already successfully loaded, and
skipping things that we cannot load, as usual.
- Once we have attempted to load everything from storage, if there
were any errors, we compute a deadline for retrying the load, with
an exponentially increasing delay. If there were no errors, then
the cache is considered complete, and there will be no retry.
This has the nice behavior that we recover from transient failures
eventually, while the exponential back-off ensures that we don't waste
too much time attempting to load certificates that can never be
loaded.
Co-authored-by: John Doty <john.doty@databricks.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add O= restrictions in addition to OU= restrictions
* Add changelog
* Add goDoc to test
* Don't let test certificate expire.
Co-authored-by: Kit Haines <khaines@mit.edu>
* improve auth/ldap TestRotateRootWithRotationUrl test case
* add const
* Update path_config_rotate_root_test.go
* Backport VAULT-34830: enable the new workflow into ce/main (#8681)
* VAULT-34830: enable the new workflow (#8661)
* pipeline: various fixes for the cutover to the enterprise first workflow (#8686)
Various small fixes that were discovered when doing the cutover to the enterprise first merge workflow:
- The `actions-docker-build` action infers enterprise metadata magically from the repository name. Use a branch that allows configuring the repo name until it's merged upstream.
- Fix some CE-In-Enterprise outputs in our metadata job.
- Pass the recurse depth flag correctly when creating backports
- Set the package name when calling the `build-vault` composite action
- Disallow merging changes into `main` and `release/*` when executing in the `hashicorp/vault` repository. This is a hack until PSS-909 is resolved.
- Use self-hosted runners when testing arm64 CE containers in enterprise.
Conflicts:
.github/workflows/backport-automation-ent.yml
.github/workflows/test-run-enos-scenario-containers.yml
---------
* remove file that slipped in during the backport but before the changed file checks (#8706)
* UI: Creating Metadata card for configuration page (#8679) (#8709)
* card setup
* updating to pass in vals
* remove test usage
* actions(metadata): fix metadata version for ce (#8713) (#8714)
* Add support for AES-CBC to transit (#8367) (#8741)
* add key types and encryption for cbc
* add decryption
* start adding tests
* add tests for policy functions
* add convergent case
* add enterprise check and key creation test cases
* fix key generation and add import/export
* add tests and fixes
* add changelog
* linter
* refactor policy functions and fix IV
* add ce change
* fix function calls
* fix factories in function call
* fix IV test case
* test fixes
* add cbc keys to read
* change iv
* fix merge errors
* make fmt
* change error name and add iv error
* fix tests
* UI: Create version card (#8710) (#8744)
* setup version card
* folder restructure
* Adding todos, removing test
* [VAULT-38605] Add self-enrollment option to the TOTP Login MFA method (#8711) (#8731)
* [VAULT-38601] Modify response to MFA enforced requests to enable TOTP self-enrollment (#8723) (#8746)
* Fix token creation in a namespace (#8461) (#8747)
* fix and test for token creation in namespace
* add changelog
* add nil check
* change existing test to work with change
* fix imports
* add error and more specificity in changelog
* enos(sample): don't double sample (#8752) (#8770)
* enos: remove double sample observe
* ci(build): fix notification on artifacts build failure
* changelog: add hash link to changes that originate from enterprise (#8745) (#8775)
* pipeline(backport): use --strategy-option=theirs (#8767) (#8780)
* VAULT-37630: Recover as a copy (#8640) (#8798)
* recover as a copy implementation
* get policy tests passing
* add helpers and testing support
* fixes
* revert a couple of changes
* more tests
* switch to query param
* correctly update source path with the namespace
* only add openapi recover source path if there's a path parameter
* add changelog
* check for no mount in path
* [UI] VAULT-37386 Plugin management: General Settings Route + Templates (#8726) (#8801)
* Move components and routes over to new PR
* Move components to secrets-engine folder
* Use native FormData
* Update params that are passed in
* Add loading state
* Add comments
* Update jsdoc description
* Remove unused action
* Remove debugger
* Fix linting errors
* Add version card component and fix merge conflict issues
* VAULT-38193 Add database observations to Vault (#8727) (#8802)
* VAULT-38193 database observations (WIP)
* VAULT-38193 database observations
* nil check
* make it consistent
* Clean up
* update vault-plugin-secrets-openldap to v0.16.1 (#8820) (#8821)
* update vault-plugin-secrets-openldap to v0.16.1
* changelog
* VAULT-39129: Updating enos tutorial scenario link (#8831) (#8835)
* [VAULT-39153] pipeline(backport): remove docs and pipeline from allowed ce inactive (#8819) (#8842)
Docs have been moved since the tool was written so that exclusion is no
longer needed. Since the defaults were added the `pipeline` group has
expanded to include all `.github`, which we don't want to always
backport. It seems unlike that `pipeline` tooling changes are likely to
be required often on inactive branches so we'll exclude all together for
now.
* [VAULT-39157] enos(cloud): add basic vault cloud scenario (#8828) (#8847)
* [VAULT-39157] enos(cloud): add basic vault cloud scenario
Add the skeleton of a Vault Cloud scenario whereby we create an HCP
network, Vault Cloud cluster, and admin token.
In subsequent PR's we'll wire up building images, waiting on builds, and
ultimately fully testing the resulting image.
* copywrite: add headers
---------
* Upgrade to CRT schema 2 to fix crt-report-dispatch event (#8572) (#8809)
* api/client: support setting extra headers with new logical request interface. (#8808) (#8858)
* [VAULT-39208]: actions: update action pins (#8864) (#8865)
* UI: Create Lease Duration card component + style updates (#8815) (#8870)
* updating components to use hds flex, removing custom css
* creating layout, updating fields to use select instead of dropdown
* conditional render, remove commented code
* adding external link
* update handlers and style
* updating general settings layout so TTL doesnt stretch other cards
* typo
* [UI] Cubbyhole List View Bug (#8859) (#8871)
* fixes issue with cubbyhole list view throwing error in child namespace
* updates to use engineType prop
* Disallow writing of barrier keyring if seals aren't healthy (#8707) (#8885)
* Set the full rewrap context for barrier keyring writes
* Retain some logging at Trace but get rid of the overall context pattern.
Apply correct ctx transform
* changelog
* remove logger
* here too
* remove other unnecessary changes
* VAULT-38888 Add prefix vault to metric summary definitions into main (#8725) (#8892)
* VAULT-38888 Add prefix vault to metric summary definitions
* VAULT-38888 Add changelog for fix
* Edit changelog file name
---------
* [VAULT-39235]: pipeline(changed-files): don't group underscore prefixed changelogs as enterprise only files (#8906) (#8934)
Don't categorize changelog files that begin with an underscore as
enterprise only, otherwise they'll be removed when backporting changes
to CE.
Since we want to include links to commit SHAs in the changelog we have
to create the changelog in the context of CE and thus need to backport
all of those changes.
We also fix a few Go tests that hand not been updated to handle the
updated default inactive CE groups.
* VAULT-39010 Adding new go-discover logic (#8884) (#8931)
* testing new go-discover logic
* add changelog
* Delete website/content/partials/known-issues/aws-auto-join-fails.mdx
* Backport bump go-getter to 1.7.9 into ce/main (#8926)
* bump go-getter to 1.7.9 (#8899)
* bump go-getter to 1.7.9
* add changelog
* go mod tidy
---------
* VAULT-38463: Addressing ldap pipeline failure (#8817) (#8911)
* VAULT-38463: Addressing ldap pipeline failure
* testing ldap tests
* testing ldap tests
* debugging ldap issue
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* debugging ldap failure
* debugging ldap failure
* debugging pipeline
* adding dependency for verify secrets
* removing extra code
* undo changes
* undo changes
* Backport [VAULT-38910]: upgrade docker package to resolve GO-2025-3829 into ce/main (#8875)
* [VAULT-38910]upgrade docker package to resolve GO-2025-3829 (#8642)
* bump github.com/hashicorp/go-secure-stdlib/plugincontainer to v0.4.2
* bump github.com/docker/docker to v28.3.3+incompatible
* go mod tidy
---------
* manually copy over missing changelogs for main (#8956)
* Improve error messages in TestRotateRootWithRotationUrl for BindDN and URL checks
* Use bitnamilegacy cassandra image for tests (#8984) (#8985)
* use default cassandra image for tests
* switch to bitnamilegacy
* [VAULT-39237] actions(generate-changelog) generate changelogs in ce for active ce versions (#8973) (#8976)
Update our changelog generator to dynamically decide which repository
context that it should use when generating the changelog. If the version
given corresponds to an active CE branch then we generate the changelog
in the context of `hashicorp/vault` with the `note-ce.md` template. If
the version corresponds to an enterprise only branch we generate the
changelog in the context of `hashicorp/vault-enterprise` with the
`note-ent.md` template.
The reason we do all of this is so that we can add commit links to
changelogs that for changes that are actually in community editions.
* UI: Moving settings/mount-backend-form to secrets/mounts (#8975) (#8998)
* adding route and replacing old route usage
* adding comments
* updating secrets tests to new route
* Update CHANGELOG.md for 1.20.3 1.19.9 1.18.14 and 1.16.25 (#31527)
* changelog: fix commit URL in CE generated template (#9010) (#9013)
* VAULT-38463: Fix ldap failure (#8996) (#9001)
* Backport [VAULT-38600] Fix the name of the CE stub for mfaLoginEnterprisePaths into ce/main (#9021)
* Update CHANGELOG.md (#31528)
added "Enterprise" to 1.19, 1.18 and 1.16 minor releases
* VAULT-38796, VAULT-38889 reformat observation schema to version 2 (#9006) (#9023)
* [VAULT-39267] actions(slack): migrate to v2 action (#8964) (#8990)
* VAULT-37633: Database static role recover operations (#8922) (#8982)
* initial implementation
* fix
* tests
* changelog
* fix vet errors
* pr comments
* [VAULT-38600] Create TOTP Login MFA credential self-enrollment API endpoint (#8970) (#8999)
* VAULT-36947: Support force unloading a snapshot (#8740) (#9036)
* portion of changes for autoloading
* add test checking for panic
* add endpoint for force unloading
* separate method for force unload
* changelog
* don't redefine constants
* VAULT-39294: Deprecate recover_snapshot_id query param and use a header instead (#8834) (#9042)
* deprecate snapshot query params, use a header instead
* keep read query param, but deprecate recover one
* fix test
* remove list change
* add changelog
* rename header, allow request method
* update changelog
* VAULT-37632 allow restoring SSH CA from loaded snapshot (#8581) (#9034)
* allow restoring ssh config/ca
* add some unit tests
* address PR review
* imports and test upgrades
* linter complaints
* add PR comment and linter fixes
* address review
* Revert "Merge https://github.com/hashicorp/vault/pull/31503 into main"
This reverts commit 6f2ffcf64cd6a01cdbf685db296053adb428e26b, reversing
changes made to 681d1d5c7a2298a8b5dd403554dec2e98c3ce971.
* Update path_config_rotate_root_test.go
---------
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: jadeidev <32917209+jadeidev@users.noreply.github.com>
Co-authored-by: Dan Rivera <dan.rivera@hashicorp.com>
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Kianna <30884335+kiannaquach@users.noreply.github.com>
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
Co-authored-by: Tin Vo <tintvo08@gmail.com>
Co-authored-by: james-warren0 <95658341+james-warren0@users.noreply.github.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
Co-authored-by: Jordan Reimer <zofskeez@gmail.com>
Co-authored-by: Scott Miller <smiller@hashicorp.com>
Co-authored-by: roh-ag <rohit.agrawal@hashicorp.com>
Co-authored-by: JMGoldsmith <spartanaudio@gmail.com>
Co-authored-by: Josh Black <raskchanky@gmail.com>
Co-authored-by: Luciano Di Lalla <88449051+ldilalla-HC@users.noreply.github.com>
Co-authored-by: Robert <17119716+robmonte@users.noreply.github.com>
Co-authored-by: Bruno Oliveira de Souza <bruno.souza@hashicorp.com>
* add rotation URL field
* add docs
* add test
* fix: correct variable name for root rotation URL in rotateRootCredential function
* fix: ensure proper formatting in TestRotateRootWithRotationUrl function
* fix: improve error handling in TestRotateRootWithRotationUrl for invalid rotation URL
* feat: add rotation URL support to LDAP credential configuration
* test: enhance validation in TestRotateRootWithRotationUrl for rotation URL effects
* Update path_config_rotate_root_test.go
* add changelog and update test docs
* Update ldap.mdx
The test container that we use for many LDAP tests recently merged a
breaking change: https://github.com/rroemhild/docker-test-openldap/issues/62
Add support for using containers via references with digests and pin to the latest
version that worked. We can unpin later if so desired.
Signed-off-by: Ryan Cragun <me@ryan.ec>
`gosimports` is the preferred style for module imports and it is
enforced via CI. I've found that things often manage to drift so I've
taken the liberty to update our pre-commit hook to verify our imports
formatting before a change is committed.
Along with updating the formatting helper I've also run `make fmt` to
resolve any formatting drift that managed to make it into the codebase.
Signed-off-by: Ryan Cragun <me@ryan.ec>
* Use DRBG based RSA key generation everywhere
* switch to the conditional generator
* Use DRBG based RSA key generation everywhere
* switch to the conditional generator
* Add an ENV var to disable the DRBG in a pinch
* update go.mod
* Use DRBG based RSA key generation everywhere
* switch to the conditional generator
* Add an ENV var to disable the DRBG in a pinch
* Use DRBG based RSA key generation everywhere
* update go.mod
* fix import
* Remove rsa2 alias, remove test code
* move cryptoutil/rsa.go to sdk
* move imports too
* remove makefile change
* rsa2->rsa
* more rsa2->rsa, remove test code
* fix some overzelous search/replace
* Update to a real tag
* changelog
* copyright
* work around copyright check
* work around copyright check pt2
* bunch of dupe imports
* missing import
* wrong license
* fix go.mod conflict
* missed a spot
* dupe import
* Add an option to allow cert-auth to return metadata about client certs that fail login
* Add cl
* Update SPDX header for sdk/logical/response_test.go
* Ferry ocsp_ca_certificates over the OCSP ValidationConf
* changelog
* First check issuer, then check extraCAS
* Use the correct cert when the signature validation from issuer succeeds
* Validate via extraCas in the cert missing case as well
* dedupe logic
* remove CA test
* Improve trusted cert loading in Certificate Auth
Currently, cert auth has a cache of certName->trusted certificate data. This cache is updated lazily on login. In highly concurrent situations, several logins
of the same cert or more likely, logins not specifying role name may happen simulataneously. In the status quo, each results in going to storage, fetching the role data
(or all roles!), unmarshalling, and certificate parsing.
This change puts a lock matrix in front of the cache miss scenario, so only one of the logins will load and process the role data. In addition, we treat
the absent role name specially, caching it separately so that it cannot be flushed by eviction on the role cache.
* changelog
* cleanup
The previous logic would consider not normalize casing before comparing
the policy names which meant that a token associated to a policy with
an uppercase could not be renewed for the following auth methods:
- AppID
- Cert
- GitHub
- LDAP
- Okta
- Radius
- Userpass
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
- Cert auth had a logic error when crafting the error response when
configured with a leaf certificate to validate against. It would
generate an error response that used a nil error.
- Make the cert auth error messages the same when we fail to match
constraints
