* WIP
* VAULT-40037 Updates to PKI observations
* review feedback
* public key size
* make fmt
* issuerId for sign self issued
* remove confusing issuer_name
* remove unused var
* whoops common name
* role -> role_name
* role name
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* license: update headers to IBM Corp.
* `make proto`
* update offset because source file changed
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* fix aws auth client cache to use accound ID
* return error if no sts config found
* cache ec2 clients by account ID, region, and role
* add changelog
* fix log syntax
Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
* Initial implementation
* Use rotation_statements, handle both password and private_key
* Remove debug prints
* Merge in main
* Remove duplicated error text
* Rename keypair root rotation function
* Use NewRotateRootCredentialsWALPasswordEntry
* Add changelog file
* Move back to original file for now, for review
* put generatePassword into function
* Fix names, call helper for generatePassword
* Generalize the rotation flow and keypair path
* Fix conditional check, remove new file
* Fix changelog
* Add test file
* Fix username check var name
* Fix name variable
* Return an error when both fields are set during rotation, and return an error if somehow walEntry is nil
* Fix test godoc
* Remove print
* change rotated key bits to 4096
Co-authored-by: Robert <17119716+robmonte@users.noreply.github.com>
* add new datakeys endpoint and refactor common functionality
* add test file for new endpoint
* add check and test cases
* add endpoint to ent
* Update builtin/logical/transit/path_datakeys_ent_test.go
* address pr feedback
* fix key size
* run make fmt
* add maximum on count
---------
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Increment certificate counts in all PKI backends.
Ensure that the PkiCertificateCounter is invoked every time we store and
issue a certificate by any of the PKI backends.
Co-authored-by: Victor Rodriguez <vrizo@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
The code that loads the trusted certificate cache for cert-based
authentication ignores any error that occurs while attempting to load
any of the certificates that it finds. Undoubtedly some deployments
have broken certificates or other non-certificate files stored in
their respective back-ends, and so this is important behavior: we
don't want to fail authentication just because `README.md` is not a
valid certificate!
In addition, because listing files and loading certificates is
expensive, the server maintains a cache of trusted certificates. This
cache is populated the first time it's needed, and then used for the
lifetime of the process. If a file fails to load as a certificate,
then it is simply not included in the cache.
These two things lead to a problem when using a backend that might be
subject to transient failures: a hiccough in the certificate loading
process can cause the server to establish a cache that is missing an
otherwise valid certificate. This can then lead to clients failing to
authenticate to the server, until such time as the server is restarted
and the cache reloaded.
This change makes the certificate cache more resilient to loading
failures, by caching partial successes. With this patch, the cache
behavior becomes:
- If the cache exists *and* is either complete or it is not yet time
to attempt to reload the certificates, then the cached results are
used without reservation.
- Otherwise we attempt to load the certificates from storage:
- If the cache does not already exist then a new, empty cache is
created.
- The storage is listed, we attempt to load everything in storage,
skipping things that we have already successfully loaded, and
skipping things that we cannot load, as usual.
- Once we have attempted to load everything from storage, if there
were any errors, we compute a deadline for retrying the load, with
an exponentially increasing delay. If there were no errors, then
the cache is considered complete, and there will be no retry.
This has the nice behavior that we recover from transient failures
eventually, while the exponential back-off ensures that we don't waste
too much time attempting to load certificates that can never be
loaded.
Co-authored-by: John Doty <john.doty@databricks.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
- Instead of using a reserved oid from LetsEncrypt in our tests
and documentation (1.3.6.1.4.1.44947.1.2.4), use
1.3.6.1.4.1.32473.1.2.4, which is in the reserved space for docs
and examples based on RFC 5612
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add O= restrictions in addition to OU= restrictions
* Add changelog
* Add goDoc to test
* Don't let test certificate expire.
Co-authored-by: Kit Haines <khaines@mit.edu>
* Add role rotation info to create/update observations
* observatin enhancements
* observatin enhancements
* remove log
* duration strings instead of seconds
* the stringening
* more times
* credential type
* Add rotation schedule/period to root rotation
* more ttls
* updates
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* improve auth/ldap TestRotateRootWithRotationUrl test case
* add const
* Update path_config_rotate_root_test.go
* Backport VAULT-34830: enable the new workflow into ce/main (#8681)
* VAULT-34830: enable the new workflow (#8661)
* pipeline: various fixes for the cutover to the enterprise first workflow (#8686)
Various small fixes that were discovered when doing the cutover to the enterprise first merge workflow:
- The `actions-docker-build` action infers enterprise metadata magically from the repository name. Use a branch that allows configuring the repo name until it's merged upstream.
- Fix some CE-In-Enterprise outputs in our metadata job.
- Pass the recurse depth flag correctly when creating backports
- Set the package name when calling the `build-vault` composite action
- Disallow merging changes into `main` and `release/*` when executing in the `hashicorp/vault` repository. This is a hack until PSS-909 is resolved.
- Use self-hosted runners when testing arm64 CE containers in enterprise.
Conflicts:
.github/workflows/backport-automation-ent.yml
.github/workflows/test-run-enos-scenario-containers.yml
---------
* remove file that slipped in during the backport but before the changed file checks (#8706)
* UI: Creating Metadata card for configuration page (#8679) (#8709)
* card setup
* updating to pass in vals
* remove test usage
* actions(metadata): fix metadata version for ce (#8713) (#8714)
* Add support for AES-CBC to transit (#8367) (#8741)
* add key types and encryption for cbc
* add decryption
* start adding tests
* add tests for policy functions
* add convergent case
* add enterprise check and key creation test cases
* fix key generation and add import/export
* add tests and fixes
* add changelog
* linter
* refactor policy functions and fix IV
* add ce change
* fix function calls
* fix factories in function call
* fix IV test case
* test fixes
* add cbc keys to read
* change iv
* fix merge errors
* make fmt
* change error name and add iv error
* fix tests
* UI: Create version card (#8710) (#8744)
* setup version card
* folder restructure
* Adding todos, removing test
* [VAULT-38605] Add self-enrollment option to the TOTP Login MFA method (#8711) (#8731)
* [VAULT-38601] Modify response to MFA enforced requests to enable TOTP self-enrollment (#8723) (#8746)
* Fix token creation in a namespace (#8461) (#8747)
* fix and test for token creation in namespace
* add changelog
* add nil check
* change existing test to work with change
* fix imports
* add error and more specificity in changelog
* enos(sample): don't double sample (#8752) (#8770)
* enos: remove double sample observe
* ci(build): fix notification on artifacts build failure
* changelog: add hash link to changes that originate from enterprise (#8745) (#8775)
* pipeline(backport): use --strategy-option=theirs (#8767) (#8780)
* VAULT-37630: Recover as a copy (#8640) (#8798)
* recover as a copy implementation
* get policy tests passing
* add helpers and testing support
* fixes
* revert a couple of changes
* more tests
* switch to query param
* correctly update source path with the namespace
* only add openapi recover source path if there's a path parameter
* add changelog
* check for no mount in path
* [UI] VAULT-37386 Plugin management: General Settings Route + Templates (#8726) (#8801)
* Move components and routes over to new PR
* Move components to secrets-engine folder
* Use native FormData
* Update params that are passed in
* Add loading state
* Add comments
* Update jsdoc description
* Remove unused action
* Remove debugger
* Fix linting errors
* Add version card component and fix merge conflict issues
* VAULT-38193 Add database observations to Vault (#8727) (#8802)
* VAULT-38193 database observations (WIP)
* VAULT-38193 database observations
* nil check
* make it consistent
* Clean up
* update vault-plugin-secrets-openldap to v0.16.1 (#8820) (#8821)
* update vault-plugin-secrets-openldap to v0.16.1
* changelog
* VAULT-39129: Updating enos tutorial scenario link (#8831) (#8835)
* [VAULT-39153] pipeline(backport): remove docs and pipeline from allowed ce inactive (#8819) (#8842)
Docs have been moved since the tool was written so that exclusion is no
longer needed. Since the defaults were added the `pipeline` group has
expanded to include all `.github`, which we don't want to always
backport. It seems unlike that `pipeline` tooling changes are likely to
be required often on inactive branches so we'll exclude all together for
now.
* [VAULT-39157] enos(cloud): add basic vault cloud scenario (#8828) (#8847)
* [VAULT-39157] enos(cloud): add basic vault cloud scenario
Add the skeleton of a Vault Cloud scenario whereby we create an HCP
network, Vault Cloud cluster, and admin token.
In subsequent PR's we'll wire up building images, waiting on builds, and
ultimately fully testing the resulting image.
* copywrite: add headers
---------
* Upgrade to CRT schema 2 to fix crt-report-dispatch event (#8572) (#8809)
* api/client: support setting extra headers with new logical request interface. (#8808) (#8858)
* [VAULT-39208]: actions: update action pins (#8864) (#8865)
* UI: Create Lease Duration card component + style updates (#8815) (#8870)
* updating components to use hds flex, removing custom css
* creating layout, updating fields to use select instead of dropdown
* conditional render, remove commented code
* adding external link
* update handlers and style
* updating general settings layout so TTL doesnt stretch other cards
* typo
* [UI] Cubbyhole List View Bug (#8859) (#8871)
* fixes issue with cubbyhole list view throwing error in child namespace
* updates to use engineType prop
* Disallow writing of barrier keyring if seals aren't healthy (#8707) (#8885)
* Set the full rewrap context for barrier keyring writes
* Retain some logging at Trace but get rid of the overall context pattern.
Apply correct ctx transform
* changelog
* remove logger
* here too
* remove other unnecessary changes
* VAULT-38888 Add prefix vault to metric summary definitions into main (#8725) (#8892)
* VAULT-38888 Add prefix vault to metric summary definitions
* VAULT-38888 Add changelog for fix
* Edit changelog file name
---------
* [VAULT-39235]: pipeline(changed-files): don't group underscore prefixed changelogs as enterprise only files (#8906) (#8934)
Don't categorize changelog files that begin with an underscore as
enterprise only, otherwise they'll be removed when backporting changes
to CE.
Since we want to include links to commit SHAs in the changelog we have
to create the changelog in the context of CE and thus need to backport
all of those changes.
We also fix a few Go tests that hand not been updated to handle the
updated default inactive CE groups.
* VAULT-39010 Adding new go-discover logic (#8884) (#8931)
* testing new go-discover logic
* add changelog
* Delete website/content/partials/known-issues/aws-auto-join-fails.mdx
* Backport bump go-getter to 1.7.9 into ce/main (#8926)
* bump go-getter to 1.7.9 (#8899)
* bump go-getter to 1.7.9
* add changelog
* go mod tidy
---------
* VAULT-38463: Addressing ldap pipeline failure (#8817) (#8911)
* VAULT-38463: Addressing ldap pipeline failure
* testing ldap tests
* testing ldap tests
* debugging ldap issue
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* testing pipeline
* debugging ldap failure
* debugging ldap failure
* debugging pipeline
* adding dependency for verify secrets
* removing extra code
* undo changes
* undo changes
* Backport [VAULT-38910]: upgrade docker package to resolve GO-2025-3829 into ce/main (#8875)
* [VAULT-38910]upgrade docker package to resolve GO-2025-3829 (#8642)
* bump github.com/hashicorp/go-secure-stdlib/plugincontainer to v0.4.2
* bump github.com/docker/docker to v28.3.3+incompatible
* go mod tidy
---------
* manually copy over missing changelogs for main (#8956)
* Improve error messages in TestRotateRootWithRotationUrl for BindDN and URL checks
* Use bitnamilegacy cassandra image for tests (#8984) (#8985)
* use default cassandra image for tests
* switch to bitnamilegacy
* [VAULT-39237] actions(generate-changelog) generate changelogs in ce for active ce versions (#8973) (#8976)
Update our changelog generator to dynamically decide which repository
context that it should use when generating the changelog. If the version
given corresponds to an active CE branch then we generate the changelog
in the context of `hashicorp/vault` with the `note-ce.md` template. If
the version corresponds to an enterprise only branch we generate the
changelog in the context of `hashicorp/vault-enterprise` with the
`note-ent.md` template.
The reason we do all of this is so that we can add commit links to
changelogs that for changes that are actually in community editions.
* UI: Moving settings/mount-backend-form to secrets/mounts (#8975) (#8998)
* adding route and replacing old route usage
* adding comments
* updating secrets tests to new route
* Update CHANGELOG.md for 1.20.3 1.19.9 1.18.14 and 1.16.25 (#31527)
* changelog: fix commit URL in CE generated template (#9010) (#9013)
* VAULT-38463: Fix ldap failure (#8996) (#9001)
* Backport [VAULT-38600] Fix the name of the CE stub for mfaLoginEnterprisePaths into ce/main (#9021)
* Update CHANGELOG.md (#31528)
added "Enterprise" to 1.19, 1.18 and 1.16 minor releases
* VAULT-38796, VAULT-38889 reformat observation schema to version 2 (#9006) (#9023)
* [VAULT-39267] actions(slack): migrate to v2 action (#8964) (#8990)
* VAULT-37633: Database static role recover operations (#8922) (#8982)
* initial implementation
* fix
* tests
* changelog
* fix vet errors
* pr comments
* [VAULT-38600] Create TOTP Login MFA credential self-enrollment API endpoint (#8970) (#8999)
* VAULT-36947: Support force unloading a snapshot (#8740) (#9036)
* portion of changes for autoloading
* add test checking for panic
* add endpoint for force unloading
* separate method for force unload
* changelog
* don't redefine constants
* VAULT-39294: Deprecate recover_snapshot_id query param and use a header instead (#8834) (#9042)
* deprecate snapshot query params, use a header instead
* keep read query param, but deprecate recover one
* fix test
* remove list change
* add changelog
* rename header, allow request method
* update changelog
* VAULT-37632 allow restoring SSH CA from loaded snapshot (#8581) (#9034)
* allow restoring ssh config/ca
* add some unit tests
* address PR review
* imports and test upgrades
* linter complaints
* add PR comment and linter fixes
* address review
* Revert "Merge https://github.com/hashicorp/vault/pull/31503 into main"
This reverts commit 6f2ffcf64cd6a01cdbf685db296053adb428e26b, reversing
changes made to 681d1d5c7a2298a8b5dd403554dec2e98c3ce971.
* Update path_config_rotate_root_test.go
---------
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: jadeidev <32917209+jadeidev@users.noreply.github.com>
Co-authored-by: Dan Rivera <dan.rivera@hashicorp.com>
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Kianna <30884335+kiannaquach@users.noreply.github.com>
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
Co-authored-by: Tin Vo <tintvo08@gmail.com>
Co-authored-by: james-warren0 <95658341+james-warren0@users.noreply.github.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
Co-authored-by: Jordan Reimer <zofskeez@gmail.com>
Co-authored-by: Scott Miller <smiller@hashicorp.com>
Co-authored-by: roh-ag <rohit.agrawal@hashicorp.com>
Co-authored-by: JMGoldsmith <spartanaudio@gmail.com>
Co-authored-by: Josh Black <raskchanky@gmail.com>
Co-authored-by: Luciano Di Lalla <88449051+ldilalla-HC@users.noreply.github.com>
Co-authored-by: Robert <17119716+robmonte@users.noreply.github.com>
Co-authored-by: Bruno Oliveira de Souza <bruno.souza@hashicorp.com>
* add key types and encryption for cbc
* add decryption
* start adding tests
* add tests for policy functions
* add convergent case
* add enterprise check and key creation test cases
* fix key generation and add import/export
* add tests and fixes
* add changelog
* linter
* refactor policy functions and fix IV
* add ce change
* fix function calls
* fix factories in function call
* fix IV test case
* test fixes
* add cbc keys to read
* change iv
* fix merge errors
* make fmt
* change error name and add iv error
* fix tests
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
* add rotation URL field
* add docs
* add test
* fix: correct variable name for root rotation URL in rotateRootCredential function
* fix: ensure proper formatting in TestRotateRootWithRotationUrl function
* fix: improve error handling in TestRotateRootWithRotationUrl for invalid rotation URL
* feat: add rotation URL support to LDAP credential configuration
* test: enhance validation in TestRotateRootWithRotationUrl for rotation URL effects
* Update path_config_rotate_root_test.go
* add changelog and update test docs
* Update ldap.mdx
* remove local time logic, and force cron to be UTC
* add test comment
* update docs
* add changelog
* change mesasge
* add utc clarification to docs
* remove utc reference in root token docs
* remove doc from partial
* consider possibility of NextVaultRotation being nil on queue population
* move test
* add changelog
* fix reference to nil, and improve debug log
* use helper function to write static roles to storage
* add password check in test
* fix godoc
* fix changelog and add remediation debug line
* force ticker to run, and make sure credential doesnt rotate
* add another edge case
* fix godoc
* check ttl is less in test
* check error case and if resp is nil
* make check on ttl more robust
* Add API warning based on DB type
* Add deprecation notice
* Add warning to the top of the docs pages
* Update capabilities table
* Filter SQLConnectionProducer fields from unrecognized parameters warning
* Add test case
* PR Review Feedback
* Remove openssl test, to be included in a separate testing PR.
* Openssl test for DeltaCRL + Move Test Helpers to test_helpers
* Switch to regex instead of contains due to different whitespace when running in CI.
* require explicit value for disable_mlock
* set disable_mlock back to true for all docker tests
* fix build error
* update test config files
* change explicit mlock check to apply to integrated storage only.
* formatting and typo fixes
* added test for raft
* remove erroneous test
* remove unecessary doc line
* remove unecessary var
* pr suggestions
* test compile fix
* add mlock config value to enos tests
* enos lint
* update enos tests to pass disable_mlock value
* move mlock error to runtime to check for env var
* fixed mlock config detection logic
* call out mlock on/off tradeoffs to docs
* rewording production hardening section on mlock for clarity
* update error message when missing disable_mlock value to help customers with the previous default
* fix config doc error and update production-hardening doc to align with existing recommendations.
* remove extra check for mlock config value
* fix docker recovery test
* Update changelog/29974.txt
Explicitly call out that Vault will not start without disable_mlock included in the config.
Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>
* more docker test experimentation.
* passing disable_mlock into test cluster
* add VAULT_DISABLE_MLOCK envvar to docker tests and pass through the value
* add missing envvar for docker env test
* upate additional docker test disable_mlock values
* Apply suggestions from code review
Use active voice.
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
---------
Co-authored-by: Kuba Wieczorek <kuba.wieczorek@hashicorp.com>
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
