* wip
* crud and internal lookups working
* worked with bob on tests
* add and fix some tests
* update logical identity protobuf
* pass in empty issuer via scim and add test
* tidy tests and address linter
* VAULT-42443 Update alias API for external JWT entity binding
* test cleanup
* changes
* changelog
* update fix
* address feedback
* whoopsie
* fix stubs
* namespace guarding, more test
* further fix
* Fixes
* I'm dumb
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
Co-authored-by: davidadeleon <ddeleon@hashicorp.com>
* vault: lazily materialize external tokens
Implement stub-first auth flow to avoid token-store writes on read-only requests, materialize only on lease registration, cache materialized token IDs by fingerprint, and add coverage for standby/leader materialization behavior.
* vault: align lazy JWT materialization with main
Adjust standby materialization behavior to main branch APIs, keep lease-time forwarding conversion, and update enterprise tests to match current standby error paths.
* vault: align jwt lazy materialization follow-ups
Apply naming and test-practice follow-ups, document cache-size rationale, simplify stub-use gating, and add external JWT e2e/benchmark coverage that measures raft writes for passthrough vs leased flows.
* vault: sanitize jwt lazy materialization logs
* vault: move jwt stub audit labels to ent
* vault: keep ent token type in audit logs
Remove audit token_type override and jwt_stub audit labeling so enterprise JWT tokens are consistently reported as ent while preserving existing request handling behavior.
* vault: rename unpersisted JWT token state
Rename IsStub/JwtStub to IsUnpersisted/JwtUnpersisted and switch serialized key to jwt_unpersisted for clearer storage semantics.
* vault: use storage-backed JWT token naming
Rename JWT token persistence marker to IsStorageBacked/JwtStorageBacked and clarify docs that passthrough JWT requests may never write token state to storage.
* vault: address jwt token materialization regressions
Fix storage-backed checks for non-ent tokens, materialize JWTs for lookup endpoints, and add lookup-self regression coverage.
* vault: thread perf standby state into jwt materialization
Pass perf standby state through materializeEnterpriseTokenForLease call sites to avoid unsynchronized state reads in request handling paths.
* test: harden flaky enterprise timing checks
Increase timing tolerance in flaky CI tests without changing runtime behavior.
* test: document CI timing tolerance in flaky tests
Add comments clarifying widened waits are for CI scheduling jitter and asynchronous convergence, not behavior changes.
* vault: remove sensitive token logging
Drop clear-text token value from SSC token MAC mismatch debug logging and keep census test unchanged per request.
* vault: tighten request logging for security scan
Remove remaining error-derived logging fields in request handling token/JWT revoke paths to satisfy clear-text sensitive logging findings.
* vault: use typed no-rpc-client errors
Replace string-based no-rpc-client error matching with a dedicated error type and update perf-standby callsites to return it.
* test: harden perf-standby billing race assertions
Allow early active-node counts up to the number of standby operations before the eventual RPC delivery assertion, to avoid race-only CI flakes without changing behavior.
* revert: undo billing test race assertion changes
Revert the prior billing test hardening commit per user request to keep billing tests unchanged in PR #12909.
* vault: restore fetchCeilingPolicies after rebase
Re-add the enterprise fetchCeilingPolicies method from main that was dropped during rebase conflict resolution, fixing build failure in request_handling.go.
* vault: materialize JWT tokens for cubbyhole requests
Ensure enterprise JWT tokens are materialized for cubbyhole paths in addition to token lookup endpoints, preserving expected persisted token-entry behavior for cubbyhole-backed request flows and token-entry JWT tests.
* vault: restore JWT token headers on perf-standby forward
* tests: fix jwt passthrough profile setup
* vault: clarify forwarding helper docs
* vault: cover jwt standby token endpoint behavior
* vault: remove enterprise JWT wording in comments
* vault: preserve materialized token request identity
* vault: stabilize external JWT readonly raft assertion
* vault: address PR 12909 review feedback
* tests: remove redundant NewTestCluster lifecycle calls
* vault: sanitize request handling token error logs
* vault: remove sensitive error fields in ent token cleanup logs
* vault: restore IsJWT wording
* vault: migrate jwt materialization tests to NewTestCluster
* tests: skip external jwt benchmark in CI
* Run make fmt
* Address PR review feedback
* Remove benchmark CI skip
* Move JWT materialization tests to external suite
---------
Co-authored-by: Bianca <48203644+biazmoreira@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add audit log entries for enterprise JWT token fields
* Reduce enterprise token field comment detail
- simplify enterprise token comments in sdk/logical/request.go
- remove verbose wording about issuer/audience/authorization semantics
* Fix TestAudit_JWT_DelegationToken permission denied error
The test was failing with 'permission denied' when using a delegation token
(JWT with act claim) to access cubbyhole. The root causes were:
1. RAR (Rich Authorization Request) check: The JWT contained
'authorization_details' constraints that only allowed access to
'secret/data/users/alice' and 'secret/data/config/general', but the
test was attempting to access 'cubbyhole/test'. The RAR check in
PerformRARCheck() was correctly denying this mismatch.
2. Missing entity policies for actor ACL: For delegation tokens, the
actor's ACL is built solely from entity identity policies (not token
policies like 'default'). Without explicit policies on the actor
entity, the delegation ACL intersection check would fail.
Fixes:
- Removed 'authorization_details' from the test JWT since the test is
about verifying audit log entries for delegation tokens, not RAR
constraints
- Added 'default' policy to both subject and actor entities to ensure
both ACLs allow cubbyhole access for the delegation token intersection
- Updated test assertions to match the simplified JWT (removed
authorization_details verification)
* Use require.NoError instead of t.Fatalf for error check
* Add explicit checks for auth field presence before type assertion
Adds separate checks to verify the 'auth' and 'metadata' fields exist in
the map before attempting type assertion, preventing potential panics
and improving test clarity.
* test: tighten request metadata merge assertions
* test: simplify enterprise metadata assertions
* test: split enterprise metadata merge coverage
* style: apply gofumpt to entry formatter tests
* test: add godoc for enterprise token metadata test
---------
Co-authored-by: Bianca <48203644+biazmoreira@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* no-op commit
* Add census counting for ssh (#12396)
* add cert counting for ssh
* add system view and fix errors
* add otp counting and change units for certs
* add storage tests
* fix census errors
* run make fmt
* use incrementer and change storage to match rfc
* run make fmt
* fix interface and remove parameter
* fix errors
* Update builtin/logical/ssh/path_creds_create.go
Co-authored-by: Victor Rodriguez Rizo <vrizo@hashicorp.com>
* remove error check
* add ssh counts to billing endpoint
* fix error
* add test case
* add ssh metric to test
* add get functions and tests
* fix format
* create function for ssh metrics
* refactoring and add test cases
* replace test check
* add census counting for ssh
* fix read calls
* fix test
* add otp test
* move ssh functions
* address feedback
* fix hash
* fix hash
* change otp count
* fix tests
* change version and hash
---------
Co-authored-by: Victor Rodriguez Rizo <vrizo@hashicorp.com>
# Conflicts:
# sdk/logical/certificate_counter.go
# vault/census_manager_schema_ent.go
# vault/census_manager_schema_ent_test.go
# vault/consumption_billing_license_utilization_ent.go
# vault/consumption_billing_util.go
# vault/consumption_billing_util_test.go
# vault/external_tests/billing/consumption_billing_license_utilization_census_ent_test.go
# vault/logical_system_use_case_billing_test.go
---------
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Co-authored-by: Victor Rodriguez Rizo <vrizo@hashicorp.com>
* add cert counting for ssh
* add system view and fix errors
* add otp counting and change units for certs
* add storage tests
* fix census errors
* run make fmt
* use incrementer and change storage to match rfc
* run make fmt
* fix interface and remove parameter
* fix errors
* Update builtin/logical/ssh/path_creds_create.go
* remove error check
* add ssh counts to billing endpoint
* fix error
* add test case
* add ssh metric to test
* add get functions and tests
* fix format
* create function for ssh metrics
* refactoring and add test cases
* replace test check
* add ssh to billing overview test
---------
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Co-authored-by: Victor Rodriguez Rizo <vrizo@hashicorp.com>
* Enable mac and generate_random key usages for managed keys.
Change response of reading manage key configuration to return the string
representation of key usages rather than the numerical values.
* Add changelog entry.
* Update changelog/_12769.txt
---------
Co-authored-by: Victor Rodriguez Rizo <vrizo@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Change PkiCertificateCountManager.GetCounts() to return a CertCount.
* Add PkiDurationAdjustedCerts field to CertCount.
Add a new field to CertCount to keep track of "duration adjusted" issued
certificates.
Add an x509.Certificate argument to CertCountIncrementer.AddIssuedCertificate.
In the implementation, use the certificate's NotBefore and NotAfter fields to
calculate the validity duration for the certificate, and use that to compute the
duration adjusted units.
* Add the issued certificate to calls to AddIssuedCertificate.
* Add PkiDurationAdjustedCerts when forwarding counts.
Add pki_duration_adjusted_certificate_count to IncrementPkiCount proto.
Update replicationServiceHandler.IncrementPkiCertCountRequest to take into
account the new field.
* Run make proto.
* Update testingPkiCertificateCounter to make assertions on time adjusted counts.
* PR review: Don't use NotAfter.Sub(NotBefore), since time.Duration is max 290 years.
* PR review: Move DurationAdjustedCertificateCount to logical.pki/test_helpers.
Add Bob generated unit tests for logical.durationAdjustedCertificateCount.
* Run make fmt.
Co-authored-by: Victor Rodriguez Rizo <vrizo@hashicorp.com>
* rotation-manager: enable RM to send rotation information to plugins on registration/rotation operations (#11810)
* initial commit for sending NVR to plugins
* add changelog
* add NVR to plugin fields, add RotationInfo to GRPC request handler
* fix tests
* ensure consistent formats on times and ttls
* add translation to allow grpc data transfer
* fix tests and rename fields
* fix missed field renames in tests
* make all methods net-new for backwards compatibility
* update mock plugin and add oss stub back
* remove method with no usages
* Address wrapper comments
* Rebuild proto
* Nil check around SetRotationInfo, return n/a for no last_vault_rotation
* Fix error to match other instances
* Update fields.go
* Return nil if unset for next/last vault rotation times
---------
Co-authored-by: robmonte <17119716+robmonte@users.noreply.github.com>
* Fix return type in stub method
---------
Co-authored-by: vinay-gopalan <86625824+vinay-gopalan@users.noreply.github.com>
Co-authored-by: robmonte <17119716+robmonte@users.noreply.github.com>
* Refactor CertificateCounter.IncrementeCount to use a param object.
In preparation to start collecting more information, refactor the
CertificateCounter to take a parameter object which can be later gain more
fields.
* Rework CertificateCounter to use a fluent interface.
Rename method IncrementCount to AddCount.
Remove method AddIssuedCertificate.
Add method Incrementer, which returns an implementation of the new
CertCountIncrementer.
* Add method CertCountIncrement.Add.
* Refactor PkiCertificateCountConsumer to take a CertCountIncrement.
* Fix TestPkiCertificateCountManager_IncrementAndConsume.
* Rename type CertCountIncrement to CertCount.
* Refactor ReadStoredCounts to return a CertCount value.
Co-authored-by: Victor Rodriguez Rizo <vrizo@hashicorp.com>
* Add the ability to specify extra audit only fields from a plugin
* Add extra auditing fields within the PKI OCSP handler
* Add missing copywrite headers
* Format OCSP dates when non-zero, otherwise specify not set to be clear
* Feedback 2: Only set time fields if not zero instead of non-parsable string
* Serialize JSON fields in SDK response struct
* Perform renames based on RFC feedback
* Resolve OpenAPI test failure
* add cl
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Vault 42177 Add Backend Field (#12092)
* add a new struct for the total number of successful requests for transit and transform
* implement tracking for encrypt path
* implement tracking in encrypt path
* add tracking in rewrap
* add tracking to datakey path
* add tracking to hmac path
* add tracking to sign path
* add tracking to verify path
* unit tests for verify path
* add tracking to cmac path
* reset the global counter in each unit test
* add tracking to hmac verify
* add methods to retrieve and flush transit count
* modify the methods that store and update data protection call counts
* update the methods
* add a helper method to combine replicated and local data call counts
* add tracking to the endpoint
* fix some formatting errors
* add unit tests to path encrypt for tracking
* add unit tests to decrypt path
* fix linter error
* add unit tests to test update and store methods for data protection calls
* stub fix: do not create separate files
* fix the tracking by coordinating replicated and local data, add unit tests
* update all reference to the new data struct
* revert to previous design with just one global counter for all calls for each cluster
* complete external test
* no need to check if current count is greater than 0, remove it
* feedback: remove unnacassary comments about atomic addition, standardize comments
* leave jira id on todo comment, remove unused method
* rename mathods by removing HWM and max in names, update jira id in todo comment, update response field key name
* feedback: remove explicit counter in cmac tests, instead put in the expected number
* feedback: remove explicit tracking in the rest of the tests
* feedback: separate transit testing into its own external test
* Update vault/consumption_billing_util_test.go
Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
* update comment after test name change
* fix comments
* fix comments in test
* another comment fix
* feedback: remove incorrect comment
* fix a CE test
* fix the update method: instead of storing max, increment by the current count value
* update the unit test, remove local prefix as argument to the methods since we store only to non-replicated paths
* update the external test
* Adds a field to backend to track billing data
removed file
* Changed implementation to use a map instead
* Some more comments
* Add more implementation
* Edited grpc server backend
* Refactored a bit
* Fix one more test
* Modified map:
* Revert "Modified map:"
This reverts commit 1730fe1f358b210e6abae43fbdca09e585aaaaa8.
* Removed some other things
* Edited consumption billing files a bit
* Testing function
* Fix transit stuff and make sure tests pass
* Changes
* More changes
* More changes
* Edited external test
* Edited some more tests
* Edited and fixed tests
* One more fix
* Fix some more tests
* Moved some testing structures around and added error checking
* Fixed some nits
* Update builtin/logical/transit/path_sign_verify.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Edited some errors
* Fixed error logs
* Edited one more thing
* Decorate the error
* Update vault/consumption_billing.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
---------
Co-authored-by: Amir Aslamov <amir.aslamov@hashicorp.com>
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Edited stub function
---------
Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
Co-authored-by: Amir Aslamov <amir.aslamov@hashicorp.com>
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Co-authored-by: divyaac <divyaac@berkeley.edu>
* Basic refactoring to reuse PKI certs for SSH
* Refactored so that files are moved to CE
* Modified comment
* Renamed CertCountSystemView
* Moved forwarding function and redefined consume function
* Renamed cert view file
* Moved forwarding function and redefined consume function
Small edit
Renamed cert view file
* Fix issues with commit
* Fix consume job
* Removed error
* Update vault/logical_system_helpers.go
---------
Co-authored-by: divyaac <divya.chandrasekaran@hashicorp.com>
Co-authored-by: Victor Rodriguez <vrizo@hashicorp.com>
Collect event subscriber filters on the active node of a cluster as
"cluster wide" filters, and send them from the secondary active to the
primary active node (`SendSecondaryFilters rpc`). The primary active
node forwards events downstream to the secondary active node if the
events match the secondary cluster's subscriber filters
(`RecvPrimaryEvents rpc`). Then the events are further distributed
around the secondary cluster via the existing `RecvActiveNodeEvents`
and `SendStandbyFilters` rpc's.
Events are forwarded downstream to the secondary cluster if the mount
exists on the secondary cluster, i.e. events from mounts with
`local=true` aren't forwarded, and events from mounts that are not
replicated via paths-filter aren't forwarded.
(This is the CE portion of the above^^)
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* license: update headers to IBM Corp.
* `make proto`
* update offset because source file changed
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
Increment certificate counts in all PKI backends.
Ensure that the PkiCertificateCounter is invoked every time we store and
issue a certificate by any of the PKI backends.
Co-authored-by: Victor Rodriguez <vrizo@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* add ce side code and stubs
* add changelog
* style refactor
* try to use APIPath as mount point instead of request field
* fix linter
* return a response struct instead of a pure timestamp
* add issue time to response
* add ttl to GetRotationInformation response
* rename field for clarity
* update ttl to just seconds
* rename next and last rotation time field; describe what they are
* rename function
* catch up to ent PR
* fix patch merge mistake
* Add an option to allow cert-auth to return metadata about client certs that fail login
* Add cl
* Update SPDX header for sdk/logical/response_test.go
This PR adds the CE plumbing to expose underyling ErrOverloaded errors.
The wrapper allows the HTTP layer to correctly assign 503 status codes
in responses.
