Commit Graph

118 Commits

Author SHA1 Message Date
Steven Clark
7efc1af5d0
Do not acquire a read lock twice on tidyStatusLock during tidy-status api call. (#28556) 2024-10-02 08:58:05 -04:00
Steven Clark
2db2a9fb5d
PKI: Track last time auto tidy was run across restarts (#28488)
* Track the last PKI auto-tidy time ran for use across nodes

 - If the interval time for auto-tidy is longer then say a regularly
   scheduled restart of Vault, auto-tidy is never run. This is due to
   the time of the last run of tidy is only kept in memory and
   initialized on startup to the current time
 - Store the last run of any tidy, to maintain previous behavior, to
   a cluster local file, which is read in/initialized upon a mount
   initialization.

* Add auto-tidy configuration fields for backing off at startup

* Add new auto-tidy fields to UI

* Update api docs for auto-tidy

* Add cl

* Update field description text

* Apply Claire's suggestions from code review

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>

* Implementing PR feedback from the UI team

* remove explicit defaults and types so we retrieve from backend, decouple enabling auto tidy from duration, move params to auto settings section

---------

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
Co-authored-by: claire bontempo <cbontempo@hashicorp.com>
2024-09-26 09:47:11 -04:00
Steven Clark
789f9b7821
Add missing revocation storage lock on RevokeCert refactoring (#28259)
* Add missing revocation storage lock on RevokeCert refactoring

* Refactor internal PKI revocation API

 - Process the odd logical.Response/error returns internally to make
   the returns better consumable from other users.
 - Leverage the GetConfigWithUpdate to fetch the current CRL config, otherwise we can use older or nil configs.
2024-09-03 17:18:30 -04:00
Scott Miller
747b395da5
Revocation RevokerFactory interface, CE changes (#28203) 2024-08-27 21:35:11 +00:00
Victor Rodriguez
67515c7e4a
Ongoing refactoring of PKI revocation code (#27427)
* Move resolveIssuerCRLPath to PKI issuing package.

* Move fetchCertBySerial to PKI issuing package.

* Move fetchRevocationInfo to PKI revocation package.

* Make associateRevokedCertWithIsssuer a method of RevocationInfo.

* Move serialFromCert and normalizeSerial to PKI parsing package.

* Move writeUnifiedRevocationEntry to PKI revocation package.

* Run make fmt.
2024-06-11 09:25:14 -04:00
Victor Rodriguez
eb8496e874
Commence refactoring to decouple revocation code from main PKI backend (#27417)
* Rename crlConfig to CrlConfig.

Rename defaultCrlConfig to DefaultCrlConfig.

* Move CrlConfig and DefaultCrlConfig to new package pki/revocation.

* Rename revocationInfo to RevocationInfo.

* Move RevocationInfo to pki/revocation.

* Add StorageContext interface to PKI's revocation package.

* Add CrlBuilderType interface to pki_backend package.

The purpose of the interface is to make it possible to gradually move (refactor)
CrlBuilder to the revocation package.

* Move CrlConfig and DefaultCrlConfig to package pki_backend.

* Make StorageContext.CrlBuilder() return a CrlBuilderType.

Add methods SetLastDeltaRebuildCheckTime() and ShouldInvalidate() to
CrlBuilderType.

* Move fetchIssuerMapForRevocationChecking to PKI's revocation package.

* Run make fmt.
2024-06-10 16:41:47 +00:00
Kit Haines
927df50a95
community edition changes to enable certificate metadata ent (#26951) 2024-05-10 18:49:20 +00:00
Steven Clark
f8cad5e344
PKI cert metadata should not cross cluster boundaries (#26919) 2024-05-09 22:30:42 -04:00
Scott Miller
9f5da77fa0
Certificate Metadata, CE components (#26710)
* Certificate Metadata, CE components

* License headers

* make proto

* move pathFetchMetadata to ENT

* move pathFetchMetadata path to ENT

* correct stub sig

* Issuers may not be available in legacy CA storage, shouldn't fail issue/sign

* clarify error msg
2024-05-01 15:19:53 -05:00
Josh Black
fa13dbd381
add gosimport to make fmt and run it (#25383)
* add gosimport to make fmt and run it

* move installation to tools.sh

* correct weird spacing issue

* Update Makefile

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* fix a weird issue

---------

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2024-02-13 14:07:02 -08:00
Mike Palmiotto
43be9fc18a
Request Limiter (#25093)
This commit introduces two new adaptive concurrency limiters in Vault,
which should handle overloading of the server during periods of
untenable request rate. The limiter adjusts the number of allowable
in-flight requests based on latency measurements performed across the
request duration. This approach allows us to reject entire requests
prior to doing any work and prevents clients from exceeding server
capacity.

The limiters intentionally target two separate vectors that have been
proven to lead to server over-utilization.

- Back pressure from the storage backend, resulting in bufferbloat in
  the WAL system. (enterprise)
- Back pressure from CPU over-utilization via PKI issue requests
  (specifically for RSA keys), resulting in failed heartbeats.

Storage constraints can be accounted for by limiting logical requests
according to their http.Method. We only limit requests with write-based
methods, since these will result in storage Puts and exhibit the
aforementioned bufferbloat.

CPU constraints are accounted for using the same underlying library and
technique; however, they require special treatment. The maximum number
of concurrent pki/issue requests found in testing (again, specifically
for RSA keys) is far lower than the minimum tolerable write request
rate. Without separate limiting, we would artificially impose limits on
tolerable request rates for non-PKI requests. To specifically target PKI
issue requests, we add a new PathsSpecial field, called limited,
allowing backends to specify a list of paths which should get
special-case request limiting.

For the sake of code cleanliness and future extensibility, we introduce
the concept of a LimiterRegistry. The registry proposed in this PR has
two entries, corresponding with the two vectors above. Each Limiter
entry has its own corresponding maximum and minimum concurrency,
allowing them to react to latency deviation independently and handle
high volumes of requests to targeted bottlenecks (CPU and storage).

In both cases, utilization will be effectively throttled before Vault
reaches any degraded state. The resulting 503 - Service Unavailable is a
retryable HTTP response code, which can be handled to gracefully retry
and eventually succeed. Clients should handle this by retrying with
jitter and exponential backoff. This is done within Vault's API, using
the go-retryablehttp library.

Limiter testing was performed via benchmarks of mixed workloads and
across a deployment of agent pods with great success.
2024-01-26 14:26:21 -05:00
Steven Clark
8963ae495d
PKI: Refactor storage of certificates into a common method (#24415)
- Move the copy/pasted code to store certificates into a
   common method within the PKI plugin
2023-12-07 11:51:51 -05:00
Steven Clark
cbf6dc2c4f
PKI refactoring to start breaking apart monolith into sub-packages (#24406)
* PKI refactoring to start breaking apart monolith into sub-packages

 - This was broken down by commit within enterprise for ease of review
   but would be too difficult to bring back individual commits back
   to the CE repository. (they would be squashed anyways)
 - This change was created by exporting a patch of the enterprise PR
   and applying it to CE repository

* Fix TestBackend_OID_SANs to not be rely on map ordering
2023-12-07 09:22:53 -05:00
Steven Clark
3623dfc227
Add support for plugins to specify binary request paths (#23729)
* wip

* more pruning

* Integrate OCSP into binary paths PoC

 - Simplify some of the changes to the router
 - Remove the binary test PKI endpoint
 - Switch OCSP to use the new binary paths backend variable

* Fix proto generation and test compilation

* Add unit test for binary request handling

---------

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
2023-10-23 17:04:42 -04:00
Steven Clark
b0fef53184
Do not attempt to shutdown ACME thread on non-active nodes (#23293) 2023-09-26 16:32:52 -04:00
hashicorp-copywrite[bot]
0b12cdcfd1
[COMPLIANCE] License changes (#22290)
* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License.

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUS-1.1

* Fix test that expected exact offset on hcl file

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2023-08-10 18:14:03 -07:00
Steven Clark
66052d5431
Refactor ACME for enterprise CIEPS integration (#22188) 2023-08-03 11:24:46 -04:00
Steven Clark
1bed33a399
Add the ability to add enterprise only backend features in PKI - OSS (#21748)
To allow us to support CIEPS backend state, allow the backend to
contain enterprise only state variables. Also allow us to implement
enterprise only hooks into the various backend functions to initialize,
periodicFunc, cleanup and invalidate.
2023-07-11 10:05:23 -04:00
Steven Clark
9b9017a926
Add the ability for enterprise to override PKI backend attributes during setup (#21618) 2023-07-06 16:16:17 -04:00
Steven Clark
000d754c40
Make ACME EAB keys specific to the ACME directory they are created within (#20803)
* Update EAB management urls underneath pki/eab

 - It was decided that for ease of ACLing, the management
   paths for EAB apis should be outside of the acme path
   prefix
 - Delete duplicated tests, rely on the proper cluster
   based tests for EAB management.

* Update ACME EAB creation paths to be directory specific

 - Make the EAB creation APIs directory specific.
 - This commit is still missing the enforcement that
   they can be redeemed on a specific path.

* Enforce EAB tokens per ACME directory context like accounts

 - Do not allow an EAB from one ACME directory to be used
   in another.
 - Rework the ACME directory function to simply get the path from the request instead of parsing out the role/issuer name.
 - Add some commentary around expectations if operators change issuer names

* Add an EAB certbot integration test

 - Verify with the 3rd party certbot cli that our EAB workflow works as expected.

* Fix unit test

 - Unit test wasn't setting up r.Path within the request
   that we now use to determine the acme directory.
2023-05-30 11:49:01 -04:00
Steven Clark
34571d4d23
Enforce PKI cluster local 'path' argument is set when enabling ACME (#20838)
* Fix error handling in ACME

 - If we don't match a specific ACME error, use ErrServerInternal instead of the last error type from the internal map
 - Logger parameters need two params

* Enforce cluster local path is set when enabling ACME

* Add a warning on ACME config read api if enabled but path not set

 - This might help expose that the local path configuration on a secondary cluster was not set which would prevent ACME from running.
2023-05-30 11:00:50 -04:00
Steven Clark
32532c61d1
Address various issues related to ACME EAB (#20755)
* Fix various EAB related issues

 - List API wasn't plumbed through properly so it did not work as expected
 - Use random 32 bytes instead of an EC key for EAB key values
 - Update OpenAPI definitions

* Clean up unused EAB keys within tidy

* Move Vault EAB creation path to pki/acme/new-eab

* Update eab vault responses to match up with docs
2023-05-24 21:17:33 +00:00
Alexander Scheel
b1f0d4e495
Add nonce service to sdk/helpers, use in PKI (#20688)
* Build a better nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add internal nonce service for testing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add benchmarks for nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add statistics around how long tidy took

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Replace ACME nonces with shared nonce service

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add an initialize method to nonce services

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use the new initialize helper on nonce service in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional tests for nonces

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Format sdk/helper/nonce

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use default 90s nonce expiry in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove parallel test case as covered by benchmark

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional commentary to encrypted nonce implementation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add nonce to test_packages

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-05-23 19:44:05 +00:00
Steven Clark
0b9f4048af
Add External Account Binding support to ACME (#20523)
* Add Vault APIS to create, list, delete ACME EAB keys

 - Add Vault authenticated APIs to create, list and delete ACME
   EAB keys.
 - Add supporting tests for all new apis

* Add require_eab to acme configuration

* Add EAB support to ACME

* Add EAB support to ACME

* PR feedback 1

 - Address missing err return within DeleteEab
 - Move verifyEabPayload to acme_jws.go no code changes in this PR
 - Update error message returned for error on account storage with EAB.

* PR feedback 2

 - Verify JWK signature payload after signature verification

* Introduce an ACME eab_policy in configuration

 - Instead of a boolean on/off for require_eab, introduce named policies for ACME behaviour enforcing eab.
 - The default policy of always-required, will force new accounts to have an EAB, and all operations in the future, will make sure the account has an EAB associated with it.
 - Two other policies, not-required will allow any anonymous users to use ACME within PKI and 'new-account-required' will enforce new accounts going forward to require an EAB, but existing accounts will still be allowed to use ACME if they don't have an EAB associated with the account.
 - Having 'always-required' as a policy, will override the environment variable to disable public acme as well.

* Add missing go-docs to new tests.

* Add valid eab_policy values in error message.
2023-05-15 13:15:20 -04:00
Kit Haines
9480573cf7
Structure of ACME Tidy (#20494)
* Structure of ACME Tidy.

* The tidy endpoints/call information.

* Counts for status plumbing.

* Update typo calls, add information saving date of account creation.

* Missed some field locations.

* Set-up of Tidy command.

* Proper tidy function; lock to work with

* Remove order safety buffer.

* Missed a field.

* Read lock for account creation; Write lock for tidy (account deletion)

* Type issues fixed.

* fix range operator.

* Fix path_tidy read.

* Add fields to auto-tidy config.

* Add (and standardize) Tidy Config Response

* Test pass, consistent fields

* Changes from PR-Reviews.

* Update test to updated default due to PR-Review.
2023-05-05 15:14:41 +00:00
Alexander Scheel
37852757b0
Add custom DNS resolver to ACME configuration (#20400)
* Handle caching of ACME config

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add DNS resolvers to ACME configuration

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add custom DNS resolver to challenge verification

This required plumbing through the config, reloading it when necessary,
and creating a custom net.Resolver instance.

Not immediately clear is how we'd go about building a custom DNS
validation mechanism that supported multiple resolvers. Likely we'd need
to rely on meikg/dns and handle the resolution separately for each
container and use a custom Dialer that assumes the address is already
pre-resolved.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Improvements to Docker harness

 - Expose additional service information, allowing callers to figure out
   both the local address and the network-specific address of the
   service container, and
 - Allow modifying permissions on uploaded container files.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add infrastructure to run Bind9 in a container for tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Validate DNS-01 challenge works

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-27 15:30:29 -04:00
Kit Haines
7f4d8ca3a4
Vault 13349 acme create new global configuration endpoints (#20228)
* Base: Add configuration to ACME.

* Remove unused endpoint

* Commit todo.

* Allow Read of Configuration.

* make fmt.

* parse config.

* Don't panic.
2023-04-26 17:16:09 +00:00
Alexander Scheel
86ab4ef3e7
Add ACME revocation handlers (#20340)
* Add ACME revocation handlers

This refactors path_revoke to expose Proof of Possession verification,
which is reused by ACME to allow methods 1 and 2:

 1. Revocation of a certificate issued by the account, using account
    signature as sufficient proof.
 2. Revocation of a certificate via proving possession of its private
    key, using this private key to create the JWS signature.

We do not support the third mechanism, completing challenges equivalent
to those on the existing certificate and then performing a revocation
under an account which didn't issue the certificate but which did solve
those challenges.

We additionally create another map account->cert->order, allowing us to
quickly look up if a cert was issued by this particular account. Note
that the inverse lookup of cert->(account, order) lookup isn't yet
possible due to Vault's storage structure.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update ACME pkiext tests to revoke certs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add auth handler checks

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address review feedback

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-25 16:48:30 -04:00
Steven Clark
9bebe95455
Implement ACME CSR signing and certificate retrieval (#20266)
* WIP: Implement ACME CSR signing and certificate retrieval

* Add some validations within the ACME finalize API

 - Validate that the CSR we were given matches the DNS names
   and IP addresses within the order
 - Validate that the CSR does not share the same public as the
   account

* Gate ACME finalize order validating all authorizations are in valid state
2023-04-21 09:38:06 -04:00
Alexander Scheel
d3629a13c3
Add warnings to crl rebuilds, allowing notifying operator of empty issuer equivalency sets (#20253)
* Add infrastructure for warnings on CRL rebuilds

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add warning on issuer missing KU for CRL Signing

When an entire issuer equivalency class is missing CRL signing usage
(but otherwise has key material present), we should add a warning so
operators can either correct this issuer or create an equivalent version
with KU specified.

Resolves: https://github.com/hashicorp/vault/issues/20137

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for issuer warnings

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix return order of CRL builders

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-19 16:55:37 +00:00
Alexander Scheel
dae04a8795
Add acme challenge validation engine (#20221)
* Allow creating storageContext with timeout

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add challenge validation engine to ACME

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Initialize the ACME challenge validation engine

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Trigger challenge validation on endpoint submission

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix GetKeyThumbprint to use raw base64

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Point at localhost for testing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add cleanup of validation engine

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-19 12:31:19 -04:00
Steven Clark
3acbeddf7a
Add a helper function to build ACME API patterns (#20180)
- Add a helper function that can accept the final API path along with
   the pattern function for an ACME api definition and generate the
   various flavors for the given API
2023-04-14 18:48:33 +00:00
Steven Clark
1b673b6ab4
Implement ACME order API (#20127)
* Implement ACME new-order API
 - This is a very rough draft for the new order ACME API

* Add ACME order list API

* Implement ACME Get order API

* Misc order related fixes

 - Filter authorizations in GetOrders for valid
 - Validate notBefore and notAfter dates make sense
 - Add <order>/cert URL path to order response if set to valid

* Return account status within err authorized, if the account key verified
2023-04-14 14:54:48 +00:00
Alexander Scheel
05feed51bc
Add ACME authorizations & challenges (#20113)
* Distinguish POST-as-GET from POST-with-empty-body

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ACME authorization, identifier, and challenge types

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ability to load and save authorizations

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ACME authorizations path handling

This supports two methods: a fetch handler over the authorization, to
expose the underlying challenges, and a deactivate handler to revoke
the authorization and mark its challenges invalid.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ACME challenge path handling

These paths kick off processing and validation of the challenge by the
ACME client.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-12 16:55:25 +00:00
Steven Clark
a8e91f0248
Rework ACME workflow test to leverage Golang's ACME client library (#19949)
* Rework ACME workflow test to leverage Golang's ACME client library

 - Instead of testing manually, leverage the Golang ACME library
   to test against our implementation from the unit tests.

* Add tests for new-account and misc fixes

 - Set and return the account status for registration
 - Add handlers for the account/ api/updates
 - Switch acme/ to cluster local storage
 - Disable terms of service checks for now as we don't set the url

* PR feedback

 - Implement account deactivation
 - Create separate account update handler, to not mix account creation
   logic
 - Add kid field to account update definition
 - Add support to update contact details on an existing account
2023-04-12 09:05:42 -04:00
Anton Averchenkov
74881dd75d
openapi: Add display attributes for PKI plugin (#19422) 2023-04-06 11:10:01 -04:00
Alexander Scheel
32e3cd6972
Merge ACME package back into the PKI package (#19826)
* Squash pki/acme package down to pki folder

Without refactoring most of PKI to export the storage layer, which we
were initially hesitant about, it would be nearly impossible to have the
ACME layer handle its own storage while being in the acme/ subpackage
under the pki package.

Thus, merge the two packages together again.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Properly format errors for missing parameters

When missing required ACME request parameters, don't return Vault-level
errors, but drop into the PKI package to return properly-formatted ACME
error messages.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Error type clarifications

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix GetOk with type conversion calls

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-29 21:08:31 +00:00
Alexander Scheel
71441742fa
Add ACME new account creation handlers (#19820)
* Identify whether JWKs existed or were created, set KIDs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Reclassify ErrAccountDoesNotExist as 400 per spec

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional stub methods for ACME accounts

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Start adding ACME newAccount handlers

This handler supports two pieces of functionality:

 1. Searching for whether an existing account already exists.
 2. Creating a new account.

One side effect of our JWS parsing logic is that we needed a way to
differentiate between whether a JWK existed on disk from an account or
if it was specified in the request. This technically means we're
potentially responding to certain requests with positive results (e.g.,
key search based on kid) versus erring earlier like other
implementations do.

No account storage has been done as part of this commit.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Unify path fields handling, fix newAccount method

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-29 19:06:09 +00:00
Steven Clark
ddb5db1538
Initial ACME new-nonce API (#19822)
* Initial ACME new-nonce API implementation

* Return proper HTTP status codes for ACME new-nonce API handler
2023-03-29 18:22:48 +00:00
Steven Clark
4063d6ed09
PKI: Initial ACME directory API support (#19803)
* PKI: Initial ACME directory API support along with basic tests for error handler and the directory itself across various paths.
2023-03-29 16:29:19 +00:00
Alexander Scheel
bebdc6551d
Add tests for PKI endpoint authentication (#19704)
* Split (un,)authenticated issuer fetch endpoints

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow unauthed per-issuer unified-crl access

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests to validate endpoint authentication status

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Disable schema validation in TestBackend_IfModifiedSinceHeaders

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-23 15:54:26 +00:00
Hamid Ghaf
e55c18ed12
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Kit Haines
2dd4528ed8
Telemetry Metrics Configuration. (#18186)
* Telemetry Metrics Configuration.

* Err Shadowing Fix (woah, semgrep is cool).

* Fix TestBackend_RevokePlusTidy_Intermediate

* Add Changelog.

* Fix memory leak.  Code cleanup as suggested by Steve.

* Turn off metrics by default, breaking-change.

* Show on tidy-status before start-up.

* Fix tests

* make fmt

* Add emit metrics to periodicFunc

* Test not delivering unavailable metrics + fix.

* Better error message.

* Fixing the false-error bug.

* make fmt.

* Try to fix race issue, remove confusing comments.

* Switch metric counter variables to an atomic.Uint32

 - Switch the metric counter variables to an atomic variable type
   so that we are forced to properly load/store values to it

* Fix race-issue better by trying until the metric is sunk.

* make fmt.

* empty commit to retrigger non-race tests that all pass locally

---------

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2023-02-10 21:31:56 +00:00
Steven Clark
afd9e82e9b
Add a comment around why we are grabbing a lock to update an atomic boolean (#19087) 2023-02-09 09:12:37 -05:00
Steven Clark
f05c20ba42
PKI Unified CRL/OCSP apis should be ent only (#18913)
* PKI Unified CRL/OCSP apis should be ent only

 - Do not enable any of the unified crl/ocsp related apis on OSS.

* Rollback refactoring of pathFetchCRLViaCertPath

 - As pointed out in the PR, this method isn't actually being used at
   the moment with the <serial> handler, pathFetchValid, matching
   everything under the cert/XXXX path.

* Fix schema for ent/oss diff

 - Define the OSS vs ENT urls we want to see within the schema
   definition even if they aren't really going to be used in the end.
2023-02-01 08:35:14 -05:00
Alexander Scheel
70cb462e65
Allow unification of revocations on other clusters (#18873)
* Allow unification of revocations on other clusters

If a BYOC revocation occurred on cluster A, while the cert was initially
issued and stored on cluster B, we need to use the invalidation on the
unified entry to detect this: the revocation queues only work for
non-PoP, non-BYOC serial only revocations and thus this BYOC would be
immediately accepted on cluster A. By checking all other incoming
revocations for duplicates on a given cluster, we can ensure that
unified revocation is consistent across clusters.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use time-of-use locking for global revocation processing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-27 16:34:04 +00:00
Steven Clark
1f520ef278
Unified revocation migration code (#18866)
* Unified revocation migration code

 - Add a periodic function that will list the local revocations
   and if any are missing from the unified revocation area will
   force a write to the unified revocation folder/remote instance.

* PR Feedback

 - Do not transfer expired certificates to unified space from local
 - Move new periodic code into a periodic.go file
 - Add a flag so we only run this stuff once if all is good, with
   a force flag if we encounter errors or if unified_crl is toggled
   on

* PR feedback take 2
2023-01-27 15:49:20 +00:00
Alexander Scheel
f3697c4a58
Add unified crl building (#18792)
* Add unified CRL config storage helpers

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support to build unified CRLs

This allows us to build unified versions of both the complete and delta
CRLs. This mostly involved creating a new variant of the
unified-specific CRL builder, fetching certs from each cluster's storage
space.

Unlike OCSP, here we do not unify the node's local storage with the
cross-cluster storage: this node is the active of the performance
primary, so writes to unified storage happen exactly the same as
writes to cluster-local storage, meaning the two are always in
sync. Other performance secondaries do not rebuild the CRL, and hence
the out-of-sync avoidance that we'd like to solve with the OCSP
responder is not necessary to solve here.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ability to fetch unified CRLs

This adds to the path-fetch APIs the ability to return the unified CRLs.
We update the If-Modified-Since infrastructure to support querying the
unified CRL specific data and fetchCertBySerial to support all unified
variants. This works for both the default/global fetch APIs and the
issuer-specific fetch APIs.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Rebuild CRLs on unified status changes

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Handle rebuilding CRLs due to either changing

This allows detecting if the Delta CRL needs to be rebuilt because
either the local or the unified CRL needs to be rebuilt. We never
trigger rebuilding the unified delta on a non-primary cluster.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Ensure serials aren't added to unified CRL twice

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 19:17:34 +00:00
Alexander Scheel
b0bdb2ebd4
Write delta WAL entries for unified CRLs (#18785)
* Write delta WAL entries for unified CRLs

When we'd ordinarily write delta WALs for local CRLs, we also need to
populate the cross-cluster delta WAL. This could cause revocation to
appear to fail if the two clusters are disconnected, but notably regular
cross-cluster revocation would also fail.

Notably, this commit also changes us to not write Delta WALs when Delta
CRLs is disabled (versus previously doing it when auto rebuild is
enabled in case Delta CRLs were later asked for), and instead,
triggering rebuilding a complete CRL so we don't need up-to-date Delta
WAL info.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update IMS test for forced CRL rebuilds

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 16:56:08 +00:00
Alexander Scheel
fa7c021db9
More cross cluster queue tweaks (#18789)
* Move comment about perf-primary only invalidation

Also remove noisy debug log.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove more noisy log statements during queue

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Skip revocation entries from our current cluster

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add locking and comment about tidying revoke queue

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch to time.Since for tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor tidyStatuses into path_tidy.go

Leaving these in backend.go often causes us to miss adding useful values
to tidyStatus when we add a new config parameter.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Track the number of deleted revocation request

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow tidy to remove confirmed revocation requests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing field to tidy test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 16:52:38 +00:00