Commit Graph

1725 Commits

Author SHA1 Message Date
Steven Clark
293e8b8ac5
Fix enterprise failure of TestCRLIssuerRemoval (#23038)
This fixes the enterprise failure of the test
 ```
  === FAIL: builtin/logical/pki TestCRLIssuerRemoval (0.00s)
     crl_test.go:1456:
         	Error Trace:	/home/runner/actions-runner/_work/vault-enterprise/vault-enterprise/builtin/logical/pki/crl_test.go:1456
         	Error:      	Received unexpected error:
         	            	Global, cross-cluster revocation queue cannot be enabled when auto rebuilding is disabled as the local cluster may not have the certificate entry!
         	Test:       	TestCRLIssuerRemoval
         	Messages:   	failed enabling unified CRLs on enterprise

 ```
2023-09-13 08:11:52 -04:00
Alexander Scheel
e2ff1f1c71
Clean up unused CRL entries when issuer is removed (#23007)
* Clean up unused CRL entries when issuer is removed

When a issuer is removed, the space utilized by its CRL was not freed,
both from the CRL config mapping issuer IDs to CRL IDs and from the
CRL storage entry. We thus implement a two step cleanup, wherein
orphaned CRL IDs are removed from the config and any remaining full
CRL entries are removed from disk.

This relates to a Consul<->Vault interop issue (#22980), wherein Consul
creates a new issuer on every leadership election, causing this config
to grow. Deleting issuers manually does not entirely solve this problem
as the config does not fully reclaim space used in this entry.

Notably, an observation that when deleting issuers, the CRL was rebuilt
on secondary clusters (due to the invalidation not caring about type of
the operation); for consistency and to clean up the unified CRLs, we
also need to run the rebuild on the active primary cluster that deleted
the issuer as well.

This approach does allow cleanup on existing impacted clusters by simply
rebuilding the CRL.

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test case on CRL removal

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-09-12 20:56:47 +00:00
Scott Miller
7d4d8cb708
Reject supplied nonces for non-convergent encryption operations (#22852)
* Ignore nonces when encrypting without convergence or with convergence versions > 1

* Honor nonce use warning in non-FIPS modes

* Revert "Honor nonce use warning in non-FIPS modes"

This reverts commit 2aee3dbdc1.

* Add a test func that removes a nonce when not needed

* err out rather than ignore the nonce

* Alter unit test to cover, also cover convergent version 3

* More unit test work

* Fix test 14

* changelog

* tests not already in a nonce present path

* Update unit test to not assume warning when nonce provided incorrectly

* remove unused test field

* Fix auto-squash events experiments

When #22835 was merged, it was auto-squashed, so the `experiments`
import was removed, but the test still referenced it.

This removes the (now unnecessary) experiment from the test.

* Allow nonces for managed keys, because we have no way of knowing if the backing cipher/mode needs one

---------

Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
2023-09-08 08:07:33 -05:00
John-Michael Faircloth
16f805419f
fix rotation_window bug in error path (#22699) 2023-08-31 15:45:01 -05:00
John-Michael Faircloth
aa05ba6105
adv ttl mgmt: define schedule interface (#22590) 2023-08-28 13:14:38 -07:00
Thomas Schweizer-Bolzonello
958eb96bab
Fix typo OSCP -> OCSP (#22586) 2023-08-28 18:39:01 +00:00
Max Bowsher
479519e6df
Fix erroneous OpenAPI operation IDs (#22579)
The recent additions to the transit secret engine have created two new
endpoints which both have the incorrect (and duplicate) operation ID of
just `"update"`.

Amend to unique meaningful values.
2023-08-28 09:11:16 -04:00
Milena Zlaticanin
2d0d5c79ed
Add the ability to set seconds in cron schedule for testing purposes (#22531)
* add rotation_schedule field to db backend

* add cron schedule field

* use priority queue with scheduled rotation types

* allow marshalling of cron schedule type

* return warning on use of mutually exclusive fields

* handle mutual exclusion of rotation fields (#22306)

* handle mutual exclusion of rotation fields

* fix import

* adv ttl mgmt: add rotation_window field (#22303)

* adv ttl mgmt: add rotation_window field

* do some rotation_window validation and add unit tests

* adv ttl mgmt: Ensure initialization sets appropriate rotation schedule (#22341)

* general cleanup and refactor rotation type checks

* make NextRotationTime account for the rotation type

* add comments

* add unit tests to handle mutual exclusion (#22352)

* add unit tests to handle mutual exclusion

* revert rotation_test.go and add missing test case to path_roles_test.go

* adv ttl mgmt: add tests for init queue (#22376)

* Vault 18908/handle manual rotation (#22389)

* support manual rotation for schedule based roles

* update description and naming

* adv ttl mgmt: consider rotation window (#22448)

* consider rotation window

ensure rotations only occur within a rotation window for schedule-based
rotations

* use helper method to set priority in rotateCredential

* fix bug with priority check

* remove test for now

* add and remove comments

* add unit tests for manual rotation (#22453)

* adv ttl mgmt: add tests for rotation_window

* adv ttl mgmt: refactor window tests (#22472)

* Handle GET static-creds endpoint (#22476)

* update read static-creds endpoint to include correct resp data

* return rotation_window if set

* update

* add changelog

* add unit test for static-creds read endpoint (#22505)

* Add the ability to set seconds in cron schedule for testing purposes

* update test so we don't use global var

* update with suggestions

---------

Co-authored-by: JM Faircloth <jmfaircloth@hashicorp.com>
Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
2023-08-25 09:42:15 -07:00
John-Michael Faircloth
83f3e391c2
secrets/database: advanced TTL management for static roles (#22484)
* add rotation_schedule field to db backend

* add cron schedule field

* use priority queue with scheduled rotation types

* allow marshalling of cron schedule type

* return warning on use of mutually exclusive fields

* handle mutual exclusion of rotation fields (#22306)

* handle mutual exclusion of rotation fields

* fix import

* adv ttl mgmt: add rotation_window field (#22303)

* adv ttl mgmt: add rotation_window field

* do some rotation_window validation and add unit tests

* adv ttl mgmt: Ensure initialization sets appropriate rotation schedule (#22341)

* general cleanup and refactor rotation type checks

* make NextRotationTime account for the rotation type

* add comments

* add unit tests to handle mutual exclusion (#22352)

* add unit tests to handle mutual exclusion

* revert rotation_test.go and add missing test case to path_roles_test.go

* adv ttl mgmt: add tests for init queue (#22376)

* Vault 18908/handle manual rotation (#22389)

* support manual rotation for schedule based roles

* update description and naming

* adv ttl mgmt: consider rotation window (#22448)

* consider rotation window

ensure rotations only occur within a rotation window for schedule-based
rotations

* use helper method to set priority in rotateCredential

* fix bug with priority check

* remove test for now

* add and remove comments

* add unit tests for manual rotation (#22453)

* adv ttl mgmt: add tests for rotation_window

* adv ttl mgmt: refactor window tests (#22472)

* Handle GET static-creds endpoint (#22476)

* update read static-creds endpoint to include correct resp data

* return rotation_window if set

* update

* add changelog

* add unit test for static-creds read endpoint (#22505)

---------

Co-authored-by: Milena Zlaticanin <60530402+Zlaticanin@users.noreply.github.com>
2023-08-24 16:45:07 -05:00
Alexander Scheel
a2a4831754
Minor improvements to transit & csrs (#22494)
* Remove note

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove duplicate curve check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove CreateOperation from new Transit X.509 endpoints

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test to validate key matching works

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-08-22 10:51:37 -04:00
Gabriel Santos
1996377b4f
Transit backend: Create CSR's from keys in transit and import certificate chains (#21081)
* setup initial boilerplate code for sign csr endpoint

* add function to sign csr

* working version of sign csr endpoint

* improving errors for csr create and sign endpoint

* initial implementation for import leaf certificate endpoint

* check if more than one certificate was provided in the ceritificate chain

* improve validate cert public key matches transit key

* convert provided cert chain from PEM to DER so it can be parsed by
x509.ParseCertificates and fixing other bugs

* fix creation of csr from csrTemplate

* add missing persist of certificate chain after validations in set-certificate endpoint

* allow exporting a certificate-chain

* move function declaration to end of page

* improving variable and function names, removing comments

* fix certificate chain parsing - work in progress

* test for signCsr endpoint

* use Operations instead of Callbacks in framework.Path

* setup test for set-certificate endpoint

fix problems with sign-csr endpoint returning base64

* finish set-certificate endpoint test

* use public key KeyEntry fields instead of retrieving public key from private

* improve error message and make better distinction between client and server error

also moved check of key types before checking if key match to endpoint handler

* check if private key has been imported for key version selected when signing a csr

* improve errors

* add endpoint description and synopsis

* fix functions calls in backend as function names changed

* improve import cert chain test

* trim whitespaces on export certificate chain

* changelog

* pass context from handler function to policy Persist

* make fmt run

* fix: assign returned error from PersistCertificateChain to err so it can be evaluated

* additional validations and improvements to parseCertificateChain function

* add validation to check if there is only one certificate in the certificate chain and it is in the first position

* import cert chain test: move creation of cluster to exported test function

* move check of end-cert pub key algorithm and key transit algorithm match into a separate function

* test export certificate chain

* Update sdk/helper/keysutil/policy.go

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>

* fix validateLeafCertPosition

* reject certificate actions on policies that allow key derivation and remove derived checks

* return UserError from CreateCSR SDK function as 400 in transit API handler

* add derived check for ED5519 keys on CreateCSR SDK func

* remove unecessary calls of x509.CreateCertificateRequest

* move validate key type match back into SDK ValidateLeafCertMatch function

* add additional validations (ValidateLeafCertKeyMatch, etc) in SDK PersistCertificateChain function

* remove uncessary call of ValidateLeafCertKeyMatch in parseImportCertChainWrite

* store certificate chain as a [][]byte instead of []*x509.Certificate

* include persisted ca chain in import cert-chain response

* remove NOTE comment

* allow exporting cert-chain even if exportable is set as false

* remove NOTE comment

* add certifcate chain to formatKeyPublic if present

also added an additional check to validate if field is added when
certchain is present

---------

Co-authored-by: Alexander Scheel <alexander.m.scheel@gmail.com>
2023-08-22 08:24:56 -04:00
hashicorp-copywrite[bot]
0b12cdcfd1
[COMPLIANCE] License changes (#22290)
* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Adding explicit MPL license for sub-package.

This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.

* Updating the license from MPL to Business Source License.

Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.

* add missing license headers

* Update copyright file headers to BUS-1.1

* Fix test that expected exact offset on hcl file

---------

Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2023-08-10 18:14:03 -07:00
Steven Clark
8f646d52be
Refactor sign-intermediate API response (#22269)
* Refactor sign-intermediate API response

 - Allow the sign-intermediate response handling code to be shared
   across different API calls.

* Add missing cieps.go
2023-08-10 09:58:07 -04:00
Austin Gebauer
a70aaf24c0
secrets/db: improves error logs for static role rotation (#22253)
* secrets/db: improves error logs for static role rotation

* use logger.With to add incremental context

* adds changelog
2023-08-08 16:28:31 -07:00
Steven Clark
64460dd58b
Refactor logic on applying leaf_not_after_behavior (#22230)
- Allow us to call and apply an issuer's leaf_not_after_behavior
   to a leaf certificate from various locations.
2023-08-08 13:39:54 +00:00
Steven Clark
8a174b05b0
Add timeouts to ACME test contexts (#22205)
- The Golang ACME library will retry on failures until the passed in
   context times out. So if a test is broken it will never end until
   the context expires.
 - The timeouts are rather large to avoid some issues with slow CI
   runners, that have already triggered on a nightly 32 bit run
2023-08-04 11:09:37 -04:00
Steven Clark
3a46df2077
Use Hashicorp docker proxy for CI docker images (#22195)
* Use Hashicorp docker proxy for CI docker images

* Use library/consul as the mirror path instead of hashicorp/consul

 - Looks like the older 1.4.4 image was not published within the
   hashicorp/consul space, only newer images are.
 - Switch to library/consul which seems to have both versions
2023-08-03 13:42:20 -04:00
Steven Clark
66052d5431
Refactor ACME for enterprise CIEPS integration (#22188) 2023-08-03 11:24:46 -04:00
Kushneryk Pavel
64f92b40fc
bug(20562): allowed_domains are compared case-sensitive if they use g… (#22126)
* bug(20562): allowed_domains are compared case-sensitive if they use glob patterns

* bug(20562): review fixes
2023-08-02 11:09:10 -04:00
Steven Clark
6ff7d38f6a
Refactor sign/issue response logic to be usable by CIEPS codebase (#22053)
- Extract out the sign/issue certificate (non-CA) response into
   a function that can be used by the CIEPS sign/issue API handlers.
2023-07-25 15:11:09 -04:00
Steven Clark
068da27a2d
Use hclog instead of t.Logf within PKI ACME test suite (#22049)
- Avoid issues with t.Logf data race issues if a log message is
   called post test completion from a background go routine.
2023-07-25 13:24:39 -04:00
Max Bowsher
e057ee0750
OpenAPI: Define default response structure for ListOperations (#21934)
* OpenAPI: Define default response structure for ListOperations

Almost all Vault ListOperation responses have an identical response
schema. Update the OpenAPI generator to know this, and remove a few
instances where that standard response schema had been manually
copy/pasted into place in individual endpoints.

* changelog

* Only render StandardListResponse schema, if an operation uses it

* Teach the response schema validation test helper about the default list schema too

---------

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-07-25 11:22:33 -04:00
Max Bowsher
fe013a987a
Add missing Query: true metadata to API definitions (#21949)
* Add missing `Query: true` metadata to API definitions

Also improve the documentation comment for `Query` to guide people better how they should be setting `Query` in the future.

Endpoints affected:
- auth/approle/role/{role_name}/secret-id/destroy
- auth/approle/role/{role_name}/secret-id-accessor/destroy
- auth/token/lookup
- auth/token/lookup-self
- sys/internal/specs/openapi
- sys/wrapping/lookup
- identity/oidc/provider/{name}/authorize

There are also endpoints in the `aws` and `gcp` secrets engines which need the same treatment in their own PRs.

When working on the `auth/token/lookup-self` path, I discovered that it
had a parameter which was completely pointless - it was even documented
as unused. It only existed because the `auth/token/lookup-self` code
path was implemented by bodging the current token into the request data
and passing control to the `auth/token/lookup` handler directly -
instead of just factoring out the common code to a reusable function -
so I fixed that whilst I was there.

Note that two of the affected endpoints currently have one form of their
OpenAPI operation ID set to something mentioning "with-parameters":
- identity/oidc/provider/{name}/authorize
- sys/internal/specs/openapi

These operation IDs should be changed, as they perpetuate
a misunderstanding - both read (GET) and update (POST/PUT) forms of
these APIs are **equally** capable of being used with parameters.

* I failed to spot that the aws plugin is in-repo! Update that too.

* Remove code cleanup changes from this PR

* Wording and wrapping adjustment as requested.
2023-07-21 15:05:29 -04:00
Max Bowsher
00e13abf1f
CreateOperation should only be implemented alongside ExistenceCheck (#18492)
* CreateOperation should only be implemented alongside ExistenceCheck

Closes #12329

Vault treats all POST or PUT HTTP requests equally - they default to
being treated as UpdateOperations, but, if a backend implements an
ExistenceCheck function, CreateOperations can be separated out when the
existence check returns false.

It follows, then, that if a CreateOperation handler is implemented
without an ExistenceCheck function, this is unreachable code - a coding
error. It's a fairly minor error in the grand scheme of things, but it
causes the generated OpenAPI spec to include x-vault-createSupported for
operations on which create can never actually be invoked - and promotes
muddled understanding of the create/update feature.

In this PR:

1) Implement a new test, which checks all builtin auth methods and
   secrets engines can be successfully initialized. (This is important
   to validate the next part.)

2) Expand upon the existing coding error checks built in to
   framework.Backend, adding a check for this misuse of CreateOperation.

3) Fix up instances of improper CreateOperation within the Vault
   repository - just two, transit and mock.

Note: At this point, the newly added test will **fail**.

There are improper uses of CreateOperation in all of the following:

    vault-plugin-auth-cf
    vault-plugin-auth-kerberos
    vault-plugin-auth-kubernetes
    vault-plugin-secrets-ad
    vault-plugin-secrets-gcpkms
    vault-plugin-secrets-kubernetes
    vault-plugin-secrets-kv
    vault-plugin-secrets-openldap
    vault-plugin-secrets-terraform

each of which needs to be fixed and updated in go.mod here, before this
new check can be added.

* Add subtests

* Add in testing of KV v2, which otherwise doesn't get tested

This is a surprisingly complicated special case

* The database plugin needs special handling as well, and add in help invocations of the builtin backends too

* Fix extra package prefix

* Add changelog

* Update 6 out of 9 plugins to needed new versions

Note, this IS an upgrade despite the apparent version numbers going
down. (That's a consequence of slightly odd release management occurring
in the plugin repositories.)

* Update to deal with code changes since branch originally created

* Perform necessary update of vault-plugin-secrets-kubernetes so that CI checks on PR can run

* Fix another instance of incorrect CreateOperation, for a test-only endpoint

By being hidden behind a Go build constraint, it had evaded notice until
now.

* Add an opportunistic test of sys/internal/specs/openapi too
2023-07-18 13:44:15 +01:00
Alexander Scheel
4ec5e22ade
Fix ACME tidy to not reference acmeContext (#21870)
* Fix ACME tidy to not reference acmeCtx

acmeContext is useful for when we need to reference things with a ACME
base URL, but everything used in tidy doesn't need this URL as it is not
coming from an ACME request.

Refactor tidy to remove references to acmeContext, including dependent
functions in acme_state.go.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove spurious log message

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Draft Tidy Acme Test with Backdate Storage + Backdate Sysxsx

* Fixes to ACME tidy testing

Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly set account kid to update account status

Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add TestTidyAcmeWithSafetyBuffer

Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test for disabling tidy operation

Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add acme_account_safety_buffer to auto-tidy config

Resolve: #21872

Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests verifying tidy safety buffers

Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add account status validations and order cleanup tests

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: kitography <khaines@mit.edu>
Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2023-07-17 14:54:28 -04:00
Laurent
366693c78d
Add a parameter to allow ExtKeyUsage field usage from a role within ACME (#21702)
* imprv: Add a parameter to allow ExtKeyUsage field usage from a role

* chore: Add the changelog entry

* imprv: Reword UI and changelog

* doc: Add allow_role_extkeyusage in parameter list

* imprv: Align variable names with existing fields/codebase

* Add unit test and tweak some labels

---------

Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2023-07-17 09:31:13 -04:00
Alexander Scheel
8c7e5d7a3a
Mark config dirty if read differs from state (#21835)
* Refactor CRL writing config to passthrough cache

When reading the CRL config via API endpoint, always read through to the
disk, updating the cache in the process. Similarly, when writing to the
CRL config, read first from disk (updating the cache), and on write,
also write back through the cache, providing consistency without the
need to invalidate through markConfigDirty(...).

This will form the basis of the new pattern for config caching.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor ACME writing config to passthrough cache

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-07-14 13:12:26 +00:00
Steven Clark
d01f354aa2
Update TestProperAuthing test (#21841)
- Updating existing entries in OSS so that this can be backported.
 - The proper fix on main going forward it to remove these ENT only
   entries and move them into the enterprise only file
2023-07-14 06:56:06 -04:00
Max Bowsher
e47dd9df53
OpenAPI: Separate ListOperation from ReadOperation (#21723)
* OpenAPI: Separate ListOperation from ReadOperation

Historically, since Vault's ReadOperation and ListOperation both map to
the HTTP GET method, their representation in the generated OpenAPI has
been a bit confusing.

This was partially mitigated some time ago, by making the `list` query
parameter express whether it was required or optional - but only in
a way useful to human readers - the human had to know, for example, that
the schema of the response body would change depending on whether `list`
was selected.

Now that there is an effort underway to automatically generate API
clients from the OpenAPI spec, we have a need to fix this more
comprehensively. Fortunately, we do have a means to do so - since Vault
has opinionated treatment of trailing slashes, linked to operations
being list or not, we can use an added trailing slash on the URL path to
separate list operations in the OpenAPI spec.

This PR implements that, and then fixes an operation ID which becomes
duplicated, with this change applied.

See also hashicorp/vault-client-go#174, a bug which will be fixed by
this work.

* Set further DisplayAttrs in auth/github plugin

To mask out more duplicate read/list functionality, now being separately
generated to OpenAPI client libraries as a result of this change.

* Apply requested changes to operation IDs

I'm not totally convinced its worth the extra lines of code, but
equally, I don't have strong feelings about it, so I'll just make the
change.

* Adjust logic to prevent any possibility of generating OpenAPI paths with doubled final slashes

Even in the edge case of improper use of regex patterns and operations.

* changelog

* Fix TestSudoPaths to pass again... which snowballed a bit...

Once I looked hard at it, I found it was missing several sudo paths,
which led to additional bug fixing elsewhere.

I might need to pull some parts of this change out into a separate PR
for ease of review...

* Fix other tests

* More test fixing

* Undo scope creep - back away from fixing sudo paths not shown as such in OpenAPI, at least within this PR

Just add TODO comments for now.
2023-07-13 13:36:52 -04:00
Steven Clark
8d74143872
Add additional debug output to TestCertStorageMetrics (#21793)
- Add a little more debug to try and understand how we could get the
   following error I saw within GHA
```
    path_tidy_test.go:769: Created root and leaf certificate, deleted leaf, but a got a certificate count of 2
```
 - Fix some other issues discovered while running the test through
   stress locally
2023-07-13 09:03:23 -04:00
Austin Gebauer
bf19846b18
Adds replication state helper to framework.Backend (#21743)
* Adds replication state helper to framework.Backend

* Fix test

* adds changelog
2023-07-11 15:22:28 -07:00
Steven Clark
11691b3b37
Allow PKI enterprise to influence TestProperAuthing test case (#21761)
- Allow enterprise repository to hook into and influence the proper
   authing test so we don't have to add values to OSS for enterprise
   work
2023-07-11 13:11:28 -04:00
Steven Clark
1bed33a399
Add the ability to add enterprise only backend features in PKI - OSS (#21748)
To allow us to support CIEPS backend state, allow the backend to
contain enterprise only state variables. Also allow us to implement
enterprise only hooks into the various backend functions to initialize,
periodicFunc, cleanup and invalidate.
2023-07-11 10:05:23 -04:00
Alexander Scheel
b0fb3b1420
pre-commit hook for formatting backend code (#21682)
* Add backend format linting to pre-commit hook

By taking a slight penalty with each commit, we can ensure that
contributors follow the format behavior by default (if they run hooks),
making accidental PRs without proper formatting less likely.
Additionally, fix gofmtcheck to align with the Makefile, fixing the
corresponding fmtcheck target for use with the hook.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix formatting errors

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-07-10 16:49:47 +00:00
Steven Clark
e29842e49e
Address data races in TestCertStorageMetrics due to plugin reloading (#21679)
- There's a race within the Plugin reloading mechanism that isn't
   trivial to address. To silence some of the failures, switch this
   test to use sealing of the cores instead of the plugin reload
   mechanism
2023-07-10 09:31:34 -04:00
Steven Clark
bce4b7a3e5
Attempt to address autoloading issues within Bind backend tests (#21694)
- We've seen a few issues with bind's auto-loading of configuration
   too quickly at bad times leading to it having partial configurations
   or not all files/permissions being restored properly during it's read
   attempt.
 - See if the freeze/thaw rndc commands will help out with these timing
   issues
2023-07-07 16:29:39 -04:00
Alexander Scheel
072f0dd7c8
Add explicit tests for Extensions, other Subject attributes (#21651)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-07-07 11:29:22 -04:00
Steven Clark
9b9017a926
Add the ability for enterprise to override PKI backend attributes during setup (#21618) 2023-07-06 16:16:17 -04:00
Steven Clark
5ce57dbd00
Fix incorrect role ttl parameters in ACME tests (#21585)
- The ACME tests were using ttl_duration and max_ttl_duration instead
   of ttl and max_ttl as input parameters to roles.
 - Add missing copyright headers
2023-07-05 14:17:15 -04:00
Steven Clark
e3b3c7a8de
Fix OpenAPI spec definitions for PKI EAB APIs (#21458)
* Fix OpenAPI spec definitions for PKI EAB APIs

 - Do not generate duplicate operation ids for the various new-eab apis
 - Fill out proper operation verb for eab delete call
 - Pluralize operation verb for list-eab-keys api
 - Fill out proper response data for new-eab and list-eab-keys

* Add cl
2023-06-27 08:44:21 -04:00
Anton Averchenkov
df304012f3
openapi: Fix response schema for PKI Issue requests (#21449)
* openapi: Fix response schema for PKI Issue requests

* tests

* changelog

* another expiration for generate/rotate root

* more type fixes from @stevendpclark
2023-06-26 13:31:56 -04:00
Christopher Swenson
cf48236a3c
Move database connections map out to separate package (#21207)
The upcoming event main plugin will use a very similar pattern
as the database plugin map, so it makes sense to refactor this and move
this map out. It also cleans up the database plugin backend so that
it does not have to keep track of the lock.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-06-22 10:20:13 -07:00
Alexander Scheel
15aee2e0ba
Fix re-migration of existing CA bundles (#21316)
* Fix re-migration of existing version 1 storage bundles

Related: VAULT-17307

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test for v1->v2 migration post-issuer deletion

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add known-issue about PKI double migration

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update website/content/partials/pki-double-migration-bug.mdx

Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>

* Update website/content/partials/pki-double-migration-bug.mdx

Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>

* Update website/content/partials/pki-double-migration-bug.mdx

* Additional clarity around known issue

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Sarah Chavis <62406755+schavis@users.noreply.github.com>
2023-06-21 20:34:51 +00:00
Violet Hynes
3a46ecc389
Replace all remaining time.ParseDurations with parseutil.ParseDurationSeconds (#21357)
* Replace all time.ParseDurations with testutil.ParseDurationSeconds

* Changelog

* Import formatting

* Import formatting

* Import formatting

* Import formatting

* Semgrep rule that runs as part of CI
2023-06-20 14:37:46 -04:00
Steven Clark
042dd57811
Supporting PR for Enterprise ACME PR cluster tests (#21275)
* Supporting PR for Enterprise ACME PR cluster tests

 - Some changes within the OSS test helpers to help in the ACME Enterprise test cases.

* Don't rename existing helper method to make oss/ent merge easier
2023-06-15 21:12:49 +00:00
Matt Schultz
8cc7be234a
Adds automated ACME tests using Caddy. (#21277)
* Adds automated ACME tests using Caddy.

* Do not use CheckSignatureFrom method to validate TLS-ALPN-01 challenges

* Uncomment TLS-ALPN test.

* Fix validation of tls-alpn-01 keyAuthz

Surprisingly, this failure was not caught by our earlier, but unmerged
acme.sh tests:

> 2023-06-07T19:35:27.6963070Z [32mPASS[0m builtin/logical/pkiext/pkiext_binary.Test_ACME/group/acme.sh_tls-alpn (33.06s)

from https://github.com/hashicorp/vault/pull/20987.

Notably, we had two failures:

 1. The extension's raw value is not used, but is instead an OCTET
    STRING encoded version:

    > The extension has the following ASN.1 [X.680] format :
    >
    > Authorization ::= OCTET STRING (SIZE (32))
    >
    > The extnValue of the id-pe-acmeIdentifier extension is the ASN.1
    > DER encoding [X.690] of the Authorization structure, which
    > contains the SHA-256 digest of the key authorization for the
    > challenge.
 2. Unlike DNS, the SHA-256 is directly embedded in the authorization,
    as evidenced by the `SIZE (32)` annotation in the quote above: we
    were instead expecting this to be url base-64 encoded, which would
    have a different size.

This failure was caught by Matt, testing with Caddy. :-)

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Quick gofmt run.

* Fix challenge encoding in TLS-ALPN-01 challenge tests

* Rename a PKI test helper that retrieves the Vault cluster listener's cert to distinguish it from the method that retrieves the PKI mount's CA cert. Combine a couple of Docker file copy commands into one.

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-06-15 20:44:09 +00:00
Alexander Scheel
052719b9a8
Allow issuance of root certs no AIA when templating is enabled (#21209)
* Ignore templated AIA on root generation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test case verifying that roots generate OK

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add warning on generation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-06-14 13:35:17 -04:00
Alexander Scheel
b346b52b62
Correctly validate ACME PoP against public key (#21190)
* Correctly validate ACME PoP against public key

ACME's proof of possession based revocation uses a signature from the
private key, but only sends the public copy along with the request.
Ensure the public copy matches the certificate, instead of failing to
cast it to a private key.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add ACME revocation tests

* Clarify commentary in acmeRevocationByPoP

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
2023-06-14 12:01:16 -04:00
Steven Clark
3b14cd2061
Allow CSRs with basic constraint extension with IsCA=false (#21149)
* Allow CSRs with basic constraint extension with IsCA=false

 - We previously forbid any CSR with a basic constraint extension within the CSR.
 - It was discovered that some ACME clients (Proxmox ACME client) do send us this extension with a value of IsCA to false.
 - So allow the extension to be set within the ACME CSR with
   a value of IsCA set to false
 - Add a test for both the IsCA=true and IsCA=false use-cases and make sure we don't actually set the extension back within the generated certificate.

* PR feedback

 - Move basic constraint function to sdk, increase test coverage
 - Error out on extra characters being returned from the asn1 unmarshalling.

* make fmt
2023-06-13 13:15:37 -04:00
Steven Clark
aba75aac86
Verify FailureToTolerance prior to sealing in ACME test (#21167)
- When running the SubtestACMEStepDownNode by itself we would be sealing the active node within the cluster too quickly and would end up with the other nodes failing to become an active node with the message: not part of stable configuration, aborting election
 - Add an extra check that the raft autopilot state is healthy and that FailureToTolerance has a value of 1 or higher before letting the test continue.
2023-06-13 12:37:09 -04:00