* Seal HA: Use new SealWrappedValue type to abstract seal wrapped values
Introduce SealWrappedValue to abstract seal wrapped values.
Make SealWrappedValue capable of marshalling into a BlobInfo, when there is
plaintext or a single encryption, or to a custom serialization consisting of a
header, length and a marshalled MultiWrapValue protobuf.
* Vault-13769: Support configuring and using multiple seals for unsealing
* Make sealWrapBackend start using multiple seals
* Make seal.Access no longer implement wrapping.Wrapper.
Instead, add the Encrypt and Decrypt methods to the Access interface.
* Make raft snapshot system use funcs SealWrapValue + UnsealWrapValue.
Move the snapshot.Sealer implementation to the vault package to
avoid circular imports.
* Update sealWrapBackend to use multiple seals for encryption.
Use all the encryption wrappers when storing seal wrapped values.
Try do decrypt using the highest priority wrapper, but try all
combinations of encrypted values and wrappers if necessary.
* Allow the use of multiple seals for entropy augmentation
Add seal_name variable in entropy stanza
Add new MultiSourcer to accommodate the new entropy augmentation behavior.
* Individually health check each wrapper, and add a sys/seal-backend-status endpoint.
* Address a race, and also a failed test mock that I didn't catch
* Track partial wrapping failures...
... where one or more but not all access.Encrypts fail for a given write.
Note these failures by adding a time ordered UUID storage entry containing
the path in a special subdirectory of root storage. Adds a callback
pattern to accomplish this, with certain high value writes like initial
barrier key storage not allowing a partial failure. The followup work
would be to detect return to health and iterate through these storage
entries, rewrapping.
* Add new data structure to track seal config generation (#4492)
* Add new data structure to track seal config generation
* Remove import cycle
* Fix undefined variable errors
* update comment
* Update setSeal response
* Fix setSealResponse in operator_diagnose
* Scope the wrapper health check locks individually (#4491)
* Refactor setSeal function in server.go. (#4505)
Refactor setSeal function in server.go.
* Decouple CreateSecureRandomReaderFunc from seal package.
Instead of using a list of seal.SealInfo structs, make
CreateSecureRandomReaderFunc use a list of new EntropySourcerInfo structs. This
brakes the denpency of package configutil on the seal package.
* Move SealGenerationInfo tracking to the seal Access.
* Move SealGenerationInfo tracking to the seal Access.
The SealGenerationInfo is now kept track by a Seal's Access instead of by the
Config object. The access implementation now records the correct generation
number on seal wrapped values.
* Only store and read SealGenerationInfo if VAULT_ENABLE_SEAL_HA_BETA is true.
* Add MultiWrapValue protobuf message
MultiWrapValue can be used to keep track of different encryptions of a value.
---------
Co-authored-by: Victor Rodriguez <vrizo@hashicorp.com>
* Use generation to determine if a seal wrapped value is up-to-date. (#4542)
* Add logging to seal Access implementation.
* Seal HA buf format run (#4561)
* Run buf format.
* Add buf.lock to ensure go-kms-wrapping module is imported.
* Vault-18958: Add unit tests for config checks
* Add safety logic for seal configuration changes
* Revert "Add safety logic for seal configuration changes"
This reverts commit 7fec48035a5cf274e5a4d98901716d08d766ce90.
* changes and tests for checking seal config
* add ent tests
* remove check for empty name and add type into test cases
* add error message for empty name
* fix no seals test
---------
Co-authored-by: divyapola5 <divya@hashicorp.com>
* Handle migrations between single-wrapper and multi-wrapper autoSeals
* Extract method SetPhysicalSealConfig.
* Extract function physicalSealConfig.
The extracted function is the only code now reading SealConfig entries from
storage.
* Extract function setPhysicalSealConfig.
The extracted function is the only code now writing SealConfig entries from
storage (except for migration from the old recovery config path).
* Move SealConfig to new file vault/seal_config.go.
* Add SealConfigType quasy-enumeration.
SealConfigType is to serve as the typed values for field SealConfig.Type.
* Rename Seal.RecoveryType to RecoverySealConfigType.
Make RecoverySealConfigType return a SealConfigType instead of a string.
* Rename Seal.BarrierType to BarrierSealConfigType.
Make BarrierSealConfigType return a SealConfigType.
Remove seal.SealType (really a two-step rename to SealConfigType).
* Add Seal methods ClearBarrierConfig and ClearRecoveryConfig.
* Handle autoseal <-> multiseal migrations.
While going between single-wrapper and multiple-wrapper autoseals are not
migrations that require an unwrap seal (such as going from shamir to autoseal),
the stored "barrier" SealConfig needs to be updated in these cases.
Specifically, the value of SealConfg.Type is "multiseal" for autoSeals that have
more than one wrapper; on the other hand, for autoseals with a single wrapper,
SealConfig.Type is the type of the wrapper.
* Remove error return value from NewAutoSeal constructor.
* Automatically rewrap partially seal wrapped values on an interval
* Add in rewrapping of partially wrapped values on an interval, regardless of seal health/status.
* Don't set SealGenerationInfo Rewrapped flag in the partial rewrap call.
* Unexport the SealGenerationInfo's Rewrapped field, add a mutex to it for thread safe access, and add accessor methods for it.
* Add a success callback to the manual seal rewrap process that updates the SealGenerationInfo's rewrapped field. This is done via a callback to avoid an import cycle in the SealRewrap code.
* Fix a failing seal wrap backend test which was broken by the unexporting of SealGenerationInfo's Rewrapped field.
* Nil check the seal rewrap success callback before calling it.
* Change SealGenerationInfo rewrapped parameter to an atomic.Bool rather than a sync.RWMutex for simplicity and performance.
* Add nil check for SealAccess before updating SealGenerationInfo rewrapped status during seal rewrap call.
* Update partial rewrap check interval from 10 seconds to 1 minute.
* Update a reference to SealGenerationInfo Rewrapped field to use new getter method.
* Fix up some data raciness in partial rewrapping.
* Account for possibly nil storage entry when retrieving partially wrapped value.
* Allow multi-wrapper autoSeals to include disabled seal wrappers.
* Restore propagation of wrapper configuration errors by setSeal.
Function setSeal is meant to propagate non KeyNotFound errors returned by calls
to configutil.ConfigureWrapper.
* Remove unused Access methods SetConfig and Type.
* Allow multi-wrapper autoSeals to include disabled seal wrappers.
Make it possible for an autoSeal that uses multiple wrappers to include disabled
wrappers that can be used to decrypt entries, but are skipped for encryption.
e an unwrapSeal when there are disabled seals.
* Fix bug with not providing name (#4580)
* add suffix to name defaults
* add comment
* only change name for disabled seal
* Only attempt to rewrap partial values when all seals are healthy.
* Only attempt to rewrap partial values when all seals are healthy.
* Change logging level from info to debug for notice about rewrap skipping based on seal health.
* Remove stale TODOs and commented out code.
---------
Co-authored-by: rculpepper <rculpepper@hashicorp.com>
Co-authored-by: Larroyo <95649169+DeLuci@users.noreply.github.com>
Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: Divya Pola <87338962+divyapola5@users.noreply.github.com>
Co-authored-by: Matt Schultz <matt.schultz@hashicorp.com>
Co-authored-by: divyapola5 <divya@hashicorp.com>
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
For now, only the leader of a cluster can handle subscription requests,
so we forward the connection request otherwise.
We forward using a 307 temporary redirect (the fallback way).
Forwarding a request over gRPC currently only supports a single request
and response, but a websocket connection is long-lived with potentially
many messages back and forth.
We modified the `vault events subscribe` command to honor those
redirects. `wscat` supports them with the `-L` flag.
In the future, we may add a gRPC method to handle forwarding WebSocket
requests, but doing so adds quite a bit of complexity (even over
normal request forwarding) due to the intricate nature of the `http` /
`vault.Core` interactions required. (I initially went down this path.)
I added tests for the forwarding header, and also tested manually.
(Testing with `-dev-three-node` is a little clumsy since it does not
properly support experiments, for some reason.)
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* add a test to show the bug
* do not output a "Success!" message if a specific field was requested
* Create 21545.txt
* Fix changelog name
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* Initial oss-patch apply
* Added changelog
* Renamed changelog txt
* Added the imports to the handler file
* Added a check that no two ports are the same, and modified changelog
* Edited go sum entry
* Tidy up using go mod
* Use strutil instead
* Revert go sum and go mod
* Revert sdk go sum
* Edited go.sum to before
* Edited go.sum again to initial
* Revert changes
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Updating the license from MPL to Business Source License.
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.
* add missing license headers
* Update copyright file headers to BUS-1.1
* Fix test that expected exact offset on hcl file
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
* Major overhaul of `vault operator generate-root` CLI help
Resolves#15252
A major overhaul of the `vault operator generate-root` CLI help to
surface the fact that it is actually six separate commands in one,
rather than requiring users to independently deduce this mental model
themselves.
In the process of doing so, also standardize some terminology:
* Fix places which used the phrase "operational token" instead of
"operation token" to be consistent with the prevailing terminology.
* Fix places which used the phrase "recovery operation token" instead of
"recovery token" to be consistent with the prevailing terminology.
This PR currently focusses on the CLI help, but following review and
feedback, I assume I'll need to replicate many of the same changes in
website/content/docs/commands/operator/generate-root.mdx as well.
* Fix some tab characters which should have been spaces
* Update command/operator_generate_root.go
---------
Co-authored-by: Violet Hynes <violet.hynes@hashicorp.com>
* Automatically track subloggers in allLoggers
This PR introduces a new interface called `SubloggerAdder` for tracking
allLoggers across startup phases.
The interface enables us to register a single `SubloggerHook` during
initial logger creation and hand off management of `allLoggers` during
the three phases we need to handle:
- Before `CoreConfig` is created, the `SubloggerHook`
(`AppendToAllLoggers`) appends new subloggers to
`ServerCommand.allLoggers`.
- After `CoreConfig` is created and before `NewCore` returns, new subloggers
are added to `CoreConfig.AllLoggers`. Intermediate state must also be
kept in sync within NewCore to track new subloggers before we return
to the server command and register the `Core.SubloggerAdder`
implementation.
- After `NewCore` returns to the server command, we register Core as the
implementer of `ServerCommand.SubloggerAdder` ensuring that all new
subloggers are appended to `Core.allLoggers`.
* Wire up the sublogger hook in NewTestLogger
Add a go:generate helper called stubmaker, which generates appropriate stubs on ent based on oss stubs, but only when needed (i.e. real ent funcs haven't been added yet.)
* add config changes for name and priority fields in seal stanza
* change env vars and fix tests
* add header and fix func call
* tweak limits on seals
* fix missing import
* add docstrings
* Replace all time.ParseDurations with testutil.ParseDurationSeconds
* Changelog
* Import formatting
* Import formatting
* Import formatting
* Import formatting
* Semgrep rule that runs as part of CI
* added exec and env_template config/parsing
* add tests
* we can reuse ctconfig here
* do not create a non-nil map
* check defaults
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* first go of exec server
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* convert to list
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* convert to list
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* sig test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add failing example
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* refactor for config changes
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add test for invalid signal
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* account for auth token changes
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* only start the runner once we have a token
* tests in diff branch
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* fix rename
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update command/agent/exec/exec.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* apply suggestions from code review
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* cleanup
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove unnecessary lock
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* refactor to use enum
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* dont block
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* handle default
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* make more explicit
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* cleanup
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove unused
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove unused file
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove test app
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* apply suggestions from code review
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* update comment
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add changelog
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* new channel for exec server token
* wire to run with vault agent
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* watch for child process to exit on its own
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* block before returning
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* added exec and env_template config/parsing
* add tests
* we can reuse ctconfig here
* do not create a non-nil map
* check defaults
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* first go of exec server
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* convert to list
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* convert to list
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* sig test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add failing example
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* refactor for config changes
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add test for invalid signal
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* account for auth token changes
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* only start the runner once we have a token
* tests in diff branch
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* fix rename
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update command/agent/exec/exec.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* apply suggestions from code review
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* cleanup
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove unnecessary lock
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* refactor to use enum
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* dont block
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* handle default
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* make more explicit
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* cleanup
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove unused
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove unused file
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove test app
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* apply suggestions from code review
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* update comment
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add changelog
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* watch for child process to exit on its own
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Add ACME health checks to pki health-check CLI
- Verify we have the required header values listed within allowed_response_headers: 'Replay-Nonce', 'Link', 'Location'
- Make sure the local cluster config path variable contains an URL with an https scheme
* Split ACME health checks into two separate verifications
- Promote ACME usage through the enable_acme_issuance check, if ACME is disabled currently
- If ACME is enabled verify that we have a valid
'path' field within local cluster configuration as well as the proper response headers allowed.
- Factor out response header verifications into a separate check mainly to work around possible permission issues.
* Only recommend enabling ACME on mounts with intermediate issuers
* Attempt to connect to the ACME directory based on the cluster path variable
- Final health check is to attempt to connect to the ACME directory based on the cluster local 'path' value. Only if we successfully connect do we say ACME is healthy.
* Fix broken unit test
* upgrade go-jose library to v3
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
* chore: fix unnecessary import alias
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
* upgrade go-jose library to v2 in vault
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
---------
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
* added exec and env_template config/parsing
* add tests
* we can reuse ctconfig here
* do not create a non-nil map
* check defaults
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* convert to list
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* convert to list
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* sig test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add failing example
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add test for invalid signal
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update command/agent/config/config.go
* use latest consul-template
* fix build
* fix test
* fix test fixtures
* make fmt
* test docs
* rename file
* env var -> environment variable
* default to SIGTERM
* empty line
* explicit naming
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* clean typo
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* replace $ HOME with /home/username in examples
* remove empty line
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <anton.averchenkov@hashicorp.com>
* refactor code focused on DevTLS mode into new function
* add tests for configureDevTLS function
* replace testcase comments with fields in testcase struct
