43 Commits

Author SHA1 Message Date
Matt Keeler
cdc5fa796e
Introduce hashicorp/go-metrics compatibility to the SDK module (#29358)
* Introduce hashicorp/go-metrics compatibility

Compatability is attained with build tags

Using tag armonmetrics or no tag at all will result in the library utilizing github.com/armon/go-metrics for metrics emission

Using tag hashicorpmetrics will result in the library utilizing the updated github.com/hashicorp/go-metrics for metrics emission.

Bump root module go.mod to pull in indirect hashicorp/go-metrics dependency from the sdk module

Update module readme.

Updates

* Finish the sentence.

* Update sdk/README.md

Co-authored-by: Paul Banks <pbanks@hashicorp.com>

* Fix up errant usage of non-compat module

* Fix go fmt

---------

Co-authored-by: Paul Banks <pbanks@hashicorp.com>
Co-authored-by: Josh Black <raskchanky@gmail.com>
2025-04-10 11:20:49 -05:00
Johan Brandhorst-Satzkorn
8d83c5d047
physical: use permitpool from go-secure-stdlib (#29331)
* sdk/physical: use permitpool from go-secure-stdlib

* physical: use permitpool from go-secure-stdlib

* fixup! sdk/physical: use permitpool from go-secure-stdlib

* fixup! sdk/physical: use permitpool from go-secure-stdlib
2025-01-24 12:33:44 -05:00
Andy Bao
4b456ffcec
Fix OSS sealunwrapper adding extra get + put request to all storage get requests (#29050)
* fix OSS sealunwrapper adding extra get + put request to all storage requests

* Add changelog entry
2024-11-29 18:19:45 +01:00
miagilepner
41c0a3040b
VAULT-31748: add removable node HA backend interface and raft implementation (#28726) 2024-10-30 18:19:49 +01:00
Paul Banks
c839854483
Overload error support for Adaptive Overload Protection (Enterprise) (#26688)
* Overload error support for Enterprise

* Remove TODO comment
2024-04-29 22:11:23 +01:00
Paul Banks
c605d1a846
Plumbing to support mount table path registration (#26176) 2024-03-27 13:25:31 +00:00
Paul Banks
3a2a922b26
Raft config refactor for mount entry size limit (#25992)
* CE parts for mount-namespace entry limit

* Remove redundant code from refactor

* Add doc comment note about ent-only use of interface

* Add CHANGELOG
2024-03-19 17:28:23 +00:00
Hamid Ghaf
a4be0c940c
vault-23135 - fix modify storage keys ending with .temp causes overwr… (#25395)
* vault-23135 - fix modify storage keys ending with .temp causes overwriting

* CL

* use t.TempDir
2024-02-15 17:34:37 +00:00
Josh Black
fa13dbd381
add gosimport to make fmt and run it (#25383)
* add gosimport to make fmt and run it

* move installation to tools.sh

* correct weird spacing issue

* Update Makefile

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* fix a weird issue

---------

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2024-02-13 14:07:02 -08:00
Paul Banks
ce942bfee9
Add configurable latency when performing Inmem writes. (#25129) 2024-01-31 11:33:20 +00:00
Josh Black
ef26498e9c
Implement raft-wal (#21460)
* Implement raft-wal

* go mod tidy

* add metrics, fix a panic

* fix the panic for real this time

* PR feedback

* refactor tests to use a helper and reduce duplication

* add a test to verify we don't use raft-wal if raft.db exists

* add config to enable the verifier

* add tests for parsing verification intervals

* run the verifier in the background

* wire up the verifier

* go mod tidy

* refactor config parsing

* remove unused function

* trying to get the verifier working

* wire up some more verifier bits

* sorted out an error, added a new test, lots of debug logging that needs to come out

* fix a bug and remove all the debugging statements

* make sure we close raft-wal stablestore too

* run verifier tests for both boltdb and raft-wal

* PR feedback

* Vault 20270 docker test raft wal (#24463)

* adding a migration test from boltdb to raftwal and back
adding a migration test using snapshot restore

* feedback

* Update physical/raft/raft.go

Co-authored-by: Paul Banks <pbanks@hashicorp.com>

* PR feedback

* change verifier function

* make this shorter

* add changelog

* Fix Close behavior

* make supporting empty logs more explicit

* add some godocs

---------

Co-authored-by: hamid ghaf <hamid@hashicorp.com>
Co-authored-by: Hamid Ghaf <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: Paul Banks <pbanks@hashicorp.com>
2024-01-25 10:08:50 -08:00
Paul Banks
2d88a454d9
Add support for larger transactions in Raft (#24991)
* Add support for larger transactions in Raft

* Add CHANGELOG

* Appease the new lint rules
2024-01-24 14:38:39 +00:00
Paul Banks
0fa36a36ae
Prevent split-brain active node writes when using Consul (#23013)
* Add test to demonstrate a split-brain active node when using Consul

* Add Consul session check to prevent split-brain updates

* It's not right

Co-authored-by: Josh Black <raskchanky@gmail.com>

---------

Co-authored-by: Josh Black <raskchanky@gmail.com>
2023-09-22 16:16:01 +01:00
Hamid Ghaf
5374b3b046
add autoloaded license path to the cache exempt list (#22363)
* add autoloaded license path to the cache exempt list

* CL
2023-08-16 13:31:02 +00:00
Victor Rodriguez
58f029d608
Add paths for seal config to cache exceptions. (#21223)
Add paths for seal config to cache exceptions.

Add barrierSealConfigPath and recoverySealConfigPlaintextPath to
cacheExceptionsPaths in order to avoid a race that causes some
nodes to always see a nil value.
2023-06-14 21:09:29 +00:00
Josh Black
0ff9059967
OSS changes for fixing bug in update-primary (#21119) 2023-06-09 14:35:41 -07:00
Hamid Ghaf
e55c18ed12
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Violet Hynes
825315fd56
VAULT-6938 Remove license from being cache exempt (#17265)
* VAULT-6938 Remove license from being cache exempt

* VAULT-6938 Add changelog

* VAULT-6938 Typo in changelog name

* Update changelog/17265.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-09-26 10:26:07 -04:00
Josh Black
c1cf97adac
merkle sync undo logs (#17103) 2022-09-13 10:03:19 -07:00
shujun10086
26258a7ec9
Fix keyring file missing after Vault restart (#15946) 2022-06-15 10:22:42 -07:00
Nick Cabatoff
859cf45d05
Return an error when trying to store a too-large key with Raft (#13282) 2021-11-25 14:07:03 -05:00
Pratyoy Mukhopadhyay
fa03adb718
[VAULT-4034] Revert back to caching nil values (#13013)
* Revert "[VAULT-4034] Only cache non-nil values (#12993)"

This reverts commit 67e1ed06c7199856f8493f416df55f06603881c6.

* Update sdk/physical/cache.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-11-02 12:00:37 -07:00
Pratyoy Mukhopadhyay
67e1ed06c7
[VAULT-4034] Only cache non-nil values (#12993)
* Only cache non-nil values

* Add changelog
2021-11-01 13:47:45 -07:00
Nick Cabatoff
7144c2c0da
Make latency injector duration atomic. This is a prerequisite for a deadlock fix to the ent integ tests. (#11875) 2021-06-16 11:48:39 -04:00
Brian Kassouf
a24653cc5c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Brian Kassouf
2df57a0418
Fix a few static analysis findings (#11307) 2021-04-07 16:48:40 -07:00
Scott Miller
e5316a9e2f
OSS side barrier encryption tracking and automatic rotation (#11007)
* Automatic barrier key rotation, OSS portion

* Fix build issues

* Vendored version

* Add missing encs field, not sure where this got lost.
2021-02-25 14:27:25 -06:00
Michel Vocks
ee5ebeb30f
Fix license caching issue (#10424)
* Fix license caching issue

* Add changelog entry
2020-12-02 18:21:14 +01:00
Connor
3ad3c3d476
Add telemetry for LRU cache (#10079)
Vault creates an LRU cache that is used when interacting with the
physical backend. Add telemetry when the cache is hit, missed, written
to and deleted from. Use the MetricSink from ClusterMetrics
2020-10-13 10:11:54 -05:00
Jeff Mitchell
84930c02dd Remove EncryptedBlobInfo in Entry 2020-02-14 17:03:40 -05:00
Jeff Mitchell
ef196d9612 Revert "Move encrypted blob info (#8357)"
This reverts commit 5a3c55b6dfe89d0348d78acafcb83b60cf614d38.
2020-02-14 17:02:33 -05:00
Jeff Mitchell
5a3c55b6df
Move encrypted blob info (#8357) 2020-02-14 16:45:37 -05:00
Jeff Mitchell
157e805b97
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00
Mike Jarmy
df01a4307d
Introduce optional service_registration stanza (#7887)
* move ServiceDiscovery into methods

* add ServiceDiscoveryFactory

* add serviceDiscovery field to vault.Core

* refactor ConsulServiceDiscovery into separate struct

* cleanup

* revert accidental change to go.mod

* cleanup

* get rid of un-needed struct tags in vault.CoreConfig

* add service_discovery parser

* add ServiceDiscovery to config

* cleanup

* cleanup

* add test for ConfigServiceDiscovery to Core

* unit testing for config service_discovery stanza

* cleanup

* get rid of un-needed redirect_addr stuff in service_discovery stanza

* improve test suite

* cleanup

* clean up test a bit

* create docs for service_discovery

* check if service_discovery is configured, but storage does not support HA

* tinker with test

* tinker with test

* tweak docs

* move ServiceDiscovery into its own package

* tweak a variable name

* fix comment

* rename service_discovery to service_registration

* tweak service_registration config

* Revert "tweak service_registration config"

This reverts commit 5509920a8ab4c5a216468f262fc07c98121dce35.

* simplify naming

* refactor into ./serviceregistration/consul
2019-12-06 09:46:39 -05:00
Brian Kassouf
3ce40c12bb
Port over cache refresh changes (#7599) 2019-10-08 13:23:43 -07:00
Vu Pham
df173e70fc Added OCI Object Storage Plugin (#6985) 2019-09-04 11:33:16 -07:00
Brian Kassouf
7744e08d94
storage/raft: When restoring a snapshot preseal first (#7011)
* storage/raft: When restoring a snapshot preseal first

* best-effort allow standbys to apply the restoreOp before sealing active node

* Don't cache the raft tls key

* Update physical/raft/raft.go

* Move pending raft peers to core

* Fix race on close bool

* Extend the leaderlease time for tests

* Update raft deps

* Fix audit hashing

* Fix race with auditing
2019-07-03 13:56:30 -07:00
Jeff Mitchell
ab520a9119 Remove data races around error/latency injector rand objects 2019-06-05 01:37:40 -04:00
Jeff Mitchell
f7bb5a2e56
Port over some SP v2 bits (#6516)
* Port over some SP v2 bits

Specifically:

* Add too-large handling to Physical (Consul only for now)
* Contextify some identity funcs
* Update SP protos

* Add size limiting to inmem storage
2019-05-01 13:47:41 -04:00
Jeff Mitchell
71d635293d Move physical/file to sdk 2019-04-15 14:51:33 -04:00
Jeff Mitchell
1d5aeccb60 Update protobufs, sdk changes 2019-04-12 23:13:14 -04:00
Jeff Mitchell
5103b7426a Update physical_util.go with entry.go 2019-04-12 22:56:04 -04:00
Jeff Mitchell
170521481d
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00