Commit Graph

34 Commits

Author SHA1 Message Date
Brian Kassouf
230ed81150
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Jeff Mitchell
86c1cdf9e6
Move awsutil over to the go-secure-stdlib version (#12128)
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
2021-07-20 20:42:00 -04:00
Lars Lehtonen
3857cdbeaf
physical: deprecate errwrap.Wrapf() (#11692) 2021-05-31 12:54:05 -04:00
Brian Kassouf
a24653cc5c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Scott Miller
9a7b23440f
Fix err shadowing (#11296) 2021-04-07 11:25:23 -05:00
Conrad Lara
1a41ae8213
Potential data loss in DynamoDB backend (#10181)
fixes hashicorp/vault#5836

DynamoDB may when throttled return a 2xx response while not committing
all submitted items to the database.

Depending upon load all actions in a BatchWriteUpdate may be throttled
with ProvisionedThroughputExceededException in which case AWS SDK handles
the retry. If some messages were throttled but not all
ProvisionedThroughputExceededException is not returned to the SDK and it
is up to us to resubmit the request.

Using an exponential backoff as recommended in AWS SDK for times we possibly
get partially throttled repeatedly.
2021-04-07 09:44:42 -05:00
Theron Voran
10c0adad72
Add logging during awskms auto-unseal (#9794)
Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.

Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
2020-09-28 14:06:49 -07:00
ncabatoff
2b3aef242b
Run CI tests in docker instead of a machine. (#8948) 2020-09-15 10:01:26 -04:00
Jeff Mitchell
157e805b97
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00
ncabatoff
29cdb196de
Use a much longer delay before giving up and failing the test. (#7646)
On circleci we've seen the test run very slowly so it's often failing here.
2019-10-14 11:06:02 -04:00
EdwinRobbins
fdcf68db67 DynamoDB: Make Unlock key delete conditional on being old leader's (#6637) 2019-04-30 17:44:47 -07:00
Jim Kalafut
a7b9987543
Exit DynamoDB tryToLock when stop channel is closed (#6640)
* Exit DynamoDB tryToLock when stop channel is closed

If the stop channel is closed (e.g. an error is returned which triggers
close(stop) in Lock), this loop will spin and use 100% CPU.

* Ensure ticker is stopped
2019-04-29 22:51:48 -07:00
ncabatoff
0f1569b7f5
Merge multiple functions for creating consul containers into one. (#6612)
Merge both functions for creating mongodb containers into one.
Add retries to docker container cleanups.
Require $VAULT_ACC be set to enable AWS tests.
2019-04-22 12:26:10 -04:00
Jeff Mitchell
170521481d
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Maarten van der Hoef
77a7f861ce Spelling fix (#6523) 2019-04-03 06:57:32 -07:00
Jim Kalafut
246d93dd75 Reset retry count after a successful lock check (#6522) 2019-04-02 17:08:49 -07:00
Mahmoud Abdelsalam
98079d81a1 Fix dynamodb HA lock race (#6512)
* Fix DynamoDB HA race issue

* Add test for race condition (which fails on the released DynamoDB code)
2019-04-02 16:19:21 -07:00
Jeff Mitchell
9af595ec61
Don't read AWS env vars (#5974)
* Don't read AWS env vars

Let AWS SDK env cred chain provider do it for us

Fixes #5965
2019-01-04 15:03:57 -05:00
Calvin Leung Huang
6d6fbf2f57
Clean up HABackend tests (#5617) 2018-11-01 10:31:09 -07:00
Calvin Leung Huang
ece7569aca Clean up container on connection failure, switch to ory/dockertest on package postgresql (#5050) 2018-08-07 09:56:33 -07:00
Jeff Mitchell
f85efad941 Remove unnecessary log scoping in some physical backends 2018-07-13 10:57:30 -04:00
Jeff Mitchell
dba48d5265
Re-add dockertest and fix up imports and update script (#4909) 2018-07-11 17:49:13 -04:00
Jeff Mitchell
f493d2436e
Add an idle timeout for the server (#4760)
* Add an idle timeout for the server

Because tidy operations can be long-running, this also changes all tidy
operations to behave the same operationally (kick off the process, get a
warning back, log errors to server log) and makes them all run in a
goroutine.

This could mean a sort of hard stop if Vault gets sealed because the
function won't have the read lock. This should generally be okay
(running tidy again should pick back up where it left off), but future
work could use cleanup funcs to trigger the functions to stop.

* Fix up tidy test

* Add deadline to cluster connections and an idle timeout to the cluster server, plus add readheader/read timeout to api server
2018-06-16 18:21:33 -04:00
Jim Kalafut
1b5b20435b
Add DynamoDB marshaling update test (#4746)
This test fails before the d3604289be update.
2018-06-12 08:22:02 -07:00
Matthew Surabian
1886e45e36 DynamoDB Nested Values Bug (#4570)
* Add tests to ExerciseBackend to expose nested-values bug

* Update DynamoDB physical backend Delete and hasChildren logic to prevent overzealous cleanup of folders and values
2018-05-16 13:30:56 -04:00
Matthew Surabian
39f2d66eb7 DynamoDB Deprecation Fixes (#4534)
* Use the AWS SDK's UnmarshalMap method for dynamodb backend, not the deprecated ConvertFromMap method

* Use the AWS SDK's MarshalMap method for dynamodb backend, not the deprecated ConvertToMap method

* Use the AWS SDK's session.NewSession method for dynamodb backend, not the deprecated session.New method

* Fix variable name awserr that colides with imported package in dynamodb backend
2018-05-10 08:25:51 -04:00
Becca Petrin
eb7c0ab84e Clean up error string formatting (#4304) 2018-04-09 14:35:21 -04:00
Vishal Nayak
e2bb2ec3b9
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin
792d219aa9 Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Nicholas Watkins
af974c2112 Allow configuration of dynamodb storage to specify the max retries of aws sdk (#4115) 2018-03-19 15:53:23 -04:00
Brian Kassouf
fbe5f87278
Add compile tests to verify physical stores satisfy the correct interfaces (#3820) 2018-01-19 17:44:24 -08:00
Brian Kassouf
8142b42d95 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Vlad Ungureanu
81f968fbe2 Remove unused recovery field in dynamodb backend (#3569) 2017-11-13 15:46:02 -05:00
Jeff Mitchell
0ac531d3f4 Migrate physical backends into separate packages (#3106) 2017-08-03 13:24:27 -04:00