* Update config.go
* added validation and parsing
* tests
* move pki external config structs and validation into separate file
* update copywrite
* update configuration
* updates
* Moved tests to pki_external_config.go, comments, refactoring
* refactor
* add tests
* linter fix
* Consolidate to table tests
* consolidate to table tests
* remove APIVersion from PKIExternalCA
* added comments for explaining each struct
* Added ParsePKIExternalCA Test
* Update tests
* Added remaining constraints
* Added destination.template field
* changes
* Added validateListenerAddr
* refactor
* more comments
* changes
* Check for duplicates across blocks
* Make RSA bits a required field
* moved template to the top level
* added comment for test explanation
* move template to the top level
* Move pki config into pkiexternalca directory
* fix linting error
* move pkiconfig back into config folder
* fix failing unit tests
* added comments
* update to preserve order of templatePKIExternalCARefs
* Added comment descriptions for each struct member
* update to include warning
* bring in warning logger from upstream into the pki config parser
* Set default umask to 077
* added comments to each field in agent config
* execute tests in parallel
* combine tests into Validate
* Use assertion error func for tests
* assert error strings
* Removed warning for now
* removed normalization on values during validation
* added tests to ensure that user values are not overridden
* remove testparse
* Update command/agent/config/config.go
* change improvement to feature in changelog
* updated to add line number in error
* Added _ent suffix to files
* Implement CA manager for ACME-based workflows (#12827)
* Implement CA manager for ACME-based workflows
* refactor tests into table tests
* update with suggestions
* format
* fix challenge cleanup
* make fmt
* update with suggestions
* add _ent + build flags
* Add a runtime component for pkiexternalca (#12838)
* Implement CA manager for ACME-based workflows
* Add a runtime component for pkiexternalca
* make fmt
* refactor tests into table tests
* update with suggestions
* format
* fix challenge cleanup
* make fmt
* update with suggestions
* update with suggestions
* add _ent + build flags
* fix linters
* delete duplicate files
* fix changelog
* rename test files
* fix linter
* try to bypass false positive linter err
* fix
* Rename file
* fix linter
* fix linter
* remove go:build enterprise commends from _ent files
* update order statuses to use kebab case + fix scanner failures
* add missing order status
* Template Integration For pki_external_ca resources (#13069)
* Implement CA manager for ACME-based workflows
* Add a runtime component for pkiexternalca
* make fmt
* refactor tests into table tests
* update with suggestions
* initial commit
* fix test failure
* changes
* remove logger check
* remove redundant config by name check
* convert to table tests
* added comments
* updates
* Fix tests
* fix nil pointer issue
* move changes to _ent files
* remove ce duplicate files
* updates
* update template.go
* added changelog.txt
* create template_pem_ent_test.go
* added comment explanation
* update ca_manager_ent.go
* update changelog
* separate ce stubs into server_ce.go and common code into server.go
* Moved helper functions to bottom of test file. Added godocs.
* Make pkiExternalCA name required in template
* remove go:build enterprise commends from _ent files
* rename to template_pem_ent
* include ent tag in server_ent.go
* remove enterprise tag comment from server_ent.go
* create pki_external_config_ce.go
* update template_pem_ent_integration_test.go
* rename integration test
---------
---------
Co-authored-by: Jaired Jawed <jaired.jawed@hashicorp.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
Co-authored-by: Zlaticanin <60530402+Zlaticanin@users.noreply.github.com>
Co-authored-by: Milena Zlaticanin <Milena.Zlaticanin@ibm.com>
* license: update headers to IBM Corp.
* `make proto`
* update offset because source file changed
Signed-off-by: Ryan Cragun <me@ryan.ec>
Co-authored-by: Ryan Cragun <me@ryan.ec>
* VAULT-28192 fix Agent and Proxy consuming large amounts of CPU for auto-auth self-healing
* Changelog
* Update changelog
* drain incoming if we get invalid token
---------
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* add gosimport to make fmt and run it
* move installation to tools.sh
* correct weird spacing issue
* Update Makefile
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* fix a weird issue
---------
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Updating the license from MPL to Business Source License.
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.
* add missing license headers
* Update copyright file headers to BUS-1.1
* Fix test that expected exact offset on hcl file
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
* move private function to internal pkg for sharing
* rename to mc
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* rename to NewConfig
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* VAULT-12940 test for templating user agent
* VAULT-12940 User agent work so far
* VAULT-12940 Vault Agent uses Vault Agent specific User-Agent header when issuing requests
* VAULT-12940 Clean-up and godocs
* VAULT-12940 changelog
* VAULT-12940 Fix test checking headers
* VAULT-12940 Fix test checking headers
* VAULT-12940 Fix test checking headers
* VAULT-12940 Fix test checking headers
* VAULT-12940 copy/paste typos
* VAULT-12940 improve comments, use make(http.Header)
* VAULT-12940 small typos and clean-up
* [VAULT-1618] Agent error log level is mismatched
`logLevelToStringPtr` translates `go-hclog`'s `ERROR` to `"ERROR"` for
Consul Template's runner, but that expects `ERR` and is quite strict
about it.
This will address https://github.com/hashicorp/vault-k8s/issues/223
after it is set as the default image in `vault-k8s`.
I didn't find a simple way to test this other than starting up a full
server and agent and letting them run, which is unfortunately fairly
slow.
I confirmed that this addresses the original issue by modifying the helm
chart with the values in this commit and patching the log level to `err`.
* VAULT-1618 Add changelog/14424.txt
* VAULT-1618 Update changelog/14424.txt based on @kalafut suggestion
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
* VAULT-1618 Move cancel and server stop into defer in tests
* VAULT-1618 Triggering CircleCI tests
* VAULT-1618 Replace ioutil with os functions for agent template tests
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Uses a bufconn listener between consul-template and vault-agent when
caching is enabled and either templates or a listener is defined. This
means no listeners need to be defined in vault-agent for just
templating. Always routes consul-template through the vault-agent
cache (instead of only when persistent cache is enabled).
Uses a local transportDialer interface in config.Cache{}.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
* agent: restart template runner on retry for unlimited retries
* template: log error message early
* template: delegate retries back to template if param is set to true
* agent: add and use the new template config stanza
* agent: fix panic, fix existing tests
* changelog: add changelog entry
* agent: add tests for exit_on_retry_failure
* agent: properly check on agent exit cases, add separate tests for missing key vs missing secrets
* agent: add note on difference between missing key vs missing secret
* docs: add docs for template_config
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* docs: fix exit_on_retry_failure, fix Functionality section
* docs: update interaction title
* template: add internal note on behavior for persist case
* docs: update agent, template, and template-config docs
* docs: update agent docs on retry stanza
* Apply suggestions from code review
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Update changelog/11775.txt
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* agent/test: rename expectExit to expectExitFromError
* agent/test: add check on early exits on the happy path
* Update website/content/docs/agent/template-config.mdx
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Remove template_retry config section. Add new vault.retry section which only has num_retries field; if num_retries is 0 or absent, default it to 12 for backwards compat with pre-1.7 template retrying. Setting num_retries=-1 disables retries.
Configured retries are used for both templating and api proxy, though if template requests go through proxy (currently requires persistence enabled) we'll only configure retries for the latter to avoid duplicate retrying. Though there is some duplicate retrying already because whenever the template server does a retry when not going through the proxy, the Vault client it uses allows for 2 behind-the-scenes retries for some 400/500 http error codes.
Adds the option of a write-through cache, backed by boltdb
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
* feat(agent): add retry configuration for vault agent
* feat(agent): add test fixtures for retry
* fix(retry): move retry stanza to top level as template_retry
* fix(retry): add retry config to ServerConfig struct
* fix(retry): point config parser to parse template_retry instead of retry
* remove netlify config (#10711)
* Fix build (#10749)
* Move the declaration to a OSS build tag file to not have it collide w… (#10750)
* Move the declaration to a OSS build tag file to not have it collide with ent declarations
* Add comment
* Remove comment to trigger ci
* Unconditionally use the root namespace when calling sys/seal-status. (#10742)
* feat(agent): add retry configuration for vault agent
* feat(agent): add test fixtures for retry
* fix(retry): move retry stanza to top level as template_retry
* fix(retry): add retry config to ServerConfig struct
* fix(retry): point config parser to parse template_retry instead of retry
Co-authored-by: Hridoy Roy <roy@hashicorp.com>
Co-authored-by: Jeff Escalante <jescalan@users.noreply.github.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
Co-authored-by: Mark Gritter <mgritter@hashicorp.com>
* agent: return a non-zero exit code on error
* agent/template: always return on template server error, add case for error_on_missing_key
* agent: fix tests by updating Run params to use an errCh
* agent/template: add permission denied test case, clean up test var
* agent: use unbuffered errCh, emit fatal errors directly to the UI output
* agent: use oklog's run.Group to schedule subsystem runners (#9761)
* agent: use oklog's run.Group to schedule subsystem runners
* agent: clean up unused DoneCh, clean up agent's main Run func
* agent/template: use ts.stopped.CAS to atomically swap value
* fix tests
* fix tests
* agent/template: add timeout on TestRunServer
* agent: output error via logs and return a generic error on non-zero exit
* fix TestAgent_ExitAfterAuth
* agent/template: do not restart ct runner on new incoming token if exit_after_auth is set to true
* agent: drain ah.OutputCh after sink exits to avoid blocking on the channel
* use context.WithTimeout, expand comments around ordering of defer cancel()
* add regression test for #7883
* Add logic to count render events and match them to expected
* remove the WAIT label and make some changes to remove the break statements
* change the 'randomness' of the templateContents test helper method
* Vault Agent Template: parse templates (#7540)
* add template config parsing, but it's wrong b/c it's not using mapstructure
* parsing consul templates in agent config
* add additional test to configuration parsing, to cover basics
* another test fixture, rework simple test into table
* refactor into table test
* rename test
* remove flattenKeys and add other test fixture
* Update command/agent/config/config.go
Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>
* return the decode error instead of swallowing it
* Update command/agent/config/config_test.go
Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>
* go mod tidy
* change error checking style
* Add agent template doc
* TemplateServer: render secrets with Consul Template (#7621)
* add template config parsing, but it's wrong b/c it's not using mapstructure
* parsing consul templates in agent config
* add additional test to configuration parsing, to cover basics
* another test fixture, rework simple test into table
* refactor into table test
* rename test
* remove flattenKeys and add other test fixture
* add template package
* WIP: add runner
* fix panic, actually copy templates, etc
* rework how the config.Vault is created and enable reading from the environment
* this was supposed to be a part of the prior commit
* move/add methods to testhelpers for converting some values to pointers
* use new methods in testhelpers
* add an unblock channel to block agent until a template has been rendered
* add note
* unblock if there are no templates
* cleanups
* go mod tidy
* remove dead code
* simple test to starT
* add simple, empty templates test
* Update package doc, error logs, and add missing close() on channel
* update code comment to be clear what I'm referring to
* have template.NewServer return a (<- chan) type, even though it's a normal chan, as a better practice to enforce reading only
* Update command/agent.go
Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>
* update with test
* Add README and doc.go to the command/agent directory (#7503)
* Add README and doc.go to the command/agent directory
* Add link to website
* address feedback for agent.go
* updated with feedback from Calvin
* Rework template.Server to export the unblock channel, and remove it from the NewServer function
* apply feedback from Nick
* fix/restructure rendering test
* Add pointerutil package for converting types to their pointers
* Remove pointer helper methods; use sdk/helper/pointerutil instead
* update newRunnerConfig to use pointerutil and empty strings
* only wait for unblock if template server is initialized
* update test structure
* some test cleanup
* follow up tests
* remove debugging, fix issue in replacing runner config
* need to handle first render/token
* Simplify the blocking logic to support exit after auth
* fix channel name
* expand TestAgent_Template to include multiple scenarios
* cleanup
* test cleanups after feedback
* Vault Agent Template: parse templates (#7540)
* add template config parsing, but it's wrong b/c it's not using mapstructure
* parsing consul templates in agent config
* add additional test to configuration parsing, to cover basics
* another test fixture, rework simple test into table
* refactor into table test
* rename test
* remove flattenKeys and add other test fixture
* Update command/agent/config/config.go
Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>
* return the decode error instead of swallowing it
* Update command/agent/config/config_test.go
Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>
* go mod tidy
* change error checking style
* Add agent template doc
* TemplateServer: render secrets with Consul Template (#7621)
* add template config parsing, but it's wrong b/c it's not using mapstructure
* parsing consul templates in agent config
* add additional test to configuration parsing, to cover basics
* another test fixture, rework simple test into table
* refactor into table test
* rename test
* remove flattenKeys and add other test fixture
* add template package
* WIP: add runner
* fix panic, actually copy templates, etc
* rework how the config.Vault is created and enable reading from the environment
* this was supposed to be a part of the prior commit
* move/add methods to testhelpers for converting some values to pointers
* use new methods in testhelpers
* add an unblock channel to block agent until a template has been rendered
* add note
* unblock if there are no templates
* cleanups
* go mod tidy
* remove dead code
* simple test to starT
* add simple, empty templates test
* Update package doc, error logs, and add missing close() on channel
* update code comment to be clear what I'm referring to
* have template.NewServer return a (<- chan) type, even though it's a normal chan, as a better practice to enforce reading only
* Update command/agent.go
Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>
* update with test
* Add README and doc.go to the command/agent directory (#7503)
* Add README and doc.go to the command/agent directory
* Add link to website
* address feedback for agent.go
* updated with feedback from Calvin
* Rework template.Server to export the unblock channel, and remove it from the NewServer function
* apply feedback from Nick
* fix/restructure rendering test
* Add pointerutil package for converting types to their pointers
* Remove pointer helper methods; use sdk/helper/pointerutil instead
* update newRunnerConfig to use pointerutil and empty strings
* only wait for unblock if template server is initialized
* drain the token channel in this test
* conditionally send on channel
