Vault Agent Template (#7652)

* Vault Agent Template: parse templates  (#7540)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* Update command/agent/config/config.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* return the decode error instead of swallowing it

* Update command/agent/config/config_test.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* go mod tidy

* change error checking style

* Add agent template doc

* TemplateServer: render secrets with Consul Template (#7621)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* add template package

* WIP: add runner

* fix panic, actually copy templates, etc

* rework how the config.Vault is created and enable reading from the environment

* this was supposed to be a part of the prior commit

* move/add methods to testhelpers for converting some values to pointers

* use new methods in testhelpers

* add an unblock channel to block agent until a template has been rendered

* add note

* unblock if there are no templates

* cleanups

* go mod tidy

* remove dead code

* simple test to starT

* add simple, empty templates test

* Update package doc, error logs, and add missing close() on channel

* update code comment to be clear what I'm referring to

* have template.NewServer return a (<- chan) type, even though it's a normal chan, as a better practice to enforce reading only

* Update command/agent.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* update with test

* Add README and doc.go to the command/agent directory (#7503)

* Add README and doc.go to the command/agent directory

* Add link to website

* address feedback for agent.go

* updated with feedback from Calvin

* Rework template.Server to export the unblock channel, and remove it from the NewServer function

* apply feedback from Nick

* fix/restructure rendering test

* Add pointerutil package for converting types to their pointers

* Remove pointer helper methods; use sdk/helper/pointerutil instead

* update newRunnerConfig to use pointerutil and empty strings

* only wait for unblock if template server is initialized

* drain the token channel in this test

* conditionally send on channel
This commit is contained in:
Clint 2019-10-18 16:21:46 -05:00 committed by GitHub
parent 3e97f5cf62
commit 012c165b02
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 995 additions and 59 deletions

View File

@ -29,10 +29,12 @@ import (
"github.com/hashicorp/vault/command/agent/auth/jwt"
"github.com/hashicorp/vault/command/agent/auth/kubernetes"
"github.com/hashicorp/vault/command/agent/cache"
"github.com/hashicorp/vault/command/agent/config"
agentConfig "github.com/hashicorp/vault/command/agent/config"
"github.com/hashicorp/vault/command/agent/sink"
"github.com/hashicorp/vault/command/agent/sink/file"
"github.com/hashicorp/vault/command/agent/sink/inmem"
"github.com/hashicorp/vault/command/agent/template"
gatedwriter "github.com/hashicorp/vault/helper/gated-writer"
"github.com/hashicorp/vault/sdk/helper/consts"
"github.com/hashicorp/vault/sdk/helper/logging"
@ -215,44 +217,48 @@ func (c *AgentCommand) Run(args []string) int {
c.UI.Info("No auto_auth block found in config file, not starting automatic authentication feature")
}
if config.Vault != nil {
c.setStringFlag(f, config.Vault.Address, &StringVar{
Name: flagNameAddress,
Target: &c.flagAddress,
Default: "https://127.0.0.1:8200",
EnvVar: api.EnvVaultAddress,
})
c.setStringFlag(f, config.Vault.CACert, &StringVar{
Name: flagNameCACert,
Target: &c.flagCACert,
Default: "",
EnvVar: api.EnvVaultCACert,
})
c.setStringFlag(f, config.Vault.CAPath, &StringVar{
Name: flagNameCAPath,
Target: &c.flagCAPath,
Default: "",
EnvVar: api.EnvVaultCAPath,
})
c.setStringFlag(f, config.Vault.ClientCert, &StringVar{
Name: flagNameClientCert,
Target: &c.flagClientCert,
Default: "",
EnvVar: api.EnvVaultClientCert,
})
c.setStringFlag(f, config.Vault.ClientKey, &StringVar{
Name: flagNameClientKey,
Target: &c.flagClientKey,
Default: "",
EnvVar: api.EnvVaultClientKey,
})
c.setBoolFlag(f, config.Vault.TLSSkipVerify, &BoolVar{
Name: flagNameTLSSkipVerify,
Target: &c.flagTLSSkipVerify,
Default: false,
EnvVar: api.EnvVaultSkipVerify,
})
// create an empty Vault configuration if none was loaded from file. The
// follow-up setStringFlag calls will populate with defaults if otherwise
// omitted
if config.Vault == nil {
config.Vault = new(agentConfig.Vault)
}
c.setStringFlag(f, config.Vault.Address, &StringVar{
Name: flagNameAddress,
Target: &c.flagAddress,
Default: "https://127.0.0.1:8200",
EnvVar: api.EnvVaultAddress,
})
c.setStringFlag(f, config.Vault.CACert, &StringVar{
Name: flagNameCACert,
Target: &c.flagCACert,
Default: "",
EnvVar: api.EnvVaultCACert,
})
c.setStringFlag(f, config.Vault.CAPath, &StringVar{
Name: flagNameCAPath,
Target: &c.flagCAPath,
Default: "",
EnvVar: api.EnvVaultCAPath,
})
c.setStringFlag(f, config.Vault.ClientCert, &StringVar{
Name: flagNameClientCert,
Target: &c.flagClientCert,
Default: "",
EnvVar: api.EnvVaultClientCert,
})
c.setStringFlag(f, config.Vault.ClientKey, &StringVar{
Name: flagNameClientKey,
Target: &c.flagClientKey,
Default: "",
EnvVar: api.EnvVaultClientKey,
})
c.setBoolFlag(f, config.Vault.TLSSkipVerify, &BoolVar{
Name: flagNameTLSSkipVerify,
Target: &c.flagTLSSkipVerify,
Default: false,
EnvVar: api.EnvVaultSkipVerify,
})
infoKeys := make([]string, 0, 10)
info := make(map[string]string)
@ -294,10 +300,14 @@ func (c *AgentCommand) Run(args []string) int {
return 1
}
// ctx and cancelFunc are passed to the AuthHandler, SinkServer, and
// TemplateServer that periodically listen for ctx.Done() to fire and shut
// down accordingly.
ctx, cancelFunc := context.WithCancel(context.Background())
var method auth.AuthMethod
var sinks []*sink.SinkConfig
var namespace string
if config.AutoAuth != nil {
for _, sc := range config.AutoAuth.Sinks {
switch sc.Type {
@ -327,7 +337,8 @@ func (c *AgentCommand) Run(args []string) int {
// Check if a default namespace has been set
mountPath := config.AutoAuth.Method.MountPath
if config.AutoAuth.Method.Namespace != "" {
mountPath = path.Join(config.AutoAuth.Method.Namespace, mountPath)
namespace = config.AutoAuth.Method.Namespace
mountPath = path.Join(namespace, mountPath)
}
authConfig := &auth.AuthConfig{
@ -486,14 +497,21 @@ func (c *AgentCommand) Run(args []string) int {
defer c.cleanupGuard.Do(listenerCloseFunc)
}
var ssDoneCh, ahDoneCh chan struct{}
// Listen for signals
// TODO: implement support for SIGHUP reloading of configuration
// signal.Notify(c.signalCh)
var ssDoneCh, ahDoneCh, tsDoneCh, unblockCh chan struct{}
var ts *template.Server
// Start auto-auth and sink servers
if method != nil {
enableTokenCh := len(config.Templates) > 0
ah := auth.NewAuthHandler(&auth.AuthHandlerConfig{
Logger: c.logger.Named("auth.handler"),
Client: c.client,
WrapTTL: config.AutoAuth.Method.WrapTTL,
EnableReauthOnNewCredentials: config.AutoAuth.EnableReauthOnNewCredentials,
EnableTemplateTokenCh: enableTokenCh,
})
ahDoneCh = ah.DoneCh
@ -504,8 +522,20 @@ func (c *AgentCommand) Run(args []string) int {
})
ssDoneCh = ss.DoneCh
// create an independent vault configuration for Consul Template to use
vaultConfig := c.setupTemplateConfig()
ts = template.NewServer(&template.ServerConfig{
Logger: c.logger.Named("template.server"),
VaultConf: vaultConfig,
Namespace: namespace,
ExitAfterAuth: config.ExitAfterAuth,
})
tsDoneCh = ts.DoneCh
unblockCh = ts.UnblockCh
go ah.Run(ctx, method)
go ss.Run(ctx, ah.OutputCh, sinks)
go ts.Run(ctx, ah.TemplateTokenCh, config.Templates)
}
// Server configuration output
@ -536,6 +566,15 @@ func (c *AgentCommand) Run(args []string) int {
}
}()
// If the template server is running and we've assinged the Unblock channel,
// wait for the template to render
if unblockCh != nil {
select {
case <-ctx.Done():
case <-ts.UnblockCh:
}
}
select {
case <-ssDoneCh:
// This will happen if we exit-on-auth
@ -549,6 +588,10 @@ func (c *AgentCommand) Run(args []string) int {
if ssDoneCh != nil {
<-ssDoneCh
}
if tsDoneCh != nil {
<-tsDoneCh
}
}
return 0
@ -648,3 +691,21 @@ func (c *AgentCommand) removePidFile(pidPath string) error {
}
return os.Remove(pidPath)
}
// setupTemplateConfig creates a config.Vault struct for use by Consul Template.
// Consul Template does not currently allow us to pass in a configured API
// client, unlike the AuthHandler and SinkServer that reuse the client created
// in this Run() method. Here we build a config.Vault struct for use by the
// Template Server that matches the configuration used to create the client
// (c.client), but in a struct of type config.Vault so that Consul Template can
// create it's own api client internally.
func (c *AgentCommand) setupTemplateConfig() *config.Vault {
return &config.Vault{
Address: c.flagAddress,
CACert: c.flagCACert,
CAPath: c.flagCAPath,
ClientCert: c.flagClientCert,
ClientKey: c.flagClientKey,
TLSSkipVerify: c.flagTLSSkipVerify,
}
}

15
command/agent/README.md Normal file
View File

@ -0,0 +1,15 @@
# Vault Agent
Vault Agent is a client daemon that provides Auth-Auth, Caching, and Template
features.
Vault Agent provides a number of different helper features, specifically
addressing the following challenges:
- Automatic authentication
- Secure delivery/storage of tokens
- Lifecycle management of these tokens (renewal & re-authentication)
See the usage documentation on the Vault website here:
- https://www.vaultproject.io/docs/agent/

View File

@ -30,11 +30,13 @@ type AuthConfig struct {
type AuthHandler struct {
DoneCh chan struct{}
OutputCh chan string
TemplateTokenCh chan string
logger hclog.Logger
client *api.Client
random *rand.Rand
wrapTTL time.Duration
enableReauthOnNewCredentials bool
enableTemplateTokenCh bool
}
type AuthHandlerConfig struct {
@ -42,6 +44,7 @@ type AuthHandlerConfig struct {
Client *api.Client
WrapTTL time.Duration
EnableReauthOnNewCredentials bool
EnableTemplateTokenCh bool
}
func NewAuthHandler(conf *AuthHandlerConfig) *AuthHandler {
@ -50,11 +53,13 @@ func NewAuthHandler(conf *AuthHandlerConfig) *AuthHandler {
// This is buffered so that if we try to output after the sink server
// has been shut down, during agent shutdown, we won't block
OutputCh: make(chan string, 1),
TemplateTokenCh: make(chan string, 1),
logger: conf.Logger,
client: conf.Client,
random: rand.New(rand.NewSource(int64(time.Now().Nanosecond()))),
wrapTTL: conf.WrapTTL,
enableReauthOnNewCredentials: conf.EnableReauthOnNewCredentials,
enableTemplateTokenCh: conf.EnableTemplateTokenCh,
}
return ah
@ -77,6 +82,7 @@ func (ah *AuthHandler) Run(ctx context.Context, am AuthMethod) {
am.Shutdown()
close(ah.OutputCh)
close(ah.DoneCh)
close(ah.TemplateTokenCh)
ah.logger.Info("auth handler stopped")
}()
@ -163,6 +169,9 @@ func (ah *AuthHandler) Run(ctx context.Context, am AuthMethod) {
}
ah.logger.Info("authentication successful, sending wrapped token to sinks and pausing")
ah.OutputCh <- string(wrappedResp)
if ah.enableTemplateTokenCh {
ah.TemplateTokenCh <- string(wrappedResp)
}
am.CredSuccess()
@ -189,6 +198,9 @@ func (ah *AuthHandler) Run(ctx context.Context, am AuthMethod) {
}
ah.logger.Info("authentication successful, sending token to sinks")
ah.OutputCh <- secret.Auth.ClientToken
if ah.enableTemplateTokenCh {
ah.TemplateTokenCh <- secret.Auth.ClientToken
}
am.CredSuccess()
}

View File

@ -87,7 +87,8 @@ consumption:
for {
select {
case <-ah.OutputCh:
// Nothing
case <-ah.TemplateTokenCh:
// Nothing
case <-time.After(stopTime.Sub(time.Now())):
if !closed {
cancelFunc()

View File

@ -8,24 +8,28 @@ import (
"strings"
"time"
ctconfig "github.com/hashicorp/consul-template/config"
"github.com/hashicorp/errwrap"
"github.com/hashicorp/go-multierror"
"github.com/hashicorp/hcl"
"github.com/hashicorp/hcl/hcl/ast"
"github.com/hashicorp/vault/helper/namespace"
"github.com/hashicorp/vault/sdk/helper/parseutil"
"github.com/mitchellh/mapstructure"
)
// Config is the configuration for the vault server.
type Config struct {
AutoAuth *AutoAuth `hcl:"auto_auth"`
ExitAfterAuth bool `hcl:"exit_after_auth"`
PidFile string `hcl:"pid_file"`
Listeners []*Listener `hcl:"listeners"`
Cache *Cache `hcl:"cache"`
Vault *Vault `hcl:"vault"`
AutoAuth *AutoAuth `hcl:"auto_auth"`
ExitAfterAuth bool `hcl:"exit_after_auth"`
PidFile string `hcl:"pid_file"`
Listeners []*Listener `hcl:"listeners"`
Cache *Cache `hcl:"cache"`
Vault *Vault `hcl:"vault"`
Templates []*ctconfig.TemplateConfig `hcl:"templates"`
}
// Vault contains configuration for connnecting to Vault servers
type Vault struct {
Address string `hcl:"address"`
CACert string `hcl:"ca_cert"`
@ -36,10 +40,12 @@ type Vault struct {
ClientKey string `hcl:"client_key"`
}
// Cache contains any configuration needed for Cache mode
type Cache struct {
UseAutoAuthToken bool `hcl:"use_auto_auth_token"`
}
// Listener contains configuration for any Vault Agent listeners
type Listener struct {
Type string
Config map[string]interface{}
@ -48,6 +54,7 @@ type Listener struct {
// RequireRequestHeader is a listener configuration option
const RequireRequestHeader = "require_request_header"
// AutoAuth is the configured authentication method and sinks
type AutoAuth struct {
Method *Method `hcl:"-"`
Sinks []*Sink `hcl:"sinks"`
@ -57,6 +64,7 @@ type AutoAuth struct {
EnableReauthOnNewCredentials bool `hcl:"enable_reauth_on_new_credentials"`
}
// Method represents the configuration for the authentication backend
type Method struct {
Type string
MountPath string `hcl:"mount_path"`
@ -66,6 +74,7 @@ type Method struct {
Config map[string]interface{}
}
// Sink defines a location to write the authenticated token
type Sink struct {
Type string
WrapTTLRaw interface{} `hcl:"wrap_ttl"`
@ -116,16 +125,18 @@ func LoadConfig(path string) (*Config, error) {
return nil, errwrap.Wrapf("error parsing 'auto_auth': {{err}}", err)
}
err = parseListeners(&result, list)
if err != nil {
if err := parseListeners(&result, list); err != nil {
return nil, errwrap.Wrapf("error parsing 'listeners': {{err}}", err)
}
err = parseCache(&result, list)
if err != nil {
if err := parseCache(&result, list); err != nil {
return nil, errwrap.Wrapf("error parsing 'cache':{{err}}", err)
}
if err := parseTemplates(&result, list); err != nil {
return nil, errwrap.Wrapf("error parsing 'template': {{err}}", err)
}
if result.Cache != nil {
if len(result.Listeners) < 1 {
return nil, fmt.Errorf("at least one listener required when cache enabled")
@ -411,3 +422,65 @@ func parseSinks(result *Config, list *ast.ObjectList) error {
result.AutoAuth.Sinks = ts
return nil
}
func parseTemplates(result *Config, list *ast.ObjectList) error {
name := "template"
templateList := list.Filter(name)
if len(templateList.Items) < 1 {
return nil
}
var tcs []*ctconfig.TemplateConfig
for _, item := range templateList.Items {
var shadow interface{}
if err := hcl.DecodeObject(&shadow, item.Val); err != nil {
return fmt.Errorf("error decoding config: %s", err)
}
// Convert to a map and flatten the keys we want to flatten
parsed, ok := shadow.(map[string]interface{})
if !ok {
return errors.New("error converting config")
}
// flatten the wait field. The initial "wait" value, if given, is a
// []map[string]interface{}, but we need it to be map[string]interface{}.
// Consul Template has a method flattenKeys that walks all of parsed and
// flattens every key. For Vault Agent, we only care about the wait input.
// Only one wait stanza is supported, however Consul Template does not error
// with multiple instead it flattens them down, with last value winning.
// Here we take the last element of the parsed["wait"] slice to keep
// consistency with Consul Template behavior.
wait, ok := parsed["wait"].([]map[string]interface{})
if ok {
parsed["wait"] = wait[len(wait)-1]
}
var tc ctconfig.TemplateConfig
// Use mapstructure to populate the basic config fields
var md mapstructure.Metadata
decoder, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{
DecodeHook: mapstructure.ComposeDecodeHookFunc(
ctconfig.StringToFileModeFunc(),
ctconfig.StringToWaitDurationHookFunc(),
mapstructure.StringToSliceHookFunc(","),
mapstructure.StringToTimeDurationHookFunc(),
),
ErrorUnused: true,
Metadata: &md,
Result: &tc,
})
if err != nil {
return errors.New("mapstructure decoder creation failed")
}
if err := decoder.Decode(parsed); err != nil {
return err
}
tcs = append(tcs, &tc)
}
result.Templates = tcs
return nil
}

View File

@ -6,6 +6,8 @@ import (
"time"
"github.com/go-test/deep"
ctconfig "github.com/hashicorp/consul-template/config"
"github.com/hashicorp/vault/sdk/helper/pointerutil"
)
func TestLoadConfigFile_AgentCache(t *testing.T) {
@ -93,8 +95,14 @@ func TestLoadConfigFile_AgentCache(t *testing.T) {
}
func TestLoadConfigFile(t *testing.T) {
os.Setenv("TEST_AAD_ENV", "aad")
defer os.Unsetenv("TEST_AAD_ENV")
if err := os.Setenv("TEST_AAD_ENV", "aad"); err != nil {
t.Fatal(err)
}
defer func() {
if err := os.Unsetenv("TEST_AAD_ENV"); err != nil {
t.Fatal(err)
}
}()
config, err := LoadConfig("./test-fixtures/config.hcl")
if err != nil {
@ -278,3 +286,106 @@ func TestLoadConfigFile_AgentCache_AutoAuth_NoSink(t *testing.T) {
t.Fatal(diff)
}
}
// TestLoadConfigFile_Template tests template definitions in Vault Agent
func TestLoadConfigFile_Template(t *testing.T) {
testCases := map[string]struct {
fixturePath string
expectedTemplates []*ctconfig.TemplateConfig
}{
"min": {
fixturePath: "./test-fixtures/config-template-min.hcl",
expectedTemplates: []*ctconfig.TemplateConfig{
&ctconfig.TemplateConfig{
Source: pointerutil.StringPtr("/path/on/disk/to/template.ctmpl"),
Destination: pointerutil.StringPtr("/path/on/disk/where/template/will/render.txt"),
},
},
},
"full": {
fixturePath: "./test-fixtures/config-template-full.hcl",
expectedTemplates: []*ctconfig.TemplateConfig{
&ctconfig.TemplateConfig{
Backup: pointerutil.BoolPtr(true),
Command: pointerutil.StringPtr("restart service foo"),
CommandTimeout: pointerutil.TimeDurationPtr("60s"),
Contents: pointerutil.StringPtr("{{ keyOrDefault \"service/redis/maxconns@east-aws\" \"5\" }}"),
CreateDestDirs: pointerutil.BoolPtr(true),
Destination: pointerutil.StringPtr("/path/on/disk/where/template/will/render.txt"),
ErrMissingKey: pointerutil.BoolPtr(true),
LeftDelim: pointerutil.StringPtr("<<"),
Perms: pointerutil.FileModePtr(0655),
RightDelim: pointerutil.StringPtr(">>"),
SandboxPath: pointerutil.StringPtr("/path/on/disk/where"),
Wait: &ctconfig.WaitConfig{
Min: pointerutil.TimeDurationPtr("10s"),
Max: pointerutil.TimeDurationPtr("40s"),
},
},
},
},
"many": {
fixturePath: "./test-fixtures/config-template-many.hcl",
expectedTemplates: []*ctconfig.TemplateConfig{
&ctconfig.TemplateConfig{
Source: pointerutil.StringPtr("/path/on/disk/to/template.ctmpl"),
Destination: pointerutil.StringPtr("/path/on/disk/where/template/will/render.txt"),
ErrMissingKey: pointerutil.BoolPtr(false),
CreateDestDirs: pointerutil.BoolPtr(true),
Command: pointerutil.StringPtr("restart service foo"),
Perms: pointerutil.FileModePtr(0600),
},
&ctconfig.TemplateConfig{
Source: pointerutil.StringPtr("/path/on/disk/to/template2.ctmpl"),
Destination: pointerutil.StringPtr("/path/on/disk/where/template/will/render2.txt"),
Backup: pointerutil.BoolPtr(true),
Perms: pointerutil.FileModePtr(0755),
Wait: &ctconfig.WaitConfig{
Min: pointerutil.TimeDurationPtr("2s"),
Max: pointerutil.TimeDurationPtr("10s"),
},
},
},
},
}
for name, tc := range testCases {
t.Run(name, func(t *testing.T) {
config, err := LoadConfig(tc.fixturePath)
if err != nil {
t.Fatalf("err: %s", err)
}
expected := &Config{
AutoAuth: &AutoAuth{
Method: &Method{
Type: "aws",
MountPath: "auth/aws",
Namespace: "my-namespace/",
Config: map[string]interface{}{
"role": "foobar",
},
},
Sinks: []*Sink{
&Sink{
Type: "file",
DHType: "curve25519",
DHPath: "/tmp/file-foo-dhpath",
AAD: "foobar",
Config: map[string]interface{}{
"path": "/tmp/file-foo",
},
},
},
},
Templates: tc.expectedTemplates,
PidFile: "./pidfile",
}
if diff := deep.Equal(config, expected); diff != nil {
t.Fatal(diff)
}
})
}
}

View File

@ -0,0 +1,49 @@
pid_file = "./pidfile"
auto_auth {
method {
type = "aws"
namespace = "/my-namespace"
config = {
role = "foobar"
}
}
sink {
type = "file"
config = {
path = "/tmp/file-foo"
}
aad = "foobar"
dh_type = "curve25519"
dh_path = "/tmp/file-foo-dhpath"
}
}
template {
destination = "/path/on/disk/where/template/will/render.txt"
create_dest_dirs = true
contents = "{{ keyOrDefault \"service/redis/maxconns@east-aws\" \"5\" }}"
command = "restart service foo"
command_timeout = "60s"
error_on_missing_key = true
perms = 0655
backup = true
left_delimiter = "<<"
right_delimiter = ">>"
sandbox_path = "/path/on/disk/where"
wait {
min = "5s"
max = "30s"
}
wait {
min = "10s"
max = "40s"
}
}

View File

@ -0,0 +1,50 @@
pid_file = "./pidfile"
auto_auth {
method {
type = "aws"
namespace = "/my-namespace"
config = {
role = "foobar"
}
}
sink {
type = "file"
config = {
path = "/tmp/file-foo"
}
aad = "foobar"
dh_type = "curve25519"
dh_path = "/tmp/file-foo-dhpath"
}
}
template {
source = "/path/on/disk/to/template.ctmpl"
destination = "/path/on/disk/where/template/will/render.txt"
create_dest_dirs = true
command = "restart service foo"
error_on_missing_key = false
perms = 0600
}
template {
source = "/path/on/disk/to/template2.ctmpl"
destination = "/path/on/disk/where/template/will/render2.txt"
perms = 0755
backup = true
wait {
min = "2s"
max = "10s"
}
}

View File

@ -0,0 +1,29 @@
pid_file = "./pidfile"
auto_auth {
method {
type = "aws"
namespace = "/my-namespace"
config = {
role = "foobar"
}
}
sink {
type = "file"
config = {
path = "/tmp/file-foo"
}
aad = "foobar"
dh_type = "curve25519"
dh_path = "/tmp/file-foo-dhpath"
}
}
template {
source = "/path/on/disk/to/template.ctmpl"
destination = "/path/on/disk/where/template/will/render.txt"
}

8
command/agent/doc.go Normal file
View File

@ -0,0 +1,8 @@
/*
Package agent implements a daemon mode of Vault designed to provide helper
features like auto-auth, caching, and templating.
Agent has it's own configuration stanza and operates as a proxy to a Vault
service.
*/
package agent

View File

@ -0,0 +1,188 @@
// Package template is responsible for rendering user supplied templates to
// disk. The Server type accepts configuration to communicate to a Vault server
// and a Vault token for authentication. Internally, the Server creates a Consul
// Template Runner which manages reading secrets from Vault and rendering
// templates to disk at configured locations
package template
import (
"context"
"strings"
ctconfig "github.com/hashicorp/consul-template/config"
"github.com/hashicorp/consul-template/manager"
"github.com/hashicorp/go-hclog"
"github.com/hashicorp/vault/command/agent/config"
"github.com/hashicorp/vault/sdk/helper/pointerutil"
)
// ServerConfig is a config struct for setting up the basic parts of the
// Server
type ServerConfig struct {
Logger hclog.Logger
// Client *api.Client
VaultConf *config.Vault
ExitAfterAuth bool
Namespace string
}
// Server manages the Consul Template Runner which renders templates
type Server struct {
// UnblockCh is used to block until a template is rendered
UnblockCh chan struct{}
// config holds the ServerConfig used to create it. It's passed along in other
// methods
config *ServerConfig
// runner is the consul-template runner
runner *manager.Runner
// Templates holds the parsed Consul Templates
Templates []*ctconfig.TemplateConfig
// TODO: remove donech?
DoneCh chan struct{}
logger hclog.Logger
exitAfterAuth bool
}
// NewServer returns a new configured server
func NewServer(conf *ServerConfig) *Server {
ts := Server{
DoneCh: make(chan struct{}),
logger: conf.Logger,
UnblockCh: make(chan struct{}),
config: conf,
exitAfterAuth: conf.ExitAfterAuth,
}
return &ts
}
// Run kicks off the internal Consul Template runner, and listens for changes to
// the token from the AuthHandler. If Done() is called on the context, shut down
// the Runner and return
func (ts *Server) Run(ctx context.Context, incoming chan string, templates []*ctconfig.TemplateConfig) {
latestToken := new(string)
ts.logger.Info("starting template server")
defer func() {
ts.logger.Info("template server stopped")
close(ts.DoneCh)
}()
if incoming == nil {
panic("incoming channel is nil")
}
// If there are no templates, close the UnblockCh
if len(templates) == 0 {
// nothing to do
ts.logger.Info("no templates found")
close(ts.UnblockCh)
return
}
// construct a consul template vault config based the agents vault
// configuration
var runnerConfig *ctconfig.Config
if runnerConfig = newRunnerConfig(ts.config, templates); runnerConfig == nil {
ts.logger.Error("template server failed to generate runner config")
close(ts.UnblockCh)
return
}
var err error
ts.runner, err = manager.NewRunner(runnerConfig, false)
if err != nil {
ts.logger.Error("template server failed to create", "error", err)
close(ts.UnblockCh)
return
}
for {
select {
case <-ctx.Done():
ts.runner.StopImmediately()
ts.runner = nil
return
case token := <-incoming:
if token != *latestToken {
ts.logger.Info("template server received new token")
ts.runner.Stop()
*latestToken = token
ctv := ctconfig.Config{
Vault: &ctconfig.VaultConfig{
Token: latestToken,
},
}
runnerConfig.Merge(&ctv)
runnerConfig.Finalize()
var runnerErr error
ts.runner, runnerErr = manager.NewRunner(runnerConfig, false)
if runnerErr != nil {
ts.logger.Error("template server failed with new Vault token", "error", runnerErr)
continue
} else {
go ts.runner.Start()
}
}
case err := <-ts.runner.ErrCh:
ts.logger.Error("template server error", "error", err.Error())
close(ts.UnblockCh)
return
case <-ts.runner.TemplateRenderedCh():
// A template has been rendered, unblock
if ts.exitAfterAuth {
// if we want to exit after auth, go ahead and shut down the runner
ts.runner.Stop()
}
close(ts.UnblockCh)
}
}
}
// newRunnerConfig returns a consul-template runner configuration, setting the
// Vault and Consul configurations based on the clients configs.
func newRunnerConfig(sc *ServerConfig, templates ctconfig.TemplateConfigs) *ctconfig.Config {
// TODO only use default Vault config
conf := ctconfig.DefaultConfig()
conf.Templates = templates.Copy()
// Setup the Vault config
// Always set these to ensure nothing is picked up from the environment
conf.Vault.RenewToken = pointerutil.BoolPtr(false)
conf.Vault.Token = pointerutil.StringPtr("")
conf.Vault.Address = &sc.VaultConf.Address
if sc.Namespace != "" {
conf.Vault.Namespace = &sc.Namespace
}
conf.Vault.SSL = &ctconfig.SSLConfig{
Enabled: pointerutil.BoolPtr(false),
Verify: pointerutil.BoolPtr(false),
Cert: pointerutil.StringPtr(""),
Key: pointerutil.StringPtr(""),
CaCert: pointerutil.StringPtr(""),
CaPath: pointerutil.StringPtr(""),
ServerName: pointerutil.StringPtr(""),
}
if strings.HasPrefix(sc.VaultConf.Address, "https") || sc.VaultConf.CACert != "" {
skipVerify := sc.VaultConf.TLSSkipVerify
verify := !skipVerify
conf.Vault.SSL = &ctconfig.SSLConfig{
Enabled: pointerutil.BoolPtr(true),
Verify: &verify,
Cert: &sc.VaultConf.ClientCert,
Key: &sc.VaultConf.ClientKey,
CaCert: &sc.VaultConf.CACert,
CaPath: &sc.VaultConf.CAPath,
}
}
conf.Finalize()
return conf
}

View File

@ -0,0 +1,159 @@
package template
import (
"context"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
"os"
"testing"
ctconfig "github.com/hashicorp/consul-template/config"
"github.com/hashicorp/go-hclog"
"github.com/hashicorp/vault/command/agent/config"
"github.com/hashicorp/vault/sdk/helper/logging"
"github.com/hashicorp/vault/sdk/helper/pointerutil"
)
// TestNewServer is a simple test to make sure NewServer returns a Server and
// channel
func TestNewServer(t *testing.T) {
server := NewServer(&ServerConfig{})
if server == nil {
t.Fatal("nil server returned")
}
if server.UnblockCh == nil {
t.Fatal("nil blocking channel returned")
}
}
func TestServerRun(t *testing.T) {
// create http test server
ts := httptest.NewServer(http.HandlerFunc(handleRequest))
defer ts.Close()
tmpDir, err := ioutil.TempDir("", "agent-tests")
defer os.RemoveAll(tmpDir)
if err != nil {
t.Fatal(err)
}
testCases := map[string]struct {
templates []*ctconfig.TemplateConfig
}{
"basic": {
templates: []*ctconfig.TemplateConfig{
&ctconfig.TemplateConfig{
Contents: pointerutil.StringPtr(templateContents),
},
},
},
}
// secretRender is a simple struct that represents the secret we render to
// disk. It's used to unmarshal the file contents and test against
type secretRender struct {
Username string `json:"username"`
Password string `json:"password"`
Version string `json:"version"`
}
for name, tc := range testCases {
t.Run(name, func(t *testing.T) {
templateTokenCh := make(chan string, 1)
for i, template := range tc.templates {
dstFile := fmt.Sprintf("%s/render_%d.txt", tmpDir, i)
template.Destination = pointerutil.StringPtr(dstFile)
}
ctx, cancelFunc := context.WithCancel(context.Background())
sc := ServerConfig{
Logger: logging.NewVaultLogger(hclog.Trace),
VaultConf: &config.Vault{
Address: ts.URL,
},
}
var server *Server
server = NewServer(&sc)
if ts == nil {
t.Fatal("nil server returned")
}
if server.UnblockCh == nil {
t.Fatal("nil blocking channel returned")
}
go server.Run(ctx, templateTokenCh, tc.templates)
// send a dummy value to trigger the internal Runner to query for secret
// info
templateTokenCh <- "test"
select {
case <-ctx.Done():
case <-server.UnblockCh:
}
// cancel to clean things up
cancelFunc()
// verify test file exists and has the content we're looking for
for _, template := range tc.templates {
if template.Destination == nil {
t.Fatal("nil template destination")
}
content, err := ioutil.ReadFile(*template.Destination)
if err != nil {
t.Fatal(err)
}
secret := secretRender{}
if err := json.Unmarshal(content, &secret); err != nil {
t.Fatal(err)
}
if secret.Username != "appuser" || secret.Password != "password" || secret.Version != "3" {
t.Fatalf("secret didn't match: %#v", secret)
}
}
})
}
}
func handleRequest(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, jsonResponse)
}
var jsonResponse = `
{
"request_id": "8af096e9-518c-7351-eff5-5ba20554b21f",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": {
"data": {
"password": "password",
"username": "appuser"
},
"metadata": {
"created_time": "2019-10-07T22:18:44.233247Z",
"deletion_time": "",
"destroyed": false,
"version": 3
}
},
"wrap_info": null,
"warnings": null,
"auth": null
}
`
var templateContents = `
{{ with secret "kv/myapp/config"}}
{
{{ if .Data.data.username}}"username":"{{ .Data.data.username}}",{{ end }}
{{ if .Data.data.password }}"password":"{{ .Data.data.password }}",{{ end }}
{{ if .Data.metadata.version}}"version":"{{ .Data.metadata.version }}"{{ end }}
}
{{ end }}
`

3
go.mod
View File

@ -48,7 +48,8 @@ require (
github.com/google/go-github v17.0.0+incompatible
github.com/google/go-metrics-stackdriver v0.0.0-20190816035513-b52628e82e2a
github.com/google/go-querystring v1.0.0 // indirect
github.com/hashicorp/consul/api v1.0.1
github.com/hashicorp/consul-template v0.22.0
github.com/hashicorp/consul/api v1.1.0
github.com/hashicorp/errwrap v1.0.0
github.com/hashicorp/go-cleanhttp v0.5.1
github.com/hashicorp/go-gcp-common v0.5.0

29
go.sum
View File

@ -16,6 +16,7 @@ github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78 h1:w+iIsaOQNcT7O
github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8=
github.com/Azure/go-autorest v11.7.1+incompatible h1:M2YZIajBBVekV86x0rr1443Lc1F/Ylxb9w+5EtSyX3Q=
github.com/Azure/go-autorest v11.7.1+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24=
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/DataDog/datadog-go v2.2.0+incompatible h1:V5BKkxACZLjzHjSgBbr2gvLA2Ae49yhc6CSY7MLy5k4=
github.com/DataDog/datadog-go v2.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
@ -151,6 +152,7 @@ github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5Kwzbycv
github.com/fatih/structs v1.1.0 h1:Q7juDM0QtcnhCpeyLGQKyg4TOIghuNXrkL32pHAUMxo=
github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/frankban/quicktest v1.4.0/go.mod h1:36zfPVQyHxymz4cH7wlDmVwDrJuljRB60qkgn7rorfQ=
github.com/frankban/quicktest v1.4.1 h1:Wv2VwvNn73pAdFIVUQRXYDFp31lXKbqblIXo/Q5GPSg=
github.com/frankban/quicktest v1.4.1/go.mod h1:36zfPVQyHxymz4cH7wlDmVwDrJuljRB60qkgn7rorfQ=
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
@ -213,6 +215,8 @@ github.com/google/btree v0.0.0-20180124185431-e89373fe6b4a h1:ZJu5NB1Bk5ms4vw0Xu
github.com/google/btree v0.0.0-20180124185431-e89373fe6b4a/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c h1:964Od4U6p2jUkFxvCydnIczKteheJEzHRToSGK3Bnlw=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0 h1:0udJVsspx3VBr5FwtLhQQtuAsVc79tTq0ocGIPAU6qo=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
@ -260,10 +264,10 @@ github.com/grpc-ecosystem/grpc-gateway v1.8.5 h1:2+KSC78XiO6Qy0hIjfc1OD9H+hsaJdJ
github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed h1:5upAirOpQc1Q53c0bnx2ufif5kANL7bfZWcc6VJWJd8=
github.com/hailocab/go-hostpool v0.0.0-20160125115350-e80d13ce29ed/go.mod h1:tMWxXQ9wFIaZeTI9F+hmhFiGpFmhOHzyShyFUhRm0H4=
github.com/hashicorp/consul/api v1.0.1 h1:LkHu3cLXjya4lgrAyZVe/CUBXgJ7AcDWKSeCjAYN9w0=
github.com/hashicorp/consul/api v1.0.1/go.mod h1:LQlewHPiuaRhn1mP2XE4RrjnlRgOeWa/ZM0xWLCen2M=
github.com/hashicorp/consul/sdk v0.1.0 h1:tTfutTNVUTDXpNM4YCImLfiiY3yCDpfgS6tNlUioIUE=
github.com/hashicorp/consul/sdk v0.1.0/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
github.com/hashicorp/consul-template v0.22.0 h1:ti5cqAekOeMfFYLJCjlPtKGwBcqwVxoZO/Y2vctwuUE=
github.com/hashicorp/consul-template v0.22.0/go.mod h1:lHrykBIcPobCuEcIMLJryKxDyk2lUMnQWmffOEONH0k=
github.com/hashicorp/consul/api v1.1.0 h1:BNQPM9ytxj6jbjjdRPioQ94T6YXriSopn0i8COv6SRA=
github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=
github.com/hashicorp/consul/sdk v0.1.1 h1:LnuDWGNsoajlhGyHJvuWW6FVqRl8JOTPqS6CPTsYjhY=
github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA=
@ -271,6 +275,8 @@ github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brv
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-cleanhttp v0.5.1 h1:dH3aiDG9Jvb5r5+bYHsikaOUIpcM0xvgMXVoDkXMzJM=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-gatedio v0.5.0 h1:Jm1X5yP4yCqqWj5L1TgW7iZwCVPGtVc+mro5r/XX7Tg=
github.com/hashicorp/go-gatedio v0.5.0/go.mod h1:Lr3t8L6IyxD3DAeaUxGcgl2JnRUpWMCsmBl4Omu/2t4=
github.com/hashicorp/go-gcp-common v0.5.0 h1:kkIQTjNTopn4eXQ1+lCiHYZXUtgIZvbc6YtAQkMnTos=
github.com/hashicorp/go-gcp-common v0.5.0/go.mod h1:IDGUI2N/OS3PiU4qZcXJeWKPI6O/9Y8hOrbSiMcqyYw=
github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd/go.mod h1:9bjs9uLqI8l75knNv3lV1kA55veR+WUPSiKIWcQHudI=
@ -329,8 +335,9 @@ github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=
github.com/hashicorp/memberlist v0.1.3 h1:EmmoJme1matNzb+hMpDuR/0sbJSUisxyqBGG676r31M=
github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=
github.com/hashicorp/memberlist v0.1.4 h1:gkyML/r71w3FL8gUi74Vk76avkj/9lYAY9lvg0OcoGs=
github.com/hashicorp/memberlist v0.1.4/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=
github.com/hashicorp/nomad/api v0.0.0-20190412184103-1c38ced33adf h1:U/40PQvWkaXCDdK9QHKf1pVDVcA+NIDVbzzonFGkgIA=
github.com/hashicorp/nomad/api v0.0.0-20190412184103-1c38ced33adf/go.mod h1:BDngVi1f4UA6aJq9WYTgxhfWSE1+42xshvstLU2fRGk=
github.com/hashicorp/raft v1.0.1/go.mod h1:DVSAWItjLjTOkVbSpWQ0j0kUADIvDaCtBxIcbNAQLkI=
@ -341,6 +348,8 @@ github.com/hashicorp/raft-snapshot v1.0.2-0.20190827162939-8117efcc5aab h1:WzGMw
github.com/hashicorp/raft-snapshot v1.0.2-0.20190827162939-8117efcc5aab/go.mod h1:5sL9eUn72lH5DzsFIJ9jaysITbHksSSszImWSOTC8Ic=
github.com/hashicorp/serf v0.8.2 h1:YZ7UKsJv+hKjqGVUUbtE3HNj79Eln2oQ75tniF6iPt0=
github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=
github.com/hashicorp/serf v0.8.3 h1:MWYcmct5EtKz0efYooPcL0yNkem+7kWxqXDi/UIh+8k=
github.com/hashicorp/serf v0.8.3/go.mod h1:UpNcs7fFbpKIyZaUuSW6EPiH+eZC7OuyFD+wc1oal+k=
github.com/hashicorp/vault-plugin-auth-alicloud v0.5.2-0.20190814210027-93970f08f2ec h1:HXVE8h6RXFsPJgwWpE+5CscsgekqtX4nhDlZGV9jEe4=
github.com/hashicorp/vault-plugin-auth-alicloud v0.5.2-0.20190814210027-93970f08f2ec/go.mod h1:TYFfVFgKF9x92T7uXouI9rLPkNnyXo/KkNcj5t+mjdM=
github.com/hashicorp/vault-plugin-auth-azure v0.5.2-0.20190814210035-08e00d801115 h1:E57y918o+c+NoI5k7ohbpZu7vRm1XZKZfC5VQVpJvDI=
@ -438,6 +447,8 @@ github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNx
github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-shellwords v1.0.5 h1:JhhFTIOslh5ZsPrpa3Wdg8bF0WI3b44EMblmU9wIsXc=
github.com/mattn/go-shellwords v1.0.5/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o=
github.com/matttproud/golang_protobuf_extensions v1.0.0 h1:YNOwxxSJzSUARoD9KRZLzM9Y858MNGCOACTvCW9TSAc=
github.com/matttproud/golang_protobuf_extensions v1.0.0/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
@ -446,8 +457,9 @@ github.com/mholt/archiver v3.1.1+incompatible h1:1dCVxuqs0dJseYEhi5pl7MYPH9zDa1w
github.com/mholt/archiver v3.1.1+incompatible/go.mod h1:Dh2dOXnSdiLxRiPoVfIr/fI1TwETms9B8CTWfeh7ROU=
github.com/michaelklishin/rabbit-hole v1.5.0 h1:Bex27BiFDsijCM9D0ezSHqyy0kehpYHuNKaPqq/a4RM=
github.com/michaelklishin/rabbit-hole v1.5.0/go.mod h1:vvI1uOitYZi0O5HEGXhaWC1XT80Gy+HvFheJ+5Krlhk=
github.com/miekg/dns v1.0.14 h1:9jZdLNd/P4+SfEJ0TNyxYpsK8N4GtfylBLqtbYN1sbA=
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/miekg/dns v1.1.15 h1:CSSIDtllwGLMoA6zjdKnaE6Tx6eVUxQ29LUgGetiDCI=
github.com/miekg/dns v1.1.15/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/mitchellh/cli v1.0.0 h1:iGBIsUe3+HZ/AD/Vd7DErOt5sU9fa8Uj7A2s1aggv1Y=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
github.com/mitchellh/copystructure v1.0.0 h1:Laisrj+bAB6b/yJwB5Bt3ITZhGJdqmxquMKeZ+mmkFQ=
@ -460,6 +472,8 @@ github.com/mitchellh/go-testing-interface v1.0.0 h1:fzU/JVNcaqHQEcVFAKeR41fkiLdI
github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
github.com/mitchellh/go-wordwrap v1.0.0/go.mod h1:ZXFpozHsX6DPmq2I0TCekCxypsnAUbP2oI0UX1GXzOo=
github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=
github.com/mitchellh/hashstructure v1.0.0 h1:ZkRJX1CyOoTkar7p/mLS5TZU4nJ1Rn/F8u9dGS02Q3Y=
github.com/mitchellh/hashstructure v1.0.0/go.mod h1:QjSHrPWS+BGUVBYkbTZWEnOh3G1DutKwClXU/ABz6AQ=
github.com/mitchellh/iochan v1.0.0 h1:C+X3KsSTLFVBr/tK1eYN/vs4rJcvsiLU338UhYPJWeY=
github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
@ -515,6 +529,7 @@ github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaR
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pierrec/lz4 v2.0.5+incompatible h1:2xWsjqPFWcplujydGg4WmhC/6fZqK42wMM8aXeqhl0I=
github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pierrec/lz4 v2.2.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pierrec/lz4 v2.2.6+incompatible h1:6aCX4/YZ9v8q69hTyiR7dNLnTA3fgtKHVVW5BCd5Znw=
github.com/pierrec/lz4 v2.2.6+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw=
@ -678,6 +693,7 @@ golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3 h1:0GoQqolDA55aaLxZyTzK/Y2eP
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859 h1:R/3boaszxrf1GEUWTVDzSKVwLmSJpwZ1yqXm8j0v2QI=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7 h1:fHDIZ2oxGnUZRN6WgWFCbYBjH9uqVPRCUVUDhs0wnbA=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@ -719,6 +735,7 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20190514135907-3a4b5fb9f71f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190523142557-0e01d883c5c5 h1:sM3evRHxE/1RuMe1FYAL3j7C7fUfIjkbE+NiDAYUF8U=
golang.org/x/sys v0.0.0-20190523142557-0e01d883c5c5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190730183949-1393eb018365/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a h1:aYOabOQFp6Vj6W1F80affTUvO9UxmJRx8K0gsfABByQ=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=

View File

@ -0,0 +1,28 @@
package pointerutil
import (
"os"
"time"
)
// StringPtr returns a pointer to a string value
func StringPtr(s string) *string {
return &s
}
// BoolPtr returns a pointer to a boolean value
func BoolPtr(b bool) *bool {
return &b
}
// TimeDurationPtr returns a pointer to a time duration value
func TimeDurationPtr(duration string) *time.Duration {
d, _ := time.ParseDuration(duration)
return &d
}
// FileModePtr returns a pointer to the given os.FileMode
func FileModePtr(o os.FileMode) *os.FileMode {
return &o
}

View File

@ -54,6 +54,8 @@ These are the currently-available general configuration option:
with code `0` after a single successful auth, where success means that a
token was retrieved and all sinks successfully wrote it
- `template` <tt>([template][template`]: \<optional\>)</tt> - Specifies options used for templating Vault secrets to files.
### vault Stanza
There can at most be one top level `vault` block and it has the following
@ -150,10 +152,21 @@ listener "tcp" {
address = "127.0.0.1:8100"
tls_disable = true
}
template {
source = "/etc/vault/server.key.ctmpl"
destination = "/etc/vault/server.key"
}
template {
source = "/etc/vault/server.crt.ctmpl"
destination = "/etc/vault/server.crt"
}
```
[vault]: /docs/agent/index.html#vault-stanza
[autoauth]: /docs/agent/autoauth/index.html
[caching]: /docs/agent/caching/index.html
[template]: /docs/agent/template/index.html
[listener]: /docs/agent/index.html#listener-stanza
[listener_main]: /docs/configuration/listener/tcp.html

View File

@ -0,0 +1,121 @@
---
layout: "docs"
page_title: "Vault Agent Templates"
sidebar_title: "TemplatesAuto-Auth"
sidebar_current: "docs-agent-templates"
description: |-
Vault Agent's Template functionality allows Vault secrets to be rendered to files
using Consul Template markup.
---
# Vault Agent Templates
Vault Agent's Template functionality allows Vault secrets to be rendered to files
using Consul Template markup.
## Functionality
The `template` stanza configures the templating engine in the Vault agent for rendering
secrets to files using Consul Template markup language. Multiple `template` stanzas
can be defined to render multiple files.
When the agent is started with templating enabled, it will attempt to acquire a
Vault token using the configured Method. On failure, it will back off for a short
while (including some randomness to help prevent thundering herd scenarios) and
retry. On success, secrets defined in the templates will be retrieved from Vault and
rendered locally.
## Configuration
The top level `template` block has multiple configurations entries:
- `source` `(object: optional)` - Path on disk to use as the input template. This
option is required if not using the `contents` option.
- `destination` `(object: required)` - Path on disk where the rendered secrets should
be created. If the parent directories If the parent directories do not exist, Vault
Agent will attempt to create them, unless `create_dest_dirs` is false.
- `create_dest_dirs` `(object: required)` - This option tells Vault Agent to create
the parent directories of the destination path if they do not exist. The default
value is true.
- `contents` `(object: optional)` - This option allows embedding the contents of
a template in the configuration file rather then supplying the `source` path to
the template file. This is useful for short templates. This option is mutually
exclusive with the `source` option.
- `command` `(object: optional)` - This is the optional command to run when the
template is rendered. The command will only run if the resulting template changes.
The command must return within 30s (configurable), and it must have a successful
exit code. Vault Agent is not a replacement for a process monitor or init system.
- `command_timeout` `(object: optional)` - This is the maximum amount of time to
wait for the optional command to return. Default is 30s.
- `error_on_missing_key` `(object: optional)` - Exit with an error when accessing
a struct or map field/key that does notexist. The default behavior will print "<no value>"
when accessing a field that does not exist. It is highly recommended you set this
to "true".
- `perms` `(object: optional)` - This is the permission to render the file. If
this option is left unspecified, Vault Agent will attempt to match the permissions
of the file that already exists at the destination path. If no file exists at that
path, the permissions are 0644.
- `backup` `(object: optional)` - This option backs up the previously rendered template
at the destination path before writing a new one. It keeps exactly one backup.
This option is useful for preventing accidental changes to the data without having
a rollback strategy.
- `left_delimiter` `(object: optional)` - Delimiter to use in the template. The
default is "{{" but for some templates, it may be easier to use a different
delimiter that does not conflict with the output file itself.
- `right_delimiter` `(object: optional)` - Delimiter to use in the template. The
default is "}}" but for some templates, it may be easier to use a different
delimiter that does not conflict with the output file itself.
- `sandbox_path` `(object: optional)` - If a sandbox path is provided, any path
provided to the `file` function is checked that it falls within the sandbox path.
Relative paths that try to traverse outside the sandbox path will exit with an error.
- `wait` `(object: required)` - This is the `minimum(:maximum)` to wait before rendering
a new template to disk and triggering a command, separated by a colon (`:`).
## Example Template
Template with Vault Agent requires the use of the `secret` function from Consul Template.
The following is an example of a template that retrieves a generic secret from Vault's
KV store:
```
{{ with secret "secret/my-secret" }}
{{ .Data.data.foo }}
{{ end }}
```
## Example Configuration
The following demonstrates configuring Vault Agent to template secrets using the
AppRole Auth method:
```python
pid_file = "./pidfile"
vault {
address = "https://127.0.0.1:8200"
}
auto_auth {
method {
type = "approle"
config = {
role_id_file_path = "/etc/vault/roleid"
secret_id_file_path = "/etc/vault/secretid"
}
}
sink {
type = "file"
config = {
path = "/tmp/file-foo"
}
}
}
template {
source = "/tmp/agent/template.ctmpl"
destination = "/tmp/agent/render.txt"
}
```