mirror of
https://github.com/hashicorp/vault.git
synced 2025-08-23 15:41:07 +02:00
270 lines
8.7 KiB
Plaintext
270 lines
8.7 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Secret data transformation
|
|
description: >-
|
|
Learn how to transform secret data with the Vault Secrets Operator.
|
|
---
|
|
|
|
# Secret data transformation
|
|
|
|
Utilizing advanced templating and data filters, the Vault Secrets Operator for Kubernetes (VSO) can
|
|
transform source secret data, secret metadata, resource labels and annotations into a format that is
|
|
compatible with your application. All secret data sources are supported. Secret transformations can
|
|
be specified directly within a secret custom resource (CR), or by references to one or more
|
|
[SecretTransformation](/vault/docs/platform/k8s/vso/api-reference#secrettransformation) custom resource
|
|
instances, or both.
|
|
|
|
## Templating
|
|
|
|
VSO utilizes the data-driven [templates for Golang](https://pkg.go.dev/text/template) to generate
|
|
secret data output. The template data input holds the secret data, secret metadata, resource labels
|
|
and annotations.
|
|
|
|
Templates are configured in a secret custom resource's
|
|
[spec.Destination.Transformation.Templates](/vault/docs/platform/k8s/vso/api-reference#transformation),
|
|
or in a SecretTransformation resource's [spec.templates](/vault/docs/platform/k8s/vso/api-reference#secrettransformationspec).
|
|
|
|
VSO provides access to a large library of template functions, some of which are documented
|
|
[below](#template-functions).
|
|
|
|
### Secret data input
|
|
|
|
Secret data is accessed through the `.Secrets` input member. It contains a map of secret
|
|
key-value pairs, which are assumed to be sensitive information fetched from a
|
|
[secret source](/vault/docs/platform/k8s/vso/sources).
|
|
|
|
For example, to include a password in your application's secret, you might specify
|
|
a template like:
|
|
```go
|
|
{{- printf "password=%s" (get .Secrets "password") -}}
|
|
```
|
|
|
|
### Secret metadata input
|
|
|
|
Secret metadata is accessed through the `.Metadata` input member. It contains a map of metadata key to
|
|
its value. The data should not contain any confidential information.
|
|
|
|
For example, to include a secret metadata value in your application's secret, you might specify
|
|
a template like:
|
|
```go
|
|
{{- printf "secretGroup=%s" (get .Metadata "secretGroup") -}}
|
|
```
|
|
|
|
### Resource annotations input
|
|
|
|
Resource annotations are accessed through the `.Annotations` input member. The annotations consist
|
|
of all `metadata.annotations` configured on the secret custom resource.
|
|
|
|
For example, to include a value from the resource's annotations in your application's secret, you
|
|
might specify a template like:
|
|
```go
|
|
{{- printf "host=%s" (get .Annotations "myapp.config/host") -}}
|
|
```
|
|
|
|
### Resource labels input
|
|
|
|
Resource labels are accessed through the `.Labels` input member. The labels consist
|
|
of all `metadata.labels` configured on the secret custom resource.
|
|
|
|
For example, to include a value from the resource's labels in your application's secret, you
|
|
might specify a template like:
|
|
```go
|
|
{{- printf "appType=%s" (get .Labels "appType") -}}
|
|
```
|
|
|
|
## Filters
|
|
|
|
Filters are used to control which source secret data fields are included in the destination secret's
|
|
data. They are specified as a set of exclude/include RE2 accepted [regular expressions](https://golang.org/s/re2syntax).
|
|
|
|
Filters are configured in the `excludes` and `includes` fields of a secret custom resource's
|
|
[spec.Destination.Transformation](/vault/docs/platform/k8s/vso/api-reference#transformation),
|
|
or in a SecretTransformation resource's [spec](/vault/docs/platform/k8s/vso/api-reference#secrettransformationspec).
|
|
|
|
All exclude patterns take precedence over any include patterns, and are never applied to templated keys.
|
|
|
|
## Examples
|
|
|
|
### Local transformation
|
|
|
|
A VaultDynamicSecret configured to sync Postgres database credentials from Vault to the Kubernetes
|
|
secret named `example-vds`.
|
|
```yaml
|
|
---
|
|
apiVersion: secrets.hashicorp.com/v1beta1
|
|
kind: VaultDynamicSecret
|
|
metadata:
|
|
namespace: example-ns
|
|
name: example-vds
|
|
annotations:
|
|
myapp.config/postgres-host: postgres-postgresql.postgres.svc.cluster.local:5432
|
|
spec:
|
|
destination:
|
|
create: true
|
|
name: app-secret
|
|
transformation:
|
|
excludes:
|
|
- .*
|
|
templates:
|
|
url:
|
|
text: |
|
|
{{- $host := get .Annotations "myapp.config/postgres-host" -}}
|
|
{{- printf "postgresql://%s:%s@%s/postgres?sslmode=disable" (get .Secrets "username") (get .Secrets "password") $host -}}
|
|
path: creds/dev-postgres
|
|
```
|
|
|
|
The resulting Kubernetes secret includes a single key named `url`, with a valid Postgres connection
|
|
URL as its value.
|
|
|
|
```yaml
|
|
url: postgresql://v-postgres-user:XUpah-password@postgres-postgresql.postgres.svc.cluster.local:5432/postgres?sslmode=disable
|
|
```
|
|
|
|
### Shared transformation
|
|
|
|
The following manifest contains shared transformation templates and filters. All `templates` it provides
|
|
will be included in the destination k8s secret. It also provides `sourceTemplates` that can be included
|
|
in any template text configured in a secret CR or within the same resource instance.
|
|
|
|
```yaml
|
|
---
|
|
apiVersion: secrets.hashicorp.com/v1beta1
|
|
kind: SecretTransformation
|
|
metadata:
|
|
name: vso-templates
|
|
namespace: example-vds
|
|
spec:
|
|
excludes:
|
|
- password|username
|
|
templates:
|
|
url:
|
|
text: '{{- template "dbUrl" . -}}'
|
|
sourceTemplates:
|
|
- name: helpers
|
|
text: |
|
|
{{/*
|
|
create a Java props from SecretInput for this app
|
|
*/}}
|
|
{{- define "appProps" -}}
|
|
{{- $host := get .Annotations "myapp.config/postgres-host" -}}
|
|
{{- printf "db.host=%s\n" $host -}}
|
|
{{- range $k, $v := .Secrets -}}
|
|
{{- printf "db.%s=%s\n" $k $v -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{/*
|
|
create a JSON config from SecretInput for this app
|
|
*/}}
|
|
{{- define "appJson" -}}
|
|
{{- $host := get .Annotations "myapp.config/postgres-host" -}}
|
|
{{- $copy := .Secrets | mustDeepCopy -}}
|
|
{{- $_ := set $copy "host" $host -}}
|
|
{{- mustToPrettyJson $copy -}}
|
|
{{- end -}}
|
|
{{/*
|
|
compose a Postgres URL from SecretInput for this app
|
|
*/}}
|
|
{{- define "dbUrl" -}}
|
|
{{- $host := get .Annotations "myapp.config/postgres-host" -}}
|
|
{{- printf "postgresql://%s:%s@%s/postgres?sslmode=disable" (get .Secrets "username") (get .Secrets "password") $host -}}
|
|
{{- end -}}
|
|
{{/*
|
|
get the app name from the VSO resource's label
|
|
*/}}
|
|
{{- define "appName" -}}
|
|
{{- get .Labels "myapp/name" -}}
|
|
{{- end -}}
|
|
```
|
|
|
|
The following `VaultDynamicSecret` manifest references the `SecretTransformation` above.
|
|
All templates and filters from the reference object will be applied to the destination secret data.
|
|
```yaml
|
|
---
|
|
apiVersion: secrets.hashicorp.com/v1beta1
|
|
kind: VaultDynamicSecret
|
|
metadata:
|
|
namespace: example-ns
|
|
name: example-vds
|
|
annotations:
|
|
myapp.config/postgres-host: postgres-postgresql.postgres.svc.cluster.local:5432
|
|
spec:
|
|
destination:
|
|
create: true
|
|
name: app-secret
|
|
transformation:
|
|
transformationRefs:
|
|
- name: vso-templates
|
|
path: creds/dev-postgres
|
|
```
|
|
|
|
The resulting Kubernetes secret includes a single key named `url`, with a valid Postgres connection
|
|
URL as its value
|
|
|
|
```yaml
|
|
url: postgresql://v-postgres-user:XUpah-password@postgres-postgresql.postgres.svc.cluster.local:5432/postgres?sslmode=disable
|
|
```
|
|
|
|
### Template functions
|
|
|
|
All template functions are provided by the [sprig](http://masterminds.github.io/sprig) library. Some common functions are mentioned below.
|
|
For the complete list of functions see [allowedSprigFuncs](https://github.com/hashicorp/vault-secrets-operator/blob/main/template/funcs.go#L26)
|
|
|
|
### String functions
|
|
|
|
`trim` removes any leading or trailing whitespaces from the input:
|
|
```
|
|
trim " host " -> `host`
|
|
```
|
|
|
|
### Encoding functions
|
|
|
|
`b64enc` base64 encodes an input value
|
|
```
|
|
b64enc "host" -> `aG9zdAo=`
|
|
```
|
|
|
|
`b64dec` base64 decodes an input value
|
|
```
|
|
b64dec "aG9zdAo=" -> `host`
|
|
```
|
|
|
|
### Map functions
|
|
|
|
`get` retrieves a value from a `map` input:
|
|
```
|
|
get .Secrets "baz" -> `qux`
|
|
```
|
|
|
|
Given a nested `map` input:
|
|
|
|
```json
|
|
{
|
|
"foo": {
|
|
"bar": "baz",
|
|
"quz": "quux"
|
|
}
|
|
}
|
|
```
|
|
|
|
`get` can retrieve a specific value:
|
|
```
|
|
get (get .Secrets "foo") "bar" -> `baz`
|
|
```
|
|
|
|
`dig` can also retrieve a specific value, or return a default if any of the keys
|
|
are not found:
|
|
```
|
|
dig "foo" "quz" "<not found>" .Secrets -> `quux`
|
|
|
|
dig "foo" "nux" "<not found>" .Secrets -> `<not found>`
|
|
```
|
|
|
|
## Related API references
|
|
|
|
- [Transformation](/vault/docs/platform/k8s/vso/api-reference#transformation)
|
|
- [HCPVaultSecretsApp](/vault/docs/platform/k8s/vso/api-reference#hcpvaultsecretsapp)
|
|
- [VaultDynamicSecret](/vault/docs/platform/k8s/vso/api-reference#vaultdynamicsecret)
|
|
- [VaultPKISecret](/vault/docs/platform/k8s/vso/api-reference#vaultpkisecret)
|
|
- [VaultStaticSecret](/vault/docs/platform/k8s/vso/api-reference#vaultstaticsecret)
|
|
- [SecretTransformation](/vault/docs/platform/k8s/vso/api-reference#secrettransformation)
|