mirror of
https://github.com/hashicorp/vault.git
synced 2025-08-06 06:37:02 +02:00
Suggested edits for identity doc updates (#29339)
* Identity dupe resolution guide first draft * initial edits * save progress * save changes * add script to find template policies * save progress * save work * push latest updates * missed one * Update website/content/docs/upgrading/deduplication/entity-group.mdx Co-authored-by: Paul Banks <pbanks@hashicorp.com> * apply additional feedback * apply feedback --------- Co-authored-by: Paul Banks <pbanks@hashicorp.com>
This commit is contained in:
Sarah Chavis
GitHub
parent
d127c4de93
commit
1b5260d696
@ -0,0 +1,364 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Resolve ACL policy templates
|
||||
description: >-
|
||||
Resolve templated ACL behavior for deduplicated entities and groups.
|
||||
---
|
||||
|
||||
# Resolve deduplication impact on ACL policy templates
|
||||
|
||||
Fix templated ACL policy behavior for entities and groups renamed during
|
||||
identity deduplication.
|
||||
|
||||
<Tip title="Assumptions">
|
||||
|
||||
- You are running Vault 1.19 or later.
|
||||
- You have [deduplication **renaming** targets in your system logs](/vault/docs/upgrading/identity-deduplication).
|
||||
- You have admin permission on the relevant Vault server or cluster.
|
||||
|
||||
</Tip>
|
||||
|
||||
|
||||
## How renaming affect ACL policies
|
||||
|
||||
Operators use
|
||||
[policy templating](/vault/docs/concepts/policies#templated-policies) to specify
|
||||
permissions that dynamically resolve based on specific properties of an
|
||||
authenticated entity. During deduplication, Vault renames entities or groups
|
||||
with duplicate identities by appending a universally unique identifier (UUID) to
|
||||
all but one of the entities to avoid future duplication.
|
||||
|
||||
If a templated ACL policy relies on an entity or group name, renaming duplicate
|
||||
identities could inadvertently change the meaning of the policy. In practice,
|
||||
identity deduplication is unlikely to grant unintended access to an existing
|
||||
resource since the renamed identity contains a UUID that should
|
||||
not collide with other resource paths. But the rename might **remove** access to
|
||||
existing resources after deduplication if the associated resource policy
|
||||
references the old entity or group name.
|
||||
|
||||
For example, assume you have the following templated ACL policy that grants
|
||||
users access to a `kv` plugin:
|
||||
|
||||
```hcl
|
||||
path "kv/users/{{identity.entity.name}}/*" {
|
||||
capabilities = ["read", "create", "update"]
|
||||
}
|
||||
```
|
||||
|
||||
The policy grants a user entity named `janine` permission to read and write
|
||||
key-value pairs to the `kv` plugin on the path `kv/users/janine/`. If there are
|
||||
duplicate entities named `janine`, Vault renames all but one of entities
|
||||
to `janine-<UUID>` during deduplication.
|
||||
|
||||
The ACL policy now gives the renamed entity permission to read and write
|
||||
While the renamed entity can still access the `kv` plugin, the change in
|
||||
no longer grants access to path `kv/users/janine/`. As a result, the user can no
|
||||
longer access their data stored under `kv/users/janine/`.
|
||||
|
||||
|
||||
## Find affected ACL policies
|
||||
|
||||
Deduplication has the potential to break templated ACL policies that rely on
|
||||
entity or group name keys:
|
||||
|
||||
Template key | Description
|
||||
------------------------------------------------------------- | ----------------
|
||||
`identity.entity.name` | Entity name
|
||||
`identity.entity.aliases.<mount accessor>.name` | Entity alias name for the given mount
|
||||
`identity.groups.ids.<group id>.name` | Group name for the given group ID
|
||||
`identity.groups.names.<group name>.id` | Group ID for the given group name
|
||||
`identity.groups.names.<group name>.metadata.<metadata key>` | Metadata for the given group name and key
|
||||
|
||||
You can use the following `bash` script to check for templated policies using
|
||||
entity or group names:
|
||||
|
||||
<Tabs>
|
||||
|
||||
<Tab heading="CLI" group="cli">
|
||||
|
||||
```bash
|
||||
policy_list=$(vault policy list)
|
||||
|
||||
for policy in ${policy_list} ; do
|
||||
|
||||
# Get the policy details
|
||||
policy_details=$(vault policy read ${policy})
|
||||
|
||||
# Check for a name based template key
|
||||
if echo "${policy_details}" | grep -q "\.name" ; then
|
||||
echo ""
|
||||
echo "Policy name: ${policy}"
|
||||
fi
|
||||
echo "${policy_details}" |
|
||||
tr -s ' ' '\n' |
|
||||
grep "\.name" --label=" template key" -H
|
||||
done
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab heading="API" group="api">
|
||||
|
||||
```bash
|
||||
policy_list=$(
|
||||
curl \
|
||||
--silent \
|
||||
--request GET \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy \
|
||||
| jq .data.policies | jq -c '.[]' | tr -d '"'
|
||||
)
|
||||
|
||||
for policy in ${policy_list} ; do
|
||||
|
||||
# Get the policy details
|
||||
policy_details=$(
|
||||
curl \
|
||||
--silent \
|
||||
--request GET \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy/${policy} \
|
||||
| jq '.data.rules' | sed 's/\\"/"/g' | sed 's/\\n/ /g'
|
||||
)
|
||||
|
||||
# Check for a name based template key
|
||||
if echo "${policy_details}" | grep -q "\.name" ; then
|
||||
echo ""
|
||||
echo "Policy name: ${policy}"
|
||||
fi
|
||||
echo "${policy_details}" |
|
||||
tr -s ' ' '\n' |
|
||||
grep "\.name" --label=" template key" -H
|
||||
done
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
</Tabs>
|
||||
|
||||
Compare the returned list of policies and template keys against the list of
|
||||
deduplication targets identified in your system logs to determine if the policy
|
||||
will break after duplication.
|
||||
|
||||
If you identify policies that may break during deduplication, consult with
|
||||
the relevant teams to determine if the policies are still used and if you should
|
||||
address the renaming problem before or after deduplication.
|
||||
|
||||
If you find policies that need remediation, there are two ways to fix user
|
||||
access:
|
||||
|
||||
1. Update relevant templated ACL policies.
|
||||
1. Move the affected resource.
|
||||
|
||||
|
||||
## Solution 1: Update relevant ACL policies
|
||||
|
||||
The easiest way to deal with broken ACL policy templates due to deduplication is
|
||||
to define new policies or add rules to existing policies that explicitly grants
|
||||
access to previous resources for the affected entities.
|
||||
|
||||
If you have a large number of duplicate identities, it may be easier to
|
||||
pre-populate a custom metadata field (e.g., `prev_name`) for the renamed
|
||||
entities. Then create a single policy that uses the custom metadata field
|
||||
instead of the entity name and attach that policy to the renamed entities.
|
||||
|
||||
<Tabs>
|
||||
|
||||
<Tab heading="CLI" group="cli">
|
||||
|
||||
Assume you have a file called `rename-targets.txt` with a list of entity names
|
||||
currently attached to a templated ACL policy, which will no longer work after
|
||||
deduplication.
|
||||
|
||||
1. Use `vault write` and the
|
||||
[`{namespace}/identity/entity/name/{name}`](/vault/api-docs/secret/identity/entity#create-update-entity-by-name)
|
||||
endpoint to update your target entries with a custom metadata field called
|
||||
`old_name` set to the **current** entity name:
|
||||
|
||||
```bash
|
||||
while read entity_name; do
|
||||
|
||||
if [[ "" = "${entity_name}" ]] ; then continue ; fi
|
||||
|
||||
# Create a payload file with the new metadata field
|
||||
echo -n '{"metadata": { "old_name": "'${entity_name}'"}}' > ./metadata.json
|
||||
|
||||
# Save the metadata to the entity
|
||||
vault write /identity/entity/name/${entity_name} @metadata.json
|
||||
|
||||
done < rename-targets.txt
|
||||
```
|
||||
|
||||
1. Create a new templatized ACL policy file that replaces the entity name
|
||||
reference with a reference to the metadata field. For example:
|
||||
|
||||
```hcl
|
||||
path "kv/users/{{identity.entity.metadata.old_name}}/*" {
|
||||
capabilities = ["read", "create", "update"]
|
||||
}
|
||||
```
|
||||
|
||||
1. Use `vault policy write` to create a new ACL policy with the policy
|
||||
definition file:
|
||||
|
||||
```shell-session
|
||||
$ vault policy write <policy_name> <path_to_policy_file>
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
<CodeBlockConfig hideClipboard="true">
|
||||
|
||||
```shell-session
|
||||
$ vault policy write "kv-access-preservation" ./dedupe-policy.hcl
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
1. Use `vault read` with the
|
||||
[`{namespace}/identity/entity/name/{name}`](/vault/api-docs/secret/identity/entity#create-update-entity-by-name)
|
||||
path to read in the existing policy assignments and add the new policy to the
|
||||
target entities. For example:
|
||||
|
||||
```bash
|
||||
policy_name="<policy_name>"
|
||||
while read entity_name; do
|
||||
|
||||
if [[ "" = "${entity_name}" ]] ; then continue ; fi
|
||||
|
||||
# Create a payload file with new policy added to any existing policy
|
||||
# assignments
|
||||
vault read \
|
||||
-field policies \
|
||||
-format json \
|
||||
/identity/entity/name/${entity_name} \
|
||||
| jq ". + [\"${policy_name}\"] | {policies: .}" > policy_update.json
|
||||
|
||||
# Update the policy assignment for the entity
|
||||
vault write /identity/entity/name/${entity_name} @policy_update.json
|
||||
|
||||
done < rename-targets.txt
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab heading="API" group="api">
|
||||
|
||||
Assume you have a file called `rename-targets.txt` with a list of entity names
|
||||
currently attached to a templated ACL policy, which will no longer work after
|
||||
deduplication.
|
||||
|
||||
1. Use the
|
||||
[`{namespace}/identity/entity/name/{name}`](/vault/api-docs/secret/identity/entity#create-update-entity-by-name)
|
||||
endpoint to update your target entries with a custom metadata field called
|
||||
`old_name` set to the **current** entity name:
|
||||
|
||||
```bash
|
||||
while read entity_name; do
|
||||
|
||||
if [[ "" = "${entity_name}" ]] ; then continue ; fi
|
||||
|
||||
# Create a payload file with the new metadata
|
||||
echo -n '{"metadata": { "old_name": "'${entity_name}'"}}' > ./metadata.json
|
||||
|
||||
# Save the metadata to the entity
|
||||
curl \
|
||||
--request POST \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
--data @./metadata.json \
|
||||
${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/identity/entity/name/${entity_name}
|
||||
|
||||
done < rename-targets.txt
|
||||
```
|
||||
|
||||
1. Create a new templatized ACL policy file that replaces the entity name
|
||||
reference with a reference to the metadata field. For example:
|
||||
|
||||
```hcl
|
||||
path "kv/users/{{identity.entity.metadata.old_name}}/*" {
|
||||
capabilities = ["read", "create", "update"]
|
||||
}
|
||||
```
|
||||
|
||||
1. Escape your policy file and make a `POST` call to the
|
||||
[`{namespace}/sys/policy/{policy_name}`](/vault/api-docs/system/policy#create-update-policy)
|
||||
with your policy details:
|
||||
|
||||
```shell-session
|
||||
$ jq -Rs '{ "policy": . | gsub("[\\r\\n\\t]"; "") }' <path_to_policy_file> |
|
||||
curl \
|
||||
--request POST \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
"$(</dev/stdin)" \
|
||||
${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy/<policy_name>
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
<CodeBlockConfig hideClipboard="true">
|
||||
|
||||
```shell-session
|
||||
$ jq -Rs '{ "policy": . | gsub("[\\r\\n\\t]"; "") }' ./dedupe-policy.hcl |
|
||||
curl \
|
||||
--request POST \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
--data "$(</dev/stdin)" \
|
||||
${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/sys/policy/kv-access-preservation
|
||||
```
|
||||
</CodeBlockConfig>
|
||||
|
||||
`/sys/mounts/{plugin_mount_path}` does not return data on success.
|
||||
|
||||
|
||||
1. Make a `GET` call to the
|
||||
[`{namespace}/identity/entity/name/{name}`](/vault/api-docs/secret/identity/entity#create-update-entity-by-name)
|
||||
endpoint to read in the existing policy assignments and add the new policy to the
|
||||
target entities. For example:
|
||||
|
||||
```bash
|
||||
policy_name="<policy_name>"
|
||||
while read entity_name; do
|
||||
|
||||
if [[ "" = "${entity_name}" ]] ; then continue ; fi
|
||||
|
||||
# Create a payload file with new policy added to any existing policy assignments
|
||||
curl \
|
||||
--request GET \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/identity/entity/name/${entity_name} \
|
||||
| jq ".data.policies + [\"${policy_name}\"] | {policies: .}" > policy_update.json
|
||||
|
||||
# Update the policy assignment for the entity
|
||||
curl \
|
||||
--request POST \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
--data @policy_update.json \
|
||||
${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/identity/entity/name/${entity_name}
|
||||
|
||||
done < rename-targets.txt
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
</Tabs>
|
||||
|
||||
|
||||
## Solution 2: Move the affected resource
|
||||
|
||||
Some resources are immovable and others are difficult to move. If you decide you
|
||||
can, and want, to move resources affected by the renaming:
|
||||
|
||||
1. Inventory the relevant resources and the associated mount paths.
|
||||
1. Determine which resources you will migrate before activating the
|
||||
deduplication flag.
|
||||
- Resources migrating **before** deduplication will be unavailable on the
|
||||
old and new paths until **deduplication** completes.
|
||||
- Resources migrating **after** deduplication will be unavailable until
|
||||
**migration** completes.
|
||||
1. Move any resources flagged for migration before deduplication and alert the
|
||||
relevant users that the paths will remain unavailable until deduplication
|
||||
completes.
|
||||
1. [Enable the deduplication option for Vault](/vault/docs/upgrading/deduplication#dedupe-flag)
|
||||
1. Move any resources flagged for migration after deduplication and alert the
|
||||
relevant users that the paths will remain unavailable until migration
|
||||
completes and the changes fully propagate.
|
||||
221
website/content/docs/upgrading/deduplication/different-case.mdx
Normal file
221
website/content/docs/upgrading/deduplication/different-case.mdx
Normal file
@ -0,0 +1,221 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Resolve different-case entity alias duplicates
|
||||
description: >-
|
||||
Fix duplicate identities for entity aliases due to case differences.
|
||||
---
|
||||
|
||||
# Fix different-case entity alias duplicates
|
||||
|
||||
Fix duplicate identities for entity aliases due to case differences.
|
||||
|
||||
**You must review different-case entity alias duplicates before enabling forced
|
||||
identity deduplication due to the potential security risk of incorrectly merging
|
||||
distinct aliases**.
|
||||
|
||||
|
||||
<Tip title="Assumptions">
|
||||
|
||||
- You are running Vault 1.19 or later.
|
||||
- You have [deduplication **merge** targets in your system logs](/vault/docs/upgrading/identity-deduplication).
|
||||
- You have admin permission on the relevant Vault server or cluster.
|
||||
|
||||
</Tip>
|
||||
|
||||
|
||||
## Why duplicates happen
|
||||
|
||||
Historical bugs in Vault allowed authN mounts to create duplicate entries of the
|
||||
same entity alias if the aliases used difference cases. For example, if an
|
||||
external LDAP system returns unnormalized usernames or changes the normalization
|
||||
scheme over time. By default, Vault uses case-insensitive matching for the
|
||||
usernames and considers `username` and `uSeRnAmE` to be the same alias. But,
|
||||
past bugs have left some users with different-case duplicates in storage.
|
||||
|
||||
To avoid the security impact of incorrectly merging identities that do not
|
||||
represent the same user, Vault **tries** to honor past behavior by switching the
|
||||
identity engine to use case-sensitive matching mode. But case-sensitive matching
|
||||
is poorly supported, deviates from the foundational security assumptions Vault
|
||||
makes, and can lead to unexpected behavior.
|
||||
|
||||
## Example server log
|
||||
|
||||
The Vault system log provides warnings about duplicate aliases, including:
|
||||
|
||||
- the alias string
|
||||
- the associated mount path (`mount accessor`)
|
||||
- the alias ID (`id`)
|
||||
- the ID for the entity linked to the alias (`canonical_id`)
|
||||
|
||||
<CodeBlockConfig hideClipboard>
|
||||
|
||||
```text
|
||||
[WARN] identity: 2 different-case entity alias duplicates found (potential security risk)
|
||||
[WARN] identity: entity-alias "alias-case" with mount accessor "auth_userpass_34aca7ec" duplicates 1 others: id=df3568a4-3b65-4104-9481-1129ecbed72f canonical_id=5f013d99-a6c7-9a00-6ad5-4ad724b14f60 force_deduplication="would merge into entity 7da76b0d-fe9b-a125-3362-2a8ff055dcf8"
|
||||
[WARN] identity: entity-alias "alias-cAsE" with mount accessor "auth_userpass_34aca7ec" duplicates 1 others: id=2992253b-1e99-4e47-b6f9-afb0c7cedf7a canonical_id=7da76b0d-fe9b-a125-3362-2a8ff055dcf8 force_deduplication="would merge others into this entity"
|
||||
[WARN] identity: entity-alias "alias-case" with mount accessor "auth_userpass_a555989a" duplicates 1 others: id=0e4bd46e-a868-4dd4-a34a-cb73f097e3a5 canonical_id=1ce06951-e2fd-0923-8ae0-a0e2c6c2378b force_deduplication="would merge into entity 37f41fcf-5e15-f13f-248d-9fa8405b1ecd"
|
||||
[WARN] identity: entity-alias "alias-cAsE" with mount accessor "auth_userpass_a555989a" duplicates 1 others: id=6cafe546-665c-4bd0-a4fd-5e699ba413c1 canonical_id=37f41fcf-5e15-f13f-248d-9fa8405b1ecd force_deduplication="would merge others into this entity"
|
||||
[WARN] identity: end of different-case entity-alias duplicates
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
When reviewing system logs for duplicate resolution, always consider the authN
|
||||
mount and namespace when grouping potential duplicates. For example, despite
|
||||
having identical, case-insensitive names in the example logs, the four aliases
|
||||
represent two distinct identities because they come from different authN mounts:
|
||||
|
||||
- `alias-case` and `alias-cAsE` are duplicates in the mount `auth_userpass_34aca7ec`
|
||||
and will merge under the alias with canonical ID `7da76b0d-fe9b-a125-3362-2a8ff055dcf8`
|
||||
during deduplication.
|
||||
- `alias-case` and `alias-cAsE` are duplicates in the mount `auth_userpass_a555989a`
|
||||
and will merge with the alias with canonical ID `37f41fcf-5e15-f13f-248d-9fa8405b1ecd`
|
||||
during deduplication.
|
||||
|
||||
|
||||
To resolve different-case duplicates, you must determine if the case difference
|
||||
is a **mergeable duplicate** or **unmergeable duplicate**:
|
||||
|
||||
- For mergeable duplicates, the case difference is **unintentional** and the
|
||||
duplicate entries represent the same logical entity. You can ignore mergeable
|
||||
duplicates and let Vault force-merge them for you to establish and enforce
|
||||
default identity matching behavior.
|
||||
|
||||
- For unmergeable duplicates, the case difference is **deliberate** and the
|
||||
source identity provider **intended** to distinguish between different
|
||||
entities using case. For example, `Alice` and `alice` are different entities
|
||||
that correspond to different humans with distinct permission sets. **You must
|
||||
resolve unmergeable duplicates before activating the deduplication flag.**
|
||||
|
||||
|
||||
|
||||
## Case 1: Resolving mergeable duplicates
|
||||
|
||||
Vault will force-merge duplicates once you enable forced identity deduplication
|
||||
with the `force-identity-deduplication` flag. Vault also merges any policies
|
||||
attached to the duplicates so the final entry has the union of all permissions
|
||||
granted to the duplicates.
|
||||
|
||||
We recommend confirming the expected merge behavior **before** enabling
|
||||
deduplication by reviewing the `force_deduplication` label in the log line to:
|
||||
|
||||
1. confirm the identified duplicates are all mergeable.
|
||||
1. confirm permissions granted by the unified policy are appropriate for the
|
||||
identity.
|
||||
|
||||
After deduplication, users can continue to log in using any case combination of
|
||||
their username, but those logins will map to a single entity gated by the same
|
||||
policies per the expected, default Vault behavior.
|
||||
|
||||
|
||||
## Case 2: Resolving unmergeable duplicates
|
||||
|
||||
<Warning>
|
||||
|
||||
We strongly recommend resolving duplicates and moving away from case-sensitive
|
||||
usernames as soon as possible to restore the security model supported by Vault.
|
||||
|
||||
Vault does not officially support case-sensitive names because it carries
|
||||
significant security risks. And unresolved duplicates could merge at any time
|
||||
due to unrelated changes in storage such as when deleting related entities or
|
||||
groups.
|
||||
|
||||
</Warning>
|
||||
|
||||
When different-case variations of an alias name actually represent different
|
||||
logical users or entities in an external system, investigate each alias before
|
||||
resolving duplicates. You **must** resolve the duplicates on each mount before
|
||||
enabling forced identity deduplication.
|
||||
|
||||
**If you confirm you need to keep the duplicated aliases**, you will need to
|
||||
reconfigure the authN mount or external service to stop creating case-sensitive
|
||||
aliases. For example, if the authN plugin allows it, you could update the plugin
|
||||
configuration to append unique IDs to alias names to differentiate between
|
||||
similar usernames.
|
||||
|
||||
Regardless of how you choose to address unmergeable duplicates, be mindful of
|
||||
how the modified behavior may disrupt users as you roll out the change.
|
||||
|
||||
<Note>
|
||||
|
||||
We are not aware of specific integrations where changing mount or external
|
||||
service behaviors is necessary and expect those situations to be rare. If you
|
||||
determine your deployment requires changes to authN mounts or external services,
|
||||
we encourage Vault Enterprise customers to work with HashiCorp support staff to
|
||||
determine the best strategy and implementation.
|
||||
|
||||
</Note>
|
||||
|
||||
**If you can confirm you do not need the duplicated alias, or the entire auth
|
||||
mount**, delete all but one of the entity aliases on each mount using the ID
|
||||
noted in the log line with `id=`.
|
||||
|
||||
<Tabs>
|
||||
|
||||
<Tab heading="CLI" group="cli">
|
||||
|
||||
Use `vault delete` with the `/identity/entity-alias/id/{id}` path to delete the
|
||||
duplicate identity:
|
||||
|
||||
```shell-session
|
||||
$ vault delete /identity/entity-alias/id/<id>
|
||||
```
|
||||
|
||||
For example, if one of the duplicate entity log entries includes
|
||||
`id=df3568a4-3b65-4104-9481-1129ecbed72f`:
|
||||
|
||||
<CodeBlockConfig hideClipboard="true">
|
||||
|
||||
```shell-session
|
||||
$ vault delete /identity/entity-alias/id/df3568a4-3b65-4104-9481-1129ecbed72f
|
||||
|
||||
Success! Data deleted (if it existed) at: identity/entity-alias/id/df3568a4-3b65-4104-9481-1129ecbed72f
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab heading="API" group="api">
|
||||
|
||||
Call the [`/identity/entity-alias/id/{id}`](/vault/api-docs/secret/identity/entity-alias#delete-entity-alias-by-id)
|
||||
endpoint to delete the duplicate entry:
|
||||
|
||||
```shell-session
|
||||
$ curl \
|
||||
--request DELETE \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
${VAULT_ADDR}/v1/identity/entity-alias/id/<id>
|
||||
```
|
||||
|
||||
For example, if one of the duplicate entity log entries includes
|
||||
`id=df3568a4-3b65-4104-9481-1129ecbed72f`:
|
||||
|
||||
<CodeBlockConfig hideClipboard="true">
|
||||
|
||||
```shell-session
|
||||
$ curl \
|
||||
--request DELETE \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
${VAULT_ADDR}/v1/identity/entity-alias/id/df3568a4-3b65-4104-9481-1129ecbed72f | jq
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
</Tab>
|
||||
|
||||
</Tabs>
|
||||
|
||||
After you resolve the duplicates by deleting them or reconfiguring the relevant
|
||||
services:
|
||||
|
||||
1. If the node is a standby, restart the node manually.
|
||||
|
||||
1. Recheck the system logs for the current node to confirm the duplicates no
|
||||
longer appear in the log during unseal.
|
||||
|
||||
|
||||
## Next steps
|
||||
|
||||
Once you confirm any remaining duplicates can be force-merged safely, you can
|
||||
[enable forced identity deduplication](/vault/docs/upgrading/deduplication#dedupe-flag).
|
||||
136
website/content/docs/upgrading/deduplication/entity-group.mdx
Normal file
136
website/content/docs/upgrading/deduplication/entity-group.mdx
Normal file
@ -0,0 +1,136 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Fix entity and group duplicates
|
||||
description: >-
|
||||
Fix duplicate identities for Vault entities and groups
|
||||
---
|
||||
|
||||
# Fix entity and group duplicates
|
||||
|
||||
Fix duplicate identities for entities and groups targeted for renaming.
|
||||
|
||||
<Tip title="Assumptions">
|
||||
|
||||
- You are running Vault 1.19 or later.
|
||||
- You have [deduplication **renaming** targets in your system logs](/vault/docs/upgrading/identity-deduplication).
|
||||
- You have admin permission on the relevant Vault server or cluster.
|
||||
|
||||
</Tip>
|
||||
|
||||
|
||||
## Why duplicates happen
|
||||
|
||||
Historical bugs in Vault allowed authN mounts to create duplicate entries of the
|
||||
same entity or group name if the names used difference cases. By default, Vault
|
||||
uses case-insensitive matching for names and considers `bob` and `BOB` to be the
|
||||
same name. But, past bugs have allowed for identical or different-case
|
||||
duplicates in entity and group names.
|
||||
|
||||
Duplicate entity and group names can cause unexpected behavior and make working
|
||||
with Vault more difficult:
|
||||
|
||||
- Fetching entities/groups by name returns the first identity found. Depending
|
||||
on your Vault version, the identity returned can vary depending on the server
|
||||
handling the request and when the last seal/unseal event occurred.
|
||||
- Deleting entities/groups by name is not guaranteed to delete all duplicates.
|
||||
- All lookup requests **must** use ID to find the right identity.
|
||||
individual identity.
|
||||
|
||||
## Example server log
|
||||
|
||||
The Vault system log provides warnings about duplicate entities and groups,
|
||||
including:
|
||||
|
||||
- the entity/group name
|
||||
- the associated namespace (`namespace ID`)
|
||||
- the entity/group ID (`id`)
|
||||
- the expected deduplication action (`force_deduplication`)
|
||||
|
||||
<CodeBlockConfig hideClipboard>
|
||||
|
||||
```text
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: 2 entity duplicates found
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "entity-cAsE" with namespace ID "root" duplicates 2 others: id=2562b42d-f603-ac6b-2591-8a20dd050897 force_deduplication="would not rename"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "entity-case" with namespace ID "root" duplicates 2 others: id=290a643d-6043-da5e-943f-3a3d09e4ecbd force_deduplication="would rename to entity-case-290a643d-6043-da5e-943f-3a3d09e4ecbd"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "entity-case" with namespace ID "root" duplicates 2 others: id=b0141be5-3f03-a1c7-a57b-02f045d04426 force_deduplication="would rename to entity-case-b0141be5-3f03-a1c7-a57b-02f045d04426"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "entity-cAsE" with namespace ID "sYMXY" duplicates 2 others: id=95f0743b-a1d5-26da-b4ef-a50490da0787 force_deduplication="would not rename"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "entity-case" with namespace ID "sYMXY" duplicates 2 others: id=9d3be96f-490a-9625-3118-896bd2b3a5f3 force_deduplication="would rename to entity-case-9d3be96f-490a-9625-3118-896bd2b3a5f3"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "entity-case" with namespace ID "sYMXY" duplicates 2 others: id=d82231d2-3716-6b9c-e80d-7d09c0409739 force_deduplication="would rename to entity-case-d82231d2-3716-6b9c-e80d-7d09c0409739"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: end of entity duplicates
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: 2 group duplicates found
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: group "group-case" with namespace ID "root" duplicates 2 others: id=8ad26e0c-8cf6-5b67-7c77-6571fa374f34 force_deduplication="would not rename"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: group "group-cAsE" with namespace ID "root" duplicates 2 others: id=9fe86ea0-f80c-1131-5be1-1d6e3b70237f force_deduplication="would rename to group-cAsE-9fe86ea0-f80c-1131-5be1-1d6e3b70237f"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: group "group-case" with namespace ID "root" duplicates 2 others: id=32dd070c-c1f8-c796-9a71-15887014b813 force_deduplication="would rename to group-case-32dd070c-c1f8-c796-9a71-15887014b813"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: group "group-case" with namespace ID "sYMXY" duplicates 2 others: id=8aaeff7e-7343-c883-1e0c-c5c9968f75a5 force_deduplication="would not rename"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: group "group-case" with namespace ID "sYMXY" duplicates 2 others: id=f11277b3-d985-4d72-d2e9-9c8c6c0db02c force_deduplication="would rename to group-case-f11277b3-d985-4d72-d2e9-9c8c6c0db02c"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: group "group-cAsE" with namespace ID "sYMXY" duplicates 2 others: id=7c753d07-b0d9-e13b-6184-48247b8f7504 force_deduplication="would rename to group-cAsE-07c753d07-b0d9-e13b-6184-48247b8f7504"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: end of group duplicates
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
Duplicate entities and groups might be exact matches or differ in case (for
|
||||
example, `Admin` and `admin`). But when reviewing system logs for duplicate
|
||||
resolution, always consider the associated namespace when grouping potential
|
||||
duplicates. For example, despite having identical, case-insensitive names in the
|
||||
example logs, Vault only targets 8 for renaming because the names belong to
|
||||
different namespaces:
|
||||
|
||||
Identity type | Old name | New name
|
||||
------------- | ------------------- | --------
|
||||
Entity | `root/entity-cAsE` | Unchanged
|
||||
Entity | `root/entity-case` | `root/entity-case-290a643d-6043-da5e-943f-3a3d09e4ecbd`
|
||||
Entity | `root/entity-case` | `root/entity-case-b0141be5-3f03-a1c7-a57b-02f045d04426`
|
||||
Entity | `sYMXY/entity-cAsE` | Unchanged
|
||||
Entity | `sYMXY/entity-case` | `sYMXY/entity-case-9d3be96f-490a-9625-3118-896bd2b3a5f3`
|
||||
Entity | `sYMXY/entity-case` | `sYMXY/entity-case-d82231d2-3716-6b9c-e80d-7d09c0409739`
|
||||
Group | `root/group-case` | Unchanged
|
||||
Group | `root/group-cAsE` | `group-cAsE-9fe86ea0-f80c-1131-5be1-1d6e3b70237f`
|
||||
Group | `root/group-case` | `group-case-32dd070c-c1f8-c796-9a71-15887014b813`
|
||||
Group | `sYMXY/group-case` | Unchanged
|
||||
Group | `sYMXY/group-case` | `group-case-f11277b3-d985-4d72-d2e9-9c8c6c0db02c`
|
||||
Group | `sYMXY/group-cAsE` | `group-cAsE-07c753d07-b0d9-e13b-6184-48247b8f7504`
|
||||
|
||||
|
||||
The automatic renaming process is security safe and preserves existing data.
|
||||
Renaming **does not** grant new permissions to existing tokens and **does not**
|
||||
delete data, so you can manually roll back the change if necessary.
|
||||
|
||||
|
||||
## Resolving entity and group duplicates
|
||||
|
||||
In most cases, renaming preserves resource access such that logins and
|
||||
permissions remain unaffected. But, there are two edge cases you may need to
|
||||
address before deduplication: **templated policies** and **external references**.
|
||||
|
||||
- If you use templated ACL policies that reference an entity or group name as
|
||||
part of the resource path, deduplication may affect access to those resources.
|
||||
|
||||
- If you use Terraform to manage Vault resources or have other systems outside
|
||||
Vault that reference entity or group names, those references will break when the
|
||||
deduplication process renames those entities/groups.
|
||||
|
||||
|
||||
For each duplicate identified by the logs, you have 3 options:
|
||||
|
||||
1. **If you know you do not need or use the renaming target**, you can use
|
||||
the
|
||||
[`/identity/entity/id/{id}`](/vault/api-docs/secret/identity/entity#delete-entity-by-id) and
|
||||
[`/identity/group/id/{id}`](/vault/api-docs/secret/identity/group#delete-group-by-id)
|
||||
endpoints to delete the identities by ID.
|
||||
|
||||
1. **If you know the risk to templated policies or external references is low or
|
||||
nonexistent**, you can opt to ignore the duplicates and address issues
|
||||
if/when they occur.
|
||||
|
||||
|
||||
1. **If you cannot confirm the risk to templated policies or external references**,
|
||||
review the guidance in the following troubleshooting guide:
|
||||
- [Resolve deduplication impact on ACL policy templates](/vault/docs/upgrading/deduplication/acl-policy-templates)
|
||||
- [Resolve deduplication impact on Terraform resource references](/vault/docs/upgrading/deduplication/external-refs)
|
||||
|
||||
## Next steps
|
||||
|
||||
Once you are comfortable that all the entity and group duplicates are properly
|
||||
addressed, you can
|
||||
[enable forced identity deduplication](/vault/docs/upgrading/deduplication#dedupe-flag).
|
||||
225
website/content/docs/upgrading/deduplication/index.mdx
Normal file
225
website/content/docs/upgrading/deduplication/index.mdx
Normal file
@ -0,0 +1,225 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Resolve duplicate identities
|
||||
description: >-
|
||||
Find duplicate identities in your Vault cluster and safely resolve them.
|
||||
---
|
||||
|
||||
# Find and resolve duplicate Vault identities
|
||||
|
||||
Bugs in Vault versions before 1.19 might lead to duplicate
|
||||
[identities](/vault/docs/concepts/identity) for entities, aliases, and groups.
|
||||
Duplicate identities in the persistent storage of a Vault cluster can cause
|
||||
unexpected behavior as duplicate identities are outside typical expectations and
|
||||
test scenarios for Vault.
|
||||
|
||||
Vault server logs include information to help you identify when duplicate
|
||||
identities exist in a given cluster. We strongly recommend identifying
|
||||
and resolving duplicates as soon as possible to return your Vault clusters to
|
||||
normal, supported behavior.
|
||||
|
||||
## Before you start
|
||||
|
||||
- **You must have Vault 1.19 or later running on all clusters**. Vault only logs
|
||||
deduplication details once you upgrade to 1.19+.
|
||||
- **You must have admin permissions for the Vault cluster**.
|
||||
|
||||
## Step 1: Look for duplicates
|
||||
|
||||
To identify duplicates in server logs:
|
||||
|
||||
1. Look for `core: post-unseal setup starting` to find the last unseal operation
|
||||
in the system logs of the active node. For example:
|
||||
|
||||
```text
|
||||
[INFO] core: post-unseal setup starting
|
||||
...<check here> ...
|
||||
[INFO] core: post-unseal setup complete
|
||||
```
|
||||
|
||||
1. Check for `DUPLICATES DETECTED` between the `setup starting` and
|
||||
`setup complete` entries in the system log. For example:
|
||||
|
||||
```text
|
||||
[WARN] identity: DUPLICATES DETECTED, see following logs for details [...]
|
||||
```
|
||||
|
||||
If you do not see entries for `DUPLICATES DETECTED`, the current cluster is
|
||||
clean and you can move to the next cluster. If you confirm none of the logs
|
||||
have duplicates, you can jump to [Step #4](#dedupe-flag).
|
||||
|
||||
<Note>
|
||||
|
||||
If you use replication, repeat the duplication check on all primary clusters and
|
||||
all **performance replication** (PR) secondary clusters. PR secondary clusters
|
||||
handle client requests and may have additional, **local** duplicates.
|
||||
|
||||
Disaster recovery (DR) secondaries do not handle client requests, so you do not
|
||||
need to check DR secondary clusters separately.
|
||||
|
||||
</Note>
|
||||
|
||||
## Step 2: Find deduplication targets
|
||||
|
||||
If you find `DUPLICATES DETECTED` in the system logs, create a list of same-case
|
||||
aliases and renaming targets from the system log to make working through
|
||||
duplicate resolution easier. If you have a large number of deduplication targets,
|
||||
you can use the following script to pull out the relevant items and create
|
||||
four files:
|
||||
|
||||
- **`unseal-process.log`** - section of the system log specific to the unseal
|
||||
process.
|
||||
- **`merge-details.txt`** - same-case identities Vault will auto-merge during
|
||||
deduplication.
|
||||
- **`rename-targets.txt`** - entities and group names in the form
|
||||
`namespace/name` Vault will rename during deduplication.
|
||||
- **`rename-details.txt`** - full rename details for rename targets.
|
||||
|
||||
```bash
|
||||
|
||||
declare -A identity_key=(
|
||||
['entity_alias']="identity: entity-alias"
|
||||
['entity_rename']="identity: entity"
|
||||
['group_rename']="identity: group"
|
||||
)
|
||||
|
||||
declare -A match_str=(
|
||||
['has_rename']="would rename to"
|
||||
['has_merge']="would merge"
|
||||
['rename']="s/.*would rename to \(.*\)\".*/\1/"
|
||||
['group_name']="s/.*identity: group \"\(.*\)\" with.*/\1/"
|
||||
['entity_name']="s/.*identity: entity \"\(.*\)\" with.*/\1/"
|
||||
['namespace']="s/.*namespace ID \"\(.*\)\" duplicates.*/\1/"
|
||||
)
|
||||
|
||||
declare -A rename_targets
|
||||
|
||||
# Grab the unseal portion of the log
|
||||
log_start="core: post-unseal setup starting"
|
||||
log_stop="core: post-unseal setup complete"
|
||||
sed "/${log_start}/,/${log_stop}/!d;/${log_stop}/q" ../short.log > unseal-process.log
|
||||
|
||||
while read line;
|
||||
do
|
||||
|
||||
# The log line relates to a renamed identity
|
||||
if [[ "${line}" == *"${match_str['has_rename']}"* ]] ; then
|
||||
|
||||
if [[ "${line}" == *"${identity_key['entity_rename']}"* ]] ; then
|
||||
type="${identity_key['entity_rename']}"
|
||||
name_match="${match_str['entity_name']}"
|
||||
fi
|
||||
if [[ "${line}" == *"${identity_key['group_rename']}"* ]] ; then
|
||||
type="${identity_key['group_rename']}"
|
||||
name_match="${match_str['group_name']}"
|
||||
fi
|
||||
|
||||
new=$(echo $line | sed -e "${match_str['rename']}")
|
||||
old=$(echo $line | sed -e "${name_match}")
|
||||
space=$(echo $line | sed -e "${match_str['namespace']}")
|
||||
rename_targets["${space}/${old}"]="[${type}] ${space}/${old} --> ${space}/${new}"
|
||||
|
||||
fi
|
||||
|
||||
# The line indicates a same-case merge operation
|
||||
if [[ "${line}" == *"${match_str['has_merge']}"* ]] ; then
|
||||
merge_details=$(echo $line | sed -e "${identity_key['entity_alias']}")
|
||||
echo ${merge_details} >> merge-details.txt
|
||||
fi
|
||||
|
||||
done < unseal-process.log
|
||||
|
||||
# Save the rename target information to files for further processing
|
||||
root_ns="root/"
|
||||
for name in "${!rename_targets[@]}"
|
||||
do
|
||||
echo ${name//"${root_ns}"/""} >> rename-targets.txt
|
||||
echo ${rename_targets["${name}"]} >> rename-details.txt
|
||||
done
|
||||
```
|
||||
|
||||
|
||||
## Step 3: Resolve any duplicates
|
||||
|
||||
If you **did not** find duplicates in the log, you can jump to
|
||||
[Step #5](#dedupe-flag).
|
||||
|
||||
If you **did** find duplicates in the logs:
|
||||
|
||||
1. Identify the duplication type for each entry in your server logs:
|
||||
- **In PR/DR deployments**: identify duplication types on your primary
|
||||
clusters.
|
||||
- **In PR deployments**: identify duplication types for **local aliases** on
|
||||
your performance secondaries. Local duplicates exist independently on the
|
||||
secondary cluster and require explicit resolution but follow the same
|
||||
process as resolving non-local different-case entity alias duplicates.
|
||||
|
||||
1. Follow the appropriate de-duplication steps based on the duplication type:
|
||||
- [Fix different-case entity alias duplicates](/vault/docs/upgrading/deduplication/different-case)
|
||||
- [Fix entity and group duplicates](/vault/docs/upgrading/deduplication/entity-group)
|
||||
|
||||
|
||||
## Step 4: Prepare for (possible) latency impacts
|
||||
|
||||
When you activate deduplication for a cluster, the cluster reloads the in-memory
|
||||
cache of all entities, aliases and groups. When deduplication replicates to
|
||||
performance replication secondaries and performance standby nodes, those nodes
|
||||
also pause and reload their identity system caches.
|
||||
|
||||
Vault remains unsealed during deduplication, but nodes may pause processing
|
||||
other requests until deduplication completes. On **moderately large** clusters,
|
||||
deduplication may take long enough to impact request latencies. On **large**
|
||||
clusters, deduplication may take longer than 30 seconds, which could cause some
|
||||
requests to timeout.
|
||||
|
||||
In general, deduplication should take less time than sealing and unsealing Vault
|
||||
and be less disruptive than a regular failover. And clusters that had duplicates
|
||||
should see their future `unseal` times reduced.
|
||||
|
||||
|
||||
## Step 5: Enforce identity de-duplication ((#dedupe-flag))
|
||||
|
||||
Once you resolve existing duplicates, you can enable the
|
||||
`force-identity-deduplication` activation flag on the primary cluster using the
|
||||
[activation flag API path](/vault/api-docs/system/activation-flags).
|
||||
|
||||
Activating feature flags is a one-time, one-way action. Once you activate a
|
||||
feature gated with a feature flag, you cannot un-activate the feature.
|
||||
|
||||
<Tabs>
|
||||
|
||||
<Tab heading="CLI" group="cli">
|
||||
|
||||
```shell-session
|
||||
vault write -f sys/activation-flags/force-identity-deduplication/activate
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab heading="API" group="api">
|
||||
|
||||
```shell-session
|
||||
$ curl \
|
||||
--request PUT \
|
||||
--header "X-Vault-Token: ${VAULT_TOKEN}" \
|
||||
${VAULT_ADDR}/v1/sys/activation-flags/force-identity-deduplication/activate
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
</Tabs>
|
||||
|
||||
You can track the start and end of deduplication on each node in the system
|
||||
by looking for `force-identity-deduplication activated` entries in the logs:
|
||||
|
||||
```
|
||||
INFO core: force-identity-deduplication activated, reloading identity store
|
||||
...
|
||||
INFO core: force-identity-deduplication activated, reloading identity store complete
|
||||
```
|
||||
|
||||
Going forward, Vault requires unique identities and re-runs the deduplication
|
||||
check as a part of the unseal process. You can review the impact on unseal time
|
||||
by reviewing system log timestamps. The difference between the log lines
|
||||
`core: post-unseal setup starting` and `core: post-unseal setup complete`
|
||||
indicates the total unseal time required for the cluster.
|
||||
160
website/content/docs/upgrading/deduplication/terraform-refs.mdx
Normal file
160
website/content/docs/upgrading/deduplication/terraform-refs.mdx
Normal file
@ -0,0 +1,160 @@
|
||||
---
|
||||
layout: docs
|
||||
page_title: Resolve Terraform config
|
||||
description: >-
|
||||
Fix external reference behavior for deduplicated entities and groups in
|
||||
Terraform config files.
|
||||
---
|
||||
|
||||
# Resolve deduplication impact on Terraform resource references
|
||||
|
||||
Fix external reference behavior in Terraform configuration files for entities
|
||||
and groups renamed during identity deduplication.
|
||||
|
||||
<Tip title="Assumptions">
|
||||
|
||||
- You are running Vault 1.19 or later.
|
||||
- You have [deduplication **renaming** targets in your system logs](/vault/docs/upgrading/identity-deduplication).
|
||||
- You have admin permission on the relevant Vault server or cluster.
|
||||
|
||||
</Tip>
|
||||
|
||||
|
||||
## How renaming affects external references
|
||||
|
||||
Renaming entities and groups can break references in Terraform (and other
|
||||
external services) when those reference refer directly to the entity or group by
|
||||
name. For example, assume you have a Terraform configuration file with named
|
||||
identity resources like the following:
|
||||
|
||||
<CodeBlockConfig highlight="11,16" hideClipboard="true">
|
||||
|
||||
```hcl
|
||||
terraform {
|
||||
required_providers {
|
||||
vault = {
|
||||
source = "hashicorp/vault"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "vault" {}
|
||||
|
||||
resource "vault_identity_entity" "BOB" {
|
||||
name = "BOB"
|
||||
policies = ["TEST"]
|
||||
}
|
||||
|
||||
resource "vault_identity_entity" "bob" {
|
||||
name = "bob"
|
||||
policies = ["test"]
|
||||
}
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
By default, Vault ignore case when matching identities, treats `BOB` and `bob`
|
||||
as the same name, and rejects the second resource as a duplicate. However, if
|
||||
your Vault cluster is running in a mode that allows both resource names due to
|
||||
historical issues, the resources might exist as separate entities.
|
||||
|
||||
If Vault identifies `bob` and `BOB` as duplicates during deduplication, it
|
||||
renames one of the identities `<name>-<uuid>`. After deduplication, Terraform
|
||||
tries to reapply the previous name for the related resource, but the in-place
|
||||
update fails because the existing resource now violates the case-insensitive
|
||||
name constraint on the Vault side.
|
||||
|
||||
For example:
|
||||
|
||||
<CodeBlockConfig highlight="2,3,11,21,28" hideClipboard="true">
|
||||
|
||||
```
|
||||
➜ tf_dupe_testing terraform apply
|
||||
vault_identity_entity.bob: Refreshing state... [id=e8c5e633-fe37-5a49-4a29-32e2643d03bd]
|
||||
vault_identity_entity.BOB: Refreshing state... [id=2577bc3f-67ab-dab7-93dc-e86f78194ff0]
|
||||
|
||||
Terraform used the selected providers to generate the following execution plan. Resource
|
||||
actions are indicated with the following symbols:
|
||||
~ update in-place
|
||||
|
||||
Terraform will perform the following actions:
|
||||
|
||||
# vault_identity_entity.bob will be updated in-place
|
||||
~ resource "vault_identity_entity" "bob" {
|
||||
+ external_policies = false
|
||||
id = "e8c5e633-fe37-5a49-4a29-32e2643d03bd"
|
||||
~ name = "bob-e8c5e633-fe37-5a49-4a29-32e2643d03bd" -> "bob"
|
||||
# (3 unchanged attributes hidden)
|
||||
}
|
||||
|
||||
...
|
||||
|
||||
vault_identity_entity.bob: Modifying... [id=e8c5e633-fe37-5a49-4a29-32e2643d03bd]
|
||||
╷
|
||||
│ Error: error updating IdentityEntity "e8c5e633-fe37-5a49-4a29-32e2643d03bd": Error making API request.
|
||||
│
|
||||
│ URL: PUT https://127.0.0.1:8200/v1/identity/entity/id/e8c5e633-fe37-5a49-4a29-32e2643d03bd
|
||||
│ Code: 400. Errors:
|
||||
│
|
||||
│ * entity name is already in use
|
||||
│
|
||||
│ with vault_identity_entity.bob,
|
||||
│ on main.tf line 17, in resource "vault_identity_entity" "bob":
|
||||
│ 17: resource "vault_identity_entity" "bob" {
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
|
||||
## Solution
|
||||
|
||||
The easiest way to deal with renamed entities and groups is to manually update the
|
||||
the associated resource in your Terraform configuration with the updated name
|
||||
before forcing deduplication.
|
||||
|
||||
<Tip>
|
||||
|
||||
Use the same process to identify and update target names for other external
|
||||
systems that reference an entity or group by name.
|
||||
|
||||
</Tip>
|
||||
|
||||
For example, if your system logs include lines like the following:
|
||||
|
||||
<CodeBlockConfig hideClipboard="true">
|
||||
|
||||
```text
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "bob" with namespace ID "admin" duplicates 1 others: id=8ad26e0c-8cf6-5b67-7c77-6571fa374881 force_deduplication="would not rename"
|
||||
2025-01-28T13:15:13.641-0800 [WARN] identity: entity "BOB" with namespace ID "admin" duplicates 1 others: id=9fe86ea0-f80c-1199-5ad1-1d01ab70237f force_deduplication="would rename to BOB-9fe86ea0-f80c-1199-5ad1-1d01ab70237f"
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
|
||||
You would update any resources associated with `BOB` in your Terraform
|
||||
configuration files. For example:
|
||||
|
||||
<CodeBlockConfig highlight="11,16" hideClipboard="true">
|
||||
|
||||
```hcl
|
||||
terraform {
|
||||
required_providers {
|
||||
vault = {
|
||||
source = "hashicorp/vault"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "vault" {}
|
||||
|
||||
resource "vault_identity_entity" "BOB-9fe86ea0-f80c-1199-5ad1-1d01ab70237f" {
|
||||
name = "BOB-9fe86ea0-f80c-1199-5ad1-1d01ab70237f"
|
||||
policies = ["TEST"]
|
||||
}
|
||||
|
||||
resource "vault_identity_entity" "bob" {
|
||||
name = "bob"
|
||||
policies = ["test"]
|
||||
}
|
||||
```
|
||||
|
||||
</CodeBlockConfig>
|
||||
@ -42,6 +42,24 @@ based on the table below.
|
||||
| CE | true | any value other than sha2-512 | An error is returned | Pure Ed25519 |
|
||||
| CE | true | sha2-512 | An error is returned (not supported on CE) | Pure Ed25519 |
|
||||
|
||||
### Identity system duplicate cleanup
|
||||
|
||||
**Users should review their server logs after upgrading to see if identity
|
||||
duplicates are reported.**
|
||||
|
||||
Vault 1.19.0 enables users impacted by historical identity duplicate bugs to
|
||||
manually trigger a one-time de-duplication process. This process restores the
|
||||
cluster to supported default behavior by resolving the duplicates.
|
||||
|
||||
Vault 1.19.0 also includes improved reporting in server logs to help diagnose
|
||||
whether this de-duplication is required. No behavior will change until the
|
||||
operator activates the relevant flag via the API.
|
||||
|
||||
To understand if you need to take action, consult
|
||||
[Resolve duplicate identities](/vault/docs/upgrading/deduplication) which
|
||||
demonstrates the log lines to watch for and how to ensure your resolve
|
||||
duplicates safely.
|
||||
|
||||
### LDAP user DN search with `upndomain`
|
||||
|
||||
The github.com/hashicorp/cap/ldap dependency has been upgraded to include a security improvement
|
||||
|
||||
@ -2551,6 +2551,31 @@
|
||||
"title": "Upgrade to Raft WAL",
|
||||
"path": "upgrading/raft-wal"
|
||||
},
|
||||
{
|
||||
"title": "Resolve duplicate identities",
|
||||
"routes": [
|
||||
{
|
||||
"title": "Process overview",
|
||||
"path": "upgrading/deduplication"
|
||||
},
|
||||
{
|
||||
"title": "Fix different-case entity alias duplicates",
|
||||
"path": "upgrading/deduplication/different-case"
|
||||
},
|
||||
{
|
||||
"title": "Fix entity and group duplicates",
|
||||
"path": "upgrading/deduplication/entity-group"
|
||||
},
|
||||
{
|
||||
"title": "Resolve ACL policy templates",
|
||||
"path": "upgrading/deduplication/acl-policy-templates"
|
||||
},
|
||||
{
|
||||
"title": "Resolve Terraform config",
|
||||
"path": "upgrading/deduplication/terraform-refs"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"title": "Upgrade to 1.19.x",
|
||||
"path": "upgrading/upgrade-to-1.19.x"
|
||||
|
||||
Loading…
Reference in New Issue
Block a user