---
layout: docs
page_title: Upgrading to Vault 1.12.x - Guides
description: |-
  This page contains the list of deprecations and important or breaking changes
  for Vault 1.12.x. Please read it carefully.
---

# Overview

This page contains the list of deprecations and important or breaking changes
for Vault 1.12.x compared to 1.11. Please read it carefully.

## Changes

### Supported storage backends

Vault Enterprise will now perform a supported storage check at startup. There is no impact on open-source Vault users.

@include 'ent-supported-storage.mdx'

@include 'consul-dataplane-upgrade-note.mdx'

### External plugin loading

Vault 1.12.0 introduced a change to how external plugins are loaded. Prior to
Vault 1.12.0 plugins were lazy loaded on startup. This means that plugin
processes were killed after a successful mount and then respawned when a
request is routed to them. Vault 1.12.0 introduced auto mutual TLS for
secrets/auth plugins so we do not lazy load them on startup anymore.

## Known issues

### Pinning to builtin plugin versions may cause failure on upgrade

1.12.0 introduced plugin versions, and with it, the ability to explicitly specify
the builtin version of a plugin when mounting an auth, database or secrets plugin.
For example, `vault auth enable -plugin-version=v1.12.0+builtin.vault approle`. If
there are any mounts where the _builtin_ version was explicitly specified in this way,
Vault may fail to start on upgrading to 1.12.1 due to the specified version no
longer being available.

To check whether a mount path is affected, read the tune information, or the
database config. The affected plugins are `snowflake-database-plugin@v0.6.0+builtin`
and any plugins with `+builtin.vault` metadata in their version.

In this example, the first two mounts are affected because `plugin_version` is
explicitly set and is one of the affected versions. The third mount is not
affected because it only has `+builtin` metadata, and is not the Snowflake
database plugin. All mounts where the version is omitted, or the plugin is
external (regardless of whether the version is specified) are unaffected.

-> **NOTE:** Make sure you use Vault CLI 1.12.0 or later to check mounts.

```shell-session
$ vault read sys/auth/approle/tune
Key                  Value
---                  -----
...
plugin_version       v1.12.0+builtin.vault

$ vault read database/config/snowflake
Key                                   Value
---                                   -----
...
plugin_name                           snowflake-database-plugin
plugin_version                        v0.6.0+builtin

$ vault read sys/auth/kubernetes/tune
Key                  Value
---                  -----
...
plugin_version       v0.14.0+builtin
```

As it is not currently possible to unset the plugin version, there are 3 possible
remediations if you have any affected mounts:

* Upgrade Vault directly to 1.12.2 once released
* Upgrade to an external version of the plugin before upgrading to 1.12.1;
  * Using the [tune API](/vault/api-docs/system/auth#tune-auth-method) for auth methods
  * Using the [tune API](/vault/api-docs/system/mounts#tune-mount-configuration) for secrets plugins
  * Or using the [configure connection](/vault/api-docs/secret/databases#configure-connection)
    API for database plugins
* Unmount and remount the path without a version specified before upgrading to 1.12.1.
  **Note:** This will delete all data and leases associated with the mount.

The bug was introduced by commit
https://github.com/hashicorp/vault/commit/c36330f4c713b886a8a23c08cbbd862a7c530fc8.

#### Impacted versions

Affects upgrading from 1.12.0 to 1.12.1. All other upgrade paths are unaffected.
1.12.2 will introduce a fix that enables upgrades from affected deployments of
1.12.0.

### Mounts associated with deprecated builtin plugins will result in core shutdown on upgrade

As of 1.12.0 Standalone (logical) DB Engines and the AppId Auth Method have been
marked with the `Pending Removal` status. Any attempt to unseal Vault with
mounts backed by one of these builtin plugins will result in an immediate
shutdown of the Vault core.

-> **NOTE** In the event that an external plugin with the same name and type as
a deprecated builtin is deregistered, any subsequent unseal of Vault will also
result in a core shutdown.

```shell-session
$ vault plugin register -sha256=c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1 auth app-id
Success! Registered plugin: app-id
$ vault auth enable -plugin-name=app-id plugin
Success! Enabled app-id auth method at: app-id/
$ vault auth list -detailed
app-id/    app-id    auth_app-id_3a8f2e24    system         system     default-service    replicated     false        false                      map[]      n/a                        0018263c-0d64-7a70-fd5c-50e05c5f5dc3    n/a        n/a                      c805cf3b69f704dfcd5176ef1c7599f88adbfd7374e9c76da7f24a32a97abfe1    n/a
$ vault plugin deregister auth app-id
Success! Deregistered plugin (if it was registered): app-id
$ vault plugin list -detailed | grep "app-id"
app-id                               auth        v1.12.0+builtin.vault                                 pending removal
```

The remediation for affected mounts is to set the
[VAULT_ALLOW_PENDING_REMOVAL_MOUNTS](/vault/docs/commands/server#vault_allow_pending_removal_mounts)
environment variable and replace any `Pending Removal` feature with the
[preferred alternative
feature](/vault/docs/deprecation/faq#q-what-should-i-do-if-i-use-mount-filters-appid-or-any-of-the-standalone-db-engines).

For more information on the phases of deprecation, see the [Deprecation Notices
FAQ](/vault/docs/deprecation/faq#q-what-are-the-phases-of-deprecation).

#### Impacted versions

Affects upgrading from any version of Vault to 1.12.x. All other upgrade paths
are unaffected.

### `vault plugin list` fails when audit logging is enabled

If audit logging is enabled, Vault will fail to audit the response from any
calls to the [`GET /v1/sys/plugins/catalog`](/vault/api-docs/system/plugins-catalog#list-plugins)
endpoint, which causes the whole request to fail and return a 500 internal
server error. From the CLI, this looks like the following:

```shell-session
$ vault plugin list
Error listing available plugins: data from server response is empty
```

It will produce errors in Vault Server's logs such as:

```text
2022-11-30T20:04:22.397Z [ERROR] audit: panic during logging: request_path=sys/plugins/catalog error="reflect: reflect.Value.Set using value obtained using unexported field"
2022-11-30T20:04:22.398Z [ERROR] core: failed to audit response: request_path=sys/plugins/catalog
  error=
  | 1 error occurred:
  | 	* panic generating audit log
  |
```

As a workaround, [listing plugins by type](/vault/api-docs/system/plugins-catalog#list-plugins-1)
will succeed:

* `vault list sys/plugins/catalog/auth`
* `vault list sys/plugins/catalog/database`
* `vault list sys/plugins/catalog/secret`

The bug was introduced by commit
https://github.com/hashicorp/vault/commit/76165052e54f884ed0aa2caa496083dc84ad1c19.

#### Impacted versions

Affects versions 1.12.0, 1.12.1, and 1.12.2. A fix will be released in 1.12.3.

### PKI OCSP GET requests return malformed request responses

If an OCSP GET request contains a '+' character, a malformed request response will be
returned instead of properly processing the request due to a double decoding issue within the
handler.

As a workaround, OCSP POST requests can be used which are unaffected.

#### Impacted versions

Affects version 1.12.3. A fix will be released in 1.12.4.

@include 'tokenization-rotation-persistence.mdx'

@include 'ocsp-redirect.mdx'

### LDAP pagination issue

There was a regression introduced in 1.12.6 relating to LDAP maximum page sizes, resulting in
an error `no LDAP groups found in groupDN [...] only policies from locally-defined groups available`.  The issue
occurs when upgrading Vault with an instance that has an existing LDAP Auth configuration.

As a workaround, disable paged searching using the following:
```shell-session
vault write auth/ldap/config max_page_size=-1
```

#### Impacted versions

Affects Vault 1.12.6.

### Slow startup time when storing PKI certificates

There was a regression introduced in 1.12.0 where Vault is slow to start because the 
PKI secret engine performs a list operation on the stored certificates. If a large number 
of certificates are stored this can cause long start times on active and standby nodes.

There is currently no workaround for this other than limiting the number of certificates stored
in Vault via the [PKI tidy](/vault/api-docs/secret/pki#tidy) or using `no_store`
flag for [PKI roles](/vault/api-docs/secret/pki#createupdate-role).

#### Impacted versions

Affects Vault 1.12.0+

@include 'pki-double-migration-bug.mdx'