Calvin Leung Huang 045836da71
raft: add support for using backend for ha_storage (#9193)
* raft: initial work on raft ha storage support

* add note on join

* add todo note

* raft: add support for bootstrapping and joining existing nodes

* raft: gate bootstrap join by reading leader api address from storage

* raft: properly check for raft-only for certain conditionals

* raft: add bootstrap to api and cli

* raft: fix bootstrap cli command

* raft: add test for setting up new cluster with raft HA

* raft: extend TestRaft_HA_NewCluster to include inmem and consul backends

* raft: add test for updating an existing cluster to use raft HA

* raft: remove debug log lines, clean up verifyRaftPeers

* raft: minor cleanup

* raft: minor cleanup

* Update physical/raft/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/ha.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/ha.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/logical_system_raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* address feedback comments

* address feedback comments

* raft: refactor tls keyring logic

* address feedback comments

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* address feedback comments

* testing: fix import ordering

* raft: rename var, cleanup comment line

* docs: remove ha_storage restriction note on raft

* docs: more raft HA interaction updates with migration and recovery mode

* docs: update the raft join command

* raft: update comments

* raft: add missing isRaftHAOnly check for clearing out state set earlier

* raft: update a few ha_storage config checks

* Update command/operator_raft_bootstrap.go

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* raft: address feedback comments

* raft: fix panic when checking for config.HAStorage.Type

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update website/pages/docs/commands/operator/raft.mdx

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* raft: remove bootstrap cli command

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft: address review feedback

* raft: revert vendored sdk

* raft: don't send applied index and node ID info if we're HA-only

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-06-23 12:04:13 -07:00

198 lines
8.9 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
layout: api
page_title: /sys/storage/raft - HTTP API
sidebar_title: <code>/sys/storage/raft</code>
description: |-
The `/sys/storage/raft` endpoints are used to manage Vault's Raft storage
backend.
---
## Join a Raft cluster
This endpoint joins a new server node to the Raft cluster. When using Shamir
seal, as soon as the Vault server is brought up, this API should be invoked
instead of `sys/init`. This API completes in 2 phases. Once this is invoked,
the joining node will receive a challenge from the Raft's leader node. This
challenge can be answered by the joining node only after a successful unseal.
Hence, the joining node should be unsealed using the unseal keys of the Raft's
leader node.
| Method | Path |
| :----- | :----------------------- |
| `POST` | `/sys/storage/raft/join` |
### Parameters
- `leader_api_addr` `(string: <required>)` Address of the leader node in the
Raft cluster to which this node is trying to join.
- `retry` `(bool: false)` - Retry joining the Raft cluster in case of
failures.
- `leader_ca_cert` `(string: "")` - CA certificate used to communicate with
Raft's leader node.
- `leader_client_cert` `(string: "")` - Client certificate used to communicate
with Raft's leader node.
- `leader_client_key` `(string: "")` - Client key used to communicate with
Raft's leader node.
### Sample Payload
```json
{
"leader_api_addr": "https://127.0.0.1:8200",
"leader_ca_cert": "-----BEGIN CERTIFICATE-----\nMIIDKTCCAhGgAwIBAgIUcd+Uyk1Tz+FhcbYP0zmynlkARoEwDQYJKoZIhvcNAQEL\nBQAwFDESMBAGA1UEAxMJbG9jYWxob3N0MB4XDTE5MDYyNDIyMjgzNVoXDTE5MDYy\nNzIyMjkwNVowADCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKdYr6rU\n21vYf7q/cpPigtBchqHvGZvpbA9DZucuUdQ0g4oAXRyShQ8omzmZOmO4A1GI3gqz\nHBePSYl+1IZCwgbPBk2CH7MhlMMINdwoEH6IxFgHNBkNK6GbwnGLyKL0Sym88ly1\n+sPP6+llS8uWNKu5GcObHLysD3Ce6QTt3usDPiw0cxp/KL1EkMi2dT7PvxTsX137\nsEsuQcylltGEtRb67xvFBP8XhQZAEGw+u4S3EmtwWMwZixB45WQhj2Ncz5U0+w8V\ncp9DSqB1QheoGPBBI62jHle05kzG85ZKmLBgHE6HEGS8biIHpannM7dgI0cRH6i2\nF69N0rcbzK+NInUCAwEAAaOBhjCBgzAOBgNVHQ8BAf8EBAMCA6gwHQYDVR0lBBYw\nFAYIKwYBBQUHAwEGCCsGAQUFBwMCMB0GA1UdDgQWBBQelJT5fthHdbCyD5zaI4tw\n7mfp8zAfBgNVHSMEGDAWgBSuG6hyoOWoiGvSA3kqwo9DirS+pzASBgNVHREBAf8E\nCDAGhwR/AAABMA0GCSqGSIb3DQEBCwUAA4IBAQCmetSH5w835RbbyHZD8e3ClHzi\n210SrvzWkx3N0JcBOjs47jlLuqVTl0HRr2xMoIkErFbhPDBXARYV8eezhQ6G5M60\n8AwVsG56rCa1l0weK2JfnEWgkwXZ/zbpZ2yNkWatWNSHdlJwGp99JTSriQYNOnMG\nWvBDA8ukoOkIJd/a8+aXZBdUiAcFvlLWmX73pYGTlnQDiIqJvhrlkgCKPvzZqvV+\njtz7kCt9EfT6sN6Xcny7GusRBSs+XfEe8u10+mbud+ufE/QSJ2D3tTYrS6eGqpVD\nrezD8jS9PfZgQfHXVaphMADyw4flaSVxfJ/ZRFxXql3oJSuj02+VX6QsXex+\n-----END CERTIFICATE-----",
"leader_client_cert": "-----BEGIN CERTIFICATE-----\nMIIDNTCCAh2gAwIBAgIULkPFG+qu7tGv21Plc1sg4eGbMPkwDQYJKoZIhvcNAQEL\nBQAwFDESMBAGA1UEAxMJbG9jYWxob3N0MB4XDTE5MDYyNDIyMjgwNVoXDTI5MDYy\nMTIyMjgzNVowFDESMBAGA1UEAxMJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEF\nAAOCAQ8AMIIBCgKCAQEAwszD+A5vWD8S1N632ElHA5Px2dk97wJBKNYc/7RPNptn\n+EOmMTXPfuA3LB92FoSMCR7ye3wvSTzyK9nqafS7U2tlOF4PJrJoNZyzrVwBVLXg\n7Pd8qQxnxbonc3bscWZuEfbFsugkPHgBtnSkyCffXKhhwM5LbJqmK5cfJRZZ0eRy\nwmOCQCJ8ZmN2KfjHiGSEw9v19CNtvFNLyfiTZZLO9M5n4dgainZZCs+vdKD7tSJf\nycwWiZ4ezOwLMIgxdbLYKVglbZsPcMVVPLTskmY8WiHUM6sy3HAbFQn20Rj7JGE6\nldR3NX80YtCMt8/d+xzBfxu4x8juxHCgZFGt3nUS0wIDAQABo38wfTAOBgNVHQ8B\nAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUrhuocqDlqIhr0gN5\nKsKPQ4q0vqcwHwYDVR0jBBgwFoAUrhuocqDlqIhr0gN5KsKPQ4q0vqcwGgYDVR0R\nBBMwEYIJbG9jYWxob3N0hwR/AAABMA0GCSqGSIb3DQEBCwUAA4IBAQBRno9NOaat\n8g1ma/6OxP1JUt1VRao4+t2GQTCJ697d2SmpHr8CYanMPog+QhDj/lderA/oS0V3\n2jruH9CyH0smFMIsLBZpnRIwdhTISbIXdU3Uvcd6nne/f7LiUUmqf8YS9SHxWPFq\nI72QvtPrsjYKCZwJsHZe071lYScjrjGnlUhhbrug2g6/ZMHJ7mndGE022zMn/XD/\njrrKE6fCDCjJ4PRrta7+G7BvsFHqMSQ+2/947TGohxW69cbNyDeiQVDQ5mpJUy2b\nnOCVVnq0nafSuvBTHCCLxjd7f+9TiB/B3qT8GA9V6LANDSdQ15MRIhpup/yosmqX\nl5goCY/j3bDh\n-----END CERTIFICATE-----",
"leader_client_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAp1ivqtTbW9h/ur9yk+KC0FyGoe8Zm+lsD0Nm5y5R1DSDigBd\nHJKFDyibOZk6Y7gDUYjeCrMcF49JiX7UhkLCBs8GTYIfsyGUwwg13CgQfojEWAc0\nGQ0roZvCcYvIovRLKbzyXLX6w8/r6WVLy5Y0q7kZw5scvKwPcJ7pBO3e6wM+LDRz\nGn8ovUSQyLZ1Ps+/FOxfXfuwSy5BzKWW0YS1FvrvG8UE/xeFBkAQbD67hLcSa3BY\nzBmLEHjlZCGPY1zPlTT7DxVyn0NKoHVCF6gY8EEjraMeV7TmTMbzlkqYsGAcTocQ\nZLxuIgelqeczt2AjRxEfqLYXr03StxvMr40idQIDAQABAoIBAHCOhhkw8hnklITX\nIAm34KSklylz2JW1eqkJfL0huogwigjYFciSBE0d0sn69fr8Wb8Nf8tSjSKLjbqd\nQ/TKEhbiSCr5yriBbb2AWDmr+OwisW1D6xaudRpN2Yrlqh1wkt2P6LPS8sehodtp\n9oEIloSqC1o1ii6czAXD3JckJzuJ66yoVyCo2oVC3NETujjJRhdRcNY0dYUGx47M\n+hQo7b4s79wlMFmccrgyJcKI7Ra3DtMrBIaVXm4WcjX2x4DduZX6L4mfG83uLWng\nhbvcEPKGamUErjPGYixKE5XOwgJT6Nzq/ZZJZpok6DbHBCtqvwwByWlfLzh8lc4o\nz+Pl5EECgYEA3pJXRM+SdVUCnWIpMFOAiNstOwz3nm0/c8GDdYYNq0ere1tKCVV0\nii+ujMQY8vEdRrp7fbqJEPJANyI6sM4Jc2nxMveFuBoEsEClZuE4KBFquW8CQU4R\nGNkzKF1W0GaTI5O4QDUag+lFG6TPCwZiejTIkt65o5bnSzGLgdC546sCgYEAwHr9\nlq9HytWkV9OLz0zbcAUEIrFAljnC32Mybsj8LzdSKIZ3wErf9Txh8zqZqMyhedxL\nOn17cpJAq5Vi71uXOaxnhkf7TPuRs8nBRRJfgyNHPXuXLkuWeuvXEe/PIC+nvOIX\nas0Ab0Pvx0fRGl4EkZ2pV9oDAEUezbVVZCoO8l8CgYEAp4p3YNfzwpj0d+5tXPBu\ngBakzJ+tQjewnP1dbLk1TuqLXjdQ2wfVyzOrbFEtCquPwy8bSICDLxt8VURR88eU\n+6kTJK+InBYR029GUtRUhmzd7qpugyQ14IOXa9ofQI8GUbCf8M1IoWWWXQHJzN3I\ngX98QHqiOU+d2k94WYb6dzMCgYA3E7p2oMdZShLRss3hzqtH/Zd2WeQSWIrjox3u\nR4Kp/Bl9UST5GDPHl7SrhcwsWgmmthusq/VkDmeE8aUyurGmJigla3mESMQjfwrX\nue8sti6PcEsNS0HPAKc1EbriCeDkkomC3RBPxk/ZZTp3YgnKpSOs6MxNCnpLAKVj\nmQlX3wKBgQCFYyYJdCKMoaoMrNjMM34sAfTqmZT4Hbxg9KM4OMDSxyvIi0ZaVcnB\nkzssvt0FOMucp4o7h8Iujt8xsgnBHwg81IV7WOw+ZwpUDAzgVc7+kCU6CLd2Q40j\n5uFnuRgiWeGm1aT4arNLWlJrDLSGbyrf59SlGJV1hmu/7SMaYIZHSQ==\n-----END RSA PRIVATE KEY-----"
}
```
### Sample Request
```shell-session
$ curl \
--header "X-Vault-Token: ..." \
--request POST \
--data @payload.json \
http://127.0.0.1:8200/v1/sys/storage/raft/join
```
**Note:** Unseal the joining node immediately after this API is invoked.
## Read Raft Configuration
This endpoint returns the details of all the nodes in the raft cluster.
| Method | Path |
| :----- | :-------------------------------- |
| `GET` | `/sys/storage/raft/configuration` |
### Sample Request
```shell-session
$ curl \
--header "X-Vault-Token: ..." \
http://127.0.0.1:8200/v1/sys/storage/raft/configuration
```
### Sample Response
```json
{
"request_id": "ca4b5a1d-38d6-e27e-e756-269521328a15",
"lease_id": "",
"lease_duration": 0,
"renewable": false,
"data": {
"config": {
"index": 24,
"servers": [
{
"address": "127.0.0.1:8201",
"leader": true,
"node_id": "raft1",
"protocol_version": "\u0003",
"voter": true
},
{
"address": "127.0.0.2:8201",
"leader": false,
"node_id": "raft2",
"protocol_version": "\u0003",
"voter": true
}
]
}
},
"warnings": null
}
```
## Remove a node from Raft cluster
This endpoint removes a node from the raft cluster.
| Method | Path |
| :----- | :------------------------------ |
| `POST` | `/sys/storage/raft/remove-peer` |
### Sample Payload
```json
{
"server_id": "raft1"
}
```
### Sample Request
```shell-session
$ curl \
--header "X-Vault-Token: ..." \
--request POST \
--data @payload.json \
http://127.0.0.1:8200/v1/sys/storage/raft/remove-peer
```
## Take a snapshot of the Raft cluster
This endpoint returns a snapshot of the current state of the raft cluster. The
snapshot is returned as binary data and should be redirected to a file.
Unavailable if Raft is used exclusively for `ha_storage`.
| Method | Path |
| :----- | :--------------------------- |
| `GET` | `/sys/storage/raft/snapshot` |
### Sample Request
```shell-session
$ curl \
--header "X-Vault-Token: ..." \
--request GET \
http://127.0.0.1:8200/v1/sys/storage/raft/snapshot > raft.snap
```
## Restore Raft using a snapshot
Installs the provided snapshot, returning the cluster to the state defined in
it. Unavailable if Raft is used exclusively for `ha_storage`.
| Method | Path |
| :----- | :--------------------------- |
| `POST` | `/sys/storage/raft/snapshot` |
### Sample Request
```shell-session
$ curl \
--header "X-Vault-Token: ..." \
--request POST \
-- data-binary @raft.snap
http://127.0.0.1:8200/v1/sys/storage/raft/snapshot
```
## Force Restore Raft using a snapshot
Installs the provided snapshot, returning the cluster to the state defined in
it. This is same as writing to `/sys/storage/raft/snapshot` except that this
bypasses checks ensuring the Autounseal or shamir keys are consistent with the
snapshot data. Unavailable if Raft is used exclusively for `ha_storage`.
| Method | Path |
| :----- | :--------------------------------- |
| `POST` | `/sys/storage/raft/snapshot-force` |
### Sample Request
```shell-session
$ curl \
--header "X-Vault-Token: ..." \
--request POST \
--data-binary @raft.snap
http://127.0.0.1:8200/v1/sys/storage/raft/snapshot-force
```