mirror of
https://github.com/siderolabs/talos.git
synced 2026-05-05 12:26:21 +02:00
docs: update documentation for Talos 1.3
This provides What's New, updates documentation with new features, clarifications and fixes. I added also simple bare-metal stubs for ISO and PXE methods. Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
This commit is contained in:
Andrey Smirnov
parent
faf49218ce
commit
474604cd27
@ -14,11 +14,6 @@ docker run -d -p 5001:5000 \
|
||||
--restart always \
|
||||
--name registry-registry.k8s.io registry:2
|
||||
|
||||
docker run -d -p 5002:5000 \
|
||||
-e REGISTRY_PROXY_REMOTEURL=https://quay.io \
|
||||
--restart always \
|
||||
--name registry-quay.io registry:2.5
|
||||
|
||||
docker run -d -p 5003:5000 \
|
||||
-e REGISTRY_PROXY_REMOTEURL=https://gcr.io \
|
||||
--restart always \
|
||||
|
||||
@ -104,7 +104,6 @@ Identify all registry prefixes from `talosctl images`, for example:
|
||||
- `gcr.io`
|
||||
- `ghcr.io`
|
||||
- `registry.k8s.io`
|
||||
- `quay.io`
|
||||
|
||||
The `talosctl cluster create` command provides conveniences for common configuration options.
|
||||
The only required flag for this guide is `--registry-mirror <endpoint>=http://10.5.0.1:6000` which redirects every pull request to the internal registry, this flag
|
||||
@ -117,7 +116,6 @@ $ sudo --preserve-env=HOME talosctl cluster create --provisioner=qemu --install-
|
||||
--registry-mirror gcr.io=http://10.5.0.1:6000 \
|
||||
--registry-mirror ghcr.io=http://10.5.0.1:6000 \
|
||||
--registry-mirror registry.k8s.io=http://10.5.0.1:6000 \
|
||||
--registry-mirror quay.io=http://10.5.0.1:6000
|
||||
validating CIDR and reserving IPs
|
||||
generating PKI and tokens
|
||||
creating state directory in "/home/user/.talos/clusters/talos-default"
|
||||
@ -157,11 +155,10 @@ machine:
|
||||
registry.k8s.io:
|
||||
endpoints:
|
||||
- http://10.5.0.1:6000/
|
||||
quay.io:
|
||||
endpoints:
|
||||
- http://10.5.0.1:6000/
|
||||
...
|
||||
```
|
||||
|
||||
Other implementations of Docker registry can be used in place of the Docker `registry` image used above to run the registry.
|
||||
If required, auth can be configured for the internal registry (and custom TLS certificates if needed).
|
||||
|
||||
Please see [pull-through cache guide]({{< relref "../talos-guides/configuration/pull-through-cache" >}}) for an example using Harbor container registry with Talos.
|
||||
|
||||
@ -67,7 +67,6 @@ sudo --preserve-env=HOME _out/talosctl-linux-amd64 cluster create \
|
||||
--cidr=172.20.0.0/24 \
|
||||
--registry-mirror docker.io=http://172.20.0.1:5000 \
|
||||
--registry-mirror kregistry.k8s.io=http://172.20.0.1:5001 \
|
||||
--registry-mirror quay.io=http://172.20.0.1:5002 \
|
||||
--registry-mirror gcr.io=http://172.20.0.1:5003 \
|
||||
--registry-mirror ghcr.io=http://172.20.0.1:5004 \
|
||||
--registry-mirror 127.0.0.1:5005=http://172.20.0.1:5005 \
|
||||
|
||||
@ -167,10 +167,18 @@ kernel commandline parameters.
|
||||
See [required kernel parameters]({{< relref "../reference/kernel" >}}).
|
||||
If creating [EC2 kubernetes clusters]({{< relref "../talos-guides/install/cloud-platforms/aws/" >}}), the configuration file can be passed in as `--user-data` to the `aws ec2 run-instances` command.
|
||||
|
||||
In any case, we need to generate the configuration which is to be provided:
|
||||
In any case, we need to generate the configuration which is to be provided.
|
||||
We start with generating a secrets bundle which should be saved in a secure location and used
|
||||
to generate machine or client configuration at any time:
|
||||
|
||||
```sh
|
||||
talosctl gen config cluster-name cluster-endpoint
|
||||
talosctl gen secrets -o secrets.yaml
|
||||
```
|
||||
|
||||
Now, we can generate the machine configuration for each node:
|
||||
|
||||
```sh
|
||||
talosctl gen config --with-secrets secrets.yaml <cluster-name> <cluster-endpoint>
|
||||
```
|
||||
|
||||
Here, `cluster-name` is an arbitrary name for the cluster, used
|
||||
@ -185,7 +193,7 @@ and port.
|
||||
For example:
|
||||
|
||||
```sh
|
||||
talosctl gen config my-cluster https://192.168.64.15:6443
|
||||
$ talosctl gen config --with-secrets secrets.yaml my-cluster https://192.168.64.15:6443
|
||||
generating PKI and tokens
|
||||
created /Users/taloswork/controlplane.yaml
|
||||
created /Users/taloswork/worker.yaml
|
||||
@ -225,7 +233,7 @@ A common example is needing to change the default installation disk.
|
||||
If you try to to apply the machine config to a node, and get an error like the below, you need to specify a different installation disk:
|
||||
|
||||
```sh
|
||||
talosctl apply-config --insecure -n 192.168.64.8 --file controlplane.yaml
|
||||
$ talosctl apply-config --insecure -n 192.168.64.8 --file controlplane.yaml
|
||||
error applying new configuration: rpc error: code = InvalidArgument desc = configuration validation failed: 1 error occurred:
|
||||
* specified install disk does not exist: "/dev/sda"
|
||||
```
|
||||
@ -237,7 +245,7 @@ You can verify which disks your nodes have by using the `talosctl disks --insecu
|
||||
For example:
|
||||
|
||||
```sh
|
||||
talosctl -n 192.168.64.8 disks --insecure
|
||||
$ talosctl -n 192.168.64.8 disks --insecure
|
||||
DEV MODEL SERIAL TYPE UUID WWID MODALIAS NAME SIZE BUS_PATH
|
||||
/dev/vda - - HDD - - virtio:d00000002v00001AF4 - 69 GB /pci0000:00/0000:00:06.0/virtio2/
|
||||
```
|
||||
@ -251,28 +259,60 @@ install:
|
||||
|
||||
to reflect `vda` instead of `sda`.
|
||||
|
||||
### Customizing Machine Configuration
|
||||
|
||||
The generated machine configuration provides sane defaults for most cases, but machine configuration
|
||||
can be modified to fit specific needs.
|
||||
|
||||
Some machine configuration options are available as flags for the `talosctl gen config` command,
|
||||
for example setting a specific Kubernetes version:
|
||||
|
||||
```sh
|
||||
talosctl gen config --with-secrets secrets.yaml --kubernetes-version 1.25.4 my-cluster https://192.168.64.15:6443
|
||||
```
|
||||
|
||||
Other modifications are done with [machine configuration patches]({{< relref "../talos-guides/configuration/patching" >}}).
|
||||
Machine configuration patches can be applied with `talosctl gen config` command:
|
||||
|
||||
```sh
|
||||
talosctl gen config --with-secrets secrets.yaml --config-patch-control-plane @cni.patch my-cluster https://192.168.64.15:6443
|
||||
```
|
||||
|
||||
> Note: `@cni.patch` means that the patch is read from a file named `cni.patch`.
|
||||
|
||||
#### Machine Configs as Templates
|
||||
|
||||
Individual machines may need different settings: for instance, each may have a
|
||||
different [static IP address]({{< relref "../advanced/advanced-networking/#static-addressing" >}}).
|
||||
|
||||
When different files are needed for machines of the same type, simply
|
||||
copy the source template (`controlplane.yaml` or `worker.yaml`) and make whatever
|
||||
modifications are needed.
|
||||
When different files are needed for machines of the same type, there are two supported flows:
|
||||
|
||||
For instance, if you had three control plane nodes and three worker nodes, you
|
||||
may do something like this:
|
||||
1. Use the `talosctl gen config` command to generate a template, and then patch
|
||||
the template for each machine with `talosctl machineconfig patch`.
|
||||
2. Generate each machine configuration file separately with `talosctl gen config` while applying patches.
|
||||
|
||||
```bash
|
||||
for i in $(seq 0 2); do
|
||||
cp controlplane.yaml cp$i.yaml
|
||||
end
|
||||
for i in $(seq 0 2); do
|
||||
cp worker.yaml w$i.yaml
|
||||
end
|
||||
For example, given a machine configuration patch which sets the static machine hostname:
|
||||
|
||||
```yaml
|
||||
# worker1.patch
|
||||
machine:
|
||||
network:
|
||||
hostname: worker1
|
||||
```
|
||||
|
||||
Then modify each file as needed.
|
||||
Either of the following commands will generate a worker machine configuration file with the hostname set to `worker1`:
|
||||
|
||||
```bash
|
||||
$ talosctl gen config --with-secrets secrets.yaml my-cluster https://192.168.64.15:6443
|
||||
created /Users/taloswork/controlplane.yaml
|
||||
created /Users/taloswork/worker.yaml
|
||||
created /Users/taloswork/talosconfig
|
||||
$ talosctl machineconfig patch worker.yaml --patch @worker1.patch --output worker1.yaml
|
||||
```
|
||||
|
||||
```sh
|
||||
talosctl gen config --with-secrets secrets.yaml --config-patch-worker @worker1.patch --output-types worker -o worker1.yaml my-cluster https://192.168.64.15:6443
|
||||
```
|
||||
|
||||
### Apply Configuration
|
||||
|
||||
|
||||
@ -7,22 +7,20 @@ description: "Table of supported Talos Linux versions and respective platforms."
|
||||
| Talos Version | 1.3 | 1.2 |
|
||||
|----------------------------------------------------------------------------------------------------------------|------------------------------------|------------------------------------|
|
||||
| Release Date | 2022-12-01 | 2022-09-01 (1.2.0) |
|
||||
| End of Community Support | 1.4.0 release (2023-03-01, TBD) | 1.3.0 release (2022-12-01, TBD) |
|
||||
| End of Community Support | 1.4.0 release (2023-03-15, TBD) | 1.3.0 release (2022-12-15, TBD) |
|
||||
| Enterprise Support | [offered by Sidero Labs Inc.](https://www.siderolabs.com/support/) | [offered by Sidero Labs Inc.](https://www.siderolabs.com/support/) |
|
||||
| Kubernetes | 1.26, 1.25, 1.24 | 1.25, 1.24, 1.23 |
|
||||
| Architecture | amd64, arm64 | amd64, arm64 |
|
||||
| **Platforms** | | |
|
||||
| - cloud | AWS, GCP, Azure, Digital Ocean, Exoscale, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud |
|
||||
| - cloud | AWS, GCP, Azure, Digital Ocean, Exoscale, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud |
|
||||
| - bare metal | x86: BIOS, UEFI; arm64: UEFI; boot: ISO, PXE, disk image | x86: BIOS, UEFI; arm64: UEFI; boot: ISO, PXE, disk image |
|
||||
| - virtualized | VMware, Hyper-V, KVM, Proxmox, Xen | VMware, Hyper-V, KVM, Proxmox, Xen |
|
||||
| - SBCs | Banana Pi M64, Jetson Nano, Libre Computer Board ALL-H3-CC, Nano Pi R4S, Pine64, Pine64 Rock64, Radxa ROCK Pi 4c, Raspberry Pi 4B, Raspberry Pi Compute Module 4 | Banana Pi M64, Jetson Nano, Libre Computer Board ALL-H3-CC, Pine64, Pine64 Rock64, Radxa ROCK Pi 4c, Raspberry Pi 4B, Raspberry Pi Compute Module 4 |
|
||||
| - local | Docker, QEMU | Docker, QEMU |
|
||||
| **Cluster API** | | |
|
||||
| [CAPI Bootstrap Provider Talos](https://github.com/siderolabs/cluster-api-bootstrap-provider-talos) | >= 0.5.5 | >= 0.5.5 |
|
||||
| [CAPI Control Plane Provider Talos](https://github.com/siderolabs/cluster-api-control-plane-provider-talos) | >= 0.4.9 | >= 0.4.9 |
|
||||
| [Sidero](https://www.sidero.dev/) | >= 0.5.5 | >= 0.5.5 |
|
||||
| **UI** | | |
|
||||
| [Theila](https://github.com/siderolabs/theila) | ✓ | ✓ |
|
||||
| [CAPI Bootstrap Provider Talos](https://github.com/siderolabs/cluster-api-bootstrap-provider-talos) | >= 0.5.6 | >= 0.5.5 |
|
||||
| [CAPI Control Plane Provider Talos](https://github.com/siderolabs/cluster-api-control-plane-provider-talos) | >= 0.4.10 | >= 0.4.9 |
|
||||
| [Sidero](https://www.sidero.dev/) | >= 0.5.7 | >= 0.5.5 |
|
||||
|
||||
## Platform Tiers
|
||||
|
||||
@ -45,6 +43,7 @@ description: "Table of supported Talos Linux versions and respective platforms."
|
||||
|
||||
### Tier 3
|
||||
|
||||
* Exoscale
|
||||
* Hetzner
|
||||
* nocloud
|
||||
* Oracle Cloud
|
||||
|
||||
@ -12,18 +12,21 @@ description: "Hardware requirements for running Talos Linux."
|
||||
<th class="px-4 py-2">Role</th>
|
||||
<th class="px-4 py-2">Memory</th>
|
||||
<th class="px-4 py-2">Cores</th>
|
||||
<th class="px-4 py-2">System Disk</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td class="border px-4 py-2">Control Plane</td>
|
||||
<td class="border px-4 py-2">2GB</td>
|
||||
<td class="border px-4 py-2">2 GiB</td>
|
||||
<td class="border px-4 py-2">2</td>
|
||||
<td class="border px-4 py-2">10 GiB</td>
|
||||
</tr>
|
||||
<tr class="bg-gray-100">
|
||||
<td class="border px-4 py-2">Worker</td>
|
||||
<td class="border px-4 py-2">1GB</td>
|
||||
<td class="border px-4 py-2">1 GiB</td>
|
||||
<td class="border px-4 py-2">1</td>
|
||||
<td class="border px-4 py-2">10 GiB</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
@ -36,18 +39,21 @@ description: "Hardware requirements for running Talos Linux."
|
||||
<th class="px-4 py-2">Role</th>
|
||||
<th class="px-4 py-2">Memory</th>
|
||||
<th class="px-4 py-2">Cores</th>
|
||||
<th class="px-4 py-2">System Disk</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td class="border px-4 py-2">Control Plane</td>
|
||||
<td class="border px-4 py-2">4GB</td>
|
||||
<td class="border px-4 py-2">4 GiB</td>
|
||||
<td class="border px-4 py-2">4</td>
|
||||
<td class="border px-4 py-2">100 GiB</td>
|
||||
</tr>
|
||||
<tr class="bg-gray-100">
|
||||
<td class="border px-4 py-2">Worker</td>
|
||||
<td class="border px-4 py-2">2GB</td>
|
||||
<td class="border px-4 py-2">2 GiB</td>
|
||||
<td class="border px-4 py-2">2</td>
|
||||
<td class="border px-4 py-2">100 GiB</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
@ -1,75 +0,0 @@
|
||||
---
|
||||
title: Theila UI for Talos
|
||||
weight: 35
|
||||
description: "An intro to Theila - a UI for Talos clusters."
|
||||
---
|
||||
|
||||
Once you have a Talos cluster running, you may find it easier to get insights on your cluster(s) using a visual user interface rather than the `talosctl` CLI.
|
||||
For this, Sidero Labs provides [Theila](https://github.com/siderolabs/theila), a simple, single-binary web-based visual user interface for Talos clusters.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You should have a Talos cluster up & running, and the `talosconfig` file for Theila to access it.
|
||||
|
||||
## Installation
|
||||
|
||||
Theila is published as a single static binary compiled for various platforms and architectures, as well as a container image.
|
||||
|
||||
### Binary
|
||||
|
||||
You can download the correct binary for your system from the [releases](https://github.com/siderolabs/theila/releases) page, or use the following commands in your terminal.
|
||||
|
||||
```bash
|
||||
curl -Lo /usr/local/bin/theila https://github.com/siderolabs/theila/releases/download/{{< theila_release >}}/theila-$(uname -s | tr "[:upper:]" "[:lower:]")-amd64
|
||||
chmod +x /usr/local/bin/theila
|
||||
```
|
||||
|
||||
## Use
|
||||
|
||||
Once installed, you can run Theila by simply running it.
|
||||
|
||||
<!-- markdownlint-disable MD001 -->
|
||||
<!-- markdownlint-disable MD022 -->
|
||||
<!-- markdownlint-disable MD025 -->
|
||||
|
||||
{{< tabpane lang="bash" >}}
|
||||
{{< tab header="Binary" >}}
|
||||
# address and port are not required and default to the values shown
|
||||
theila --address 127.0.0.1 --port 8080
|
||||
{{< /tab >}}
|
||||
{{< tab header="Docker" >}}
|
||||
docker run --rm --volume ${HOME}/.talos/config:/opt/talosconfig:ro --env TALOSCONFIG=/opt/talosconfig --publish 8080:8080 ghcr.io/siderolabs/theila --address 0.0.0.0
|
||||
{{< /tab >}}
|
||||
{{< /tabpane >}}
|
||||
|
||||
Once it is running you should be able to point a browser at [http://localhost:8080](http://localhost:8080) to open the Theila UI.
|
||||
|
||||
### Clusters
|
||||
|
||||
You can navigate around various Talos clusters using the menu at the upper-left corner (see 1.1), then selecting the specific cluster from the list (see 1.2).
|
||||
|
||||
|
||||
### Cluster Overview
|
||||
|
||||
Clicking on the "Overview" option in the menu (see 2.1) will display an overview of resource use & health of the cluster.
|
||||
|
||||
|
||||
### Nodes
|
||||
|
||||
Entering the "Nodes" section on the menu (see 3.1) will give a list of nodes in the cluster (see 3.2), along with information such as IP address, status, and any roles assigned to the node.
|
||||
Opening the node menu (see 3.3) show the actions that can be taken on a specific node.
|
||||
|
||||
|
||||
Clicking on a specific node name in the node list will open the node detail page for more information on each specific node (see 4.1), including running services and their logs (see 4.2).
|
||||
|
||||
|
||||
Clicking on the "Monitor" tab (see 5.1) allows you to watch resource use over time, with CPU and memory consumption graphs updated in real time, and a detailed list of running process each with their individual resource use (see 5.2).
|
||||
|
||||
|
||||
Lastly, the "Dmesg" tab shows all kernel messages of the node since boot.
|
||||
|
||||
### Pods
|
||||
|
||||
Using the "Pods" section on the menu (see 6.1) will list all pods in the cluster, across all namespaces.
|
||||
Clicking on the drop-down arrow (see 6.2) will open up more detailed information of the specified pod.
|
||||
|
||||
@ -6,4 +6,353 @@ description: "List of new and shiny features in Talos Linux."
|
||||
|
||||
See also [upgrade notes]({{< relref "../talos-guides/upgrading-talos/">}}) for important changes.
|
||||
|
||||
TBD
|
||||
## Component Updates
|
||||
|
||||
* Kubernetes: v1.26.0
|
||||
* Flannel: v0.20.2
|
||||
* CoreDNS: v1.10.0
|
||||
* etcd: v3.5.6
|
||||
* Linux: 5.15.82
|
||||
* containerd: v1.6.12
|
||||
|
||||
Talos is built with Go 1.19.4.
|
||||
|
||||
## Kubernetes
|
||||
|
||||
### `kube-apiserver` Custom Audit Policy
|
||||
|
||||
Talos now supports setting custom audit policy for `kube-apiserver` in the machine configuration.
|
||||
|
||||
```yaml
|
||||
cluster:
|
||||
apiServer:
|
||||
auditPolicy: |
|
||||
apiVersion: audit.k8s.io/v1
|
||||
kind: Policy
|
||||
rules:
|
||||
- level: Metadata
|
||||
```
|
||||
|
||||
### `etcd` Secrets Encryption with `secretbox` algorithm
|
||||
|
||||
By default new clusters will use `secretbox` for etcd [secrets encryption](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) instead of AES-CBC.
|
||||
If both are configured then `secretbox` will take precedence for new writes.
|
||||
Old clusters may keep using AES-CBC.
|
||||
To enable `secretbox` you need to add an encryption secret at `cluster.secretboxEncryptionSecret` after an upgrade to Talos 1.3.
|
||||
You should keep `aescbcEncryptionSecret` however, even if `secretbox` is enabled, older data will still be encrypted with AES-CBC.
|
||||
|
||||
How to generate the secret for `secretbox`:
|
||||
|
||||
```bash
|
||||
dd if=/dev/random of=/dev/stdout bs=32 count=1 | base64
|
||||
```
|
||||
|
||||
### Node Labels
|
||||
|
||||
Talos now supports specifying node labels in the machine configuration:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
nodeLabels:
|
||||
rack: rack1a
|
||||
zone: us-east-1a
|
||||
```
|
||||
|
||||
Changes to the node labels will be applied immediately without restarting `kubelet`.
|
||||
|
||||
Talos keeps track of the owned node labels in the `talos.dev/owned-labels` annotation.
|
||||
|
||||
### Static Pod Manifests
|
||||
|
||||
Talos by default (for new clusters) doesn't configure `kubelet` to watch `/etc/kubernetes/manifests` directory for static pod manifests.
|
||||
Talos-managed static pods are served via local HTTP server which prevents potential security vulnerabilities related to malicious static pods manifests
|
||||
being placed to the aforementioned directory.
|
||||
|
||||
Static pods should always be configured in `machine.pods` instead of using `machine.files` to put files to `/etc/kubernetes/manifests` directory.
|
||||
To re-enable support for `/etc/kubernetes/manifests` you may set `machine.kubelet.disableManifestsDirectory`.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
kubelet:
|
||||
disableManifestsDirectory: no
|
||||
```
|
||||
|
||||
## etcd
|
||||
|
||||
### `etcd` Consistency Check
|
||||
|
||||
Talos enables [--experimental-compact-hash-check-enabled](https://github.com/etcd-io/etcd/pull/14120) option by default to improve
|
||||
`etcd` store consistency guarantees.
|
||||
|
||||
This options is only available with `etcd` >= v3.5.5, so Talos doesn't support versions of `etcd` older than v3.5.5 (Talos 1.3.0 defaults to `etcd` v3.5.6).
|
||||
|
||||
### `etcd` Member ID
|
||||
|
||||
Talos now internally handles etcd member removal by member ID instead of member name (hostname).
|
||||
This resolves the case when member name is not accurate or empty (eg: when `etcd` hasn't fully joined yet).
|
||||
|
||||
Command `talosctl etcd remove-member` now accepts member IDs instead of member names.
|
||||
|
||||
A new resource can be used to get member ID of the Talos node:
|
||||
|
||||
```bash
|
||||
$ talosctl get etcdmember
|
||||
NODE NAMESPACE TYPE ID VERSION MEMBER ID
|
||||
10.150.0.4 etcd EtcdMember local 1 143fab7c7ccd2577
|
||||
```
|
||||
|
||||
## CRI (containerd)
|
||||
|
||||
### CRI Configuration Overrides
|
||||
|
||||
Talos no longer supports CRI config overrides placed in `/var/cri/conf.d` directory.
|
||||
|
||||
[New way to add configuration overrides]({{< relref "../talos-guides/configuration/containerd/" >}}) correctly handles merging of containerd/CRI plugin configuration.
|
||||
|
||||
### Registry Mirrors
|
||||
|
||||
Talos had an inconsistency in the way registry mirror endpoints are handled when compared with `containerd` implementation:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
registries:
|
||||
mirrors:
|
||||
docker.io:
|
||||
endpoints:
|
||||
- "https://mirror-registry/v2/mirror.docker.io"
|
||||
```
|
||||
|
||||
Talos would use endpoint `https://mirror-registry/v2/mirror.docker.io`, while `containerd` would use `https://mirror-registry/v2/mirror.docker.io/v2`.
|
||||
This inconsistency is now fixed, and Talos uses same endpoint as `containerd`.
|
||||
|
||||
New `overridePath` configuration is introduced to skip appending `/v2` both on Talos and `containerd` side:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
registries:
|
||||
mirrors:
|
||||
docker.io:
|
||||
endpoints:
|
||||
- "https://mirror-registry/v2/mirror.docker.io"
|
||||
overridePath: true
|
||||
```
|
||||
|
||||
### registry.k8s.io
|
||||
|
||||
Talos now uses `registry.k8s.io` instead of `k8s.gcr.io` for Kubernetes container images.
|
||||
|
||||
See [Kubernetes documentation](https://kubernetes.io/blog/2022/11/28/registry-k8s-io-faster-cheaper-ga/) for additional details.
|
||||
|
||||
If using registry mirrors, or in air-gapped installations you may need to update your configuration.
|
||||
|
||||
## Linux
|
||||
|
||||
### cgroups v1
|
||||
|
||||
Talos always defaults to using `cgroups v2` when Talos doesn't run in a container (when running in a container
|
||||
Talos follows the host `cgroups` mode).
|
||||
Talos can now be forced to use `cgroups v1` by setting boot kernel argument `talos.unified_cgroup_hierarchy=0`:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
install:
|
||||
extraKernelArgs:
|
||||
- "talos.unified_cgroup_hierarchy=0"
|
||||
```
|
||||
|
||||
Current `cgroups` mode can be checked with `talosctl ls /sys/fs/cgroup`:
|
||||
|
||||
`cgroups v1`:
|
||||
|
||||
```text
|
||||
blkio
|
||||
cpu
|
||||
cpuacct
|
||||
cpuset
|
||||
devices
|
||||
freezer
|
||||
hugetlb
|
||||
memory
|
||||
net_cls
|
||||
net_prio
|
||||
perf_event
|
||||
pids
|
||||
```
|
||||
|
||||
`cgroups v2`:
|
||||
|
||||
```text
|
||||
cgroup.controllers
|
||||
cgroup.max.depth
|
||||
cgroup.max.descendants
|
||||
cgroup.procs
|
||||
cgroup.stat
|
||||
cgroup.subtree_control
|
||||
cgroup.threads
|
||||
cpu.stat
|
||||
cpuset.cpus.effective
|
||||
cpuset.mems.effective
|
||||
init
|
||||
io.stat
|
||||
kubepods
|
||||
memory.numa_stat
|
||||
memory.stat
|
||||
podruntime
|
||||
system
|
||||
```
|
||||
|
||||
> Note: `cgroupsv1` is deprecated and it should be used only for compatibility with workloads which don't support `cgroupsv2` yet.
|
||||
|
||||
### Kernel Command Line `ip=` Argument
|
||||
|
||||
Talos now supports referencing interface name via `enxMAC` address notation in the `ip=` argument:
|
||||
|
||||
```text
|
||||
ip=172.20.0.2::172.20.0.1:255.255.255.0::enx7085c2dfbc59
|
||||
```
|
||||
|
||||
Talos correctly handles multiple `ip=` arguments, and also enables forcing DHCP on a specific interface:
|
||||
|
||||
```text
|
||||
vlan=eth0.137:eth0 ip=eth0.137:dhcp
|
||||
```
|
||||
|
||||
### Kernel Module Parameters
|
||||
|
||||
Talos now supports settings kernel module parameters.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
kernel:
|
||||
modules:
|
||||
- name: "br_netfilter"
|
||||
parameters:
|
||||
- nf_conntrack_max=131072
|
||||
```
|
||||
|
||||
### BTF Support
|
||||
|
||||
Talos Linux kernel now ships with [BTF (BPF Type Format)](https://www.containiq.com/post/btf-bpf-type-format) support enabled:
|
||||
|
||||
```bash
|
||||
$ talosctl -n 10.150.0.4 ls -l /sys/kernel/btf
|
||||
NODE MODE UID GID SIZE(B) LASTMOD NAME
|
||||
10.150.0.4 drwxr-xr-x 0 0 0 Dec 13 16:51:19 .
|
||||
10.150.0.4 -r--r--r-- 0 0 11578002 Dec 13 16:51:19 vmlinux
|
||||
```
|
||||
|
||||
This can be used to compile BPF programs against the kernel without kernel sources, or to load relocatable BPF programs.
|
||||
|
||||
## Platform Support
|
||||
|
||||
### Exocale Platform
|
||||
|
||||
Talos adds support for a new platform: [Exoscale](https://www.exoscale.com/).
|
||||
|
||||
Exoscale provides a firewall, TCP load balancer and autoscale groups.
|
||||
It works well with CCM and Kubernetes node autoscaler.
|
||||
|
||||
### Nano Pi R4S
|
||||
|
||||
Talos now supports the Nano Pi R4S SBC.
|
||||
|
||||
### Raspberry Generic Images
|
||||
|
||||
The Raspberry Pi 4 specific image has been deprecated and will be removed in the v1.4 release of Talos.
|
||||
Talos now ships a generic Raspberry Pi image that should support more Raspberry Pi variants.
|
||||
Refer to the [docs]({{< relref "../talos-guides/install/single-board-computers/rpi_generic/" >}}) to find which ones are supported.
|
||||
|
||||
### PlatformMetadata Resource
|
||||
|
||||
Talos now publishes information about the platform it is running on in the `PlatformMetadata` resource:
|
||||
|
||||
```yaml
|
||||
# talosctl get platformmetadata -o yaml
|
||||
spec:
|
||||
platform: equinixMetal
|
||||
hostname: ci-blue-worker-amd64-0
|
||||
region: dc
|
||||
zone: dc13
|
||||
instanceType: c3.medium.x86
|
||||
instanceId: efc0f667-XXX-XXX-XXXX-XXXXXXX
|
||||
providerId: equinixmetal://efc0f667-XXX-XXX-XXXX-XXXXXXX
|
||||
```
|
||||
|
||||
## Networking
|
||||
|
||||
### KubeSpan
|
||||
|
||||
KubeSpan MTU link size is now configurable via `network.kubespan.mtu` setting in the machine configuration.
|
||||
Default KubeSpan MTU assumes that the underlying network MTU is 1500 bytes, so if the underlying network MTU is different, KubeSpan MTU should be adjusted accordingly.
|
||||
|
||||
KubeSpan automatically publishes machine external (public) IP as a machine endpoint (as discovered by connecting to the discovery service), this allows establishing a connection
|
||||
to a machine behind NAT if the KubeSpan port 51820 is forwarded to the machine.
|
||||
|
||||
KubeSpan by default publishes all machine addresses as Wireguard endpoints and finds the set of endpoints that are reachable for each pair of machines.
|
||||
A set of endpoints can be manually filtered via `machine.network.kubespan.filters.endpoints` setting in the machine configuration.
|
||||
|
||||
### Route MTU
|
||||
|
||||
Talos now supports setting MTU for a specific route.
|
||||
|
||||
## `talosctl`
|
||||
|
||||
### Action Tracking
|
||||
|
||||
Now action tracking for commands `talosctl reboot`, `talosctl shutdown`, `talosctl reset` and `talosctl upgrade` is enabled by default.
|
||||
Previous behavior can be restored by setting `--wait=false` flag.
|
||||
|
||||
### `talosctl machineconfig patch`
|
||||
|
||||
A new subcommand, `machineconfig patch` is added to `talosctl` to allow patching of machine configuration.
|
||||
|
||||
It accepts a machineconfig file and a list of patches as input, and outputs the patched machine configuration.
|
||||
|
||||
Patches can be sourced from the command line or from a file.
|
||||
Output can be written to a file or to stdout.
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
talosctl machineconfig patch controlplane.yaml --patch '[{"op":"replace","path":"/cluster/clusterName","value":"patch1"}]' --patch @/path/to/patch2.json
|
||||
```
|
||||
|
||||
Additionally, `talosctl machineconfig gen` subcommand is introduced as an alias to `talosctl gen config`.
|
||||
|
||||
### `talosctl gen config`
|
||||
|
||||
The command `talosctl gen config` now supports generating a single type of output (e.g. controlplane machine configuration) by specifying the `--output-types` flag,
|
||||
which is useful with pre-generated secrets bundle, e.g.:
|
||||
|
||||
```bash
|
||||
$ talosctl gen secrets # this outputs secrets bundle to secrets.yaml
|
||||
$ talosctl gen config mycluster https://mycluster:6443 --with-secrets secrets.yaml --output-types controlplane -o -
|
||||
version: v1alpha1 # Indicates the schema used to decode the contents.
|
||||
debug: false # Enable verbose logging to the console.
|
||||
persist: true # Indicates whether to pull the machine config upon every boot.
|
||||
# Provides machine specific configuration options.
|
||||
machine:
|
||||
...
|
||||
```
|
||||
|
||||
### `talosctl get -o jsonpath`
|
||||
|
||||
The command `talosctl get` now supports `jsonpath` output format:
|
||||
|
||||
```bash
|
||||
$ talosctl -n 10.68.182.3 get address -o jsonpath='{.spec.address}
|
||||
10.68.182.3/31
|
||||
127.0.0.1/8
|
||||
::1/128
|
||||
192.168.11.128/32
|
||||
```
|
||||
|
||||
## Developer Experience
|
||||
|
||||
### New Go Module Path
|
||||
|
||||
Talos now uses `github.com/siderolabs/talos` and `github.com/siderolabs/talos/pkg/machinery` as a Go module path.
|
||||
|
||||
@ -31,72 +31,27 @@ To generate configuration files for a new cluster, we can use the `--with-kubesp
|
||||
This will enable peer discovery and KubeSpan.
|
||||
|
||||
```yaml
|
||||
...
|
||||
# Provides machine specific network configuration options.
|
||||
machine:
|
||||
network:
|
||||
# Configures KubeSpan feature.
|
||||
kubespan:
|
||||
enabled: true # Enable the KubeSpan feature.
|
||||
...
|
||||
# Configures cluster member discovery.
|
||||
cluster:
|
||||
discovery:
|
||||
enabled: true # Enable the cluster membership discovery feature.
|
||||
enabled: true
|
||||
# Configure registries used for cluster member discovery.
|
||||
registries:
|
||||
# Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information
|
||||
kubernetes: {}
|
||||
# Service registry is using an external service to push and pull information about cluster members.
|
||||
kubernetes: # Kubernetes registry is problematic with KubeSpan, if the control plane endpoint is routeable itself via KubeSpan.
|
||||
disabled: true
|
||||
service: {}
|
||||
...
|
||||
# Provides cluster specific configuration options.
|
||||
cluster:
|
||||
id: yui150Ogam0pdQoNZS2lZR-ihi8EWxNM17bZPktJKKE= # Globally unique identifier for this cluster.
|
||||
secret: dAmFcyNmDXusqnTSkPJrsgLJ38W8oEEXGZKM0x6Orpc= # Shared secret of cluster.
|
||||
```
|
||||
|
||||
> The default discovery service is an external service hosted for free by Sidero Labs.
|
||||
> The default value is `https://discovery.talos.dev/`.
|
||||
> Contact Sidero Labs if you need to run this service privately.
|
||||
|
||||
### Upgrading an Existing Cluster
|
||||
### Enabling for an Existing Cluster
|
||||
|
||||
In order to enable KubeSpan for an existing cluster, upgrade to the latest version of Talos ({{< release >}}).
|
||||
Once your cluster is upgraded, the configuration of each node must contain the globally unique identifier, the shared secret for the cluster, and have KubeSpan and discovery enabled.
|
||||
|
||||
> Note: Discovery can be used without KubeSpan, but KubeSpan requires at least one discovery registry.
|
||||
|
||||
#### Talos v0.11 or Less
|
||||
|
||||
If you are migrating from Talos v0.11 or less, we need to generate a cluster ID and secret.
|
||||
|
||||
To generate an `id`:
|
||||
|
||||
```sh
|
||||
$ openssl rand -base64 32
|
||||
EUsCYz+oHNuBppS51P9aKSIOyYvIPmbZK944PWgiyMQ=
|
||||
```
|
||||
|
||||
To generate a `secret`:
|
||||
|
||||
```sh
|
||||
$ openssl rand -base64 32
|
||||
AbdsWjY9i797kGglghKvtGdxCsdllX9CemLq+WGVeaw=
|
||||
```
|
||||
|
||||
Now, update the configuration of each node with the cluster with the generated `id` and `secret`.
|
||||
You should end up with the addition of something like this (your `id` and `secret` should be different):
|
||||
|
||||
```yaml
|
||||
cluster:
|
||||
id: EUsCYz+oHNuBppS51P9aKSIOyYvIPmbZK944PWgiyMQ=
|
||||
secret: AbdsWjY9i797kGglghKvtGdxCsdllX9CemLq+WGVeaw=
|
||||
```
|
||||
|
||||
> Note: This can be applied in immediate mode (no reboot required).
|
||||
|
||||
#### Talos v0.12 or More
|
||||
|
||||
Enable `kubespan` and `discovery`.
|
||||
In order to enable KubeSpan for an existing cluster, enable `kubespan` and `discovery` settings in the machine config for each machine in the cluster (`discovery` is enabled by default):
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
@ -108,6 +63,42 @@ cluster:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
KubeSpan will automatically discovery all cluster members, exchange Wireguard public keys and establish a full mesh network.
|
||||
|
||||
There are a few configuration options available to fine-tune the feature:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
network:
|
||||
kubespan:
|
||||
enabled: false
|
||||
advertiseKubernetesNetworks: false
|
||||
allowDownPeerBypass: false
|
||||
mtu: 1420
|
||||
filters:
|
||||
endpoints:
|
||||
- 0.0.0.0/0
|
||||
- ::/0
|
||||
```
|
||||
|
||||
The setting `advertiseKubernetesNetworks` controls whether the node will advertise Kubernetes service and pod networks to other nodes in the cluster over KubeSpan.
|
||||
It defaults to being disabled, which means KubeSpan only controls the node-to-node traffic, while pod-to-pod traffic is routed and encapsulated by CNI.
|
||||
This setting should not be enabled with Calico and Cilium CNI plugins, as they do their own pod IP allocation which is not visible to KubeSpan.
|
||||
|
||||
The setting `allowDownPeerBypass` controls whether the node will allow traffic to bypass WireGuard if the destination is not connected over KubeSpan.
|
||||
If enabled, there is a risk that traffic will be routed unencrypted if the destination is not connected over KubeSpan, but it allows a workaround
|
||||
for the case where a node is not connected to the KubeSpan network, but still needs to access the cluster.
|
||||
|
||||
The `mtu` setting configures the Wireguard MTU, which defaults to 1420.
|
||||
This default value of 1420 is safe to use when the underlying network MTU is 1500, but if the underlying network MTU is smaller, the KubeSpanMTU should be adjusted accordingly:
|
||||
`KubeSpanMTU = UnderlyingMTU - 80`.
|
||||
|
||||
The `filters` setting allows to hide some endpoints from being advertised over KubeSpan.
|
||||
This is useful when some endpoints are known to be unreachable between the nodes, so that KubeSpan doesn't try to establish a connection to them.
|
||||
Another use-case is hiding some endpoints if nodes can connect on multiple networks, and some of the networks are more preferable than others.
|
||||
|
||||
## Resource Definitions
|
||||
|
||||
### KubeSpanIdentities
|
||||
|
||||
@ -19,7 +19,6 @@ In the following, the focus is on the Talos Linux specific components.
|
||||
| apid | When interacting with Talos, the gRPC API endpoint you interact with directly is provided by `apid`. `apid` acts as the gateway for all component interactions and forwards the requests to `machined`. |
|
||||
| containerd | An industry-standard container runtime with an emphasis on simplicity, robustness, and portability. To learn more, see the [containerd website](https://containerd.io). |
|
||||
| machined | Talos replacement for the traditional Linux init-process. Specially designed to run Kubernetes and does not allow starting arbitrary user services. |
|
||||
| networkd | Handles all of the host level network configuration. The configuration is defined under the `networking` key |
|
||||
| kernel | The Linux kernel included with Talos is configured according to the recommendations outlined in the [Kernel Self Protection Project](http://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project). |
|
||||
| trustd | To run and operate a Kubernetes cluster, a certain level of trust is required. Based on the concept of a 'Root of Trust', `trustd` is a simple daemon responsible for establishing trust within the system. |
|
||||
| udevd | Implementation of `eudev` into `machined`. `eudev` is Gentoo's fork of udev, systemd's device file manager for the Linux kernel. It manages device nodes in /dev and handles all user space actions when adding or removing devices. To learn more, see the [Gentoo Wiki](https://wiki.gentoo.org/wiki/Eudev). |
|
||||
@ -91,22 +90,13 @@ Only the services necessary to run Kubernetes and manage the node are available.
|
||||
This includes:
|
||||
|
||||
- containerd
|
||||
- etcd
|
||||
- [kubelet](https://kubernetes.io/docs/concepts/overview/components/)
|
||||
- networkd
|
||||
- trustd
|
||||
- udevd
|
||||
|
||||
### networkd
|
||||
|
||||
Networkd handles all of the host level network configuration.
|
||||
The configuration is defined under the `networking` key.
|
||||
|
||||
By default, we attempt to issue a DHCP request for every interface on the server.
|
||||
This can be overridden by supplying one of the following kernel arguments:
|
||||
|
||||
- `talos.network.interface.ignore` - specify a list of interfaces to skip discovery on
|
||||
- `ip` - `ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>:<dns0-ip>:<dns1-ip>:<ntp0-ip>` as documented in the [kernel here](https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt)
|
||||
- ex, `ip=10.0.0.99:::255.0.0.0:control-1:eth0:off:10.0.0.1`
|
||||
The `machined` process handles all machine configuration, API handling, resource and controller management.
|
||||
|
||||
### kernel
|
||||
|
||||
|
||||
@ -191,7 +191,7 @@ KubeSpan is using only two bits of the firewall mark with the mask `0x00000060`.
|
||||
>
|
||||
> At the moment of the writing, it was confirmed that Calico CNI is using bits `0xffff0000` and
|
||||
> Cilium CNI is using bits `0xf00`, so KubeSpan is compatible with both.
|
||||
> Flannel CNI doesn't use firewall mark at all.
|
||||
> Flannel CNI uses `0x4000` mask, so it is also compatible.
|
||||
|
||||
In the routing rules table, we match on the mark `0x40` with the mask `0x60`:
|
||||
|
||||
|
||||
@ -13,14 +13,13 @@ Several of these are enforced by the Kernel Self Protection Project [KSPP](https
|
||||
**Required** parameters:
|
||||
|
||||
* `talos.platform`: can be one of `aws`, `azure`, `container`, `digitalocean`, `equinixMetal`, `gcp`, `hcloud`, `metal`, `nocloud`, `openstack`, `oracle`, `scaleway`, `upcloud`, `vmware` or `vultr`
|
||||
* `init_on_alloc=1`: required by KSPP
|
||||
* `slab_nomerge`: required by KSPP
|
||||
* `pti=on`: required by KSPP
|
||||
|
||||
**Recommended** parameters:
|
||||
|
||||
* `init_on_free=1`: advised by KSPP if minimizing stale data lifetime is
|
||||
important
|
||||
* `init_on_alloc=1`: advised by KSPP, enabled by default in kernel config
|
||||
* `init_on_free=1`: advised by KSPP, enabled by default in kernel config
|
||||
|
||||
### Available Talos-specific parameters
|
||||
|
||||
|
||||
@ -43,12 +43,12 @@ Container logs for Kubernetes pods can be retrieved with `talosctl logs -k` comm
|
||||
```sh
|
||||
$ talosctl -n 172.20.1.2 containers -k
|
||||
NODE NAMESPACE ID IMAGE PID STATUS
|
||||
172.20.1.2 k8s.io kube-system/kube-flannel-dk6d5 k8s.gcr.io/pause:3.5 1329 SANDBOX_READY
|
||||
172.20.1.2 k8s.io kube-system/kube-flannel-dk6d5 registry.k8s.io/pause:3.6 1329 SANDBOX_READY
|
||||
172.20.1.2 k8s.io └─ kube-system/kube-flannel-dk6d5:install-cni ghcr.io/siderolabs/install-cni:v0.7.0-alpha.0-1-g2bb2efc 0 CONTAINER_EXITED
|
||||
172.20.1.2 k8s.io └─ kube-system/kube-flannel-dk6d5:install-config quay.io/coreos/flannel:v0.13.0 0 CONTAINER_EXITED
|
||||
172.20.1.2 k8s.io └─ kube-system/kube-flannel-dk6d5:kube-flannel quay.io/coreos/flannel:v0.13.0 1610 CONTAINER_RUNNING
|
||||
172.20.1.2 k8s.io kube-system/kube-proxy-gfkqj k8s.gcr.io/pause:3.5 1311 SANDBOX_READY
|
||||
172.20.1.2 k8s.io └─ kube-system/kube-proxy-gfkqj:kube-proxy k8s.gcr.io/kube-proxy:v{{< k8s_release >}} 1379 CONTAINER_RUNNING
|
||||
172.20.1.2 k8s.io kube-system/kube-proxy-gfkqj registry.k8s.io/pause:3.5 1311 SANDBOX_READY
|
||||
172.20.1.2 k8s.io └─ kube-system/kube-proxy-gfkqj:kube-proxy registry.k8s.io/kube-proxy:v{{< k8s_release >}} 1379 CONTAINER_RUNNING
|
||||
|
||||
$ talosctl -n 172.20.1.2 logs -k kube-system/kube-proxy-gfkqj:kube-proxy
|
||||
172.20.1.2: 2021-11-30T19:13:20.567825192Z stderr F I1130 19:13:20.567737 1 server_others.go:138] "Detected node IP" address="172.20.0.3"
|
||||
|
||||
@ -1,15 +1,46 @@
|
||||
---
|
||||
title: "Managing PKI"
|
||||
title: "Managing Talos PKI"
|
||||
description: "How to manage Public Key Infrastructure"
|
||||
aliases:
|
||||
- ../../guides/managing-pki
|
||||
---
|
||||
|
||||
## Generating an Administrator Key Pair
|
||||
## Generating New Client Configuration
|
||||
|
||||
In order to create a key pair, you will need the root CA.
|
||||
### Using Controlplane Node
|
||||
|
||||
If you have a valid (not expired) `talosconfig` with `os:admin` role,
|
||||
a new client configuration file can be generated with `talosctl config new` against
|
||||
any controlplane node:
|
||||
|
||||
```shell
|
||||
talosctl -n CP1 config new talosconfig-reader --roles os:reader --crt-ttl 24h
|
||||
```
|
||||
|
||||
A specific [role]({{< relref "rbac" >}}) and certificate lifetime can be specified.
|
||||
|
||||
### From Secrets Bundle
|
||||
|
||||
If a secrets bundle (`secrets.yaml` from `talosctl gen secrets`) was saved while
|
||||
[generating machine configuration]({{< relref "../../introduction/getting-started/#configure-talos ">}}):
|
||||
|
||||
```shell
|
||||
talosctl gen config --with-secrets secrets.yaml --output-types talosconfig -o talosconfig <cluster-name> https://<cluster-endpoint>
|
||||
```
|
||||
|
||||
> Note: `<cluster-name>` and `<cluster-endpoint>` arguments don't matter, as they are not used for `talosconfig`.
|
||||
|
||||
### From Control Plane Machine Configuration
|
||||
|
||||
In order to create a new key pair for client configuration, you will need the root Talos API CA.
|
||||
The base64 encoded CA can be found in the control plane node's configuration file.
|
||||
Save the the CA public key, and CA private key as `ca.crt`, and `ca.key` respectively:
|
||||
|
||||
```shell
|
||||
yq eval .machine.ca.crt controlplane.yaml | base64 -d > ca.crt
|
||||
yq eval .machine.ca.key controlplane.yaml | base64 -d > ca.key
|
||||
```
|
||||
|
||||
Save the the CA public key, and CA private key as `ca.crt`, and `ca.key` respectively.
|
||||
Now, run the following commands to generate a certificate:
|
||||
|
||||
```bash
|
||||
@ -18,34 +49,24 @@ talosctl gen csr --key admin.key --ip 127.0.0.1
|
||||
talosctl gen crt --ca ca --csr admin.csr --name admin
|
||||
```
|
||||
|
||||
Now, base64 encode `admin.crt`, and `admin.key`:
|
||||
Put the base64-encoded files to the respective location to the `talosconfig`:
|
||||
|
||||
```bash
|
||||
cat admin.crt | base64
|
||||
cat admin.key | base64
|
||||
```yaml
|
||||
context: mycluster
|
||||
contexts:
|
||||
mycluster:
|
||||
endpoints:
|
||||
- CP1
|
||||
- CP2
|
||||
ca: <base64-encoded ca.crt>
|
||||
crt: <base64-encoded admin.crt>
|
||||
key: <base64-encoded admin.key>
|
||||
```
|
||||
|
||||
You can now set the `crt` and `key` fields in the `talosconfig` to the base64 encoded strings.
|
||||
|
||||
## Renewing an Expired Administrator Certificate
|
||||
|
||||
In order to renew the certificate, you will need the root CA, and the admin private key.
|
||||
The base64 encoded key can be found in any one of the control plane node's configuration file.
|
||||
Where it is exactly will depend on the specific version of the configuration file you are using.
|
||||
By default admin `talosconfig` certificate is valid for 365 days, while cluster CAs are valid for 10 years.
|
||||
In order to prevent admin `talosconfig` from expiring, renew the client config before expiration using `talosctl config new` command described above.
|
||||
|
||||
Save the the CA public key, CA private key, and admin private key as `ca.crt`, `ca.key`, and `admin.key` respectively.
|
||||
Now, run the following commands to generate a certificate:
|
||||
|
||||
```bash
|
||||
talosctl gen csr --key admin.key --ip 127.0.0.1
|
||||
talosctl gen crt --ca ca --csr admin.csr --name admin
|
||||
```
|
||||
|
||||
You should see `admin.crt` in your current directory.
|
||||
Now, base64 encode `admin.crt`:
|
||||
|
||||
```bash
|
||||
cat admin.crt | base64
|
||||
```
|
||||
|
||||
You can now set the certificate in the `talosconfig` to the base64 encoded string.
|
||||
If the `talosconfig` is expired or lost, you can still generate a new one using either the `secrets.yaml`
|
||||
secrets bundle or the control plane node's configuration file using methods described above.
|
||||
|
||||
@ -311,12 +311,18 @@ talosctl ... --patch '[{"op": "add", "path": "/machine/network/hostname", "value
|
||||
If multiple config patches are specified, they are applied in the order of appearance.
|
||||
The format of the patch (JSON patch or strategic merge patch) is detected automatically.
|
||||
|
||||
Generated machine configuration can be patched with `talosctl gen config`:
|
||||
Talos machine configuration can be patched at the moment of generation with `talosctl gen config`:
|
||||
|
||||
```shell
|
||||
talosctl gen config test-cluster https://172.20.0.1:6443 --config-patch @all.yaml --config-patch-control-plane @cp.yaml --config-patch-worker @worker.yaml
|
||||
```
|
||||
|
||||
Generated machine configuration can also be patched after the fact with `talosctl machineconfig patch`
|
||||
|
||||
```shell
|
||||
talosctl machineconfig patch worker.yaml --patch @patch.yaml -o worker1.yaml
|
||||
```
|
||||
|
||||
Machine configuration on the running Talos node can be patched with `talosctl patch`:
|
||||
|
||||
```shell
|
||||
|
||||
@ -7,7 +7,7 @@ aliases:
|
||||
|
||||
In this guide we will create a set of local caching Docker registry proxies to minimize local cluster startup time.
|
||||
|
||||
When running Talos locally, pulling images from Docker registries might take a significant amount of time.
|
||||
When running Talos locally, pulling images from container registries might take a significant amount of time.
|
||||
We spin up local caching pass-through registries to cache images and configure a local Talos cluster to use those proxies.
|
||||
A similar approach might be used to run Talos in production in air-gapped environments.
|
||||
It can be also used to verify that all the images are available in local registries.
|
||||
@ -27,7 +27,7 @@ The follow are requirements for creating the set of caching proxies:
|
||||
|
||||
## Launch the Caching Docker Registry Proxies
|
||||
|
||||
Talos pulls from `docker.io`, `registry.k8s.io`, `quay.io`, `gcr.io`, and `ghcr.io` by default.
|
||||
Talos pulls from `docker.io`, `registry.k8s.io`, `gcr.io`, and `ghcr.io` by default.
|
||||
If your configuration is different, you might need to modify the commands below:
|
||||
|
||||
```bash
|
||||
@ -41,11 +41,6 @@ docker run -d -p 5001:5000 \
|
||||
--restart always \
|
||||
--name registry-registry.k8s.io registry:2
|
||||
|
||||
docker run -d -p 5002:5000 \
|
||||
-e REGISTRY_PROXY_REMOTEURL=https://quay.io \
|
||||
--restart always \
|
||||
--name registry-quay.io registry:2.5
|
||||
|
||||
docker run -d -p 5003:5000 \
|
||||
-e REGISTRY_PROXY_REMOTEURL=https://gcr.io \
|
||||
--restart always \
|
||||
@ -58,7 +53,6 @@ docker run -d -p 5004:5000 \
|
||||
```
|
||||
|
||||
> Note: Proxies are started as docker containers, and they're automatically configured to start with Docker daemon.
|
||||
> Please note that `quay.io` proxy doesn't support recent Docker image schema, so we run older registry image version (2.5).
|
||||
|
||||
As a registry container can only handle a single upstream Docker registry, we launch a container per upstream, each on its own
|
||||
host port (5000, 5001, 5002, 5003 and 5004).
|
||||
@ -72,7 +66,6 @@ As registry containers expose their ports on the host, we can use bridge IP to d
|
||||
sudo talosctl cluster create --provisioner qemu \
|
||||
--registry-mirror docker.io=http://10.5.0.1:5000 \
|
||||
--registry-mirror registry.k8s.io=http://10.5.0.1:5001 \
|
||||
--registry-mirror quay.io=http://10.5.0.1:5002 \
|
||||
--registry-mirror gcr.io=http://10.5.0.1:5003 \
|
||||
--registry-mirror ghcr.io=http://10.5.0.1:5004
|
||||
```
|
||||
@ -92,11 +85,32 @@ On Linux, the docker bridge address can be inspected with `ip addr show docker0`
|
||||
talosctl cluster create --provisioner docker \
|
||||
--registry-mirror docker.io=http://172.17.0.1:5000 \
|
||||
--registry-mirror registry.k8s.io=http://172.17.0.1:5001 \
|
||||
--registry-mirror quay.io=http://172.17.0.1:5002 \
|
||||
--registry-mirror gcr.io=http://172.17.0.1:5003 \
|
||||
--registry-mirror ghcr.io=http://172.17.0.1:5004
|
||||
```
|
||||
|
||||
## Machine Configuration
|
||||
|
||||
The caching registries can be configured via machine configuration [patch]({{< relref "patching" >}}), equivalent to the command line flags above:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
registries:
|
||||
mirrors:
|
||||
docker.io:
|
||||
endpoints:
|
||||
- http://10.5.0.1:5000
|
||||
gcr.io:
|
||||
endpoints:
|
||||
- http://10.5.0.1:5003
|
||||
ghcr.io:
|
||||
endpoints:
|
||||
- http://10.5.0.1:5004
|
||||
registry.k8s.io:
|
||||
endpoints:
|
||||
- http://10.5.0.1:5001
|
||||
```
|
||||
|
||||
## Cleaning Up
|
||||
|
||||
To cleanup, run:
|
||||
@ -104,10 +118,54 @@ To cleanup, run:
|
||||
```bash
|
||||
docker rm -f registry-docker.io
|
||||
docker rm -f registry-registry.k8s.io
|
||||
docker rm -f registry-quay.io
|
||||
docker rm -f registry-gcr.io
|
||||
docker rm -f registry-ghcr.io
|
||||
```
|
||||
|
||||
> Note: Removing docker registry containers also removes the image cache.
|
||||
> So if you plan to use caching registries, keep the containers running.
|
||||
|
||||
## Using Harbor as a Caching Registry
|
||||
|
||||
[Harbor](https://goharbor.io/) is an open source container registry that can be used as a caching proxy.
|
||||
Harbor supports configuring multiple upstream registries, so it can be used to cache multiple registries at once behind a single endpoint.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
As Harbor puts a registry name in the pull image path, we need to set `overridePath: true` to prevent Talos and containerd from appending `/v2` to the path.
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
registries:
|
||||
mirrors:
|
||||
docker.io:
|
||||
endpoints:
|
||||
- http://harbor/v2/proxy-docker.io
|
||||
overridePath: true
|
||||
ghcr.io:
|
||||
endpoints:
|
||||
- http://harbor/v2/proxy-ghcr.io
|
||||
overridePath: true
|
||||
gcr.io:
|
||||
endpoints:
|
||||
- http://harbor/v2/proxy-gcr.io
|
||||
overridePath: true
|
||||
registry.k8s.io:
|
||||
endpoints:
|
||||
- http://harbor/v2/proxy-registry.k8s.io
|
||||
overridePath: true
|
||||
```
|
||||
|
||||
The Harbor external endpoint (`http://harbor` in this example) can be configured with authentication or custom TLS:
|
||||
|
||||
```yaml
|
||||
machine:
|
||||
registries:
|
||||
config:
|
||||
harbor:
|
||||
auth:
|
||||
username: admin
|
||||
password: password
|
||||
```
|
||||
|
||||
@ -0,0 +1,16 @@
|
||||
---
|
||||
title: "ISO"
|
||||
description: "Booting Talos on bare-metal with ISO."
|
||||
---
|
||||
|
||||
Talos can be installed on bare-metal machine using an ISO image.
|
||||
ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/).
|
||||
|
||||
Talos doesn't install itself to disk when booted from an ISO until the machine configuration is applied.
|
||||
|
||||
Please follow the [getting started guide]({{< relref "../../../introduction/getting-started" >}}) for the generic steps on how to install Talos.
|
||||
|
||||
> Note: If there is already a Talos installation on the disk, the machine will boot into that installation when booting from a Talos ISO.
|
||||
> The boot order should prefer disk over ISO, or the ISO should be removed after the installation to make Talos boot from disk.
|
||||
|
||||
See [kernel parameters reference]({{< relref "../../../reference/kernel" >}}) for the list of kernel parameters supported by Talos.
|
||||
@ -0,0 +1,37 @@
|
||||
---
|
||||
title: "PXE"
|
||||
description: "Booting Talos over the network on bare-metal with PXE."
|
||||
---
|
||||
|
||||
Talos can be installed on bare-metal using PXE service.
|
||||
There are two more detailed guides for PXE booting using [Matchbox]({{< relref "matchbox">}}) and [Digital Rebar]({{< relref "digital-rebar">}}).
|
||||
|
||||
This guide describes generic steps for PXE booting Talos on bare-metal.
|
||||
|
||||
First, download the `vmlinuz` and `initramfs` assets from the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/).
|
||||
Set up the machines to PXE boot from the network (usually by setting the boot order in the BIOS).
|
||||
There might be options specific to the hardware being used, booting in BIOS or UEFI mode, using iPXE, etc.
|
||||
|
||||
Talos requires the following kernel parameters to be set on the initial boot:
|
||||
|
||||
* `talos.platform=metal`
|
||||
* `slab_nomerge`
|
||||
* `pti=on`
|
||||
|
||||
When booted from the network without machine configuration, Talos will start in maintenance mode.
|
||||
|
||||
Please follow the [getting started guide]({{< relref "../../../introduction/getting-started" >}}) for the generic steps on how to install Talos.
|
||||
|
||||
See [kernel parameters reference]({{< relref "../../../reference/kernel" >}}) for the list of kernel parameters supported by Talos.
|
||||
|
||||
> Note: If there is already a Talos installation on the disk, the machine will boot into that installation when booting from network.
|
||||
> The boot order should prefer disk over network.
|
||||
|
||||
Talos can automatically fetch the machine configuration from the network on the initial boot using `talos.config` kernel parameter.
|
||||
A metadata service (HTTP service) can be implemented to deliver customized configuration to each node for example by using the MAC address of the node:
|
||||
|
||||
```text
|
||||
talos.config=https://metadata.service/talos/config?mac=${mac}
|
||||
```
|
||||
|
||||
> Note: The `talos.config` kernel parameter supports other substitution variables, see [kernel parameters reference]({{< relref "../../../reference/kernel" >}}) for the full list.
|
||||
@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "Sidero"
|
||||
description: "Sidero is a project created by the Talos team that has native support for Talos."
|
||||
aliases:
|
||||
title: "Sidero Metal"
|
||||
description: "Sidero Metal is a project created by the Talos team that has native support for Talos."
|
||||
aliases:
|
||||
- ../../../bare-metal-platforms/sidero
|
||||
---
|
||||
|
||||
|
||||
@ -32,11 +32,24 @@ For example, if upgrading from Talos 1.0 to Talos 1.2.4, the recommended upgrade
|
||||
|
||||
* upgrade from 1.0 to latest patch of 1.0 - to v1.0.6
|
||||
* upgrade from v1.0.6 to latest patch of 1.1 - to v1.1.2
|
||||
* upgrade from v1.1.2 to latest patch of 1.2 - to v1.2.4
|
||||
* upgrade from v1.1.2 to v1.2.4
|
||||
|
||||
## Before Upgrade to {{% release %}}
|
||||
|
||||
TBD
|
||||
### Custom CRI Configuration
|
||||
|
||||
If you were using a custom CRI configuration placed under `/var/cri/conf.d` with `.machine.files` setting, this will stop working after an upgrade to Talos 1.3.
|
||||
Talos 1.3 fixes the way CRI configuration is merged, and introduces a [new location for custom CRI configuration]({{< relref "configuration/containerd" >}}).
|
||||
The new location under `/etc/cri/conf.d` will not be accepted by Talos 1.2 and earlier, so if CRI configuration overrides are critical to boot Talos, the following upgrade flow can be used:
|
||||
|
||||
1. Update machine configuration before upgrade in `--mode=staged` (using `talosctl apply-config`, `talosctl patch mc` or `talosctl edit mc`).
|
||||
The new machine configuration will be only applied after the reboot (upgrade).
|
||||
2. Perform an upgrade, after the upgrade Talos will pick up new CRI configuration.
|
||||
|
||||
### Reboot after Upgrade
|
||||
|
||||
The new Talos 1.3.0 kernel has BTF debugging information, so `kexec` support will be disabled on upgrade preventing a `kexec` failure.
|
||||
After an upgrade `kexec` support will work again (if not disabled).
|
||||
|
||||
## Video Walkthrough
|
||||
|
||||
@ -93,7 +106,17 @@ future.
|
||||
|
||||
## Machine Configuration Changes
|
||||
|
||||
TBD
|
||||
All changes to Talos machine configuration are backwards compatible, so you can upgrade Talos and then optionally update the machine configuration to use new features.
|
||||
|
||||
* `.machine.nodeLabels` can be used to control user-defined Kubernetes node labels
|
||||
* `.cluster.secretboxEncryptionSecret` can be set to encrypt Kubernetes secrets with `secretbox` algorithm (see [What's New]({{< relref "../introduction/what-is-new" >}}) for details)
|
||||
* `.machine.kubelet.disableManifestsDirectory` can be enabled to disable kubelet static pods support from `/etc/kubernetes/manifests` directory
|
||||
* `.cluster.apiServer.auditPolicy` configures custom `kube-apiserver` audit policy
|
||||
* `mtu` setting of the network route can be used to control route-specific MTU
|
||||
* `overridePath` can be used on `machine.registries.endpoints` to skip appending `/v2` (see [Pull-Through Cache]({{< relref "configuration/pull-through-cache" >}}))
|
||||
* `.machine.features.apidCheckExtKeyUsage` can be enabled after an upgrade to 1.3 to perform more strict checks on node-to-node Talos apid communication (see [CVE-2022-36103](https://github.com/siderolabs/talos/security/advisories/GHSA-7hgc-php5-77qq))
|
||||
* `.machine.kubespan` provdes new settings for [KubeSpan]({{< relref "../kubernetes-guides/network/kubespan" >}}) support
|
||||
* `.machine.kernel.modules` provides support to pass parameters to kernel modules on load
|
||||
|
||||
## Upgrade Sequence
|
||||
|
||||
|
||||
BIN
website/static/images/harbor-endpoints.png
Normal file
BIN
website/static/images/harbor-endpoints.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 176 KiB |
BIN
website/static/images/harbor-projects.png
Normal file
BIN
website/static/images/harbor-projects.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 217 KiB |
Loading…
x
Reference in New Issue
Block a user