diff --git a/hack/start-registry-proxies.sh b/hack/start-registry-proxies.sh index 92329d445..ed4e48393 100755 --- a/hack/start-registry-proxies.sh +++ b/hack/start-registry-proxies.sh @@ -14,11 +14,6 @@ docker run -d -p 5001:5000 \ --restart always \ --name registry-registry.k8s.io registry:2 -docker run -d -p 5002:5000 \ - -e REGISTRY_PROXY_REMOTEURL=https://quay.io \ - --restart always \ - --name registry-quay.io registry:2.5 - docker run -d -p 5003:5000 \ -e REGISTRY_PROXY_REMOTEURL=https://gcr.io \ --restart always \ diff --git a/website/content/v1.3/advanced/air-gapped.md b/website/content/v1.3/advanced/air-gapped.md index 18776f3b3..79008ec99 100644 --- a/website/content/v1.3/advanced/air-gapped.md +++ b/website/content/v1.3/advanced/air-gapped.md @@ -104,7 +104,6 @@ Identify all registry prefixes from `talosctl images`, for example: - `gcr.io` - `ghcr.io` - `registry.k8s.io` -- `quay.io` The `talosctl cluster create` command provides conveniences for common configuration options. The only required flag for this guide is `--registry-mirror =http://10.5.0.1:6000` which redirects every pull request to the internal registry, this flag @@ -117,7 +116,6 @@ $ sudo --preserve-env=HOME talosctl cluster create --provisioner=qemu --install- --registry-mirror gcr.io=http://10.5.0.1:6000 \ --registry-mirror ghcr.io=http://10.5.0.1:6000 \ --registry-mirror registry.k8s.io=http://10.5.0.1:6000 \ - --registry-mirror quay.io=http://10.5.0.1:6000 validating CIDR and reserving IPs generating PKI and tokens creating state directory in "/home/user/.talos/clusters/talos-default" @@ -157,11 +155,10 @@ machine: registry.k8s.io: endpoints: - http://10.5.0.1:6000/ - quay.io: - endpoints: - - http://10.5.0.1:6000/ ... ``` Other implementations of Docker registry can be used in place of the Docker `registry` image used above to run the registry. If required, auth can be configured for the internal registry (and custom TLS certificates if needed). + +Please see [pull-through cache guide]({{< relref "../talos-guides/configuration/pull-through-cache" >}}) for an example using Harbor container registry with Talos. diff --git a/website/content/v1.3/advanced/developing-talos.md b/website/content/v1.3/advanced/developing-talos.md index 0b0ee4037..4cca23379 100644 --- a/website/content/v1.3/advanced/developing-talos.md +++ b/website/content/v1.3/advanced/developing-talos.md @@ -67,7 +67,6 @@ sudo --preserve-env=HOME _out/talosctl-linux-amd64 cluster create \ --cidr=172.20.0.0/24 \ --registry-mirror docker.io=http://172.20.0.1:5000 \ --registry-mirror kregistry.k8s.io=http://172.20.0.1:5001 \ - --registry-mirror quay.io=http://172.20.0.1:5002 \ --registry-mirror gcr.io=http://172.20.0.1:5003 \ --registry-mirror ghcr.io=http://172.20.0.1:5004 \ --registry-mirror 127.0.0.1:5005=http://172.20.0.1:5005 \ diff --git a/website/content/v1.3/introduction/getting-started.md b/website/content/v1.3/introduction/getting-started.md index 2ad284304..3afea3c41 100644 --- a/website/content/v1.3/introduction/getting-started.md +++ b/website/content/v1.3/introduction/getting-started.md @@ -167,10 +167,18 @@ kernel commandline parameters. See [required kernel parameters]({{< relref "../reference/kernel" >}}). If creating [EC2 kubernetes clusters]({{< relref "../talos-guides/install/cloud-platforms/aws/" >}}), the configuration file can be passed in as `--user-data` to the `aws ec2 run-instances` command. -In any case, we need to generate the configuration which is to be provided: +In any case, we need to generate the configuration which is to be provided. +We start with generating a secrets bundle which should be saved in a secure location and used +to generate machine or client configuration at any time: ```sh - talosctl gen config cluster-name cluster-endpoint +talosctl gen secrets -o secrets.yaml +``` + +Now, we can generate the machine configuration for each node: + +```sh +talosctl gen config --with-secrets secrets.yaml ``` Here, `cluster-name` is an arbitrary name for the cluster, used @@ -185,7 +193,7 @@ and port. For example: ```sh - talosctl gen config my-cluster https://192.168.64.15:6443 +$ talosctl gen config --with-secrets secrets.yaml my-cluster https://192.168.64.15:6443 generating PKI and tokens created /Users/taloswork/controlplane.yaml created /Users/taloswork/worker.yaml @@ -225,7 +233,7 @@ A common example is needing to change the default installation disk. If you try to to apply the machine config to a node, and get an error like the below, you need to specify a different installation disk: ```sh -talosctl apply-config --insecure -n 192.168.64.8 --file controlplane.yaml +$ talosctl apply-config --insecure -n 192.168.64.8 --file controlplane.yaml error applying new configuration: rpc error: code = InvalidArgument desc = configuration validation failed: 1 error occurred: * specified install disk does not exist: "/dev/sda" ``` @@ -237,7 +245,7 @@ You can verify which disks your nodes have by using the `talosctl disks --insecu For example: ```sh -talosctl -n 192.168.64.8 disks --insecure +$ talosctl -n 192.168.64.8 disks --insecure DEV MODEL SERIAL TYPE UUID WWID MODALIAS NAME SIZE BUS_PATH /dev/vda - - HDD - - virtio:d00000002v00001AF4 - 69 GB /pci0000:00/0000:00:06.0/virtio2/ ``` @@ -251,28 +259,60 @@ install: to reflect `vda` instead of `sda`. +### Customizing Machine Configuration + +The generated machine configuration provides sane defaults for most cases, but machine configuration +can be modified to fit specific needs. + +Some machine configuration options are available as flags for the `talosctl gen config` command, +for example setting a specific Kubernetes version: + +```sh +talosctl gen config --with-secrets secrets.yaml --kubernetes-version 1.25.4 my-cluster https://192.168.64.15:6443 +``` + +Other modifications are done with [machine configuration patches]({{< relref "../talos-guides/configuration/patching" >}}). +Machine configuration patches can be applied with `talosctl gen config` command: + +```sh +talosctl gen config --with-secrets secrets.yaml --config-patch-control-plane @cni.patch my-cluster https://192.168.64.15:6443 +``` + +> Note: `@cni.patch` means that the patch is read from a file named `cni.patch`. + #### Machine Configs as Templates Individual machines may need different settings: for instance, each may have a different [static IP address]({{< relref "../advanced/advanced-networking/#static-addressing" >}}). -When different files are needed for machines of the same type, simply -copy the source template (`controlplane.yaml` or `worker.yaml`) and make whatever -modifications are needed. +When different files are needed for machines of the same type, there are two supported flows: -For instance, if you had three control plane nodes and three worker nodes, you -may do something like this: +1. Use the `talosctl gen config` command to generate a template, and then patch + the template for each machine with `talosctl machineconfig patch`. +2. Generate each machine configuration file separately with `talosctl gen config` while applying patches. -```bash -for i in $(seq 0 2); do - cp controlplane.yaml cp$i.yaml -end -for i in $(seq 0 2); do - cp worker.yaml w$i.yaml -end +For example, given a machine configuration patch which sets the static machine hostname: + +```yaml +# worker1.patch +machine: + network: + hostname: worker1 ``` -Then modify each file as needed. +Either of the following commands will generate a worker machine configuration file with the hostname set to `worker1`: + +```bash +$ talosctl gen config --with-secrets secrets.yaml my-cluster https://192.168.64.15:6443 +created /Users/taloswork/controlplane.yaml +created /Users/taloswork/worker.yaml +created /Users/taloswork/talosconfig +$ talosctl machineconfig patch worker.yaml --patch @worker1.patch --output worker1.yaml +``` + +```sh +talosctl gen config --with-secrets secrets.yaml --config-patch-worker @worker1.patch --output-types worker -o worker1.yaml my-cluster https://192.168.64.15:6443 +``` ### Apply Configuration diff --git a/website/content/v1.3/introduction/support-matrix.md b/website/content/v1.3/introduction/support-matrix.md index 335d21e77..60d075cc7 100644 --- a/website/content/v1.3/introduction/support-matrix.md +++ b/website/content/v1.3/introduction/support-matrix.md @@ -7,22 +7,20 @@ description: "Table of supported Talos Linux versions and respective platforms." | Talos Version | 1.3 | 1.2 | |----------------------------------------------------------------------------------------------------------------|------------------------------------|------------------------------------| | Release Date | 2022-12-01 | 2022-09-01 (1.2.0) | -| End of Community Support | 1.4.0 release (2023-03-01, TBD) | 1.3.0 release (2022-12-01, TBD) | +| End of Community Support | 1.4.0 release (2023-03-15, TBD) | 1.3.0 release (2022-12-15, TBD) | | Enterprise Support | [offered by Sidero Labs Inc.](https://www.siderolabs.com/support/) | [offered by Sidero Labs Inc.](https://www.siderolabs.com/support/) | | Kubernetes | 1.26, 1.25, 1.24 | 1.25, 1.24, 1.23 | | Architecture | amd64, arm64 | amd64, arm64 | | **Platforms** | | | -| - cloud | AWS, GCP, Azure, Digital Ocean, Exoscale, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | +| - cloud | AWS, GCP, Azure, Digital Ocean, Exoscale, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | AWS, GCP, Azure, Digital Ocean, Hetzner, OpenStack, Oracle Cloud, Scaleway, Vultr, Upcloud | | - bare metal | x86: BIOS, UEFI; arm64: UEFI; boot: ISO, PXE, disk image | x86: BIOS, UEFI; arm64: UEFI; boot: ISO, PXE, disk image | | - virtualized | VMware, Hyper-V, KVM, Proxmox, Xen | VMware, Hyper-V, KVM, Proxmox, Xen | | - SBCs | Banana Pi M64, Jetson Nano, Libre Computer Board ALL-H3-CC, Nano Pi R4S, Pine64, Pine64 Rock64, Radxa ROCK Pi 4c, Raspberry Pi 4B, Raspberry Pi Compute Module 4 | Banana Pi M64, Jetson Nano, Libre Computer Board ALL-H3-CC, Pine64, Pine64 Rock64, Radxa ROCK Pi 4c, Raspberry Pi 4B, Raspberry Pi Compute Module 4 | | - local | Docker, QEMU | Docker, QEMU | | **Cluster API** | | | -| [CAPI Bootstrap Provider Talos](https://github.com/siderolabs/cluster-api-bootstrap-provider-talos) | >= 0.5.5 | >= 0.5.5 | -| [CAPI Control Plane Provider Talos](https://github.com/siderolabs/cluster-api-control-plane-provider-talos) | >= 0.4.9 | >= 0.4.9 | -| [Sidero](https://www.sidero.dev/) | >= 0.5.5 | >= 0.5.5 | -| **UI** | | | -| [Theila](https://github.com/siderolabs/theila) | ✓ | ✓ | +| [CAPI Bootstrap Provider Talos](https://github.com/siderolabs/cluster-api-bootstrap-provider-talos) | >= 0.5.6 | >= 0.5.5 | +| [CAPI Control Plane Provider Talos](https://github.com/siderolabs/cluster-api-control-plane-provider-talos) | >= 0.4.10 | >= 0.4.9 | +| [Sidero](https://www.sidero.dev/) | >= 0.5.7 | >= 0.5.5 | ## Platform Tiers @@ -45,6 +43,7 @@ description: "Table of supported Talos Linux versions and respective platforms." ### Tier 3 +* Exoscale * Hetzner * nocloud * Oracle Cloud diff --git a/website/content/v1.3/introduction/system-requirements.md b/website/content/v1.3/introduction/system-requirements.md index 04b2ac7dd..dbd43933d 100644 --- a/website/content/v1.3/introduction/system-requirements.md +++ b/website/content/v1.3/introduction/system-requirements.md @@ -12,18 +12,21 @@ description: "Hardware requirements for running Talos Linux." Role Memory Cores + System Disk Control Plane - 2GB + 2 GiB 2 + 10 GiB Worker - 1GB + 1 GiB 1 + 10 GiB @@ -36,18 +39,21 @@ description: "Hardware requirements for running Talos Linux." Role Memory Cores + System Disk Control Plane - 4GB + 4 GiB 4 + 100 GiB Worker - 2GB + 2 GiB 2 + 100 GiB diff --git a/website/content/v1.3/introduction/theila.md b/website/content/v1.3/introduction/theila.md deleted file mode 100644 index 46dcace1c..000000000 --- a/website/content/v1.3/introduction/theila.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: Theila UI for Talos -weight: 35 -description: "An intro to Theila - a UI for Talos clusters." ---- - -Once you have a Talos cluster running, you may find it easier to get insights on your cluster(s) using a visual user interface rather than the `talosctl` CLI. -For this, Sidero Labs provides [Theila](https://github.com/siderolabs/theila), a simple, single-binary web-based visual user interface for Talos clusters. - -## Prerequisites - -You should have a Talos cluster up & running, and the `talosconfig` file for Theila to access it. - -## Installation - -Theila is published as a single static binary compiled for various platforms and architectures, as well as a container image. - -### Binary - -You can download the correct binary for your system from the [releases](https://github.com/siderolabs/theila/releases) page, or use the following commands in your terminal. - -```bash -curl -Lo /usr/local/bin/theila https://github.com/siderolabs/theila/releases/download/{{< theila_release >}}/theila-$(uname -s | tr "[:upper:]" "[:lower:]")-amd64 -chmod +x /usr/local/bin/theila -``` - -## Use - -Once installed, you can run Theila by simply running it. - - - - - -{{< tabpane lang="bash" >}} -{{< tab header="Binary" >}} -# address and port are not required and default to the values shown -theila --address 127.0.0.1 --port 8080 -{{< /tab >}} -{{< tab header="Docker" >}} -docker run --rm --volume ${HOME}/.talos/config:/opt/talosconfig:ro --env TALOSCONFIG=/opt/talosconfig --publish 8080:8080 ghcr.io/siderolabs/theila --address 0.0.0.0 -{{< /tab >}} -{{< /tabpane >}} - -Once it is running you should be able to point a browser at [http://localhost:8080](http://localhost:8080) to open the Theila UI. - -### Clusters - -You can navigate around various Talos clusters using the menu at the upper-left corner (see 1.1), then selecting the specific cluster from the list (see 1.2). -![Fig 1 Talos cluster selection with Theila](/images/theila-cluster-selection.png) - -### Cluster Overview - -Clicking on the "Overview" option in the menu (see 2.1) will display an overview of resource use & health of the cluster. -![Fig 2 Talos cluster overview](/images/theila-cluster-overview.png) - -### Nodes - -Entering the "Nodes" section on the menu (see 3.1) will give a list of nodes in the cluster (see 3.2), along with information such as IP address, status, and any roles assigned to the node. -Opening the node menu (see 3.3) show the actions that can be taken on a specific node. -![Fig 3 Node list](/images/theila-nodes.png) - -Clicking on a specific node name in the node list will open the node detail page for more information on each specific node (see 4.1), including running services and their logs (see 4.2). -![Fig 4 Node detail](/images/theila-node-detail.png) - -Clicking on the "Monitor" tab (see 5.1) allows you to watch resource use over time, with CPU and memory consumption graphs updated in real time, and a detailed list of running process each with their individual resource use (see 5.2). -![Fig 5 Node monitoring](/images/theila-node-monitor.png) - -Lastly, the "Dmesg" tab shows all kernel messages of the node since boot. - -### Pods - -Using the "Pods" section on the menu (see 6.1) will list all pods in the cluster, across all namespaces. -Clicking on the drop-down arrow (see 6.2) will open up more detailed information of the specified pod. -![Fig 6 Pods](/images/theila-pods.png) diff --git a/website/content/v1.3/introduction/what-is-new.md b/website/content/v1.3/introduction/what-is-new.md index 6ea587937..558eff39a 100644 --- a/website/content/v1.3/introduction/what-is-new.md +++ b/website/content/v1.3/introduction/what-is-new.md @@ -6,4 +6,353 @@ description: "List of new and shiny features in Talos Linux." See also [upgrade notes]({{< relref "../talos-guides/upgrading-talos/">}}) for important changes. -TBD +## Component Updates + +* Kubernetes: v1.26.0 +* Flannel: v0.20.2 +* CoreDNS: v1.10.0 +* etcd: v3.5.6 +* Linux: 5.15.82 +* containerd: v1.6.12 + +Talos is built with Go 1.19.4. + +## Kubernetes + +### `kube-apiserver` Custom Audit Policy + +Talos now supports setting custom audit policy for `kube-apiserver` in the machine configuration. + +```yaml +cluster: + apiServer: + auditPolicy: | + apiVersion: audit.k8s.io/v1 + kind: Policy + rules: + - level: Metadata +``` + +### `etcd` Secrets Encryption with `secretbox` algorithm + +By default new clusters will use `secretbox` for etcd [secrets encryption](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) instead of AES-CBC. +If both are configured then `secretbox` will take precedence for new writes. +Old clusters may keep using AES-CBC. +To enable `secretbox` you need to add an encryption secret at `cluster.secretboxEncryptionSecret` after an upgrade to Talos 1.3. +You should keep `aescbcEncryptionSecret` however, even if `secretbox` is enabled, older data will still be encrypted with AES-CBC. + +How to generate the secret for `secretbox`: + +```bash +dd if=/dev/random of=/dev/stdout bs=32 count=1 | base64 +``` + +### Node Labels + +Talos now supports specifying node labels in the machine configuration: + +```yaml +machine: + nodeLabels: + rack: rack1a + zone: us-east-1a +``` + +Changes to the node labels will be applied immediately without restarting `kubelet`. + +Talos keeps track of the owned node labels in the `talos.dev/owned-labels` annotation. + +### Static Pod Manifests + +Talos by default (for new clusters) doesn't configure `kubelet` to watch `/etc/kubernetes/manifests` directory for static pod manifests. +Talos-managed static pods are served via local HTTP server which prevents potential security vulnerabilities related to malicious static pods manifests +being placed to the aforementioned directory. + +Static pods should always be configured in `machine.pods` instead of using `machine.files` to put files to `/etc/kubernetes/manifests` directory. +To re-enable support for `/etc/kubernetes/manifests` you may set `machine.kubelet.disableManifestsDirectory`. + +Example: + +```yaml +machine: + kubelet: + disableManifestsDirectory: no +``` + +## etcd + +### `etcd` Consistency Check + +Talos enables [--experimental-compact-hash-check-enabled](https://github.com/etcd-io/etcd/pull/14120) option by default to improve +`etcd` store consistency guarantees. + +This options is only available with `etcd` >= v3.5.5, so Talos doesn't support versions of `etcd` older than v3.5.5 (Talos 1.3.0 defaults to `etcd` v3.5.6). + +### `etcd` Member ID + +Talos now internally handles etcd member removal by member ID instead of member name (hostname). +This resolves the case when member name is not accurate or empty (eg: when `etcd` hasn't fully joined yet). + +Command `talosctl etcd remove-member` now accepts member IDs instead of member names. + +A new resource can be used to get member ID of the Talos node: + +```bash +$ talosctl get etcdmember +NODE NAMESPACE TYPE ID VERSION MEMBER ID +10.150.0.4 etcd EtcdMember local 1 143fab7c7ccd2577 +``` + +## CRI (containerd) + +### CRI Configuration Overrides + +Talos no longer supports CRI config overrides placed in `/var/cri/conf.d` directory. + +[New way to add configuration overrides]({{< relref "../talos-guides/configuration/containerd/" >}}) correctly handles merging of containerd/CRI plugin configuration. + +### Registry Mirrors + +Talos had an inconsistency in the way registry mirror endpoints are handled when compared with `containerd` implementation: + +```yaml +machine: + registries: + mirrors: + docker.io: + endpoints: + - "https://mirror-registry/v2/mirror.docker.io" +``` + +Talos would use endpoint `https://mirror-registry/v2/mirror.docker.io`, while `containerd` would use `https://mirror-registry/v2/mirror.docker.io/v2`. +This inconsistency is now fixed, and Talos uses same endpoint as `containerd`. + +New `overridePath` configuration is introduced to skip appending `/v2` both on Talos and `containerd` side: + +```yaml +machine: + registries: + mirrors: + docker.io: + endpoints: + - "https://mirror-registry/v2/mirror.docker.io" + overridePath: true +``` + +### registry.k8s.io + +Talos now uses `registry.k8s.io` instead of `k8s.gcr.io` for Kubernetes container images. + +See [Kubernetes documentation](https://kubernetes.io/blog/2022/11/28/registry-k8s-io-faster-cheaper-ga/) for additional details. + +If using registry mirrors, or in air-gapped installations you may need to update your configuration. + +## Linux + +### cgroups v1 + +Talos always defaults to using `cgroups v2` when Talos doesn't run in a container (when running in a container +Talos follows the host `cgroups` mode). +Talos can now be forced to use `cgroups v1` by setting boot kernel argument `talos.unified_cgroup_hierarchy=0`: + +```yaml +machine: + install: + extraKernelArgs: + - "talos.unified_cgroup_hierarchy=0" +``` + +Current `cgroups` mode can be checked with `talosctl ls /sys/fs/cgroup`: + +`cgroups v1`: + +```text +blkio +cpu +cpuacct +cpuset +devices +freezer +hugetlb +memory +net_cls +net_prio +perf_event +pids +``` + +`cgroups v2`: + +```text +cgroup.controllers +cgroup.max.depth +cgroup.max.descendants +cgroup.procs +cgroup.stat +cgroup.subtree_control +cgroup.threads +cpu.stat +cpuset.cpus.effective +cpuset.mems.effective +init +io.stat +kubepods +memory.numa_stat +memory.stat +podruntime +system +``` + +> Note: `cgroupsv1` is deprecated and it should be used only for compatibility with workloads which don't support `cgroupsv2` yet. + +### Kernel Command Line `ip=` Argument + +Talos now supports referencing interface name via `enxMAC` address notation in the `ip=` argument: + +```text +ip=172.20.0.2::172.20.0.1:255.255.255.0::enx7085c2dfbc59 +``` + +Talos correctly handles multiple `ip=` arguments, and also enables forcing DHCP on a specific interface: + +```text +vlan=eth0.137:eth0 ip=eth0.137:dhcp +``` + +### Kernel Module Parameters + +Talos now supports settings kernel module parameters. + +Example: + +```yaml +machine: + kernel: + modules: + - name: "br_netfilter" + parameters: + - nf_conntrack_max=131072 +``` + +### BTF Support + +Talos Linux kernel now ships with [BTF (BPF Type Format)](https://www.containiq.com/post/btf-bpf-type-format) support enabled: + +```bash +$ talosctl -n 10.150.0.4 ls -l /sys/kernel/btf +NODE MODE UID GID SIZE(B) LASTMOD NAME +10.150.0.4 drwxr-xr-x 0 0 0 Dec 13 16:51:19 . +10.150.0.4 -r--r--r-- 0 0 11578002 Dec 13 16:51:19 vmlinux +``` + +This can be used to compile BPF programs against the kernel without kernel sources, or to load relocatable BPF programs. + +## Platform Support + +### Exocale Platform + +Talos adds support for a new platform: [Exoscale](https://www.exoscale.com/). + +Exoscale provides a firewall, TCP load balancer and autoscale groups. +It works well with CCM and Kubernetes node autoscaler. + +### Nano Pi R4S + +Talos now supports the Nano Pi R4S SBC. + +### Raspberry Generic Images + +The Raspberry Pi 4 specific image has been deprecated and will be removed in the v1.4 release of Talos. +Talos now ships a generic Raspberry Pi image that should support more Raspberry Pi variants. +Refer to the [docs]({{< relref "../talos-guides/install/single-board-computers/rpi_generic/" >}}) to find which ones are supported. + +### PlatformMetadata Resource + +Talos now publishes information about the platform it is running on in the `PlatformMetadata` resource: + +```yaml +# talosctl get platformmetadata -o yaml +spec: + platform: equinixMetal + hostname: ci-blue-worker-amd64-0 + region: dc + zone: dc13 + instanceType: c3.medium.x86 + instanceId: efc0f667-XXX-XXX-XXXX-XXXXXXX + providerId: equinixmetal://efc0f667-XXX-XXX-XXXX-XXXXXXX +``` + +## Networking + +### KubeSpan + +KubeSpan MTU link size is now configurable via `network.kubespan.mtu` setting in the machine configuration. +Default KubeSpan MTU assumes that the underlying network MTU is 1500 bytes, so if the underlying network MTU is different, KubeSpan MTU should be adjusted accordingly. + +KubeSpan automatically publishes machine external (public) IP as a machine endpoint (as discovered by connecting to the discovery service), this allows establishing a connection +to a machine behind NAT if the KubeSpan port 51820 is forwarded to the machine. + +KubeSpan by default publishes all machine addresses as Wireguard endpoints and finds the set of endpoints that are reachable for each pair of machines. +A set of endpoints can be manually filtered via `machine.network.kubespan.filters.endpoints` setting in the machine configuration. + +### Route MTU + +Talos now supports setting MTU for a specific route. + +## `talosctl` + +### Action Tracking + +Now action tracking for commands `talosctl reboot`, `talosctl shutdown`, `talosctl reset` and `talosctl upgrade` is enabled by default. +Previous behavior can be restored by setting `--wait=false` flag. + +### `talosctl machineconfig patch` + +A new subcommand, `machineconfig patch` is added to `talosctl` to allow patching of machine configuration. + +It accepts a machineconfig file and a list of patches as input, and outputs the patched machine configuration. + +Patches can be sourced from the command line or from a file. +Output can be written to a file or to stdout. + +Example: + +```bash +talosctl machineconfig patch controlplane.yaml --patch '[{"op":"replace","path":"/cluster/clusterName","value":"patch1"}]' --patch @/path/to/patch2.json +``` + +Additionally, `talosctl machineconfig gen` subcommand is introduced as an alias to `talosctl gen config`. + +### `talosctl gen config` + +The command `talosctl gen config` now supports generating a single type of output (e.g. controlplane machine configuration) by specifying the `--output-types` flag, +which is useful with pre-generated secrets bundle, e.g.: + +```bash +$ talosctl gen secrets # this outputs secrets bundle to secrets.yaml +$ talosctl gen config mycluster https://mycluster:6443 --with-secrets secrets.yaml --output-types controlplane -o - +version: v1alpha1 # Indicates the schema used to decode the contents. +debug: false # Enable verbose logging to the console. +persist: true # Indicates whether to pull the machine config upon every boot. +# Provides machine specific configuration options. +machine: +... +``` + +### `talosctl get -o jsonpath` + +The command `talosctl get` now supports `jsonpath` output format: + +```bash +$ talosctl -n 10.68.182.3 get address -o jsonpath='{.spec.address} +10.68.182.3/31 +127.0.0.1/8 +::1/128 +192.168.11.128/32 +``` + +## Developer Experience + +### New Go Module Path + +Talos now uses `github.com/siderolabs/talos` and `github.com/siderolabs/talos/pkg/machinery` as a Go module path. diff --git a/website/content/v1.3/kubernetes-guides/network/kubespan.md b/website/content/v1.3/kubernetes-guides/network/kubespan.md index f5c5290a6..36c4316e8 100644 --- a/website/content/v1.3/kubernetes-guides/network/kubespan.md +++ b/website/content/v1.3/kubernetes-guides/network/kubespan.md @@ -31,72 +31,27 @@ To generate configuration files for a new cluster, we can use the `--with-kubesp This will enable peer discovery and KubeSpan. ```yaml -... - # Provides machine specific network configuration options. +machine: network: - # Configures KubeSpan feature. kubespan: enabled: true # Enable the KubeSpan feature. -... - # Configures cluster member discovery. +cluster: discovery: - enabled: true # Enable the cluster membership discovery feature. + enabled: true # Configure registries used for cluster member discovery. registries: - # Kubernetes registry uses Kubernetes API server to discover cluster members and stores additional information - kubernetes: {} - # Service registry is using an external service to push and pull information about cluster members. + kubernetes: # Kubernetes registry is problematic with KubeSpan, if the control plane endpoint is routeable itself via KubeSpan. + disabled: true service: {} -... -# Provides cluster specific configuration options. -cluster: - id: yui150Ogam0pdQoNZS2lZR-ihi8EWxNM17bZPktJKKE= # Globally unique identifier for this cluster. - secret: dAmFcyNmDXusqnTSkPJrsgLJ38W8oEEXGZKM0x6Orpc= # Shared secret of cluster. ``` > The default discovery service is an external service hosted for free by Sidero Labs. > The default value is `https://discovery.talos.dev/`. > Contact Sidero Labs if you need to run this service privately. -### Upgrading an Existing Cluster +### Enabling for an Existing Cluster -In order to enable KubeSpan for an existing cluster, upgrade to the latest version of Talos ({{< release >}}). -Once your cluster is upgraded, the configuration of each node must contain the globally unique identifier, the shared secret for the cluster, and have KubeSpan and discovery enabled. - -> Note: Discovery can be used without KubeSpan, but KubeSpan requires at least one discovery registry. - -#### Talos v0.11 or Less - -If you are migrating from Talos v0.11 or less, we need to generate a cluster ID and secret. - -To generate an `id`: - -```sh -$ openssl rand -base64 32 -EUsCYz+oHNuBppS51P9aKSIOyYvIPmbZK944PWgiyMQ= -``` - -To generate a `secret`: - -```sh -$ openssl rand -base64 32 -AbdsWjY9i797kGglghKvtGdxCsdllX9CemLq+WGVeaw= -``` - -Now, update the configuration of each node with the cluster with the generated `id` and `secret`. -You should end up with the addition of something like this (your `id` and `secret` should be different): - -```yaml -cluster: - id: EUsCYz+oHNuBppS51P9aKSIOyYvIPmbZK944PWgiyMQ= - secret: AbdsWjY9i797kGglghKvtGdxCsdllX9CemLq+WGVeaw= -``` - -> Note: This can be applied in immediate mode (no reboot required). - -#### Talos v0.12 or More - -Enable `kubespan` and `discovery`. +In order to enable KubeSpan for an existing cluster, enable `kubespan` and `discovery` settings in the machine config for each machine in the cluster (`discovery` is enabled by default): ```yaml machine: @@ -108,6 +63,42 @@ cluster: enabled: true ``` +## Configuration + +KubeSpan will automatically discovery all cluster members, exchange Wireguard public keys and establish a full mesh network. + +There are a few configuration options available to fine-tune the feature: + +```yaml +machine: + network: + kubespan: + enabled: false + advertiseKubernetesNetworks: false + allowDownPeerBypass: false + mtu: 1420 + filters: + endpoints: + - 0.0.0.0/0 + - ::/0 +``` + +The setting `advertiseKubernetesNetworks` controls whether the node will advertise Kubernetes service and pod networks to other nodes in the cluster over KubeSpan. +It defaults to being disabled, which means KubeSpan only controls the node-to-node traffic, while pod-to-pod traffic is routed and encapsulated by CNI. +This setting should not be enabled with Calico and Cilium CNI plugins, as they do their own pod IP allocation which is not visible to KubeSpan. + +The setting `allowDownPeerBypass` controls whether the node will allow traffic to bypass WireGuard if the destination is not connected over KubeSpan. +If enabled, there is a risk that traffic will be routed unencrypted if the destination is not connected over KubeSpan, but it allows a workaround +for the case where a node is not connected to the KubeSpan network, but still needs to access the cluster. + +The `mtu` setting configures the Wireguard MTU, which defaults to 1420. +This default value of 1420 is safe to use when the underlying network MTU is 1500, but if the underlying network MTU is smaller, the KubeSpanMTU should be adjusted accordingly: +`KubeSpanMTU = UnderlyingMTU - 80`. + +The `filters` setting allows to hide some endpoints from being advertised over KubeSpan. +This is useful when some endpoints are known to be unreachable between the nodes, so that KubeSpan doesn't try to establish a connection to them. +Another use-case is hiding some endpoints if nodes can connect on multiple networks, and some of the networks are more preferable than others. + ## Resource Definitions ### KubeSpanIdentities diff --git a/website/content/v1.3/learn-more/components.md b/website/content/v1.3/learn-more/components.md index b1a1496ea..35cc020c1 100644 --- a/website/content/v1.3/learn-more/components.md +++ b/website/content/v1.3/learn-more/components.md @@ -19,7 +19,6 @@ In the following, the focus is on the Talos Linux specific components. | apid | When interacting with Talos, the gRPC API endpoint you interact with directly is provided by `apid`. `apid` acts as the gateway for all component interactions and forwards the requests to `machined`. | | containerd | An industry-standard container runtime with an emphasis on simplicity, robustness, and portability. To learn more, see the [containerd website](https://containerd.io). | | machined | Talos replacement for the traditional Linux init-process. Specially designed to run Kubernetes and does not allow starting arbitrary user services. | -| networkd | Handles all of the host level network configuration. The configuration is defined under the `networking` key | | kernel | The Linux kernel included with Talos is configured according to the recommendations outlined in the [Kernel Self Protection Project](http://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project). | | trustd | To run and operate a Kubernetes cluster, a certain level of trust is required. Based on the concept of a 'Root of Trust', `trustd` is a simple daemon responsible for establishing trust within the system. | | udevd | Implementation of `eudev` into `machined`. `eudev` is Gentoo's fork of udev, systemd's device file manager for the Linux kernel. It manages device nodes in /dev and handles all user space actions when adding or removing devices. To learn more, see the [Gentoo Wiki](https://wiki.gentoo.org/wiki/Eudev). | @@ -91,22 +90,13 @@ Only the services necessary to run Kubernetes and manage the node are available. This includes: - containerd +- etcd - [kubelet](https://kubernetes.io/docs/concepts/overview/components/) - networkd - trustd - udevd -### networkd - -Networkd handles all of the host level network configuration. -The configuration is defined under the `networking` key. - -By default, we attempt to issue a DHCP request for every interface on the server. -This can be overridden by supplying one of the following kernel arguments: - -- `talos.network.interface.ignore` - specify a list of interfaces to skip discovery on -- `ip` - `ip=:::::::::` as documented in the [kernel here](https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt) - - ex, `ip=10.0.0.99:::255.0.0.0:control-1:eth0:off:10.0.0.1` +The `machined` process handles all machine configuration, API handling, resource and controller management. ### kernel diff --git a/website/content/v1.3/learn-more/kubespan.md b/website/content/v1.3/learn-more/kubespan.md index ccc2cfd1f..a8f79e287 100644 --- a/website/content/v1.3/learn-more/kubespan.md +++ b/website/content/v1.3/learn-more/kubespan.md @@ -191,7 +191,7 @@ KubeSpan is using only two bits of the firewall mark with the mask `0x00000060`. > > At the moment of the writing, it was confirmed that Calico CNI is using bits `0xffff0000` and > Cilium CNI is using bits `0xf00`, so KubeSpan is compatible with both. -> Flannel CNI doesn't use firewall mark at all. +> Flannel CNI uses `0x4000` mask, so it is also compatible. In the routing rules table, we match on the mark `0x40` with the mask `0x60`: diff --git a/website/content/v1.3/reference/kernel.md b/website/content/v1.3/reference/kernel.md index eae236ec9..c584f986a 100644 --- a/website/content/v1.3/reference/kernel.md +++ b/website/content/v1.3/reference/kernel.md @@ -13,14 +13,13 @@ Several of these are enforced by the Kernel Self Protection Project [KSPP](https **Required** parameters: * `talos.platform`: can be one of `aws`, `azure`, `container`, `digitalocean`, `equinixMetal`, `gcp`, `hcloud`, `metal`, `nocloud`, `openstack`, `oracle`, `scaleway`, `upcloud`, `vmware` or `vultr` -* `init_on_alloc=1`: required by KSPP * `slab_nomerge`: required by KSPP * `pti=on`: required by KSPP **Recommended** parameters: -* `init_on_free=1`: advised by KSPP if minimizing stale data lifetime is - important +* `init_on_alloc=1`: advised by KSPP, enabled by default in kernel config +* `init_on_free=1`: advised by KSPP, enabled by default in kernel config ### Available Talos-specific parameters diff --git a/website/content/v1.3/talos-guides/configuration/logging.md b/website/content/v1.3/talos-guides/configuration/logging.md index af572b0ef..7f49d1697 100644 --- a/website/content/v1.3/talos-guides/configuration/logging.md +++ b/website/content/v1.3/talos-guides/configuration/logging.md @@ -43,12 +43,12 @@ Container logs for Kubernetes pods can be retrieved with `talosctl logs -k` comm ```sh $ talosctl -n 172.20.1.2 containers -k NODE NAMESPACE ID IMAGE PID STATUS -172.20.1.2 k8s.io kube-system/kube-flannel-dk6d5 k8s.gcr.io/pause:3.5 1329 SANDBOX_READY +172.20.1.2 k8s.io kube-system/kube-flannel-dk6d5 registry.k8s.io/pause:3.6 1329 SANDBOX_READY 172.20.1.2 k8s.io └─ kube-system/kube-flannel-dk6d5:install-cni ghcr.io/siderolabs/install-cni:v0.7.0-alpha.0-1-g2bb2efc 0 CONTAINER_EXITED 172.20.1.2 k8s.io └─ kube-system/kube-flannel-dk6d5:install-config quay.io/coreos/flannel:v0.13.0 0 CONTAINER_EXITED 172.20.1.2 k8s.io └─ kube-system/kube-flannel-dk6d5:kube-flannel quay.io/coreos/flannel:v0.13.0 1610 CONTAINER_RUNNING -172.20.1.2 k8s.io kube-system/kube-proxy-gfkqj k8s.gcr.io/pause:3.5 1311 SANDBOX_READY -172.20.1.2 k8s.io └─ kube-system/kube-proxy-gfkqj:kube-proxy k8s.gcr.io/kube-proxy:v{{< k8s_release >}} 1379 CONTAINER_RUNNING +172.20.1.2 k8s.io kube-system/kube-proxy-gfkqj registry.k8s.io/pause:3.5 1311 SANDBOX_READY +172.20.1.2 k8s.io └─ kube-system/kube-proxy-gfkqj:kube-proxy registry.k8s.io/kube-proxy:v{{< k8s_release >}} 1379 CONTAINER_RUNNING $ talosctl -n 172.20.1.2 logs -k kube-system/kube-proxy-gfkqj:kube-proxy 172.20.1.2: 2021-11-30T19:13:20.567825192Z stderr F I1130 19:13:20.567737 1 server_others.go:138] "Detected node IP" address="172.20.0.3" diff --git a/website/content/v1.3/talos-guides/configuration/managing-pki.md b/website/content/v1.3/talos-guides/configuration/managing-pki.md index 5d14dcf7d..529dec1b8 100644 --- a/website/content/v1.3/talos-guides/configuration/managing-pki.md +++ b/website/content/v1.3/talos-guides/configuration/managing-pki.md @@ -1,15 +1,46 @@ --- -title: "Managing PKI" +title: "Managing Talos PKI" description: "How to manage Public Key Infrastructure" aliases: - ../../guides/managing-pki --- -## Generating an Administrator Key Pair +## Generating New Client Configuration -In order to create a key pair, you will need the root CA. +### Using Controlplane Node + +If you have a valid (not expired) `talosconfig` with `os:admin` role, +a new client configuration file can be generated with `talosctl config new` against +any controlplane node: + +```shell +talosctl -n CP1 config new talosconfig-reader --roles os:reader --crt-ttl 24h +``` + +A specific [role]({{< relref "rbac" >}}) and certificate lifetime can be specified. + +### From Secrets Bundle + +If a secrets bundle (`secrets.yaml` from `talosctl gen secrets`) was saved while +[generating machine configuration]({{< relref "../../introduction/getting-started/#configure-talos ">}}): + +```shell +talosctl gen config --with-secrets secrets.yaml --output-types talosconfig -o talosconfig https:// +``` + +> Note: `` and `` arguments don't matter, as they are not used for `talosconfig`. + +### From Control Plane Machine Configuration + +In order to create a new key pair for client configuration, you will need the root Talos API CA. +The base64 encoded CA can be found in the control plane node's configuration file. +Save the the CA public key, and CA private key as `ca.crt`, and `ca.key` respectively: + +```shell +yq eval .machine.ca.crt controlplane.yaml | base64 -d > ca.crt +yq eval .machine.ca.key controlplane.yaml | base64 -d > ca.key +``` -Save the the CA public key, and CA private key as `ca.crt`, and `ca.key` respectively. Now, run the following commands to generate a certificate: ```bash @@ -18,34 +49,24 @@ talosctl gen csr --key admin.key --ip 127.0.0.1 talosctl gen crt --ca ca --csr admin.csr --name admin ``` -Now, base64 encode `admin.crt`, and `admin.key`: +Put the base64-encoded files to the respective location to the `talosconfig`: -```bash -cat admin.crt | base64 -cat admin.key | base64 +```yaml +context: mycluster +contexts: + mycluster: + endpoints: + - CP1 + - CP2 + ca: + crt: + key: ``` -You can now set the `crt` and `key` fields in the `talosconfig` to the base64 encoded strings. - ## Renewing an Expired Administrator Certificate -In order to renew the certificate, you will need the root CA, and the admin private key. -The base64 encoded key can be found in any one of the control plane node's configuration file. -Where it is exactly will depend on the specific version of the configuration file you are using. +By default admin `talosconfig` certificate is valid for 365 days, while cluster CAs are valid for 10 years. +In order to prevent admin `talosconfig` from expiring, renew the client config before expiration using `talosctl config new` command described above. -Save the the CA public key, CA private key, and admin private key as `ca.crt`, `ca.key`, and `admin.key` respectively. -Now, run the following commands to generate a certificate: - -```bash -talosctl gen csr --key admin.key --ip 127.0.0.1 -talosctl gen crt --ca ca --csr admin.csr --name admin -``` - -You should see `admin.crt` in your current directory. -Now, base64 encode `admin.crt`: - -```bash -cat admin.crt | base64 -``` - -You can now set the certificate in the `talosconfig` to the base64 encoded string. +If the `talosconfig` is expired or lost, you can still generate a new one using either the `secrets.yaml` +secrets bundle or the control plane node's configuration file using methods described above. diff --git a/website/content/v1.3/talos-guides/configuration/patching.md b/website/content/v1.3/talos-guides/configuration/patching.md index ad0c36779..8945828d2 100644 --- a/website/content/v1.3/talos-guides/configuration/patching.md +++ b/website/content/v1.3/talos-guides/configuration/patching.md @@ -311,12 +311,18 @@ talosctl ... --patch '[{"op": "add", "path": "/machine/network/hostname", "value If multiple config patches are specified, they are applied in the order of appearance. The format of the patch (JSON patch or strategic merge patch) is detected automatically. -Generated machine configuration can be patched with `talosctl gen config`: +Talos machine configuration can be patched at the moment of generation with `talosctl gen config`: ```shell talosctl gen config test-cluster https://172.20.0.1:6443 --config-patch @all.yaml --config-patch-control-plane @cp.yaml --config-patch-worker @worker.yaml ``` +Generated machine configuration can also be patched after the fact with `talosctl machineconfig patch` + +```shell +talosctl machineconfig patch worker.yaml --patch @patch.yaml -o worker1.yaml +``` + Machine configuration on the running Talos node can be patched with `talosctl patch`: ```shell diff --git a/website/content/v1.3/talos-guides/configuration/pull-through-cache.md b/website/content/v1.3/talos-guides/configuration/pull-through-cache.md index a98d582ac..c7718dc44 100644 --- a/website/content/v1.3/talos-guides/configuration/pull-through-cache.md +++ b/website/content/v1.3/talos-guides/configuration/pull-through-cache.md @@ -7,7 +7,7 @@ aliases: In this guide we will create a set of local caching Docker registry proxies to minimize local cluster startup time. -When running Talos locally, pulling images from Docker registries might take a significant amount of time. +When running Talos locally, pulling images from container registries might take a significant amount of time. We spin up local caching pass-through registries to cache images and configure a local Talos cluster to use those proxies. A similar approach might be used to run Talos in production in air-gapped environments. It can be also used to verify that all the images are available in local registries. @@ -27,7 +27,7 @@ The follow are requirements for creating the set of caching proxies: ## Launch the Caching Docker Registry Proxies -Talos pulls from `docker.io`, `registry.k8s.io`, `quay.io`, `gcr.io`, and `ghcr.io` by default. +Talos pulls from `docker.io`, `registry.k8s.io`, `gcr.io`, and `ghcr.io` by default. If your configuration is different, you might need to modify the commands below: ```bash @@ -41,11 +41,6 @@ docker run -d -p 5001:5000 \ --restart always \ --name registry-registry.k8s.io registry:2 -docker run -d -p 5002:5000 \ - -e REGISTRY_PROXY_REMOTEURL=https://quay.io \ - --restart always \ - --name registry-quay.io registry:2.5 - docker run -d -p 5003:5000 \ -e REGISTRY_PROXY_REMOTEURL=https://gcr.io \ --restart always \ @@ -58,7 +53,6 @@ docker run -d -p 5004:5000 \ ``` > Note: Proxies are started as docker containers, and they're automatically configured to start with Docker daemon. -> Please note that `quay.io` proxy doesn't support recent Docker image schema, so we run older registry image version (2.5). As a registry container can only handle a single upstream Docker registry, we launch a container per upstream, each on its own host port (5000, 5001, 5002, 5003 and 5004). @@ -72,7 +66,6 @@ As registry containers expose their ports on the host, we can use bridge IP to d sudo talosctl cluster create --provisioner qemu \ --registry-mirror docker.io=http://10.5.0.1:5000 \ --registry-mirror registry.k8s.io=http://10.5.0.1:5001 \ - --registry-mirror quay.io=http://10.5.0.1:5002 \ --registry-mirror gcr.io=http://10.5.0.1:5003 \ --registry-mirror ghcr.io=http://10.5.0.1:5004 ``` @@ -92,11 +85,32 @@ On Linux, the docker bridge address can be inspected with `ip addr show docker0` talosctl cluster create --provisioner docker \ --registry-mirror docker.io=http://172.17.0.1:5000 \ --registry-mirror registry.k8s.io=http://172.17.0.1:5001 \ - --registry-mirror quay.io=http://172.17.0.1:5002 \ --registry-mirror gcr.io=http://172.17.0.1:5003 \ --registry-mirror ghcr.io=http://172.17.0.1:5004 ``` +## Machine Configuration + +The caching registries can be configured via machine configuration [patch]({{< relref "patching" >}}), equivalent to the command line flags above: + +```yaml +machine: + registries: + mirrors: + docker.io: + endpoints: + - http://10.5.0.1:5000 + gcr.io: + endpoints: + - http://10.5.0.1:5003 + ghcr.io: + endpoints: + - http://10.5.0.1:5004 + registry.k8s.io: + endpoints: + - http://10.5.0.1:5001 +``` + ## Cleaning Up To cleanup, run: @@ -104,10 +118,54 @@ To cleanup, run: ```bash docker rm -f registry-docker.io docker rm -f registry-registry.k8s.io -docker rm -f registry-quay.io docker rm -f registry-gcr.io docker rm -f registry-ghcr.io ``` > Note: Removing docker registry containers also removes the image cache. > So if you plan to use caching registries, keep the containers running. + +## Using Harbor as a Caching Registry + +[Harbor](https://goharbor.io/) is an open source container registry that can be used as a caching proxy. +Harbor supports configuring multiple upstream registries, so it can be used to cache multiple registries at once behind a single endpoint. + +![Harbor Endpoints](/images/harbor-endpoints.png) + +![Harbor Projects](/images/harbor-projects.png) + +As Harbor puts a registry name in the pull image path, we need to set `overridePath: true` to prevent Talos and containerd from appending `/v2` to the path. + +```yaml +machine: + registries: + mirrors: + docker.io: + endpoints: + - http://harbor/v2/proxy-docker.io + overridePath: true + ghcr.io: + endpoints: + - http://harbor/v2/proxy-ghcr.io + overridePath: true + gcr.io: + endpoints: + - http://harbor/v2/proxy-gcr.io + overridePath: true + registry.k8s.io: + endpoints: + - http://harbor/v2/proxy-registry.k8s.io + overridePath: true +``` + +The Harbor external endpoint (`http://harbor` in this example) can be configured with authentication or custom TLS: + +```yaml +machine: + registries: + config: + harbor: + auth: + username: admin + password: password +``` diff --git a/website/content/v1.3/talos-guides/install/bare-metal-platforms/iso.md b/website/content/v1.3/talos-guides/install/bare-metal-platforms/iso.md new file mode 100644 index 000000000..e8da16a3b --- /dev/null +++ b/website/content/v1.3/talos-guides/install/bare-metal-platforms/iso.md @@ -0,0 +1,16 @@ +--- +title: "ISO" +description: "Booting Talos on bare-metal with ISO." +--- + +Talos can be installed on bare-metal machine using an ISO image. +ISO images for `amd64` and `arm64` architectures are available on the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). + +Talos doesn't install itself to disk when booted from an ISO until the machine configuration is applied. + +Please follow the [getting started guide]({{< relref "../../../introduction/getting-started" >}}) for the generic steps on how to install Talos. + +> Note: If there is already a Talos installation on the disk, the machine will boot into that installation when booting from a Talos ISO. +> The boot order should prefer disk over ISO, or the ISO should be removed after the installation to make Talos boot from disk. + +See [kernel parameters reference]({{< relref "../../../reference/kernel" >}}) for the list of kernel parameters supported by Talos. diff --git a/website/content/v1.3/talos-guides/install/bare-metal-platforms/pxe.md b/website/content/v1.3/talos-guides/install/bare-metal-platforms/pxe.md new file mode 100644 index 000000000..b255e9e2c --- /dev/null +++ b/website/content/v1.3/talos-guides/install/bare-metal-platforms/pxe.md @@ -0,0 +1,37 @@ +--- +title: "PXE" +description: "Booting Talos over the network on bare-metal with PXE." +--- + +Talos can be installed on bare-metal using PXE service. +There are two more detailed guides for PXE booting using [Matchbox]({{< relref "matchbox">}}) and [Digital Rebar]({{< relref "digital-rebar">}}). + +This guide describes generic steps for PXE booting Talos on bare-metal. + +First, download the `vmlinuz` and `initramfs` assets from the [Talos releases page](https://github.com/siderolabs/talos/releases/latest/). +Set up the machines to PXE boot from the network (usually by setting the boot order in the BIOS). +There might be options specific to the hardware being used, booting in BIOS or UEFI mode, using iPXE, etc. + +Talos requires the following kernel parameters to be set on the initial boot: + +* `talos.platform=metal` +* `slab_nomerge` +* `pti=on` + +When booted from the network without machine configuration, Talos will start in maintenance mode. + +Please follow the [getting started guide]({{< relref "../../../introduction/getting-started" >}}) for the generic steps on how to install Talos. + +See [kernel parameters reference]({{< relref "../../../reference/kernel" >}}) for the list of kernel parameters supported by Talos. + +> Note: If there is already a Talos installation on the disk, the machine will boot into that installation when booting from network. +> The boot order should prefer disk over network. + +Talos can automatically fetch the machine configuration from the network on the initial boot using `talos.config` kernel parameter. +A metadata service (HTTP service) can be implemented to deliver customized configuration to each node for example by using the MAC address of the node: + +```text +talos.config=https://metadata.service/talos/config?mac=${mac} +``` + +> Note: The `talos.config` kernel parameter supports other substitution variables, see [kernel parameters reference]({{< relref "../../../reference/kernel" >}}) for the full list. diff --git a/website/content/v1.3/talos-guides/install/bare-metal-platforms/sidero.md b/website/content/v1.3/talos-guides/install/bare-metal-platforms/sidero.md index f3a531ed3..5be9a9164 100644 --- a/website/content/v1.3/talos-guides/install/bare-metal-platforms/sidero.md +++ b/website/content/v1.3/talos-guides/install/bare-metal-platforms/sidero.md @@ -1,7 +1,7 @@ --- -title: "Sidero" -description: "Sidero is a project created by the Talos team that has native support for Talos." -aliases: +title: "Sidero Metal" +description: "Sidero Metal is a project created by the Talos team that has native support for Talos." +aliases: - ../../../bare-metal-platforms/sidero --- diff --git a/website/content/v1.3/talos-guides/upgrading-talos.md b/website/content/v1.3/talos-guides/upgrading-talos.md index e594c50b7..0086c9d08 100644 --- a/website/content/v1.3/talos-guides/upgrading-talos.md +++ b/website/content/v1.3/talos-guides/upgrading-talos.md @@ -32,11 +32,24 @@ For example, if upgrading from Talos 1.0 to Talos 1.2.4, the recommended upgrade * upgrade from 1.0 to latest patch of 1.0 - to v1.0.6 * upgrade from v1.0.6 to latest patch of 1.1 - to v1.1.2 -* upgrade from v1.1.2 to latest patch of 1.2 - to v1.2.4 +* upgrade from v1.1.2 to v1.2.4 ## Before Upgrade to {{% release %}} -TBD +### Custom CRI Configuration + +If you were using a custom CRI configuration placed under `/var/cri/conf.d` with `.machine.files` setting, this will stop working after an upgrade to Talos 1.3. +Talos 1.3 fixes the way CRI configuration is merged, and introduces a [new location for custom CRI configuration]({{< relref "configuration/containerd" >}}). +The new location under `/etc/cri/conf.d` will not be accepted by Talos 1.2 and earlier, so if CRI configuration overrides are critical to boot Talos, the following upgrade flow can be used: + +1. Update machine configuration before upgrade in `--mode=staged` (using `talosctl apply-config`, `talosctl patch mc` or `talosctl edit mc`). + The new machine configuration will be only applied after the reboot (upgrade). +2. Perform an upgrade, after the upgrade Talos will pick up new CRI configuration. + +### Reboot after Upgrade + +The new Talos 1.3.0 kernel has BTF debugging information, so `kexec` support will be disabled on upgrade preventing a `kexec` failure. +After an upgrade `kexec` support will work again (if not disabled). ## Video Walkthrough @@ -93,7 +106,17 @@ future. ## Machine Configuration Changes -TBD +All changes to Talos machine configuration are backwards compatible, so you can upgrade Talos and then optionally update the machine configuration to use new features. + +* `.machine.nodeLabels` can be used to control user-defined Kubernetes node labels +* `.cluster.secretboxEncryptionSecret` can be set to encrypt Kubernetes secrets with `secretbox` algorithm (see [What's New]({{< relref "../introduction/what-is-new" >}}) for details) +* `.machine.kubelet.disableManifestsDirectory` can be enabled to disable kubelet static pods support from `/etc/kubernetes/manifests` directory +* `.cluster.apiServer.auditPolicy` configures custom `kube-apiserver` audit policy +* `mtu` setting of the network route can be used to control route-specific MTU +* `overridePath` can be used on `machine.registries.endpoints` to skip appending `/v2` (see [Pull-Through Cache]({{< relref "configuration/pull-through-cache" >}})) +* `.machine.features.apidCheckExtKeyUsage` can be enabled after an upgrade to 1.3 to perform more strict checks on node-to-node Talos apid communication (see [CVE-2022-36103](https://github.com/siderolabs/talos/security/advisories/GHSA-7hgc-php5-77qq)) +* `.machine.kubespan` provdes new settings for [KubeSpan]({{< relref "../kubernetes-guides/network/kubespan" >}}) support +* `.machine.kernel.modules` provides support to pass parameters to kernel modules on load ## Upgrade Sequence diff --git a/website/static/images/harbor-endpoints.png b/website/static/images/harbor-endpoints.png new file mode 100644 index 000000000..c01b60a69 Binary files /dev/null and b/website/static/images/harbor-endpoints.png differ diff --git a/website/static/images/harbor-projects.png b/website/static/images/harbor-projects.png new file mode 100644 index 000000000..9a70b8715 Binary files /dev/null and b/website/static/images/harbor-projects.png differ