807 Commits

Author SHA1 Message Date
Noel Georgi
e6a4583ba8
feat: support generating unsigned UKIs
Support generating unsigned UKI's.

Also plumb in support to `talosctl cluster create` to boot off UKI's.
This doesn't work yet as installer needs more work.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2025-01-10 18:39:57 +05:30
Andrey Smirnov
bbd6067d42
fix: partition alignment on disks with 4k sectors
Fixes #10097

See https://github.com/siderolabs/go-blockdevice/pull/121

I added an option to QEMU provisioner to create disks with custom block
sizes (supported for some disk types).

Unfortunately, this case can't be built as a regression as QEMU's
firmware boots fine with ESP partition at 256/1024/2048 LBA.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2025-01-09 19:54:43 +04:00
Andrey Smirnov
b72bda0a42
fix: talosctl support and race tests
1. Don't set max cgroups limit if race mode is enabled (only in test
   mode). When e.g. apid/trustd are built with race detector on, they
   consume 10x the memory.
2. Fix a data race in `talosctl support` when showing UI progress.
3. Fix an issue pulling `kubeconfig` in `talosctl support` - pull from
   endpoints (controlplanes) without setting any nodes.

Fixes #10036

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-25 21:05:27 +04:00
Andrey Smirnov
8212e4864d
refactor: use quirks in kernel args
Make default args depend on quirks, and also pass quirks down to
platform code.

Reduces amount of hacks, but it is functionally equivalent.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-23 18:52:06 +04:00
Noel Georgi
a5660ed778
feat: pcirebind controller
Add a controller to support rebinding drivers for PCI devices.

Fixes: https://github.com/siderolabs/extensions/pull/488

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-12-20 17:35:37 +05:30
Andrey Smirnov
4c3261626f
docs: fix several typos
Fixes #10000

Fixes #10001

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-20 14:53:54 +04:00
KillianCdP
5bfd829bf9
docs: fix 'containter' typo
Update talosctl docs.

Signed-off-by: KillianCdP <castillon.killian@protonmail.ch>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-17 13:38:08 +04:00
Andrey Smirnov
3e9e027efb
test: add an option to boot from an USB stick
Emulates a boot from a USB stick.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-12 19:59:19 +04:00
Andrey Smirnov
dd61ad8610
fix: lock provisioning order of user disk partitions
Fixes #9877

As a side-effect, fix alignment of user disks for newer QEMU versions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-05 16:12:22 +04:00
Noel Georgi
347b758465
chore: support saving cluster logs on destroy
Support saving cluster logs on destroy

Fixes: #9808

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-29 14:39:15 +05:30
Andrey Smirnov
bef4d51508
fix: make system_disk condition work properly before install
The problem was with specific disk selector `!system_disk` - in previous
implementation, as `system_disk` defaulted to false even if the system
disk is not known yet, this might result in picking up a disk which is
going to be system disk before system disk is picked.

In new implementation, as `system_disk` is not set before it is
detected, the condition containing `system_disk` (in either way) would
fail to execute and volume provision will be delayed until system disk
is detected.

Also:

Fixes #9809

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-27 19:39:39 +04:00
Noel Georgi
939c555f9a
fix: imager disk image-cache generator
Move things around so `talosctl` is not dependent on `go-blockdevice`.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-26 21:47:08 +05:30
Noel Georgi
1bac0b183a
feat: support generating disk images with image cache
Add support for generating disk images with image cache.

Fixes: #9616

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-26 18:52:25 +05:30
Noel Georgi
77cf84fb57
feat: support generating iso with imagecache
Support generating iso with imagecache.

Part-of: #9616

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-21 20:40:05 +05:30
Noel Georgi
5de6275b8f
chore: image cache generator improvements
* Save image in OCI format, so imager can either use as OCI input or from
a registry.
* Support caching layers to a path, so subsequent runs are faster

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-21 20:09:50 +05:30
Andrey Smirnov
cc768037f8
feat: implement block device wipe
Fixes #9731

The wipe doesn't require a reboot, but it requires the blockdevice not
to be used as a volume.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-20 15:46:37 +04:00
Dmitriy Matrenichev
4dc58cfdf3
chore: small fixes
QoL PR

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-11-19 22:21:55 +03:00
Noel Georgi
95b2fc946e
feat: image cache gen
Fixes: #9611

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-19 19:08:38 +05:30
Dmitriy Matrenichev
e26d0043e0
chore: code cleanup
More usage of slices package, less usage of package sort.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-11-14 12:25:56 +03:00
Noel Georgi
682718d4c9
fix: use imager incoming version for extension validation
Use the version coming from imager to validate extension constraints.

Part of : #9694

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-12 15:26:59 +05:30
Noel Georgi
2001167058
chore(ci): save support zip always after tests
Save `support.zip` always, also use a different folder for saving logs,
so we can save artifacts of multi cluster tests.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-08 19:18:06 +05:30
Andrey Smirnov
05c620957c
feat: allow extra mounts for docker-based talosctl cluster create
Fixes #9607

Use docker CLI syntax, support any kind of mounts supported by docker
CLI.

Also drop modules from `talos` container image, as it's useless to
provide modules in container mode.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-05 16:10:08 +04:00
Andrey Smirnov
d39393879a
fix: rework the 'metal-iso' config acquisition
Fixes #9538

Re-do the implementation by using the volume management primitives, so
that we can avoid/skip old code. This should fix all issues related to
the partition/whole disk.

Fix issues in the volume management (exposed, as we haven't used it this
way before).

Build a test case in `talosctl cluster create` to inject machine config
via `metal-iso`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-10-24 16:18:50 +04:00
Noel Georgi
1b22df48a4
chore: support debug shell for advanced development
Support dropping into a very minimal debug shell.

```bash
sudo -E --preserve-env=HOME _out/talosctl-linux-amd64 cluster create --provisioner=qemu $REGISTRY_MIRROR_FLAGS --controlplanes=1 --workers=0 --with-bootloader=false --with-debug-shell
```

Co-authored-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-10-19 16:56:24 +02:00
Andrey Smirnov
e0434d77d7
feat: update dependencies
Bring in new tools, pkgs, update Go dependencies and others.

In preparation for Talos 1.9.0-alpha.0.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-10-17 22:12:50 +04:00
Andrey Smirnov
bc4c21f41a
test: add json logs test environment
Add an option to `talosctl cluster create` to start a JSON log receiver,
and enabled it optionally.

Enable in `integration-qemu`.

See #9510

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-10-17 16:23:26 +04:00
Dmitriy Matrenichev
59a78da42c
chore: add proto-codec/codec
Unify usage of proto codec v2 across our projects.
Bump grpc library to 1.67.1 and ensure that we it still works with HTTP/2 ALPN value changes.

For https://github.com/siderolabs/talos/issues/9404

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-10-17 00:12:42 +03:00
Dmitriy Matrenichev
7ff1cedfe3
chore: update siderolabs/crypto module and return proper ALPN
Fixes #9463

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-10-16 22:12:49 +03:00
Noel Georgi
a859cff364
chore: use virtio driver for disks in arm64
ARM64 doesn't support `ide` as a disk driver for disks, use `virtio`
instead.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-10-16 17:43:13 +05:30
Dmitriy Matrenichev
4d902021bb
fix: do not use pflag csv comma reader for config-patch
Modules pflag and cobra use csv.Reader for `StringSliceVar` method. This doesn't work well with JSON, and we do not need this at all.
Drop it.

Fixes #9493

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-10-14 11:31:23 +03:00
Andrey Smirnov
908fd8789c
feat: support cgroup deep analysis in talosctl
The new command `talosctl cgroups` fetches cgroups snapshot from the
machine, parses it fully, enhances with additional information (e.g.
resolves pod names), and presents a customizable view of cgroups
configuration (e.g. limits) and current consumption.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-30 18:57:12 +04:00
Andrey Smirnov
392c4798f0
feat: prepare for Talos 1.9
Update tools, pkgs, extras.

Brings in Go 1.23.1, Linux 6.6.52, new xfsprogs, etc.

Fork docs.

Add new version contract, etc.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-27 21:23:48 +04:00
Andrey Smirnov
8166a58b36
fix: filter out non-printable characters in process line
Otherwise the output might be distorted by characters like `\n`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-19 21:19:47 +04:00
Andrey Smirnov
8d6884a8e2
test: add a test for inline machine config trusted roots
Run SideroLink API server via TLS with self-signed certificate, inject
that certificate into Talos via `talos.config.inline=`.

Fix a couple of place where our special TLS root CA provider supporting
reloading on the fly was not used.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-12 21:44:36 +04:00
Dmitriy Matrenichev
cd7c682662
chore: disallow duplicate documents on decoder level
Required for #9275

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-09-06 15:50:13 +03:00
Andrey Smirnov
3038ccfa88
feat: add configuration for EPHEMERAL volume
Fixes #9261

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-06 14:11:35 +04:00
Artem Chernyshev
faffa4c3f1
fix: never unarchive initramfs when loading boot assets in talosctl
The initramfs unarchive won't work as it's extension is `xz` while the
actual compression is `zst`.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-09-05 20:09:57 +03:00
Andrey Smirnov
6f7c3a8e5c
fix: build of talosctl on non-Linux arches
Move META constants out to machinery, and fix up imports. The internal
`pkg/meta` package shold not be consumed in public-facing commands.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 22:17:38 +04:00
Andrey Smirnov
b453385bd9
feat: support volume configuration, provisioning, etc
This implements the first round of changes, replacing the volume backend
with the new implementation, while keeping most of the external
interfaces intact.

See #8367

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 18:32:34 +04:00
Andrey Smirnov
be2ebf6b4d
chore: bump dependencies
Update tools, pkgs, extras, Go dependencies, Go tools, etc.

Linux 6.6.47 and containerd 2.0.0-rc.4.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-29 20:44:37 +04:00
Noel Georgi
19a44c2b0b
chore: drop console ttyS0 argument
Drop `console=ttyS0` argument for metal images/installer.

`console=ttyS0` causes lot of issues with bare metal hardware when
trying to use a physical serial port.

Ref:

* https://bugzilla.redhat.com/show_bug.cgi?id=1839923
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=763601;msg=17
* https://www.kernel.org/doc/html/latest/admin-guide/serial-console.html
* https://github.com/coreos/fedora-coreos-tracker/issues/567

Fixes: #8695
Fixes: #8657
Fixes: #8127

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-27 22:24:59 +05:30
Dmitry Sharshakov
4834a61a8e
feat: report SELinux labels
This will be useful for debugging SELinux implementation. Make API report other xattrs for further development like IMA/EVM

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-08-26 16:19:38 +03:00
Noel Georgi
36f83eea9f
chore: make qemu check flag consistent with code
Restructure code as per changes from #9198.

This makes the flag name to be in sync with what it actually does.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-20 20:33:56 +05:30
Andrey Smirnov
61a1c946bf
feat: bundle (some) CNI plugins with Talos core
Fixes https://github.com/siderolabs/extensions/issues/448

Bundle some CNI standard plugins plus Flannel CNI plugin (as Flannel is
the default CNI in Talos) in the Talos `initramfs`.

With this change, no plugin install is required, so the `install-cni`
step is dropped from the Flannel default manifest.

The bundled plugins:

```
$ talosctl -n 172.20.0.2 ls -lH /opt/cni/bin/
NODE         MODE         UID   GID   SIZE(B)   LASTMOD       NAME
172.20.0.2   drwxr-xr-x   0     0     109 B     7 hours ago   .
172.20.0.2   -rwxr-xr-x   0     0     3.2 MB    7 hours ago   bridge
172.20.0.2   -rwxr-xr-x   0     0     3.3 MB    7 hours ago   firewall
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   flannel
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   host-local
172.20.0.2   -rwxr-xr-x   0     0     2.4 MB    7 hours ago   loopback
172.20.0.2   -rwxr-xr-x   0     0     2.8 MB    7 hours ago   portmap
```

The `initramfs` for amd64 grows 67 -> 73 MiB with this change.

The path `/opt/cni/bin` is still an overlay mount, so extra plugins can
be dropped to this directory (no change here).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-14 14:33:18 +04:00
Jean-Francois Roy
fd54dc191d
feat(talosctl): append microsoft secure boot certs
This patch adds a flag to `secureboot.database.Generate` to append the
Microsoft UEFI secure boot DB and KEK certificates to the appropriate
ESLs, in addition to complimentary command line flags.

This patch also includes a copy of said Microsoft certificates. The
certificates are downloaded from an official Microsoft repo.

Signed-off-by: Jean-Francois Roy <jf@devklog.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-22 14:15:42 +04:00
Andrey Smirnov
1b8c9ccbb0
fix: enforce secureboot enroll option only for supported releases
Follow up for #9005

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-17 14:16:26 +04:00
Andrey Smirnov
b07338f547
feat: provide machine config document to update trusted CA roots
Fixes #8867

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-12 19:28:31 +04:00
Andrey Smirnov
cf5effabb2
feat: provide an option to enforce SecureBoot for TPM enrollment
Fixes #8995

There is no security impact, as the actual SecureBoot
state/configuration is measured into the PCR 7 and the disk encryption
key unsealing is tied to this value.

This is more to provide a way to avoid accidentally encrypting to the
TPM while SecureBoot is not enabled.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-11 22:21:47 +04:00
Noel Georgi
a727a1d97a
chore: make using action tracker easier
Refactor so that action tracker accepts an interface.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-07-11 21:51:29 +05:30
Dmitriy Matrenichev
dad9c40c73
chore: simplify code
- replace `interface{}` with `any` using `gofmt -r 'interface{} -> any -w'`
- replace `a = []T{}` with `var a []T` where possible.
- replace `a = []T{}` with `a = make([]T, 0, len(b))` where possible.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-07-08 18:14:00 +03:00