174 Commits

Author SHA1 Message Date
Noel Georgi
e6a4583ba8
feat: support generating unsigned UKIs
Support generating unsigned UKI's.

Also plumb in support to `talosctl cluster create` to boot off UKI's.
This doesn't work yet as installer needs more work.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2025-01-10 18:39:57 +05:30
Andrey Smirnov
bbd6067d42
fix: partition alignment on disks with 4k sectors
Fixes #10097

See https://github.com/siderolabs/go-blockdevice/pull/121

I added an option to QEMU provisioner to create disks with custom block
sizes (supported for some disk types).

Unfortunately, this case can't be built as a regression as QEMU's
firmware boots fine with ESP partition at 256/1024/2048 LBA.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2025-01-09 19:54:43 +04:00
Andrey Smirnov
4761a9e6aa
chore: update dependencies
Go modules, tools, pkgs, etc.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-26 14:48:31 +04:00
Andrey Smirnov
8212e4864d
refactor: use quirks in kernel args
Make default args depend on quirks, and also pass quirks down to
platform code.

Reduces amount of hacks, but it is functionally equivalent.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-23 18:52:06 +04:00
Noel Georgi
a5660ed778
feat: pcirebind controller
Add a controller to support rebinding drivers for PCI devices.

Fixes: https://github.com/siderolabs/extensions/pull/488

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-12-20 17:35:37 +05:30
Andrey Smirnov
58236066dd
fix: support image cache on VFAT USB stick
Scenario: copy contents of the ISO to the USB VFAT stick.

Make sure VFAT filesystem has a label `TALOS_*`.

Fixes #9936

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-13 17:15:24 +04:00
Andrey Smirnov
3e9e027efb
test: add an option to boot from an USB stick
Emulates a boot from a USB stick.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-12 19:59:19 +04:00
Christian Luetke-Stetzkamp
470b75563a
fix: use mtu network option for podman
Fixes: #7881

Signed-off-by: Christian Luteke-Stetzkamp <christian@lkamp.de>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-11 19:23:07 +04:00
Utku Ozdemir
fe04571527
fix: power on the machine on reboot request in qemu power api
IPMI powers on a machine when a PowerCycle command is sent, making a prior power on call unnecessary. We probably want to do the same thing in our API-based power implementation.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-12-11 13:09:17 +01:00
Andrey Smirnov
707a77bf64
test: fix user namespace test, TPM2 fixes
Make sure the test runs on a specific node, wait for swtpm to be up.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-09 20:16:04 +04:00
Andrey Smirnov
dd61ad8610
fix: lock provisioning order of user disk partitions
Fixes #9877

As a side-effect, fix alignment of user disks for newer QEMU versions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-05 16:12:22 +04:00
Noel Georgi
347b758465
chore: support saving cluster logs on destroy
Support saving cluster logs on destroy

Fixes: #9808

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-29 14:39:15 +05:30
Andrey Smirnov
581c0851d8
feat: update dependencies
Kubernetes: 1.32.0-rc.0
Flannel: 0.26.1
Coredns: 1.12.0

Plus some small bumps.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-28 19:51:02 +04:00
Andrey Smirnov
db1c707689
chore: move enabling SELinux by default to 1.10
For 1.9, we keep it disabled until it gets more production ready in
1.10.

It still can be enabled with `selinux=1`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-28 13:59:50 +04:00
Dmitry Sharshakov
1a8cc5f8b2
feat: add SELinux labels to volumes
Label mounted filesystems like ephemeral, overlay mounts, as well as data directories (going to become volumes later).

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-11-21 14:23:43 +01:00
Dmitriy Matrenichev
e26d0043e0
chore: code cleanup
More usage of slices package, less usage of package sort.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-11-14 12:25:56 +03:00
Sam Stelfox
aea98940b7
fix: arch linux search paths and names for QEMU provisioner
Addresses old and recent change combination that prevents qemu provisioning on
ArchLinux by adding a default search path and the filenames the package
maintainer used.

Signed-off-by: Sam Stelfox <sstelfox@bedroomprogrammers.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-12 15:11:24 +04:00
Noel Georgi
2001167058
chore(ci): save support zip always after tests
Save `support.zip` always, also use a different folder for saving logs,
so we can save artifacts of multi cluster tests.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-08 19:18:06 +05:30
Andrey Smirnov
0406a05a98
chore: update pkgs to ones built with gcc 14.2
There are no changes to versions of the pkgs themselves.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-07 17:58:02 +04:00
Noel Georgi
1800f81044
fix: selinux handling and apparmor tests
Conditionally mount selinuxfs only if it's present.

Fix AppArmor tests, `apparmor` and other minor LSM's and set
`apparmor=1`.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-07 07:50:00 +05:30
Noel Georgi
5112547d6b
chore: generate support zip for crashdump
Generate support zip on crashdump.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-06 15:54:21 +05:30
Andrey Smirnov
398f714cff
feat: update Linux 6.6.59, runc 1.2.1
Update Go dependencies, other releases.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-05 20:10:27 +04:00
Andrey Smirnov
05c620957c
feat: allow extra mounts for docker-based talosctl cluster create
Fixes #9607

Use docker CLI syntax, support any kind of mounts supported by docker
CLI.

Also drop modules from `talos` container image, as it's useless to
provide modules in container mode.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-11-05 16:10:08 +04:00
Dmitry Sharshakov
960a040491
feat: start enabling SELinux
Part of: #9127

Label executables and processes, build, load and manage SELinux policy, enable audit support.

Labeling filesystems, devices and runtime files will be done in further changes, see the full PR.

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-11-04 16:56:53 +01:00
Andrey Smirnov
dc0c6acbd7
refactor: remove unmaintained github.com/vishvananda/netlink
Replace it with other based on mdlayher/netlink packages.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-10-28 19:18:15 +04:00
Andrey Smirnov
d39393879a
fix: rework the 'metal-iso' config acquisition
Fixes #9538

Re-do the implementation by using the volume management primitives, so
that we can avoid/skip old code. This should fix all issues related to
the partition/whole disk.

Fix issues in the volume management (exposed, as we haven't used it this
way before).

Build a test case in `talosctl cluster create` to inject machine config
via `metal-iso`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-10-24 16:18:50 +04:00
Noel Georgi
1b22df48a4
chore: support debug shell for advanced development
Support dropping into a very minimal debug shell.

```bash
sudo -E --preserve-env=HOME _out/talosctl-linux-amd64 cluster create --provisioner=qemu $REGISTRY_MIRROR_FLAGS --controlplanes=1 --workers=0 --with-bootloader=false --with-debug-shell
```

Co-authored-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
2024-10-19 16:56:24 +02:00
Andrey Smirnov
bc4c21f41a
test: add json logs test environment
Add an option to `talosctl cluster create` to start a JSON log receiver,
and enabled it optionally.

Enable in `integration-qemu`.

See #9510

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-10-17 16:23:26 +04:00
Noel Georgi
fc89dc2164
fix: support extra-disks when using iso
When using `iso` and `extra-disks` we're getting errors like below for
any nodes than the first node.

```text
qemu-system-aarch64: -cdrom _out/metal-arm64-secureboot.iso: drive with bus=0, unit=2 (index=2) exists
```

Fix by explicitly specifying the the media is cdrom, so qemu doesn't
index.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-10-16 21:41:58 +05:30
Andrey Smirnov
8d6884a8e2
test: add a test for inline machine config trusted roots
Run SideroLink API server via TLS with self-signed certificate, inject
that certificate into Talos via `talos.config.inline=`.

Fix a couple of place where our special TLS root CA provider supporting
reloading on the fly was not used.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-12 21:44:36 +04:00
Andrey Smirnov
bc8bf9e8a8
feat: update Linux 6.6.49
Bring in new pkgs, runc 1.2.0-rc.3.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-09-04 19:32:51 +04:00
Andrey Smirnov
b453385bd9
feat: support volume configuration, provisioning, etc
This implements the first round of changes, replacing the volume backend
with the new implementation, while keeping most of the external
interfaces intact.

See #8367

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 18:32:34 +04:00
Andrey Smirnov
be2ebf6b4d
chore: bump dependencies
Update tools, pkgs, extras, Go dependencies, Go tools, etc.

Linux 6.6.47 and containerd 2.0.0-rc.4.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-29 20:44:37 +04:00
Noel Georgi
106c17d0b5
chore: aarch64 qemu local secureboot support
Support booting with SecureBoot on aarch64 with `talosctl cluster
create` with QEMU provisioner.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-28 18:47:45 +05:30
Noel Georgi
6f969e3645
chore: improve cluster create UX on aarch64
Improve cluster create UX on aarch64.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-16 15:55:26 +05:30
Noel Georgi
2ac8d2274f
chore: support unsupported flag for mkfs
Support `unsupported` flag for mkfs, so that `STATE` partition with size
less than 300M can be created by `mkfs.xfs`.

This allows to bring in newer `xfsprogs` that can repair corrupted FS
better.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-08 20:21:02 +05:30
Utku Ozdemir
980f9ebc07
fix: fix log format in cluster provisioning
We missed passing the log formatting options while printing logs in a couple places, causing the logs to be in inconsistent format.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-07-16 11:43:26 +02:00
Andrey Smirnov
736c1485e2
fix: change the UEFI firmware search path order
Ensure that SecureBoot enabled images come before regular ones.

With Ubuntu 24.04 `ovmf` package, due to the ordering of the search
paths `talosctl` might pick up a wrong image and disable SecureBoot.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-11 21:56:33 +04:00
Dmitriy Matrenichev
dad9c40c73
chore: simplify code
- replace `interface{}` with `any` using `gofmt -r 'interface{} -> any -w'`
- replace `a = []T{}` with `var a []T` where possible.
- replace `a = []T{}` with `a = make([]T, 0, len(b))` where possible.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-07-08 18:14:00 +03:00
Andrey Smirnov
be35f380cc
chore: update pkgs/tools/extras
This brings in Go 1.22.5 and new Flannel CNI plugin.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-03 20:38:55 +04:00
Andrey Smirnov
b4c871e4b7
chore: bump dependencies
Update Go modules and other dependencies.

Fix linting of the Dockerfile.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-02 14:46:51 +04:00
Andrey Smirnov
6f6a5d1057
chore: upgrade to rtnetlink/v2 library
The v1 version is no longer supported.

The major change is the decoding of link data, but we're not using it,
as we have our own decoders/encoders for a long time.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-13 19:20:48 +04:00
Andrey Smirnov
7fcb521a6a
feat: use hydrophone instead of sonobuoy
Fixes #8790

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-11 16:51:45 +04:00
Andrey Smirnov
f07b79f4a8
feat: provide disk detection based on new blockdevices
Uses go-siderolabs/go-blockdevice/v2 for all the hard parts,
provides new resource `Disk` which describes all disks in the system.

Additional resource `SystemDisk` always point to the system disk (based
on the location of `META` partition).

The `Disks` API (and `talosctl disks`) provides a view now into the
`talosctl get disks` to keep backwards compatibility.

QEMU provisioner can now create extra disks of various types: IDE, AHCI,
SCSI, NVME, this allows to test detection properly.

The new resource will be the foundation for volume provisioning (to pick
up the disk to provision the volume on).

Example:

```
talosctl -n 172.20.0.5 get disks
NODE         NAMESPACE   TYPE   ID        VERSION   SIZE          READ ONLY   TRANSPORT   ROTATIONAL   WWID                                                               MODEL            SERIAL
172.20.0.5   runtime     Disk   loop0     1         65568768      true
172.20.0.5   runtime     Disk   nvme0n1   1         10485760000   false       nvme                     nvme.1b36-6465616462656566-51454d55204e564d65204374726c-00000001   QEMU NVMe Ctrl   deadbeef
172.20.0.5   runtime     Disk   sda       1         10485760000   false       virtio      true                                                                            QEMU HARDDISK
172.20.0.5   runtime     Disk   sdb       1         10485760000   false       sata        true         t10.ATA     QEMU HARDDISK                           QM00013        QEMU HARDDISK
172.20.0.5   runtime     Disk   sdc       1         10485760000   false       sata        true         t10.ATA     QEMU HARDDISK                           QM00001        QEMU HARDDISK
172.20.0.5   runtime     Disk   vda       1         12884901888   false       virtio      true
```

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-07 20:18:32 +04:00
Andrey Smirnov
357d7754fd
fix: clean up VM runners on cluster destroy
This never worked properly, as `Wait()` doesn't work for child
processes, and `talosctl cluster destroy` is not a child of processes
created by `talosctl cluster create`.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-05 21:53:24 +04:00
Andrey Smirnov
41f92e0ba4
chore: update Go to 1.22.4, other updates
Bump go modules, adjust the code.

New linter warnings.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-05 20:59:52 +04:00
Andrey Smirnov
b7afe2669b
feat: update Linux 6.6.30
Update tools/pkgs to the latest version, brings in all updates.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-13 17:14:03 +04:00
Utku Ozdemir
0821b9c50b
feat: add --non-masquerade-cidrs flag to talosctl cluster create
Allow skipping NAT for the given destinations from a cluster network. This option makes it possible to form an etcd cluster from clusters in different networks created by running `talosctl cluster create` command multiple times using different CIDRs: they simply should have the CIDR of the other clusters passed with `--non-masquerade-cidrs`.

Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
2024-04-23 13:30:13 +02:00
Andrey Smirnov
3433fa13bf
feat: use container DNS when in container mode
More specifically, pick up `/etc/resolv.conf` contents by default when
in container mode, and use that as a base resolver for the host DNS.

Fixes #8303

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-16 17:01:36 +04:00
Dmitry Sharshakov
653f838b09
feat: support multiple Docker cluster in talosctl cluster create
Dynamically map Kubernetes and Talos API ports to an available port on
the host, so every cluster gets its own unique set of parts.

As part of the changes, refactor the provision library and interfaces,
dropping old weird interfaces replacing with (hopefully) much more
descriprive names.

Signed-off-by: Dmitry Sharshakov <dmitry.sharshakov@siderolabs.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-04 21:21:39 +04:00