184 Commits

Author SHA1 Message Date
Andrey Smirnov
8212e4864d
refactor: use quirks in kernel args
Make default args depend on quirks, and also pass quirks down to
platform code.

Reduces amount of hacks, but it is functionally equivalent.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-12-23 18:52:06 +04:00
Noel Georgi
939c555f9a
fix: imager disk image-cache generator
Move things around so `talosctl` is not dependent on `go-blockdevice`.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-26 21:47:08 +05:30
Noel Georgi
1bac0b183a
feat: support generating disk images with image cache
Add support for generating disk images with image cache.

Fixes: #9616

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-26 18:52:25 +05:30
Noel Georgi
77cf84fb57
feat: support generating iso with imagecache
Support generating iso with imagecache.

Part-of: #9616

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-21 20:40:05 +05:30
Noel Georgi
682718d4c9
fix: use imager incoming version for extension validation
Use the version coming from imager to validate extension constraints.

Part of : #9694

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-11-12 15:26:59 +05:30
Andrey Smirnov
6f7c3a8e5c
fix: build of talosctl on non-Linux arches
Move META constants out to machinery, and fix up imports. The internal
`pkg/meta` package shold not be consumed in public-facing commands.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 22:17:38 +04:00
Andrey Smirnov
b453385bd9
feat: support volume configuration, provisioning, etc
This implements the first round of changes, replacing the volume backend
with the new implementation, while keeping most of the external
interfaces intact.

See #8367

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-08-30 18:32:34 +04:00
Noel Georgi
19a44c2b0b
chore: drop console ttyS0 argument
Drop `console=ttyS0` argument for metal images/installer.

`console=ttyS0` causes lot of issues with bare metal hardware when
trying to use a physical serial port.

Ref:

* https://bugzilla.redhat.com/show_bug.cgi?id=1839923
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=763601;msg=17
* https://www.kernel.org/doc/html/latest/admin-guide/serial-console.html
* https://github.com/coreos/fedora-coreos-tracker/issues/567

Fixes: #8695
Fixes: #8657
Fixes: #8127

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-08-27 22:24:59 +05:30
Jean-Francois Roy
fd54dc191d
feat(talosctl): append microsoft secure boot certs
This patch adds a flag to `secureboot.database.Generate` to append the
Microsoft UEFI secure boot DB and KEK certificates to the appropriate
ESLs, in addition to complimentary command line flags.

This patch also includes a copy of said Microsoft certificates. The
certificates are downloaded from an official Microsoft repo.

Signed-off-by: Jean-Francois Roy <jf@devklog.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-22 14:15:42 +04:00
Dmitriy Matrenichev
dad9c40c73
chore: simplify code
- replace `interface{}` with `any` using `gofmt -r 'interface{} -> any -w'`
- replace `a = []T{}` with `var a []T` where possible.
- replace `a = []T{}` with `a = make([]T, 0, len(b))` where possible.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-07-08 18:14:00 +03:00
Andrey Smirnov
be35f380cc
chore: update pkgs/tools/extras
This brings in Go 1.22.5 and new Flannel CNI plugin.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-07-03 20:38:55 +04:00
Andrey Smirnov
82d9cd3229
fix: add upgrade errata for arm64/zboot kernels
Fixes #8854

Talos 1.8.0 instroduces EFI ZBoot compression, and kexec from 1.7.0 to
compressed kernel doesn't work.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-06-03 20:10:24 +04:00
Andrey Smirnov
92a274e9a0
fix: workaround problems with udevd races
When `udevd` rescans block device partitions while Talos is doing
partitions, it might be that Talos can hit the following error
while trying to open/mount a partition:

```
no such device or address
```

Previous attempts to fix that were using `ENODEV`, while the proper code
is `ENXIO`.

Also take exclusive lock while working with user disks to prevent
concurrent udevd rescan.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-05-24 21:22:07 +04:00
Artem Chernyshev
42ac5cd0c2
fix: check for nil machine config during installation
Otherwise we get `nil reference` exception during maintenance mode
upgrade with partial machine configs.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-05-08 16:34:55 +03:00
Andrey Smirnov
4c0c626b78
feat: use zstd compression in place of xz
Initramfs and kernel are compressed with zstd.

Extensions are compressed with zstd for Talos 1.8+.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-29 18:09:12 +04:00
Andrey Smirnov
0a785802ea
fix: overlay installer operations
1. Use overlay installer to build the `cmdline` when running in
   install/upgrade mode.

2. Pull down the overlay installer with the arch specific to the
   installer being generated, vs. the arch of the `imager`.

3. Print a message when running an overlay installer.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-16 20:07:44 +04:00
Artem Chernyshev
3dd1f4e88c
chore: extract pkg/imager/quirks to pkg/machinery
To make it possible to use it without pulling the whole Talos.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-04-15 21:37:47 +03:00
Andrey Smirnov
090143b030
fix: allow platform cmdline args to be platform-specific
Fix Equnix Metal (where proper arm64 args are known) and metal platform
(using generic arm64 console arg).

Other platforms might need to be updated, but correct settings are not
known at the moment.

Fixes #8529

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-02 14:41:39 +04:00
Niklas Wik
0fc24eeb09
feat: provide insecure flag to imager
provides flag for imager to pull images insecurely from private registries

Signed-off-by: Niklas Wik <niklas.wik@nokia.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-04-01 15:53:56 +04:00
Andrey Smirnov
7d43c9aa6b
chore: annotate installer errors
I want to catch a spurious error `ENODEV`, where exactly it comes from.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-03-21 16:58:34 +04:00
Dmitriy Matrenichev
19f15a840c
chore: bump golangci-lint to 1.57.0
Fix all discovered issues.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-03-21 01:06:53 +03:00
Noel Georgi
952801d8b2
fix: handle overlay partition options
Handling of Overlay PartitionOpts was missed in the previous code.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-14 15:39:59 +05:30
Noel Georgi
d118a852b9
feat: implement Install for imager overlays
Implement `Install` for imager overlays.
Also add support for generating installers.

Depends on: #8377

Fixes: #8350
Fixes: #8351
Fixes: #8350

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-03-12 22:46:29 +05:30
Artem Chernyshev
3c8f51d707
chore: move cli formatters and version modules to machinery
To be used in the `go-talos-support` module without importing the whole
Talos repo.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2024-03-07 16:29:15 +03:00
Noel Georgi
8125e754b8
feat: imager overlay
Support overlays for imager.
The `Install` interface is not wired yet, it will be done as a different
PR.

This should be a no-op for existing imager.

Part of: #8350

Signed-off-by: Noel Georgi <git@frezbo.dev>
2024-02-29 20:44:31 +05:30
Andrey Smirnov
d81d490003
chore: update CoreDNS renovate source
As we're using a mirrored image from `registry.k8s.io`, use that as a
source instead of GitHub. Mirrored image appears with some delay after
an official CoreDNS release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-27 17:12:25 +04:00
Andrey Smirnov
8872a7a210
fix: ignore 'no such device' in addition to 'no such file'
This errors pops up when `udevd` rescans the partition table with Talos
trying to mount a device concurrently.

This feels to be something new with Linux 6.6 probably.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-02-26 20:00:05 +04:00
Dmitriy Matrenichev
fa3b933705
chore: replace fmt.Errorf with errors.New where possible
This time use `eg` from `x/tools` repo tool to do this.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-14 17:39:30 +03:00
Dmitriy Matrenichev
5324d39167
chore: bump stuff
Also fix .golangci.yml file.

Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
2024-02-09 19:19:25 +03:00
Andrey Smirnov
a5e13c696d
fix: retry blockdevice open in the installer
We had these retries in other places, but not here.

This seems to happen more frequently with Linux 6.6 update, the tl;dr is
same: `udevd` tries to rescan the partition table at the wrong moment,
preventing Talos installer to open the partition which was just created.

It's a race, so workaround it by retrying the call.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-31 22:17:20 +04:00
Andrey Smirnov
e0dfbb8fba
fix: allow META encoded values to be compressed
Fixes #8186

This is planned to be backported to Talos 1.6.3.

This allows to pass large META values (YAML for platform network
configuration) which might otherwise exceed the limit for kernel
command line params.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2024-01-23 17:24:18 +04:00
Andrey Smirnov
0a30ef7845
fix: imager should support different Talos versions
Add some quirks to make images generated with newer Talos compatible
with images generated by older Talos.

Specifically, reset options were adding in Talos 1.4, so we shouldn't
add them for older versions.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-22 16:13:34 +04:00
Andrey Smirnov
760f793d55
fix: use correct prefix when installing SBC files
When creating an image under non-default mount prefix, it should be
used explicitly when copying SBC files.

See https://github.com/siderolabs/image-factory/issues/65

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-15 19:46:10 +04:00
Andrey Smirnov
d8a435f0e4
fix: initialize boot assets with defaults early
The problem was that bootloaders were correctly picking up defaults for
`installer` mode (vs. `imager` mode), but DTB and other SBC stuff wasn't
properly initialized, so installing on SBC fails.

Now all options are properly initialized with defaults early in the
process.

Fixes #8009

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-12-01 17:47:05 +04:00
Noel Georgi
7a4a92854f
feat: support sanitized kernel args
Support dropping kernel args that start with `-`.

Fixes: #7613

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-11-28 16:23:05 +05:30
Andrey Smirnov
e22ab440d7
feat: update Linux 6.1.61, containerd 1.7.8, runc 1.1.10
Bump tools/pkgs/extras.

Update Go dependencies.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-09 20:17:28 +04:00
Andrey Smirnov
6dc776b8aa
fix: when writing to META in the installer/imager, use fixed name
Use fixed partition name instead of trying to auto-discover by label.

Auto-discovery by label might hit completely wrong blockdevice.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-11-01 20:34:41 +04:00
Andrey Smirnov
cbe6e7622d
fix: generate images for SBCs using imager
See https://github.com/siderolabs/image-factory/issues/43

Two fixes:

* pass path to the dtb, uboot and rpi-firmware explicitly
* include dtb, uboot and rpi-firmware into arm64 installer image when
  generated via imager (regular arm64 installer was fine)

(The generation of SBC images was not broken for Talos itself, but only
when used via Image Factory).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-30 13:46:58 +04:00
Serge Logvinov
401e894112
feat: customize image size
Can possible to change boot image size.
Change the default image size for some cloud platform.

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-10-16 15:51:13 +04:00
Andrey Smirnov
a52d3cda3b
chore: update gen and COSI runtime
No actual changes, adapting to use new APIs.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-09-22 12:13:13 +04:00
Andrey Smirnov
b8fb55d5c2
fix: use a mount prefix when installing a bootloader
This is not a problem in general, but when running multiple image
generation procedures using the same mount point is a problem.

This is a no-op if `MountPrefix` is not set (when installing/upgrading
vs. creating an image).

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-31 22:21:41 +04:00
Andrey Smirnov
44f59a8049
feat: improve imager APIs
* report the final output path of the asset
* allow 'cmdline' output (just to get the kernel cmdline, e.g. for PXE
  booting)
* support pre-pulled container images for extensions

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-31 20:52:58 +04:00
Andrey Smirnov
3c9f7a7de6
chore: re-enable nolintlint and typecheck linters
Drop startup/rand.go, as since Go 1.20 `rand.Seed` is done
automatically.

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-25 01:05:41 +04:00
Andrey Smirnov
ccfa8de117
fix: automatically change rpi_4 board on upgrade
As `rpi_4` was removed in Talos 1.5, automatically migrate to
`rpi_generic`.

Fixes #7632

Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
2023-08-21 18:05:42 +04:00
Andrey Smirnov
e0f383598e
chore: clean up the output of the imager
Use `Progress`, and options to pass around the way messages are written.

Fixed some tiny issues in the code, but otherwise no functional changes.

To make colored output work with `docker run`, switched back image
generation to use volume mount for output (old mode is still
functioning, but it's not the default, and it works when docker is not
running on the same host).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-08-07 16:00:14 +04:00
Andrey Smirnov
87fe8f1a2a
feat: implement image generation profiles
Support full configuration for image generation, including image
outputs, support most features (where applicable) for all image output
types, unify image generation process.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-08-02 19:13:44 +04:00
Noel Georgi
94e9891c1b
chore: bump sd-boot to v254-rc1
Bump sd-boot.
Fix parsing PE executable offsets.
Set the PE file alignment to be 512 bytes.

Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-07-11 15:52:57 +05:30
Artem Chernyshev
ce63abb219
feat: add KMS assisted encryption key handler
Talos now supports new type of encryption keys which rely on Sealing/Unsealing randomly generated bytes with a KMS server:

```
systemDiskEncryption:
  ephemeral:
    keys:
      - kms:
          endpoint: https://1.2.3.4:443
        slot: 0
```
gRPC API definitions and a simple reference implementation of the KMS server can be found in this
[repository](https://github.com/siderolabs/kms-client/blob/main/cmd/kms-server/main.go).

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2023-07-07 19:02:39 +03:00
Andrey Smirnov
35d6adcb9a
fix: provide stashed META values before installation
Previously, if META values were supplied to the Talos ISO via
environment variable, they will be written down and available after the
install. With this fix, values are also readable and available before
the installation runs (in maintenance mode).

Most of the PR is refactoring `meta.Value(s)` to be a shared library
which is used by the installer/imager and (now) Talos.

Also fixes an issue with not returning properly `NotExist` error when
META is not yet available as a partition on disk.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2023-06-27 20:57:43 +04:00
Tim Jones
53389b1e72
feat: auto-enroll secure boot keys
Uses the auto-enrollment feature of sd-boot to enroll required UEFI Secure
Boot keys.

Fixes: #7373

Signed-off-by: Tim Jones <tim.jones@siderolabs.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
2023-06-24 00:44:56 +05:30