Fixes#4815
This implements the following steps:
* machine configuration updates
* pulling and unpacking system extension images
* validating, listing system extensions
* re-packing system extensions
* preserving installed extensions in `/etc/extensions.yaml`
Once extension is enabled, raw information can be queried with:
```
$ talosctl -n 172.20.0.2 cat /etc/extensions.yaml
layers:
- image: 000.ghcr.io-smira-gvisor-c927b54-dirty.sqsh
metadata:
name: gvisor
version: 20220117.0-v1.0.0
author: Andrew Rynhard
description: |
This system extension provides gVisor using containerd's runtime handler.
compatibility:
talos:
version: '> v0.15.0-alpha.1'
```
This was tested with the `gvisor` system extension.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
* cloud-init for scaleway
* set ipv6 to the interface
Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
* cloud-init for hcloud
* set ipv6 to the interface
Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
This relies on changes in GRUB and other utilities to respect
`SOURCE_DATE_EPOCH`.
Variable `SOURCE_DATE_EPOCH` is set to the timestamp of the last git
commit which makes it deterministic, but still changes for each
release/commit.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
GCP does not consider generated .tar file to be valid.
This reverts commit b2507b41d250b989b9c13ad23e16202cd53a18d2.
Refs #4023.
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
Sparse file generation replaced with Go native calls.
Final artifact `.tar` reproducible with new tar flags and using GNU tar
instead of busybox one, but as the image itself is not reproducible,
this only helps a bit.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Preset `-0` for xz means fast compression but low compression level.
Changing this to `-6` (default) means that result is 10% smaller (tested
with RPi4 image).
Enable parallel compression with number of threads equal to number of
CPUs to make it compress even faster then with `-0`:
* `-0`: 15s
* `-6`: 60s
* `-6 -T 0`: 10s (on my machine, depends on number of cores)
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
We need to be able to run an install with `docker run`. This checks if
we are running from docker and skips overlay mount checks if we are, as
docker creates a handful of overlay mounts by default that we can't
workaround (not easily at least).
Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
The structure of the controllers is really similar to addresses and
routes:
* `LinkSpec` resource describes desired link state
* `LinkConfig` controller generates `LinkSpecs` based on machine
configuration and kernel cmdline
* `LinkMerge` controller merges multiple configuration sources into a
single `LinkSpec` paying attention to the config layer priority
* `LinkSpec` controller applies the specs to the kernel state
Controller `LinkStatus` (which was implemented before) watches the
kernel state and publishes current link status.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
VMware vSphere doens't allow talos.config to be set when deploying the OVA
due to missing vApp properties. Added talos.config to the ovf template to include
talos.config to fix this.
Fixes talos-systems#3669
Signed-off-by: Joost Coelingh <joost.coelingh@eu.equinix.com>
VMware vSphere considers the OVA invalid, seemingly because it considers
VirtualSCSI incompatible with osType otherLinux64Guest. Updating the osType
to other3xLinux64Guest fixes this
Fixes talos-systems#3515
Signed-off-by: Lennard Klein <lennard.klein@eu.equinix.com>
This has two big visible changes:
* `installer` image now contains assets for both `amd64` and `arm64`, so
it can be used to generate any Talos image (including RPi on amd64 host)
* Talos is using cross-compilation instead of emulation to build
non-native architectures: on amd64, Go amd64 compiler produces binaries
for both arm64 and amd64
(before this change: Go arm64 compiler via QEMU produces arm64 binaries on amd64)
CI implications: we no longer require arm64 nodes.
Changes walkthrough:
* `installer` container now keeps assets under `/usr/install/<arch>`
* Dockerfile build starts forcing toolchain/base image to use the build
host native architecture, not target architecture
* lots of duplication for amd64/arm64 as we want to combine assets for
both arches in a single image (e.g. we have multi-arch amd64/arm64
installer image, each arch has native installer binary, but both arches
contain full set of amd64/arm64 assets)
* fixed a small bug preventing arm64 on amd64 talosctl cluster create
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
The OVF environment is a way to supply guestinfo to guests. It is
a datastructure (XML) put in `extraConfig` (commonly referred to as
`guestinfo`) under the key `ovfenv`.
This OVF env is said to be the proper way to supply customization data
to guests (ie, not through `extraConfig`), and on some platforms (eg,
vCD), it is even the only option.
This change also enables the actual OVF transport in the OVA.
Signed-off-by: Jorik Jonker <jorik.jonker@eu.equinix.com>
Talos validates machine configuration at boot time, and refuses to boot
if machine configuration is invalid.
As machine configuration validation rules might change over time, we
need to prevent a scenario when after an upgrade machine configuration
becomes invalid, as there's no way to roll back properly.
Machine configuration is submitted over stdin to the installer
container, and installer container validates it using the new version of
Talos (which is going to be installed).
If the config is not sent over stdin, installer assumes old version of
Talos and proceeds.
This should be backported to 0.9 to allow config validation on upgrade
to 0.10.
Fixes#3419
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Overlay mount in `mountinfo` don't show up as mounts for any particular
block device, so the existing check doesn't catch them.
This was discovered as our current master can't upgrade because of
overlay mount for `/opt` and `apid` image in `/opt/apid` (which will be
fixed in a separate PR).
Without the check, installer fails on resetting partition table for the
disk effectively wiping the node (`device or resource busy` error).
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This fixes a case of upgrade from 0.9.0-alpha.4 to 0.9.0-beta.0. With
introduced proper partition alignment and physical block size != 512,
partitions before ephemeral will be moved around a bit (due to the
alignment), and `STATE` partition size might change a bit.
If encryption is enabled, contents are preserved as raw bytes, so
partition size should be exactly same during restore.
Drop code (mostly tests) which handled 0.6 to 0.7 upgrades.
On upgrade with preserve don't touch any partitions, at least for 0.8 ->
0.9 layout hasn't changed.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This PR introduces the first part of disk encryption support.
New config section `systemDiskEncryption` was added into MachineConfig.
For now it contains only Ephemeral partition encryption.
Encryption itself supports two kinds of keys for now:
- node id deterministic key.
- static key which is hardcoded in the config and mainly used for test
purposes.
Talosctl cluster create can now be told to encrypt ephemeral partition
by using `--encrypt-ephemeral` flag.
Additionally:
- updated pkgs library version.
- changed Dockefile to copy cryptsetup deps from pkgs.
Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
Filesystem creation step is moved on the later stage: when Talos mounts
the partition for the first time.
Now it checks if the partition doesn't have any filesystem and formats
it right before mounting.
Additionally refactored mount options a bit:
- replaced separate options with a set of binary flags.
- implemented pre-mount and post-unmount hooks.
And fixed typos in couple of places and increased timeout for `apid ready`.
Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
This fixes spurious race conditions when user disks are partitioned
and formatted in `mountUserDisks` task. While this task runs, `udevd` is
running to allow various `/dev/` symlinks to be used for user disks.
At the same time `udevd` might trigger syscall `BLKRRPART` at any time
concurrently with Talos which leads to a race on kernel side when Talos
tries to update kernel partition table while kernel does it on its own
as a result of `udevd` call.
As part of the fix, `RereadPartitionTable()` calls were removed (they
trigger `BLKRRPART` and they're not needed as Talos updates partition
table on its own).
Some cleanups to make sure blockdevice is open/closed just in matching
pairs (no lingering open blockdevice instances). This is import for
`WithExclusiveLock()` calls, as it would lead to a deadlock if previous
blockdevice instance is not closed.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This refactoring is required to simplify the work to be done to support
disk encryption.
Tried to minimize amount of queries done by `blockdevice` `probe`
methods.
Instead, where we have `runtime.Runtime` we get all required blockdevices
there from blockdevice cache stored in `State().Machine().Disk()`.
This opens a way to store encryption settings in the `Partition`
objects.
Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
Fixes#3011
See also https://github.com/talos-systems/go-procfs/pull/8
We don't want to allow all the kernel args to be overridden, as this
might compromise KSPP, but we would rather allow some args to be
overridden explicitly.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
That change should make Talos updates more straightforward in any
projects that depend on Talos.
Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
SBC should always overwrite default kernel params.
Otherwise we will always get duplicate values for some of them.
Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
Idea is to add an option to perform "selective" reset: default reset
operation is to wipe all partitions (triggering reinstall), while spec
allows only to wipe some of the operations.
Other operations are performed exactly in the same way for any reset
flow.
Possible use case: reset only `EPHEMERAL` partition.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Regular upgrade path takes just one reboot, but it requires all the
processes to be stopped on the node before upgrade might proceed. Under
some circumstances and with potential Talos bugs it might not work
rendering Talos upgrades almost impossible.
Staged upgrades build upon regular install flow to run the upgrade on
the node reboot. Such upgrades require two reboots of the node, and it
requires two pulls of the installer image, but they should be much less
suspicious to the failure. Once the upgrade is staged, node can be
rebooted in any possible way, including hard reset and upgrade is
performed on the next boot.
New ADV format was implemented as well to allow to store install image
ref/options across reboots. New format allows for bigger values and
takes 50% of the `META` partition. Old ADV is still kept for
compatibility reasons.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This makes sure Talos won't pick up any potential leftover data on fresh
install. On upgrade contents of META partitions are preserved anyways.
Fixes#2919
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
By publishing SBC images as compressed raw images, tools like etcher can flash SD cards
by using URLs to the release asset. It is also common in this community to publish compressed
images instead of tarballs.
Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
This allows boards to provide kernel args at install time. We need this so that
we can set the console.
Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
This introduces the notion of a "board" in Talos. A board is an interface that is capable
of modifying the installation in specific ways for a given SBC. This also adds support for the
libretech_all_h3_cc_h5.
Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
This PR adds the ability for us to deploy Talos in openstack. Tested in
local devstack with a supplied userdata file. It also adds support to
the Makefile for building the openstack image so it'll be published with
next release.
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
This changes installer image/iso output to be tar via stdout
(optionally), so that we can copy back artifacts back from remote docker
daemon.
Fixes#2776
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Fixes were applied automatically.
Import ordering might be questionable, but it's strict:
* stdlib
* other packages
* same package imports
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
