2626 Commits

Author SHA1 Message Date
Andrey Smirnov
eefe1c21c3
feat: add new etcd members in learner mode
Fixes #3714

This provides more safe way to join new members to the etcd cluster.

See https://etcd.io/docs/v3.4/learning/design-learner/

With learner mode join there are few differences:

* new nodes are joined one by one, because etcd enforces a single
learner member in the cluster
* learner members are not counted in quorum calculations, so while
learner catches up with the master node, quorum is not affected and
cluster is still operational

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-12 17:56:57 +03:00
Andrey Smirnov
b1c66fbad1
feat: implement Equinix Metal support for virtual (shared) IP
Talos supports automatic virtual IP for the control plane with pure
layer 2 connectivity. Equinix Metal API supports assigning shared IPs to
the nodes, this PR combines existing virtual IP functionality with calls
to EM API to move the IP address on EM side to the leader node.

The only thing which should be supplied in the machine configuration is
the Equinix Metal API token, every other setting is automatically
discovered automatically by Talos.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-12 17:26:59 +03:00
Andrew Rynhard
62242f979e
chore: require GPG signatures
GPG signatures are now required.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2021-08-11 17:49:31 +00:00
Andrey Smirnov
faecae44fd feat: make ISO builds reproducible
This relies on changes in GRUB and other utilities to respect
`SOURCE_DATE_EPOCH`.

Variable `SOURCE_DATE_EPOCH` is set to the timestamp of the last git
commit which makes it deterministic, but still changes for each
release/commit.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-11 09:20:07 -07:00
Andrey Smirnov
887c2326a4 release(v0.12.0-alpha.0): prepare release
This is the official v0.12.0-alpha.0 release.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
pkg/machinery/v0.12.0-alpha.0 v0.12.0-alpha.0
2021-08-11 08:49:34 -07:00
Andrey Smirnov
a15f01844f fix: move etcd PKI under /system/secrets
Fixes #3944

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-10 16:16:23 -07:00
Andrey Smirnov
eb02afe18b fix: match correctly routes on the address family
When looking up existing routes in the kernel routing table, make sure
routes are filtered by the address family (IPv4, IPv6). This is usually
redundant, as either destination or gateway enforces the family, but for
the default routes without the destination, we need to add this check.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-10 13:29:58 -07:00
Andrey Smirnov
cb948accfe feat: allow multiple addresses per interface
Fixes #3940

This changes machine configuration in a backwards compatible way to
allow multiple `.addresses` to be specified for the interface (and for
VLANs). Old field `.cidr` is still processed and used, but new
deployments should move to `.addresses`. This allows to stop using
interface duplication workarounds to assign multiple IPs to the
interface.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-10 13:29:37 -07:00
Alexey Palazhchenko
e030b2e8bb chore: use k8s 1.21.3 in CAPI tests for now
Refs #4046.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-10 13:28:37 -07:00
Artem Chernyshev
e08b4f8f9e feat: implement sysctl controllers
Fixed: https://github.com/talos-systems/talos/issues/3686

Replaced sequencer tasks for KSPP and Kubernetes required sysctl props
by the ones set by controllers.

KernelParam flow includes of 3 controllers and 2 resources:
- `KernelParamConfigController` - handles user sysctls coming from v1alpha1
config.
- `KernelParamDefaultsController` - handles our built-in KSPP and K8s
required sysctls.
- `KernelParamSpecController` - consumes `KernelParamSpec`s created by the
previous two controllers, applies them and updates the corresponding
`KernelParamStatus`.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-08-10 13:21:49 -07:00
Alexey Palazhchenko
fdf6b2433c chore: revert "improve artifacts generation reproducibility"
GCP does not consider generated .tar file to be valid.

This reverts commit b2507b41d250b989b9c13ad23e16202cd53a18d2.
Refs #4023.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-10 11:50:42 -07:00
Andrey Smirnov
b68ed1eb89 fix: make route resources ID match closer routing table primary key
New primary key is the following:

* table (if `main`, leave it empty)
* family
* gateway
* destination
* metric (priority)

There might be more ways we can improve this (probably gateway should be
removed?), but this should be much better model to represent routes in
the system.

This was discovered in KubeSpan development (@Ulexus).

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-10 07:11:09 -07:00
Artem Chernyshev
585f633710 fix: correctly handle nodoc for struct fields
Also fix the const for `docgen:nodoc` for skipped the whole struct.

Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
2021-08-10 06:24:29 -07:00
Andrey Smirnov
f2d394dc42 docs: add AMIs for v0.11.5
AMIs as uploaded via the CI release pipeline.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-09 14:35:45 -07:00
Serge Logvinov
d0970cbfd6 feat: bootstrap token limit
Token can use only for bootstrap nodes

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-08-09 12:38:55 -07:00
Serge Logvinov
5285a46d78 fix: maintenance mode reason message
More friendly message why maintenance mode is starting

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-08-09 12:37:35 -07:00
Andrey Smirnov
009d15e8dc chore: use etcd client TryLock function on upgrade
We are running etcd 3.4, so it's time to implement the TODO item.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-09 12:12:45 -07:00
Alexey Palazhchenko
4dae9ea55c chore: use vtprotobuf compiled marshaling in Talos API
Use `vtprobuf` optimized Marshal/Unmarshal methods which do not depend
on reflection to reduce memory and CPU usage while using Talos API.

See https://github.com/planetscale/vtprotobuf and
https://vitess.io/blog/2021-06-03-a-new-protobuf-generator-for-go/

Co-authored-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-09 08:42:13 -07:00
Andrey Smirnov
7ca5749ad4 chore: bump dependencies via dependabot
PRs #4033 #4034 #4035 #4036 #4037 #4038 #4039 #4040

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-09 08:32:02 -07:00
Andrey Smirnov
b2507b41d2 chore: improve artifacts generation reproducibility
Sparse file generation replaced with Go native calls.

Final artifact `.tar` reproducible with new tar flags and using GNU tar
instead of busybox one, but as the image itself is not reproducible,
this only helps a bit.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-09 06:54:47 -07:00
Andrey Smirnov
1f7dad234b chore: update PKGS version (512 cpus, new ca-certficates)
This pulls in a change to bump number of max CPUs in the kernel to 512
and update ca-certificates.

Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
2021-08-09 06:54:14 -07:00
Andrew Rynhard
1a2e78a24e fix: update go-blockdevice
Brings in the latest version of go-blockdevice to fix and issues with
the PMBR.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2021-08-08 09:09:03 -07:00
Andrey Smirnov
6d6ed1170f chore: use parallel xz with higher compression level
Preset `-0` for xz means fast compression but low compression level.
Changing this to `-6` (default) means that result is 10% smaller (tested
with RPi4 image).

Enable parallel compression with number of threads equal to number of
CPUs to make it compress even faster then with `-0`:

* `-0`: 15s
* `-6`: 60s
* `-6 -T 0`: 10s (on my machine, depends on number of cores)

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-06 12:51:31 -07:00
Andrey Smirnov
571f7db1bb chore: workaround GitHub new release notes limit
GitHub recently introduced limit of 25000 byte for the release notes,
which breaks our release process, as our notes are over the limit.

PR https://github.com/talos-systems/talos/pull/4021 introduced support
for GFM release notes, which are at least 50% smaller.

This PR enables GFM, adds a check for release notes size and pushes
release-notes step to every build so that we known in advance if release
notes are over the limit (vs. the moment we push the release).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-06 11:56:26 -07:00
Alexey Palazhchenko
09d70b7eaf feat: update Kubernetes to v1.22.0
Closes #3967.
Closes #3997.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@talos-systems.com>
2021-08-06 09:06:32 -07:00
Andrey Smirnov
f25f10e73e feat: add an option to disable PSP
This feature comes as PSP is deprecated and going to be removed in 1.25.
In preparation for that, add an option to disable PSP which was always
enabled in Talos by default.

To keep backwards compatibility, PSP is still enabled by default.

See also #3971

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Co-authored-by: Adam Szucs-Matyas <szucsitg@gmail.com>
2021-08-06 06:45:33 -07:00
Andrey Smirnov
7c6e4cf230 feat: allow both DHCP and static addressing for the interface
No need to do hacks with multiple interface entries just to get multiple
addressing methods

Fixes #2804

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-05 14:48:18 -07:00
Andrey Smirnov
3c566dbc30 fix: remove admission plugins enabled by default from the list
This allows to disable these plugins via `extraArgs` and shortens
argument list for the `kube-apiserver`.

There are no functional changes, as these plugins are enabled by default
anyways.

Based on #3971

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Co-authored-by: Adam Szucs-Matyas <szucsitg@gmail.com>
Signed-off-by: Adam Szucs-Matyas <szucsitg@gmail.com>
2021-08-04 22:59:22 -07:00
Andrey Smirnov
69ead37353 fix: preserve PMBR bootable flag correctly
See https://github.com/talos-systems/go-blockdevice/pull/41

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-04 22:58:15 -07:00
Andrey Smirnov
dee6305170 fix: align partitions with minimal I/O size
Also print discovered blockdevice properties before partitioning the
device.

See https://github.com/talos-systems/go-blockdevice/pull/40

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-04 11:51:00 -07:00
Andrey Smirnov
628902297d feat: update GRUB to 2.06
See https://github.com/talos-systems/pkgs/pull/310

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-04 10:07:21 -07:00
Serge Logvinov
b9d04928d9 feat: move system processes to cgroups
* use cgroup v2
* cgroups: /init, /system, /system/runtime
* kubelet cgroup metrics

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-08-04 09:00:38 -07:00
Andrey Smirnov
0b8681b4b4 fix: resolve several issues with Wireguard link specs
* correctly merge wireguard specs across multiple configuration layers
(partially stolen from #3577)

* fix erroneous wireguard reconfig when listen port in the config is
zero

* add tests for link merging (once again, partially stolen from #3577)

* fix ugly bug with LinkSpec Type merging (I believe it's a major source
of pain for you, Seán, in your PR).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-03 13:25:19 -07:00
Serge Logvinov
f8f4bf3bae docs: add disk encryptions examples
* Update partition encryption mount options

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-08-03 12:54:54 -07:00
Andrey Smirnov
79b8fa64b9 feat: update containerd to 1.5.5
* https://github.com/containerd/containerd/releases/tag/v1.5.5

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-03 10:26:21 -07:00
Andrey Smirnov
539f42090e chore: bump dependencies via dependabot
Fixes #3993

Fixes #3994

Fixes #3995

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-03 10:25:17 -07:00
Andrey Smirnov
0c7ce1cd81 feat: remove remnants of bootkube support
Fixes #3951

Bootkube support was removed in Talos 0.9. Talos versions 0.9-0.11
support conversion of self-hosted bootkube-based control plane to the
new style control plane running as static pods managed by Talos.

This commit removes all backwards compatibility and removes conversion
code.

For the k8s controllers, `BootstrapStatus` is removed and a dependency
on `etcd` service status is added (as it was implicitly there via
`BootstrapStatus`).

Remove control plane conversion code.

In k8s upgrade code, remove self-hosted part.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-08-03 07:55:42 -07:00
Alexey Palazhchenko
d4f9804f86 chore: fix typos
"Optimizations" wasn't spelled correctly.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-08-03 07:42:15 -07:00
Artem Chernyshev
5f027615ff feat: expose more encryption options to the machine config
Fixes: https://github.com/talos-systems/talos/issues/3606

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-07-27 11:19:26 -07:00
Alexey Palazhchenko
585152a0be chore: bump dependencies
Closes #3983.
Closes #3984.
Closes #3985.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-07-26 04:37:25 -07:00
Serge Logvinov
fc66ec5969 feat: set oom score for main processes
This PR change oom score:
* containerd to -999
* apid, trustd to -998
* cri to -100
* etcd to -998

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-07-24 20:29:03 -07:00
Serge Logvinov
df54584a33 fix: drop linux capabilities
apid, trusted, etcd run as root and do not need this permissions

Signed-off-by: Serge Logvinov <serge.logvinov@sinextra.dev>
2021-07-23 18:00:37 -07:00
Spencer Smith
f65d0b739b docs: add 0.11.3 AMIs
This PR adds AMIs for 0.11.3

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-07-23 09:40:36 -04:00
Spencer Smith
7332d63695 fix: bump pkgs for new kernel 5.10.52
This PR pulls in new pkgs to ensure we're patched against CVE-2021-33909

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-07-22 13:18:42 -07:00
Artem Chernyshev
70d2505b7c fix: do not require ToVersion to be set when detecting version
We do not know the upgrade version when checking components versions in
Theila.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-07-21 08:51:26 -07:00
Alexey Palazhchenko
0953b19985 chore: update extras to bring a new CNI bundle
https://github.com/containernetworking/plugins/releases/tag/v0.9.1

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-07-20 07:27:13 -07:00
Rui Lopes
b6c47f866a fix: set the /etc/os-release HOME_URL parameter
the previous url was failing as described in https://github.com/talos-systems/talos/issues/3968

Signed-off-by: Rui Lopes <rgl@ruilopes.com>
2021-07-20 05:20:48 -07:00
Alexey Palazhchenko
c780821d0b feat: update containerd to 1.5.3, runc to 1.0.1
Update pkgs and constants.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-07-19 08:56:32 -07:00
Artem Chernyshev
f8f1c83a75 feat: detect the lowest Kubernetes version in upgrade-k8s CLI command
Scan all pods in `kube-system` and find `kube-proxy`, `kube-scheduler`,
`kube-controller-manager` and `kube-apiserver` ones, then check the
lowest version amongst them.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-07-19 08:24:04 -07:00
Artem Chernyshev
55e17ccdd1 chore: bump dependencies
Fixes: https://github.com/talos-systems/talos/pull/3954 https://github.com/talos-systems/talos/pull/3955 https://github.com/talos-systems/talos/pull/3956 https://github.com/talos-systems/talos/pull/3957 https://github.com/talos-systems/talos/pull/3958 https://github.com/talos-systems/talos/pull/3959 https://github.com/talos-systems/talos/pull/3960 https://github.com/talos-systems/talos/pull/3961 https://github.com/talos-systems/talos/pull/3962 https://github.com/talos-systems/talos/pull/3963 https://github.com/talos-systems/talos/pull/3964

And update kubelet to 1.21.3.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-07-19 06:06:01 -07:00