See https://github.com/siderolabs/talos/discussions/10641
The problem is that `overridePath` config option should be per-endpoint,
not a global one.
As we can't easily change v1alpha1 config, change the interface instead,
and update the controller which generates final registry config to
use per-endpoint setting.
Ensure that `registryd` endpoint never uses `overridePath`.
E.g., with this PR:
```toml
[host]
[host.'http://127.0.0.1:3172']
capabilities = ['pull', 'resolve']
[host.'http://172.20.0.1:5006/v2/']
capabilities = ['pull', 'resolve']
override_path = true
```
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
This is not used for quite some time now, it was used previously to
import tarballs of apid and other services images.
We are not using it anymore.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Fixes#9613
This has two changes:
* adjust Talos registry resolver to match containerd (CRI) resolver: use
by default upstream as a fallback
* add a machine config option to skip upstream as a fallback, and adjust
CRI configuration accordingly
See https://github.com/containerd/containerd/blob/main/docs/hosts.md#registry-configuration---examples
for details on CRI's `hosts.toml`.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
After a fix a while ago, the condition was hard to understand - but we
should skip this block as long as there's no TLS config, which might
mean either being nil or having default values.
I found this while debugging #9594, but it doesn't change anything.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
The new command `talosctl cgroups` fetches cgroups snapshot from the
machine, parses it fully, enhances with additional information (e.g.
resolves pod names), and presents a customizable view of cgroups
configuration (e.g. limits) and current consumption.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Provide `XDG_RUNTIME_DIR` environment variable, this specifically fixes
the `kubectl exec` action when `/tmp` is filled up.
Update containerd configuration to version 3 and fix it up.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Signed-off-by: Noel Georgi <git@frezbo.dev>
- replace `interface{}` with `any` using `gofmt -r 'interface{} -> any -w'`
- replace `a = []T{}` with `var a []T` where possible.
- replace `a = []T{}` with `a = make([]T, 0, len(b))` where possible.
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Drop BurntSushi one, and use /v2 of pelletier package.
There is indirect use of v1 which should hopefully go away once we move
away from sonobouy.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Fixes#6391
Implement a set of APIs and commands to manage images in the CRI, and
pre-pull images on Kubernetes upgrades.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
See #7230
Refactor more config interfaces, move config accessor interfaces
to different package to break the dependency loop.
Make `.RawV1Alpha1()` method typed to avoid type assertions everywhere.
No functional changes.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
- github.com/containerd/typeurl to v2.1.1
- github.com/aws/aws-sdk-go to v1.44.264
- alpine to 3.18.0
- node to 20.2.0-alpine
- github.com/containernetworking/plugins to v1.3.0
- github.com/docker/docker to v23.0.6+incompatible
- github.com/hetznercloud/hcloud-go to v1.45.1
- github.com/insomniacslk/dhcp to v0.0.0-20230516061539-49801966e6cb
- github.com/rivo/tview to v0.0.0-20230511053024-822bd067b165
- tools to v1.5.0-alpha.0-7-gd2dde48
- pkgs to v1.5.0-alpha.0-16-g7958db1
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
talosctl netstat -k show all host and non-hostnetwork pods sockets/connections.
talosctl netstat namespace/pod shows sockets/connections of a specific pod +
autocompletes in the shell.
Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
This PR adds first 12 symbols from container ID and adds them to `talosctl -k containers` each container output.
That way we can ensure that we get the logs from proper container even if there is a newer one.
Closes#6886
Co-authored-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Talos always supported that, but CRI config lacked support for it.
Now with recent containerd the new `_default` host is used as a
fallback, so this re-enables the support and updates the docs.
See https://github.com/containerd/containerd/pull/8065
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
There was inconsistency in the way `/v2` was appended to registry
endpoint path between containerd (CRI) and Talos:
* Talos only appended `/v2` to empty paths
* containerd appended `/v2` if it's not the suffix already
Fix Talos to act same as containerd, and introduce a setting
`overridePath` which stops both Talos and `containerd` from appending
`/v2` (should be required with e.g. Harbor registry mirror).
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
There's a cyclic dependency on siderolink library which imports talos
machinery back. We will fix that after we get talos pushed under a new
name.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
This the first step towards replacing all import paths to be based on
`siderolabs/` instead of `talos-systems/`.
All updates contain no functional changes, just refactorings to adapt to
the new path structure.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
I had to do several things:
- contextcheck now supports Go 1.18 generics, but I had to disable it because of this https://github.com/kkHAIKE/contextcheck/issues/9
- dupword produces to many false positives, so it's also disabled
- revive found all packages which didn't have a documentation comment before. And tehre is A LOT of them. I updated some of them, but gave up at some point and just added them to exclude rules for now.
- change lint-vulncheck to use `base` stage as base
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
We should preserve the order of keys in generated `hosts.toml`, but
go-toml library has no real way to do that on marshaling, so fix the
previous workaround, as it was generating invalid TOML.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
Fixes#5652
This reworks and unifies HTTP client/transport management in Talos:
* cleanhttp is used everywhere consistently
* DefaultClient is using pooled client, other clients use regular
transport
* like before, Proxy vars are inspected on each request (but now
consistently)
* manifest download functions now recreate the client on each run to
pick up latest changes
* system CA list is picked up from a fixed locations, and supports
reloading on changes
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
There should be no functional change with this PR.
The primary driver is supporting strategic merge configuration patches.
For such type of patches machine config should be loaded from incomplete
fragments, so it becomes critically important to distinguish between a
field having zero value vs. field being set in YAML.
E.g. with following struct:
```go
struct { AEnabled *bool `yaml:"a"` }
```
It's possible to distinguish between:
```yaml
a: false
```
and no metion of `a` in YAML.
Merging process trewats zero values as "not set" (skips them when
merging), so it's important to allow overriding value to explicit
`false`.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
Containerd doesn't support merging plugin configuration from multiple
sources, and Talos has several pieces which configure CRI plugin:
(see https://github.com/containerd/containerd/issues/5837)
* base config
* registry mirror config
* system extensions
* ...
So we implement our own simple way of merging config parts (by simply
concatenating text files) to build a final `cri.toml`.
At the same time containerd migrated to a new format to specify registry
mirror configuration, while old way (via CRI config) is going to be
removed in 1.7.0. New way also allows to apply most of registry
configuration (except for auth) on the fly.
Also, containerd was updated to 1.6.0-rc.0 and runc to 1.1.0.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
See https://github.com/containerd/cri/pull/1543Fixes#4274
Fix is applied on two levels:
* for Talos-initiated pulls, update API call
* for Kubernetes-initiated pulls, update CRI plugin config
Comparison of `/var` usage before/after, as reported by
`talosctl mounts` (in GiB):
| | before | after |
|--------------|:------:|------:|
| controlplane | 1.98 | 1.74 |
| worker | 1.17 | 1.01 |
It's hard to measure effect on pulls to system containerd, like
`installer` image, as it's ephemeral, but it should also reduce space
usage in `tmpfs`.
Also fixes output of `talosctl mounts`.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
When running with cgroupsv2 and the deeply nested nature of our CI, we
need to take extra steps to make sure tests are working fine.
Some tests were disabled under cgroupsv2 as I can't make them work.
Signed-off-by: Andrey Smirnov <andrey.smirnov@talos-systems.com>
The structure of the controllers is really similar to addresses and
routes:
* `LinkSpec` resource describes desired link state
* `LinkConfig` controller generates `LinkSpecs` based on machine
configuration and kernel cmdline
* `LinkMerge` controller merges multiple configuration sources into a
single `LinkSpec` paying attention to the config layer priority
* `LinkSpec` controller applies the specs to the kernel state
Controller `LinkStatus` (which was implemented before) watches the
kernel state and publishes current link status.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Fixes#3538
See also talos-systems/pkgs#276
As new containerd is now Go module-based, it pulls many more
dependencies if simply imported in `go.mod`, so I had to replace the
reference to the constant in `pkg/machinery/` to `containerd` volume
with simple value to avoid pulling Kubernetes dependencies into
`pkg/machinery`.
Also updates the kernel to include PR talos-systems/pkgs#275 for AES-NI
support.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This removes container images for the aforementioned services, they are
now built into `machined` executable which launches one or another
service based on `argv[0]`.
Containers are started with rootfs directory which contains only a
single executable file for the service.
This creates rootfs on squashfs for each container in
`/opt/<container>`.
Service `networkd` is not touched as it's handled in #3350.
This removes all the image imports, snapshots and other things which
were associated with the existing way to run containers.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
