152 Commits

Author SHA1 Message Date
Alexey Palazhchenko
4fe6912143 test: better talosctl ls tests
Refs #3018.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-05-20 03:29:21 -07:00
Andrey Smirnov
76e38b7b82 feat: update Kubernetes to 1.21.1
See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-05-13 08:05:08 -07:00
Alexey Palazhchenko
9996d4b028 chore: use REGISTRY_MIRROR_FLAGS if defined
REGISTRY_MIRROR_FLAGS can be already defined in the environment with .env or .envrc file.
Use it for easier and faster local e2e tests.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-04-19 09:13:14 -07:00
Andrey Smirnov
e7a9164b1e test: implement talosctl conformance command to run e2e tests
Command implements two modes:

* `fast`: conformance suite is run at maximum speed
* `certified`: conformance suite is run in serial mode, results
  are capture to produce artifacts ready for CNCF submission process

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-04-16 09:17:51 -07:00
Alexey Palazhchenko
37a5edf04a feat: update Kubernetes to 1.21.0 release
See CHANGELOG:
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md

Closes #3329.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-04-09 20:08:20 +03:00
Spencer Smith
e6b4e524ff test: update CAPA to 0.6.4
This PR pulls in an updated cluster api aws version, ensuring the CRDs
are closer to what's expected when we patch the CAPA image later in the
setup. We will eventually move to 0.6.5 as soon as it's cut.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-04-07 14:37:20 -04:00
Alexey Palazhchenko
8737ea716a feat: allow external cloud provides configration
Closes #3312.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-04-06 22:54:24 -07:00
Alexey Palazhchenko
a9451f5712 feat: update Kubernetes to 1.21.0-beta.1
See CHANGELOG:
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md

Refs #3329.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-03-30 03:07:03 -07:00
Alexey Palazhchenko
dc21d9b4b0 chore: remove old file
To prevent confusion.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-03-25 08:39:54 -07:00
Alexey Palazhchenko
ed272e604e feat: update Kubernetes to 1.21.0-beta.0
See CHANGELOG:
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md

Refs #3329.

Signed-off-by: Alexey Palazhchenko <alexey.palazhchenko@gmail.com>
2021-03-24 07:36:54 -07:00
Andrey Smirnov
f0512dfce9 feat: update Kubernetes to 1.20.5
See CHANGELOG:
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md#changelog-since-v1204

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-03-19 03:14:46 -07:00
Spencer Smith
78d384ebb6 test: update aws cloud provider version
This PR makes sure we set a cloud provider version, which will make our
AWS tests pass again.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-03-15 13:45:50 -07:00
Andrey Smirnov
ca8a5596c7 chore: fix provision tests after changes to build-container
CNI was removed from build-container which works fine for
`talosctl cluster create` clusters as it installs its own CNI, but fails
for upgrade tests as they were never updated for the CNI bundle.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-03-12 09:59:15 -08:00
Andrey Smirnov
d7cdc8cc15 feat: implement simple layer 2 shared IP for CP
This adds a VIP (virtual IP) option to the network configuration of an
interface, which will allow a set of nodes to share a floating IP
address among them.  For now, this is restricted to control plane use
and only a single shared IP is supported.

Fixes #3111

Signed-off-by: Seán C McCord <ulexus@gmail.com>
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-26 14:14:34 -08:00
Andrey Smirnov
4e19b597ab test: add integration test with Canal CNI and reset API
Canal CNI is known to be trying to reach out to k8s control plane on pod
teardown.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-24 11:34:02 -08:00
Andrey Smirnov
84ad6cbb1a chore: switch CI to stop embedding local registry into the builds
This adds new `IMAGE_REGISTRY` variable (similar to `IMAGE_TAG`) which
affects only the registry image gets pushed to, but it's not built into
the binaries and images as a default registry.

This fixes a problem when release builds reference our CI local
registry.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-24 18:05:37 +03:00
Andrey Smirnov
e9fc54f6e3 feat: update Kubernetes to 1.20.3
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md#changelog-since-v1202

Also updater pkgs for:

* talos-systems/pkgs#238 (raspberrypi-firmware update)
* talos-systems/pkgs#242 (Linux 5.10.17 + init_on_free=0)

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-02-19 05:22:34 -08:00
Artem Chernyshev
54d6a45217 feat: add state encryption support
State partition encryption support adds a new section to the machine config.
And a new step to the sequencer flow which saves encryption
configuration object as json serialized value in the META partition.

Everything else is the same as is for the ephemeral partition.
Additionally enabled state partition encryption in the disk encryption
integration tests.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-02-18 06:55:22 -08:00
Artem Chernyshev
58ff2c9808 feat: implement ephemeral partition encryption
This PR introduces the first part of disk encryption support.
New config section `systemDiskEncryption` was added into MachineConfig.
For now it contains only Ephemeral partition encryption.

Encryption itself supports two kinds of keys for now:
- node id deterministic key.
- static key which is hardcoded in the config and mainly used for test
purposes.

Talosctl cluster create can now be told to encrypt ephemeral partition
by using `--encrypt-ephemeral` flag.

Additionally:
- updated pkgs library version.
- changed Dockefile to copy cryptsetup deps from pkgs.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2021-02-17 13:39:04 -08:00
Spencer Smith
6207fa517b test: update aws templates
This PR moves to using our new cluster-api-templates for AWS.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-02-17 16:08:48 -05:00
Spencer Smith
f1d1f72b58 chore(ci): update gcp templates
This PR moves to using the official CAPG provider release, as well as
making use of our snazzy new capi templates.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2021-02-17 10:33:41 -05:00
Andrey Smirnov
d71ac4c4ff feat: update Kubernetes to 1.20.2
Minor point release, official changelog:

https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2021-01-15 09:06:18 -08:00
Andrey Smirnov
a8dd2ff30d fix: checkpoint controller-manager and scheduler
Default manifests created by bootkube so far were only enabling
pod-checkpointer for kube-apiserver. This seems to have issues with
single-node control plane scenario, when without scheduler and
controller-manager node might fall into `NodeAffinity` state.

See https://github.com/talos-systems/bootkube-plugin/pull/23

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-12-28 11:53:17 -08:00
Artem Chernyshev
73c81c501e fix: pass disk image flags to e2e-qemu cluster create command
Forgot to add it in the original PR.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2020-12-22 23:57:31 -08:00
Artem Chernyshev
6540e9bf70 feat: support disk image in talosctl cluster create
Fixes: https://github.com/talos-systems/talos/issues/2973

Can now supply disk image using `--disk-image-path` flag.
May need to enable `--with-apply-config` if it's necessary to bootstrap
nodes properly.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2020-12-22 17:06:00 +03:00
Andrey Smirnov
b1d4814308 feat: update Kubernetes to 1.20.1
See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-12-21 23:52:29 +03:00
Andrey Smirnov
9d1ac81be5 chore: lower MTU to 1450 for the tests in the CI
This should help with the CNI encapsulation in the cluster.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-12-17 17:14:07 +03:00
Andrew Rynhard
6f979d463c test: add ISO test
Adds a simple test for the ISO. Boots the ISO, and then uses the `apply-config` command
in `talosctl` to create a cluster.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2020-12-10 13:04:29 -08:00
Andrey Smirnov
80184393bc feat: update kernel to 5.9.13, new KSPP requirements
Pulls in following changes:

* https://github.com/talos-systems/toolchain/pull/20
* https://github.com/talos-systems/tools/pull/116
* https://github.com/talos-systems/pkgs/pull/214
* https://github.com/talos-systems/pkgs/pull/215
* https://github.com/talos-systems/pkgs/pull/216
* https://github.com/talos-systems/pkgs/pull/217
* https://github.com/talos-systems/go-procfs/pull/4

New empty amd64 images for u-boot & rpi-firmware reduce the size of
amd64 installer image.

For backwards compatibility QEMU provisioner still injects "legacy" KSPP
kernel args into initial boot environment.

Installer correctly upgrades KSPP options when moving from one version
of Talos to another.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-12-10 12:41:58 -08:00
Andrey Smirnov
872e792dbc feat: update Kubernetes to 1.20.0
Official K8s release matching Talos 0.8.0.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-12-09 06:11:48 -08:00
Andrey Smirnov
11c2b8f80c test: bump defaults for provision tests resources
Our defaults are too low now.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-12-07 07:01:41 -08:00
Andrey Smirnov
621968977e feat: update kubernetes to 1.20.0-rc.0
Talos 0.8 is going to ship with K8s 1.20.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-12-02 10:50:58 -08:00
Andrey Smirnov
28ba6e416e feat: update Kubernetes to v1.20.0-beta.2
Talos 0.8 is going to ship with K8s 1.20.x.

Changes to support new `control-plane` label,
upgrade-k8s supports automated fixups for 1.20.

See also: https://github.com/talos-systems/bootkube-plugin/pull/22

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-11-25 06:39:14 -08:00
Andrey Smirnov
1add26b42a chore: bump K8s to 1.19.4 in e2e scripts with CABPT version
This should fix the problem with the kubelet image.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-11-16 07:18:44 -08:00
Andrey Smirnov
61facf700a chore: build arm64 images in CI
This changes installer image/iso output to be tar via stdout
(optionally), so that we can copy back artifacts back from remote docker
daemon.

Fixes #2776

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-11-13 12:34:48 -08:00
Andrey Smirnov
df6ad3fa80 feat: upgrade Kubernetes default version to 1.19.4
k8s.io modules don't have 1.19.4 tag yet :(

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-11-12 08:51:04 -08:00
Andrey Smirnov
350d75eb46 feat: build talosctl-cni-bundle, use it in talosctl for QEMU
This builds a bundle with CNI plugins for talosctl which is
automatically downloaded by `talosctl` if CNI plugins are missing.

CNI directories are moved by default to the `~/.talos/cni` path.

Also add a bunch of pre-flight checks to the QEMU provisioner to make it
easier to bootstrap the Talos QEMU cluster.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-10-30 16:30:37 -07:00
Artem Chernyshev
061b296530 feat: allow specifying user-disks in talosctl cluster create
User-disks are supported by QEMU and Firecracker providers.
Can be defined by using the following parameters:
```
--user-disk /mount/path:1GB
```

Can get more than 1 user disk.
Same set of user disks will be created for all master and worker nodes.

Additionally enable user-disks in qemu e2e test.

Signed-off-by: Artem Chernyshev <artem.0xD2@gmail.com>
2020-10-30 08:44:08 -07:00
Andrew Rynhard
1b0ed13231 docs: move to gridsome
Brings in a new theme, improved content, and restructured layout.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2020-10-26 21:14:14 -07:00
Spencer Smith
7b4633b35d chore: update CI scripts
This PR pulls in the latest version of our CAPI providers, as well as
makes some minor tweaks to our bash scripts to disable terminal output
of commands during certain actions.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-10-22 09:00:41 -07:00
Andrew Rynhard
7017327059 chore: update qemu hack script to use ISO
This can serve as an example of providing the config via an ISO, and
simplify local setups a bit.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2020-09-30 10:58:42 -07:00
Andrey Smirnov
ff0d4b305a feat: build Talos images/artifacts for amd64/arm64
By default, build outside of Drone works the same and builds only amd64
version, loads images back into dockerd, etc.

If multiple platforms are used, multi-arch images are built which can't
be exported to docker or to `.tar` image, they're always pushed to the
registry (even for PR builds to our internal CI registry).

Artifacts as files (initramfs, kernel) now have `-arch` suffix:
`vmlinuz-amd64`, `initramfs-amd64.xz`. "Magic" script normalizes output
paths depending on whether single platform or multiple platforms were
given.

VM provisioners accept magic `${ARCH}` in initramfs/kernel paths which
gets replaced by cluster architecture.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-09-27 10:32:07 -07:00
Andrew Rynhard
7d2741fc4b chore: migrate to ghcr.io
Move to GHCR.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2020-09-23 15:06:30 -07:00
Andrey Smirnov
788cd15c29 test: add e2e test to the provision (upgrade) tests
Add sonobuoy runner code with log fetching on failure. Use hand-picked
set of e2e tests to run: verify basic pod functionality, verify service
connectivity.

Add option `--run-e2e` to the `talosctl health` to run quick e2e test to
verify cluster health.

Add option to run provision tests with custom CNI, run one track of
provision tests with Cilium.

Bump Cilium to 1.8.2.

Talos 0.6 won't uncordon node automatically after upgrade from 0.5, as
0.5 doesn't put annotation. Workaround that in upgrade tests.

Bump upgrade test version to 0.6.0 release.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-09-08 13:26:31 -07:00
Andrew Rynhard
1a4059a553 feat: add grub bootloader
This moves to using grub instead of syslinux.

BREAKING CHANGE: Single node upgrades will fail in this change. This
will also break the A/B fallback setup since this version introduces
an entirely new partition scheme, that any fallback will not know about.
We plan on addressing these issues in a follow up change.

Signed-off-by: Andrew Rynhard <andrew@rynhard.io>
2020-09-01 12:06:43 -07:00
Andrey Smirnov
59adf7315d feat: provide option to run Talos under UEFI in QEMU
This also adds integration pipeline tests for UEFI.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-08-28 12:51:10 -07:00
Spencer Smith
c07ce17b7a fix: update e2e scripts to work with python3
This PR replaces python with python3

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-08-11 12:38:52 -07:00
Spencer Smith
303c477051 chore: update capi CI manifests to use control planes
This PR will update the CI testing to make use of our control plane
provider, as well as the other CAPI components.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-08-11 11:14:44 -04:00
Andrey Smirnov
55f3249783 test: use registry mirrors in CI
This relies on registry caching mirrors running in the CI.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-07-31 16:30:41 +03:00
Andrey Smirnov
58aa2b75bb test: destroy clusters in e2e tests (qemu/firecracker)
As the build runs inside containers which are part of a single pod, we
need to clean up networking bits (bridge interface, etc.), so that it
doesn't cause problems for other steps.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-07-31 06:21:09 -07:00