185 Commits

Author SHA1 Message Date
Andrey Smirnov
edb40437ec feat: add support for osctl logs -f
Now default is not to follow the logs (which is similar to `kubectl logs`).

Integration test was added for `Logs()` API and `osctl logs` command.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-05 13:58:52 -08:00
Spencer Smith
509ec5b6ff chore: update gcp disk sizes
This PR updates the disks to 100GB for hopes of better disk perf.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-05 13:55:40 -08:00
Spencer Smith
264c5440ef chore: rewrite basic integration in go instead of bash
This PR will be the start of several. It rewrites the basic integration
in go. We'll do these one at a time.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-05 15:55:19 -05:00
Andrew Rynhard
1d3cc0038b feat: use containerd-shim-runc-v2
This configures the CRI containerd to use containerd-shim-runc-v2.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 14:36:18 -08:00
Spencer Smith
878f95b20b chore: support image specification in drone step function
This PR adds the ability to override the image in our default build step
function in jsonnet. Needed so we can override the image easily for
basic integration tests.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-04 11:23:56 -05:00
Andrew Rynhard
9f9fd02ceb chore: fix conformance
The `--e2e-parallel` flag seems to skip all tests when running in
certified-conformance mode. This reverts that change, and also adds a
check that fails if the conformance tests do not pass. This ensures that
we are not publishing broken versions of our edge release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 17:08:57 -08:00
Andrew Rynhard
712275dfea chore: upgrade sonobuoy
This upgrades sonouoy and additionally adds the `--e2e-parallel` flag to
hopefully speed things up.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-28 12:13:17 -08:00
Andrew Rynhard
70b9186be0 chore: push edge tag on succesful conformance
This adds a step to the conformance pipeline that pushes all containers
with the tag "edge." This Will allow us to start using and edge
"channel" for upgrades.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 08:10:25 -08:00
Andrew Rynhard
031c65be47 feat: add IMA policy
This creates an IMA policy at boot. It uses the default TCB policy with
a dont_measure rule for XFS.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 16:49:48 -08:00
Andrew Rynhard
103620dc5c chore: add ability to specify custom intaller to libvirt setup
This is useful when developing Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 14:57:18 -08:00
Spencer Smith
1ade2673d7 chore: disable all azure e2e temporarily
This PR removes all refs to the azure e2e/integration/conformance tests
for now, since we need to wait on some upstream CAPI fixes and the test
is currently broken.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-25 12:01:56 -05:00
Andrew Rynhard
ae83221e4a test: add integration test for full boot sequence
This adds an integration test that can be ran on a KVM enabled Linux
machine. It makes use of docker, matchbox, dnsmasq, libvirt, and HAproxy
to create an HA cluster.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-15 09:02:52 -08:00
Andrew Rynhard
4680f66bc5 docs: add autogenerated config reference
This adds a small program to parse our config structs and generate
markdown from them. This will allow us to enforce a standard and require
documentation for fields as they get added.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 08:38:39 -08:00
Brad Beam
8988c1c6a0 feat: Disable networkd configuration if ip kernel parameter is specified
This allows the kernel argument `ip` to take precedence over networking configuration. Documentation for
this parameter can be found here https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-10 12:07:01 -08:00
Spencer Smith
6d5bbaf7c8 chore: re-enable e2e for aws clusters
This PR adds in the necessary manifests and fixes to deploy aws clusters
as part of e2e testing.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-07 15:32:14 -05:00
Andrey Smirnov
8fdf71789e test: add 'integration-test' to e2e runs
Also refactored `integration-test` build as a generic step to be shared
by basic-integration and e2e-integration steps.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-07 06:30:34 -08:00
Spencer Smith
ce7a0e36cc chore: re-enable e2e testing
This PR will re-enable e2e testing by using the new cluster api
bootstrap provider and various infra providers.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-05 16:53:38 -05:00
Andrey Smirnov
b0aef2cf22 test: add integration test framework
This is just first steps and core foundation.

It can be used like:

```
make integration.test
osctl cluster create
build/integration.test -test.v
```

This should run the test against the Docker instance.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-05 17:21:38 +03:00
Brad Beam
a4e1479b07 refactor: Move kubeconfig to machined
This moves the Kubeconfig api endpoint to machined and consolidates the
"read a file" code into machined. This also changes Kubeconfig to
use the CopyOut method which changes Kubeconfig to a streaming grpc call.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-04 14:45:23 -08:00
Andrew Rynhard
a3dc6adec1 chore: remove unused files
This removes unused files in hack.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-31 22:46:38 -07:00
Andrew Rynhard
03a26f5836 chore: prepare release v0.3.0-alpha.5
This is the official v0.3.0-alpha.5 release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-31 15:35:41 -07:00
Brad Beam
ca76ccd4af feat: Add support for creating VMware images
This PR adds support for generating VMware compatible images as an ova.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-31 13:39:54 -05:00
Andrew Rynhard
3c6d0135d0 feat: upgrade Kubernetes to 1.16.2
This brings in 1.16.2 modules and bumps the default hyperkube image.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-30 06:35:12 -07:00
Andrew Rynhard
73d76307b0 chore: add Digital Ocean image to release
This will add a step to build the Digital Ocean image.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-28 15:37:34 -07:00
Brad Beam
573cce8d18 feat: Add APId
This PR introduces APId. This service replaces the frontend functionality
previously provided by OSD. The main driver for this is two fold:

1. Create a single purpose application to expose the talos api

2. Make use of code generation to DRY api changes

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-25 13:02:33 -05:00
Andrew Rynhard
7a4b4d42b5 docs: add v0.3 AWS guide
This adds documentation for v0.3 AWS users.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-23 15:20:07 -07:00
Andrey Smirnov
811fd6706a chore: make Slack notifications more fancy
Uses some examples found on the Internet :)

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-22 14:23:25 -07:00
Andrew Rynhard
638d36bce7 fix: ensure control plane endpoint is set
We were mistakenly overwriting the control plane endpoint in the
`generate` command. This fixes that and adds a simple validation of the
endpoint field in the config. We should expand the validation to ensure
that a valid IP or DNS name have been provided.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-21 19:44:20 -07:00
Andrey Smirnov
7f6fdc3e6f chore: run 'git fetch --tags' as first step
This is critical to get correct `gitmeta image tag`.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-21 15:12:15 -07:00
Brad Beam
e6bf92ce31 feat(osd): Enable hitting multiple OSD endpoints
This enables the ability to specify additional <talos> endpoints to connect to
to pull back data.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-10-16 15:30:25 -05:00
Andrew Rynhard
6c33547452 fix: add slub_debug=P to ISO kernel args
This option is required by KSPP.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 10:57:56 -07:00
Andrew Rynhard
792a35e8ae fix: use talos.config instead of talos.userdata
The new kernel parameter talos.config should be used instead of
tallos.userdata.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 10:44:59 -07:00
Andrew Rynhard
80e3876df5 feat: remove proxyd
We have decided that proxyd is not the best architectue for HA
Kubernetes. Our recommendation to users will be to create a load
balancer instead.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 08:11:00 -07:00
Andrew Rynhard
fef151748b feat: use the unified pkgs repo artifacts
This moves to using a single revision of pkgs. It includes a few
changes:

- kernel with KVM host support
- containerd v1.3.0

This change brings in a kernel with host KVM support. This will allow us
to use VMs within Talos for things like integrations tests. This also
allows users to do things with KVM as they see fit.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-14 07:18:17 -07:00
Spencer Smith
5d5f530bb0 chore: update sonobuoy for conformance tests
This PR updates the sonobuoy version. We're currently running
conformance tests with 0.15.x

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-10-10 18:26:05 -07:00
Spencer Smith
313ca2cb23 chore: re-enable end to end tests
This PR will add the bits necessary to make use of changes to our
v1alpha1 cluster api provider for CI testing. This is needed since we've
had machine config changes.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-10-10 17:32:44 -04:00
Andrey Smirnov
c2cb0f9778 chore: enable 'wsl' linter and fix all the issues
I wish there were less of them :)

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-10 01:16:29 +03:00
Andrey Smirnov
bb5f5cc754 chore: bump golangci-lint to 1.20
Memory usage reduced around 8-10x: now it stays stable at 1GB.

I disabled some of the new linters, and one rule which is violated a
lot.

I might make sense to go back and enable `wsl` fixing all the issues
(leaving that for another PR).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-10-09 22:21:08 +03:00
Andrew Rynhard
4454afef2f feat: default docker based cluster to 1 master
The goal with the docker based cluster is to provide developers with an
easy way to run Kubernetes on their local machines. Most of the time,
they won't need more than 1 master. This defaults cluster creation to 1
master.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-08 19:04:54 -07:00
Andrew Rynhard
b29391f0be feat: use bootkube for cluster creation
This replaces kubeadm with bootkube.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-07 17:17:57 -07:00
Andrew Rynhard
4ae8186107 feat: add configurator interface
This moves from translating a config into an internal config
representation, to using an interface. The idea is that an interface
gives us stronger compile time checks, and will prevent us from having to copy
from on struct to another. As long as a concrete type implements the
Configurator interface, it can be used to provide instructions to Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-10-04 07:53:09 -07:00
Seán C McCord
5686ba2db3 feat: Allow env override of hack/qemu image location
This fixes #1220

Signed-off-by: Seán C McCord <ulexus@gmail.com>
2019-09-29 07:10:20 -07:00
Andrew Rynhard
27adda4d9d chore: use the official Drone git plugin
The changes we needed in the clone plugin have been merged. We should
use the official plugin to minimize what we have to maintain.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-23 22:45:31 -07:00
Andrew Rynhard
82c706a0fb feat: upgrade Kubernetes to v1.16.0
Brings in Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-19 20:19:29 -07:00
Andrew Rynhard
6efd6fbe08 chore: move gRPC API to public
In order for other projects to make use of our APIs, they must not
reside underneath the internal directory. This moves the protobuf
definitions to a top-level "api" directory and scopes them according to
their domain. This change also removes generated code from the gitignore
file so that users don't have to generate the code themseleves.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-19 08:55:13 -07:00
Andrew Rynhard
20302eb8f6 chore: fix AWS image dependency
We no longer need to wait for the installer image to be pushed before
creating the AWS image.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-17 21:12:03 -07:00
Andrew Rynhard
472f1aa6e8 chore: upgrade Sonobuoy to v0.15.4
This version has a fix for a bug that is affecting us.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-17 14:52:10 -07:00
Andrew Rynhard
3e62973b2c chore: upgrade conformange image
This upgrade the kube-conformance image used by sonobouy to
v1.16.0-rc.2.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-16 16:05:24 -07:00
Andrew Rynhard
ab4e058489 feat: upgrade Kubernetes to v1.16.0-rc.2
This brings in the release candidate for Kubernetes v1.16.0.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-16 14:56:55 -07:00
Andrew Rynhard
75746266ce feat: upgrade Kubernetes to v1.16.0-rc.1
This brings in the latest RC of 1.16.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-09-12 20:20:48 -07:00