216 Commits

Author SHA1 Message Date
Andrew Rynhard
88667641df chore: refactor E2E scripts
This PR aims to simplify our E2E scripts.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-26 20:47:25 -08:00
Andrew Rynhard
c359caef3d chore: fix CI
We need `DOCKER_NET` to be set.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-24 12:42:43 -08:00
Andrew Rynhard
f87c6d74d3 chore: use firecracker in basic-integration
This adds a basic integration step that uses firecracker.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-23 05:52:22 -08:00
Spencer Smith
e0181c85eb feat: allow ability to customize containerd
This PR will allow for any toml files added into `/var/cri/conf.d` to be
picked up and parsed as a containerd config. This should allow users a
nice way to add additional configs by passing extra files in machine
config like:

```
machine:
  ...
  files:
    - content: |
        [metrics]
          address = "0.0.0.0:11234"
      path: /var/cri/conf.d/metrics.toml
      op: create
```

Will close #1718.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-22 17:08:10 -05:00
Spencer Smith
60260c85d1 feat: upgrade kubernetes version to 1.17.1
This PR will bring in the latest point release of k8s 1.17

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-17 09:39:26 -08:00
Andrew Rynhard
40f803de66 chore: run sonobuoy in quick mode
This adds sonobuoy's quick mode test to basic integration.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-17 09:25:43 -08:00
Andrew Rynhard
6533a41da7 chore: fix E2E script
The basic integration cluster name was changed in a previous PR. This
aligns the E2E script with the new naming conventions, and mounts the
correct integration test binary.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-10 07:53:26 -08:00
Andrew Rynhard
d824d0bfdb chore: publish boot.tar.gz
This adds a convenience tarball that includes vmlinuz, and initramfs.xz
in a single tarball.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-09 12:38:21 -08:00
Andrew Rynhard
d123d24b93 chore: allow docgen to ignore a struct
Using a well known comment (docgen: nodoc), we can now tell docgen to
ignore certain structs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-09 11:44:43 -05:00
Spencer Smith
04639824b9 chore: disable iso artifact publication
This PR will disable iso publication for now. We plan to reincorporate the
ability to use ISOs once we've researched #1722.

Will close #1442

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-07 07:01:46 -08:00
Andrew Rynhard
794d9e6066 chore: update all target in Makefile
We should build the most common things by default.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-06 11:08:27 -08:00
Andrew Rynhard
f0732cafcf chore: fix release dependency
The GitHub release should depend on the push step instead of
push-latest.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-02 06:25:21 -08:00
Andrew Rynhard
0778214f1f chore: fix push events
Now that we have a push target and push-% target, we can simplify the drone
conditions. This updates the conditions so that the latest channel updates
on pushes to master, the edge channel updates on successful nightly cron, and
an image with the standard tag is pushed in all events except pull requests.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-01 13:00:12 -08:00
Andrew Rynhard
288d4d0b51 chore: push latest tag on tag events
This ensures that the latest tag is updated on git tag events.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-01 11:41:49 -08:00
Andrew Rynhard
e6a16d5572 chore: use the correct condition for latest and edge pushes
This updates the drone conditions to push the latest tag only
for pushes to the master branch. Additionally, the edge tag will be
pushed only when the nightly cron is executed.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-01 10:23:11 -08:00
Andrew Rynhard
7f2483e848 chore: fix releases
The GitHub release plugin doesn't allow directories, and has no way to
tell it to ignore a path. The workaround is to be explicit about what
files we want in a release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-30 18:13:14 -08:00
Andrey Smirnov
ebd40bd0eb chore: use osctl cluster --wait in basic-integration
There are few workarounds for Drone way of running integration test:
DinD runs as a separate pod, and we can only access its exposed on the
"host" ports, while from Talos cluster this endpoint is not reachable.

So internally Talos nodes still use addresses like "10.5.0.2", while
test is using "docker" to access it (that's name of the `docker` service
in the pipeline).

When running locally, 127.0.0.1 is used as endpoint, which should work
fine both on OS X and Linux.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-30 15:15:42 -08:00
Andrew Rynhard
9faf4907f2 chore: exclude cron events in push-latest step
We shouldn't push latest on cron events.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-27 17:23:50 -08:00
Andrew Rynhard
0b23727ad3 chore: fix conformance
The drone linter complained about duplicate steps. This removes them.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-27 15:38:14 -08:00
Andrew Rynhard
c8d3da5376 chore: add more functions to the release script
This adds functionality to update the CHANGELOG, and cherry-pick a
commit into a release branch.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-26 08:52:56 -08:00
Andrew Rynhard
6aa4a9e305 chore: remove gitmeta references
We no longer depend on gitmeta.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-26 07:05:09 -08:00
Andrew Rynhard
5a7eb631b2 feat: add installer command to installer container
This replaces the entrypoint.sh shell script with a go binary.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-26 06:41:25 -08:00
Andrew Rynhard
e4a1bc3cf9 chore: add help menu to the Makefile
This adds a help  menu to the Makefile. It documents all build
dependencies, and how to get started.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-25 11:11:41 -08:00
Andrew Rynhard
831f5524a1 chore: refactor Makefile to be more DRY
This PR aims to make the Makefile more DRY.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-24 10:48:32 -08:00
Andrew Rynhard
6602a85976 chore: use docker buildx
This replaces buildkit and buildctl with the docker buildx plugin.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-24 08:30:39 -08:00
Andrew Rynhard
31baa14e36 feat: add support for tftp download
This adds support for downloading the machine config over TFTP. This
will allow users to avoid having to setup an HTTP server, and use
whatever they are using for PXE.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-18 09:28:38 -08:00
Andrew Rynhard
49de6e96b3 chore: fix KVM test
This adds hostname and domain name options to the DHCP response.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-18 09:13:03 -08:00
Brad Beam
9584b47cd7 feat: Upgrade kubernetes to 1.17.0
Primarily doc/constant changes.

Added additionnal bits to `docs` target in makefile to generate osctl
docs as well as config files. Explicitly define a HOME variable so we
get consistent home directories for talosconfig variables in our docs.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-12-10 16:03:35 -08:00
Andrew Rynhard
5fb12f735a chore: make the CNI configurable in local KVM test
This changes the local KVM setup to use a configurable CNI URL.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-10 08:08:26 -08:00
Andrey Smirnov
399aeda0b9 feat: rename confusing target options, --endpoints, etc.
Fixes #1610

1. In `talosconfig`, deprecate `Target` in favor of `Endpoints`
(client-side LB to come next).

2. In `osctl`, use `--nodes` in place of `--target`.

3. In `osctl` add option `--endpoints` to override `Endpoints` for the
call.

Other changes are just updates to catch up with the changes. Most
probably I missed something... And CAPI provider needs update.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-10 02:23:54 +03:00
Andrew Rynhard
f7f85d7585 chore: upgrade sonobuoy to v0.16.5
This brings in the latest sonobuoy.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-05 14:36:04 -08:00
Andrey Smirnov
edb40437ec feat: add support for osctl logs -f
Now default is not to follow the logs (which is similar to `kubectl logs`).

Integration test was added for `Logs()` API and `osctl logs` command.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-05 13:58:52 -08:00
Spencer Smith
509ec5b6ff chore: update gcp disk sizes
This PR updates the disks to 100GB for hopes of better disk perf.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-05 13:55:40 -08:00
Spencer Smith
264c5440ef chore: rewrite basic integration in go instead of bash
This PR will be the start of several. It rewrites the basic integration
in go. We'll do these one at a time.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-05 15:55:19 -05:00
Andrew Rynhard
1d3cc0038b feat: use containerd-shim-runc-v2
This configures the CRI containerd to use containerd-shim-runc-v2.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 14:36:18 -08:00
Spencer Smith
878f95b20b chore: support image specification in drone step function
This PR adds the ability to override the image in our default build step
function in jsonnet. Needed so we can override the image easily for
basic integration tests.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-04 11:23:56 -05:00
Andrew Rynhard
9f9fd02ceb chore: fix conformance
The `--e2e-parallel` flag seems to skip all tests when running in
certified-conformance mode. This reverts that change, and also adds a
check that fails if the conformance tests do not pass. This ensures that
we are not publishing broken versions of our edge release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 17:08:57 -08:00
Andrew Rynhard
712275dfea chore: upgrade sonobuoy
This upgrades sonouoy and additionally adds the `--e2e-parallel` flag to
hopefully speed things up.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-28 12:13:17 -08:00
Andrew Rynhard
70b9186be0 chore: push edge tag on succesful conformance
This adds a step to the conformance pipeline that pushes all containers
with the tag "edge." This Will allow us to start using and edge
"channel" for upgrades.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 08:10:25 -08:00
Andrew Rynhard
031c65be47 feat: add IMA policy
This creates an IMA policy at boot. It uses the default TCB policy with
a dont_measure rule for XFS.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 16:49:48 -08:00
Andrew Rynhard
103620dc5c chore: add ability to specify custom intaller to libvirt setup
This is useful when developing Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 14:57:18 -08:00
Spencer Smith
1ade2673d7 chore: disable all azure e2e temporarily
This PR removes all refs to the azure e2e/integration/conformance tests
for now, since we need to wait on some upstream CAPI fixes and the test
is currently broken.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-25 12:01:56 -05:00
Andrew Rynhard
ae83221e4a test: add integration test for full boot sequence
This adds an integration test that can be ran on a KVM enabled Linux
machine. It makes use of docker, matchbox, dnsmasq, libvirt, and HAproxy
to create an HA cluster.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-15 09:02:52 -08:00
Andrew Rynhard
4680f66bc5 docs: add autogenerated config reference
This adds a small program to parse our config structs and generate
markdown from them. This will allow us to enforce a standard and require
documentation for fields as they get added.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 08:38:39 -08:00
Brad Beam
8988c1c6a0 feat: Disable networkd configuration if ip kernel parameter is specified
This allows the kernel argument `ip` to take precedence over networking configuration. Documentation for
this parameter can be found here https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-10 12:07:01 -08:00
Spencer Smith
6d5bbaf7c8 chore: re-enable e2e for aws clusters
This PR adds in the necessary manifests and fixes to deploy aws clusters
as part of e2e testing.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-07 15:32:14 -05:00
Andrey Smirnov
8fdf71789e test: add 'integration-test' to e2e runs
Also refactored `integration-test` build as a generic step to be shared
by basic-integration and e2e-integration steps.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-07 06:30:34 -08:00
Spencer Smith
ce7a0e36cc chore: re-enable e2e testing
This PR will re-enable e2e testing by using the new cluster api
bootstrap provider and various infra providers.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-05 16:53:38 -05:00
Andrey Smirnov
b0aef2cf22 test: add integration test framework
This is just first steps and core foundation.

It can be used like:

```
make integration.test
osctl cluster create
build/integration.test -test.v
```

This should run the test against the Docker instance.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-05 17:21:38 +03:00
Brad Beam
a4e1479b07 refactor: Move kubeconfig to machined
This moves the Kubeconfig api endpoint to machined and consolidates the
"read a file" code into machined. This also changes Kubeconfig to
use the CopyOut method which changes Kubeconfig to a streaming grpc call.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-04 14:45:23 -08:00