102 Commits

Author SHA1 Message Date
Andrew Rynhard
ac54a3cb86 chore: add ability to promote to a release
Although the GitHub release plug requires a tag and will fail on a
promotion, this is still useful as it will allow us to mimic a release
before we tag.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-11 11:51:53 -07:00
Andrew Rynhard
2ee769d19e chore: add image test step
Instead of building platform specific images in the default pipeline, we
should build just one image as part of our basic testing to make sure
installations work as expected.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-11 10:51:33 -07:00
Andrew Rynhard
c34ce3a4ed chore: reenable AMI publishing
This was removed during the refactor of our Drone file.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-11 10:07:57 -07:00
Andrew Rynhard
817380bad6 chore: refactor the Jsonnet file
This change improves the drone jsonnet file by making it more DRY and
structuring it in a way that makes it much easier to follow.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-11 09:23:30 -07:00
Andrew Rynhard
620efe52ef chore: fix push step dependencies
We should wait until basic integration is done.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-10 03:52:29 -07:00
Andrey Smirnov
ae54f7e40d fix: stalls in local Docker cluster boot
Problem was triggered by udevd trigger, root cause is not clear, but
workaround is to disable it for container mode.

Implement CPU/mem limits for `osctl cluster create`, apply defaults,
bump defaults for cicd.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-08-10 13:31:47 +03:00
Andrew Rynhard
b965239672 chore: fix clone logic
This is another attempt at fixing the clone logic to make it work when
building the master branch.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-09 23:04:43 -07:00
Andrew Rynhard
217b7e2f9d chore: fix broken clone
This fixes and issue with cloning the master branch caused by git
refusing to fetch into the current branch.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-09 22:44:54 -07:00
Andrew Rynhard
8786916fd0 chore: build drone YAML via jsonnet
This PR aims to DRY the drone config file by using Jsonnet to generate
it.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-08-09 22:30:37 -07:00
Brad Beam
e60a57e186 chore: Fix up adhoc e2e tests
- Wait a little after cluster comes up
- Change interaction with CONFORMANCE variable to work around
  set -eou pipefail restrictions
- Set sonobouy runner version to latest to work with alpha
  version

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-09 13:55:14 -05:00
Brad Beam
bfc1646cd9 chore(ci): Add e2e promotion pipeline
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-08-08 11:27:57 -05:00
Spencer Smith
eea33a2254 chore: enable CIS testing in conformance runs
This PR will run through the kube-bench tests as part of our nightly
conformance runs

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-08-07 17:06:03 -04:00
Spencer Smith
902577b4dc feat: upgrade kubernetes to v1.16.0-alpha.3
This PR updates the kubernetes version constant, as well as pulls in the
new kubeadm image with the last alpha of v1.16.0 baked in. Additionally,
moves the CNI daemon sets to apps/v1, since they're now out of beta.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-08-07 16:05:07 -04:00
Spencer Smith
9e02c77c0a chore: add azure e2e testing
This PR will allow us to run an azure e2e test in parallel with our
current GCE implementation.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-08-07 12:16:32 -04:00
Andrey Smirnov
71640662e0 chore(init): rearrange phase handling to push shutdown to main
This re-arranges phases a bit so that shutdown actions are pushed back
to the top-level main.go of machined.

Small rudimentary event.Bus is introduce to facilitate event passing
(shutdown/restart) between various machined components and main.go. This
might be not the best implementation, just something to allow this
message passing without global variables or such.

Machined API was refactored to run as goroutine service.

ACPI & signal handlers re-built as phase tasks, and activated for
non-container, container modes respectively.

As part of the fix, now `docker stop` triggers correct shutdown of Talos
(not a big deal, but good for testing).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-08-02 08:42:12 -07:00
Spencer Smith
38dfddbab3 feat: break up osctl cluster create and basic/e2e tests
This PR will break cluster create apart from the other steps in
integration tests. It will allow us to run the cluster create, then use
it for parallel e2e builds in different cloud environments.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-08-01 10:55:24 -04:00
Andrey Smirnov
587011e250 chore: remove hack/dev/ scripts & docker-compose
They are outdated, `osctl cluster` implements cluster up/down in a
better way. K8s manifests are left intact, they are used in integration
tests.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-07-30 00:47:58 +03:00
Andrew Rynhard
e63c882b89 refactor: split machined into phases
This change aims to standardize the boot process. It introduces the
concept of a phase, which is comprised of tasks. Phases are ran in serial and
the tasks that make up a phase are ran concurrently.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-29 12:40:03 -07:00
Andrew Rynhard
6852fa969f chore: create raw image as sparse file
This change reduces the size of raw disk significantly by creating it as
a sparse file.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-25 11:28:07 -07:00
Andrew Rynhard
0ec17e4169 feat: run rootfs from squashfs
This change moves the rootfs to a squashfs image.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-25 08:38:31 -07:00
Andrey Smirnov
8c59adb9dc chore: allow to run tests only for specified packages
This allows to do `make test TESTPKGS=./internal/app/machined`.

Also update Dockerfile slug as
https://github.com/moby/buildkit/pull/1081 was merged into master.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-07-23 22:17:22 +03:00
Spencer Smith
089890f36b chore: setup gce for e2e builds
This PR will provide a basis for running e2e tests on GCE several times
a day. We'll need to add a cron event to the drone repo once merged.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-07-22 12:46:02 -04:00
Andrey Smirnov
9f9acf1f05 chore: run tests in the buildkit itself
This relies on two PRs to the buildkit:

* https://github.com/moby/buildkit/pull/1081
* https://github.com/moby/buildkit/pull/1085

Sysfs fix was merged to upstream, so updated tag, while using
`Dockerfile` slug I can switch to dockerfile2llb with support for
`--security=insecure`.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-07-19 07:53:49 -07:00
Brad Beam
e9482a4041 fix: Fix integration of extra kernel args
Switch from `StringSliceVar` to `StringArrayVar` to maintain commas
in kernel args.

Update entrypoint script to allow specifying extra kernel args.

Remove default console settings in kernel config.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-07-16 14:38:55 -05:00
Andrew Rynhard
1e9548d149 feat: use new pkgs for initramfs and rootfs
This brings in the newly compiled libraries and binaries from our new
pkg builds.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-15 10:32:29 -07:00
Andrey Smirnov
82fe5b55e5 chore: make unit-tests use isolated instances of containerd
This makes test launch their own isolated instance of containerd with
its own root/state directories and listening socket address. Each test
brings this instance up/down on its own.

Add options to override containerd address in the code (used only in the
tests).

Enable parallel go test runs once again.

P.S. I wish I could share that 'SetupSuite' phase across the tests, but
afaik there's no way in Go to share `_test.go` code across packages. If
we put it as normal package, this might pull in test dependencies (like
`testify`) into production code, which I don't like.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-07-10 19:46:32 +03:00
Andrew Rynhard
d82444a42d
chore: revert #816 (#829)
This reverts commit 11f8392e53986da6c124037ffab06064ec04aedd.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-04 08:43:34 -07:00
Andrew Rynhard
11f8392e53
chore: fix GCE image creation (#816)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-07-03 18:06:47 -07:00
Andrey Smirnov
0131f839fa
fix: update hack/dev for new userdata location (#777)
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-06-26 00:48:04 +03:00
Andrew Rynhard
69c5adae40
chore: remove ready plugin from CoreDNS (#764)
This plugin is not available in CoreDNS v1.3.1

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-06-21 23:45:40 -07:00
Andrew Rynhard
99cb06d9c7
chore: update floating IPs for E2E test (#762)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-06-21 23:05:17 -07:00
Spencer Smith
0d42a23007 chore: add e2e test (#736)
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-06-21 22:10:43 -07:00
Andrey Smirnov
e86ef87fe8 chore: don't run tests in parallel across packages (#748)
We run tests in parallel mode (`go test -p 4`), default is to run in
parallel in fact. But tests are not isolated, as some of them launch
containerd on a fixed file socket (as socket path is hardcoded in
Talos), and that might lead to any weirdness when tests try to
launch containerd concurrently on the same file socket.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-06-20 16:30:21 -07:00
Spencer Smith
4ba12fecd8
feat(ci): enable nightly e2e tests (#716)
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-06-17 12:36:48 -04:00
Spencer Smith
95b107d884
chore(ci): modularize integration test (#722)
Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-06-06 09:28:53 -04:00
Brad Beam
8537e7eeb6
feat(init): Add support for control plane join config (#700)
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-05-31 12:21:00 -05:00
Andrey Smirnov
f704cb2cc3
refactor(osctl): DRY up osctl sources by using common client setup (#686)
Remove duplicated code which was setting up grpc client with common
method. Should have no functional changes otherwise.

Add args len check where missing.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-05-27 22:55:20 +03:00
Andrew Rynhard
90b5b83b8d
chore: improve the basic integration test (#685)
This PR ensures that we have 3 master nodes.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-27 10:44:09 -07:00
Brad Beam
d8249c8779
refactor(init): Allow kubeadm init on controlplane (#658)
* refactor(init): Allow kubeadm init on controlplane

This shifts the cluster formation from init(bootstrap) and join(control plane)
to init(control plane).

This makes use of the previously implemented initToken to provide a TTL for
cluster initialization to take place and allows us to mostly treat all control
plane nodes equal. This also sets up the path for us to handle master upgrades
and not be concerned with odd behavior when upgrading the previously defined
init node.

To facilitate kubeadm init across all control plane nodes, we make use of the
initToken to run `kubeadm init phase certs` command to generate any missing
certificates once. All other control plane nodes will attempt to sync the
necessary certs/files via all defined trustd endpoints and being the startup
process.

* feat(init): Add service runner context to PreFunc

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-05-24 16:05:49 -05:00
Brad Beam
a64de7ed51
feat(init): Add initToken parameter to userdata (#664)
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-05-20 14:23:38 -05:00
Andrew Rynhard
18a1536b01
feat: use osctl in installer (#654)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-15 16:14:30 -07:00
Andrey Smirnov
4bf649f14c chore: workaround flaky tests (#651)
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-05-13 15:00:59 -07:00
Andrew Rynhard
3c52f5e686
feat: add support for UEFI (#642)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-12 10:11:33 -07:00
Andrew Rynhard
4a0c4d84f3
chore: fix creation of syslinux config file (#639)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-10 07:54:35 -07:00
Andrew Rynhard
64c48353e2
feat: add ability to create multiple entries in extlinux.conf (#636)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-10 04:50:26 -07:00
Andrew Rynhard
86e24601b1
chore: publish AMIs on tags (#633)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-09 13:04:41 -07:00
Andrew Rynhard
86e17c91fb
feat: update partition layout to accomodate upgrades (#621)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-07 13:31:34 -07:00
Andrew Rynhard
67a8343590
chore: build iso image (#616)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-05 13:13:01 -07:00
Andrew Rynhard
00eb0658aa
feat: add support for ISO based installations (#606)
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-05-02 21:30:06 -07:00
Brad Beam
a1ea03aba4
feat: Add calico manifests for local dev setup (#608)
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-05-02 13:29:24 -05:00