1178 Commits

Author SHA1 Message Date
Andrew Rynhard
d4c202438c refactor: set CRI config to /etc/cri/containerd.toml
This changes the CRI specific containerd instance's config to a
different path.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 19:32:00 -08:00
Seán C McCord
9d9b958fba fix: reverse preference order of network config
Kernel config should always play second to a file-based config.

Fixes #1588

Signed-off-by: Seán C McCord <ulexus@gmail.com>
2019-12-04 17:01:05 -08:00
Andrew Rynhard
1f4c17269d feat: add universal TUN/TAP device driver support
This is required when doing anything with KVM.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 15:25:39 -08:00
Andrew Rynhard
1d3cc0038b feat: use containerd-shim-runc-v2
This configures the CRI containerd to use containerd-shim-runc-v2.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 14:36:18 -08:00
Andrey Smirnov
fc52025490 fix: provide peer remote address for 'NODE': as default in osctl
This change is pretty mechanical, just wrap every API so that remote
peer address is used as default for `resp.Metadata.Hostname`.

This makes `NODE:` non-empty in all the API calls.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-05 00:11:55 +03:00
Andrew Rynhard
43e6703b8b feat: upgrade containerd to v1.3.2
This brings in the latest version of Containerd.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-04 10:19:51 -08:00
Spencer Smith
878f95b20b chore: support image specification in drone step function
This PR adds the ability to override the image in our default build step
function in jsonnet. Needed so we can override the image easily for
basic integration tests.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-04 11:23:56 -05:00
Andrey Smirnov
5b316f7ea3 feat: osctl logs now supports multiple targets
Update cli so that with multiple targets work pretty much like
`tail -f one.log two.log`.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-04 17:54:58 +03:00
Spencer Smith
739ce61efa feat: support output directory for osctl config generate
This PR adds a `--output-dir/-o` flag to osctl config generate and will
attempt to create the directory path and write the generated files to
that location. Will close #1509.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-03 17:29:38 -05:00
Spencer Smith
23bfa8b299 chore: validate url input for osctl config generate
This PR closes #1372.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-03 17:23:18 -05:00
Spencer Smith
190f0c6281 feat: support client only version for osctl
This PR adds the `--client` flag to `osctl version` so that we exit
before attempting to contact the server to get its version. Will close #1363

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-03 17:06:23 -05:00
Spencer Smith
7c1d950686 docs: clarify vmware instructions
This PR clarifies a few nits about the vmware docs. We were referring to
alpha.2 in lots of places. I moved this to a note that just says to set
TALOS_VERSION to `v0.3.0-alpha.10` or similar. Also clarifies the path
to the ova could be any /path/to/downloaded/file.

Should close #1572 when we've gotten some extra info about the hardware
versions.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-03 16:40:50 -05:00
Andrew Rynhard
3e5ca30aa5 refactor: simplify NewTemporaryClientFromPKI
This is a simple refactor that reduces the number of arguments required
by `NewTemporaryClientFromPKI`.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-03 09:10:24 -08:00
Andrey Smirnov
96a7289f06 test: fix integration version test as 'NODE:' might be missing
When invoked without `-t`, `osctl` shouldn't print `NODE:` anymore.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-03 07:45:41 -08:00
Tim Gerla
4debea6856 feat: allow deep-linking to specific docs pages
This changes the way the sidebar anchors are constructed, and if
there is an anchor in the URL, the Content component will render
the requested page, instead of just the top-level docs page.

Signed-off-by: Tim Gerla <tim@gerla.net>
2019-12-03 07:31:48 -08:00
Andrew Rynhard
c09c8cec24 chore: prepare release v0.3.0-alpha.10
This is the official v0.3.0-alpha.10 release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
v0.3.0-alpha.10
2019-12-02 15:40:40 -08:00
Andrew Rynhard
9745c3a504 fix: update kernel version constant
This is needed in order for integration tests to pass.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-02 15:26:28 -08:00
Andrey Smirnov
f0a080a340 fix: osctl panic when metadata is nil
If `osctl` invocation doesn't have `-t`, response doesn't contain
`.Metadata` field, so we need to check for it explicitly.

Some commands in osctl had this check, some didn't, now all the commands
have this check.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-02 23:49:34 +03:00
Brad Beam
05c1659126 feat(networkd): Add support for kernel nfsroot arguments.
This adds support for parsing/honoring the `ip=` kernel argument that can
be supplied to configure an interface on the host.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-12-02 09:57:05 -08:00
Spencer Smith
c8f7336569 feat: support force flag for osctl kubeconfig
This PR allows you to specify the kubeconfig command like `osctl
kubeconfig . --force` or `osctl kubeconfig . -f` to delete the
kubeconfig file before we write it out.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-02 11:40:32 -05:00
Andrew Rynhard
aef38d0e11 fix: prevent nil pointer panic
This KubeletConfig should be checked for `nil`, and initialized if
needed, before attempting to access its' fields.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-02 08:04:43 -08:00
Andrew Rynhard
30e40f6d18 docs: add automated upgrades proposal
This is a proposal for the new way of running Talos and performing upgrades in v0.2.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-02 07:07:55 -08:00
Andrew Rynhard
c630a936dd docs: fix documentation link
Updates the documentation link in the README.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 22:43:42 -08:00
Andrew Rynhard
9c0b0a1c46 chore: upgrade packages
This brings in the latest set of packages with the following changes:

- Linux v5.3.14
- Pinned ca-certificates (2019-11-27)

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 21:10:19 -08:00
Andrew Rynhard
21c4aa8aa6 feat: enable webhook authorization mode
This moves to using Webhook mode instead of the default AlwaysAllow for
kubelet API authorization.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 17:34:00 -08:00
Andrew Rynhard
9f9fd02ceb chore: fix conformance
The `--e2e-parallel` flag seems to skip all tests when running in
certified-conformance mode. This reverts that change, and also adds a
check that fails if the conformance tests do not pass. This ensures that
we are not publishing broken versions of our edge release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 17:08:57 -08:00
Andrew Rynhard
6ab9877b72 chore: update bootkube
This brings in changes from upstream bootkube. It fixes an issue with
the pod-checkpointer that would cause the pod-checkpointer to fail if
the kubelet's read-only port were disabled. It also adds a dedicated
certificate for the API server's `kubelet-client-*` args, which will allow the
usage of the `authentication-token-webhook` flag in the kubelet.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-01 14:00:50 -08:00
Andrey Smirnov
ad2f2574d7 fix: provide a way for client TLS config to use Provider
In `tls.Config`, there are two hooks for getting certificate for client
and server config. So we need separate configuration methods to
configure them both.

Required in apid to provide refreshing TLS client cert to
grpc.ClientConn.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-29 23:35:23 +03:00
Andrey Smirnov
5b7bea2471 feat: use grpc-proxy in apid
This replaces codegen version of apid proxying with
talos-systems/grpc-proxy based version. Proxying is transparent, it
doesn't require exact information about methods and response types. It
requires some common layout response to enhance it properly with node
metadata or errors.

There should be no signifcant changes to the API with the previous
version, but it's worth mentioning a few changes:

1. grpc.ClientConn is established just once per upstream (either local
service or remote apid instance).

2. When called without `-t` (`targets`), apid proxies immediately down
to local service skipping proxying to itself (as before), which results
in empty node metadata in response (before it had local node IP). Might
revert this later to proxy to itself (?).

3. Streaming APIs are now fully supported with multiple targets, but
message definition doesn't contain `ResponseMetadata`, so streaming APIs
are broken now with targets (needs a fix).

4. Errors are now returned as responses with `Error` field set in
`ResponseMetadata`, this requires client library update and `osctl` to
handle it properly.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-11-29 22:57:25 +03:00
Andrew Rynhard
712275dfea chore: upgrade sonobuoy
This upgrades sonouoy and additionally adds the `--e2e-parallel` flag to
hopefully speed things up.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-28 12:13:17 -08:00
Andrew Rynhard
9ea041c7d9 feat: upgrade packages
This brings in fix patches for:

- ca-certificates
- eudev
- libressl
- libseccomp

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 13:24:25 -08:00
Tim Gerla
83d9e01217 feat: add Google Analytics tracking to the project website
- add the Google Analytics tracking package and tracking ID.

Signed-off-by: Tim Gerla <tim@gerla.net>
2019-11-27 10:56:15 -08:00
Andrew Rynhard
aaefcbd891 fix: recover control plane on reboot
This brings in a patched version of the pod-checkpointer. It fixes a bug
that prevented the static pod-checkpointer from being scheduled,
preventing recovery of the control plane on a reboot of all control
plane nodes.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 10:47:51 -08:00
Andrew Rynhard
f3882e7e0a fix: ensure etcd comes back up on reboot of all members
This adds a check for the existence of data in the etcd data directory,
that we then use to conditionally set the initial cluster args to handle
the case of a reboot of all etcd members at once. Without this, etcd
fails to come back up, and effectively kills the cluster.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 10:06:15 -08:00
Andrew Rynhard
48d5aac0fc feat: enable aggregation layer
This moves to using our official bootkube repo. The latest changes in
the branch we are using enables the aggregation layer. This should fix
our conformance.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 08:40:58 -08:00
Andrew Rynhard
70b9186be0 chore: push edge tag on succesful conformance
This adds a step to the conformance pipeline that pushes all containers
with the tag "edge." This Will allow us to start using and edge
"channel" for upgrades.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 08:10:25 -08:00
Brad Beam
a6ab1ec2a5 chore(networkd): Ignore bonded interfaces without config
This change sets bonded devices to ignored if there is no user supplied
configuration. Without configuration, a bonded interfaces doesnt provide
any value. This should speed up initial boot times by preventing address
discovery on this interface.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-27 08:10:07 -08:00
Andrew Rynhard
c9a91b7d9d fix: require mode flag when validating
This flag should be required.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-27 08:09:49 -08:00
Andrew Rynhard
4efccd96ea refactor: rename virtual package to pseudo
This aligns the nomenclature for filesystems like /dev and /proc with
what is used in the kernel code.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 22:32:48 -08:00
Brad Beam
119bf3e7bb feat(networkd): Add support for bonding
This includes a healthy refactor of the networkd code as well.
- Move netlink functionality to nic package
- Networkd facilitates the orchestration of the underlying interface configuration
- Networkd now stores the state of each interface configuration. This
  should allow us to expose this information via api in the future.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-26 20:08:31 -08:00
Andrew Rynhard
4bec94f655 fix: don't measure overlayfs
Measuring overlayfs spams the measurement logs. Since these are mostly files
in pods, we don't get a lot of benefit from measuring them. This adds a rule
to ignore overlayfs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 19:11:49 -08:00
Andrew Rynhard
031c65be47 feat: add IMA policy
This creates an IMA policy at boot. It uses the default TCB policy with
a dont_measure rule for XFS.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 16:49:48 -08:00
Andrew Rynhard
3f49a15c06 feat: enable IMA measurement and appraisal
This updates the kernel to make use of a version that has IMA
measurement and appraisal enabled. It is not yet enforced. Additionally,
this adds the securityfs mount at /sys/kernel/security.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-26 03:15:58 -08:00
Andrew Rynhard
103620dc5c chore: add ability to specify custom intaller to libvirt setup
This is useful when developing Talos.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 14:57:18 -08:00
Andrew Rynhard
596fcff741 chore: prepare release v0.3.0-alpha.9
This is the official v0.3.0-alpha.9 release.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
v0.3.0-alpha.9
2019-11-25 12:28:46 -08:00
Andrew Rynhard
ac089dc330 feat: add read API
This adds an API for reading arbitrary files.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 10:46:50 -08:00
Andrew Rynhard
43ad18fbee feat: allow sysctl writes
This adds the ability to write sysctls on boot by specifying them in the
machine config.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 10:46:28 -08:00
Andrew Rynhard
e78e1655f1 feat: upgrade packages
This brings in the following changes:

- Linux 5.3.13
- Containerd 1.3.1

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 10:41:47 -08:00
Spencer Smith
1ade2673d7 chore: disable all azure e2e temporarily
This PR removes all refs to the azure e2e/integration/conformance tests
for now, since we need to wait on some upstream CAPI fixes and the test
is currently broken.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-25 12:01:56 -05:00
Brad Beam
28ee910899 chore: Fix formatting ( make fmt )
Not sure if there was an update in the fmt code path, but these are the
results after running `make fmt`.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-23 13:50:52 -08:00