147 Commits

Author SHA1 Message Date
Andrey Smirnov
fae5e6915d chore: rework firecracker code around upstream Go SDK + PRs
This removes use of private fork with custom `ip=` kernel argument
handling and switches fully to upstream version of it.

Firecracker Go SDK version is `master` + following PRs:

* https://github.com/firecracker-microvm/firecracker-go-sdk/pull/167
* https://github.com/firecracker-microvm/firecracker-go-sdk/pull/177
* https://github.com/firecracker-microvm/firecracker-go-sdk/pull/178

MTU handling support was implemented as well.

Changes:

* hostname to each node is passed via `talos.hostname=` kernel arg
* IP configuration is generated by SDK from CNI result
* fixed bugs with wrong netmask
* nameservers & MTU is passed via Talos config

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-29 02:35:15 +03:00
Brad Beam
defbcf3856 docs(apid): Add apid docs
Describes apid and introduces some workflows to illustrate what apid does.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2020-01-28 11:36:13 -08:00
Andrew Rynhard
d36b3a50d6 docs: remove invalid field from docs
This removes `extraDiskArgs` from the kubelet configuration field. This
never really was a thing.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-28 07:35:27 -08:00
Andrew Rynhard
6d1a2f7b6d fix: implement kubelet extra mounts
This implements old behavior from our initial config that allows for
specifying extra mounts for the kubelet container.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-28 06:44:27 -08:00
Andrew Rynhard
428aeef556 docs: fix machined component
This fixes the sidebar to only have machined, not init and machined.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-23 15:50:28 -08:00
Andrew Rynhard
4a3629dbfd docs: update metal section
This simpifies the metal section overview.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-23 05:53:54 -08:00
Spencer Smith
e0181c85eb feat: allow ability to customize containerd
This PR will allow for any toml files added into `/var/cri/conf.d` to be
picked up and parsed as a containerd config. This should allow users a
nice way to add additional configs by passing extra files in machine
config like:

```
machine:
  ...
  files:
    - content: |
        [metrics]
          address = "0.0.0.0:11234"
      path: /var/cri/conf.d/metrics.toml
      op: create
```

Will close #1718.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-22 17:08:10 -05:00
Andrew Rynhard
b4dc1526ee docs: remove pre-release from v0.3 docs
This change updates the docs toggle menu to remove "pre-release" from
the v0.3 option.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-22 09:16:22 -08:00
Andrey Smirnov
9da687d2a3 test: firecracker provisioner fixes, implement cluster destroy
This implements `osctl cluster destroy` for Firecracker, adds
new utility command `osctl cluser show`.

Firecracker mode now has control process for firecracker VMs, allowing
clean reboots and background operations.

Lots of small fixes to Firecracker mode, clean CNI shutdown, cleaning up
netns, etc.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2020-01-21 17:11:06 -08:00
Spencer Smith
67e50f6f50 feat: allow for bootkube images to be customized
This PR allows for pod checkpointer and coredns images to be customized
for bootkube. We can already customize the hyperkube image and all other
images used by bootkube are CNI-related and can be customized with the
"custom" CNI setup.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-21 11:17:28 -08:00
Spencer Smith
60260c85d1 feat: upgrade kubernetes version to 1.17.1
This PR will bring in the latest point release of k8s 1.17

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-17 09:39:26 -08:00
Andrew Rynhard
d933ca9a88 docs: add missing docs
The `docs` target needed to be ran.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2020-01-15 19:08:41 -08:00
Spencer Smith
4b81907bd3 feat: allow additional manifests to be provided to bootkube
This PR will add an `additionalManifests` field to the config data that
allows users to specify a list of URLs that they'd like to fetch
manifests from. These manifests will then be added to the bootkube asset
directory and applied during the bootkube service.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-15 08:48:41 -08:00
Tim Gerla
d6f5ff3414 feat: add a basic architectural diagram and a call to action
- add an architecture diagram
- add a call-to-action button on the front page
- clean up some lint issues
- adjust the way the "chips" image is displayed
- move the K8s certified logo to the "Features" section

Signed-off-by: Tim Gerla <tim@gerla.net>
2020-01-10 07:44:47 -08:00
Spencer Smith
d0476dbc93 docs: reorganize components sidebar and add ntpd
This PR alphabetizes the components and adds in the missing ntpd link.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-10 10:22:41 -05:00
Spencer Smith
75d9f7b454 feat: support configurable docker-based clusters
This PR will allow users to issue `osctl config generate`, tweak the
configs to their liking, then use those configs to call `osctl cluster
create`.

Example workflow:

```
osctl config generate my-cluster https://10.5.0.2:6443 -o ./my-cluster

** tweaky tweak **

osctl cluster create --name my-cluster --input-dir "$PWD/my-cluster"
```

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2020-01-08 14:11:56 -05:00
Andrey Smirnov
ebd40bd0eb chore: use osctl cluster --wait in basic-integration
There are few workarounds for Drone way of running integration test:
DinD runs as a separate pod, and we can only access its exposed on the
"host" ports, while from Talos cluster this endpoint is not reachable.

So internally Talos nodes still use addresses like "10.5.0.2", while
test is using "docker" to access it (that's name of the `docker` service
in the pipeline).

When running locally, 127.0.0.1 is used as endpoint, which should work
fine both on OS X and Linux.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-30 15:15:42 -08:00
Andrey Smirnov
0081ac5fac refactor: extract Talos cluster provisioner as common code
This extracts Docker Talos cluster provisioner as common code
which might be shared between `osctl cluster` and integration-test.

There should be almost no functional changes.

As proof of concept, abstract cluster readiness checks were implemented
based on provisioned cluster state. It implements same checks as
`basic-integration.sh` in pure Go via Talos/K8s clients.

`conditions` package was promoted from machined-internal to
`internal/pkg` as it is used to run the checks.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-27 12:14:19 -08:00
Andrew Rynhard
5a7eb631b2 feat: add installer command to installer container
This replaces the entrypoint.sh shell script with a go binary.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-26 06:41:25 -08:00
Andrew Rynhard
e4a1bc3cf9 chore: add help menu to the Makefile
This adds a help  menu to the Makefile. It documents all build
dependencies, and how to get started.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-25 11:11:41 -08:00
Andrey Smirnov
26f222e6db refactor: osctl code cleanup, no functional changes
Fixes #1666

1. Remove custom validation of Args, use cobra-provided validators.
2. Always use errors to stop the execution flow, don't rely on
`log.Fatal` and `panic` for normal flows. This makes sure `defer` always
has a chance to run, connection is shut down in a clean way.
3. Command `docs` is hidden, as it's not for users.
4. Global variable `globalCtx` is removed, `WithClient` is used to pass
context to the command.
5. `setupClientE` renamed to `WithClient`, `setupClient` removed.
6. Code from `cmd/root.go` moved to `pkg/helpers` when possible.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-20 00:32:26 +03:00
Andrey Smirnov
c24ce2fd5f feat: humanize timestamp and size in osctl list output
Fixes #1565

Examples:

```
$ osctl list -l
MODE          SIZE(B)   LASTMOD           NAME
drwxr-xr-x    4096      Dec 17 16:37:19   .
-rwxr-xr-x    0         Dec 17 16:37:19   .dockerenv
drwxr-xr-x    4096      Dec 17 16:35:20   bin
drwxr-xr-x    4096      Dec 17 16:37:20   boot
drwxr-xr-x    5480      Dec 17 16:37:19   dev
drwxr-xr-x    4096      Dec 17 16:37:19   etc
drwxr-xr-x    4096      Dec 17 16:35:19   lib
drwxr-xr-x    4096      Dec 17 16:35:21   mnt
drwxr-xr-x    4096      Dec 17 16:39:17   opt
dr-xr-xr-x    0         Dec 17 16:37:19   proc
drwxr-x---    4096      Dec  5 06:39:44   root
drwxr-xr-x    4096      Dec 17 16:39:06   run
drwxr-xr-x    4096      Dec 17 16:35:20   sbin
dr-xr-xr-x    0         Dec 17 16:37:19   sys
dtrwxrwxrwx   4096      Dec 17 16:38:05   tmp
drwxr-xr-x    4096      Dec 17 16:35:21   usr
drwxr-xr-x    4096      Dec 17 16:37:19   var
```

```
$ osctl list -lH
MODE          SIZE(B)   LASTMOD          NAME
drwxr-xr-x    4.1 kB    18 minutes ago   .
-rwxr-xr-x    0 B       18 minutes ago   .dockerenv
drwxr-xr-x    4.1 kB    20 minutes ago   bin
drwxr-xr-x    4.1 kB    18 minutes ago   boot
drwxr-xr-x    5.5 kB    18 minutes ago   dev
drwxr-xr-x    4.1 kB    18 minutes ago   etc
drwxr-xr-x    4.1 kB    20 minutes ago   lib
drwxr-xr-x    4.1 kB    20 minutes ago   mnt
drwxr-xr-x    4.1 kB    16 minutes ago   opt
dr-xr-xr-x    0 B       18 minutes ago   proc
drwxr-x---    4.1 kB    1 week ago       root
drwxr-xr-x    4.1 kB    16 minutes ago   run
drwxr-xr-x    4.1 kB    20 minutes ago   sbin
dr-xr-xr-x    0 B       18 minutes ago   sys
dtrwxrwxrwx   4.1 kB    17 minutes ago   tmp
drwxr-xr-x    4.1 kB    20 minutes ago   usr
drwxr-xr-x    4.1 kB    18 minutes ago   var
```

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-17 23:21:28 +03:00
Andrey Smirnov
4c18f21088 feat: osctl bash/zsh completion support
Fixes #1500

Most of the help lines borrowed from kubectl. Basic completion seems to
work, but the `osctl` binary should be named `osctl` (not
`osctl-linux-amd64` or such).

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-17 16:27:51 +03:00
Andrew Rynhard
fa4fb4d444 feat: add create and overwrite file operations
This adds `create` and `overwrite` file operations so that we can have
better sanity checks around extra files.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-11 11:49:27 -08:00
Andrew Rynhard
ad863a7f92 refactor: rename protobuf services, RPCs, and messages
This PR brings our protobuf files into conformance with the protobuf
style guide, and community conventions. It is purely renames, along with
generated docs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-11 11:41:40 -08:00
Andrew Rynhard
fe61a3591f docs: update osctl kubeconfig references
The usage of `osctl kubeconfig` has changed. This updates the docs to
reflect the change.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-10 19:08:31 -08:00
Brad Beam
9584b47cd7 feat: Upgrade kubernetes to 1.17.0
Primarily doc/constant changes.

Added additionnal bits to `docs` target in makefile to generate osctl
docs as well as config files. Explicitly define a HOME variable so we
get consistent home directories for talosconfig variables in our docs.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-12-10 16:03:35 -08:00
Andrey Smirnov
399aeda0b9 feat: rename confusing target options, --endpoints, etc.
Fixes #1610

1. In `talosconfig`, deprecate `Target` in favor of `Endpoints`
(client-side LB to come next).

2. In `osctl`, use `--nodes` in place of `--target`.

3. In `osctl` add option `--endpoints` to override `Endpoints` for the
call.

Other changes are just updates to catch up with the changes. Most
probably I missed something... And CAPI provider needs update.

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-10 02:23:54 +03:00
Tim Gerla
9a2fd989c9 fix: improve the project site meta description
This change should slightly improve the search engine placement of our
docs/project site by being a bit more descriptive.

Signed-off-by: Tim Gerla <tim@gerla.net>
2019-12-09 12:24:23 -08:00
Alex Szakaly
9efcf06dca docs: simplify corporate proxy
It is possible to append to file since 84354c59414,
hence no need to create custom Docker image.

Signed-off-by: Alex Szakaly <alex.szakaly@gmail.com>
2019-12-09 11:18:21 -08:00
Tim Gerla
343cba04d3 fix: update node dependencies for project website
Update node dependencies on the project website to fix a
GitHub-flagged vulnerability in serialize-javascript.

Signed-off-by: Tim Gerla <tim@gerla.net>
2019-12-09 10:15:19 -08:00
Andrey Smirnov
4f22581464 docs: update generated osctl documentation
`--context`, `osctl logs -f`, `osctl read`

Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
2019-12-06 14:13:41 -08:00
Spencer Smith
c3deb3e439 docs: update with new cni abilities
This PR updates the docs for cni

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-06 15:37:53 -05:00
Spencer Smith
7c1d950686 docs: clarify vmware instructions
This PR clarifies a few nits about the vmware docs. We were referring to
alpha.2 in lots of places. I moved this to a note that just says to set
TALOS_VERSION to `v0.3.0-alpha.10` or similar. Also clarifies the path
to the ova could be any /path/to/downloaded/file.

Should close #1572 when we've gotten some extra info about the hardware
versions.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-12-03 16:40:50 -05:00
Tim Gerla
4debea6856 feat: allow deep-linking to specific docs pages
This changes the way the sidebar anchors are constructed, and if
there is an anchor in the URL, the Content component will render
the requested page, instead of just the top-level docs page.

Signed-off-by: Tim Gerla <tim@gerla.net>
2019-12-03 07:31:48 -08:00
Andrew Rynhard
30e40f6d18 docs: add automated upgrades proposal
This is a proposal for the new way of running Talos and performing upgrades in v0.2.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-12-02 07:07:55 -08:00
Tim Gerla
83d9e01217 feat: add Google Analytics tracking to the project website
- add the Google Analytics tracking package and tracking ID.

Signed-off-by: Tim Gerla <tim@gerla.net>
2019-11-27 10:56:15 -08:00
Brad Beam
119bf3e7bb feat(networkd): Add support for bonding
This includes a healthy refactor of the networkd code as well.
- Move netlink functionality to nic package
- Networkd facilitates the orchestration of the underlying interface configuration
- Networkd now stores the state of each interface configuration. This
  should allow us to expose this information via api in the future.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-26 20:08:31 -08:00
Andrew Rynhard
43ad18fbee feat: allow sysctl writes
This adds the ability to write sysctls on boot by specifying them in the
machine config.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-25 10:46:28 -08:00
Andrew Rynhard
dc19cdab9d docs: add matchbox getting started guide
This adds a basic guide on using Talos with Matchbox.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-23 07:20:49 -08:00
Brad Beam
91ac62389d docs: Add examples to networkd
Add simple examples around static addressing and assigning multiple addresses
to a single interface.

Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
2019-11-14 22:11:40 -08:00
Spencer Smith
cd3827f9de docs: update gcp docs
This PR will uplift the GCP documentation to match our current way to
deploy talos via CLI.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-14 10:03:31 -05:00
Andrew Rynhard
e1fc9017d2 feat: allow extra arguments to be passed to etcd
This allows for an arbitrary set of args to be passed to etcd. Due to
the fact the we do the discovery of the current cluster state, we
blacklist initial-cluster. Additionally, we black list args that would
makes sense to modify, like the path to the certs.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-14 05:53:00 -08:00
Spencer Smith
f0c8feb56c docs: Update azure doc
This PR updates the azure doc to match the newer processes of deploying
talos.

Signed-off-by: Spencer Smith <robertspencersmith@gmail.com>
2019-11-13 19:20:16 -08:00
Andrew Rynhard
127fa54a76 docs: add docs command to osctl
This allows osctl to generate documentation for itself.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-13 17:14:53 -08:00
Andrew Rynhard
714ae3e7c1 docs: add a project dropdown
The dropdown is a better place to group information about the project that
isn't documentation. This places the roadmap and use cases under the project
dropdown.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 21:48:34 -08:00
Andrew Rynhard
6c55234793 docs: remove stale docs
The KVM, Xen, and Digital Rebar docs are either not accurate or just a
blank page. We should remove them and add them back when we are ready
to.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 21:30:10 -08:00
Andrew Rynhard
5f2da192f2 docs: fix proxy Dockerfile example
Small fixes to the Dockerfile example for proxy setups.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 20:49:25 -08:00
Andrew Rynhard
ded96fa0fb docs: disable PurgeCSS
PurgeCSS seems to be removing portions of the styles in
docs/website/assets/css. This disables it until we can figure out a
better solution.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 20:47:22 -08:00
Andrew Rynhard
eed169e457 chore: format docs to one sentence per line
Small nit fix.

Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
2019-11-11 15:23:35 -08:00