This PR only touches `Version` method, but I will expand it to other
methods in the next PR.
When proxying to many upstreams, errors are wrapped as responses as we
can't return error and response from grpc call. Reflect-based function
was introduced to filter out responses which contain errors as
multierror. Reflection was used, as each response is a different Go
type, and we can't write a generic function for it.
osctl was updated to support having both resp & err not nil. One failed
response shouldn't result in error.
Re-enabled integration test for multiple targets and version
consistency, need e2e validation.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This change is pretty mechanical, just wrap every API so that remote
peer address is used as default for `resp.Metadata.Hostname`.
This makes `NODE:` non-empty in all the API calls.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This replaces codegen version of apid proxying with
talos-systems/grpc-proxy based version. Proxying is transparent, it
doesn't require exact information about methods and response types. It
requires some common layout response to enhance it properly with node
metadata or errors.
There should be no signifcant changes to the API with the previous
version, but it's worth mentioning a few changes:
1. grpc.ClientConn is established just once per upstream (either local
service or remote apid instance).
2. When called without `-t` (`targets`), apid proxies immediately down
to local service skipping proxying to itself (as before), which results
in empty node metadata in response (before it had local node IP). Might
revert this later to proxy to itself (?).
3. Streaming APIs are now fully supported with multiple targets, but
message definition doesn't contain `ResponseMetadata`, so streaming APIs
are broken now with targets (needs a fix).
4. Errors are now returned as responses with `Error` field set in
`ResponseMetadata`, this requires client library update and `osctl` to
handle it properly.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This moves the Kubeconfig api endpoint to machined and consolidates the
"read a file" code into machined. This also changes Kubeconfig to
use the CopyOut method which changes Kubeconfig to a streaming grpc call.
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
- Added common.proto to host NodeMetadata
- go_package names were fixed up so imports are generated with the proper
package names
- fixed up build work (dockerfile) to prevent copying the previously
generated go proto files. This fixes a bug where we could incorrectly
copy the previously generated protobuf instead of a new one generated
at an incorrect location/name/etc.
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
This enables the ability to specify additional <talos> endpoints to connect to
to pull back data.
Signed-off-by: Brad Beam <brad.beam@talos-systems.com>
This removes the github.com/pkg/errors package in favor of the official
error wrapping in go 1.13.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
Memory usage reduced around 8-10x: now it stays stable at 1GB.
I disabled some of the new linters, and one rule which is violated a
lot.
I might make sense to go back and enable `wsl` fixing all the issues
(leaving that for another PR).
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This detangles the gRPC client code from the userdata code. The
motivation behind this is to make creating clients more simple and not
dependent on our configuration format.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This makes working with the API much cleaner as a client. Using gob
doesn't give the client a well-known type to work with in the API
definition.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
In order for other projects to make use of our APIs, they must not
reside underneath the internal directory. This moves the protobuf
definitions to a top-level "api" directory and scopes them according to
their domain. This change also removes generated code from the gitignore
file so that users don't have to generate the code themseleves.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This implements 'default deny' policy for service operations via the
API: services do not allow operations.
Service whitelists itself for stop/start/restart by implementing the
interface and returning boolean flag which might depend on userdata.
Machined APIs `Stop/Start` were renamed to `ServiceStop`/`ServiceStart`
to avoid confusion with osd API `Restart` which is not related to
services. Old APIs are deprecated and compatibility code forwards old
APIs to the new code.
`ServiceRestart` API was introduced to distinguish restart action from
stop/start (previously restart was implemented as stop+start in the
CLI).
Service udevd-trigger was whitelisted for all operations (allows
stopping hanging run, restarting to trigger once again).
Services proxyd & ntpd were whitelisted for restart and start (start is
whitelisted to help with service stuck in stopped state while restarting).
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
It is now possible to `start`/`stop`/`restart` any service via `osctl`
commands.
There are some changes in `ServiceRunner` to support re-use (re-entering
running state). `Services` singleton now tracks service running state to
avoid calling `Start()` on already running `ServiceRunner` instance.
Method `Start()` was renamed to `LoadAndStart()` to break up service
loading (adding to the list of service) and actual service start.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This change allows for more accurate mount reporting as /proc/mounts is
a symlink to /proc/self/mounts and contains mounts that are relative to
the running process. In our case this was osd. This caused inaccurate
reporting of mounts since they were relative to osd when we really
wanted mounts relative to machined.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
This PR moves the reset API to the init API definition.
It leverages the same code we use for upgrades.
Signed-off-by: Andrew Rynhard <andrew@andrewrynhard.com>
Service `osd` doesn't have access to rootfs, as it is running in a
container, so move API to `init` which has unconstrained access to
rootfs. (This is in line with another API, `osctl cp`).
Fixes: #752
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
Actual API is implemented in the `init`, as it has access to root
filesystem. `osd` proxies API back to `init` with some tricks to support
grpc streaming.
Given some absolute path, `init` produces and streams back .tar.gz
archive with filesystem contents.
`osctl cp` works in two modes. First mode streams data to stdout, so
that we can do e.g.: `osctl cp /etc - | tar tz`. Second mode extracts
archive to specified location, dropping ownership info and adjusting
permissions a bit. Timestamps are not preserved.
If full dump with owner/permisisons is required, it's better to stream
data to `tar xz`, for quick and dirty look into filesystem contents
under unprivileged user it's easier to use in-place extraction.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
I couldn't find any use for the `timeout` flag nor the value passed in
the API, but it block much more useful and present in other commands
flag 'target'.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This moves cli code (rendering output, etc.) out of 'client' package, so
that client package is usable outside of cli.
Consistently accept context as first param to API methods, so that we
can build graceful request cancellation.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This returns list of all the services registered, with their current
status, past events, health state, etc.
New CLI is `osctl service [<id>]`: without `<id>` it prints list of all
the services, with specific `<id>` it provides details for a service.
I decided to create "parallel" data structures in protobuf as Go
structures don't map nicely onto what protoc generates: pointers vs.
values, additional fields like mutexes, etc. Probably there's a better
approach, I'm open for it.
For CLI, I tried to keep CLI stuff in `cmd/` package, and I also created
simple wrapper to remove duplicated code which sets up client for each
command.
Examples:
```
$ osctl service
SERVICE STATE HEALTH LAST CHANGE LAST EVENT
containerd Running OK 21s ago Health check successful
kubeadm Running ? 2s ago Started task kubeadm (PID 280) for container kubeadm
kubelet Running ? 0s ago Started task kubelet (PID 383) for container kubelet
ntpd Running ? 14s ago Started task ntpd (PID 129) for container ntpd
osd Running ? 14s ago Started task osd (PID 126) for container osd
proxyd Waiting ? 14s ago Waiting for conditions
trustd Running ? 14s ago Started task trustd (PID 125) for container trustd
udevd Running ? 14s ago Started task udevd (PID 130) for container udevd
```
```
$ osctl service proxyd
ID proxyd
STATE Running
HEALTH ?
EVENTS [Preparing]: Running pre state (22s ago)
[Waiting]: Waiting for conditions (22s ago)
[Preparing]: Creating service runner (6s ago)
[Running]: Started task proxyd (PID 461) for container proxyd (6s ago)
```
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
This implements insecure over-file-socket gRPC API for init with two
first simplest APIs: reboot and shutdown (poweroff).
File socket is mounted only to `osd` service, so it is the only service
which can access init API. Osd forwards reboot/shutdown already
implemented APIs to init which actually executes these.
This enables graceful shutdown/reboot with service shutdown, sync, etc.
Signed-off-by: Andrey Smirnov <smirnov.andrey@gmail.com>
