Delete files older than 30 days. Also bump minimal Go version to 1.23 and fix lint issues.
Closes#546
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
This commit implements session tracking and log audit for those types:
- [x] auth.PublicKey
- [x] auth.AccessPolicy
- [x] auth.User
- [x] auth.Identity
- [x] omni.Machine
- [x] omni.MachineLabels
- [x] omni.Cluster
- [x] omni.MachineSet (only empty owners for update, log create and delete in all cases)
- [x] omni.MachineSetNode (only empty owners for update, log create and delete in all cases)
- [x] omni.ConfigPatch
- [x] Talos API Access
- [x] Kubernetes API access
Output example:
```
{"event_type":"update","resource_type":"Machines.omni.sidero.dev","event_ts":1723137771180,"event_data":{"session":{"user_agent":"Omni-Internal-Agent"},"machine":{"id":"18cec051-d975-483d-8d43-10ac6421648a","is_connected":true,"management_address":"fdae:41e4:649b:9303:da9b:1ed:a725:c3dd","labels":{"omni.sidero.dev/address":"fdae:41e4:649b:9303:da9b:1ed:a725:c3dd"}}}}
{"event_type":"update","resource_type":"Machines.omni.sidero.dev","event_ts":1723137771180,"event_data":{"session":{"user_agent":"Omni-Internal-Agent"},"machine":{"id":"18cec051-d975-483d-8d43-10ac6421648a","is_connected":true,"management_address":"fdae:41e4:649b:9303:da9b:1ed:a725:c3dd","labels":{"omni.sidero.dev/address":"fdae:41e4:649b:9303:da9b:1ed:a725:c3dd"}}}}
{"event_type":"update","resource_type":"Machines.omni.sidero.dev","event_ts":1723137771181,"event_data":{"session":{"user_agent":"Omni-Internal-Agent"},"machine":{"id":"18cec051-d975-483d-8d43-10ac6421648a","is_connected":true,"management_address":"fdae:41e4:649b:9303:da9b:1ed:a725:c3dd","labels":{"omni.sidero.dev/address":"fdae:41e4:649b:9303:da9b:1ed:a725:c3dd"}}}}
{"event_type":"create","resource_type":"MachineLabels.omni.sidero.dev","event_ts":1723137787549,"event_data":{"session":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"ea002172-b9da-423f-bd1d-b443b8a7b43c","role":"Admin","email":"dmitry.matrenichev@siderolabs.com","fingerprint":"da7b997eb68449a12bebc6a3bf4f59beaf167209"},"machine_labels":{"id":"18cec051-d975-483d-8d43-10ac6421648a","labels":{"222":""}}}}
{"event_type":"update","resource_type":"MachineLabels.omni.sidero.dev","event_ts":1723137787553,"event_data":{"session":{"user_agent":"Omni-Internal-Agent"},"machine_labels":{"id":"18cec051-d975-483d-8d43-10ac6421648a","labels":{"222":""}}}}
{"event_type":"update","resource_type":"MachineLabels.omni.sidero.dev","event_ts":1723137811532,"event_data":{"session":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"ea002172-b9da-423f-bd1d-b443b8a7b43c","role":"Admin","email":"dmitry.matrenichev@siderolabs.com","fingerprint":"da7b997eb68449a12bebc6a3bf4f59beaf167209"},"machine_labels":{"id":"18cec051-d975-483d-8d43-10ac6421648a","labels":{"222":"","333":""}}}}
{"event_type":"update","resource_type":"MachineLabels.omni.sidero.dev","event_ts":1723137811610,"event_data":{"session":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"ea002172-b9da-423f-bd1d-b443b8a7b43c","role":"Admin","email":"dmitry.matrenichev@siderolabs.com","fingerprint":"da7b997eb68449a12bebc6a3bf4f59beaf167209"},"machine_labels":{"id":"18cec051-d975-483d-8d43-10ac6421648a","labels":{"222":"","333":""}}}}
{"event_type":"update","resource_type":"MachineLabels.omni.sidero.dev","event_ts":1723137811611,"event_data":{"session":{"user_agent":"Omni-Internal-Agent"},"machine_labels":{"id":"18cec051-d975-483d-8d43-10ac6421648a","labels":{"222":"","333":""}}}}
{"event_type":"destroy","resource_type":"MachineLabels.omni.sidero.dev","event_ts":1723137811621,"event_data":{"session":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"ea002172-b9da-423f-bd1d-b443b8a7b43c","role":"Admin","email":"dmitry.matrenichev@siderolabs.com","fingerprint":"da7b997eb68449a12bebc6a3bf4f59beaf167209"},"machine_labels":{"id":"18cec051-d975-483d-8d43-10ac6421648a","labels":{"222":"","333":""}}}}
{"event_type":"create","resource_type":"Users.omni.sidero.dev","event_ts":1723141793888,"event_data":{"new_user":{"role":"Admin","id":"7903a72c-87af-43b8-94dc-82bd961ab768"},"session":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"ea002172-b9da-423f-bd1d-b443b8a7b43c","role":"Admin","email":"dmitry.matrenichev@siderolabs.com","fingerprint":"da7b997eb68449a12bebc6a3bf4f59beaf167209"}}}
{"event_type":"create","resource_type":"Identities.omni.sidero.dev","event_ts":1723141793981,"event_data":{"new_user":{"id":"7903a72c-87af-43b8-94dc-82bd961ab768","email":"some-user-email@email.com"},"session":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"ea002172-b9da-423f-bd1d-b443b8a7b43c","role":"Admin","email":"dmitry.matrenichev@siderolabs.com","fingerprint":"da7b997eb68449a12bebc6a3bf4f59beaf167209"}}}
```
Closes#37
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
This PR implements audit logs. To enable it you have to set the `--audit-log-dir` flag
to a directory where the audit logs will be stored. The audit logs are stored in a JSON format.
Example:
```json
{"event_type":"update","resource_type":"PublicKeys.omni.sidero.dev","event_ts":1722537710182,"event_data":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"a19a7a38-1793-4262-a9ef-97bc00c7a155","role":"Admin","email":"useremail@userdomain.com","confirmation_type":"auth0","fingerprint":"15acb974f769bdccd38a4b28f282b78736b80bc7","public_key_expiration":1722565909}}
```
Keep in mind that `event_ts` are in milliseconds instead of seconds.
Field `event_data` contains all relevant information about the event.
To enabled it in the development environment you will have to add the
`--audit-log-dir /tmp/omni-data/audit-logs` line to `docker-compose.override.yml`
or run `generate-certs` again.
For #37
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Improve the exposed service reliability by using a TCP loadbalancer between the nodes exposing the service.
Rework the exposed service proxy registry to be a COSI controller instead to simplify the logic, improve reliability and testability.
Closessiderolabs/omni#396.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Convert goroutine panics to errors or error logs.
Disallow usage of `golang.org/x/sync/errgroup` package in the backend by `depguard` linter. This linter configuration depends on: https://github.com/siderolabs/kres/pull/417
Rekres the project to include the feature (also bump Go to 1.22.4), but revert `PROTOBUF_GO_VERSION` and `GRPC_GATEWAY_VERSION` manually to not break the frontend.
Disallowing the named `go` statement was not possible at the moment using existing linters, raised an issue in `forbidigo` for it: https://github.com/ashanbrown/forbidigo/issues/47Closessiderolabs/omni#373.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Rework the workload service proxying feature to support the new domain format in addition to the existing one.
Old format:
```
p-g3a4ana-demo.omni.siderolabs.io
```
New format:
```
g3a4ana-demo.proxy-us.omni.siderolabs.io
```
The old format required a new DNS records to be added for each new workload service, causing issues with the resolution on clients. The new format addresses it by leveraging wildcard records.
Additionally, build the full exposed service URL on the backend and make it a field on `ExposedService` resource, so they can be accessed using `omnictl get exposedservice`.
Part of siderolabs/omni#17.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
With new gRPC (both gateway and modules) it uses `grpc.NewClient` call to create clients.
It no longer support custom addresses without a `passthrough:` prefix. Previous fix didn't
account for that in some places, so this one changes the structure of `Transport` to always
return address in proper form for external users.
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Run a discovery service instance inside Omni (enabled by default).
It listens only on the SideroLink interface on port 8093.
Clusters can opt in to use this embedded discovery service instead of the `discovery.talos.dev`. It is added as a new cluster feature both on frontend and in cluster templates.
Closessiderolabs/omni#20.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Make the controller run tasks that can collect machine status from each
machine.
Instead of changing the `MachineStatusSnapshot` directly in the
siderolink events handler pass these events to the controller through
the channel, so that all events are handled in the same place.
If either event comes from siderolink or if task runner gets the machine
status it updates the `MachineStatusSnapshot` resource.
Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
Use the Talos resource API as well as the siderolink event sink to determine the status of a machine.
Follow the agreed decision tree of:
- if the update came over the same channel as before, use it
- if the update came over a different channel than before, and the timestamp is newer than the previous update, use it
- otherwise, drop it
Closessiderolabs/omni#41.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
When determining the schematic ID of a machine, instead of relying the ID on the schematic ID meta-extension, compute the ID by gathering the extensions on the machine. This way, the extension ID will not contain the META values, labels or the kernel args.
This ID is actually the ID we need, as when we compare the desired schematic with the actual one during a Talos upgrade, we are only interested in the changes in the list of extensions.
This does not cause the kernel args, labels, etc. to disappear, as they are used at installation time and preserved afterward (e.g., during upgrades).
Additionally:
- Remove the list of extensions from the `Schematic` resource, as it relied upon the schematics always being created through Omni. This is not always the case - i.e., when a partial join config is used. Therefore, instead of relying on it, we store the list of extensions by directly reading them from the machine and storing them on the `MachineStatus` resource.
- Skip setting the schematic META section at all if there are no labels set on Download Installation Media screen.
Closessiderolabs/omni#55.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Works the same way as `talosctl support` but also grabs some relevant
Omni resources to help with the diagnostics.
Uses `go-talos-support` common module to collect Talos data.
Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
Omni is source-available under BUSL.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Co-Authored-By: Artem Chernyshev <artem.chernyshev@talos-systems.com>
Co-Authored-By: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Co-Authored-By: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Co-Authored-By: Philipp Sauter <philipp.sauter@siderolabs.com>
Co-Authored-By: Noel Georgi <git@frezbo.dev>
Co-Authored-By: evgeniybryzh <evgeniybryzh@gmail.com>
Co-Authored-By: Tim Jones <tim.jones@siderolabs.com>
Co-Authored-By: Andrew Rynhard <andrew@rynhard.io>
Co-Authored-By: Spencer Smith <spencer.smith@talos-systems.com>
Co-Authored-By: Christian Rolland <christian.rolland@siderolabs.com>
Co-Authored-By: Gerard de Leeuw <gdeleeuw@leeuwit.nl>
Co-Authored-By: Steve Francis <67986293+steverfrancis@users.noreply.github.com>
Co-Authored-By: Volodymyr Mazurets <volodymyrmazureets@gmail.com>
