Table size queries (`omni_sqlite_subsystem_size_bytes`) filtered by dbstat name, missing index sizes. Join with sqlite_master to attribute index pages to their parent table.
DB size (`omni_sqlite_db_size_bytes`) used dbstat sum which excludes freelist pages. Use page_count * page_size to match actual file size.
Add `omni_sqlite_db_freelist_size_bytes` metric to track wasted space.
Signed-off-by: Oguz Kilcan <oguz.kilcan@siderolabs.com>
(cherry picked from commit e85ab384c343ca26987ce65ca1f9f4a9ee7d2361)
Introduce token-bucket based bandwidth rate limiting for the SideroLink WireGuard tunnel, configurable via services.siderolink.bandwidthLimitMbps and services.siderolink.bandwidthLimitBurstBytes config fields (with corresponding CLI flag fallbacks).
Rate limiting is applied in both directions: outbound via a wrapped conn.Bind and inbound via a TUN input packet filter. A shared limiter drops packets exceeding the budget, relying on TCP congestion control to throttle senders. Disabled by default (0 = unlimited).
Also adds a Grafana service to docker-compose with pre-built Omni dashboards for local development observability.
Signed-off-by: Oguz Kilcan <oguz.kilcan@siderolabs.com>
Replace the audit log download button with a modal which includes a date range for filtering to keep file sizes down. Includes a warning if trying to download logs for longer than 7 days. Modal includes a bytes downloaded indicator to convey progres. Download is canceled if modal is closed.
Signed-off-by: Edward Sammut Alessi <edward.sammutalessi@siderolabs.com>
Remove the flags for turning on SQLite storage for:
- Discovery service state
- Audit logs
- Machine logs
Instead, migrate them unconditionally to SQLite on the next startup.
Remove many flags which are no longer meaningful. Only keep the ones which are required for the migrations.
Additionally: Make the `--sqlite-storage-path` (or its config counterpart `.storage.sqlite.path`) required with no default value, as a default value does not make sense for it in most of the cases.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Update the node version used by the frontend to the latest LTS version 24.11.0
Signed-off-by: Edward Sammut Alessi <edward.sammutalessi@siderolabs.com>
default / e2e-backups (push) Blocked by required conditions
default / e2e-cluster-import (push) Blocked by required conditions
default / e2e-forced-removal (push) Blocked by required conditions
default / e2e-omni-upgrade (push) Blocked by required conditions
default / e2e-scaling (push) Blocked by required conditions
default / e2e-short (push) Blocked by required conditions
default / e2e-short-secureboot (push) Blocked by required conditions
default / e2e-templates (push) Blocked by required conditions
default / e2e-upgrades (push) Blocked by required conditions
default / e2e-workload-proxy (push) Blocked by required conditions
Switch from bun to node. Also replaces bun test with vitest. Updated apexcharts to satisfy peerDependencies and removed unused dependencies. Fixed type issues that were being silently missed in the previous setup.
Fixes#1398
Signed-off-by: Edward Sammut Alessi <edward.sammutalessi@siderolabs.com>
default / e2e-backups (push) Blocked by required conditions
default / e2e-cluster-import (push) Blocked by required conditions
default / e2e-forced-removal (push) Blocked by required conditions
default / e2e-omni-upgrade (push) Blocked by required conditions
default / e2e-scaling (push) Blocked by required conditions
default / e2e-short (push) Blocked by required conditions
default / e2e-short-secureboot (push) Blocked by required conditions
default / e2e-templates (push) Blocked by required conditions
default / e2e-upgrades (push) Blocked by required conditions
default / e2e-workload-proxy (push) Blocked by required conditions
The inspector will start on `0.0.0.0:12000` if the compose is started
`WITH_DEBUG=true`.
Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
default / e2e-backups (push) Blocked by required conditions
default / e2e-forced-removal (push) Blocked by required conditions
default / e2e-scaling (push) Blocked by required conditions
default / e2e-short (push) Blocked by required conditions
default / e2e-short-secureboot (push) Blocked by required conditions
default / e2e-templates (push) Blocked by required conditions
default / e2e-upgrades (push) Blocked by required conditions
default / e2e-workload-proxy (push) Blocked by required conditions
User-owned config patches are only deleted when the resource they are assigned to (cluster/machine set/cluster machine/machine) is deleted.
This causes some dangling/orphan config patches to accumulate over time - namely the patches with no matching owner.
Implement a controller to clean them up after some period of time.
Signed-off-by: Utku Ozdemir <utku.ozdemir@siderolabs.com>
This PR implements audit logs. To enable it you have to set the `--audit-log-dir` flag
to a directory where the audit logs will be stored. The audit logs are stored in a JSON format.
Example:
```json
{"event_type":"update","resource_type":"PublicKeys.omni.sidero.dev","event_ts":1722537710182,"event_data":{"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36","ip_address":"<snip>","user_id":"a19a7a38-1793-4262-a9ef-97bc00c7a155","role":"Admin","email":"useremail@userdomain.com","confirmation_type":"auth0","fingerprint":"15acb974f769bdccd38a4b28f282b78736b80bc7","public_key_expiration":1722565909}}
```
Keep in mind that `event_ts` are in milliseconds instead of seconds.
Field `event_data` contains all relevant information about the event.
To enabled it in the development environment you will have to add the
`--audit-log-dir /tmp/omni-data/audit-logs` line to `docker-compose.override.yml`
or run `generate-certs` again.
For #37
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Bump vault container version in the compose script.
Bump auth0 and uuid lib versions in the `package.json`.
Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
Dropped some dependencies which no longer worked:
- monaco-editor-vue3 - replaced with a tiny wrapper.
- polyfill for streaming classes - replaced with vite plugin for
polyfills.
- buffer - replaced base64 encoding with calls from the `fetch.pb.ts`.
Dropped the code that sets up monaco in `main.ts` in favor of Vite
monaco plugin that does that for you.
Updated `monaco-yaml` as the old version didn't work with Vite.
Switched from `Jest` to `bun:test`.
Use TypeScript in the tailwind config, as it imports `colors.ts` now.
Had to use TypeScript for `colors` as Vite can't import CommonJS
modules.
Add eslint linter to the `lint` dependencies, enable a bit more linters
and fix all detected errors.
Fixes: https://github.com/siderolabs/omni/issues/294
Signed-off-by: Artem Chernyshev <artem.chernyshev@talos-systems.com>
This PR adds the support for WG over GRPC. New field `VirtualAddrport`
in `SiderolinkSpec` should allow for both
setting the virtual addr and loading it after the Omni restart.
Signed-off-by: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Omni is source-available under BUSL.
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
Co-Authored-By: Artem Chernyshev <artem.chernyshev@talos-systems.com>
Co-Authored-By: Utku Ozdemir <utku.ozdemir@siderolabs.com>
Co-Authored-By: Dmitriy Matrenichev <dmitry.matrenichev@siderolabs.com>
Co-Authored-By: Philipp Sauter <philipp.sauter@siderolabs.com>
Co-Authored-By: Noel Georgi <git@frezbo.dev>
Co-Authored-By: evgeniybryzh <evgeniybryzh@gmail.com>
Co-Authored-By: Tim Jones <tim.jones@siderolabs.com>
Co-Authored-By: Andrew Rynhard <andrew@rynhard.io>
Co-Authored-By: Spencer Smith <spencer.smith@talos-systems.com>
Co-Authored-By: Christian Rolland <christian.rolland@siderolabs.com>
Co-Authored-By: Gerard de Leeuw <gdeleeuw@leeuwit.nl>
Co-Authored-By: Steve Francis <67986293+steverfrancis@users.noreply.github.com>
Co-Authored-By: Volodymyr Mazurets <volodymyrmazureets@gmail.com>
