This alert will never return anything as the left side of the query has the labels `[component, environment, instance, job, type]` while the right side has `[component, environment, instance, job]`. The `type` label was added to `prometheus_tsdb_head_samples_appended_total` in this PR but the mixin wasn't updated for the new label: https://github.com/prometheus/prometheus/pull/11395 This was found with [pint](https://github.com/cloudflare/pint) PromQL linting Signed-off-by: Will Bollock <wbollock@linode.com>
Prometheus Mixin
This is work in progress. We aim for it to become a good role model for alerts and dashboards eventually, but it is not quite there yet.
The Prometheus Mixin is a set of configurable, reusable, and extensible alerts and dashboards for Prometheus.
To use them, you need to have jsonnet (v0.13+) and jb installed. If you
have a working Go development environment, it's easiest to run the following:
$ go install github.com/google/go-jsonnet/cmd/jsonnet@latest
$ go install github.com/google/go-jsonnet/cmd/jsonnetfmt@latest
$ go install github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@latest
Note: The make targets lint and fmt need the jsonnetfmt binary, which is
available from v.0.16.0 in the Go implementation of jsonnet. If your jsonnet version is older than 0.16.0 you have to either upgrade or install the C++ version of
jsonnetfmt if you want to use make lint
or make fmt.
Next, install the dependencies by running the following command in this directory:
$ jb install
You can then build a prometheus_alerts.yaml with the alerts and a directory
dashboards_out with the Grafana dashboard JSON files:
$ make prometheus_alerts.yaml
$ make dashboards_out
For more advanced uses of mixins, see https://github.com/monitoring-mixins/docs.