From 9f0b52d73a683fd10d11f554ab1d68dc30b18b2c Mon Sep 17 00:00:00 2001 From: Gabriel Filion Date: Mon, 7 Oct 2024 18:57:10 -0400 Subject: [PATCH] docs: Describe how time() is set to start at 0 in unit tests The return value of functions relating to the current time, e.g. time(), is set by promtool to start at timestamp 0 at the start of a test's evaluation. This has the very nice consequence that tests can run reliably without depending on when they are run. It does, however, mean that tests will give out results that can be unexpected by users. If this behaviour is documented, then users will be empowered to write tests for their rules that use time-dependent functions. (Closes: prometheus/docs#1464) Signed-off-by: Gabriel Filion --- docs/configuration/unit_testing_rules.md | 27 ++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/configuration/unit_testing_rules.md b/docs/configuration/unit_testing_rules.md index d237c8cf88..13b0445c7c 100644 --- a/docs/configuration/unit_testing_rules.md +++ b/docs/configuration/unit_testing_rules.md @@ -275,3 +275,30 @@ groups: summary: "Instance {{ $labels.instance }} down" description: "{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 5 minutes." ``` + +### Time within tests + +It should be noted that in all tests, either in `alert_test_case` or +`promql_test_case`, the output from all functions related to the current time, +for example the `time()` and `day_of_*()` functions, will output a consistent value +for tests. + +At the start of the test evaluation, `time()` returns 0 and therefore when under test +`time()` will return a value of `0 + eval_time`. + +If you need to write tests for alerts that use functions relating to the current +time, make sure that the values given to your `input_series` are placed far +enough in the past, relative to the evaluation time described above. The values +can for example be negative timestamps so that with a very small `eval_time` the +alert can be expected to trigger. + +Another method that's known to work is to instead bump `eval_time` in the future +so that the timestamp output by `time()` will be a higher value and the values +in `input_series` will be far enough apart from that point in time so that the +alerts will trigger. This method has the downside of making promtool generate a +timeseries database that contains a value for each `input_series` for each +`interval` for the given test. This can become very slow relatively easily and +can end up consuming a lot of RAM for running your test. By instead using values +for `input_series` relative to the timestamp described above even though the +values go into negative numbers, you can keep `eval_time` fairly lower and avoid +making your tests run very slowly.