1292 Commits

Author SHA1 Message Date
Julien
8f75da3fd6
Merge pull request #16754 from simonpasquier/fix-promql-panic
fix: prevent invalid array access in aggregate expression
2025-06-20 13:53:59 +02:00
Simon Pasquier
71217a6e43
fix: prevent invalid array access in aggregate expression
This commit fixes the evaluation of invalid expressions like
`sum(rate(`. Before that, it would trigger a panic in the PromQL engine
because it tried to access an index which is out of range.

The bug was probably introduced by 06d0b063ea.

Signed-off-by: Simon Pasquier <spasquie@redhat.com>
2025-06-19 15:11:38 +02:00
Kapil Lamba
69906bb4f5
Add script for converting PromQL tests to new syntax format (#16562)
Signed-off-by: Kapil Lamba <kapillamba4@gmail.com>
Co-authored-by: Neeraj Gartia <80708727+NeerajGartia21@users.noreply.github.com>
2025-06-18 23:57:39 +02:00
Charles Korn
8d9dfa075d
promql: reuse histogramStatsIterator where possible, and expose it for other implementations to use (#16686)
* Expose type
* Add `Reset` method

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2025-06-18 23:53:46 +02:00
Björn Rabenstein
f69c83f5f1
Merge pull request #16702 from bboreham/simpler-detect
[Refactor] PromQL: Simplify detectHistogramStatsDecoding
2025-06-18 13:02:55 +02:00
Michael Hoffmann
ec98e7f069 promql: fix ts_of_last_over_time for histogram vectors
This PR fixes a bug in ts_of_last_over_time where the float samples
where used when computing the last timestamp of the histogram samples.

Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
2025-06-16 13:09:49 +00:00
Julien
998840c496
Merge pull request #16713 from roidelapluie/fix-pow-dur-expr
promql: Fix unary operator precedence in duration expressions
2025-06-16 11:13:34 +02:00
Julien Pivotto
6d56e77e0e promql: Fix unary operator precedence in duration expressions
Split unary operator handling in duration expressions into two specific
cases to fix precedence conflicts:
- Handle unary operators with number literals directly
- Handle unary operators with parenthesized expressions separately

This prevents unary minus from incorrectly binding to subsequent
operators in expressions like `foo offset -1^1`, ensuring it parses
as `(foo offset -1) ^ 1` rather than `foo offset (-1^1)`.

Fixes #16711

Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
2025-06-13 13:40:50 +02:00
Michael Hoffmann
a5fa9431d8 promql: add ts_of_(max,min,last)_over_time functions
This commit adds the ts_of_(max,min,last)_over_time functions behind the experimental feature flag.

Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
2025-06-12 15:03:28 +00:00
Innokentii Konstantinov
94d5e0f41c
engine: fix PossibleNonCounterInfo annotation for rate and increase. (#16718)
* Comment exposed field

Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com>

* Remove invalid test

Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com>

* Fix PossibleNonCounterInfo annotation.

Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com>

* lint

Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com>

---------

Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com>
2025-06-11 14:09:57 +01:00
beorn7
68b1bcc648 promql: typo fix in comment
Signed-off-by: beorn7 <beorn@grafana.com>
2025-06-10 22:06:04 +02:00
beorn7
c3da460474 promql: Deactivate three failing tests for the time being
These tests fail on darwin/arm64.

One is expected, because the test demonstrates the limits of the
numerical accuracy of our methods, and different inaccurate outcomes
on different hardware are expected.

The other two are mysterious at the moment, see
https://github.com/prometheus/prometheus/issues/16714 for detailed
discussion and debugging.

Signed-off-by: beorn7 <beorn@grafana.com>
2025-06-10 21:48:52 +02:00
Björn Rabenstein
8fc1750bcc
Merge pull request #16569 from prometheus/beorn7/promql2
promql: Simplify avg aggregation and avg_over_time
2025-06-07 00:16:20 +02:00
beorn7
64431830b8 Add more avg_over_time test cases with extreme values
These tests were initially created by @crush-on-anechka. I modified
them slightly.

Signed-off-by: beorn7 <beorn@grafana.com>
2025-06-06 23:50:08 +02:00
Bryan Boreham
91eab63d38 [Refactor] PromQL: Simplify detectHistogramStatsDecoding
Restarting the depth-first walk on each leg of a binary expression is
convoluted. ISTM the correct logic is to walk the path backwards to the
first relevant function.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-06-06 14:52:59 +01:00
Arthur Silva Sens
8d0a747237
PROM-39: Provide PromQL info annotations when rate()/increase() over series without __type__="counter" label (#16632)
* Provide PromQL info annotations when rate()/increase() over series without counter label

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Address comments

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-06-06 10:01:20 -03:00
beorn7
ff67596a82 promql: Simplify avg aggregation and avg_over_time
As it turns out, if we combine Kahan summation and incremental mean
calculation properly, it works quite well and we do not need to switch
between simple mean calculation and incremental calculation based on
overflow.

This simplifies the code quite a bit.

Signed-off-by: beorn7 <beorn@grafana.com>
2025-06-05 23:26:56 +02:00
Björn Rabenstein
41594cebb4
Merge pull request #16566 from prometheus/beorn7/promql
promqltest: Adjust more tests to "left-open" ranges
2025-06-05 16:45:38 +02:00
beorn7
a4d57fce74 promqltest: Adjust more tests to "left-open" ranges
This was an oversight because the old tests still happened to pass
with the new behavior, but important test data was excluded at the
left end of the interval, rendering some tests not actually testing
what we want to test.

In the past, we also applied different strategies to adjust the test
(extend the range from 1m to 2m, or set the evaluation timestamp to
45s). This commit unifies things and reduces redundancy.

Signed-off-by: beorn7 <beorn@grafana.com>
2025-06-05 14:06:51 +02:00
György Krajcsovits
bccb1d6459
fix(promql): do not loose information about buckets when doing the detect
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-06-04 10:24:50 +02:00
György Krajcsovits
df94ecebab
fix(promql): histogram_count inconsistent
The problem is in the counter reset detection. The code that loads the
samples is matrixIterSlice which uses the typed Buffer iterator, which
will preload the integer histogram samples, however the last sample is
always(!) loaded as a float histogram sample in matrixIterSlice and the
optimized iterator fails to detect counter resets in that case.

Also the iterator does not reset lastH, lastFH properly.

Ref: https://github.com/prometheus/prometheus/issues/16681

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-06-03 12:31:27 +02:00
György Krajcsovits
e6b838391a
test(promql): histogram_count inconsistent
Ref: https://github.com/prometheus/prometheus/issues/16681

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-06-03 12:11:59 +02:00
tongjicoder
4fe20fa340 chore: fix some comments
Signed-off-by: tongjicoder <tongjicoder@icloud.com>
2025-05-27 23:14:41 +08:00
Julien
1d9dfde989
Merge pull request #16041 from prymitive/parenExprEnd
Add a test for aggregation wrapped in ParenExpr
2025-05-22 09:32:17 +02:00
György Krajcsovits
772d5ab433
Merge branch 'main' into krajo/intern-custom-values 2025-05-20 08:23:15 +02:00
Bartlomiej Plotka
8e6b008608
feature: type-and-unit-labels (PROM-39 implementation) (#16228)
* feature: type-and-unit-labels (extended MetricIdentity)

Experimental implementation of https://github.com/prometheus/proposals/pull/39

Previous (unmerged) experiments:
* https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels
* https://github.com/prometheus/prometheus/pull/16025

Signed-off-by: bwplotka <bwplotka@gmail.com>

feature: type-and-unit-labels (extended MetricIdentity)

Experimental implementation of https://github.com/prometheus/proposals/pull/39

Previous (unmerged) experiments:
* https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels
* https://github.com/prometheus/prometheus/pull/16025

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Fix compilation errors

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

Lint

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

Revert change made to protobuf 'Accept' header

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

Fix compilation errors for 'dedupelabels' tag

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Rectored into schema.Metadata

Signed-off-by: bwplotka <bwplotka@gmail.com>

* texparse: Added tests for PromParse

Signed-off-by: bwplotka <bwplotka@gmail.com>

* add OM tests.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* add proto tests

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* add schema label tests.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* fix tests.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* add promql tests.

Signed-off-by: bwplotka <bwplotka@gmail.com>

* lint

Signed-off-by: bwplotka <bwplotka@gmail.com>

* Addressed comments.

Signed-off-by: bwplotka <bwplotka@gmail.com>

---------

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-05-17 09:37:25 +00:00
Neeraj Gartia
8b0d33e5b2
promql: support variable scalar parameter in aggregations in range queries (#16404)
This fixes the regression introduced in https://github.com/prometheus/prometheus/issues/15971 while preserving the performance improvements.

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2025-05-11 15:40:31 +02:00
Neeraj Gartia
591242901a
promql: Refactor some functions to make them more DRY (#16532)
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2025-05-11 15:16:15 +02:00
Julien
59874fd89c
Merge pull request #16478 from KofClubs/range-vector-1001ms
promql: function selector sometimes misses a sample on dense samples
2025-05-08 12:02:10 +02:00
George Krajcsovits
f0471ff74b
Merge pull request #16552 from charleskorn/charleskorn/fix-16551
promql: don't emit a value from `histogram_fraction` or `histogram_quantile` if classic and native histograms are present at the same timestamp
2025-05-08 09:28:14 +02:00
György Krajcsovits
6dc6785473
chore(engine): add simple NHCB benchmark
Copy the benchmark for native histograms with exponential buckets and
adopt to native histograms with custom buckets.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-05-07 13:54:34 +02:00
Lukasz Mierzwa
7f58b4f784 Fix position in TestPreprocessAndWrapWithStepInvariantExpr
This is failing after 26088d01b9518b3315805186dcee534b986bf79f and it seems that currently tested position is incorrect, fix it

Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
2025-05-07 11:29:15 +01:00
György Krajcsovits
06d0b063ea fix(parser): parenthesis around aggregate expression parsing
The position range of nested aggregate expression was wrong, for the
expression "(sum(foo))" the position of "sum(foo)" should be 1-9, but
the parser could not decide the end of the expression on pos 9, instead
it read ahead to pos 10 and then emitted the aggregate. But we only
kept the last closing position (10) and wrote that into the aggregate.

The reason for this is that the parser cannot know from "(sum(foo)" alone
if the aggregate is finished. It could be finished as in "(sum(foo))" but
equally it could continue with group modifier as "(sum(foo) by (bar))".

The fix is to track ")" tokens in a stack in addition to the lastClosing
position, which is overloaded with other things like offset number tracking.


Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-05-07 11:17:39 +01:00
Lukasz Mierzwa
ba690a8c4c Add a test for aggregation wrapped in ParenExpr
Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com>
2025-05-07 11:17:24 +01:00
Björn Rabenstein
1d847f70c9
Merge pull request #16556 from NeerajGartia21/promql/promql-test-bug
[FIX] Promql-test: Fix error message for new eval syntax when `expect` line does not match with regex
2025-05-06 13:35:27 +02:00
Neeraj Gartia
a177376d72 fix promql-test error message and readme
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2025-05-05 23:23:35 +05:30
Charles Korn
2c0f02a702
promql: don't emit a value from histogram_fraction or histogram_quantile if classic and native histograms are present at the same timestamp
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2025-05-05 16:23:08 +10:00
Arve Knudsen
e7e3ab2824
Fix linting issues found by golangci-lint v2.0.2 (#16368)
* Fix linting issues found by golangci-lint v2.0.2

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-05-03 19:05:13 +02:00
Aman
26bddcf068
promql: histogram_stddev and histogram_stdvar should use arithmetic mean for custom buckets
Signed-off-by: amanycodes <amanycodes@gmail.com>
2025-04-24 14:48:58 +02:00
Zhang Zhanpeng
0658923bf0 fix conversion from time range in seconds to time duration at parser
Signed-off-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com>
2025-04-24 16:29:33 +08:00
Michael Hoffmann
d6d9f97bac
promql: histogram_fraction for bucket histograms (#16095)
* promql: histogram_fraction for bucket histograms

This PR extends the histogram_fraction function to also work with classic bucket histograms. This is beneficial because it allows expressions like sum(increase(my_bucket{le="0.5"}[10m]))/sum(increase(my_total[10m])) to be written without knowing the actual values of the "le" label, easing the transition to native histograms later on.
It also feels natural since histogram_quantile also can deal with classic histograms.

Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>

* promql: histogram_fraction for bucket histograms

* Add documentation and reduce code duplication
* Fix a bug in linear interpolation between bucket boundaries
* Add more PromQL tests

Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>

* Update docs/querying/functions.md

Co-authored-by: Björn Rabenstein <github@rabenste.in>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>

---------

Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Co-authored-by: Björn Rabenstein <github@rabenste.in>
2025-04-23 00:28:31 +02:00
Julien
f4ca1368ff
Merge pull request #16249 from roidelapluie/arythmetics
PromQL: allow arithmetic operations in durations in PromQL parser
2025-04-22 10:17:41 +02:00
Neeraj Gartia
a511164615
[REFACTOR] PromQL: DRY stddev and stdvar functions (#16451)
* DRY stddev and stdvar

---------

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2025-04-20 11:17:03 +02:00
David Leadbeater
e647f7954f
promtool: Add feature flags for promql features (#16443)
These are supported in the main prometheus binary but the feature flags
weren't supported in promtool.

Fixes #16412.

Signed-off-by: David Leadbeater <dgl@dgl.cx>
2025-04-17 10:29:44 +01:00
Björn Rabenstein
ba8609fbd0
Merge pull request #16352 from NeerajGartia21/promql/subquery_patch
[BUG] PromQL: Return early when `endtime` < `startime`
2025-04-16 01:00:19 +02:00
Julien Pivotto
e57512acfc Address feedback
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
2025-04-07 12:58:34 +02:00
Neeraj Gartia
f83e964231 fix NH load expression with incrementation more than once
Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2025-04-05 23:37:39 +05:30
Julien Pivotto
2cf2bf5b8f Address feedback
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
2025-04-03 13:19:55 +02:00
Julien Pivotto
2f6ad79edf Move calculation to the engine
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
2025-04-03 10:56:02 +02:00
Julien Pivotto
9cfd6f7df8 PromQL duration expr: Add a check for durations our of range
Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com>
2025-04-03 10:56:02 +02:00