451 Commits

Author SHA1 Message Date
Peter Štibraný
6d76f09c58
Extract interface from ActivityQueryTracker and allows passing custom implementation (#10071)
* Extract interface from ActivityQueryTracker and allows passing custom implementation.

Signed-off-by: Peter Štibraný <pstibrany@gmail.com>
2022-01-29 23:55:27 +01:00
Matej Gera
2c61d29b2a
Tracing: Migrate to OpenTelemetry library (#9724)
Signed-off-by: Matej Gera <matejgera@gmail.com>
2022-01-25 11:08:04 +01:00
beorn7
b39f2739e5 PromQL: Always enable negative offset and @ modifier
This follows the line of argument that the invariant of not looking
ahead of the query time was merely emerging behavior and not a
documented stable feature. Any query that looks ahead of the query
time was simply invalid before the introduction of the negative offset
and the @ modifier.

Signed-off-by: beorn7 <beorn@grafana.com>
2022-01-11 17:08:55 +01:00
beorn7
53ca375345 promql: Add a guard against a nil histogram in sum aggregation
This can happen if the aggregation starts with a float and later
encounters a histogram. In that case, the newly encountered histogram
would have been added to a nil histogram.

This should be tested, of course, but that's best done within the
PromQL testing framework, which we still need to enable for histograms
(for which we have a TODO in the code and now also a card in the GH
project).

Signed-off-by: beorn7 <beorn@grafana.com>
2021-12-15 14:33:44 +01:00
Ganesh Vernekar
f580248759
Support + operator for sparse histograms (#9949)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-12-06 23:06:58 +05:30
Ganesh Vernekar
187a767292
Implement sum() for sparse histograms (#9948)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-12-06 21:38:10 +05:30
Ganesh Vernekar
4a43349aca
histogram_quantile for sparse histograms (#9935)
* MergeFloatBucketIterator for []FloatBucketIterator

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* histogram_quantile for histograms

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix histogram_quantile

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Unit test and enhancements

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Iterators to iterate buckets in reverse and all buckets together including zero bucket

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Consider all buckets for histogram_quantile and fix the implementation

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Remove unneeded code

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-12-06 19:17:22 +05:30
Björn Rabenstein
4ce01e9770
storage: Rename ...Values methods to At... (#9889)
This mirrors #9888 for the richer iterators we have with histograms in
the game.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-29 16:23:04 +05:30
Björn Rabenstein
d677aa4b29
storage: Consolidate iterator method names (Values -> At) (#9888)
`BufferedSeriesIterator` and `MemoizedSeriesIterator` use a method
called `Values` for exactly the purpose for which all other iterators
of the same kind use a method called `At`. That alone is confusing,
but on top of that, the `Values` method only returns a single sample,
not multiple values. I assume the naming has historical reasons. This
commit makes it more consistent. It is now easier to read, and now
`BufferedSeriesIterator` and `MemoizedSeriesIterator` implement
`chunkenc.Iterator` like many other iterators, too.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-29 11:16:40 +01:00
Björn Rabenstein
7e42acd3b1
tsdb: Rework iterators (#9877)
- Pick At... method via return value of Next/Seek.
- Do not clobber returned buckets.
- Add partial FloatHistogram suppert.

Note that the promql package is now _only_ dealing with
FloatHistograms, following the idea that PromQL only knows float
values.

As a byproduct, I have removed the histogramSeries metric. In my
understanding, series can have both float and histogram samples, so
that metric doesn't make sense anymore.

As another byproduct, I have converged the sampleBuf and the
histogramSampleBuf in memSeries into one. The sample type stored in
the sampleBuf has been extended to also contain histograms even before
this commit.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-29 13:24:23 +05:30
beorn7
8e4e8726bb promql: Fix another ChunkEncoding call
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-22 21:05:49 +01:00
beorn7
5d4db805ac Merge branch 'main' into sparsehistogram 2021-11-17 19:57:31 +01:00
beorn7
9de3ab60df promql: improve histogram support in engine.go
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-16 13:20:24 +01:00
beorn7
73858d7f82 storage: histogram support in memoized_iterator
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-15 21:55:58 +01:00
beorn7
4c28d9fac7 Move to histogram.Histogram pointers
This is to avoid copying the many fields of a histogram.Histogram all
the time.

This also fixes a bunch of formerly broken tests.

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-12 23:17:35 +01:00
Thomas Jackson
f0003bc0ba
Don't drop ParenExpr when creating StepInvariantExpr (#9591)
* Add test case to showcase the problem in #9590

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>

* Don't unwrap ParenExpr in newStepInvariantExpr

Fixes #9590

Signed-off-by: Thomas Jackson <jacksontj.89@gmail.com>
2021-11-10 20:16:24 +05:30
beorn7
c954cd9d1d Move packages out of deprecated pkg directory
This creates a new `model` directory and moves all data-model related
packages over there:
  exemplar labels relabel rulefmt textparse timestamp value

All the others are more or less utilities and have been moved to `util`:
  gate logging modetimevfs pool runtime

Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-09 08:03:10 +01:00
beorn7
8f92c90897 Add TODOs and some minor tweaks
Signed-off-by: beorn7 <beorn@grafana.com>
2021-11-07 17:12:04 +01:00
Ganesh Vernekar
c8b267efd6
Get histograms from TSDB to the rate() function implementation
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-11-03 19:04:18 +05:30
Mateusz Gozdek
1a6c2283a3 Format Go source files using 'gofumpt -w -s -extra'
Part of #9557

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2021-11-02 19:52:34 +01:00
Bryan Boreham
a278ea4b58
promql: copy data when short-circuiting (#9552)
* promql: copy data when short-circuiting

Because the range query loop re-uses the output buffer each time round,
we must copy results into the buffer rather than using input as output.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-10-20 16:03:02 +02:00
Julien Pivotto
a18224d02d
make aggregations deterministic (#9459)
* Add deterministic test for aggregations

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Make aggregations deterministic

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>

* Increase testing

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2021-10-17 15:16:38 +05:30
ziollek
55f9147b44
Add atan2 to scalar operators - issue #9485 (#9515)
* Add atan2 to scalar operators

Signed-off-by: Tomasz Ziolkowski <tomasz.ziolkowski@allegro.pl>
2021-10-15 16:03:11 +02:00
Levi Harrison
8547a2bd86 Add atan2 binary operator
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-09-23 10:30:46 -04:00
Bryan Boreham
5a754bc043
Short-circuit vector binary ops (#9362)
In degenerate cases we can save the effort of building a map.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-09-21 17:37:36 +05:30
Bryan Boreham
c4942ef3b7
Optimise query_range by computing join signatures just once (#9360)
* Add benchmark case for many-to-one join

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* query_range: compute join signatures just once

For an expression like `a + on(p,q) b`, extract the `p,q` part from each
series once, instead of re-computing at every step of the range.

Although there was a cache, computing the key by concatenating all
labels was expensive.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-09-21 15:58:39 +05:30
Bryan Boreham
7d105277fe
Optimise topk where k==1 (#9365)
* Add benchmark for query_range with topk

Modify sample data so values within a metric differ

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Optimise topk where k==1

In this case we don't need a heap to keep track of values; just a single
slot is fine.

Simplify the initialization of the heap: since all cases start off as a
single-item heap we can just assign the value directly.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>

* Allow at least one slot in results for topk, quantile

k isn't set for quantile, but we need space to start collecting values

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2021-09-21 15:57:28 +05:30
Darshan Chaudhary
c4f2e9eec5
Add present_over_time (#9097)
* Add present_over_time

Signed-off-by: darshanime <deathbullet@gmail.com>

* Add tests for present_over_time

Signed-off-by: darshanime <deathbullet@gmail.com>

* Address PR comments

Signed-off-by: darshanime <deathbullet@gmail.com>

* Add documentation for present_over_time

Signed-off-by: darshanime <deathbullet@gmail.com>

* Update documentation

Signed-off-by: darshanime <deathbullet@gmail.com>

* Update documentation comment

Signed-off-by: darshanime <deathbullet@gmail.com>
2021-07-29 12:38:11 +02:00
darshanime
364c40be57 Add Stringer to Query interface
Signed-off-by: darshanime <deathbullet@gmail.com>
2021-07-11 19:23:34 +05:30
Levi Harrison
b5f6f8fb36 Switched to go-kit/log
Signed-off-by: Levi Harrison <git@leviharrison.dev>
2021-06-11 12:28:36 -04:00
yeya24
d698e062dc improve grouping label match logic
Signed-off-by: yeya24 <yb532204897@gmail.com>
2021-04-16 22:04:58 -04:00
Marco Pracucci
6719071a0f
Optimize aggregations in PromQL engine (#8594)
* Optimize aggregations in PromQL engine

Signed-off-by: Marco Pracucci <marco@pracucci.com>
2021-03-19 17:52:29 +01:00
Marco Pracucci
7bbab380b6
Added tracing span to evaluator.eval()
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2021-03-15 15:05:47 +01:00
Marco Pracucci
b92c03023d
Optimized vector selector
Signed-off-by: Marco Pracucci <marco@pracucci.com>
2021-03-11 14:32:56 +01:00
pschou
f80b52be69
Merge branch 'main' into dev_neg_offset 2021-02-23 20:52:57 -05:00
schou
75d932a172 var init for bool
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:26:35 -05:00
schou
22bfc11738 aggregate booleans for ease of reading
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:26:35 -05:00
schou
22cd48868a adding feature flag, promql-negative-offset
Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-23 20:25:56 -05:00
pschou
aff3c702ab
promql: Add sgn, clamp and last_over_time functions (#8457)
* Add sgn, clamp and last_over_time functions

Signed-off-by: schou <pschou@users.noreply.github.com>
2021-02-20 16:34:52 +01:00
Ganesh Vernekar
86c71856e8
Add start() and end() pre-processors for @ modifier (#8425)
* Add start() and end() pre-processors for @ modifier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix reviews

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix review comments

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-02-09 21:33:16 +05:30
Marcelo E. Magallon
75d86c6747 Update golangci-lint to 1.36.0
In the previous version, 1.18.0, the "megacheck" linter paid attention
to the '//lint:ignore' comment, but that is no longer there.

Newer version pay attention to '//nolint:<linter>,<linter>,...'
comments, optionally followed by a "second" comment introduced by '//'.

Update the directives to use this style.

This is related to prometheus/blackbox_exporter#738 and
prometheus/blackbox_exporter#745.

Signed-off-by: Marcelo E. Magallon <marcelo.magallon@grafana.com>
2021-02-04 08:53:33 -06:00
Ganesh Vernekar
b18fde996e
Fix timestamp() function for @ modifier
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-02-03 19:13:12 +05:30
Ganesh Vernekar
9199fcb8d1
'@ <timestamp>' modifier (#8121)
This commit adds `@ <timestamp>` modifier as per this design doc: https://docs.google.com/document/d/1uSbD3T2beM-iX4-Hp7V074bzBRiRNlqUdcWP6JTDQSs/edit.

An example query:

```
rate(process_cpu_seconds_total[1m]) 
  and
topk(7, rate(process_cpu_seconds_total[1h] @ 1234))
```

which ranks based on last 1h rate and w.r.t. unix timestamp 1234 but actually plots the 1m rate.

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-01-20 16:27:39 +05:30
Ganesh Vernekar
d30da66d77
Fix timestamp() method for vector selector inside paren (#8164)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2020-11-09 18:21:50 +05:30
Harkishen Singh
fc8e769d71
Use ASSIGN when using = inside braces (#7911)
* Fix EQL when using = inside braces.

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* EQL => EQLC and ASSIGN => EQL

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>

* Aligned yacc code.

Signed-off-by: Harkishen-Singh <harkishensingh@hotmail.com>
2020-09-09 15:40:02 +05:30
Vijay Samuel
00ee73ef91
Export members of EvalNodeHelper to facilitate usage in external functions (#7860)
Signed-off-by: Vijay Samuel <vjsamuel@ebay.com>
2020-08-27 19:30:10 +01:00
Julien Pivotto
6f9e7ff750
Drop metric name in bool comparison between two instant vectors (#7819)
* Drop metric name in bool comparison between two instant vectors

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-08-22 21:04:03 +02:00
Julien Pivotto
20ab94fedf
Hints: Separating out the range and offsets of PromQL subqueries (#7667)
Fix #7629

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-08-11 07:21:39 +01:00
Annanay Agarwal
118aeab02c
Make context key type public (#7748)
Signed-off-by: Annanay <annanayagarwal@gmail.com>
2020-08-05 09:51:36 +01:00
Julien Pivotto
22acb87e09
refactoring: make sure that query_duration_seconds metrics are the same (#7668)
* refactoring: make sure that query_duration_seconds are the same

Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2020-07-25 11:55:59 +02:00