14260 Commits

Author SHA1 Message Date
Bryan Boreham
6b36a5592a
Merge pull request #14618 from machine424/para
test(cmd/prometheus): speed up test execution by t.Parallel() when possible
2024-10-20 18:19:51 +01:00
Ayoub Mrini
d8c1605930
Merge pull request #15164 from machine424/quantile
feat: normalize "le" and "quantile" labels values upon ingestion
2024-10-19 21:13:03 +02:00
machine424
cf128a0472
test(cmd/prometheus): speed up test execution by t.Parallel() when possible
turn some loops into subtests to make use of t.Parallel()

requires Go 1.22 to make use of https://go.dev/blog/loopvar-preview

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-18 17:52:59 +02:00
machine424
8bcb4d865d
feat: normalize "le" and "quantile" labels values upon ingestion
Signed-off-by: machine424 <ayoubmrini424@gmail.com>

Co-authored-by: beorn7 <beorn@grafana.com>
2024-10-18 17:37:29 +02:00
Ayoub Mrini
98dcd28b1a
Merge pull request #15170 from machine424/awldi
fix(discovery): Handle cache.DeletedFinalStateUnknown in node informers' DeleteFunc
2024-10-18 17:33:08 +02:00
Alex Greenbank
421a3c22ea
scrape: provide a fallback format (#15136)
scrape: Remove implicit fallback to the Prometheus text format

Remove implicit fallback to the Prometheus text format in case of invalid/missing Content-Type and fail the scrape instead. Add ability to specify a `fallback_scrape_protocol` in the scrape config.

---------

Signed-off-by: alexgreenbank <alex.greenbank@grafana.com>
Signed-off-by: Alex Greenbank <alex.greenbank@grafana.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2024-10-18 17:12:31 +02:00
Ayoub Mrini
5505c83a4d
Merge pull request #15167 from machine424/impor
feat: ProtobufParse.formatOpenMetricsFloat: improve float formatting by using strconv.AppendFloat instead of fmt.Sprint
2024-10-18 15:35:21 +02:00
Alan Protasio
c78d5b94af
Disallowing configure AM with the v1 api (#13883)
* Stop supporting Alertmanager v1

* Disallowing configure AM with the v1 api

Signed-off-by: alanprot <alanprot@gmail.com>

* Update config/config_test.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update config/config.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Addressing coments

Signed-off-by: alanprot <alanprot@gmail.com>

* Update notifier/notifier.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update config/config_test.go

Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

---------

Signed-off-by: alanprot <alanprot@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>
2024-10-18 15:23:14 +02:00
machine424
18b81ad79d
feat: ProtobufParse.formatOpenMetricsFloat: improve float formatting by using strconv.AppendFloat instead of fmt.Sprint
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-18 15:19:55 +02:00
Bryan Boreham
754c104a3e
Merge pull request #15173 from prometheus/merge-2.55-into-main-3
Merge release-2.55 into main (interim)
2024-10-18 10:28:20 +01:00
George Krajcsovits
763cbdf35f
Merge pull request #15180 from prometheus/ooo-nh-corrupt-chunk
fix(tsdb): populateWithDelChunkSeriesIterator corrupting chunk meta
2024-10-18 10:49:02 +02:00
György Krajcsovits
a4083f14e8 Fix populateWithDelChunkSeriesIterator corrupting chunk meta
When handling recoded histogram chunks the min time of the chunk is
updated by mistake. It should only update when the chunk is completely new.
Otherwise the ongoing chunk's meta will be later than the previously
written samples in it.

Same bug as https://github.com/prometheus/prometheus/pull/14629

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-18 10:34:22 +02:00
György Krajcsovits
e6a682f046 Reproduce populateWithDelChunkSeriesIterator corrupting chunk meta
When handling recoded histogram chunks the min time of the chunk is
updated by mistake. It should only update when the chunk is completely
new.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-18 10:34:22 +02:00
Bartlomiej Plotka
efc43d0714
s/scrape_classic_histograms/always_scrape_classic_histograms (3.0 breaking change) (#15178)
This is for readability, especially when we can converting to nhcb option.

See discussion https://cloud-native.slack.com/archives/C077Z4V13AM/p1729155873397889

Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-18 08:32:15 +01:00
Yi
2cabd1b707
config: remove expand-external-labels flag in release 3.0 (#14657)
remove expand-external-labels feature flag

and enabled env arg expansion for external labels by default.

Signed-off-by: jyz0309 <45495947@qq.com>
2024-10-17 10:25:05 +02:00
George Krajcsovits
6ce21b8399
Merge pull request #15172 from krajorama/update-client-golang
chore(deps): update client_golang from 1.20.4 to 1.20.5
2024-10-16 15:26:41 +02:00
Bryan Boreham
5dd5aec0e6
Merge pull request #14599 from machine424/sigsegv
test(tsdb): add a reproducer for https://github.com/prometheus/prometheus/issues/14422
2024-10-16 14:16:54 +01:00
Bryan Boreham
a846bf9a5e Merge branch 'release-2.55' into merge-2.55-into-main-3 2024-10-16 14:08:54 +01:00
György Krajcsovits
4271670a82 chore(deps): update client_golang from 1.20.4 to 1.20.5
Ref: https://github.com/prometheus/client_golang/releases/tag/v1.20.5

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-16 15:02:13 +02:00
Joshua Hesketh
5a4e4f6936
Fix stddev/stdvar when aggregating histograms, NaNs, and infinities (#14941)
promql: Fix stddev/stdvar when aggregating histograms, NaNs, and Infs

Native histograms are ignored when calculating stddev or stdvar.

However, for the first series of each group, a `groupedAggregation` is
always created. If the first series that was encountered is a histogram
then it acts as the equivalent of a 0 point.

This change creates the first `groupedAggregation` with the `seen` field set to `false` if the point is a
histogram, thus ignoring it like the rest of the aggregation function does. A new `groupedAggregation`
will then be created once an actual float value is encountered.

This commit also sets the `floatValue` field of the `groupedAggregation` to `NaN`, if the first
float value of a group is `NaN` or `±Inf`, so that the outcome is consistently `NaN` once those
values are in the mix.

(The added tests fail without this change).

Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
Signed-off-by: beorn7 <beorn@grafana.com>

---------

Signed-off-by: Joshua Hesketh <josh@nitrotech.org>
Signed-off-by: beorn7 <beorn@grafana.com>
Co-authored-by: beorn7 <beorn@grafana.com>
2024-10-16 15:00:46 +02:00
akunszt
08a7162502
discovery: aws/ec2 unit tests (#14364)
* discovery: add aws/ec2 unit tests 

* discovery: initial skeleton for aws/ec2 unit tests

This is a - very likely - not too useful unit test for the AWS SD. It is
commited so other people can check the basic logic and the
implementation.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: fix linter complains about ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: add basic unit test for aws

This tests only the basic labelling, not including the VPC related
information.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: fix linter complains about ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: other linter fixes in aws/ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: implement remaining tests for aws/ec2

The coverage is not 100% but I think it is a good starting point if
someone wants to improve that.

Currently it covers all the AWS API calls.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: make linter happy in aws/ec2_test.go

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: make utility funtcions private

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discover: no global variable in the aws/ec2 test

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: common body for some tests in ec2

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: try to make golangci-lint happy

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: make every non-test function private

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: test for errors first in TestRefresh

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: move refresh tests into the function

This way people can find both the test cases and the execution of the
test at the same place.

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: fix copyright date

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: remove misleading comment

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: rename test for easier identification

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: use static values for the test cases

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discover: try to make the linter happy

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: drop redundant data from ec2 and use common ptr functions

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: use Error instead of Equal

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

* discovery: merge refreshAZIDs tests into one

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>

---------

Signed-off-by: Arpad Kunszt <akunszt@hiya.com>
2024-10-16 14:36:37 +02:00
Bryan Boreham
249969d7ff
Merge pull request #15168 from aknuds1/arve/refactor-range-eval
[Release 2.55] Refactor evaluator.rangeEval by splitting out gatherVector method
2024-10-16 13:19:36 +01:00
Arve Knudsen
e05e97cdd7 evaluator.rangeEval: Split out gatherVector method
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-16 14:01:03 +02:00
Bryan Boreham
04ec65b74a
Merge pull request #15166 from aknuds1/arve/upgrade-enterprise-certificate-proxy
[Release 2.55] [BUILD] Upgrade github.com/googleapis/enterprise-certificate-proxy to v0.3.4
2024-10-16 12:52:25 +01:00
George Krajcsovits
55419275ab
Merge pull request #15159 from prometheus/nhcb-scrape-optimize1
convertnhcb: use CutSuffix instead of regex replace for histogram name
2024-10-16 10:25:18 +02:00
machine424
b1c356beea
fix(discovery): Handle cache.DeletedFinalStateUnknown in node informers' DeleteFunc
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-16 10:20:37 +02:00
machine424
ab2475c426
test(tsdb): add a reproducer for https://github.com/prometheus/prometheus/issues/14422
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2024-10-15 20:39:25 +02:00
Arve Knudsen
90cc7e5723 Upgrade github.com/googleapis/enterprise-certificate-proxy to v0.3.4
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-15 16:37:58 +02:00
Bryan Boreham
e7f23c049f
Merge pull request #15163 from prometheus/backport-14900-2-55
[Release 2.55] [REFACTOR] promql.Engine: Refactor vector selector evaluation into a method
2024-10-15 15:33:32 +01:00
Arve Knudsen
f7b396a1dc promql.Engine: Refactor vector selector evaluation into a method (#14900)
New method is named `evalVectorSelector`.

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-15 14:57:54 +01:00
Bryan Boreham
e26a3dba16
Merge pull request #15162 from prometheus/cp-15210-2-55
[Release 2.55] [BUGFIX] TSDB: Don't read in-order chunks from before head MinTime
2024-10-15 14:52:07 +01:00
Bryan Boreham
105c692f77 [BUGFIX] TSDB: Don't read in-order chunks from before head MinTime
Because we are reimplementing the `IndexReader` to fetch in-order and
out-of-order chunks together, we must reproduce the behaviour of
`Head.indexRange()`, which floors the minimum time queried at `head.MinTime()`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-10-15 14:23:05 +01:00
Neeraj Gartia
d4b1f9eb33
Corrects the behaviour of binary opperators between histogram and float (#14726)
promql: corrects binary operators functioning for mixed sample with histogram and float

For invalid pairings of sample types, an annotation is added now.

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>

---------

Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com>
2024-10-15 14:44:36 +02:00
Bryan Boreham
1e1f6ab9df
Merge pull request #15120 from bboreham/floor-ino-mint
[BUGFIX] TSDB: Don't read in-order chunks from before head MinTime
2024-10-15 10:27:38 +01:00
Arthur Silva Sens
911c3efb96
Merge pull request #14976 from prometheus/prep-3.0-beta.1
Prepare 3.0.0-beta.1
v3.0.0-beta.1 v0.300.0-beta.1
2024-10-14 12:44:41 -03:00
György Krajcsovits
78de9bd10f convertnhcb: use CutSuffix instead of regex replace for histogram name
This is much quicker.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-14 16:50:05 +02:00
George Krajcsovits
522149a2ae
model: move classic NHCB conversion into its own file (#15156)
* model: move classic to NHCB conversion into its own file

In preparation for #14978.

Author: Jeanette Tan <jeanette.tan@grafana.com>  2024-07-03 11:56:48
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com>
Co-authored-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Better naming from review comment

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Add doc strings.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Co-authored-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-10-14 14:23:11 +02:00
Manik Rana
c3b8582cf4
[PERF] textparse: optimize OM p.isCreatedSeries() (#15150)
* refac: remove p.Metric usage

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* perf: use bytes instead of strings

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: comments

Signed-off-by: Manik Rana <Manikrana54@gmail.com>

---------

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
2024-10-14 10:18:15 +01:00
George Krajcsovits
b8867f8ead
Merge pull request #15142 from krajorama/fix-appendhistogram-race
bugfix: data race in head.Appender.AppendHistogram and Commit
2024-10-14 08:13:39 +02:00
M Viswanath Sai
16bba78f15
discovery: Improve Azure test coverage to 50% (#14586)
* azure sd: separate refresh and refreshAzure
* azure sd: create a client with mocked servers for tests
* add test for refresh function

---------

Signed-off-by: mviswanathsai <mviswanath.sai.met21@itbhu.ac.in>
2024-10-13 10:24:51 +02:00
dependabot[bot]
7ce9cfa6e4
Bump the k8s-io group with 3 updates (#15030)
Bumps the k8s-io group with 3 updates: [k8s.io/api](https://github.com/kubernetes/api), [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery) and [k8s.io/client-go](https://github.com/kubernetes/client-go).


Updates `k8s.io/api` from 0.31.0 to 0.31.1
- [Commits](https://github.com/kubernetes/api/compare/v0.31.0...v0.31.1)

Updates `k8s.io/apimachinery` from 0.31.0 to 0.31.1
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.31.0...v0.31.1)

Updates `k8s.io/client-go` from 0.31.0 to 0.31.1
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kubernetes/client-go/compare/v0.31.0...v0.31.1)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-io
- dependency-name: k8s.io/apimachinery
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-io
- dependency-name: k8s.io/client-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-io
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-12 10:40:23 +02:00
dependabot[bot]
1e3953ab13
Bump github.com/docker/docker (#15033)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 27.2.0+incompatible to 27.3.1+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v27.2.0...v27.3.1)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-12 10:38:57 +02:00
Oleg Zaytsev
50ef0dc954
Fix MemPostings.Add and MemPostings.Get data race (#15141)
* Tests for Mempostings.{Add,Get} data race
* Fix MemPostings.{Add,Get} data race

We can't modify the postings list that are held in MemPostings as they
might already be in use by some readers.

* Modify BenchmarkHeadStripeSeriesCreate to have common labels

If there are no common labels on the series, we don't excercise the
ordering part of MemSeries, as we're just creating slices of one element
for each label value.

---------

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2024-10-11 15:21:15 +02:00
Julien
e6f9e2dde3
Merge pull request #15119 from roidelapluie/dochttp
docs: extract HTTP client option documentation in their own sections
2024-10-11 13:10:34 +02:00
Ayoub Mrini
d9a54284f5
Merge pull request #14987 from machine424/yafix
fix(notifier): avoid dropping known alertmanagers after each ApplyConfig
2024-10-10 21:14:35 +02:00
György Krajcsovits
bb70370d72 TSDB head: fix race between AppendHistogram and Commit
Move writing memSeries lastHistogramValue and lastFloatHistogramValue
after series creation under lock.

The resulting code isn't totally correct in the sense that we're setting
these values before Commit() , so they might be overwritten/rolled back
later.

Also Append of stale sample checks the values without lock, so there's
still a potential race.

The correct solution would be to set these only in Commit() which we
actually do, but then Commit() would also need to process samples in
order and not floats first, then histograms, then float histograms - which
leads to not knowing what stale marker to write for histograms.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-10 16:59:15 +02:00
György Krajcsovits
631fadc4ca Unit test for data race in head.Appender.AppendHistogram
Two Appenders race when creating a series with a native histogram
as the memSeries will be common and the lastHistogram field is written
without lock.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-10 14:10:07 +02:00
Manik Rana
032ca9ef96
[PERF] textparse: further optimzations for OM CreatedTimestamps (#15097)
* feat: Added more tests; some changes/optimizations when pair-programming with Bartek.

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: imports

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: gofumpt

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: use an efficient replacement to p.Metric

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* feat: reduce mem allocs + comments

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: gofumpt

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* chore: use single quotes

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: rename

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: rename to seriesHash

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: switch condition order

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* refac: switch condition order

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* feat: stronger checking

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>

* chore: fmt

Signed-off-by: Manik Rana <manikrana54@gmail.com>

* refac: pass pointer of buf into seriesHash()

Signed-off-by: Manik Rana <manikrana54@gmail.com>

---------

Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-10 12:01:13 +01:00
Björn Rabenstein
8b545bab2f
Merge pull request #15026 from huochexizhan/main
fix: fix slice init length
2024-10-10 12:55:28 +02:00
Björn Rabenstein
720a2599a6
Merge pull request #15135 from prometheus/beorn7/docs
docs: Update chunk layot for NHCB
2024-10-10 12:00:51 +02:00