16360 Commits

Author SHA1 Message Date
Ganesh Vernekar
64808d4f56
Merge pull request #16968 from pipiland2612/Remove_label_index
tsdb: Remove writing Label Index and Label Offset Table in the index
2025-08-05 15:12:44 -07:00
Sujal Shah
17c58f5fce wal: ignore os.ErrNotExist errors in DirSize during WAL size calculation
This change updates `DirSize` to ignore `os.ErrNotExist` errors,
since they are expected during normal WAL cleanup. All other errors
continue to propagate.

Fixes: #17005
Signed-off-by: Sujal Shah <sujalshah28092004@gmail.com>
2025-08-05 22:41:46 +05:30
Bryan Boreham
31eb3b286d [TESTS] PromQL: shrink work done in BenchmarkRangeQuery
Previously, BenchmarkRangeQuery would run each case with three data sizes
(1, 10, 100) and three range lengths (1, 100, 1000) for nine variations.

With 36 cases, running with `-count=6` to get dependable results, would
take 40-50 minutes in total.

This PR removes the middle option in both dimensions, thus shrinking to
four variations and about 20 minutes to run everything.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-08-05 14:21:23 +01:00
Bryan Boreham
e068c7332d [REFACTOR] TSDB: Clarify intersectPostings
This is intended to make `intersectPostings` easier to follow.

Instead of cryptic `arr` and `cur`, name the members `postings` and
`current`.

Instead of updating `cur` to intermediate values encountered during
operations, introduce a local variable `target` meaning the ref we might
expect to find next, and only update `current` when an intersection is
found.

Name the function which implements seeking `Seek` instead of `doNext`.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-08-05 13:09:29 +01:00
Alan Protasio
25aee26a57
Improving "Sparse postings" intersection (#13971)
Lets take the given example:

P1: [2, 5, 9, 18, 21]
P2: [3, 7, 14, 19, 21]
P3: [1, 21]

Currently, we would only advance through P1 and P2 until discovering
an intersection and then checking P3. In essence, the traversal order
was: 2, 3, 5, 7, 9, 14, 18, 19, 21 (intersection found).

With the proposed change, P3 is also examined even if P1 and P2
haven't found an intersection yet. This adjustment allows for the
possibility of skipping some iterations.

Post-change, the traversal order becomes: 2, 3, 21 (3 iterations instead of 9).

Signed-off-by: alanprot <alanprot@gmail.com>
2025-08-05 12:22:54 +01:00
George Krajcsovits
457a2381f9
fix(parser): revert pr 16041 and 16754 (#16996)
* fix(parser): revert pr 16041 and 16754

Fix for https://github.com/prometheus/prometheus/issues/16053
turned out to be too complicated and lead to more errors than it solved.
See https://github.com/prometheus/prometheus/pull/16875.

* add TODOs to wrong end positions
* restore test and fix initialization bug

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-08-05 10:48:08 +02:00
Owen Williams
74610b7c89
config: address edge case where local config specifies validation mode only (#16923)
This check ensures that local ScrapeConfigs that only specify Legacy validation do not inherit the default global AllowUTF8 escaping setting, which is an invalid combination of settings.

---------

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-08-04 13:53:50 -04:00
Matthieu MOREL
9f782164b4 chore: enable fatcontext linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-08-04 18:24:10 +02:00
Matthieu MOREL
0484a6d547 chore: enable-all rule from govet
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-08-04 18:06:36 +02:00
Owen Williams
5cc49720ac
otlp: update otlptranslator package (#16961)
* otlp: update otlptranslator package

Signed-off-by: Owen Williams <owen.williams@grafana.com>

* lint

Signed-off-by: Owen Williams <owen.williams@grafana.com>

* lint

Signed-off-by: Owen Williams <owen.williams@grafana.com>

* catch err

Signed-off-by: Owen Williams <owen.williams@grafana.com>

---------

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-08-04 18:02:16 +02:00
Matthieu MOREL
cef219c31c chore: enable unused-receiver rule from revive
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-08-04 09:43:33 +00:00
Matthias Rampke
789fddf25a
Merge pull request #16970 from prometheus/mr/repo-sync-manual-trigger
repo sync: allow manual triggering
2025-08-01 23:46:52 +02:00
Matthias Rampke
9ad799e0b1
repo sync: allow manual triggering
Add a [workflow dispatch](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#workflow_dispatch)
trigger to allow [manual triggering](https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow)
of the repo sync workflow.

I am often behind on merging the repo sync PRs. The script does not update them,
and I don't have the nerve right now to figure _that_ out. As a workaround, it
would be useful to trigger it manually after merging the outstanding ones, so I
can get a fresh round on the same day.

Signed-off-by: Matthias Rampke <matthias@prometheus.io>
2025-08-01 22:24:03 +02:00
pipiland2612
8b24acb729 Remove label index and labe offset index
Signed-off-by: pipiland2612 <nguyen.t.dang.minh@gmail.com>
2025-08-01 13:50:49 +03:00
Joe Adams
19dbe80376
go mod tidy
Signed-off-by: Joe Adams <github@joeadams.io>
2025-07-31 21:23:25 -04:00
Joe Adams
cdfb67467f
Review feedback
Signed-off-by: Joe Adams <github@joeadams.io>
2025-07-31 21:22:43 -04:00
Julius Volz
2e709c6567
Merge pull request #16695 from sujalshah-bit/block_endpoint
api: Create `/status/tsdb/blocks` endpoint.
2025-07-31 18:15:49 +02:00
David Ashpole
ff3882fd35
Merge pull request #16914 from dashpole/simplify_target
Simplify target_info addition in the otlptranslator
2025-07-30 13:21:01 -04:00
Bartlomiej Plotka
e35c09d84d
Merge pull request #16946 from thampiotr/fix-memory-corruption-parser
Fix labels memory corruption when using protobuf encoding
2025-07-30 13:16:58 +01:00
Piotr
be46837c12 fix memory corruption in labels when scraping with protbuf
Signed-off-by: Piotr <17101802+thampiotr@users.noreply.github.com>
2025-07-30 11:22:59 +01:00
Joe Adams
56a3bbf5c5
Fix import formatting
Signed-off-by: Joe Adams <github@joeadams.io>
2025-07-29 23:17:23 -04:00
Joe Adams
eab9b696f2
Upgrade AWS SDK to v2
AWS SDK v1 is end of life soon, so migrate to the V2 SDK. The credential loading should work more consistently with other projects that use the SDK and load credentials from the appropriate locations including from environment variables. This affects the EC2 and Lightsail service discovery features.

Signed-off-by: Joe Adams <github@joeadams.io>
2025-07-29 23:06:05 -04:00
machine424
13ed544426 chore: Volunteer Ayoub Mrini as Shepherd for Release 3.6
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2025-07-29 18:23:59 +02:00
Piotr
0b3ed4020c reproduce decoder labels memory corruption
Signed-off-by: Piotr <17101802+thampiotr@users.noreply.github.com>
2025-07-29 17:22:04 +01:00
Alex Greenbank
7a151d8959
update email address (#16941)
Signed-off-by: Alex Greenbank <alex@greenbank.org>
2025-07-29 14:25:12 +01:00
AxcelXander
a85618854a
Merge pull request #16721 from AxcelXander/fix-issue-14414-metadata-test
remote: Add metadata validation to TestSampleDelivery for v2 protocol
2025-07-29 14:18:21 +01:00
Bryan Boreham
1ada3ced5a
Merge pull request #16921 from bboreham/release-extra
[DOCS] Small extra note on release process
2025-07-28 17:35:51 +01:00
Bryan Boreham
e9cf792c47
Merge pull request #16895 from bboreham/noinline-kahan
[BUGFIX] PromQL: Noinline kahanSumInc, to reduce numerical errors
2025-07-28 15:55:19 +01:00
Julien
fe67b30e9e
Merge pull request #16731 from roidelapluie/rename-master-main
docs: Direct link to remote read proto to the main branch
2025-07-28 16:34:55 +02:00
Bryan Boreham
d9e5748a27
[REFACTOR] Scraping: Remove unnecessary yolostring calls (#16927)
Go will not allocate when reading from a map with a key cast from []byte to string.

Also remove some yoloString calls in package `textparse` - call a more suitable library function.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-07-28 14:54:32 +01:00
Arthur Silva Sens
2c04f2d7b1
PROM-39: Add type and unit labels to OTLP endpoint (#16630)
* PROM-39: Add type and unit labels to OTLP endpoint

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Extract label addition into helper function

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Wire feature flag and web handler configuration

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Apply suggestions from code review

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Use lowercase for units too

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Use otlptranslator.UnitNamer to build units

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Address copilot's comment

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Verify label presence before adding them

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Overwrite type/unit labels when already set

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* sed/addTypeAndUnitLabels/enableTypeAndUnitLabels/

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Reduce duplicated code

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-07-25 08:53:13 -03:00
Carrie Edwards
44b0fbba1e
No info annotation for rate/increase when type is histogram (#16915)
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-07-24 10:11:37 -07:00
George Krajcsovits
3f59fe1a80
fix(chunkenc): appending histograms with empty buckets (#16893)
* test(chunkenc): appending histograms with empty buckets and gaps

Append such native histograms that have empty buckets and gaps
between the indexes of those buckets.

There is a special case for appending counter native histograms to a chunk in TSDB: if we append a histogram that is missing some buckets that are already in chunk, then usually that's a counter reset. However if the missing bucket is empty, meaning its value is 0, then we don't consider it missing.

For this case to trigger , we need to write empty buckets into the chunk. Normally native histograms are compacted when we emit them , so this is very rare and compact make sure that there are no multiple continuous empty buckets with gaps between them.

The code that I've added in #14513 did not take into account that you can bypass compact and write histograms with many empty buckets, with gaps between them. These are still valid, so the code has to account for them.

Main fix in the expandIntSpansAndBuckets and expandFloatSpansAndBuckets function. I've also refactored them for clarity. Consequently needed to fix insert and adjustForInserts to also allow gaps between inserts.

I've added some new test cases (data driven test would be nice here, too many cases). And removed the deprecated old function that didn't deal with empty buckets at all.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2025-07-24 18:01:02 +02:00
Bryan Boreham
442512fbb9 [DOCS] Small extra note on release process
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-07-24 16:44:01 +01:00
Julius Volz
13b55ffc81
UI: More inclusive metadata handling for _count/_sum/_bucket suffixes (#16910)
Although these suffixes always need to be removed before querying metadata for
metrics that follow the Prometheus naming best practices, there can also be
metrics that don't follow these naming practices and have these suffixes
without being part of either a histogram or a summary metric.

Fixes https://github.com/prometheus/prometheus/issues/16907

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2025-07-24 13:54:58 +02:00
buda
f8b3fce845
remote write 2.0: add write handler benchmark for V2 format (#16687)
Signed-off-by: Sandro Jijavadze <sandrojijavadze@protonmail.com>
2025-07-23 13:51:06 -07:00
David Ashpole
c8ac3e46ca
simplify target_info addition in the otlptranslator
Signed-off-by: David Ashpole <dashpole@google.com>
2025-07-23 16:49:17 +00:00
Darshan Chaudhary
9b00c296a8
[PERF] PromQL: Pass concrete types to promql functions (#16797)
Currently, the promql functions take the interface slice []parser.Value as an argument,
which is being implemented by the conrete types Vector, Matrix etc. This PR replaces
the interface with the concrete types, resulting in improved performance.
The inspiration for this PR came from #16698 which does this for binops. 
I extended the idea to all promql functions

Signed-off-by: darshanime <deathbullet@gmail.com>

* pass single Matrix

Signed-off-by: darshanime <deathbullet@gmail.com>

---------

Signed-off-by: darshanime <deathbullet@gmail.com>
2025-07-23 17:30:03 +01:00
machine424
9a0bbb60bc test(tsdb): disable TestDelayedCompaction/delayed_compaction_enabled on windows
as flaky because of Time imprecision

fixes https://github.com/prometheus/prometheus/issues/16450

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2025-07-22 15:30:05 +01:00
Olivier Lemasle
da9b8b2f8d chore(configuration.md): Fix typo
Signed-off-by: Olivier Lemasle <o.lemasle@gmail.com>
2025-07-22 13:50:21 +01:00
Julius Volz
19a8f8c8ea
Merge pull request #16901 from prometheus/configurable-notice-alerts
Make query warning/info notice display configurable
2025-07-21 14:24:53 +02:00
Julius Volz
1354bb90be Make query warning/info notice display configurable
Fixes https://github.com/prometheus/prometheus/issues/16888

Signed-off-by: Julius Volz <julius.volz@gmail.com>
2025-07-21 13:03:46 +02:00
Harry John
b09cf6be8d
fix(promql): Ensure native histogram values copied in subqueries (#16879)
Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>
2025-07-21 08:54:54 +02:00
Bryan Boreham
358499f868 [BUGFIX] PromQL: Noinline kahanSum, to avoid compiler optimisations
Observed on go1.24.4 darwin/arm64, the compensation variable is not correctly calculated.

Note you may have to run the tests several times to see it, to get the right ordering of series.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-07-19 15:19:29 +01:00
Arthur Silva Sens
3602785a89
Support including scope metadata as metric labels (#16878)
* Support including scope metadata as metric labels

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Ensure Scope Name, Version and Schema URL aren't overriden by attributes

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-07-18 13:44:05 -03:00
Charles Korn
46acc974c0
fix(remote): Unregister metrics emitted by remote.WriteStorage when closed (#16868)
* Unregister metrics emitted by `remote.WriteStorage` when closed

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Address PR feedback: add test

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2025-07-17 11:32:15 +02:00
Ayoub Mrini
9dc274687b
Merge pull request #16831 from machine424/nsmeta
feat(discovery/kubernetes): allow attaching namespace metadata
2025-07-17 10:30:27 +01:00
Bryan Boreham
6b37b42f6f
Merge pull request #16616 from prymitive/stringLabels
Rename labels.go -> labels_slicelabels.go
2025-07-17 09:28:29 +01:00
machine424
b74c3ce153
chore(configuration.md): adjust kubernetes DS config
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2025-07-17 09:55:10 +02:00
machine424
a9f6fdd910
feat(discovery/kubernetes): allow attaching namespace metadata
to ingress and service roles.

with the help of claude-4-sonnet

Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2025-07-17 09:53:16 +02:00