Commit Graph

1609 Commits

Author SHA1 Message Date
Carrie Edwards
90339416e1 Fix tests
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:25:29 -07:00
Carrie Edwards
9f78ddded2 Fix linting
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:14:25 -07:00
Carrie Edwards
455ab8806f Fix linting
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:14:25 -07:00
Carrie Edwards
db3a7845c5 Fix FromMetrics tests
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:14:25 -07:00
Carrie Edwards
d650307ab4 Fix linting and temporality tests
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:14:23 -07:00
Carrie Edwards
1990beabe9 Do not require type and unit labels enabled to add temporality label per proposal update
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:13:31 -07:00
Carrie Edwards
6c064893e4 Add tests for delta temporality support
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:13:31 -07:00
Carrie Edwards
ff693401eb Set metadata type as gauge in otelMetricTypeToPromMetric
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:13:29 -07:00
Carrie Edwards
9c32a69bf3 Add temporality label for metrics ingested in OTLP endpoint
Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-08-05 14:11:47 -07:00
Owen Williams
5cc49720ac
otlp: update otlptranslator package (#16961)
* otlp: update otlptranslator package

Signed-off-by: Owen Williams <owen.williams@grafana.com>

* lint

Signed-off-by: Owen Williams <owen.williams@grafana.com>

* lint

Signed-off-by: Owen Williams <owen.williams@grafana.com>

* catch err

Signed-off-by: Owen Williams <owen.williams@grafana.com>

---------

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-08-04 18:02:16 +02:00
David Ashpole
ff3882fd35
Merge pull request #16914 from dashpole/simplify_target
Simplify target_info addition in the otlptranslator
2025-07-30 13:21:01 -04:00
AxcelXander
a85618854a
Merge pull request #16721 from AxcelXander/fix-issue-14414-metadata-test
remote: Add metadata validation to TestSampleDelivery for v2 protocol
2025-07-29 14:18:21 +01:00
Arthur Silva Sens
2c04f2d7b1
PROM-39: Add type and unit labels to OTLP endpoint (#16630)
* PROM-39: Add type and unit labels to OTLP endpoint

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Extract label addition into helper function

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Wire feature flag and web handler configuration

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Apply suggestions from code review

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Use lowercase for units too

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Use otlptranslator.UnitNamer to build units

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Address copilot's comment

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Verify label presence before adding them

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Overwrite type/unit labels when already set

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* sed/addTypeAndUnitLabels/enableTypeAndUnitLabels/

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Reduce duplicated code

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-07-25 08:53:13 -03:00
buda
f8b3fce845
remote write 2.0: add write handler benchmark for V2 format (#16687)
Signed-off-by: Sandro Jijavadze <sandrojijavadze@protonmail.com>
2025-07-23 13:51:06 -07:00
David Ashpole
c8ac3e46ca
simplify target_info addition in the otlptranslator
Signed-off-by: David Ashpole <dashpole@google.com>
2025-07-23 16:49:17 +00:00
Arthur Silva Sens
3602785a89
Support including scope metadata as metric labels (#16878)
* Support including scope metadata as metric labels

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* Ensure Scope Name, Version and Schema URL aren't overriden by attributes

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-07-18 13:44:05 -03:00
Charles Korn
46acc974c0
fix(remote): Unregister metrics emitted by remote.WriteStorage when closed (#16868)
* Unregister metrics emitted by `remote.WriteStorage` when closed

Signed-off-by: Charles Korn <charles.korn@grafana.com>

* Address PR feedback: add test

Signed-off-by: Charles Korn <charles.korn@grafana.com>

---------

Signed-off-by: Charles Korn <charles.korn@grafana.com>
2025-07-17 11:32:15 +02:00
Bryan Boreham
36ec1e4fb0
Merge branch 'main' into merge-3.5-into-main
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2025-07-14 15:29:00 +01:00
Owen Williams
d2f1f4fb27
config: Add UnderscoreEscapingWithoutSuffixes translation strategy (#16849)
The last permutation of the translation options does underscore translation but does not add suffixes.
This translation option already exists in Mimir as otel_metric_suffixes_enabled, indicating external demand for this strategy.
There is an accompanying update to prometheus-docs to explain the use of this mode: https://github.com/prometheus/docs/pull/2688

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-07-10 11:27:23 -04:00
Arthur Silva Sens
4b9d0fb92f
Revert: OTLP Support including scope metadata as metric labels (#16842)
Reverts #16730 and #16760

This is being done because we've noticed a problem in the spec that could
lead to name collisions if attributes name, version or schema_url are added
to the scope. They would collide with the already reserved labels
otel_scope_name, otel_scope_version and otel_scope_schema_url.

Since this new configuration option never made it into a release, we can
safely remove it from the 3.5 release. We'll sort this out for the 3.6 release

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-07-08 10:37:19 +00:00
Charles Korn
1e58d792a5
storage/remote: fix "http: read on closed response body" errors if chunkedSeriesSet.Next is called again after the series set is exhausted (#16838)
Signed-off-by: Charles Korn <charles.korn@grafana.com>
2025-07-07 09:23:34 +02:00
Michael Hoffmann
21b1536b5a
storage: add projection fields to select hints (#16423)
This commit adds Projection metadata to SelectHints so that downstream
storage implementations can use it to save effort when answering to
Select calls.

Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com>
2025-07-06 12:57:19 +02:00
Arve Knudsen
f561aa795d
OTLP receiver: Generate target_info samples between the earliest and latest samples per resource (#16737)
* OTLP receiver: Generate target_info samples between the earliest and latest samples per resource

Modify the OTLP receiver to generate target_info samples between the earliest
and latest samples per resource instead of only one for the latest timestamp.
The samples are spaced lookback delta/2 apart.

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-07-04 14:38:16 +00:00
Arthur Silva Sens
0502f2d8fb
Bump otlptranslator to latest SHA
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-07-02 14:55:51 -03:00
Arve Knudsen
9c791faade
OTLP receiver: Don't append _total suffix to non-monotonic OTel sums (#16776)
* OTLP receiver: Don't append _total suffix to non-monotonic OTel sums

Fix the OTLP receiver so the suffix _total isn't appended to metrics
converted from non-monotonic OTel sum metrics, if otlp.translation_strategy is
UnderscoreEscapingWithSuffixes or NoUTF8EscapingWithSuffixes.

Also add translation tests.

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-06-25 18:18:45 +02:00
Arve Knudsen
c701636034
OTLP receiver: Rename otlp.convert_scope_metadata to otlp.promote_scope_metadata (#16760)
* Rename otlp.convert_scope_metadata to otlp.promote_scope_metadata

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Julius Hinze <juliusmh@proton.me>
2025-06-20 13:09:02 +00:00
Arve Knudsen
964bd7d1a9
OTLP: Support including scope metadata as metric labels (#16730)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-06-18 09:13:51 +02:00
AxcelXander
472f0de661
Enhance TestDropOldTimeSeries to test both v1 and v2 protocols (#16709)
- Wrapped existing test logic in a loop to run with both protocol versions
- Ensures consistent behavior across protocol versions for dropping old time series

Signed-off-by: AxcelXander <tyz666@bu.edu>
Co-authored-by: AxcelXander <tyz666@bu.edu>
2025-06-11 13:06:45 -07:00
zenador
40a27ce875
Make translatorMetricFromOtelMetric public (#16683)
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2025-06-03 12:39:21 +02:00
Arthur Silva Sens
24057883a1
Update otlptranslator with new API (#16626)
* Update otlptranslator with new API

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-05-29 09:24:02 -03:00
Antonio Jimenez
2834a665ed
Add support for promoting all OTel resource attributes (#16426)
Add support for promoting all OTel resource attributes via `promote_all_resource_attributes`,
except for those ignored using 'ignore_resource_attributes'.

---------

Signed-off-by: Antonio Jimenez <antonjim@thousandEyes.com>
Signed-off-by: Antonio Jimenez <123171955+antonjim-te@users.noreply.github.com>
2025-05-26 18:15:01 +02:00
bragi92
14fc57e4cf
remote_write azure auth : allow empty client_id to suppport system assigned managed identity (#16421)
* squash (#1)

* remote-write: allow empty azure client_id to support system assigned managed identity

* add blank line for tests

* remote-write: allow empty azure client_id to support system assigned managed identity

Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>

* add blank line for tests

Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>

---------

Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>

* treat empty client_id as system-assigned identity; this is a valid case

Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>

* rename file 

Signed-off-by: bragi92 <kadubey@microsoft.com>

---------

Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
2025-05-24 15:01:49 +02:00
Arthur Silva Sens
5a98246f50
Fix flakiness in TestOTLPWriteHandler (#16608) 2025-05-17 05:35:26 -03:00
Arve Knudsen
e7e3ab2824
Fix linting issues found by golangci-lint v2.0.2 (#16368)
* Fix linting issues found by golangci-lint v2.0.2

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-05-03 19:05:13 +02:00
zenador
d73b0f34e3
Strip newline in error when reading from remote client (#16487)
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2025-04-27 13:59:33 +00:00
SungJin1212
b594733a07
Fix PRW2 data diff error message typo (#16481)
Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
2025-04-25 09:27:22 +02:00
Fiona Liao
7ec63b1fa1
Add primitive support for ingesting OTLP delta metrics as-is (#16360)
* Add simple delta support

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Rename delta2cumulative part

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Whoops bad refactor

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add example yml

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Feature flag instead and histogram hint handling

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Delete otel_delta.yml - outdated

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Renaming to native delta support

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add more explanatory comments

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add more explanation to histograms

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Correct comment on d2c consumer

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add tests for counters and fix bug

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add histogram tests

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add docs

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Sort series to make test deterministic

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* More formatting

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Change flag name to ingestion

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Explain where rate calculation can go wrong

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add warning about duplicate timestamps

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Update docs/feature_flags.md

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Fix tests

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Remove unnecessary if

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Add warning to d2c section

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Make unknown type error when getting temporality

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Correct type comment - not planning to add delta metric metadata type

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Remove unused param for empty type

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Rewrite temporality logic to be clearer

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

* Change spurious to unnecessary - better description

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>

---------

Signed-off-by: Fiona Liao <fiona.liao@grafana.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-04-23 09:58:02 -03:00
Arthur Silva Sens
d9c0ad1e61
feat: Support 'NoTranslation' mode in OTLP endpoint (#16441)
* feat: Support 'NoTranslation' mode in OTLP endpoint

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-04-22 13:13:30 -03:00
Jeanette Tan
b91c66cdf9 Move newline to inside the expected error
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2025-04-16 17:58:19 +08:00
Jeanette Tan
ab2d17d7a0 Move unit test to existing suite
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2025-04-16 03:57:42 +08:00
Jeanette Tan
f1b6694837 Add unit test
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2025-04-16 03:57:42 +08:00
Jeanette Tan
88bf1b1a68 Allow unwrapping of errors when reading from remote client
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2025-04-16 03:57:42 +08:00
György Krajcsovits
1cdc956d27 test(rw2): add nhcb testcases to remote write 2.0
Ref: https://github.com/prometheus/prometheus/issues/15021

Also modified spansToSpansProto to not allocate empty bucket spans array
when converting internal model to remote write model.
Otherwise the test TestDecodeWriteV2Request fails since empty array
is marshaled/unmarshaled as nil so we don't get back the exact same
thing.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-04-08 15:45:56 +02:00
Oleg Zaytsev
c5b92f71d4
fix: storage/remote.pool interned refs count and flaky test (#16335)
* Fix storage/remote.pool interned refs count and flaky test

I saw TestIntern_MultiRef_Concurrent failing on a different PR saying 'expected refs to be 1 but it was 2'.

I took a look, and it definitely can be racy, especially with a time.Sleep() of just 1ms.

I'm fixing that by explicitly waiting until it has been released, and by repeating that 1000 times, otherwise it's just a recipe for a future flaky test.

OTOH, I also took a look at the implementation and saw that we were not holding the RLock() when increasing the references count, so when releasing there was a race condition for the cleanup, I fixed that by holding RLock() while increasing the references count.

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

* s/Equalf/Equal/

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>

---------

Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
2025-03-27 21:04:30 +01:00
Jeanette Tan
3c865c6649 Allow NewReadClient to take in more config options
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2025-03-26 00:09:15 +08:00
Matthieu MOREL
5fa1146e21
chore: enable gci linter (#16245)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2025-03-22 15:46:13 +00:00
Arve Knudsen
f5f22e7201
Remove golangci-lint exclusions for OTLP code (#16250)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-03-20 18:43:22 +01:00
Carrie Edwards
c90b387d89
Optionally translate OTel histograms to NHCB (#15850)
* Optionally translate OTEL histograms to NHCB

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add tests for explicit histogram to NHCB translation

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Linting

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add test for translation with flag

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Refactor to re-use bucket conversion function

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Update prompb

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add comment explaining explicit bounds and use defined CustomBucketSchema

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add TODO for limiting max bucket count

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Fix imports and remove extra blank line

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Update configuration docs and CHANGELOG

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Update CHANGELOG

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Use otlptranslator package

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

---------

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-03-20 09:18:51 -07:00
Arthur Silva Sens
b39672736a
Migrate otlptranslator package to prometheus/otlptranslator (#16240)
* Migrate otlptranslator package to prometheus/otlp-prometheus-translator

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-03-20 16:13:26 +01:00
Patryk Prus
ebeeb1c823
Fix flaky TestReadClient/timeout test
Signed-off-by: Patryk Prus <p@trykpr.us>
2025-03-18 13:35:40 -04:00