Commit Graph

325 Commits

Author SHA1 Message Date
Arthur Silva Sens
4b9d0fb92f
Revert: OTLP Support including scope metadata as metric labels (#16842)
Reverts #16730 and #16760

This is being done because we've noticed a problem in the spec that could
lead to name collisions if attributes name, version or schema_url are added
to the scope. They would collide with the already reserved labels
otel_scope_name, otel_scope_version and otel_scope_schema_url.

Since this new configuration option never made it into a release, we can
safely remove it from the 3.5 release. We'll sort this out for the 3.6 release

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-07-08 10:37:19 +00:00
Owen Williams
aa1d46a9da
scrape: set validation and escaping defaults in default config vars (#16751)
Fixes https://github.com/prometheus/prometheus/issues/16750

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-06-25 11:14:13 -04:00
Arve Knudsen
c701636034
OTLP receiver: Rename otlp.convert_scope_metadata to otlp.promote_scope_metadata (#16760)
* Rename otlp.convert_scope_metadata to otlp.promote_scope_metadata

---------

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Julius Hinze <juliusmh@proton.me>
2025-06-20 13:09:02 +00:00
Arve Knudsen
964bd7d1a9
OTLP: Support including scope metadata as metric labels (#16730)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2025-06-18 09:13:51 +02:00
Owen Williams
5a1cce4fbb
config: fix typo in comments (#16734)
Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-06-17 08:24:30 -04:00
Jan-Otto Kröpke
6d10fce2b6
Merge pull request #16699 from prometheus/release-3.4 2025-06-06 17:04:53 +02:00
Arthur Silva Sens
ed0de41b73
bugfix: Default OTLP config not respected when otlp: block is unset (#16693)
Co-authored-by: Jan-Otto Kröpke <mail@jkroepke.de>
2025-06-04 22:52:39 +02:00
Antonio Jimenez
2834a665ed
Add support for promoting all OTel resource attributes (#16426)
Add support for promoting all OTel resource attributes via `promote_all_resource_attributes`,
except for those ignored using 'ignore_resource_attributes'.

---------

Signed-off-by: Antonio Jimenez <antonjim@thousandEyes.com>
Signed-off-by: Antonio Jimenez <123171955+antonjim-te@users.noreply.github.com>
2025-05-26 18:15:01 +02:00
machine424
690c9da817 chore(config): add guidelines for adding a new RuntimeConfig field based on learnings from GoGC addition
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
2025-05-21 16:16:05 +02:00
Will Hegedus
33578fedb3 fix(config): respect GOGC environment variable if no "runtime" block exists
Fixes: https://github.com/prometheus/prometheus/issues/16334
Related to:
- https://github.com/prometheus/prometheus/pull/15238
- https://github.com/prometheus/prometheus/pull/16052

Currently, when the GOGC environment variable is set -- and no `runtime`
block is set in the Prometheus config file -- it is ignored and the
default value of 75% is always used.

However, if there is an empty runtime block (e.g. `runtime: {}`), _then_
the GOGC environment variable is checked.

This PR changes this behavior to consistently check and use the GOGC
environment variable when it is set (unless the `gogc` field is set in
the `runtime` block of the loaded config file, in which case it still
gives that precedence).

Co-authored-by: Adam Rambo <arambo@protonmail.com>
Signed-off-by: Will Hegedus <whegedus@akamai.com>
2025-05-12 07:37:20 -04:00
hardlydearly
ba4b058b7a refactor: use slices.Contains to simplify code
Signed-off-by: hardlydearly <799511800@qq.com>
2025-05-09 08:27:10 +02:00
George Krajcsovits
2726a18ae3
Merge pull request #16452 from chardch/global-always-scrape-classic-hist
Add global config option for always_scrape_classic_histograms
2025-05-06 09:29:30 +02:00
chardch
a1c157aaef Add global config option for always_scrape_classic_histograms
Addresses https://github.com/prometheus/prometheus/issues/16371
This will help with migrating to native histograms with `convert_classic_histograms_to_nhcb` since users may still need to keep the classic histograms during a migration

Signed-off-by: chardch <otwordsne@gmail.com>
2025-05-05 10:14:04 -07:00
Arthur Silva Sens
d9c0ad1e61
feat: Support 'NoTranslation' mode in OTLP endpoint (#16441)
* feat: Support 'NoTranslation' mode in OTLP endpoint

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-04-22 13:13:30 -03:00
chardch
357b5ed640 Fix small issues and clarify documentation
Signed-off-by: chardch <otwordsne@gmail.com>
2025-03-29 08:35:11 -07:00
chardch
2f59d38054 Add global config option for convert_classic_histograms_to_nhcb
Addresses the global part of https://github.com/prometheus/prometheus/issues/13532

Signed-off-by: chardch <otwordsne@gmail.com>
2025-03-27 10:55:27 -07:00
Owen Williams
6566c5a2b3
scrape: Add config option for escaping scheme request. (#16066)
The new metric_name_escaping_scheme config option works in parallel with metric_name_validation_scheme and controls which escaping scheme is requested when scraping. When not specified, the scheme will request underscores if the validation scheme is set to legacy, and will request allow-utf-8 when the validation scheme is set to utf8. This setting allows users to allow utf8 names if they like, but explicitly request an escaping scheme rather than UTF-8.

Fixes https://github.com/prometheus/prometheus/issues/16034

Built on https://github.com/prometheus/prometheus/pull/16080

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-03-26 18:27:28 -04:00
Carrie Edwards
c90b387d89
Optionally translate OTel histograms to NHCB (#15850)
* Optionally translate OTEL histograms to NHCB

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add tests for explicit histogram to NHCB translation

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Linting

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add test for translation with flag

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Refactor to re-use bucket conversion function

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Update prompb

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add comment explaining explicit bounds and use defined CustomBucketSchema

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Add TODO for limiting max bucket count

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Fix imports and remove extra blank line

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Update configuration docs and CHANGELOG

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Update CHANGELOG

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

* Use otlptranslator package

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>

---------

Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com>
2025-03-20 09:18:51 -07:00
Arthur Silva Sens
95f49dd84b
Bump prometheus/common to v0.63.0 (#16210)
* Bump prometheus/common to v0.63.0

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

* nolint usage of deprecated model.NameValidationScheme

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-03-13 20:42:42 +01:00
Owen Williams
94b43c5d4c utf8: Remove support for legacy global validation setting
Global and Data Source configurations can specify legacy mode, but Prometheus now requires that the overall validation mode be set to UTF-8

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2025-03-13 10:47:24 -04:00
Danial Eskandari
5efaa84be1
config: set gogc default value when config body is empty (#16052)
* fix: set gogc default value when config body is empty

Signed-off-by: Danial Eskandari <doneskandari@gmail.com>

* refactor: explicitly check value 75 in `TestGoGCDefaultValueOnEmptyConfigBody`

add GoGC assertion in `TestEmptyConfig`, also removed the no longer needed runtime config assignment in `TestEmptyGlobalBlock`

Signed-off-by: Danial Eskandari <doneskandari@gmail.com>

* refactor: remove `TestGoGCDefaultValueOnEmptyConfigBody` to reduce duplicate assertions

Signed-off-by: Danial Eskandari <doneskandari@gmail.com>

---------

Signed-off-by: Danial Eskandari <doneskandari@gmail.com>
2025-02-27 11:25:52 +01:00
Matt Hughes
5868e36d91 scraper: fix UTF-8 scraping header always sent with PrometheusText1.0.0
The `Accept` header should not include `escape=allow-utf-8` unless
explicitly requested.

Conveniently, there was already a test covering this header's value, it
just required updating so it also asserts that this value in the header
is not set in the cases we don't expect it to be set. I also converted
those tests into table tests to help make failures clearer.

Issue: https://github.com/prometheus/prometheus/issues/15857
Signed-off-by: Matt Hughes <mhughes@uw.co.uk>
2025-02-20 11:33:16 +00:00
Arve Knudsen
beca06c629
Clarify the NoUTF8EscapingWithSuffixes and UnderscoreEscapingWithSuffixes modes (#15567)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-12-17 08:40:53 +01:00
bwplotka
134a0c7d78 Improved error.
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-12-12 10:57:27 +00:00
Bartlomiej Plotka
aa9fef75ac
Apply suggestions from code review
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-12-12 10:06:16 +00:00
bwplotka
f44bba31b3 config: Remove validation GetScrapeConfigs; require using config.Load.
We should never modify (or even shallow copy) Config after config.Load;
added comments and modified GetScrapeConfigs to do so. For GetScrapeConfigs
the validation (even repeated) was likely doing writes (because global fields was 0). We GetScrapeConfigs concurrently
in tests and ApplyConfig causing test races. In prod there were
races but likelyt only to replace 0 with 0, so not too severe.

I removed validation since I don't see anyone using our config.Config without Load.
I had to refactor one test that was doing it, all others use yaml config.

Fixes #15538
Previous attempt: https://github.com/prometheus/prometheus/pull/15634

Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-12-11 13:31:01 +00:00
György Krajcsovits
174f828d70 Rename to keep_identifying_resource_attributes
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-11-29 15:19:14 +01:00
György Krajcsovits
ffabc04829 Make the behavior configurable
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-11-29 15:12:57 +01:00
Đurica Yuri Nikolić
101b1c307f
[ENHANCEMEN] Remote-Write: optionally use a DNS resolver that picks a random IP (#15329)
When a remote-write is executed towards a host name that is resolved to multiple IP addresses, this PR introduces a possibility to force creation of new connections used for the remote-write request to a randomly chosen IP address from the ones corresponding to the host name. The default behavior remains unchanged, i.s., the IP address used for the connection creation remains the one chosen by Go.

This is an experimental feature, it is disabled by default.

Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>
2024-11-15 15:41:49 +00:00
SuperQ
bc1bc5c118
Update sigv4 library
Migrate use of prometheus/common/sigv4 to prometheus/sigv4.

Related to: https://github.com/prometheus/common/issues/709

Signed-off-by: SuperQ <superq@gmail.com>
2024-11-08 22:35:44 +01:00
Arthur Silva Sens
03995775a7
Allow UTF-8 characters in metric and label names as opt-in feature (#15258)
* Allow UTF-8 characters in metric and label names as opt-in feature

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
2024-11-08 10:02:48 -03:00
Matthieu MOREL
af1a19fc78 enable errorf rule from perfsprint linter
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-11-06 16:50:36 +01:00
Bryan Boreham
20fdc8f541 [CHANGE] Remote-write: default enable_http2 to false
Remote-write creates several shards to parallelise sending, each with
its own http connection. We do not want them all combined onto one
socket by http2.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-10-24 22:27:06 +02:00
György Krajcsovits
4283ae73dc Rename convert_classic_histograms to convert_classic_histograms_to_nhcb
On reviewer request.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-21 13:22:58 +02:00
György Krajcsovits
8c1b5a6251 Merge branch 'main' into nhcb-scrape-impl 2024-10-21 11:00:41 +02:00
Alex Greenbank
421a3c22ea
scrape: provide a fallback format (#15136)
scrape: Remove implicit fallback to the Prometheus text format

Remove implicit fallback to the Prometheus text format in case of invalid/missing Content-Type and fail the scrape instead. Add ability to specify a `fallback_scrape_protocol` in the scrape config.

---------

Signed-off-by: alexgreenbank <alex.greenbank@grafana.com>
Signed-off-by: Alex Greenbank <alex.greenbank@grafana.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2024-10-18 17:12:31 +02:00
Alan Protasio
c78d5b94af
Disallowing configure AM with the v1 api (#13883)
* Stop supporting Alertmanager v1

* Disallowing configure AM with the v1 api

Signed-off-by: alanprot <alanprot@gmail.com>

* Update config/config_test.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update config/config.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Addressing coments

Signed-off-by: alanprot <alanprot@gmail.com>

* Update notifier/notifier.go

Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

* Update config/config_test.go

Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>

---------

Signed-off-by: alanprot <alanprot@gmail.com>
Signed-off-by: Alan Protasio <alanprot@gmail.com>
Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com>
Co-authored-by: Jan Fajerski <jan--f@users.noreply.github.com>
2024-10-18 15:23:14 +02:00
Bartlomiej Plotka
efc43d0714
s/scrape_classic_histograms/always_scrape_classic_histograms (3.0 breaking change) (#15178)
This is for readability, especially when we can converting to nhcb option.

See discussion https://cloud-native.slack.com/archives/C077Z4V13AM/p1729155873397889

Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-18 08:32:15 +01:00
György Krajcsovits
04b827dd77 Merge branch 'main' into nhcb-scrape-impl 2024-10-17 12:34:44 +02:00
Yi
2cabd1b707
config: remove expand-external-labels flag in release 3.0 (#14657)
remove expand-external-labels feature flag

and enabled env arg expansion for external labels by default.

Signed-off-by: jyz0309 <45495947@qq.com>
2024-10-17 10:25:05 +02:00
George Krajcsovits
a1700aab3a
Apply suggestions from code review
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
2024-10-14 11:13:58 +02:00
György Krajcsovits
a2805e7934 Merge branch 'main' into nhcb-scrape-impl 2024-10-09 11:53:26 +02:00
TJ Hoplock
6ebfbd2d54 chore!: adopt log/slog, remove go-kit/log
For: #14355

This commit updates Prometheus to adopt stdlib's log/slog package in
favor of go-kit/log. As part of converting to use slog, several other
related changes are required to get prometheus working, including:
- removed unused logging util func `RateLimit()`
- forward ported the util/logging/Deduper logging by implementing a small custom slog.Handler that does the deduping before chaining log calls to the underlying real slog.Logger
- move some of the json file logging functionality to use prom/common package functionality
- refactored some of the new json file logging for scraping
- changes to promql.QueryLogger interface to swap out logging methods for relevant slog sugar wrappers
- updated lots of tests that used/replicated custom logging functionality, attempting to keep the logical goal of the tests consistent after the transition
- added a healthy amount of `if logger == nil { $makeLogger }` type conditional checks amongst various functions where none were provided -- old code that used the go-kit/log.Logger interface had several places where there were nil references when trying to use functions like `With()` to add keyvals on the new *slog.Logger type

Signed-off-by: TJ Hoplock <t.hoplock@gmail.com>
2024-10-07 15:58:50 -04:00
György Krajcsovits
71fd2d93a9 Merge branch 'main' into nhcb-scrape-impl
# Conflicts:
#	config/config.go
#	scrape/scrape.go
2024-09-25 13:43:57 +02:00
Jeanette Tan
62e7f0438d implement basic conversion of classic to nhcb in scrape
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
2024-09-25 13:36:56 +02:00
Jan Fajerski
fa318711f4 Merge branch 'main' into 3.0-main-sync-24-09-09
Conflicts:
	cmd/prometheus/main.go
	docs/command-line/prometheus.md
	docs/feature_flags.md
	web/ui/build_ui.sh
	web/web.go
    Resolved by dropping the UTF-8 feature flag and adding the
    `auto-reload-config` feature flag.
    For the new web ui pick all changes from `main`.
2024-09-09 15:44:22 +02:00
Owen Williams
88bb05c3e8 utf8: enable utf-8 support by default
This change causes Prometheus to allow all UTF-8 characters in metric and label names.
This means that names that were previously invalid and would have been previously rejected will be allowed through.

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-09-06 08:48:11 -04:00
Julien
ce0f09b125 Scrape: Add scrape_failure_log_file to log Scrape Failures
Signed-off-by: Julien <roidelapluie@o11y.eu>
2024-09-05 11:01:40 +02:00
Justin Lei
3a82cd5a7e
Add streaming remote read to ReadClient (#11379)
* Add streaming remote read to ReadClient

Signed-off-by: Justin Lei <justin.lei@grafana.com>

* Apply suggestions from code review

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Justin Lei <justin.lei@grafana.com>

* Remote read instrumentation tweaks

Signed-off-by: Justin Lei <lei.justin@gmail.com>

* Minor cleanups

Signed-off-by: Justin Lei <lei.justin@gmail.com>

* In-line handleChunkedResponse

Signed-off-by: Justin Lei <lei.justin@gmail.com>

* Fix lints

Signed-off-by: Justin Lei <lei.justin@gmail.com>

* Explicitly call cancel() when needed

Signed-off-by: Justin Lei <lei.justin@gmail.com>

* Update chunkedSeries, chunkedSeriesIterator for new interfaces

Signed-off-by: Justin Lei <lei.justin@gmail.com>

* Adapt remote.chunkedSeries to use prompb.ChunkedSeries

Signed-off-by: Justin Lei <lei.justin@gmail.com>

* Fix lint

Signed-off-by: Justin Lei <lei.justin@gmail.com>

---------

Signed-off-by: Justin Lei <justin.lei@grafana.com>
Signed-off-by: Justin Lei <lei.justin@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
2024-08-28 07:23:54 +01:00
Owen Williams
436a439ed2 fix(utf8): fix config logic for name validation
We should only overwrite the ScrapeConfig if it is empty.

Added tests

part of https://github.com/prometheus/prometheus/issues/13095

Signed-off-by: Owen Williams <owen.williams@grafana.com>
2024-08-23 15:16:48 -04:00