bwplotka
adc4150597
om2: add om2 complex type format for PoC
...
This change is for demo purposes, exploring the benefits (and downsides)
for the complex type format for OM2 captured in
https://github.com/prometheus/docs/pull/2679 .
This assumes Prometheus stores NS and NHCB (and NH) going forward (for
best case efficiency), but is expected to work for classic mode too with
little overhead (benchmarks will tell us).
Part of the PromCon talk we do with @krajorama
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-10-07 12:37:22 +01:00
NamanParlecha
594f9d63a5
refactor(textparse): Introduce Variadic options in textParse.New ( #17155 )
...
* refactor(textparse): introduce ParserOptions struct for cleaner parser initialization
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(fuzz): update fuzzParseMetricWithContentType to use ParserOptions
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(parser): simplify ParserOptions usage in tests and implementations
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(parse): using variadic options
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(parser): add fallbackType & SymbolTable to variadic options
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(parser): private fields
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(scrape): compose parser options
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(parser): add comments
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(parser): update to use ParserOptions struct for configuration
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(scrape): remove unused parserOptions field from scrapeLoop
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
* refactor(parser): update ParserOptions field names and add comments for clarity
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
---------
Signed-off-by: Naman-B-Parlecha <namanparlecha@gmail.com>
2025-09-11 10:49:42 +01:00
George Krajcsovits
31e4d84edd
refactor(textparse): allow for parsers with direct NHCB support ( #17153 )
...
Hide adding NHCB parser on top another parser in New() function
so we can easily add direct NHCB capable parsers.
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2025-09-06 11:45:44 +02:00
Bartlomiej Plotka
8e6b008608
feature: type-and-unit-labels (PROM-39 implementation) ( #16228 )
...
* feature: type-and-unit-labels (extended MetricIdentity)
Experimental implementation of https://github.com/prometheus/proposals/pull/39
Previous (unmerged) experiments:
* https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels
* https://github.com/prometheus/prometheus/pull/16025
Signed-off-by: bwplotka <bwplotka@gmail.com>
feature: type-and-unit-labels (extended MetricIdentity)
Experimental implementation of https://github.com/prometheus/proposals/pull/39
Previous (unmerged) experiments:
* https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels
* https://github.com/prometheus/prometheus/pull/16025
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Fix compilation errors
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Lint
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Revert change made to protobuf 'Accept' header
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Fix compilation errors for 'dedupelabels' tag
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
* Rectored into schema.Metadata
Signed-off-by: bwplotka <bwplotka@gmail.com>
* texparse: Added tests for PromParse
Signed-off-by: bwplotka <bwplotka@gmail.com>
* add OM tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* add proto tests
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* add schema label tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* fix tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* add promql tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* lint
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Addressed comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
---------
Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
2025-05-17 09:37:25 +00:00
Bartlomiej Plotka
dc85d677d8
textparse: Optimize CreatedTimestamp; It returns int64 value now. ( #16072 )
...
..instead of *int64. This is as an optimization and ease of use. We already
accepted in many places (proto histograms, PRW) that CT (or any timestamp really) 0
means not set.
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-03-07 12:43:13 +00:00
Bartlomiej Plotka
733a5e9eb4
textparse: Optimized protobuf parser with custom streaming unmarshal. ( #15731 )
...
* textparse: Optimized protobuf parser with custom streaming decoder.
Signed-off-by: bwplotka <bwplotka@gmail.com>
Update model/textparse/protobufparse.go
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Addressing comments.
Signed-off-by: bwplotka <bwplotka@gmail.com>
decoder: reuse histograms and summaries.
Signed-off-by: bwplotka <bwplotka@gmail.com>
optimize help returning (5% of mem utilization).
Signed-off-by: bwplotka <bwplotka@gmail.com>
Apply suggestions from code review
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Update prompb/io/prometheus/client/decoder.go
Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Fix build.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Update model/textparse/protobufparse.go
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
---------
Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
2025-02-13 10:38:35 +00:00
Bartlomiej Plotka
00b69efabb
model/textparse: Change parser interface Metric(...) string to Labels(...) ( #16012 )
...
* model/textparse: Change parser interface Metric(...) string to Labels(...)
Simplified the interface given no one is using the return argument.
Renamed for clarity too.
Found and discussed https://github.com/prometheus/prometheus/pull/15731#discussion_r1950916842
Signed-off-by: bwplotka <bwplotka@gmail.com>
* Fixed comments; optimized not needed copy for om and text.
Signed-off-by: bwplotka <bwplotka@gmail.com>
---------
Signed-off-by: bwplotka <bwplotka@gmail.com>
2025-02-12 15:47:56 +00:00
George Krajcsovits
2182b83271
feat(nhcb): implement created timestamp handling ( #15198 )
...
Call through to the underlaying parser if we are not in a histogram
and the entry is a series or exponential native histogram. Otherwise store
and retrieve CT for NHCB.
* fix(omparser): losing exemplars when CT is parsed
Fixes : #15137
Ignore exemplars while peeking ahead during CT parsing.
Simplify state reset with defer().
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
2024-10-24 07:38:58 +02:00
Alex Greenbank
421a3c22ea
scrape: provide a fallback format ( #15136 )
...
scrape: Remove implicit fallback to the Prometheus text format
Remove implicit fallback to the Prometheus text format in case of invalid/missing Content-Type and fail the scrape instead. Add ability to specify a `fallback_scrape_protocol` in the scrape config.
---------
Signed-off-by: alexgreenbank <alex.greenbank@grafana.com>
Signed-off-by: Alex Greenbank <alex.greenbank@grafana.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2024-10-18 17:12:31 +02:00
Bartlomiej Plotka
f6e110d588
textparse: Refactored main testing utils for reusability; fixed proto Units. ( #15095 )
...
Signed-off-by: bwplotka <bwplotka@gmail.com>
2024-10-07 12:17:44 +01:00
Matthieu MOREL
ab64966e9d
fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()" ( #15094 )
...
* fix: use "ErrorContains" or "EqualError" instead of "Contains(t, err.Error()" and "Equal(t, err.Error()"
---------
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2024-10-06 16:35:29 +00:00
Manik Rana
f1c57a95ed
change: No longer ingest OM _created as timeseries if feature-flag 'enable-ct-zero-ingestion' is enabled; fixed OM text CT conversion bug ( #14738 )
...
* chore: revert TypeRequiresCT to private
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* feat: init NewOpenMetricsParser with skipCT true
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: allow opt-in to OM CT ingestion
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use textparse interface to set om options
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: set skipOMSeries in test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* wip: add tests for OM CR parse
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: merge ct tests
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: add cases for OM text
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: check correct test cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: use both scrape protocols in config
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: fix inputs and output tests for OM
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: cleanup
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: rename skipOMSeries to skipOMCTSeries
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
* fix: finish refac
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* refac: move setup code outside test
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* tests: verify _created lines create new metric in certain cases
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* fix: post merge fixes
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* manager: Fixed CT OMText conversion bug; Refactored tests.
Signed-off-by: bwplotka <bwplotka@gmail.com>
* chore: lint
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: gofumpt
Signed-off-by: Manik Rana <manikrana54@gmail.com>
* chore: imports
Signed-off-by: Manik Rana <manikrana54@gmail.com>
---------
Signed-off-by: Manik Rana <manikrana54@gmail.com>
Signed-off-by: Manik Rana <Manikrana54@gmail.com>
Signed-off-by: bwplotka <bwplotka@gmail.com>
Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: bwplotka <bwplotka@gmail.com>
2024-10-02 11:52:03 +01:00
Bryan Boreham
eff3a13e19
model/textparse: parsers take a labels SymbolTable
...
This allows strings to be interned to save memory.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
2024-02-26 11:45:25 +00:00
beorn7
9e500345f3
textparse/scrape: Add option to scrape both classic and native histograms
...
So far, if a target exposes a histogram with both classic and native
buckets, a native-histogram enabled Prometheus would ignore the
classic buckets. With the new scrape config option
`scrape_classic_histograms` set, both buckets will be ingested,
creating all the series of a classic histogram in parallel to the
native histogram series. For example, a histogram `foo` would create a
native histogram series `foo` and classic series called `foo_sum`,
`foo_count`, and `foo_bucket`.
This feature can be used in a migration strategy from classic to
native histograms, where it is desired to have a transition period
during which both native and classic histograms are present.
Note that two bugs in classic histogram parsing were found and fixed
as a byproduct of testing the new feature:
1. Series created from classic _gauge_ histograms didn't get the
_sum/_count/_bucket prefix set.
2. Values of classic _float_ histograms weren't parsed properly.
Signed-off-by: beorn7 <beorn@grafana.com>
2023-05-13 01:32:25 +02:00
Julien Pivotto
f695df843f
Improve content-type error handling
...
- Call err everywhere
- Change log message to underscore-separated field
Followup on #10186
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
2022-02-08 11:02:51 +01:00
Matheus Pimenta
8d8ce641a4
error for invalid media type should not be completely swallowed ( #10186 )
...
* error for invalid media type should not be completely swallowed
Signed-off-by: Matheus Pimenta <matheuscscp@gmail.com>
2022-02-08 10:57:56 +01:00
