prometheus/tsdb/chunkenc
George Krajcsovits 3f59fe1a80
fix(chunkenc): appending histograms with empty buckets (#16893)
* test(chunkenc): appending histograms with empty buckets and gaps

Append such native histograms that have empty buckets and gaps
between the indexes of those buckets.

There is a special case for appending counter native histograms to a chunk in TSDB: if we append a histogram that is missing some buckets that are already in chunk, then usually that's a counter reset. However if the missing bucket is empty, meaning its value is 0, then we don't consider it missing.

For this case to trigger , we need to write empty buckets into the chunk. Normally native histograms are compacted when we emit them , so this is very rare and compact make sure that there are no multiple continuous empty buckets with gaps between them.

The code that I've added in #14513 did not take into account that you can bypass compact and write histograms with many empty buckets, with gaps between them. These are still valid, so the code has to account for them.

Main fix in the expandIntSpansAndBuckets and expandFloatSpansAndBuckets function. I've also refactored them for clarity. Consequently needed to fix insert and adjustForInserts to also allow gaps between inserts.

I've added some new test cases (data driven test would be nice here, too many cases). And removed the deprecated old function that didn't deal with empty buckets at all.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
Co-authored-by: Björn Rabenstein <beorn@grafana.com>
2025-07-24 18:01:02 +02:00
..
bstream_test.go tsdb/chunkenc.Pool: Refactor Get and Put 2024-05-08 13:37:25 +02:00
bstream.go [PERF] Chunk encoding: simplify writeByte 2024-09-18 13:19:04 +01:00
chunk_test.go enable errorf rule from perfsprint linter 2024-11-06 16:50:36 +01:00
chunk.go Fix: chunkenc.MockSeriesIterator 2024-08-30 16:44:36 +02:00
float_histogram_test.go fix(chunkenc): appending histograms with empty buckets (#16893) 2025-07-24 18:01:02 +02:00
float_histogram.go fix(chunkenc): appending histograms with empty buckets (#16893) 2025-07-24 18:01:02 +02:00
histogram_meta_test.go fix(chunkenc): appending histograms with empty buckets (#16893) 2025-07-24 18:01:02 +02:00
histogram_meta.go fix(chunkenc): appending histograms with empty buckets (#16893) 2025-07-24 18:01:02 +02:00
histogram_test.go fix(chunkenc): appending histograms with empty buckets (#16893) 2025-07-24 18:01:02 +02:00
histogram.go fix(chunkenc): appending histograms with empty buckets (#16893) 2025-07-24 18:01:02 +02:00
varbit_test.go Use more varbit in histogram chunks 2021-10-13 20:03:35 +02:00
varbit.go Fix language in docs and comments (#14041) 2024-05-08 17:57:09 +02:00
xor_test.go chunkenc: Slightly optimize xorWrite/xoRead (#11476) 2022-10-20 15:08:01 +05:30
xor.go [PERF] Chunk encoding: combine timestamp writes 2024-09-18 13:19:21 +01:00