mirrors/prometheus
1
0
Fork 0
mirror of https://github.com/prometheus/prometheus.git synced 2025-08-28 18:11:04 +02:00
Code Issues Projects Releases Wiki Activity
882 Commits 294 Branches 481 Tags
Commit Graph

119 Commits

Author SHA1 Message Date
Fabian Reinartz
96d7f540d4 Persist series without allocating the full set 
Change index persistence for series to not be accumulated in memory
before being written as one large batch. `Labels` and `ChunkMeta`
objects are reused.
This cuts down memory spikes during compaction of multiple blocks
significantly.

As part of the the Index{Reader,Writer} now have an explicit notion of
symbols and series must be inserted in order.
 2017-08-06 12:06:41 +02:00
Fabian Reinartz
3951d8cc29 Fix selection of compactable blocks 2017-08-04 12:02:34 +02:00
Fabian Reinartz
5d7b5994d6 Fix compaction range selection 2017-07-13 16:13:59 +02:00
Goutham Veeramachaneni
401bdfd4a6 Make Chunks return the number of samples inside 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 18:45:12 +02:00
Goutham Veeramachaneni
1263a68875 Recompact blocks with large number of tombstones 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 18:27:31 +02:00
Goutham Veeramachaneni
9a2ab732b5 Choose ranges by retention and min-block-duration 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-12 17:30:34 +02:00
Goutham Veeramachaneni
ce23adb579 Initial compaction implementation for block-ranges 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-07-07 16:32:18 +02:00
Fabian Reinartz
03f95ee096 Fix directory file descriptor leak. 2017-06-12 00:05:04 +02:00
Fabian Reinartz
05e411a8eb Improve heuristic to spread chunks across block 2017-06-08 11:30:32 +02:00
Fabian Reinartz
f006e2d1ab Merge pull request #92 from prometheus/sources 
Add list of source head blocks to meta.json
 2017-06-07 11:50:44 +02:00
Fabian Reinartz
683b4c8c2a Add list of source head blocks to meta.json 2017-06-07 11:06:45 +02:00
Goutham Veeramachaneni
a1c8425357
Initial implementation of HeadBlock Snapshots 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-06-05 13:48:31 +05:30
Goutham Veeramachaneni
3a5ae6b1a4 Merge pull request #82 from Gouthamve/deletes-1 
Implement deletions
 2017-05-27 22:06:45 +05:30
Goutham Veeramachaneni
44e9ae38b5
Incorporate PR feedback. 
* Expose Stone as it is used in an exported method.
* Move from tombstoneReader to []Stone for the same reason as above.
* Make WAL reading a little cleaner.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 21:26:31 +05:30
Fabian Reinartz
b3b9245ad6 Add and fix metrics 2017-05-26 15:13:03 +02:00
Goutham Veeramachaneni
6febabeb28
Final delete fixes. 
* Make sure no reads happen on the block when delete is in progress.
* Fix bugs in compaction.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-26 16:31:45 +05:30
Goutham Veeramachaneni
f29fb62fba
Make TombstoneReader a Getter. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-24 11:24:24 +05:30
Goutham Veeramachaneni
244b73fce1
Rename for clarity and consistency. 
Misc. changes for code cleanliness.

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 16:42:36 +05:30
Goutham Veeramachaneni
8434019ad9
Merge branch 'master' into deletes-1 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-22 12:58:38 +05:30
Goutham Veeramachaneni
0b70333ef6
Add tests for tombstones. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-21 23:20:05 +05:30
Goutham Veeramachaneni
3dbb400bef
Initial implementation of Delete on DB. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-20 00:35:50 +05:30
Fabian Reinartz
7e6961b4ac Fix import and var name 2017-05-18 17:30:52 +02:00
Fabian Reinartz
d2673d8659 Completely drop empty blocks 2017-05-18 16:12:17 +02:00
Fabian Reinartz
39df7e2bba Switch blocks to ULID directories, drop sequenc numbers 2017-05-18 16:09:30 +02:00
Goutham Veeramachaneni
22c1b5b492
Make SeriesSets use tombstones. 
Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-17 14:49:42 +05:30
Goutham Veeramachaneni
5579efbd5b
Initial implentation of Deletes on persistedBlock 
Very much a WIP

Signed-off-by: Goutham Veeramachaneni <cs14btech11014@iith.ac.in>
 2017-05-14 14:36:26 +05:30
Fabian Reinartz
778103b450 Add liecence file and headers 2017-04-10 20:59:45 +02:00
Fabian Reinartz
10c7c9acbe Adjust import names to new repository organisation 2017-04-04 11:27:26 +02:00
Fabian Reinartz
789e8224ff Fix wrong comparison in head block resorting 2017-03-21 12:12:33 +01:00
Fabian Reinartz
9c93f8f2aa Fix various races 
This fixes different race condition encoutnered when running Prometheus.
It reduces the overall performance in the synthetic benchmark a fair bit
but has no indiciations of impacting a real-world setup notably.
 2017-03-20 14:45:27 +01:00
Fabian Reinartz
3635569257 Trigger reload correctly on interrupted compaction 2017-03-20 10:41:43 +01:00
Fabian Reinartz
a8e8903350 Use ChunkMeta references for clarity 
This has been a common source of hard to debug issues. Its a premature
and unbenchmarked optimization and semantically, we want ChunkMetas to
be references in all changed cases.
 2017-03-14 15:40:16 +01:00
Fabian Reinartz
04783831e2 Correctly close compacted blocks. 2017-03-06 09:33:55 +01:00
Fabian Reinartz
21b97d1e04 Fix compaction selection procedure 2017-03-02 15:10:13 +01:00
Fabian Reinartz
327e07e8d0 Properly sync moves and removes in compactor 2017-03-02 14:35:06 +01:00
Fabian Reinartz
92120448c2 Properly cleanup compacted dirs, fixes, docs 2017-03-02 14:32:09 +01:00
Fabian Reinartz
2c3e778d90 Compactor interface 2017-03-02 09:17:30 +01:00
Fabian Reinartz
2c3b56350a Write meta.json file atomically 2017-03-01 17:19:57 +01:00
Fabian Reinartz
db5c88ea9a Misc compaction fixes 2017-02-28 08:40:51 +01:00
Fabian Reinartz
306831f151 Add per-block state ULID 2017-02-27 10:46:15 +01:00
Fabian Reinartz
9a5dfadb09 Tweak IndexWriter interface, serialize directly into dir 2017-02-25 07:24:20 +01:00
Fabian Reinartz
78780cd2ba Segment chunk file 
This adds write path support for segmented chunk data files.
Files of 512MB are pre-allocated and written to. If the file size
is exceeded, the next file is started. On completion, files
are truncated to their final size.
 2017-02-23 10:50:22 +01:00
Fabian Reinartz
a3b47c4929 Create default logger for DB 2017-02-19 16:04:37 +01:00
Fabian Reinartz
f734773214 Replace per-file locking with single PID lock file 
File locks have a multitude of problems that make them hard to use
correctly. As they are just advisory, they are only meaningful to
prevent accidents like running the same process twice.
A simple PID file lock works reliably in those cases and is simpler.
 2017-02-19 13:01:19 +01:00
Fabian Reinartz
c808928b90 SeriesWriter -> ChunkWriter 
This commit simplifies a SeriesWriter into a ChunkWriter and detaches
it entirely from the notion of a series and the series index.
 2017-02-19 11:27:31 +01:00
Fabian Reinartz
a61a31a5d7 compaction: add fast-path for compacting mem blocks 2017-01-30 09:49:36 +01:00
Fabian Reinartz
c20cc44b06 Add docs, write sequence number to meta.json 2017-01-29 08:11:47 +01:00
Fabian Reinartz
30d8866c13 Limit compaction range, make cut and compact concurrent 2017-01-20 07:58:19 +01:00
Fabian Reinartz
67d185ceb9 Compact based on compaction generation 2017-01-19 19:45:52 +01:00
Fabian Reinartz
d4779b374c Properly track and write meta file 2017-01-19 14:01:38 +01:00