Fabian Reinartz
300f4e2abf
Use separate lock for series creation
...
This uses the head block's own lock to only lock if new series were
encountered.
In the general append case we just need to hold a
2017-01-06 18:10:50 +01:00
Fabian Reinartz
63e12807da
Don't update head postings mapper on every append
2017-01-06 16:43:18 +01:00
Fabian Reinartz
71efd2e08d
Periodically fsync WAL, make head cut async
2017-01-06 15:18:06 +01:00
Fabian Reinartz
c61b310210
Naive size-based compaction
...
This adds naive compaction that tries to compact three
blocks of roughly equal size.
It decides based on samples present in a block and has no
safety measures considering the actual file size.
2017-01-06 13:53:05 +01:00
Fabian Reinartz
2eb544c98e
Change file names and maker parsing safer
2017-01-06 13:13:22 +01:00
Fabian Reinartz
96c2bd249f
Handle compaction trigger and reinitializing in DB
2017-01-06 13:03:23 +01:00
Fabian Reinartz
3ed2c2a14b
Rename Partition to regular DB, DB to PartitionedDB
2017-01-06 11:40:09 +01:00
Fabian Reinartz
937cdb579c
Switch to sequential block names
...
This changes block directory names from the int64 timestamp
to sequential numbering.
2017-01-06 10:45:03 +01:00
Fabian Reinartz
4590b61343
Rename shard to partition
2017-01-06 08:08:02 +01:00
Fabian Reinartz
9790aa98ac
Add postings wrapper that emits head postings in label set order
...
This adds a position mapper that takes series from a head block
in the order they were appended and creates a mapping representing
them in order of their label sets.
Write-repair of the postings list would cause very expensive writing.
Hence, we keep them as they are and only apply the postition mapping
at the very end, after a postings list has been sufficienctly reduced
through intersections etc.
2017-01-05 16:05:42 +01:00
Fabian Reinartz
5aa7f7cce8
Compact head block into persisted block
2017-01-04 21:11:15 +01:00
Fabian Reinartz
3f72d5d027
Fix last timestamp initialization
...
This initializes the chunkDesc's last timestamp to the minimum
value so initial samples with a timestamp of 0 (e.g. in tests)
are not accidentally dropped.
2017-01-04 14:06:40 +01:00
Fabian Reinartz
40cf215fba
Hotfix missing sorting of head block series index
2017-01-04 09:47:20 +01:00
Fabian Reinartz
ec99f99d3d
Fix and test bug in shardSeriesSet
2017-01-03 19:02:42 +01:00
Fabian Reinartz
ac49f8c15e
Consolidate persistence and compaction
2017-01-03 16:49:37 +01:00
Fabian Reinartz
e7f04d14d5
Lock mmapped files
2017-01-03 10:09:20 +01:00
Fabian Reinartz
91b65b55e7
Run persistence in separate goroutine
2017-01-02 22:24:35 +01:00
Fabian Reinartz
a648ef5252
Convert persister into function
2017-01-02 16:58:47 +01:00
Fabian Reinartz
dbd2b21d2e
Make persistence atomic
2017-01-02 14:41:13 +01:00
Fabian Reinartz
62b8ded0a5
Rename iterator value getters to At()
2017-01-02 13:33:06 +01:00
Fabian Reinartz
5c45a1cc6f
Remove shardSeriesSet caching
2017-01-02 12:05:52 +01:00
Fabian Reinartz
beb842a856
Change block interface
...
This changes the block interface to directly expose index
and series readers rather than the direct querier
2017-01-02 11:12:28 +01:00
Fabian Reinartz
5d75a3dc7b
Add basic compaction
...
This adds a basic compactor that will merge two persisted blocks into
one. It simply fully rewrites the index and concatenates the chunk
lists.
It just writes into the current working dir and doesn't properly handle
which blocks to compact for now.
2017-01-02 10:34:55 +01:00
Fabian Reinartz
5d7ec06e04
Use ChunkMeta in SeriesWriter
2017-01-01 20:01:17 +01:00
Fabian Reinartz
c00d17e691
Modify IndexReader API to accomodate compaction
...
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.
A LabelIndices method is added to query for existing label
value indices.
2016-12-31 16:24:04 +01:00
Fabian Reinartz
1e1a37b15b
Remove double-reference in chunk hashmap
2016-12-31 10:19:02 +01:00
Fabian Reinartz
675f0886f0
Append to chunks cannot error
2016-12-31 10:10:27 +01:00
Fabian Reinartz
7280533c42
Add basic shard metrics
2016-12-31 09:48:49 +01:00
Fabian Reinartz
a009247ab7
Add matching of empty label
2016-12-30 19:36:28 +01:00
Fabian Reinartz
eb4f366988
Add return error to Appender.Add
2016-12-29 11:03:39 +01:00
Fabian Reinartz
3a5468f251
Lock shards during querying and shutdown
2016-12-28 11:41:44 +01:00
Fabian Reinartz
286293802b
Add merge postings
2016-12-28 11:02:19 +01:00
Fabian Reinartz
ab7fbc05ad
Zero timestamp as base, use binary search list postings
2016-12-28 08:50:20 +01:00
Fabian Reinartz
dd72b52098
Add postings intersection
2016-12-27 11:32:10 +01:00
Fabian Reinartz
f8111cef0e
Fix chunk series iterator seeking
2016-12-26 16:55:44 +01:00
Fabian Reinartz
201d7687b6
Fix uniqueness of new series
2016-12-26 16:55:32 +01:00
Fabian Reinartz
5efe1d178e
labels: add Compare and String methods
2016-12-24 13:49:35 +01:00
Fabian Reinartz
0d0c5cfaf1
labels: add string constructor, expose matcher
2016-12-24 10:19:46 +01:00
Fabian Reinartz
787199a88e
Fix erroneous value assignments
2016-12-22 20:57:00 +01:00
Fabian Reinartz
8aba95048a
Rename to OpenHeadBlock
2016-12-22 20:00:24 +01:00
Fabian Reinartz
cabd7e4ebc
Add small testdata file
2016-12-22 19:15:37 +01:00
Fabian Reinartz
e7edae39b2
Use page buffered writer for WAL
2016-12-22 16:14:34 +01:00
Fabian Reinartz
9c6a72aadd
Load head with WALs correctly
2016-12-22 15:54:39 +01:00
Fabian Reinartz
1dde3b6d31
Add WAL decoder+loading and benchmarks
2016-12-22 15:18:33 +01:00
Fabian Reinartz
0b8c77361e
Add initial WAL writing
2016-12-22 12:05:24 +01:00
Fabian Reinartz
2a825f6c28
Consolidate mem index into HeadBlock
2016-12-22 01:12:28 +01:00
Fabian Reinartz
869cccf080
Test and fixes for buffered iterator
2016-12-21 16:06:33 +01:00
Fabian Reinartz
0a94f58f1a
Fix test import of labels, simplify constructor names
2016-12-21 15:12:26 +01:00
Fabian Reinartz
da2beb3e6d
Fix zero division, add buffer series iterator
2016-12-21 13:04:51 +01:00
Fabian Reinartz
dbca3453fb
Add label clone benchmark
2016-12-21 10:37:38 +01:00
