Fabian Reinartz
5aa7f7cce8
Compact head block into persisted block
2017-01-04 21:11:15 +01:00
Fabian Reinartz
3f72d5d027
Fix last timestamp initialization
...
This initializes the chunkDesc's last timestamp to the minimum
value so initial samples with a timestamp of 0 (e.g. in tests)
are not accidentally dropped.
2017-01-04 14:06:40 +01:00
Fabian Reinartz
ec99f99d3d
Fix and test bug in shardSeriesSet
2017-01-03 19:02:42 +01:00
Fabian Reinartz
ac49f8c15e
Consolidate persistence and compaction
2017-01-03 16:49:37 +01:00
Fabian Reinartz
91b65b55e7
Run persistence in separate goroutine
2017-01-02 22:24:35 +01:00
Fabian Reinartz
a648ef5252
Convert persister into function
2017-01-02 16:58:47 +01:00
Fabian Reinartz
dbd2b21d2e
Make persistence atomic
2017-01-02 14:41:13 +01:00
Fabian Reinartz
62b8ded0a5
Rename iterator value getters to At()
2017-01-02 13:33:06 +01:00
Fabian Reinartz
5d75a3dc7b
Add basic compaction
...
This adds a basic compactor that will merge two persisted blocks into
one. It simply fully rewrites the index and concatenates the chunk
lists.
It just writes into the current working dir and doesn't properly handle
which blocks to compact for now.
2017-01-02 10:34:55 +01:00
Fabian Reinartz
c00d17e691
Modify IndexReader API to accomodate compaction
...
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.
A LabelIndices method is added to query for existing label
value indices.
2016-12-31 16:24:04 +01:00
Fabian Reinartz
1e1a37b15b
Remove double-reference in chunk hashmap
2016-12-31 10:19:02 +01:00
Fabian Reinartz
675f0886f0
Append to chunks cannot error
2016-12-31 10:10:27 +01:00
Fabian Reinartz
7280533c42
Add basic shard metrics
2016-12-31 09:48:49 +01:00
Fabian Reinartz
a009247ab7
Add matching of empty label
2016-12-30 19:36:28 +01:00
Fabian Reinartz
eb4f366988
Add return error to Appender.Add
2016-12-29 11:03:39 +01:00
Fabian Reinartz
3a5468f251
Lock shards during querying and shutdown
2016-12-28 11:41:44 +01:00
Fabian Reinartz
ab7fbc05ad
Zero timestamp as base, use binary search list postings
2016-12-28 08:50:20 +01:00
Fabian Reinartz
f8111cef0e
Fix chunk series iterator seeking
2016-12-26 16:55:44 +01:00
Fabian Reinartz
787199a88e
Fix erroneous value assignments
2016-12-22 20:57:00 +01:00
Fabian Reinartz
8aba95048a
Rename to OpenHeadBlock
2016-12-22 20:00:24 +01:00
Fabian Reinartz
9c6a72aadd
Load head with WALs correctly
2016-12-22 15:54:39 +01:00
Fabian Reinartz
1dde3b6d31
Add WAL decoder+loading and benchmarks
2016-12-22 15:18:33 +01:00
Fabian Reinartz
0b8c77361e
Add initial WAL writing
2016-12-22 12:05:24 +01:00
Fabian Reinartz
2a825f6c28
Consolidate mem index into HeadBlock
2016-12-22 01:12:28 +01:00
Fabian Reinartz
da2beb3e6d
Fix zero division, add buffer series iterator
2016-12-21 13:04:51 +01:00
Fabian Reinartz
ede733ab6c
Extract labels package
2016-12-21 09:39:01 +01:00
Fabian Reinartz
ee217adc7e
Redfine append interface, remove old Prometheus storage from bench
2016-12-21 00:02:37 +01:00
Fabian Reinartz
d9ca4b47f5
Fix offset errors, fix persisted postings order
2016-12-20 13:14:55 +01:00
Fabian Reinartz
1b23d62e3f
Properly close files before reopening
2016-12-19 22:37:03 +01:00
Fabian Reinartz
00a503129b
Use contextualized and traced errors in reader
2016-12-19 22:29:49 +01:00
Fabian Reinartz
bad93d8d57
Extract head serialization into Head method
2016-12-18 14:43:27 +01:00
Fabian Reinartz
b08f82fa4e
Pre-select relevant chunks on series access.
...
This adds interval metadata to indexed chunks. The queried interval
is used to filter chunks when queried from the index to save
unnecessary accesses of the chunks file.
This is especially relevant for series that come and go often and larger
files.
2016-12-16 12:13:17 +01:00
Fabian Reinartz
bd77103a49
Add stats serialization, load querier of all blocks
2016-12-15 16:14:33 +01:00
Fabian Reinartz
b2f1db5666
Add unsafe string and slice conversions
2016-12-15 11:56:41 +01:00
Fabian Reinartz
5424a0cf75
Rename SeriesShard to Shard
2016-12-15 08:36:09 +01:00
Fabian Reinartz
9873e18b75
Add loading of persisted blocks
2016-12-15 08:36:07 +01:00
Fabian Reinartz
ca89080128
Misc fixes for initial Prometheus integration
2016-12-14 18:38:46 +01:00
Fabian Reinartz
fc992fafc2
Change querier interface, initial implementations
2016-12-14 15:39:23 +01:00
Fabian Reinartz
6eeb0ef01c
Add queriers and partial mocks
2016-12-13 15:26:58 +01:00
Fabian Reinartz
5e02e28f9c
Add proper mmap calls
2016-12-11 15:49:36 +01:00
Fabian Reinartz
14dbc59f2b
cleanup and switching removal of unsafe calls.
2016-12-10 18:09:57 +01:00
Fabian Reinartz
3a528c3078
Write plain postings list index
2016-12-10 09:44:00 +01:00
Fabian Reinartz
8cbc95c316
Write label value indices
2016-12-09 21:40:38 +01:00
Fabian Reinartz
1e0edf367b
Write index with symbol table
2016-12-09 21:23:34 +01:00
Fabian Reinartz
40a451694f
Refactor persistence into interfaces
2016-12-09 20:45:46 +01:00
Fabian Reinartz
62f9dc311c
misc
2016-12-09 16:54:38 +01:00
Fabian Reinartz
74f8dfd95d
Persist blocks periodically
2016-12-09 13:41:38 +01:00
Fabian Reinartz
0cf8bb9e53
Move sub-indexes into single index structure
2016-12-09 10:41:51 +01:00
Fabian Reinartz
8aa99a3ebd
misc
2016-12-09 10:00:14 +01:00
Fabian Reinartz
2c34a15fe6
Add initial seriailization of block data
2016-12-08 17:43:10 +01:00
Fabian Reinartz
3ef7da33c8
Restructure files
2016-12-08 12:21:03 +01:00
Fabian Reinartz
b845f8d3a1
Reduce test data allocations
2016-12-08 11:59:54 +01:00
Fabian Reinartz
52276c6966
Bucket samples before appending.
...
This pre-sorts samples into buckets before appending them to reduce
locking of shards.
2016-12-07 17:10:49 +01:00
Fabian Reinartz
9b459458d0
Docs and interface definitions
2016-12-05 21:26:19 +01:00
Fabian Reinartz
83574b1565
Add new interfaces and skeleton
2016-12-04 13:16:11 +01:00
Fabian Reinartz
6f93a699e6
wipe
2016-12-02 17:49:05 +01:00
Fabian Reinartz
f9f11c4a08
Add main db implementation
2016-11-15 10:34:25 +01:00
