diff --git a/Makefile b/Makefile index 2a2e22e..ad4a208 100644 --- a/Makefile +++ b/Makefile @@ -1,8 +1,9 @@ -# $Id: Makefile,v 1.10 2008-02-01 10:24:38 dim Exp $ +# $Id: Makefile,v 1.11 2008-02-12 17:31:33 dim Exp $ # # Makefile for debian packaging purpose, make install not intended to work. DOCS = pgloader.1.txt +TODO = TODO.txt CVSROOT = $(shell cat CVS/Root) VERSION = $(shell ./pgloader.py --version | cut -d' ' -f3) @@ -34,6 +35,9 @@ install: html: $(DOCS) asciidoc -a toc $< +todo: $(TODO) + asciidoc -a toc $< + site: html scp ${DOCS:.txt=.html} cvs.pgfoundry.org:htdocs diff --git a/TODO.txt b/TODO.txt new file mode 100644 index 0000000..e3315c3 --- /dev/null +++ b/TODO.txt @@ -0,0 +1,130 @@ += TODO LIST = + +== Multi Threaded Import == + +* Terminate +split_file_reading+ behavior implementation (status: debug) + +* Implement /round robin reader/ behavior + +== Constraint Exclusion support == + +=== User level configuration + +At user level, you will have to add a +constraint_exclusion = on+ +parameter to pgloader section configuration for it to bother checking +if the destination table has some children etc. + +You'll need to provide also a global ce_path parameter (where to find user +python constraint exclusion modules) and a +ce_modules+ parameter for each +section where +constraint_exclusion = on+: + ce_modules = columnA:module:class, columnB:module:class + +As the +ce_path+ could point to any number of modules where a single +type is supported by several modules, I'll let the user choose which +module to use. + +=== Constraint exclusion modules + +The modules will provide one or several class(es) (kind of a packaging +issue), each one will have to register which datatypes and operators +they know about. Here's some pseudo-code of a module, which certainly +is the best way to express a code design idea: + + class MyCE: +    def __init__(self, operator, constant, cside='r'): +       """ CHECK ( col operator constant ) => cside = 'r', could be 'l' """ +       ... + +    @classmethod +    def support_type(cls, type): +       return type in ['integer', 'bigint', 'smallint', 'real', 'double'] + +    @classmethod +    def support_operator(cls, op): +        return op in ['=', '>', '<', '>=', '<=', '%'] + +    def check(self, op, data): +       if op == '>' : return self.gt(data) +       ... + +    def gt(self, data): +       if cside == 'l': +          return self.constant > data +       elif cside == 'r': +          return data > self.constant + +This way pgloader will be able to support any datatype (user datatype +like IP4R included) and operator (+@@+, +~<=+ or whatever). For +pgloader to handle a +CHECK()+ constraint, though, it'll have to be +configured to use a CE class supporting the used operators and +datatypes. + +=== PGLoader constraint exclusion support + +The +CHECK()+ constraint being a tree of check expressions[*] linked +by logical operators, pgloader will have to build some logic tree of +MyCE (user CE modules) and evaluate all the checks in order to be able +to choose the input line partition. + +[*]: +check((a % 10) = 1)+ makes an expression tree containing 2 check +nodes + +After having parsed +pg_constraint.consrc+ (not +conbin+ which seems +too much an internal dump for using it from user code) and built a +CHECK tree for each partition, pgloader will try to decide if it's +about range partitioning (most common case). + +If each partition CHECK tree is +AND((a>=b, a