Davlet Panech
82b2667ef7
Handle SystemExit separately from Exception (python <= 2.4)
2010-09-20 17:15:43 -04:00
Dimitri Fontaine
a062db184d
Avoid calling self.fd.tell() when we're reading from stdin.
...
That means the options --load-from-stdin and --boundaries are not compatible.
2010-04-29 13:20:44 +02:00
Dimitri Fontaine
cbd167c0e8
Implement --load-from-stdin
2010-04-15 16:55:10 +02:00
Dimitri Fontaine
38c8a079f2
Bugfix: MAX_PARALLEL_SECTIONS should not be None, even if not given on command line.
2010-04-07 14:11:10 +02:00
Dimitri Fontaine
411a9a53d7
Add support for PG options as arguments and in the config file, with assorted fixes
2010-04-05 21:17:22 +02:00
Dimitri Fontaine
201209259a
Implement --reject-log and --reject-data options.
2010-04-05 01:02:30 +02:00
Dimitri Fontaine
ac0a16f9b2
Add support for --max-parallel-sections and --section-threads from command line.
2010-04-05 00:32:46 +02:00
Dimitri Fontaine
b7b0bbc62d
Support for processing a filename rather than a section, and add --field-separator
2010-04-04 20:42:40 +02:00
dim
090de905b2
Implement skip_head_lines in configuration file, some more bugfixes
2009-03-10 17:56:16 +00:00
dim
848595f49b
Some more python2.3 support changes
2008-09-18 12:58:12 +00:00
dim
5a63f4f609
Protect the database connection establishing into the parralel sections semaphore, allow field_sep to be configured to \t, and some fixes
2008-05-29 09:16:13 +00:00
dim
b7c7e6a7c1
Support for fixed format datafile
2008-05-21 10:33:06 +00:00
dim
e15ac609a8
Allow user to force psycopg version to use, as psycopg2 is not always the best choice
2008-02-27 12:30:13 +00:00
dim
45176adec7
Don't wait for non started threads (PGLoader init could error out)
2008-02-25 14:11:33 +00:00
dim
a27a801f62
Synchronize threads starts and ends with an Event
2008-02-15 21:14:45 +00:00
dim
192da034c3
Handle more nicely connection errors - from python stacktrace to ERROR message
2008-02-15 09:53:06 +00:00
dim
a905ac7521
FIX check_path calls, from user bug report
2008-02-14 20:15:30 +00:00
dim
a068a5ea6a
FIX just-introduced bug: no reader init meant self.db.copy_sep = None
2008-02-14 14:57:47 +00:00
dim
e641b18fb3
Support for 'rrqueue_size' per-section configuration parameter, defaults to copy_every
2008-02-13 09:42:26 +00:00
dim
a2195ec2e7
Better handling of split_file_reading case, Still In Progress
2008-02-12 12:24:15 +00:00
dim
b534962192
Using semaphores to control how many max threads run at any given time
2008-02-12 11:27:18 +00:00
dim
fc8adf1831
First implementation of parallel loading, each thread loading a part of the input file (stat, seek, etc)
2008-02-11 17:38:52 +00:00
dim
9e17976ef8
First attempt at a multi-threaded pgloader, seems to be ok though not configurable yet
2008-02-05 13:41:45 +00:00
dim
526836deca
database object is now initialized from PGLoader init
2008-02-05 11:45:30 +00:00
dim
60ffb4ba3f
pgloader no longer has to run VACUUM after having loaded all the tables
2008-02-05 11:20:07 +00:00
dim
8605b14084
* pgloader -V now VACUUM each table separately, no more vacuumdb issued
...
* New option -D to DISABLE Triggers while loading (ENABLE them one done)
2008-02-01 10:24:39 +00:00
dim
70b3def565
New logging mecanism init fix, when nothing is configured
2007-12-10 10:35:59 +00:00
dim
a530dedd66
* Rework DatabaseError and consider dichotomy in this case too.
...
* New options lc_messages and log_file
2007-12-07 21:13:35 +00:00
dim
efecde9f5b
new logging options documentation
2007-12-07 14:37:28 +00:00
dim
5da357c262
Beginning of real logging implementation for pgloader messages
2007-12-04 23:08:30 +00:00
dim
d9f328ce4d
debian packaging
2007-11-29 21:30:25 +00:00
dim
0426dcb4f0
Fix default reformat_path checking; Fix template configuration usage
...
with user-defined columns (defining copy_columns from template).
2007-11-27 21:52:57 +00:00
dim
0c1ad56106
FIX: reformat_path could be not set after configuration reading
2007-11-27 14:28:17 +00:00
dim
1e7a16b869
Configuration now knows about templates, and command line about a reformat
...
path option, --reformat_path= or -R.
2007-11-26 21:33:24 +00:00
dim
3a8ac261c8
Support for template sections, WIP
2007-11-26 17:36:49 +00:00
dim
86b33defef
DEBUG and documentation of reformat option, plus other fixes for having all
...
the test cases pass again.
2007-11-20 15:11:49 +00:00
dim
e9b884cb4e
Global input_encoding configuration, per-section datestyle configuration, fixes.
2007-11-19 16:44:20 +00:00
dim
5cbd3531aa
Adding the 'input_encoding' option in the configuration file, per
...
section.
2007-11-18 23:04:48 +00:00
dim
dbaeef0ad2
pgloader version 2.2.3, with User Defined Columns
...
* User Defined Columns
* Temporary files with copy data content now suffixed .pgloader
* New option --version
* Fix TextReader newline_escapes configuration option reading
* Fix Reader reject initialisation
* Skip database related settings when in DRY_RUN mode (-n)
* List all command line options from man page synopsis
2007-11-14 21:09:36 +00:00
dim
57ea0f443e
Improve summary output
2007-11-05 14:17:01 +00:00
dim
d32472bb3e
Add support for --quiet and --summary options, in order to provide some
...
way to only see errors on output.
2007-10-20 14:28:50 +00:00
dim
921db51d65
* Support for datestyle setting
...
* Support for omiting column numbering
* Change documentation source format from SGML to asciidoc
* New version 2.2.1
2007-08-23 10:38:09 +00:00
dim
8ed1e0ff2c
New version of pgloader (2.0.2):
...
* new developpers
* python code replaces tcl code
* one config file for many tables (one command line for loading them all)
* manpage
* debian package
* supports multi-line input file (without quotes)
* Informix large objects support (loading to TEXT or BYTEA)
* configurable amount of rows per COPY instruction
* trailing slash optionnal removal (support informix UNLOAD file format)
* begin processing at any line in the file, by number or row id
* dry-run option, to validate input reading without connecting to database
* pedantic option, to stop processing on warning
2006-11-19 21:18:42 +00:00
