mirrors/pgloader
1
0
Fork 0
mirror of https://github.com/dimitri/pgloader.git synced 2025-08-07 23:07:00 +02:00
Code Issues Projects Releases Wiki Activity
152 Commits 7 Branches 39 Tags 23 MiB
pgloader-v2
Commit Graph

40 Commits

Author SHA1 Message Date
Guillaume Lelarge
8c03a124aa Add an example of the truncate option. 2011-02-25 11:51:24 +01:00
Dimitri Fontaine
1711ef97ff prepare for 2.3.3~dev3 release, some testers need that 2010-11-09 14:39:37 +01:00
Dimitri Fontaine
b8160e821a Add an example with null = 2010-11-09 12:31:26 +01:00
Dimitri Fontaine
f154d9bf83 Implement --boundaries to load a part of a file only (given in bytes). 2010-04-15 16:59:06 +02:00
Dimitri Fontaine
1738aaa6aa Add forgotten exemple, fixed input cols. 2010-04-15 16:55:37 +02:00
Dimitri Fontaine
cbd167c0e8 Implement --load-from-stdin 2010-04-15 16:55:10 +02:00
Dimitri Fontaine
1b6c0f4735 Implement support for the CVS field_size_limit, new in python 2.5. 2010-04-06 16:35:39 +02:00
Dimitri Fontaine
411a9a53d7 Add support for PG options as arguments and in the config file, with assorted fixes 2010-04-05 21:17:22 +02:00
Dimitri Fontaine
99df251963 Adding missed file, parallel data exemple 2010-04-04 19:32:48 +02:00
dim
090de905b2 Implement skip_head_lines in configuration file, some more bugfixes 2009-03-10 17:56:16 +00:00
dim
5a63f4f609 Protect the database connection establishing into the parralel sections semaphore, allow field_sep to be configured to \t, and some fixes 2008-05-29 09:16:13 +00:00
dim
b7c7e6a7c1 Support for fixed format datafile 2008-05-21 10:33:06 +00:00
dim
174ed31302 Adding support for columns=* configuration 2008-02-25 14:21:53 +00:00
dim
84738febfd FIX round-robin reader case when not all configured threads are needed to load the file 2008-02-16 19:17:03 +00:00
dim
ca54951de7 Generalize unbuffered file reading to a file-like class and use it in csvreader and textreader. Almost ok now, or so it seems 2008-02-14 22:25:25 +00:00
dim
610d3d6c2e Split file reading works with text format (problem was file iterator buffering) 2008-02-14 17:17:55 +00:00
dim
a068a5ea6a FIX just-introduced bug: no reader init meant self.db.copy_sep = None 2008-02-14 14:57:47 +00:00
dim
41f1a40e49 Factorize readers file opening code 2008-02-14 12:18:49 +00:00
dim
4c45cc9d1c DEBUG attempts of split_read_file(), renaming of internal methods, tests using text instead of csv 2008-02-13 17:12:30 +00:00
dim
e641b18fb3 Support for 'rrqueue_size' per-section configuration parameter, defaults to copy_every 2008-02-13 09:42:26 +00:00
dim
bac5d2b482 First implementation of Round Robin Reader threading method, Work In Progress --- does not work at all yet 2008-02-12 21:58:42 +00:00
dim
a2195ec2e7 Better handling of split_file_reading case, Still In Progress 2008-02-12 12:24:15 +00:00
dim
b534962192 Using semaphores to control how many max threads run at any given time 2008-02-12 11:27:18 +00:00
dim
fc8adf1831 First implementation of parallel loading, each thread loading a part of the input file (stat, seek, etc) 2008-02-11 17:38:52 +00:00
dim
a530dedd66 * Rework DatabaseError and consider dichotomy in this case too. 
* New options lc_messages and log_file
 2007-12-07 21:13:35 +00:00
dim
f0d6a8a37e Systematic usage of new logger module 2007-12-07 14:24:29 +00:00
dim
5da357c262 Beginning of real logging implementation for pgloader messages 2007-12-04 23:08:30 +00:00
dim
1116b2ceac FIX debian packaging wrt pysupport proper usage. 
pgloader modules now are installed into /usr/share/python-support/pgloader
 2007-12-01 10:15:29 +00:00
dim
d9f328ce4d debian packaging 2007-11-29 21:30:25 +00:00
dim
88db262b89 Catch IOError: commit code, not wrong example/pgloader.conf 2007-11-28 21:06:14 +00:00
dim
d99fc7ba4c Catch IOError when opening data file from textreader (such as no such file or directory) 2007-11-28 21:02:57 +00:00
dim
3a8ac261c8 Support for template sections, WIP 2007-11-26 17:36:49 +00:00
dim
a80e62642e packaging of 2.2.4 2007-11-21 10:27:22 +00:00
dim
86b33defef DEBUG and documentation of reformat option, plus other fixes for having all 
the test cases pass again.
 2007-11-20 15:11:49 +00:00
dim
5cbd3531aa Adding the 'input_encoding' option in the configuration file, per 
section.
 2007-11-18 23:04:48 +00:00
dim
dbaeef0ad2 pgloader version 2.2.3, with User Defined Columns 
* User Defined Columns
  * Temporary files with copy data content now suffixed .pgloader
  * New option --version
  * Fix TextReader newline_escapes configuration option reading
  * Fix Reader reject initialisation
  * Skip database related settings when in DRY_RUN mode (-n)
  * List all command line options from man page synopsis
 2007-11-14 21:09:36 +00:00
dim
921db51d65 * Support for datestyle setting 
* Support for omiting column numbering
  * Change documentation source format from SGML to asciidoc
  * New version 2.2.1
 2007-08-23 10:38:09 +00:00
dim
8ed8219e37 CSV support, can load not-all-columns of data files, can load to table with more cols than data file 2007-06-04 12:14:18 +00:00
dim
29934d7112 Support partial columns definition for COPY 2007-01-19 11:30:09 +00:00
dim
8ed1e0ff2c New version of pgloader (2.0.2): 
* new developpers
* python code replaces tcl code
* one config file for many tables (one command line for loading them all)
* manpage
* debian package
* supports multi-line input file (without quotes)
* Informix large objects support (loading to TEXT or BYTEA)
* configurable amount of rows per COPY instruction
* trailing slash optionnal removal (support informix UNLOAD file format)
* begin processing at any line in the file, by number or row id
* dry-run option, to validate input reading without connecting to database
* pedantic option, to stop processing on warning
 2006-11-19 21:18:42 +00:00