mirror of
https://github.com/dimitri/pgloader.git
synced 2025-08-09 07:47:00 +02:00
104 lines
4.0 KiB
Plaintext
104 lines
4.0 KiB
Plaintext
In this directory you'll find out some pgloader usage examples.
|
|
|
|
To use them, please first create a pgloader database, then for each example
|
|
the tables it needs, then issue the pgloader command:
|
|
|
|
$ createdb --encoding=utf-8 pgloader
|
|
$ cd examples
|
|
$ psql pgloader < simple/simple.sql
|
|
$ ../pgloader.py -Tvc pgloader.conf simple
|
|
|
|
If you want to load data from all examples, create tables for all of them
|
|
first, then run pgloader without argument.
|
|
|
|
The provided examples are:
|
|
|
|
. simple
|
|
|
|
This dataset shows basic case, with trailing separator and data
|
|
reordering.
|
|
|
|
. errors
|
|
|
|
Same test, but with impossible dates. Should report some errors. If it
|
|
does not report errors, check you're not using psycopg 1.1.21.
|
|
|
|
Should report 3 errors out of 7 lines (4 updates).
|
|
|
|
. clob
|
|
|
|
This dataset shows some text large object importing to PostgreSQL text
|
|
datatype.
|
|
|
|
. cluttured
|
|
|
|
A dataset with newline escaped and multi-line input (without quoting)
|
|
Beware of data reordering, too.
|
|
|
|
. csv
|
|
|
|
A dataset with csv delimiter ',' and quoting '"'.
|
|
|
|
. partial
|
|
|
|
A dataset from which we only load some columns of the provided one.
|
|
|
|
. serial
|
|
|
|
In this dataset the id field is ommited, it's a serial which will be
|
|
automatically set by PostgreSQL while COPYing.
|
|
|
|
. reformat
|
|
|
|
A timestamp column is formated the way MySQL dump its timestamp,
|
|
which is not the same as the way PostgreSQL reads them. The
|
|
reformat.mysql module is used to reformat the data on-the-fly.
|
|
|
|
. udc
|
|
|
|
A used defined column test, where all file columns are not used but
|
|
a new constant one, not found in the input datafile, is added while
|
|
loading data.
|
|
|
|
You can launch all those pgloader tests in one run, provided you created the
|
|
necessary tables:
|
|
|
|
$ for sql in */*sql; do psql pgloader < $sql; done
|
|
$ ../pgloader.py -Tsc pgloader.conf
|
|
|
|
errors WARNING COPY error, trying to find on which line
|
|
errors WARNING COPY data buffer saved in /tmp/errors.AhWvAv.pgloader
|
|
errors WARNING COPY error recovery done (2/3) in 0.064s
|
|
errors WARNING COPY error, trying to find on which line
|
|
errors WARNING COPY data buffer saved in /tmp/errors.BclHtj.pgloader
|
|
errors WARNING COPY error recovery done (1/1) in 0.054s
|
|
errors ERROR 3 errors found into [errors] data
|
|
errors ERROR please read /tmp/errors.rej.log for errors log
|
|
errors ERROR and /tmp/errors.rej for data still to process
|
|
errors ERROR 3 database errors occured
|
|
reformat WARNING COPY error, trying to find on which line
|
|
reformat WARNING COPY data buffer saved in /tmp/reformat.6P4WCD.pgloader
|
|
reformat WARNING COPY error recovery done (1/4) in 0.034s
|
|
reformat ERROR 1 errors found into [reformat] data
|
|
reformat ERROR please read /tmp/reformat.rej.log for errors log
|
|
reformat ERROR and /tmp/reformat.rej for data still to process
|
|
reformat ERROR 1 database errors occured
|
|
|
|
Table name | duration | size | copy rows | errors
|
|
====================================================================
|
|
allcols | 0.025s | - | 8 | 0
|
|
clob | 0.034s | - | 7 | 0
|
|
cluttered | 0.061s | - | 6 | 0
|
|
csv | 0.035s | - | 6 | 0
|
|
errors | 0.113s | - | 4 | 3
|
|
fixed | 0.045s | - | 3 | 0
|
|
partial | 0.030s | - | 7 | 0
|
|
reformat | 0.036s | - | 4 | 1
|
|
serial | 0.029s | - | 7 | 0
|
|
simple | 0.050s | - | 7 | 0
|
|
udc | 0.020s | - | 5 | 0
|
|
====================================================================
|
|
Total | 0.367s | - | 64 | 4
|
|
|
|
Please note errors test should return 3 errors and reformat 1 error.
|