mirrors/pgloader
1
0
Fork 0
mirror of https://github.com/dimitri/pgloader.git synced 2026-05-04 10:31:02 +02:00
Code Issues Projects Releases Wiki Activity

In passing docs review.

Browse Source 
Also modernize the test/census-place.load to use the newer fixed file
format options, and show that in the docs.
This commit is contained in:
Dimitri Fontaine 2014-10-18 22:42:31 +02:00
parent 1bc467f1a4
commit 68416a79b3
7 changed files with 53 additions and 57 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
test/census-places.load
View File

@ -17,22 +17,20 @@ LOAD ARCHIVE
FROM FILENAME MATCHING ~/places2k.txt/
WITH ENCODING latin1
(
-- name start length
usps 0 2,
fips 2 2,
fips_code 4 5,
"LocationName" 9 64,
p 73 9,
h 82 9,
land 91 14,
water 105 14,
ldm 119 14,
wtm 131 14,
lat 143 10,
long 153 11
usps from 0 for 2,
fips from 2 for 2,
fips_code from 4 for 5,
"LocationName" from 9 for 64 [trim right whitespace],
p from 73 for 9,
h from 82 for 9,
land from 91 for 14,
water from 105 for 14,
ldm from 119 for 14,
wtm from 131 for 14,
lat from 143 for 10,
long from 153 for 11
)
INTO postgresql:///pgloader?places
(
usps, fips, fips_code,
"LocationName" text using (right-trim LocationName)
usps, fips, fips_code, "LocationName"
);

6
web/src/csv.md
View File

@ -5,9 +5,9 @@ specifications. pgloader allows you to describe those specs in its command.
## The Command
To load data with [pgloader](http://pgloader.tapoueh.org/) you need to
define in a *command* the operations in some details. Here's our example for
loading CSV data:
To load data with [pgloader](http://pgloader.io/) you need to define in a
*command* the operations in some details. Here's our example for loading CSV
data:
LOAD CSV
FROM 'path/to/file.csv' (x, y, a, b, c, d)

10
web/src/dBase.md
View File

@ -6,9 +6,9 @@ support in modern tools, pgloader is right there on the list too!
## The Command
To load data with [pgloader](http://pgloader.tapoueh.org/) you need to
define in a *command* the operations in some details. Here's our example for
loading a dBase file, using a file provided by the french administration.
To load data with [pgloader](http://pgloader.io/) you need to define in a
*command* the operations in some details. Here's our example for loading a
dBase file, using a file provided by the french administration.
You can find more files from them at the
[Insee](http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement.asp)
@ -50,8 +50,8 @@ Let's start the `pgloader` command with our `dbf-zip.load` command file:
----------------- --------- --------- --------- --------------
Total import time 9181 9181 0 1.906s
We can see that [http://pgloader.tapoueh.org](pgloader) did download the
file from its HTTP URL location then *unziped* it before the loading itself.
We can see that [http://pgloader.io](pgloader) did download the file from
its HTTP URL location then *unziped* it before the loading itself.
Note that the output of the command has been edited to facilitate its
browsing online.

40
web/src/fixed.md
View File

@ -6,9 +6,9 @@ blank-padded when the data is shorter than the full reserved range.
## The Command
To load data with [pgloader](http://pgloader.tapoueh.org/) you need to
define in a *command* the operations in some details. Here's our example for
loading Fixed Width Data, using a file provided by the US census.
To load data with [pgloader](http://pgloader.io/) you need to define in a
*command* the operations in some details. Here's our example for loading
Fixed Width Data, using a file provided by the US census.
You can find more files from them at the
[Census 2000 Gazetteer Files](http://www.census.gov/geo/maps-data/data/gazetteer2000.html).
@ -29,29 +29,27 @@ Here's our command:
loc_name varchar(64)
);
$$
LOAD FIXED
FROM FILENAME MATCHING ~/places2k.txt/
WITH ENCODING latin1
(
-- name start length
usps 0 2,
fips 2 2,
fips_code 4 5,
loc_name 9 64,
p 73 9,
h 82 9,
land 91 14,
water 105 14,
ldm 119 14,
wtm 131 14,
lat 143 10,
long 153 11
usps from 0 for 2,
fips from 2 for 2,
fips_code from 4 for 5,
"LocationName" from 9 for 64 [trim right whitespace],
p from 73 for 9,
h from 82 for 9,
land from 91 for 14,
water from 105 for 14,
ldm from 119 for 14,
wtm from 131 for 14,
lat from 143 for 10,
long from 153 for 11
)
INTO postgresql:///pgloader?places
(
usps, fips, fips_code,
loc_name text using (right-trim loc_name)
usps, fips, fips_code, "LocationName"
);
You can see the full list of options in the
@ -91,8 +89,8 @@ Let's start the `pgloader` command with our `census-places.load` command file:
----------------- --------- --------- --------- --------------
Total import time 25375 25375 0 3.019s
We can see that [http://pgloader.tapoueh.org](pgloader) did download the
file from its HTTP URL location then *unziped* it before the loading itself.
We can see that [http://pgloader.io](pgloader) did download the file from
its HTTP URL location then *unziped* it before the loading itself.
Note that the output of the command has been edited to facilitate its
browsing online.

6
web/src/geolite.md
View File

@ -7,9 +7,9 @@ content into your database directly.
## The Command
To load data with [pgloader](http://pgloader.tapoueh.org/) you need to
define in a *command* the operations in some details. Here's our example for
loading the Geolite data:
To load data with [pgloader](http://pgloader.io/) you need to define in a
*command* the operations in some details. Here's our example for loading the
Geolite data:
/*
* Loading from a ZIP archive containing CSV files. The full test can be

10
web/src/mysql.md
View File

@ -43,8 +43,8 @@ You can see the full list of options in the
of the options you see here.
Note that here pgloader will benefit from the meta-data information found in
the MySQL database to create a PostgreSQL table capable of hosting the data
as described, then load the data.
the MySQL database to create a PostgreSQL database capable of hosting the
data as described, then load the data.
In particular, some specific *casting rules* are given here, to cope with
date values such as `0000-00-00` that MySQL allows and PostgreSQL rejects
@ -56,9 +56,9 @@ Finaly note that we are using the *MATERIALIZE VIEWS* clause of pgloader:
the selected views here will be migrated over to PostgreSQL *with their
contents*.
It's possible to use the *MATERIALIZE VIEWS* clause and given both the name
and the SQL (in MySQL dialect) definition of view, then pgloader created the
view for the duration of the data loading.
It's possible to use the *MATERIALIZE VIEWS* clause and give both the name
and the SQL (in MySQL dialect) definition of view, then pgloader creates the
view at bofore loading the data, then drops it again at the end.
## Loading the data

10
web/src/sqlite.md
View File

@ -7,7 +7,7 @@ very good at. pgloader can help you there.
## The Command
To load data with [pgloader](http://pgloader.tapoueh.org/) you need to
To load data with [pgloader](http://pgloader.io/) you need to
define in a *command* the operations in some details. Here's our command:
load database
@ -23,8 +23,8 @@ You can see the full list of options in the
of the options you see here.
Note that here pgloader will benefit from the meta-data information found in
the SQLite file to create a PostgreSQL table capable of hosting the data as
described, then load the data.
the SQLite file to create a PostgreSQL database capable of hosting the data
as described, then load the data.
## Loading the data
@ -65,8 +65,8 @@ Let's start the `pgloader` command with our `sqlite.load` command file:
---------------------- --------- --------- --------- --------------
Total streaming time 15607 15607 0 0.476s
We can see that [http://pgloader.tapoueh.org](pgloader) did download the
file from its HTTP URL location then *unziped* it before the loading itself.
We can see that [http://pgloader.io](pgloader) did download the file from
its HTTP URL location then *unziped* it before loading it.
Also, the *WARNING* messages we see here are expected as the PostgreSQL
database is empty when running the command, and pgloader is using the SQL