Commit Graph

847 Commits

Author SHA1 Message Date
Dimitri Fontaine
0068a45e1c Fix parsing of qualified target table names, see #186.
We used to parse qualified table names as a simple string, which then
breaks attempts to be smart about how to quote idenfifiers. Some sources
are known to accept dots in quoted table names and we need to be able to
process that properly without tripping on qualified table names too
late.

Current code might not be the best approach as it's just using either a
cons or a string for table names internally, rather than defining a
proper data structure with a schema and a name slot.

Well, that's for a later cleanup patch, I happen to be lazy tonight.
2015-04-17 23:22:30 +02:00
Dimitri Fontaine
5ac396799a Be careful about the OS return code, fix #190.
Define a bunch of OS return codes and use them wisely, or at least in a
better way than just doing (uiop:quit) whenever there's something wrong,
without any difference whatsover to the caller.

Now we return a non-zero error code when we know something wrong did
happen. Which is more useful.
2015-04-17 22:30:04 +02:00
Dimitri Fontaine
11976d2c23 Fix census-places URL location of the source file. 2015-04-17 21:58:19 +02:00
Dimitri Fontaine
2a6ad888d0 Implement basic HTTP status checking in fetch method. 2015-04-17 21:37:21 +02:00
Dimitri Fontaine
cb94993064 Fix #202, blind attempt, passes tests. 2015-04-06 19:47:38 +02:00
Dimitri Fontaine
77394bd029 Merge pull request #201 from jdufresne/fix-typo
Fix typo
2015-04-02 20:37:54 +02:00
Jon Dufresne
b88ef6bdea Fix typo. 2015-04-02 09:45:44 -07:00
Jon Dufresne
8038931f5a Remove trailing whitespace. 2015-04-02 09:45:36 -07:00
Dimitri Fontaine
8f57c90809 Merge pull request #196 from benesch/makefile-updates
Match pgloader-standalone target to pgloader target
2015-03-23 22:19:55 +01:00
Nikhil Benesch
fe35487048 Match pgloader-standalone target to pgloader target 2015-03-23 17:03:34 -04:00
Dimitri Fontaine
36eeb6f438 Fix header lines length to be dynamic too... 2015-03-21 17:35:34 +01:00
Dimitri Fontaine
f9d70dee5c Merge pull request #193 from chmouel/patch-1
Trivial spellling mistakes
2015-03-18 10:30:17 +01:00
Chmouel Boudjnah
9173ce82ba Trivial spellling mistakes
I was just browsing the code and my english mispelling OCD kicked in.
2015-03-18 09:40:54 +01:00
Dimitri Fontaine
48ab15a77d Auto-adjust summary table name column width.
Per gripe from Marcos, who argues that for a human readable format
breaking when table names are wider than expected at compile time is
quite a strange position to defend.
2015-03-16 14:55:46 +01:00
Dimitri Fontaine
095e1c7bbc Merge pull request #192 from wmorin/fix-typo
Fix explanation typo. Drop table if it exists
2015-03-12 17:25:24 +01:00
Willy Morin
eca61f8771 Fix explanation typo. Drop table if it exists 2015-03-12 11:42:21 +01:00
Dimitri Fontaine
9361b2113e Merge pull request #191 from mtyson01/specdates
Fix incorrect dates in specfile
2015-03-09 10:56:41 +01:00
Matt Tyson
24e6fc0384 Fix incorrect dates in specfile
RPM complains about these dates being invalid during the build process
2015-03-09 00:47:07 +00:00
Dimitri Fontaine
7d2d09ce68 Add the option to preserve MySQL index names, fix #187.
See test/parse/hans.goeuro.load for an example usage of the new option.

In passing, any error when creating indexes is now properly reported and
logged, which was missing previously. Oops.
2015-03-07 20:19:47 +01:00
Dimitri Fontaine
d8510b031c Cleanup the MS SQL schema introspection queries, see #183. 2015-02-20 19:13:21 +01:00
Dimitri Fontaine
48f451bdbc Implement the option to disable triggers when loading data.
This option is dangerous and allows to skip ALL triggers when loading
data against PostgreSQL. This includes foreign key constraints
definitions and will allow loading data out of order.

When using both the options "create no table" and "disable triggers" it
will be possible to load data into a schema prepared by your favorite
external tool, at the cost of not validating FK constraints. Use with
care.

Fix #167.
2015-02-19 15:05:10 +01:00
Dimitri Fontaine
4f099e3ddc Merge pull request #179 from pborreli/typos
Fixed typos
2015-02-19 10:34:24 +01:00
Dimitri Fontaine
47288d2818 Fix whitespace and indentation. 2015-02-19 10:30:42 +01:00
Victor Kryukov
c38ef4c235 Make quoting identifiers more robust: do not quote already quoted string, and double quotes when quoting. Fix #180. 2015-02-19 10:26:44 +01:00
Dimitri Fontaine
5b19776d5b MS SQL casting rules for floats: there's no scale. See #177. 2015-02-19 10:15:35 +01:00
Pascal Borreli
1a18b5cfac Fixed typos 2015-02-18 23:17:16 +00:00
Dimitri Fontaine
7fd1ddaa5f Handle MS SQL columns of float types without scale, fix #177.
The default for MS SQL float types is to only have a precision defined,
as described in https://msdn.microsoft.com/en-us/library/ms173773.aspx,
but the pgloader code didn't know what to do with a float without scale.
2015-02-18 23:43:27 +01:00
Dimitri Fontaine
c5f0aeec93 The default DBF encoding still is ASCII. 2015-02-18 23:27:35 +01:00
Dimitri Fontaine
55584406fa Add encoding support for db3 sources, fix #176.
It appears that db3 files are not limited to the ASCII character
encoding that they were designed with, so let's clue pgloader about
that.

This commit build
770cbe3526
and the pgloader Makefile has been updated to momentarily fetch cl-db3
from github rather than Quicklisp so that it's possible to enjoy the new
feature immediately.
2015-02-18 22:40:03 +01:00
Dimitri Fontaine
8f4eb73cad Use the "table name" option when loading from DBF, fix #175.
The command parser for DBF sources wasn't (anymore?) considering the
value of the "table name = " option, only considering the option from
the connection string.

Make it so that the option now takes preference over the DSN tablename
option when both are set.
2015-02-18 14:52:14 +01:00
Dimitri Fontaine
a8e728a740 Document with extra auto_increment casting rules guard, fix #173. 2015-02-17 22:14:54 +01:00
Dimitri Fontaine
b00e1bba7f Allow backslash and at-sign in user names, fix #171.
When connection to MS SQL it's sometimes needed to be able to use @ or \
as part of the username, which is now possible.
2015-02-11 11:56:52 +01:00
Dimitri Fontaine
3c1f18f4f5 Merge pull request #169 from pinaraf/fixDotInName
Allow dots in user names
2015-02-06 20:10:06 +01:00
Pierre Ducroquet
894df76b36 Introduce a new rule specific for username string
This allows dots in username.
2015-02-04 23:22:38 +01:00
Dimitri Fontaine
7ccbb45763 Be smarter about MS SQL bit to boolean conversion, should fix #162.
This is a blind patch given that I couldn't CREATE TABLE as per the bug
report to try and see by myself what's happening. Better have some tests
going on though.
2015-02-03 13:39:09 +01:00
Dimitri Fontaine
100e942c22 Force quoting of some identifiers, per pg rules, fix #161.
PostgreSQL requires that an idenfitier begin with letters or underscore
only, so an identifier that begins with a digit must be quoted. In the
current coding pgloader will unecessarily quote some identifiers that
begin with a unicode accentuated letter, but that's only cosmetic and
isn't worth worrying about (famous last words).
2015-01-29 10:40:12 +01:00
Dimitri Fontaine
d02f5c3992 Allow quoting table names in pgsql uri, fix #163. 2015-01-29 10:10:40 +01:00
Dimitri Fontaine
3e5115c493 Fix MS SQL index names.
As MS SQL index names may contain dots, replace dots with underscores.
That should fix #158 once again.
2015-01-26 16:56:39 +01:00
Dimitri Fontaine
95961a42b8 Replace dots with underscores in MS SQL fk names.
That should help fixing #158 where MS SQL uses the following name for
one of its fkey: fk_dbo.track_dbo.artist_artistid. PostgreSQL refuses fk
names with dots in it.
2015-01-24 17:23:08 +01:00
Dimitri Fontaine
13faf3e4f8 Blind try at fixing #158.
The call to format-pgsql-create-fkey was passing the fkey cons rather
than just the fk definition structure as an argument.
2015-01-23 19:59:04 +01:00
Dimitri Fontaine
4e7f925a9b Typo fix in the docs. 2015-01-22 11:38:49 +01:00
Dimitri Fontaine
cefc636a6e Merge pull request #156 from seletz/seletz-fix-typo-in-manpage
Update pgloader.1.md
2015-01-22 11:15:13 +01:00
Stefan Eletzhofer
9efe68504f Update pgloader.1.md
Fixed a very minor formatting change in the man page markdown
source.
2015-01-22 11:10:22 +01:00
Dimitri Fontaine
28f6a87165 Interim release and forward porting from rel_3_2_stable.
Given the slashdot effect and some bad luck, the binary artefacts of the
3.2.0 release are not currently available, and anyway contain known bugs
that have been fixed meanwhile thanks to early adopters who did open
issues on github.

So we hastily publish the current master's branch version as a github
release with binary files.
2015-01-22 02:25:14 +01:00
Dimitri Fontaine
02ecd8866b Fix Makefile and bash multi-line escaping...
Travis is unhappy about the spelling of the command, so let's not try to
be cleverer here and accept a runaway line here. See #153 too.
2015-01-21 19:56:49 +01:00
Dimitri Fontaine
8ce967744c Add support for http_proxy environment variable to install quicklisp.
See #153.
2015-01-21 15:57:29 +01:00
Dimitri Fontaine
ce5a61face Catch PostgreSQL internal errors too, fixes #155. 2015-01-21 13:01:28 +01:00
Dimitri Fontaine
86fab763a8 Fix "typo" where SBCL's init and save hook were swapped... fix #153. 2015-01-20 10:06:10 +01:00
Dimitri Fontaine
425266798b Merge pull request #152 from zickzackv/patch-2
add note to install universal binaries of freetds
2015-01-19 12:14:31 +01:00
zickzackv
511fc40ee6 add note to install universal binaries of freetds
Otherwise cffi tries to load the 64-bit version of the library which fails
2015-01-19 09:10:02 +01:00