mirrors/pgloader
1
0
Fork 0
mirror of https://github.com/dimitri/pgloader.git synced 2025-08-08 23:37:00 +02:00
Code Issues Projects Releases Wiki Activity
1,568 Commits 7 Branches 39 Tags 23 MiB
fix/order-mysql-tables-by-decreasing-row-count
Commit Graph

1568 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dimitri Fontaine
6e324a1f74 Merge pull request #98 from cbbrowne/master 
Some wordsmithing on the docs
 2014-07-24 19:04:09 +02:00
Dimitri Fontaine
368dd88b48 Implement GETENV facility for reading FROM and INTO clauses. 
It might be important to be able to use the exact same pgloader commands
file but adapt its source and target depending on the environment where
the command is to be run (production, development, staging, etc).

Introduce the new sub-clause GETENV 'variable-name' to that effect.

The regression test facility that we have now isn't nearly sophisticated
enough to support this, so the feature isn't yet covered.
 2014-07-24 18:28:59 +02:00
Christopher Browne
ccb22d410b more wordsmithing 2014-07-24 11:03:33 -04:00
Dimitri Fontaine
798ae1e1a8 Remove MATERIALIZE ALL VIEWS from the TODO, it's been done. 2014-07-24 16:28:16 +02:00
Christopher Browne
f0ea64d627 Wordsmithing; fixing typos and such 2014-07-23 18:34:51 -04:00
Dimitri Fontaine
3b2119cae4 Don't catch all and any errors for retrying batches. 
In particular a non existing table or column shouldn't be considered as
an error we can just retry upon receiving.
 2014-07-21 15:38:53 +02:00
Dimitri Fontaine
a090f2fe3d Fix CCL compat where nil can't be a plist key. 2014-07-18 12:49:14 +02:00
Dimitri Fontaine
4aa8b0946f Get rid of unused sample file. 2014-07-18 11:07:25 +02:00
Dimitri Fontaine
cb4b2a3334 Convert SQLite tinyint to PostgreSQL smallint, fixes #97. 2014-07-18 11:06:37 +02:00
Dimitri Fontaine
9eff1bb4d8 Travis: Adapt test/ixf.load to work against 9.1. 2014-07-17 17:35:41 +02:00
Dimitri Fontaine
ebef038c8f Document the LOAD FROM IXF command. 2014-07-17 16:59:12 +02:00
Dimitri Fontaine
5a2b98856f Update the main SQLite test database. 2014-07-17 16:56:28 +02:00
Dimitri Fontaine
07b5aa3ed6 Add BEFORE/AFTER LOAD clauses to IXF and DBF commands. 2014-07-17 16:56:13 +02:00
Dimitri Fontaine
f352b39284 Travis: quickload local pgloader, not the QL dist one... 2014-07-17 10:19:00 +02:00
Dimitri Fontaine
e4948a3627 Move some logic up to the parser. 
As per grammar review at  https://github.com/nikodemus/esrap/issues/26,
improve the situation.
 2014-07-16 23:25:46 +02:00
Dimitri Fontaine
0f3103da2d Improve the parsing rules, avoiding eating all possible chars. 
Help improving the situation for #56.
 2014-07-16 23:09:11 +02:00
Dimitri Fontaine
5a636522de Fix make dependencies. 2014-07-14 22:34:43 +02:00
Dimitri Fontaine
9ddf117a90 In-passing desultory cleanup. 2014-07-14 21:54:50 +02:00
Dimitri Fontaine
3e0526c957 Implement early support for IXF files. 2014-07-14 21:53:50 +02:00
Dimitri Fontaine
f86dcf86fb Fix download link for latest tarball. 2014-07-14 14:34:26 +02:00
Dimitri Fontaine
d00837f8fc Fix --upgrade-config basic usage. 2014-07-13 16:35:53 +02:00
Dimitri Fontaine
f5fc5cce73 Review MySQL casts rules with auto_increment, fix #96. 2014-07-12 22:41:20 +02:00
Dimitri Fontaine
53a7e47058 New MySQL default Cast Rule for bit(1) to boolean, fix #93. 
We need a new transformation function that work with a vector of
integers as input.
 2014-07-03 11:47:59 +02:00
Dimitri Fontaine
6d49d9e10a Add a "real" column test case in SQLite main test, Closes #73. 2014-06-29 16:33:42 +02:00
Dimitri Fontaine
55655ed927 Fix fixed-file column name quoting, as we did for CSV, fixes #70. 2014-06-29 16:25:30 +02:00
Dimitri Fontaine
6a7f3c2417 Add some docs about oversized batches, fix #86. 2014-06-29 16:06:09 +02:00
Dimitri Fontaine
2577308ad6 Ensure field level processing is happening in all cases. 
When using both field level specifications and transformation functions,
the transformation needs to happen on the result of the field processing
rather than on the raw field.

Fixes #92.
 2014-06-26 02:08:49 +02:00
Dimitri Fontaine
ab75e6c626 Improve ragged right code comments, follow-up to #90. 2014-06-25 13:29:26 +02:00
Dimitri Fontaine
2ac374bcc8 Improve internal testing a bit. 
This should get into a full reproducible regression test against MySQL
some day.
 2014-06-25 12:43:57 +02:00
Dimitri Fontaine
a70a0ddd77 Keep typemod, default and not null when lacking casting rules, fixes #91. 2014-06-25 12:39:51 +02:00
Dimitri Fontaine
d1100efa28 Fix ragged lines support for fixed files, fixing #90. 
It's possible for the data to stop before the end of a specified column,
in which case we still want to accept whatever shortened data we have.
 2014-06-24 18:58:26 +02:00
Dimitri Fontaine
f7d251ed86 Fix quoting of TRUNCATE command, fix #84. 
That patch is not a principaled approach at fixing the problem but
should allow for not messing up with fully qualified table names.

A proper way to do it would be to have a pgsql object name structure
composed of the catalog, the schema and the name as separate entries,
with assorted API to print that object properly. That's for another day
though.
 2014-06-20 13:10:39 +02:00
Dimitri Fontaine
252abdeaa7 Find field attributes case-insensitively, fix #80. 
That's a follow-up to the case management cleanup for fields and
columns, wherein the "null if" property would stop being applied
properly.
 2014-06-17 21:53:53 +02:00
Dimitri Fontaine
88eba90776 Handle "ragged right" fixed width files, fix #82. 2014-06-17 17:24:07 +02:00
Dimitri Fontaine
9ad98c5c2a Handle errors when parsing fixed width files, per #82. 2014-06-17 17:10:04 +02:00
Dimitri Fontaine
899f0a3291 Improve nil handling of transformation functions, fix #81. 
Several transformation functions where not handling nil value
specifically, preventing the compiler to do its best and producing
WARNING at pgloader run time.
 2014-06-17 16:13:35 +02:00
Dimitri Fontaine
63e6b506be Travis; tweak some tests for PostgreSQL 9.1 compat. 2014-06-17 01:00:33 +02:00
Dimitri Fontaine
9c7f88c0d2 Travis: pre-install extension ip4r as superuser. 2014-06-17 00:47:56 +02:00
Dimitri Fontaine
e8828289cf Travis: try again with no SSL. 2014-06-17 00:33:35 +02:00
Dimitri Fontaine
f123843cdc Travis: apt-get install openssl rather than libssl... 2014-06-17 00:31:24 +02:00
Dimitri Fontaine
1d9c2251c5 Travis: add apt.postgresql.org and ip4r. 2014-06-17 00:28:46 +02:00
Dimitri Fontaine
34da7fac10 Travis: more tweaks. 2014-06-17 00:15:48 +02:00
Dimitri Fontaine
c14d620d83 Travis: let's see some debug information. 2014-06-16 21:15:22 +02:00
Dimitri Fontaine
0e16b221f0 Travis: we need to use GNU awk here! 2014-06-16 20:24:20 +02:00
Dimitri Fontaine
9d15376604 Yet another travis tweak. 2014-06-16 20:19:30 +02:00
Dimitri Fontaine
0882a18b52 Yet another attempt at getting the travis specs right. 2014-06-16 20:12:35 +02:00
Dimitri Fontaine
32b4cf23e8 New test case showing off the 'null if' source field option, see #80. 2014-06-16 19:59:08 +02:00
Dimitri Fontaine
61ff60d285 Another tweak of the travis build setup. 2014-06-16 18:30:19 +02:00
Dimitri Fontaine
4c43e5e3c3 Use http://pgsql.tapoueh.org/sbcl/sbcl_1.2.0-1_amd64.deb in travis builds. 2014-06-16 18:21:23 +02:00
Dimitri Fontaine
477c019fe1 Put the pv silencer on the right command line. 2014-06-16 18:03:36 +02:00