Dimitri Fontaine
c50164e53d
Manage the whole class of "integrity errors" also when retrying a batch...
2014-01-24 15:10:03 +01:00
Dimitri Fontaine
69b550a46e
Make use of the new usage function...
2014-01-24 10:14:51 +01:00
Dimitri Fontaine
be4cc804c0
Show usage and help when the command line options are not recognized.
2014-01-24 09:22:02 +01:00
Dimitri Fontaine
e8fcb15c27
Fix another hasty commit erroneously containing a for-tests change.
2014-01-23 23:29:27 +01:00
Dimitri Fontaine
b374d4bc8b
The current retry method has no need for *copy-batch-split*.
2014-01-23 23:28:25 +01:00
Dimitri Fontaine
d132bafc07
Refrain from parsing a non-existing command file...
2014-01-23 23:17:34 +01:00
Dimitri Fontaine
3f61c66a79
Also handle extra columns in CSV parsing.
2014-01-23 15:15:42 +01:00
Dimitri Fontaine
516ef08c37
Allow loading ragged CSV files.
2014-01-23 15:07:05 +01:00
Dimitri Fontaine
4cbe4b3218
Manage the whole class of "integrity violation" errors.
2014-01-23 14:59:46 +01:00
Dimitri Fontaine
59e87b84a0
Release Candidate 8.
2014-01-23 00:26:08 +01:00
Dimitri Fontaine
9455752805
Review and simplify batch retry processing.
2014-01-23 00:15:57 +01:00
Dimitri Fontaine
7c238f45f2
Fix batch retry handling, broken in previous refactoring. Fixes #22 .
2014-01-22 22:52:18 +01:00
Dimitri Fontaine
a5c661dd4a
Cleanup error recovery logging.
2014-01-22 17:57:56 +01:00
Dimitri Fontaine
ccb888164c
Fix where to find relative filenames from within commands.
2014-01-22 17:11:01 +01:00
Dimitri Fontaine
c13d7bbae7
Bug fix when processing plain filenames.
2014-01-22 11:00:12 +01:00
Dimitri Fontaine
a51a712b6a
Fix asd dependencies, cleanup useless and misplaced compilation options.
2014-01-21 14:37:26 +01:00
Dimitri Fontaine
6e4a3e2165
Fix parsing COPY error message without column information, see issue #22 .
2014-01-20 17:19:41 +01:00
Dimitri Fontaine
c56bbab0c4
Fix #24 by allowing cast rules adding only transformation functions.
2014-01-20 16:00:09 +01:00
Dimitri Fontaine
afc64cc30d
Cast MySQL smallint with auto_increment to PostgreSQL serial, fixes #22 .
2014-01-16 10:07:21 +01:00
Dimitri Fontaine
80b6c46aae
Version 3.0.97.
2014-01-15 22:53:43 +01:00
Dimitri Fontaine
aa49e8eec2
Fix the log-filename when operating from the command line.
2014-01-15 22:48:45 +01:00
Dimitri Fontaine
6ccb1871f5
Fix parsing dotted hostnames.
2014-01-15 10:48:56 +01:00
Dimitri Fontaine
07c614c170
Switch to the newer cl-csv API.
...
Thanks to the work at https://github.com/AccelerationNet/cl-csv/pull/12 we
can now use the main branch of cl-csv again.
2014-01-11 18:28:24 +01:00
Dimitri Fontaine
d5890f7779
Release 3.0.96, debian packaging included.
2013-12-27 12:22:03 +01:00
Dimitri Fontaine
ef358c0b7d
Take benefits of PostgreSQL COPY error CONTEXT.
...
This message has the line number where the erroneous data was found on the
server, and given the pre-processing we already done at that point, it's
easy to convert that number into an index into the current batch, an array.
To do do, we need Postmodern to expose the CONTEXT error message and we need
to parse it. The following pull request cares about the Postmodern side of
things:
https://github.com/marijnh/Postmodern/pull/46
The parsing is done as simply as possible, only assuming that the error
message is using comma separators and having the line number in second
position. The parsing as done here should still work with localized message
strings.
CONTEXT: COPY errors, line 3, column b: "2006-13-11"
This change should significantly reduce the cost of error processing.
2013-12-25 21:43:22 +01:00
Dimitri Fontaine
36bfd923fa
Fix monitor logging with no argument is given to the format string.
2013-12-25 21:25:20 +01:00
Dimitri Fontaine
cd1d5ad597
Fix missing *pg-dbname* in CSV processing, resulting in broken logging.
2013-12-25 16:50:14 +01:00
Dimitri Fontaine
7d94d4ff62
Small code cleanup.
2013-12-25 16:20:37 +01:00
Dimitri Fontaine
f2bec5fcd1
Fix the main command line "driver" to use the new with-monitor API.
2013-12-24 19:54:22 +01:00
Dimitri Fontaine
d6edc54272
Ensure ordering of sections (table-names) in the summary.
2013-12-24 19:31:27 +01:00
Dimitri Fontaine
b2c9e0d2dc
Refactor the whole logging infrastructure not to depend on threads sharing streams.
2013-12-24 19:08:55 +01:00
Dimitri Fontaine
e31c5cbe83
Set the default batch size (in rows) back to 25000.
2013-12-23 13:17:04 +01:00
Dimitri Fontaine
8bf4bffb2c
Fix SQLite stats, wasn't counting read rows.
2013-12-23 13:16:04 +01:00
Dimitri Fontaine
8dda84b0e8
Some more refactoring, MySQL copy-database function is still too big.
2013-12-20 17:37:25 +01:00
Dimitri Fontaine
fe302af221
Refactor the dbname API to feed from the connection string directly.
2013-12-20 17:24:02 +01:00
Dimitri Fontaine
5a464a0fae
Remove some dead code (was intended for later use).
2013-12-20 17:07:43 +01:00
Dimitri Fontaine
4748104a38
Refactor the too big copy-database function for MySQL.
2013-12-20 15:06:40 +01:00
Dimitri Fontaine
aa934f1a5c
Review, fix some comments and indenting.
2013-12-20 11:08:26 +01:00
Dimitri Fontaine
49f2567b0d
Improve stats on meta-data and before sections.
2013-12-20 10:50:27 +01:00
Dimitri Fontaine
ce14542a15
Respect the Materialize Views ordering given in the command when loading data.
2013-12-20 10:35:00 +01:00
Dimitri Fontaine
bb561a0c4d
Fetch all MySQL meta-data from a single connection.
2013-12-20 10:21:31 +01:00
Dimitri Fontaine
e09eb3fbb2
There is always a need for a before state when migrating from MySQL...
2013-12-20 10:19:36 +01:00
Dimitri Fontaine
aca04b1514
Fix problem found when trying to load the code with CCL.
2013-12-19 23:08:02 +01:00
Dimitri Fontaine
3a5d618cc8
Switch to using vectors for representing rows, minimize consing.
2013-12-18 23:30:11 +01:00
Dimitri Fontaine
0a6608061c
Due to the bug fixed and the new batch implementation, version is now 3.0.95.
2013-12-18 18:56:12 +01:00
Dimitri Fontaine
82c4bc9e9e
Switch the pgsql batch implementation to using arrays to reduce consing.
2013-12-18 18:31:06 +01:00
Dimitri Fontaine
fe2fb517ac
Typo fix.
2013-12-18 18:26:31 +01:00
Gleb Golubitsky
50e8fca96b
Fixing "quote identifiers" option for MySQL conversions
...
Signed-off-by: Gleb Golubitsky <sectoid@gnolltech.org>
2013-12-18 15:06:22 +02:00
Dimitri Fontaine
1ef024c5c7
Fix handling of reserved table names in SELECT and COPY commands.
2013-12-18 12:06:41 +01:00
Dimitri Fontaine
633b6ae64f
Export with-mysql-connection facility for REPL testing.
2013-12-18 12:05:56 +01:00
