Commit Graph

10 Commits

Author SHA1 Message Date
Dimitri Fontaine
643875a266 Improve CSV error handling, thanks to cl-csv continue restart. 2014-02-08 17:51:15 +01:00
Dimitri Fontaine
8ac2cc4930 Skip empty lines when reading from files. 2014-01-24 15:11:15 +01:00
Dimitri Fontaine
3f61c66a79 Also handle extra columns in CSV parsing. 2014-01-23 15:15:42 +01:00
Dimitri Fontaine
516ef08c37 Allow loading ragged CSV files. 2014-01-23 15:07:05 +01:00
Dimitri Fontaine
a5c661dd4a Cleanup error recovery logging. 2014-01-22 17:57:56 +01:00
Dimitri Fontaine
ef358c0b7d Take benefits of PostgreSQL COPY error CONTEXT.
This message has the line number where the erroneous data was found on the
server, and given the pre-processing we already done at that point, it's
easy to convert that number into an index into the current batch, an array.

To do do, we need Postmodern to expose the CONTEXT error message and we need
to parse it. The following pull request cares about the Postmodern side of
things:

  https://github.com/marijnh/Postmodern/pull/46

The parsing is done as simply as possible, only assuming that the error
message is using comma separators and having the line number in second
position. The parsing as done here should still work with localized message
strings.

  CONTEXT: COPY errors, line 3, column b: "2006-13-11"

This change should significantly reduce the cost of error processing.
2013-12-25 21:43:22 +01:00
Dimitri Fontaine
d99b859c3f Improve and fix COPY error handling, transactions, connections, and GUCs. 2013-11-13 23:54:41 +01:00
Dimitri Fontaine
f619054c79 Improve database connection parsing to use sensible defaults. 2013-10-19 22:39:09 +02:00
Dimitri Fontaine
f768f74cbb Add support for comments (-- and /* */ style) in the command language. 2013-10-18 11:19:24 +02:00
Dimitri Fontaine
89aaabd179 Port tests from pgloader 2.x, implement trailing-sep. 2013-10-13 22:10:07 +02:00