Commit Graph

847 Commits

Author SHA1 Message Date
Dimitri Fontaine
ea5a373ad9 Fix loading examples to avoid loading with the index already there. 2013-10-01 00:11:54 +02:00
Dimitri Fontaine
6b3e5d6338 Some tries at optimisations. 2013-10-01 00:11:26 +02:00
Dimitri Fontaine
1fd0fb68f2 Get rid of the global null-as processing, which means nothing. 2013-10-01 00:11:13 +02:00
Dimitri Fontaine
2ddaba4284 Fix MySQL schema casting WRT identifers.
In MySQL schema cast, take care to downcase or quote the colum names
references in the index definitions, and replace non alphanumeric characters
with #\_ when downcasing.
2013-09-30 22:09:07 +02:00
Dimitri Fontaine
ccc153b9e1 Don't report a separation line when there's no finally section to separate... 2013-09-30 19:54:26 +02:00
Dimitri Fontaine
64589134c8 Temporary hack for debugging, error wasn't there anyway. 2013-09-30 19:48:42 +02:00
Dimitri Fontaine
d3991690cf Fix handling of CSV loading when not given fields nor columns lists. 2013-09-30 19:48:21 +02:00
Dimitri Fontaine
2ff0d11332 Fix a typo in the com.informatimago.clext ASD dependency declaration. 2013-09-30 17:31:28 +02:00
Dimitri Fontaine
32d183efa5 Rework the summary handling in LOAD FROM ARCHIVE commands. 2013-09-30 16:04:01 +02:00
Dimitri Fontaine
9c1592c4eb Switch to using the unzip command rather than the CL implementation... 2013-09-30 14:23:19 +02:00
Dimitri Fontaine
2a6c974f8e Handle input file encodings. 2013-09-30 00:26:41 +02:00
Dimitri Fontaine
47257a173c Add some logs in the archive code. 2013-09-30 00:26:07 +02:00
Dimitri Fontaine
5ca4348e7e Fix handling of per-field NULL IF properties. 2013-09-29 22:26:20 +02:00
Dimitri Fontaine
9bea12ed3e Fix the LOAD FROM ARCHIVE command code paths until it actually works. 2013-09-28 23:45:24 +02:00
Dimitri Fontaine
a951d55f4c Don't try to probe-file tmpdir when it's nil. 2013-09-27 10:28:49 +02:00
Dimitri Fontaine
6e3767bb37 Implement generic field to columns projection with support for user code.
And use that new facility in the Archive mode of operations, which is a
practical unit of testing and milestone to reach. We have enough code to
actually compile and try running the parser's output.
2013-09-25 00:01:46 +02:00
Dimitri Fontaine
b4e530981c WIP implementing full archive fetching and downloading. 2013-09-24 18:34:05 +02:00
Dimitri Fontaine
4172374b61 Use CREATE TABLE IF NOT EXISTS in the db3 import. 2013-09-24 11:05:57 +02:00
Dimitri Fontaine
2ab8c74c49 Update docs to point ot the main Postmodern tree. 2013-09-23 15:34:39 +02:00
Dimitri Fontaine
fd578e7c63 Document the temporary dependency towards the patched version of Postmodern. 2013-09-23 15:28:20 +02:00
Dimitri Fontaine
699fcb21f2 Refactor to new Postmodern patch. 2013-09-23 15:25:02 +02:00
Dimitri Fontaine
7151a2ea62 Refactor transaction handling, depend on a patch to postmodern. 2013-09-23 11:30:20 +02:00
Dimitri Fontaine
fd2809277f Code cleanup: use with-stats-collection. 2013-09-23 09:58:41 +02:00
Dimitri Fontaine
ac4f9ff654 Code review, client messages, logs, summary, index naming and building. 2013-09-23 00:09:11 +02:00
Dimitri Fontaine
031a8bd15a First pass at thinking parallel: create indexes while copying next tables... 2013-09-22 16:37:04 +02:00
Dimitri Fontaine
e75954c319 Remove the global kernel parameter, create as needed. 2013-09-22 15:12:43 +02:00
Dimitri Fontaine
1a28a03e1f Add option "schema only" for MySQL migrations, cleanup code, fix identifer case for indexes. 2013-09-21 17:35:03 +02:00
Dimitri Fontaine
c9dd959fbd Implement MySQL migration option "downcase|quote identifier". 2013-09-20 22:52:45 +02:00
Dimitri Fontaine
020cd7d3ed Fix the code to handle lack of target type in MySQL cast rules parsing. 2013-09-20 22:51:44 +02:00
Dimitri Fontaine
dd38c9c154 Implement a command to load data from DBF files. 2013-09-20 15:00:26 +02:00
Dimitri Fontaine
7b97c687d7 Add a test case for parsing several commands at once. 2013-09-20 13:32:43 +02:00
Dimitri Fontaine
bc4927bbbb Allow to omit the target type definition in a MySQL cast definition. 2013-09-20 13:32:27 +02:00
Dimitri Fontaine
69b2cc119c Implement a first version of the LOAD CSV FROM command. 2013-09-20 13:14:49 +02:00
Dimitri Fontaine
3f89a00731 Fix CSV copy-from-file to be usable as-is, and fix conn spec passing. 2013-09-20 11:39:08 +02:00
Dimitri Fontaine
ba16f6f6b7 Alway quote identifiers when converting MySQL schema to PostgreSQL. 2013-09-20 10:26:02 +02:00
Dimitri Fontaine
2135c45763 Review default type casting rules from MySQL to PostgreSQL, include blobs. 2013-09-20 10:18:45 +02:00
Dimitri Fontaine
b072298937 It's unfortunately not possible to SET gucs once already in COPY "mode". 2013-09-19 23:32:53 +02:00
Dimitri Fontaine
bfcefce45f Fix passing *pg-settings* around in lparallel kernel bindings. 2013-09-19 23:32:17 +02:00
Dimitri Fontaine
24db93affd Fix handling of typemod, in particular with numeric data types. 2013-09-19 22:44:02 +02:00
Dimitri Fontaine
be0738d70a Implement casting of MySQL ENUM types. 2013-09-19 15:04:56 +02:00
Dimitri Fontaine
e6d4c73c1b Make Xach's db3 lib into its own asdf piece and integrate it with pgloader. 2013-09-19 00:42:35 +02:00
Dimitri Fontaine
bd6004e6b5 M-x delete-trailing-whitespace 2013-09-18 21:58:49 +02:00
Dimitri Fontaine
4aeec53ab4 Fix the syslog messages consumer to actually fill in data queues. 2013-09-18 17:41:27 +02:00
Dimitri Fontaine
b9d0edb0ee Fix pgloader.pgsql:get-connection-spec for port handling. 2013-09-18 17:40:43 +02:00
Dimitri Fontaine
0d43c31164 Have pgloader.syslog actually send data to PostgreSQL. 2013-09-18 15:38:55 +02:00
Dimitri Fontaine
20bbf827a3 Review LOAD MESSAGES FROM syntax to allow multiple grammars. 2013-09-18 13:11:04 +02:00
Dimitri Fontaine
ee1655b9cb Parse the new syslog facility command, and fix usage of semicolons in passing. 2013-09-17 18:20:59 +02:00
Dimitri Fontaine
eb04b34208 Remove unused files. 2013-09-17 10:57:47 +02:00
Dimitri Fontaine
b83228fa3e Small documentation enhancement. 2013-09-09 22:32:29 +02:00
Dimitri Fontaine
9fa7891346 Add an example of start-syslog-server usage as comments. 2013-09-09 22:31:43 +02:00