13 Commits

Author SHA1 Message Date
Dimitri Fontaine
8476c1a359 Allow setting search_path with multiple schemas.
The PostgreSQL search_path allows multiple schemas and might even need
it to be able to reference types and other tables. Allow setting more
than one schema by using the fact that PostgreSQL schema names don't
need to be individually quoted, and passing down the exact content of
the SET search_path value down to PostgreSQL.

Fix #359.
2016-03-20 20:54:08 +01:00
Dimitri Fontaine
c724018840 Implement ALTER TABLE clause for MySQL migrations.
The new ALTER TABLE facility allows to act on tables found in the MySQL
database before the migration happens. In this patch the only provided
actions are RENAME TO and SET SCHEMA, which fixes #224.

In order to be able to provide the same option for MS SQL users, we will
have to make it work at the SCHEMA level (ALTER SCHEMA ... RENAME TO
...) and modify the internal schema-struct so that the schema slot of
our table instances are a schema instance rather than its name.

Lacking MS SQL test database and instance, the facility is not yet
provided for that source type.
2016-03-06 21:51:33 +01:00
Dimitri Fontaine
eb45bf0338 Expose concurrency settings to the end users.
Add the workers and concurrency settings to the LOAD commands for
database sources so that users can tweak them now, and add mentions of
them in the documentation too.

From the documentation string of the copy-from method as found in
src/sources/common/methods.lisp:

   We allow WORKER-COUNT simultaneous workers to be active at the same time
   in the context of this COPY object. A single unit of work consist of
   several kinds of workers:

     - a reader getting raw data from the COPY source with `map-rows',
     - N transformers preparing raw data for PostgreSQL COPY protocol,
     - N writers sending the data down to PostgreSQL.

   The N here is setup to the CONCURRENCY parameter: with a CONCURRENCY of
   2, we start (+ 1 2 2) = 5 concurrent tasks, with a CONCURRENCY of 4 we
   start (+ 1 4 4) = 9 concurrent tasks, of which only WORKER-COUNT may be
   active simultaneously.

Those options should find their way in the remaining sources, that's for
a follow-up patch tho.
2016-01-15 23:22:32 +01:00
Dimitri Fontaine
d4b58a1f78 Review MySQL default options.
The default values for MySQL WITH clause options wasn't really tested
and broke on simple cases, the new set of defaults is known to work in
many cases (most?).

Other combinations of options will need some review work, and we might
need to consider preventing some of them, that's for another patch tho.
2014-09-21 12:19:20 -05:00
Dimitri Fontaine
a2370938b6 MATERIALIZE ALL VIEWS.
Complete the MySQL migration feature.
2014-05-26 18:03:50 +02:00
Dimitri Fontaine
4bc4cc53a7 Add support for connecting to PostgreSQL with Unix Domain Sockets, fixing #15. 2013-11-22 17:19:48 +01:00
Dimitri Fontaine
8fa7a86013 Implement typemod and default guards in the MySQL CAST clause. 2013-11-17 22:33:06 +01:00
Dimitri Fontaine
718ac80560 Implement a Materiaze Views option for MySQL. 2013-11-16 21:41:07 +01:00
Dimitri Fontaine
fe75f3df54 Improve the set of options for loading from databases, assorted fixes. 2013-11-05 23:07:03 +01:00
Dimitri Fontaine
450c06b6fb Add a "with foreign keys" option to the MySQL Command. 2013-11-05 19:16:12 +01:00
Dimitri Fontaine
6593995832 Add support for BEFORE LOAD DO and FINALLY DO blocks in the MySQL command. 2013-11-05 17:07:13 +01:00
Dimitri Fontaine
2477b68e15 Implement filtering against the list of tables to migrate, in MySQL. 2013-11-05 14:48:05 +01:00
Dimitri Fontaine
561eb29175 Add a test case for the sakila database. 2013-11-04 21:24:23 +01:00