Commit Graph

13 Commits

Author SHA1 Message Date
Dimitri Fontaine
a392328dad Allow any ordering of guards and extra cast rule clauses.
It used to be that extra were forced to being parsed before guards, but
there's no reason why a user wouldn't think to write its clauses the other
way round, so add support for that as well.

See #779.
2018-04-29 19:00:20 +02:00
Dimitri Fontaine
e4dca1a086 Implement support for MySQL useSSL=true|false option.
The MySQL connection string parameter for SSL usage is useSSL, so map an
option name to our expected values for sslmode in database connection
strings.

See #748.
2018-03-16 16:41:40 +01:00
Dimitri Fontaine
4612e68435 Implement support for new casting rules guards and actions.
Namely the actions are “keep extra” and “drop extra” and the casting rule
guard is “with extra on update current timestamp”. Having support for those
elements in the casting rules allow such a definition as the following:

      type timestamp with extra on update current timestamp
        to "timestamp with time zone" drop extra

The effect of such as cast rule would be to ignore the MySQL extra
definition and then refrain pgloader from creating the PostgreSQL triggers
that implement the same behavior.

Fix #735.
2018-01-31 15:17:05 +01:00
Dimitri Fontaine
7b08b6e3d3 Refrain from creating tables in “data only” operations.
We forgot that rule in the case of creating the target tables for the
materializing views commands, which led to surprising and wrong behavior.

Fix #721, and add a new test case while at it.
2018-01-25 23:32:31 +01:00
Dimitri Fontaine
07cdf3e7e5 Use MySQL column names in MySQL queries.
The query for concurrency-support didn't get the memo that we should ignore
PostgreSQL identifier-case when querying the source MySQL database. Fix the
query string to include column names as given by the MySQL catalogs.

In bug report #703, the problem is found in PostgreSQL queries. This has
been fixed before already. Trying to reproduce the bug produced an error in
the concurrency-support query instead, so let's fix this one.

Fix #703.
2017-12-22 14:15:46 +01:00
Dimitri Fontaine
b7d87a9eb1 Fix MySQL bit(1) casting function.
When this function was written, pgloader would get an array of numbers over
the wire, nowadays it looks like it's receiving an array of characters
instead (in other words, a string).

Improve the `bits-to-boolean` function to accept either input, and raise an
error in another case.

My theory is that something changed either in MySQL (with version 10) or in
the Qmynd driver somehow... but tonight we just go easy and fix the bug
locally rather than try and understand where it might be coming from.

Fixes #684.
2017-12-03 23:06:54 +01:00
Dimitri Fontaine
5c60f8c35c Implement a new type casting guard: unsigned.
MySQL allows using unsigned data types and pgloader should then target a
signed type of a larger capacity so that values can fit. For example, the
data definition “smallint(5) unsigned” should be casted to “integer”.

This patch allows user defined cast rules to be written against “unsigned”
data types as per their MySQL catalog representation.

See #678.
2017-11-22 10:26:03 -08:00
Dimitri Fontaine
1d7706c045 Fix the MySQL encoding error handling.
The error handling would try and read past the error buffer in some cases,
when the BABEL lib would give a position that's after the buffer read.

Fix #661.
2017-11-13 11:27:47 +01:00
Dimitri Fontaine
a9afddf8ed Accept quoted namestrings as target type names for cast rules.
This allows passing "double precision" rather than float8, for example.

Fix #650.
2017-10-21 21:03:58 +02:00
Dimitri Fontaine
b36f36b74e Add a (local) test case. 2017-10-16 17:25:44 +02:00
Dimitri Fontaine
460fe6cc77 Fix quoting of default values for MariaDB 10 support.
The default values quoting changed in MariaDB 10, and we need to adjust in
pgloader: extra '' chars could defeat the default matching logic:

  "'0000-00-00'" is different from "0000-00-00"
2017-09-19 11:29:53 +02:00
Dimitri Fontaine
8a361a0ff8 Add support for multiple on update columns per table.
The MySQL special syntax "on update current_timestamp()" used to support
only a single column per table (in MySQL), and so did pgloader. In MariaDB
version 10 it's now possible to have several column with that special
treatment, so adapt pgloader to migrate that too.

What pgloader does is recognize that several columns are to receive the same
pre-update processing, and creates a single function that does the both of
them, as in the following example, from pgloader logs in a test case:

    CREATE OR REPLACE FUNCTION mysql.on_update_current_timestamp_onupdate()
      RETURNS trigger
      LANGUAGE plpgsql
      AS
    $$
    BEGIN
       NEW.update_date = now();
       NEW.calc_date = now();
       RETURN NEW;
    END;
    $$;
    CREATE TRIGGER on_update_current_timestamp
            BEFORE UPDATE ON mysql.onupdate
          FOR EACH ROW
      EXECUTE PROCEDURE mysql.on_update_current_timestamp_onupdate();

Fixes #629.
2017-09-15 01:04:57 +02:00
Dimitri Fontaine
b7347a567c Add test cases for MySQL.
At the moment it's a very manual process, and it might get automated
someday. Meanwhile it's still useful to have.

See #569 for an issue that got a test case added.
2017-09-14 15:59:10 +02:00