The current instructions build SBCL then pgloader all from sources in a
debian VM. It might be a good idea to maintain a debian+SBCL docker
image and build pgloader on top of that.
Handling the errors within the thread is useful when debugging pgloader
interactively, but not so much when started from the command line, where
it would hand the program forever with threads waiting for interactive
debugger actions to be taken, with no way for the user to actually take
action.
First, the index names in MS SQL, as in MySQL, are only unique per
table, whereas they need to be globally unique (per database) in
PostgreSQL. So reuse the infrastructure we had for MySQL here.
Second, the way we trick table names in index and fkey structures means
that we already did quote the names and we don't want to quote them
again, so add a new possible *identifier-case* value to handle the case
where nothing is to be done, pretty please.
Rather than doing ALTER TABLE directly, use CREATE UNIQUE INDEX in the
all in parallel concurrent index build per table, and only in the end
game "upgrade" that unique index into a PRIMARY KEY using ALTER TABLE.
The reason why it's a good idea to do that is to avoid an ACCESS
EXCLUSIVE LOCK at ALTER TABLE time, which is killing our index build
concurrency.
We got unsync again with the qmynd driver (see #124) and the WIP branch
of esrap has been in the works for a long time now. We now manually
fetch the "proper" version of those.
When querying the default values of MySQL tables against MySQL catalogs,
the default value is always returned as a string. When the column having
the default actually is a "binary" column, we want the transformation
functions to receive a proper vector of bytes.
This patch adds some hard-coded rules to be smarted about the situation
for columns of type "binary".
Now it's possible to parse a command to load data from MS SQL. The
parser was until now parsing all database URI within the same common
rule and that isn't possible anymore if we want to distinguish in
between source database right from the parser, which we actually want to
do.
This patch also implement in-passing fixes all over the place, including
the transformation function float-to-string that only happened to work
on double-float data.
Converting the table definitions (with type casting) seems to work. Also
did experiment a little with actuallt fetching some data... and had to
edit the cl-mssql driver, which is temporarily monkey patched.
MySQL ENUM syntax is more complex than it first looked, allowing for
escaping and spaces, so use a more complex parser here. Fortunately, the
syntax looks like a CSV row enough that cl-csv can be re-used for that.
This allows users to benefit from the same flexible machinery when using
SQLite as when using MySQL, and also allows to add some more default
cast rules too.
The metabang-bind lib offers a nice bind macro that solves the problem
of ignoring bindings in destructuring-bind, and allows a let* approach
to nested destructuring (wven when mixed with let declarations).
Using that lib (that we already indirectly depend on anyway) simplifies
the parser code substantially.
