pgloader/test/csv-filename-pattern.load
Dimitri Fontaine 7dd69a11e1 Implement concurrency and workers for files sources.
More than the syntax and API tweaks, this patch also make it so that a
multi-file specification (using e.g. ALL FILENAMES IN DIRECTORY) can be
loaded with several files in the group in parallel.

To that effect, tweak again the md-connection and md-copy
implementations.
2016-01-16 22:53:55 +01:00

16 lines
423 B
Fish

load csv
from all filenames matching ~<matching.*csv$>
in directory 'data'
having fields (id, field)
into postgres:///pgloader?matching
with fields optionally enclosed by '"',
fields terminated by ',',
truncate,
workers = 8,
concurrency = 1
before load do
$$ drop table if exists matching; $$,
$$ create table matching(id int, field text); $$;