Docs cleanup.

Don't maintain generated files in git, it's useless (thanks mainly to
readthedocs), also remove the previous format of the docs.

.gitignore

@@ -16,3 +16,4 @@ web/howto/quickstart.html
 web/howto/sqlite.html
 .DS_Store
 system-index.txt
+docs/_build

docs/_build/html/.buildinfo

@@ -1,4 +0,0 @@
-# Sphinx build info version 1
-# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 59392f424cbb8621f1b34cf4df7fbec1
-tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/_build/html/_sources/bugreport.rst.txt

@@ -1,49 +0,0 @@
-Reporting Bugs
-==============
-
-pgloader is a software and as such contains bugs. Most bugs are easy to
-solve and taken care of in a short delay. For this to be possible though,
-bug reports need to follow those recommandations:
-
-  - include pgloader version,
-  - include problematic input and output,
-  - include a description of the output you expected,
-  - explain the difference between the ouput you have and the one you expected,
-  - include a self-reproducing test-case
-
-Test Cases to Reproduce Bugs
-----------------------------
-
-Use the *inline* source type to help reproduce a bug, as in the pgloader tests::
-
-  LOAD CSV
-     FROM INLINE
-     INTO postgresql://dim@localhost/pgloader?public."HS"
-
-     WITH truncate,
-          fields terminated by '\t',
-          fields not enclosed,
-          fields escaped by backslash-quote,
-          quote identifiers
-
-      SET work_mem to '128MB',
-          standard_conforming_strings to 'on',
-          application_name to 'my app name'
-
-   BEFORE LOAD DO
-    $$ create extension if not exists hstore; $$,
-    $$ drop table if exists "HS"; $$,
-    $$ CREATE TABLE "HS"
-       (
-          id     serial primary key,
-          kv     hstore
-       )
-    $$;
-  
-  
-  1	email=>foo@example.com,a=>b
-  2	test=>value
-  3	a=>b,c=>"quoted hstore value",d=>other
-  4	baddata
-
-

docs/_build/html/_sources/index.rst.txt

@@ -1,33 +0,0 @@
-.. pgloader documentation master file, created by
-   sphinx-quickstart on Tue Dec  5 19:23:32 2017.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-Welcome to pgloader's documentation!
-====================================
-
-.. toctree::
-   :maxdepth: 2
-   :caption: Table Of Contents:
-
-   intro
-   tutorial/tutorial
-   pgloader
-   ref/csv
-   ref/fixed
-   ref/copy
-   ref/dbf
-   ref/ixf
-   ref/archive
-   ref/mysql
-   ref/sqlite
-   ref/mssql
-   ref/transforms
-   bugreport
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`

docs/_build/html/_sources/intro.rst.txt

@@ -1,63 +0,0 @@
-Introduction
-============
-
-pgloader loads data from various sources into PostgreSQL. It can
-transform the data it reads on the fly and submit raw SQL before and
-after the loading.  It uses the `COPY` PostgreSQL protocol to stream
-the data into the server, and manages errors by filling a pair of
-*reject.dat* and *reject.log* files.
-
-pgloader knows how to read data from different kind of sources:
-
-  * Files
-    * CSV
-    * Fixed Format
-    * DBF
-  * Databases
-    * SQLite
-    * MySQL
-    * MS SQL Server
-
-The level of automation provided by pgloader depends on the data source
-type. In the case of CSV and Fixed Format files, a full description of the
-expected input properties must be given to pgloader. In the case of a
-database, pgloader connects to the live service and knows how to fetch the
-metadata it needs directly from it.
-
-Continuous Migration
---------------------
-
-pgloader is meant to migrate a whole database in a single command line and
-without any manual intervention. The goal is to be able to setup a
-*Continuous Integration* environment as described in the `Project
-Methodology <http://mysqltopgsql.com/project/>`_ document of the `MySQL to
-PostgreSQL <http://mysqltopgsql.com/project/>`_ webpage.
-
-  1. Setup your target PostgreSQL Architecture
-  2. Fork a Continuous Integration environment that uses PostgreSQL
-  3. Migrate the data over and over again every night, from production
-  4. As soon as the CI is all green using PostgreSQL, schedule the D-Day
-  5. Migrate without suprise and enjoy! 
-
-In order to be able to follow this great methodology, you need tooling to
-implement the third step in a fully automated way. That's pgloader.
-
-Commands
---------
-
-pgloader implements its own *Command Language*, a DSL that allows to specify
-every aspect of the data load and migration to implement. Some of the
-features provided in the language are only available for a specific source
-type.
-
-Command Line
-------------
-
-The pgloader command line accepts those two variants::
-
-    pgloader [<options>] [<command-file>]...
-    pgloader [<options>] SOURCE TARGET
-
-Either you have a *command-file* containing migration specifications in the
-pgloader *Command Language*, or you can give a *Source* for the data and a
-PostgreSQL database connection *Target* where to load the data into.

docs/_build/html/_sources/pgloader-usage-examples.rst.txt

@@ -1,163 +0,0 @@
-Pgloader Usage Examples
-=======================
-
-Currently not included, because redundant with the tutorial.
-
-Usage Examples
---------------
-
-Review the command line options and pgloader's version::
-
-    pgloader --help
-    pgloader --version
-
-Loading from a complex command
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Use the command file as the pgloader command argument, pgloader will parse
-that file and execute the commands found in it::
-
-    pgloader --verbose ./test/csv-districts.load
-
-CSV
-^^^
-
-Load data from a CSV file into a pre-existing table in your database, having
-pgloader guess the CSV properties (separator, quote and escape character)::
-
-    pgloader ./test/data/matching-1.csv pgsql:///pgloader?tablename=matching
-
-Load data from a CSV file into a pre-existing table in your database, with
-expanded options::
-
-    pgloader --type csv                                   \
-             --field id --field field                     \
-             --with truncate                              \
-             --with "fields terminated by ','"            \
-             ./test/data/matching-1.csv                   \
-             postgres:///pgloader?tablename=matching
-
-In that example the whole loading is driven from the command line, bypassing
-the need for writing a command in the pgloader command syntax entirely. As
-there's no command though, the extra inforamtion needed must be provided on
-the command line using the `--type` and `--field` and `--with` switches.
-
-For documentation about the available syntaxes for the `--field` and
-`--with` switches, please refer to the CSV section later in the man page.
-
-Note also that the PostgreSQL URI includes the target *tablename*.
-
-Reading from STDIN
-^^^^^^^^^^^^^^^^^^
-
-File based pgloader sources can be loaded from the standard input, as in the
-following example::
-
-    pgloader --type csv                                         \
-             --field "usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong" \
-             --with "skip header = 1"                          \
-             --with "fields terminated by '\t'"                \
-             -                                                 \
-             postgresql:///pgloader?districts_longlat          \
-             < test/data/2013_Gaz_113CDs_national.txt
-
-The dash (`-`) character as a source is used to mean *standard input*, as
-usual in Unix command lines. It's possible to stream compressed content to
-pgloader with this technique, using the Unix pipe:
-
-    gunzip -c source.gz | pgloader --type csv ... - pgsql:///target?foo
-
-Loading from CSV available through HTTP
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The same command as just above can also be run if the CSV file happens to be
-found on a remote HTTP location::
-
-    pgloader --type csv                                                     \
-             --field "usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong" \
-             --with "skip header = 1"                                       \
-             --with "fields terminated by '\t'"                             \
-             http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt     \
-             postgresql:///pgloader?districts_longlat
-
-Some more options have to be used in that case, as the file contains a
-one-line header (most commonly that's column names, could be a copyright
-notice). Also, in that case, we specify all the fields right into a single
-`--field` option argument.
-
-Again, the PostgreSQL target connection string must contain the *tablename*
-option and you have to ensure that the target table exists and may fit the
-data. Here's the SQL command used in that example in case you want to try it
-yourself::
-
-    create table districts_longlat
-    (
-             usps        text,
-             geoid       text,
-             aland       bigint,
-             awater      bigint,
-             aland_sqmi  double precision,
-             awater_sqmi double precision,
-             intptlat    double precision,
-             intptlong   double precision
-    );
-
-Also notice that the same command will work against an archived version of
-the same data, e.g.
-http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz.
-
-Finally, it's important to note that pgloader first fetches the content from
-the HTTP URL it to a local file, then expand the archive when it's
-recognized to be one, and only then processes the locally expanded file.
-
-In some cases, either because pgloader has no direct support for your
-archive format or maybe because expanding the archive is not feasible in
-your environment, you might want to *stream* the content straight from its
-remote location into PostgreSQL. Here's how to do that, using the old battle
-tested Unix Pipes trick::
-
-    curl http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz \
-    | gunzip -c                                                        \
-    | pgloader --type csv                                              \
-               --field "usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong"
-               --with "skip header = 1"                                \
-               --with "fields terminated by '\t'"                      \
-               -                                                       \
-               postgresql:///pgloader?districts_longlat
-
-Now the OS will take care of the streaming and buffering between the network
-and the commands and pgloader will take care of streaming the data down to
-PostgreSQL.
-
-Migrating from SQLite
-^^^^^^^^^^^^^^^^^^^^^
-
-The following command will open the SQLite database, discover its tables
-definitions including indexes and foreign keys, migrate those definitions
-while *casting* the data type specifications to their PostgreSQL equivalent
-and then migrate the data over::
-
-    createdb newdb
-    pgloader ./test/sqlite/sqlite.db postgresql:///newdb
-
-Migrating from MySQL
-^^^^^^^^^^^^^^^^^^^^
-
-Just create a database where to host the MySQL data and definitions and have
-pgloader do the migration for you in a single command line::
-
-    createdb pagila
-    pgloader mysql://user@localhost/sakila postgresql:///pagila
-
-Fetching an archived DBF file from a HTTP remote location
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-It's possible for pgloader to download a file from HTTP, unarchive it, and
-only then open it to discover the schema then load the data::
-
-    createdb foo
-    pgloader --type dbf http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/historiq2013.zip postgresql:///foo
-
-Here it's not possible for pgloader to guess the kind of data source it's
-being given, so it's necessary to use the `--type` command line switch.
-

docs/_build/html/_sources/pgloader.rst.txt

@@ -1,713 +0,0 @@
-PgLoader Reference Manual
-=========================
-
-pgloader loads data from various sources into PostgreSQL. It can
-transform the data it reads on the fly and submit raw SQL before and
-after the loading.  It uses the `COPY` PostgreSQL protocol to stream
-the data into the server, and manages errors by filling a pair of
-*reject.dat* and *reject.log* files.
-
-pgloader operates either using commands which are read from files::
-
-    pgloader commands.load
-
-or by using arguments and options all provided on the command line::
-
-    pgloader SOURCE TARGET
-
-Arguments
----------
-
-The pgloader arguments can be as many load files as needed, or a couple of
-connection strings to a specific input file.
-
-Source Connection String
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-The source connection string format is as follows::
-
-    format:///absolute/path/to/file.ext
-    format://./relative/path/to/file.ext
-
-Where format might be one of `csv`, `fixed`, `copy`, `dbf`, `db3` or `ixf`.::
-
-    db://user:pass@host:port/dbname
-
-Where db might be of `sqlite`, `mysql` or `mssql`.
-
-When using a file based source format, pgloader also support natively
-fetching the file from an http location and decompressing an archive if
-needed. In that case it's necessary to use the `--type` option to specify
-the expected format of the file. See the examples below.
-
-Also note that some file formats require describing some implementation
-details such as columns to be read and delimiters and quoting when loading
-from csv.
-
-For more complex loading scenarios, you will need to write a full fledge
-load command in the syntax described later in this document.
-
-Target Connection String
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-The target connection string format is described in details later in this
-document, see Section Connection String.
-
-Options
--------
-
-Inquiry Options
-^^^^^^^^^^^^^^^
-
-Use these options when you want to know more about how to use `pgloader`, as
-those options will cause `pgloader` not to load any data.
-
-  * `-h`, `--help`
-
-    Show command usage summary and exit.
-
-  * `-V`, `--version`
-
-    Show pgloader version string and exit.
-
-  * `-E`, `--list-encodings`
-
-    List known encodings in this version of pgloader.
-
-  * `-U`, `--upgrade-config`
-    
-    Parse given files in the command line as `pgloader.conf` files with the
-    `INI` syntax that was in use in pgloader versions 2.x, and output the
-    new command syntax for pgloader on standard output.
-
-
-General Options
-^^^^^^^^^^^^^^^
-
-Those options are meant to tweak `pgloader` behavior when loading data.
-
-  * `-v`, `--verbose`
-    
-    Be verbose.
-
-  * `-q`, `--quiet`
-    
-    Be quiet.
-
-  * `-d`, `--debug`
-    
-    Show debug level information messages.
-
-  * `-D`, `--root-dir`
-    
-    Set the root working directory (default to "/tmp/pgloader").
-
-  * `-L`, `--logfile`
-    
-    Set the pgloader log file (default to "/tmp/pgloader.log").
-
-  * `--log-min-messages`
-    
-    Minimum level of verbosity needed for log message to make it to the
-    logfile. One of critical, log, error, warning, notice, info or debug.
-
-  * `--client-min-messages`
-    
-    Minimum level of verbosity needed for log message to make it to the
-    console. One of critical, log, error, warning, notice, info or debug.
-
-  * `-S`, `--summary`
-    
-    A filename where to copy the summary output. When relative, the filename
-    is expanded into `*root-dir*`.
-
-    The format of the filename defaults to being *human readable*. It is
-    possible to have the output in machine friendly formats such as *CSV*,
-    *COPY* (PostgreSQL's own COPY format) or *JSON* by specifying a filename
-    with the extension resp. `.csv`, `.copy` or `.json`.
-
-  * `-l <file>`, `--load-lisp-file <file>`
-    
-    Specify a lisp <file> to compile and load into the pgloader image before
-    reading the commands, allowing to define extra transformation function.
-    Those functions should be defined in the `pgloader.transforms` package.
-    This option can appear more than once in the command line.
-
-  * `--dry-run`
-
-    Allow testing a `.load` file without actually trying to load any data.
-    It's useful to debug it until it's ok, in particular to fix connection
-    strings.
-
-  * `--on-error-stop`
-
-    Alter pgloader behavior: rather than trying to be smart about error
-    handling and continue loading good data, separating away the bad one,
-    just stop as soon as PostgreSQL refuses anything sent to it. Useful to
-    debug data processing, transformation function and specific type
-    casting.
-
-  * `--self-upgrade <directory>`
-
-    Specify a <directory> where to find pgloader sources so that one of the
-    very first things it does is dynamically loading-in (and compiling to
-    machine code) another version of itself, usually a newer one like a very
-    recent git checkout.
-
-Command Line Only Operations
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Those options are meant to be used when using `pgloader` from the command
-line only, rather than using a command file and the rich command clauses and
-parser. In simple cases, it can be much easier to use the *SOURCE* and
-*TARGET* directly on the command line, then tweak the loading with those
-options:
-
-  * `--with "option"`
-
-    Allows setting options from the command line. You can use that option as
-    many times as you want. The option arguments must follow the *WITH*
-    clause for the source type of the `SOURCE` specification, as described
-    later in this document.
-
-  * `--set "guc_name='value'"`
-
-    Allows setting PostgreSQL configuration from the command line. Note that
-    the option parsing is the same as when used from the *SET* command
-    clause, in particular you must enclose the guc value with single-quotes.
-
-  * `--field "..."`
-
-    Allows setting a source field definition. Fields are accumulated in the
-    order given on the command line. It's possible to either use a `--field`
-    option per field in the source file, or to separate field definitions by
-    a comma, as you would do in the *HAVING FIELDS* clause.
-
-  * `--cast "..."`
-
-    Allows setting a specific casting rule for loading the data.
-
-  * `--type csv|fixed|db3|ixf|sqlite|mysql|mssql`
-
-    Allows forcing the source type, in case when the *SOURCE* parsing isn't
-    satisfying.
-
-  * `--encoding <encoding>`
-
-    Set the encoding of the source file to load data from.
-
-  * `--before <filename>`
-
-    Parse given filename for SQL queries and run them against the target
-    database before loading the data from the source. The queries are parsed
-    by pgloader itself: they need to be terminated by a semi-colon (;) and
-    the file may include `\i` or `\ir` commands to *include* another file.
-
-  * `--after <filename>`
-
-    Parse given filename for SQL queries and run them against the target
-    database after having loaded the data from the source. The queries are
-    parsed in the same way as with the `--before` option, see above.
-
-More Debug Information
-^^^^^^^^^^^^^^^^^^^^^^
-
-To get the maximum amount of debug information, you can use both the
-`--verbose` and the `--debug` switches at the same time, which is equivalent
-to saying `--client-min-messages data`. Then the log messages will show the
-data being processed, in the cases where the code has explicit support for
-it.
-
-Batches And Retry Behaviour
----------------------------
-
-To load data to PostgreSQL, pgloader uses the `COPY` streaming protocol.
-While this is the faster way to load data, `COPY` has an important drawback:
-as soon as PostgreSQL emits an error with any bit of data sent to it,
-whatever the problem is, the whole data set is rejected by PostgreSQL.
-
-To work around that, pgloader cuts the data into *batches* of 25000 rows
-each, so that when a problem occurs it's only impacting that many rows of
-data. Each batch is kept in memory while the `COPY` streaming happens, in
-order to be able to handle errors should some happen.
-
-When PostgreSQL rejects the whole batch, pgloader logs the error message
-then isolates the bad row(s) from the accepted ones by retrying the batched
-rows in smaller batches. To do that, pgloader parses the *CONTEXT* error
-message from the failed COPY, as the message contains the line number where
-the error was found in the batch, as in the following example::
-
-    CONTEXT: COPY errors, line 3, column b: "2006-13-11"
-
-Using that information, pgloader will reload all rows in the batch before
-the erroneous one, log the erroneous one as rejected, then try loading the
-remaining of the batch in a single attempt, which may or may not contain
-other erroneous data.
-
-At the end of a load containing rejected rows, you will find two files in
-the *root-dir* location, under a directory named the same as the target
-database of your setup. The filenames are the target table, and their
-extensions are `.dat` for the rejected data and `.log` for the file
-containing the full PostgreSQL client side logs about the rejected data.
-
-The `.dat` file is formatted in PostgreSQL the text COPY format as documented
-in `http://www.postgresql.org/docs/9.2/static/sql-copy.html#AEN66609`.
-
-A Note About Performance
-------------------------
-
-pgloader has been developed with performance in mind, to be able to cope
-with ever growing needs in loading large amounts of data into PostgreSQL.
-
-The basic architecture it uses is the old Unix pipe model, where a thread is
-responsible for loading the data (reading a CSV file, querying MySQL, etc)
-and fills pre-processed data into a queue. Another threads feeds from the
-queue, apply some more *transformations* to the input data and stream the
-end result to PostgreSQL using the COPY protocol.
-
-When given a file that the PostgreSQL `COPY` command knows how to parse, and
-if the file contains no erroneous data, then pgloader will never be as fast
-as just using the PostgreSQL `COPY` command.
-
-Note that while the `COPY` command is restricted to read either from its
-standard input or from a local file on the server's file system, the command
-line tool `psql` implements a `\copy` command that knows how to stream a
-file local to the client over the network and into the PostgreSQL server,
-using the same protocol as pgloader uses.
-
-A Note About Parallelism
-------------------------
-
-pgloader uses several concurrent tasks to process the data being loaded:
-
-  - a reader task reads the data in and pushes it to a queue,
-  
-  - at last one write task feeds from the queue and formats the raw into the
-    PostgreSQL COPY format in batches (so that it's possible to then retry a
-    failed batch without reading the data from source again), and then sends
-    the data to PostgreSQL using the COPY protocol.
-
-The parameter *workers* allows to control how many worker threads are
-allowed to be active at any time (that's the parallelism level); and the
-parameter *concurrency* allows to control how many tasks are started to
-handle the data (they may not all run at the same time, depending on the
-*workers* setting).
-
-We allow *workers* simultaneous workers to be active at the same time in the
-context of a single table. A single unit of work consist of several kinds of
-workers:
-
-  - a reader getting raw data from the source,
-  - N writers preparing and sending the data down to PostgreSQL.
-
-The N here is setup to the *concurrency* parameter: with a *CONCURRENCY* of
-2, we start (+ 1 2) = 3 concurrent tasks, with a *concurrency* of 4 we start
-(+ 1 4) = 9 concurrent tasks, of which only *workers* may be active
-simultaneously.
-
-The defaults are `workers = 4, concurrency = 1` when loading from a database
-source, and `workers = 8, concurrency = 2` when loading from something else
-(currently, a file). Those defaults are arbitrary and waiting for feedback
-from users, so please consider providing feedback if you play with the
-settings.
-
-As the `CREATE INDEX` threads started by pgloader are only waiting until
-PostgreSQL is done with the real work, those threads are *NOT* counted into
-the concurrency levels as detailed here.
-
-By default, as many `CREATE INDEX` threads as the maximum number of indexes
-per table are found in your source schema. It is possible to set the `max
-parallel create index` *WITH* option to another number in case there's just
-too many of them to create.
-
-Source Formats
---------------
-
-pgloader supports the following input formats:
-
-  - csv, which includes also tsv and other common variants where you can
-    change the *separator* and the *quoting* rules and how to *escape* the
-    *quotes* themselves;
-
-  - fixed columns file, where pgloader is flexible enough to accomodate with
-    source files missing columns (*ragged fixed length column files* do
-    exist);
-
-  - PostgreSLQ COPY formatted files, following the COPY TEXT documentation
-    of PostgreSQL, such as the reject files prepared by pgloader;
-
-  - dbase files known as db3 or dbf file;
-
-  - ixf formated files, ixf being a binary storage format from IBM;
-
-  - sqlite databases with fully automated discovery of the schema and
-    advanced cast rules;
-
-  - mysql databases with fully automated discovery of the schema and
-    advanced cast rules;
-
-  - MS SQL databases with fully automated discovery of the schema and
-    advanced cast rules.
-
-Pgloader Commands Syntax
-------------------------
-
-pgloader implements a Domain Specific Language allowing to setup complex
-data loading scripts handling computed columns and on-the-fly sanitization
-of the input data. For more complex data loading scenarios, you will be
-required to learn that DSL's syntax. It's meant to look familiar to DBA by
-being inspired by SQL where it makes sense, which is not that much after
-all.
-
-The pgloader commands follow the same global grammar rules. Each of them
-might support only a subset of the general options and provide specific
-options.
-
-::
-
-    LOAD <source-type>
-	     FROM <source-url>
-           [ HAVING FIELDS <source-level-options> ]
-		 INTO <postgresql-url>
-           [ TARGET TABLE [ "<schema>" ]."<table name>" ]
-           [ TARGET COLUMNS <columns-and-options> ]
-
-	[ WITH <load-options> ]
-
-	[ SET <postgresql-settings> ]
-
-    [ BEFORE LOAD [ DO <sql statements> | EXECUTE <sql file> ] ... ]
-    [  AFTER LOAD [ DO <sql statements> | EXECUTE <sql file> ] ... ]
-	;
-
-The main clauses are the `LOAD`, `FROM`, `INTO` and `WITH` clauses that each
-command implements. Some command then implement the `SET` command, or some
-specific clauses such as the `CAST` clause.
-
-Templating with Mustache
-------------------------
-
-pgloader implements the https://mustache.github.io/ templating system so
-that you may have dynamic parts of your commands. See the documentation for
-this template system online.
-
-A specific feature of pgloader is the ability to fetch a variable from the
-OS environment of the pgloader process, making it possible to run pgloader
-as in the following example::
-
-    $ DBPATH=sqlite/sqlite.db pgloader ./test/sqlite-env.load
-    
-or in several steps::
-
-    $ export DBPATH=sqlite/sqlite.db
-    $ pgloader ./test/sqlite-env.load
-
-The variable can then be used in a typical mustache fashion::
-
-    load database
-         from '{{DBPATH}}'
-         into postgresql:///pgloader;
-
-It's also possible to prepare a INI file such as the following::
-
-    [pgloader]
-    
-    DBPATH = sqlite/sqlite.db
-
-And run the following command, feeding the INI values as a *context* for
-pgloader templating system::
-
-    $ pgloader --context ./test/sqlite.ini ./test/sqlite-ini.load
-
-The mustache templates implementation with OS environment support replaces
-former `GETENV` implementation, which didn't work anyway.
-
-Common Clauses
---------------
-
-Some clauses are common to all commands:
-
-FROM
-^^^^
-
-The *FROM* clause specifies where to read the data from, and each command
-introduces its own variant of sources. For instance, the *CSV* source
-supports `inline`, `stdin`, a filename, a quoted filename, and a *FILENAME
-MATCHING* clause (see above); whereas the *MySQL* source only supports a
-MySQL database URI specification.
-
-INTO
-^^^^
-
-The PostgreSQL connection URI must contains the name of the target table
-where to load the data into. That table must have already been created in
-PostgreSQL, and the name might be schema qualified.
-
-Then *INTO* option also supports an optional comma separated list of target
-columns, which are either the name of an input *field* or the white space
-separated list of the target column name, its PostgreSQL data type and a
-*USING* expression.
-
-The *USING* expression can be any valid Common Lisp form and will be read
-with the current package set to `pgloader.transforms`, so that you can use
-functions defined in that package, such as functions loaded dynamically with
-the `--load` command line parameter.
-
-Each *USING* expression is compiled at runtime to native code.
-
-This feature allows pgloader to load any number of fields in a CSV file into
-a possibly different number of columns in the database, using custom code
-for that projection.
-
-WITH
-^^^^
-
-Set of options to apply to the command, using a global syntax of either:
-
-   - *key = value*
-   - *use option*
-   - *do not use option*
-
-See each specific command for details.
-
-All data sources specific commands support the following options:
-
-  - *on error stop*
-  - *batch rows = R*
-  - *batch size = ... MB*
-  - *prefetch rows = ...*
-
-See the section BATCH BEHAVIOUR OPTIONS for more details.
-
-In addition, the following settings are available:
-
-   - *workers = W*
-   - *concurrency = C*
-   - *max parallel create index = I*
-
-See section A NOTE ABOUT PARALLELISM for more details.
-
-SET
-^^^
-
-This clause allows to specify session parameters to be set for all the
-sessions opened by pgloader. It expects a list of parameter name, the equal
-sign, then the single-quoted value as a comma separated list.
-
-The names and values of the parameters are not validated by pgloader, they
-are given as-is to PostgreSQL.
-
-BEFORE LOAD DO
-^^^^^^^^^^^^^^
-
-You can run SQL queries against the database before loading the data from
-the `CSV` file. Most common SQL queries are `CREATE TABLE IF NOT EXISTS` so
-that the data can be loaded.
-
-Each command must be *dollar-quoted*: it must begin and end with a double
-dollar sign, `$$`. Dollar-quoted queries are then comma separated. No extra
-punctuation is expected after the last SQL query.
-
-BEFORE LOAD EXECUTE
-^^^^^^^^^^^^^^^^^^^
-
-Same behaviour as in the *BEFORE LOAD DO* clause. Allows you to read the SQL
-queries from a SQL file. Implements support for PostgreSQL dollar-quoting
-and the `\i` and `\ir` include facilities as in `psql` batch mode (where
-they are the same thing).
-
-AFTER LOAD DO
-^^^^^^^^^^^^^
-
-Same format as *BEFORE LOAD DO*, the dollar-quoted queries found in that
-section are executed once the load is done. That's the right time to create
-indexes and constraints, or re-enable triggers.
-
-AFTER LOAD EXECUTE
-^^^^^^^^^^^^^^^^^^
-
-Same behaviour as in the *AFTER LOAD DO* clause. Allows you to read the SQL
-queries from a SQL file. Implements support for PostgreSQL dollar-quoting
-and the `\i` and `\ir` include facilities as in `psql` batch mode (where
-they are the same thing).
-
-Connection String
-^^^^^^^^^^^^^^^^^
-
-The `<postgresql-url>` parameter is expected to be given as a *Connection URI*
-as documented in the PostgreSQL documentation at
-http://www.postgresql.org/docs/9.3/static/libpq-connect.html#LIBPQ-CONNSTRING.
-
-::
-
-    postgresql://[user[:password]@][netloc][:port][/dbname][?option=value&...]
-
-Where:
-
-  - *user*
-
-    Can contain any character, including colon (`:`) which must then be
-    doubled (`::`) and at-sign (`@`) which must then be doubled (`@@`).
-
-    When omitted, the *user* name defaults to the value of the `PGUSER`
-    environment variable, and if it is unset, the value of the `USER`
-    environment variable.
-
-  - *password*
-
-	Can contain any character, including the at sign (`@`) which must then
-	be doubled (`@@`). To leave the password empty, when the *user* name
-	ends with at at sign, you then have to use the syntax user:@.
-
-    When omitted, the *password* defaults to the value of the `PGPASSWORD`
-    environment variable if it is set, otherwise the password is left
-    unset.
-    
-    When no *password* is found either in the connection URI nor in the
-    environment, then pgloader looks for a `.pgpass` file as documented at
-    https://www.postgresql.org/docs/current/static/libpq-pgpass.html. The
-    implementation is not that of `libpq` though. As with `libpq` you can
-    set the environment variable `PGPASSFILE` to point to a `.pgpass` file,
-    and pgloader defaults to `~/.pgpass` on unix like systems and
-    `%APPDATA%\postgresql\pgpass.conf` on windows. Matching rules and syntax
-    are the same as with `libpq`, refer to its documentation.
-
-  - *netloc*
-
-    Can be either a hostname in dotted notation, or an ipv4, or an Unix
-    domain socket path. Empty is the default network location, under a
-    system providing *unix domain socket* that method is preferred, otherwise
-    the *netloc* default to `localhost`.
-
-	It's possible to force the *unix domain socket* path by using the syntax
-	`unix:/path/to/where/the/socket/file/is`, so to force a non default
-	socket path and a non default port, you would have:
-
-	    postgresql://unix:/tmp:54321/dbname
-
-    The *netloc* defaults to the value of the `PGHOST` environment
-    variable, and if it is unset, to either the default `unix` socket path
-    when running on a Unix system, and `localhost` otherwise.
-
-    Socket path containing colons are supported by doubling the colons
-    within the path, as in the following example:
-    
-        postgresql://unix:/tmp/project::region::instance:5432/dbname
-
-  - *dbname*
-
-	Should be a proper identifier (letter followed by a mix of letters,
-	digits and the punctuation signs comma (`,`), dash (`-`) and underscore
-	(`_`).
-
-    When omitted, the *dbname* defaults to the value of the environment
-    variable `PGDATABASE`, and if that is unset, to the *user* value as
-    determined above.
-
-  - *options*
-
-    The optional parameters must be supplied with the form `name=value`, and
-    you may use several parameters by separating them away using an
-    ampersand (`&`) character.
-
-    Only some options are supported here, *tablename* (which might be
-    qualified with a schema name) *sslmode*, *host*, *port*, *dbname*,
-    *user* and *password*.
-
-    The *sslmode* parameter values can be one of `disable`, `allow`,
-    `prefer` or `require`.
-
-    For backward compatibility reasons, it's possible to specify the
-    *tablename* option directly, without spelling out the `tablename=`
-    parts.
-
-    The options override the main URI components when both are given, and
-    using the percent-encoded option parameters allow using passwords
-    starting with a colon and bypassing other URI components parsing
-    limitations.
-
-Regular Expressions
-^^^^^^^^^^^^^^^^^^^
-
-Several clauses listed in the following accept *regular expressions* with
-the following input rules:
-
-  - A regular expression begins with a tilde sign (`~`),
-
-  - is then followed with an opening sign,
-
-  - then any character is allowed and considered part of the regular
-    expression, except for the closing sign,
-
-  - then a closing sign is expected.
-
-The opening and closing sign are allowed by pair, here's the complete list
-of allowed delimiters::
-
-    ~//
-    ~[]
-    ~{}
-    ~()
-    ~<>
-    ~""
-    ~''
-    ~||
-    ~##
-
-Pick the set of delimiters that don't collide with the *regular expression*
-you're trying to input. If your expression is such that none of the
-solutions allow you to enter it, the places where such expressions are
-allowed should allow for a list of expressions.
-
-Comments
-^^^^^^^^
-
-Any command may contain comments, following those input rules:
-
-  - the `--` delimiter begins a comment that ends with the end of the
-    current line,
-
-  - the delimiters `/*` and `*/` respectively start and end a comment, which
-    can be found in the middle of a command or span several lines.
-
-Any place where you could enter a *whitespace* will accept a comment too.
-
-Batch behaviour options
-^^^^^^^^^^^^^^^^^^^^^^^
-
-All pgloader commands have support for a *WITH* clause that allows for
-specifying options. Some options are generic and accepted by all commands,
-such as the *batch behaviour options*, and some options are specific to a
-data source kind, such as the CSV *skip header* option.
-
-The global batch behaviour options are:
-
-  - *batch rows*
-
-    Takes a numeric value as argument, used as the maximum number of rows
-    allowed in a batch. The default is `25 000` and can be changed to try
-    having better performance characteristics or to control pgloader memory
-    usage;
-
-  - *batch size*
-
-    Takes a memory unit as argument, such as *20 MB*, its default value.
-    Accepted multipliers are *kB*, *MB*, *GB*, *TB* and *PB*. The case is
-    important so as not to be confused about bits versus bytes, we're only
-    talking bytes here.
-
-  - *prefetch rows*
-
-    Takes a numeric value as argument, defaults to `100000`. That's the
-    number of rows that pgloader is allowed to read in memory in each reader
-    thread. See the *workers* setting for how many reader threads are
-    allowed to run at the same time.
-
-Other options are specific to each input source, please refer to specific
-parts of the documentation for their listing and covering.
-
-A batch is then closed as soon as either the *batch rows* or the *batch
-size* threshold is crossed, whichever comes first. In cases when a batch has
-to be closed because of the *batch size* setting, a *debug* level log
-message is printed with how many rows did fit in the *oversized* batch.
-

docs/_build/html/_sources/ref/archive.rst.txt

@@ -1,104 +0,0 @@
-Loading From an Archive
-=======================
-
-This command instructs pgloader to load data from one or more files contained
-in an archive. Currently the only supported archive format is *ZIP*, and the
-archive might be downloaded from an *HTTP* URL.
-
-Here's an example::
-
-    LOAD ARCHIVE
-       FROM /Users/dim/Downloads/GeoLiteCity-latest.zip
-       INTO postgresql:///ip4r
-
-       BEFORE LOAD
-         DO $$ create extension if not exists ip4r; $$,
-            $$ create schema if not exists geolite; $$,
-
-         EXECUTE 'geolite.sql'
-
-       LOAD CSV
-            FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
-                 WITH ENCODING iso-8859-1
-                 (
-                    locId,
-                    country,
-                    region     null if blanks,
-                    city       null if blanks,
-                    postalCode null if blanks,
-                    latitude,
-                    longitude,
-                    metroCode  null if blanks,
-                    areaCode   null if blanks
-                 )
-            INTO postgresql:///ip4r?geolite.location
-                 (
-                    locid,country,region,city,postalCode,
-                    location point using (format nil "(~a,~a)" longitude latitude),
-                    metroCode,areaCode
-                 )
-            WITH skip header = 2,
-                 fields optionally enclosed by '"',
-                 fields escaped by double-quote,
-                 fields terminated by ','
-
-      AND LOAD CSV
-            FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
-                 WITH ENCODING iso-8859-1
-                 (
-                    startIpNum, endIpNum, locId
-                 )
-            INTO postgresql:///ip4r?geolite.blocks
-                 (
-                    iprange ip4r using (ip-range startIpNum endIpNum),
-                    locId
-                 )
-            WITH skip header = 2,
-                 fields optionally enclosed by '"',
-                 fields escaped by double-quote,
-                 fields terminated by ','
-
-       FINALLY DO
-         $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;
-
-The `archive` command accepts the following clauses and options.
-
-Archive Source Specification: FROM
-----------------------------------
-
-Filename or HTTP URI where to load the data from. When given an HTTP URL the
-linked file will get downloaded locally before processing.
-
-If the file is a `zip` file, the command line utility `unzip` is used to
-expand the archive into files in `$TMPDIR`, or `/tmp` if `$TMPDIR` is unset
-or set to a non-existing directory.
-
-Then the following commands are used from the top level directory where the
-archive has been expanded.
-
-Archive Sub Commands
---------------------
-
-  - command [ *AND* command ... ]
-
-    A series of commands against the contents of the archive, at the moment
-    only `CSV`,`'FIXED` and `DBF` commands are supported.
-
-    Note that commands are supporting the clause *FROM FILENAME MATCHING*
-    which allows the pgloader command not to depend on the exact names of
-    the archive directories.
-
-    The same clause can also be applied to several files with using the
-    spelling *FROM ALL FILENAMES MATCHING* and a regular expression.
-
-    The whole *matching* clause must follow the following rule::
-
-      FROM [ ALL FILENAMES | [ FIRST ] FILENAME ] MATCHING
-
-Archive Final SQL Commands
---------------------------
-      
-  - *FINALLY DO*
-
-    SQL Queries to run once the data is loaded, such as `CREATE INDEX`.
-

docs/_build/html/_sources/ref/copy.rst.txt

@@ -1,115 +0,0 @@
-Loading COPY Formatted Files
-============================
-
-This commands instructs pgloader to load from a file containing COPY TEXT
-data as described in the PostgreSQL documentation. Here's an example::
-
-    LOAD COPY
-         FROM copy://./data/track.copy
-              (
-                trackid, track, album, media, genre, composer,
-                milliseconds, bytes, unitprice
-              )
-         INTO postgresql:///pgloader
-       TARGET TABLE track_full
-
-         WITH truncate
-
-          SET work_mem to '14MB',
-              standard_conforming_strings to 'on'
-
-    BEFORE LOAD DO
-         $$ drop table if exists track_full; $$,
-         $$ create table track_full (
-              trackid      bigserial,
-              track        text,
-              album        text,
-              media        text,
-              genre        text,
-              composer     text,
-              milliseconds bigint,
-              bytes        bigint,
-              unitprice    numeric
-            );
-         $$;
-
-The `COPY` format command accepts the following clauses and options.
-
-COPY Formatted Files Source Specification: FROM
------------------------------------------------
-
-Filename where to load the data from. This support local files, HTTP URLs
-and zip files containing a single dbf file of the same name. Fetch such a
-zip file from an HTTP address is of course supported.
-
-  - *inline*
-
-    The data is found after the end of the parsed commands. Any number of
-    empty lines between the end of the commands and the beginning of the
-    data is accepted.
-
-  - *stdin*
-
-    Reads the data from the standard input stream.
-
-  - *FILENAMES MATCHING*
-
-    The whole *matching* clause must follow the following rule::
-
-      [ ALL FILENAMES | [ FIRST ] FILENAME ]
-      MATCHING regexp
-      [ IN DIRECTORY '...' ]
-
-    The *matching* clause applies given *regular expression* (see above for
-    exact syntax, several options can be used here) to filenames. It's then
-    possible to load data from only the first match of all of them.
-
-    The optional *IN DIRECTORY* clause allows specifying which directory to
-    walk for finding the data files, and can be either relative to where the
-    command file is read from, or absolute. The given directory must exists.
-
-COPY Formatted File Options: WITH
----------------------------------
-
-
-When loading from a `COPY` file, the following options are supported:
-
-  - *delimiter*
-
-    Takes a single character as argument, which must be found inside single
-    quotes, and might be given as the printable character itself, the
-    special value \t to denote a tabulation character, or `0x` then an
-    hexadecimal value read as the ASCII code for the character.
-
-    This character is used as the *delimiter* when reading the data, in a
-    similar way to the PostgreSQL `COPY` option.
-
-  - *null*
-
-    Takes a quoted string as an argument (quotes can be either double quotes
-    or single quotes) and uses that string as the `NULL` representation in
-    the data.
-
-    This is similar to the *null* `COPY` option in PostgreSQL.
-
-  - *truncate*
-
-    When this option is listed, pgloader issues a `TRUNCATE` command against
-    the PostgreSQL target table before reading the data file.
-
-  - *disable triggers*
-
-    When this option is listed, pgloader issues an `ALTER TABLE ... DISABLE
-    TRIGGER ALL` command against the PostgreSQL target table before copying
-    the data, then the command `ALTER TABLE ... ENABLE TRIGGER ALL` once the
-    `COPY` is done.
-
-    This option allows loading data into a pre-existing table ignoring the
-    *foreign key constraints* and user defined triggers and may result in
-    invalid *foreign key constraints* once the data is loaded. Use with
-    care.
-
-  - *skip header*
-
-    Takes a numeric value as argument. Instruct pgloader to skip that many
-    lines at the beginning of the input file.

docs/_build/html/_sources/ref/csv.rst.txt

@@ -1,243 +0,0 @@
-Loading CSV data
-================
-
-This command instructs pgloader to load data from a `CSV` file. Here's an
-example::
-
-    LOAD CSV
-       FROM 'GeoLiteCity-Blocks.csv' WITH ENCODING iso-646-us
-            HAVING FIELDS
-            (
-               startIpNum, endIpNum, locId
-            )
-       INTO postgresql://user@localhost:54393/dbname
-            TARGET TABLE geolite.blocks
-            TARGET COLUMNS
-            (
-               iprange ip4r using (ip-range startIpNum endIpNum),
-               locId
-            )
-       WITH truncate,
-            skip header = 2,
-            fields optionally enclosed by '"',
-            fields escaped by backslash-quote,
-            fields terminated by '\t'
-
-        SET work_mem to '32 MB', maintenance_work_mem to '64 MB';
-
-The `csv` format command accepts the following clauses and options.
-
-CSV Source Specification: FROM
-------------------------------
-
-Filename where to load the data from. Accepts an *ENCODING* option. Use the
-`--list-encodings` option to know which encoding names are supported.
-
-The filename may be enclosed by single quotes, and could be one of the
-following special values:
-
-  - *inline*
-
-    The data is found after the end of the parsed commands. Any number
-    of empty lines between the end of the commands and the beginning of
-    the data is accepted.
-
-  - *stdin*
-
-    Reads the data from the standard input stream.
-
-  - *FILENAMES MATCHING*
-
-    The whole *matching* clause must follow the following rule::
-
-            [ ALL FILENAMES | [ FIRST ] FILENAME ]
-            MATCHING regexp
-            [ IN DIRECTORY '...' ]
-
-    The *matching* clause applies given *regular expression* (see above
-    for exact syntax, several options can be used here) to filenames.
-    It's then possible to load data from only the first match of all of
-    them.
-
-    The optional *IN DIRECTORY* clause allows specifying which directory
-    to walk for finding the data files, and can be either relative to
-    where the command file is read from, or absolute. The given
-    directory must exists.
-
-Fields Specifications
----------------------
-    
-The *FROM* option also supports an optional comma separated list of *field*
-names describing what is expected in the `CSV` data file, optionally
-introduced by the clause `HAVING FIELDS`.
-
-Each field name can be either only one name or a name following with
-specific reader options for that field, enclosed in square brackets and
-comma-separated. Supported per-field reader options are:
-
-  - *terminated by*
-
-     See the description of *field terminated by* below.
-
-     The processing of this option is not currently implemented.
-
-  - *date format*
-
-    When the field is expected of the date type, then this option allows
-    to specify the date format used in the file.
-
-    Date format string are template strings modeled against the
-    PostgreSQL `to_char` template strings support, limited to the
-    following patterns:
-
-      - YYYY, YYY, YY for the year part
-      - MM for the numeric month part
-      - DD for the numeric day part
-      - HH, HH12, HH24 for the hour part
-      - am, AM, a.m., A.M.
-      - pm, PM, p.m., P.M.
-      - MI for the minutes part
-      - SS for the seconds part
-      - MS for the milliseconds part (4 digits)
-      - US for the microseconds part (6 digits)
-      - unparsed punctuation signs: - . * # @ T / \ and space
-
-    Here's an example of a *date format* specification::
-
-        column-name [date format 'YYYY-MM-DD HH24-MI-SS.US']
-
-  - *null if*
-
-     This option takes an argument which is either the keyword *blanks*
-     or a double-quoted string.
-
-     When *blanks* is used and the field value that is read contains
-     only space characters, then it's automatically converted to an SQL
-     `NULL` value.
-
-     When a double-quoted string is used and that string is read as the
-     field value, then the field value is automatically converted to an
-     SQL `NULL` value.
-
-  - *trim both whitespace*, *trim left whitespace*, *trim right whitespace*
-
-    This option allows to trim whitespaces in the read data, either from
-    both sides of the data, or only the whitespace characters found on
-    the left of the streaing, or only those on the right of the string.
-
-CSV Loading Options: WITH
--------------------------
-
-When loading from a `CSV` file, the following options are supported:
-
-  - *truncate*
-
-     When this option is listed, pgloader issues a `TRUNCATE` command
-     against the PostgreSQL target table before reading the data file.
-
-  - *drop indexes*
-
-    When this option is listed, pgloader issues `DROP INDEX` commands
-    against all the indexes defined on the target table before copying
-    the data, then `CREATE INDEX` commands once the `COPY` is done.
-
-    In order to get the best performance possible, all the indexes are
-    created in parallel and when done the primary keys are built again
-    from the unique indexes just created. This two step process allows
-    creating the primary key index in parallel with the other indexes,
-    as only the `ALTER TABLE` command needs an *access exclusive lock*
-    on the target table.
-
-  - *disable triggers*
-
-    When this option is listed, pgloader issues an `ALTER TABLE ...
-    DISABLE TRIGGER ALL` command against the PostgreSQL target table
-    before copying the data, then the command `ALTER TABLE ... ENABLE
-    TRIGGER ALL` once the `COPY` is done.
-
-    This option allows loading data into a pre-existing table ignoring
-    the *foreign key constraints* and user defined triggers and may
-    result in invalid *foreign key constraints* once the data is loaded.
-    Use with care.
-
-  - *skip header*
-
-    Takes a numeric value as argument. Instruct pgloader to skip that
-    many lines at the beginning of the input file.
-
-  - *csv header*
-
-    Use the first line read after *skip header* as the list of csv field
-    names to be found in the CSV file, using the same CSV parameters as
-    for the CSV data.
-
-  - *trim unquoted blanks*
-
-    When reading unquoted values in the `CSV` file, remove the blanks
-    found in between the separator and the value. That behaviour is the
-    default.
-
-  - *keep unquoted blanks*
-
-    When reading unquoted values in the `CSV` file, keep blanks found in
-    between the separator and the value.
-
-  - *fields optionally enclosed by*
-
-    Takes a single character as argument, which must be found inside single
-    quotes, and might be given as the printable character itself, the
-    special value \t to denote a tabulation character, the special value \'
-    to denote a single-quote, or `0x` then an hexadecimal value read as the
-    ASCII code for the character.
-
-    The following options specify the same enclosing character, a single quote::
-
-      fields optionally enclosed by '\''
-      fields optionally enclosed by '0x27'
-
-    This character is used as the quoting character in the `CSV` file,
-    and defaults to double-quote.
-
-  - *fields not enclosed*
-
-    By default, pgloader will use the double-quote character as the
-    enclosing character. If you have a CSV file where fields are not
-    enclosed and are using double-quote as an expected ordinary
-    character, then use the option *fields not enclosed* for the CSV
-    parser to accept those values.
-
-  - *fields escaped by*
-
-    Takes either the special value *backslash-quote* or *double-quote*,
-    or any value supported by the *fields terminated by* option (see
-    below). This value is used to recognize escaped field separators
-    when they are to be found within the data fields themselves.
-    Defaults to *double-quote*.
-
-  - *csv escape mode*
-
-    Takes either the special value *quote* (the default) or *following*
-    and allows the CSV parser to parse either only escaped field
-    separator or any character (including CSV data) when using the
-    *following* value.
-
-  - *fields terminated by*
-
-    Takes a single character as argument, which must be found inside
-    single quotes, and might be given as the printable character itself,
-    the special value \t to denote a tabulation character, or `0x` then
-    an hexadecimal value read as the ASCII code for the character.
-
-    This character is used as the *field separator* when reading the
-    `CSV` data.
-
-  - *lines terminated by*
-
-    Takes a single character as argument, which must be found inside
-    single quotes, and might be given as the printable character itself,
-    the special value \t to denote a tabulation character, or `0x` then
-    an hexadecimal value read as the ASCII code for the character.
-
-    This character is used to recognize *end-of-line* condition when
-    reading the `CSV` data.
-

docs/_build/html/_sources/ref/dbf.rst.txt

@@ -1,53 +0,0 @@
-Loading DBF data
-=================
-
-This command instructs pgloader to load data from a `DBF` file. Here's an
-example::
-
-    LOAD DBF
-	    FROM http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/reg2013.dbf
-        INTO postgresql://user@localhost/dbname
-        WITH truncate, create table;
-
-The `dbf` format command accepts the following clauses and options.
-
-DBF Source Specification: FROM
-------------------------------
-
-Filename where to load the data from. This support local files, HTTP URLs
-and zip files containing a single dbf file of the same name. Fetch such a
-zip file from an HTTP address is of course supported.
-
-DBF Loading Options: WITH
--------------------------
-
-When loading from a `DBF` file, the following options are supported:
-
-  - *truncate*
-
-    When this option is listed, pgloader issues a `TRUNCATE` command against
-    the PostgreSQL target table before reading the data file.
-
-  - *disable triggers*
-
-    When this option is listed, pgloader issues an `ALTER TABLE ... DISABLE
-    TRIGGER ALL` command against the PostgreSQL target table before copying
-    the data, then the command `ALTER TABLE ... ENABLE TRIGGER ALL` once the
-    `COPY` is done.
-
-    This option allows loading data into a pre-existing table ignoring the
-    *foreign key constraints* and user defined triggers and may result in
-    invalid *foreign key constraints* once the data is loaded. Use with
-    care.
-
-  - *create table*
-
-    When this option is listed, pgloader creates the table using the meta
-    data found in the `DBF` file, which must contain a list of fields with
-    their data type. A standard data type conversion from DBF to PostgreSQL
-    is done.
-
-  - *table name*
-
-    This options expects as its value the possibly qualified name of the
-    table to create.

docs/_build/html/_sources/ref/fixed.rst.txt

@@ -1,182 +0,0 @@
-Loading Fixed Cols File Formats
-===============================
-
-This command instructs pgloader to load data from a text file containing
-columns arranged in a *fixed size* manner. Here's an example::
-
-    LOAD FIXED
-         FROM inline
-              (
-               a from  0 for 10,
-               b from 10 for  8,
-               c from 18 for  8,
-               d from 26 for 17 [null if blanks, trim right whitespace]
-              )
-         INTO postgresql:///pgloader
-       TARGET TABLE fixed
-              (
-                 a, b,
-                 c time using (time-with-no-separator c),
-                 d
-              )
-
-         WITH truncate
-
-          SET work_mem to '14MB',
-              standard_conforming_strings to 'on'
-
-    BEFORE LOAD DO
-         $$ drop table if exists fixed; $$,
-         $$ create table fixed (
-             a integer,
-             b date,
-             c time,
-             d text
-            );
-         $$;
-
-     01234567892008052011431250firstline
-        01234562008052115182300left blank-padded
-     12345678902008052208231560another line
-      2345609872014092914371500                 
-      2345678902014092914371520
-
-The `fixed` format command accepts the following clauses and options.
-
-Fixed File Format Source Specification: FROM
---------------------------------------------
-
-Filename where to load the data from. Accepts an *ENCODING* option. Use the
-`--list-encodings` option to know which encoding names are supported.
-
-The filename may be enclosed by single quotes, and could be one of the
-following special values:
-
-  - *inline*
-
-     The data is found after the end of the parsed commands. Any number
-     of empty lines between the end of the commands and the beginning of
-     the data is accepted.
-
-  - *stdin*
-
-     Reads the data from the standard input stream.
-
-  - *FILENAMES MATCHING*
-
-    The whole *matching* clause must follow the following rule::
-
-        [ ALL FILENAMES | [ FIRST ] FILENAME ]
-        MATCHING regexp
-        [ IN DIRECTORY '...' ]
-
-    The *matching* clause applies given *regular expression* (see above
-    for exact syntax, several options can be used here) to filenames.
-    It's then possible to load data from only the first match of all of
-    them.
-
-    The optional *IN DIRECTORY* clause allows specifying which directory
-    to walk for finding the data files, and can be either relative to
-    where the command file is read from, or absolute. The given
-    directory must exists.
-
-Fields Specifications
----------------------
-
-The *FROM* option also supports an optional comma separated list of *field*
-names describing what is expected in the `FIXED` data file.
-
-Each field name is composed of the field name followed with specific reader
-options for that field. Supported per-field reader options are the
-following, where only *start* and *length* are required.
-
-  - *start*
-
-    Position in the line where to start reading that field's value. Can
-    be entered with decimal digits or `0x` then hexadecimal digits.
-
-  - *length*
-
-    How many bytes to read from the *start* position to read that
-    field's value. Same format as *start*.
-
-Those optional parameters must be enclosed in square brackets and
-comma-separated:
-
-  - *terminated by*
-
-     See the description of *field terminated by* below.
-
-     The processing of this option is not currently implemented.
-
-  - *date format*
-
-    When the field is expected of the date type, then this option allows
-    to specify the date format used in the file.
-
-    Date format string are template strings modeled against the
-    PostgreSQL `to_char` template strings support, limited to the
-    following patterns:
-
-      - YYYY, YYY, YY for the year part
-      - MM for the numeric month part
-      - DD for the numeric day part
-      - HH, HH12, HH24 for the hour part
-      - am, AM, a.m., A.M.
-      - pm, PM, p.m., P.M.
-      - MI for the minutes part
-      - SS for the seconds part
-      - MS for the milliseconds part (4 digits)
-      - US for the microseconds part (6 digits)
-      - unparsed punctuation signs: - . * # @ T / \ and space
-
-    Here's an example of a *date format* specification::
-
-        column-name [date format 'YYYY-MM-DD HH24-MI-SS.US']
-
-  - *null if*
-
-    This option takes an argument which is either the keyword *blanks*
-    or a double-quoted string.
-
-    When *blanks* is used and the field value that is read contains only
-    space characters, then it's automatically converted to an SQL `NULL`
-    value.
-
-    When a double-quoted string is used and that string is read as the
-    field value, then the field value is automatically converted to an
-    SQL `NULL` value.
-
-  - *trim both whitespace*, *trim left whitespace*, *trim right whitespace*
-
-    This option allows to trim whitespaces in the read data, either from
-    both sides of the data, or only the whitespace characters found on
-    the left of the streaing, or only those on the right of the string.
-
-Fixed File Format Loading Options: WITH
----------------------------------------
-
-When loading from a `FIXED` file, the following options are supported:
-
-  - *truncate*
-
-    When this option is listed, pgloader issues a `TRUNCATE` command
-    against the PostgreSQL target table before reading the data file.
-
-  - *disable triggers*
-
-    When this option is listed, pgloader issues an `ALTER TABLE ...
-    DISABLE TRIGGER ALL` command against the PostgreSQL target table
-    before copying the data, then the command `ALTER TABLE ... ENABLE
-    TRIGGER ALL` once the `COPY` is done.
-
-    This option allows loading data into a pre-existing table ignoring
-    the *foreign key constraints* and user defined triggers and may
-    result in invalid *foreign key constraints* once the data is loaded.
-    Use with care.
-
-  - *skip header*
-
-    Takes a numeric value as argument. Instruct pgloader to skip that
-    many lines at the beginning of the input file.
-

docs/_build/html/_sources/ref/ixf.rst.txt

@@ -1,66 +0,0 @@
-Loading IXF Data
-================
-
-This command instructs pgloader to load data from an IBM `IXF` file. Here's
-an example::
-
-    LOAD IXF
-        FROM data/nsitra.test1.ixf
-        INTO postgresql:///pgloader
-      TARGET TABLE nsitra.test1
-        WITH truncate, create table, timezone UTC
-
-      BEFORE LOAD DO
-       $$ create schema if not exists nsitra; $$,
-       $$ drop table if exists nsitra.test1; $$;
-
-The `ixf` format command accepts the following clauses and options.
-
-IXF Source Specification: FROM
-------------------------------
-
-Filename where to load the data from. This support local files, HTTP URLs
-and zip files containing a single ixf file of the same name. Fetch such a
-zip file from an HTTP address is of course supported.
-
-IXF Loading Options: WITH
--------------------------
-
-When loading from a `IXF` file, the following options are supported:
-
-  - *truncate*
-
-    When this option is listed, pgloader issues a `TRUNCATE` command against
-    the PostgreSQL target table before reading the data file.
-
-  - *disable triggers*
-
-    When this option is listed, pgloader issues an `ALTER TABLE ... DISABLE
-    TRIGGER ALL` command against the PostgreSQL target table before copying
-    the data, then the command `ALTER TABLE ... ENABLE TRIGGER ALL` once the
-    `COPY` is done.
-
-    This option allows loading data into a pre-existing table ignoring the
-    *foreign key constraints* and user defined triggers and may result in
-    invalid *foreign key constraints* once the data is loaded. Use with
-    care.
-
-  - *create table*
-
-    When this option is listed, pgloader creates the table using the meta
-    data found in the `DBF` file, which must contain a list of fields with
-    their data type. A standard data type conversion from DBF to PostgreSQL
-    is done.
-
-  - *table name*
-
-    This options expects as its value the possibly qualified name of the
-    table to create.
-
-  - *timezone*
-
-    This options allows to specify which timezone is used when parsing
-    timestamps from an IXF file, and defaults to *UTC*. Expected values are
-    either `UTC`, `GMT` or a single quoted location name such as
-    `'Universal'` or `'Europe/Paris'`.
-

docs/_build/html/_sources/ref/mssql.rst.txt

@@ -1,159 +0,0 @@
-Migrating a MS SQL Database to PostgreSQL
-=========================================
-
-This command instructs pgloader to load data from a MS SQL database.
-Automatic discovery of the schema is supported, including build of the
-indexes, primary and foreign keys constraints.
-
-Here's an example::
-
-    load database
-         from mssql://user@host/dbname
-         into postgresql:///dbname
-
-    including only table names like 'GlobalAccount' in schema 'dbo'
-
-    set work_mem to '16MB', maintenance_work_mem to '512 MB'
-
-    before load do $$ drop schema if exists dbo cascade; $$;
-
-The `mssql` command accepts the following clauses and options.
-
-MS SQL Database Source Specification: FROM
-------------------------------------------
-
-Connection string to an existing MS SQL database server that listens and
-welcome external TCP/IP connection. As pgloader currently piggybacks on the
-FreeTDS driver, to change the port of the server please export the `TDSPORT`
-environment variable.
-
-MS SQL Database Migration Options: WITH
----------------------------------------
-
-When loading from a `MS SQL` database, the same options as when loading a
-`MySQL` database are supported. Please refer to the MySQL section. The
-following options are added:
-
-  - *create schemas*
-
-    When this option is listed, pgloader creates the same schemas as found
-    on the MS SQL instance. This is the default.
-
-  - *create no schemas*
-
-    When this option is listed, pgloader refrains from creating any schemas
-    at all, you must then ensure that the target schema do exist.
-
-MS SQL Database Casting Rules
------------------------------
-    
-CAST
-^^^^
-
-The cast clause allows to specify custom casting rules, either to overload
-the default casting rules or to amend them with special cases.
-
-Please refer to the MySQL CAST clause for details.
-
-MS SQL Partial Migration
-------------------------
-
-
-INCLUDING ONLY TABLE NAMES LIKE
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table name patterns used to limit the
-tables to migrate to a sublist. More than one such clause may be used, they
-will be accumulated together.
-
-Example::
-
-  including only table names lile 'GlobalAccount' in schema 'dbo'
-
-EXCLUDING TABLE NAMES LIKE
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table name patterns used to exclude
-table names from the migration. This filter only applies to the result of
-the *INCLUDING* filter.
-
-::
-   
-   excluding table names matching 'LocalAccount' in schema 'dbo'
-
-MS SQL Schema Transformations
------------------------------
-
-ALTER SCHEMA '...' RENAME TO '...'
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Allows to rename a schema on the flight, so that for instance the tables
-found in the schema 'dbo' in your source database will get migrated into the
-schema 'public' in the target database with this command::
-
-  alter schema 'dbo' rename to 'public'
-
-ALTER TABLE NAMES MATCHING ... IN SCHEMA '...'
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-See the MySQL explanation for this clause above. It works the same in the
-context of migrating from MS SQL, only with the added option to specify the
-name of the schema where to find the definition of the target tables.
-
-The matching is done in pgloader itself, with a Common Lisp regular
-expression lib, so doesn't depend on the *LIKE* implementation of MS SQL,
-nor on the lack of support for regular expressions in the engine.
-
-MS SQL Driver setup and encoding
---------------------------------
-
-pgloader is using the `FreeTDS` driver, and internally expects the data to
-be sent in utf-8. To achieve that, you can configure the FreeTDS driver with
-those defaults, in the file `~/.freetds.conf`::
-
-    [global]
-        tds version = 7.4
-        client charset = UTF-8
-
-Default MS SQL Casting Rules
-----------------------------
-
-When migrating from MS SQL the following Casting Rules are provided:
-
-Numbers::
-
-  type tinyint to smallint
-
-  type float to float   using float-to-string
-  type real to real     using float-to-string
-  type double to double precision     using float-to-string
-  type numeric to numeric     using float-to-string
-  type decimal to numeric     using float-to-string
-  type money to numeric     using float-to-string
-  type smallmoney to numeric     using float-to-string
-
-Texts::
-
-  type char      to text drop typemod
-  type nchat     to text drop typemod
-  type varchar   to text drop typemod
-  type nvarchar  to text drop typemod
-  type xml       to text drop typemod
-
-Binary::
-
-  type binary    to bytea using byte-vector-to-bytea
-  type varbinary to bytea using byte-vector-to-bytea
-
-Date::
-
-  type datetime    to timestamptz
-  type datetime2   to timestamptz
-
-Others::
-
-  type bit to boolean
-  type hierarchyid to bytea
-  type geography to bytea
-  type uniqueidentifier to uuid using sql-server-uniqueidentifier-to-uuid
-

docs/_build/html/_sources/ref/mysql.rst.txt

@@ -1,623 +0,0 @@
-Migrating a MySQL Database to PostgreSQL
-========================================
-
-This command instructs pgloader to load data from a database connection. The
-only supported database source is currently *MySQL*, and pgloader supports
-dynamically converting the schema of the source database and the indexes
-building.
-
-A default set of casting rules are provided and might be overloaded and
-appended to by the command.
-
-Here's an example using as many options as possible, some of them even being
-defaults. Chances are you don't need that complex a setup, don't copy and
-paste it, use it only as a reference!
-
-::
-   
-    LOAD DATABASE
-         FROM      mysql://root@localhost/sakila
-         INTO postgresql://localhost:54393/sakila
-
-     WITH include drop, create tables, create indexes, reset sequences,
-          workers = 8, concurrency = 1,
-          multiple readers per thread, rows per range = 50000
-
-      SET PostgreSQL PARAMETERS
-          maintenance_work_mem to '128MB',
-          work_mem to '12MB',
-          search_path to 'sakila, public, "$user"'
-    
-      SET MySQL PARAMETERS
-          net_read_timeout  = '120',
-          net_write_timeout = '120'
-
-     CAST type bigint when (= precision 20) to bigserial drop typemod,
-          type date drop not null drop default using zero-dates-to-null,
-          -- type tinyint to boolean using tinyint-to-boolean,
-          type year to integer
-
-     MATERIALIZE VIEWS film_list, staff_list
-
-     -- INCLUDING ONLY TABLE NAMES MATCHING ~/film/, 'actor'
-     -- EXCLUDING TABLE NAMES MATCHING ~<ory>
-     -- DECODING TABLE NAMES MATCHING ~/messed/, ~/encoding/ AS utf8
-     -- ALTER TABLE NAMES MATCHING 'film' RENAME TO 'films'
-     -- ALTER TABLE NAMES MATCHING ~/_list$/ SET SCHEMA 'mv'
-     
-     ALTER TABLE NAMES MATCHING ~/_list$/, 'sales_by_store', ~/sales_by/
-      SET SCHEMA 'mv'
-    
-     ALTER TABLE NAMES MATCHING 'film' RENAME TO 'films'
-     ALTER TABLE NAMES MATCHING ~/./ SET (fillfactor='40')
-    
-     ALTER SCHEMA 'sakila' RENAME TO 'pagila'
-
-     BEFORE LOAD DO
-       $$ create schema if not exists pagila; $$,
-       $$ create schema if not exists mv;     $$,
-       $$ alter database sakila set search_path to pagila, mv, public; $$;
-
-
-The `database` command accepts the following clauses and options.
-
-MySQL Database Source Specification: FROM
------------------------------------------
-
-Must be a connection URL pointing to a MySQL database.
-
-If the connection URI contains a table name, then only this table is
-migrated from MySQL to PostgreSQL.
-
-See the `SOURCE CONNECTION STRING` section above for details on how to write
-the connection string. The MySQL connection string accepts the same
-parameter *sslmode* as the PostgreSQL connection string, but the *verify*
-mode is not implemented (yet).
-    
-Environment variables described in
-<http://dev.mysql.com/doc/refman/5.0/en/environment-variables.html> can be
-used as default values too. If the user is not provided, then it defaults to
-`USER` environment variable value. The password can be provided with the
-environment variable `MYSQL_PWD`. The host can be provided with the
-environment variable `MYSQL_HOST` and otherwise defaults to `localhost`. The
-port can be provided with the environment variable `MYSQL_TCP_PORT` and
-otherwise defaults to `3306`.
-
-MySQL Database Migration Options: WITH
---------------------------------------
-
-When loading from a `MySQL` database, the following options are supported,
-and the default *WITH* clause is: *no truncate*, *create schema*, *create
-tables*, *include drop*, *create indexes*, *reset sequences*, *foreign
-keys*, *downcase identifiers*, *uniquify index names*.
-
-  - *include drop*
-
-    When this option is listed, pgloader drops all the tables in the target
-    PostgreSQL database whose names appear in the MySQL database. This
-    option allows for using the same command several times in a row until
-    you figure out all the options, starting automatically from a clean
-    environment. Please note that `CASCADE` is used to ensure that tables
-    are dropped even if there are foreign keys pointing to them. This is
-    precisely what `include drop` is intended to do: drop all target tables
-    and recreate them.
-
-    Great care needs to be taken when using `include drop`, as it will
-    cascade to *all* objects referencing the target tables, possibly
-    including other tables that are not being loaded from the source DB.
-
-  - *include no drop*
-
-    When this option is listed, pgloader will not include any `DROP`
-    statement when loading the data.
-
-  - *truncate*
-
-    When this option is listed, pgloader issue the `TRUNCATE` command
-    against each PostgreSQL table just before loading data into it.
-
-  - *no truncate*
-
-    When this option is listed, pgloader issues no `TRUNCATE` command.
-
-  - *disable triggers*
-
-    When this option is listed, pgloader issues an `ALTER TABLE ... DISABLE
-    TRIGGER ALL` command against the PostgreSQL target table before copying
-    the data, then the command `ALTER TABLE ... ENABLE TRIGGER ALL` once the
-    `COPY` is done.
-
-    This option allows loading data into a pre-existing table ignoring the
-    *foreign key constraints* and user defined triggers and may result in
-    invalid *foreign key constraints* once the data is loaded. Use with
-    care.
-
-  - *create tables*
-
-    When this option is listed, pgloader creates the table using the meta
-    data found in the `MySQL` file, which must contain a list of fields with
-    their data type. A standard data type conversion from DBF to PostgreSQL
-    is done.
-
-  - *create no tables*
-
-    When this option is listed, pgloader skips the creation of table before
-    loading data, target tables must then already exist.
-
-    Also, when using *create no tables* pgloader fetches the metadata from
-    the current target database and checks type casting, then will remove
-    constraints and indexes prior to loading the data and install them back
-    again once the loading is done.
-
-  - *create indexes*
-
-    When this option is listed, pgloader gets the definitions of all the
-    indexes found in the MySQL database and create the same set of index
-    definitions against the PostgreSQL database.
-
-  - *create no indexes*
-
-    When this option is listed, pgloader skips the creating indexes.
-        
-  - *drop indexes*
-  
-    When this option is listed, pgloader drops the indexes in the target
-    database before loading the data, and creates them again at the end
-    of the data copy.
-
-  - *uniquify index names*, *preserve index names*
-
-    MySQL index names are unique per-table whereas in PostgreSQL index names
-    have to be unique per-schema. The default for pgloader is to change the
-    index name by prefixing it with `idx_OID` where `OID` is the internal
-    numeric identifier of the table the index is built against.
-
-    In somes cases like when the DDL are entirely left to a framework it
-    might be sensible for pgloader to refrain from handling index unique
-    names, that is achieved by using the *preserve index names* option.
-
-    The default is to *uniquify index names*.
-
-    Even when using the option *preserve index names*, MySQL primary key
-    indexes named "PRIMARY" will get their names uniquified. Failing to do
-    so would prevent the primary keys to be created again in PostgreSQL
-    where the index names must be unique per schema.
-
-  - *drop schema*
-  
-    When this option is listed, pgloader drops the target schema in the
-    target PostgreSQL database before creating it again and all the objects
-    it contains. The default behavior doesn't drop the target schemas.
-
-  - *foreign keys*
-
-    When this option is listed, pgloader gets the definitions of all the
-    foreign keys found in the MySQL database and create the same set of
-    foreign key definitions against the PostgreSQL database.
-
-  - *no foreign keys*
-
-    When this option is listed, pgloader skips creating foreign keys.
-
-  - *reset sequences*
-
-    When this option is listed, at the end of the data loading and after the
-    indexes have all been created, pgloader resets all the PostgreSQL
-    sequences created to the current maximum value of the column they are
-    attached to.
-
-    The options *schema only* and *data only* have no effects on this
-    option.
-
-  - *reset no sequences*
-
-    When this option is listed, pgloader skips resetting sequences after the
-    load.
-
-    The options *schema only* and *data only* have no effects on this
-    option.
-
-  - *downcase identifiers*
-
-    When this option is listed, pgloader converts all MySQL identifiers
-    (table names, index names, column names) to *downcase*, except for
-    PostgreSQL *reserved* keywords.
-
-    The PostgreSQL *reserved* keywords are determined dynamically by using
-    the system function `pg_get_keywords()`.
-
-  - *quote identifiers*
-
-    When this option is listed, pgloader quotes all MySQL identifiers so
-    that their case is respected. Note that you will then have to do the
-    same thing in your application code queries.
-
-  - *schema only*
-
-    When this option is listed pgloader refrains from migrating the data
-    over. Note that the schema in this context includes the indexes when the
-    option *create indexes* has been listed.
-
-  - *data only*
-
-    When this option is listed pgloader only issues the `COPY` statements,
-    without doing any other processing.
-
-  - *single reader per thread*, *multiple readers per thread*
-  
-    The default is *single reader per thread* and it means that each
-    MySQL table is read by a single thread as a whole, with a single
-    `SELECT` statement using no `WHERE` clause.
-    
-    When using *multiple readers per thread* pgloader may be able to
-    divide the reading work into several threads, as many as the
-    *concurrency* setting, which needs to be greater than 1 for this
-    option to kick be activated.
-    
-    For each source table, pgloader searches for a primary key over a
-    single numeric column, or a multiple-column primary key index for
-    which the first column is of a numeric data type (one of `integer`
-    or `bigint`). When such an index exists, pgloader runs a query to
-    find the *min* and *max* values on this column, and then split that
-    range into many ranges containing a maximum of *rows per range*.
-    
-    When the range list we then obtain contains at least as many ranges
-    than our concurrency setting, then we distribute those ranges to
-    each reader thread.
-    
-    So when all the conditions are met, pgloader then starts as many
-    reader thread as the *concurrency* setting, and each reader thread
-    issues several queries with a `WHERE id >= x AND id < y`, where `y -
-    x = rows per range` or less (for the last range, depending on the
-    max value just obtained.
-  
-  - *rows per range*
-  
-    How many rows are fetched per `SELECT` query when using *multiple
-    readers per thread*, see above for details.
-
-  - *SET MySQL PARAMETERS*
-  
-    The *SET MySQL PARAMETERS* allows setting MySQL parameters using the
-    MySQL `SET` command each time pgloader connects to it.
-
-MySQL Database Casting Rules
-----------------------------
-
-The command *CAST* introduces user-defined casting rules.
-
-The cast clause allows to specify custom casting rules, either to overload
-the default casting rules or to amend them with special cases.
-
-A casting rule is expected to follow one of the forms::
-
-    type <mysql-type-name> [ <guard> ... ] to <pgsql-type-name> [ <option> ... ]
-    column <table-name>.<column-name> [ <guards> ] to ...
-
-It's possible for a *casting rule* to either match against a MySQL data type
-or against a given *column name* in a given *table name*. That flexibility
-allows to cope with cases where the type `tinyint` might have been used as a
-`boolean` in some cases but as a `smallint` in others.
-
-The *casting rules* are applied in order, the first match prevents following
-rules to be applied, and user defined rules are evaluated first.
-
-The supported guards are:
-
-  - *when unsigned*
-
-    The casting rule is only applied against MySQL columns of the source
-    type that have the keyword *unsigned* in their data type definition.
-
-    Example of a casting rule using a *unsigned* guard::
-        
-      type smallint when unsigned to integer drop typemod
-
-  - *when default 'value'*
-
-    The casting rule is only applied against MySQL columns of the source
-    type that have given *value*, which must be a single-quoted or a
-    double-quoted string.
-
-  - *when typemod expression*
-
-    The casting rule is only applied against MySQL columns of the source
-    type that have a *typemod* value matching the given *typemod
-    expression*. The *typemod* is separated into its *precision* and *scale*
-    components.
-
-    Example of a cast rule using a *typemod* guard::
-
-      type char when (= precision 1) to char keep typemod
-
-    This expression casts MySQL `char(1)` column to a PostgreSQL column of
-    type `char(1)` while allowing for the general case `char(N)` will be
-    converted by the default cast rule into a PostgreSQL type `varchar(N)`.
-
-  - *with extra auto_increment*
-
-    The casting rule is only applied against MySQL columns having the
-    *extra* column `auto_increment` option set, so that it's possible to
-    target e.g. `serial` rather than `integer`.
-
-    The default matching behavior, when this option isn't set, is to match
-    both columns with the extra definition and without.
-
-    This means that if you want to implement a casting rule that target
-    either `serial` or `integer` from a `smallint` definition depending on
-    the *auto_increment* extra bit of information from MySQL, then you need
-    to spell out two casting rules as following::
-
-      type smallint  with extra auto_increment
-        to serial drop typemod keep default keep not null,
-
-      type smallint
-        to integer drop typemod keep default keep not null
-
-The supported casting options are:
-
-  - *drop default*, *keep default*
-
-    When the option *drop default* is listed, pgloader drops any
-    existing default expression in the MySQL database for columns of the
-    source type from the `CREATE TABLE` statement it generates.
-
-    The spelling *keep default* explicitly prevents that behaviour and
-    can be used to overload the default casting rules.
-
-  - *drop not null*, *keep not null*, *set not null*
-
-    When the option *drop not null* is listed, pgloader drops any
-    existing `NOT NULL` constraint associated with the given source
-    MySQL datatype when it creates the tables in the PostgreSQL
-    database.
-
-    The spelling *keep not null* explicitly prevents that behaviour and
-    can be used to overload the default casting rules.
-
-    When the option *set not null* is listed, pgloader sets a `NOT NULL`
-    constraint on the target column regardless whether it has been set
-    in the source MySQL column.
-
-  - *drop typemod*, *keep typemod*
-
-    When the option *drop typemod* is listed, pgloader drops any
-    existing *typemod* definition (e.g. *precision* and *scale*) from
-    the datatype definition found in the MySQL columns of the source
-    type when it created the tables in the PostgreSQL database.
-
-    The spelling *keep typemod* explicitly prevents that behaviour and
-    can be used to overload the default casting rules.
-
-  - *using*
-
-    This option takes as its single argument the name of a function to
-    be found in the `pgloader.transforms` Common Lisp package. See above
-    for details.
-
-    It's possible to augment a default cast rule (such as one that
-    applies against `ENUM` data type for example) with a *transformation
-    function* by omitting entirely the `type` parts of the casting rule,
-    as in the following example::
-
-      column enumerate.foo using empty-string-to-null
-
-MySQL Views Support
--------------------
-
-MySQL views support allows pgloader to migrate view as if they were base
-tables. This feature then allows for on-the-fly transformation from MySQL to
-PostgreSQL, as the view definition is used rather than the base data.
-
-MATERIALIZE VIEWS
-^^^^^^^^^^^^^^^^^
-
-This clause allows you to implement custom data processing at the data
-source by providing a *view definition* against which pgloader will query
-the data. It's not possible to just allow for plain `SQL` because we want to
-know a lot about the exact data types of each column involved in the query
-output.
-
-This clause expect a comma separated list of view definitions, each one
-being either the name of an existing view in your database or the following
-expression::
-
-  *name* `AS` `$$` *sql query* `$$`
-
-The *name* and the *sql query* will be used in a `CREATE VIEW` statement at
-the beginning of the data loading, and the resulting view will then be
-dropped at the end of the data loading.
-
-MATERIALIZE ALL VIEWS
-^^^^^^^^^^^^^^^^^^^^^
-
-Same behaviour as *MATERIALIZE VIEWS* using the dynamic list of views as
-returned by MySQL rather than asking the user to specify the list.
-
-MySQL Partial Migration
------------------------
-
-INCLUDING ONLY TABLE NAMES MATCHING
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table names or *regular expression* used
-to limit the tables to migrate to a sublist.
-
-Example::
-
-  including only table names matching ~/film/, 'actor'
-
-EXCLUDING TABLE NAMES MATCHING
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table names or *regular expression* used
-to exclude table names from the migration. This filter only applies to the
-result of the *INCLUDING* filter.
-
-::
-  
-  excluding table names matching ~<ory>
-
-MySQL Encoding Support
-----------------------
-      
-DECODING TABLE NAMES MATCHING
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table names or *regular expressions*
-used to force the encoding to use when processing data from MySQL. If the
-data encoding known to you is different from MySQL's idea about it, this is
-the option to use.
-
-::
-  
-  decoding table names matching ~/messed/, ~/encoding/ AS utf8
-
-You can use as many such rules as you need, all with possibly different
-encodings.
-
-MySQL Schema Transformations
-----------------------------
-    
-ALTER TABLE NAMES MATCHING
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table names or *regular expressions*
-that you want to target in the pgloader *ALTER TABLE* command. The only two
-available actions are *SET SCHEMA* and *RENAME TO*, both take a quoted
-string as parameter::
-
-    ALTER TABLE NAMES MATCHING ~/_list$/, 'sales_by_store', ~/sales_by/
-     SET SCHEMA 'mv'
-   
-    ALTER TABLE NAMES MATCHING 'film' RENAME TO 'films'
-    
-    ALTER TABLE NAMES MATCHING ~/./ SET (fillfactor='40')
-
-You can use as many such rules as you need. The list of tables to be
-migrated is searched in pgloader memory against the *ALTER TABLE* matching
-rules, and for each command pgloader stops at the first matching criteria
-(regexp or string).
-
-No *ALTER TABLE* command is sent to PostgreSQL, the modification happens at
-the level of the pgloader in-memory representation of your source database
-schema. In case of a name change, the mapping is kept and reused in the
-*foreign key* and *index* support.
-
-The *SET ()* action takes effect as a *WITH* clause for the `CREATE TABLE`
-command that pgloader will run when it has to create a table.
-
-MySQL Migration: limitations
-----------------------------
-
-The `database` command currently only supports MySQL source database and has
-the following limitations:
-
-  - Views are not migrated,
-
-    Supporting views might require implementing a full SQL parser for the
-    MySQL dialect with a porting engine to rewrite the SQL against
-    PostgreSQL, including renaming functions and changing some constructs.
-
-    While it's not theoretically impossible, don't hold your breath.
-
-  - Triggers are not migrated
-
-    The difficulty of doing so is not yet assessed.
-
-  - Of the geometric datatypes, only the `POINT` database has been covered.
-    The other ones should be easy enough to implement now, it's just not
-    done yet.
-
-Default MySQL Casting Rules
----------------------------
-
-When migrating from MySQL the following Casting Rules are provided:
-
-Numbers::
-
-  type int with extra auto_increment to serial when (< precision 10)
-  type int with extra auto_increment to bigserial when (<= 10 precision)
-  type int to int       when  (< precision 10)
-  type int to bigint    when  (<= 10 precision)
-  type tinyint   with extra auto_increment to serial
-  type smallint  with extra auto_increment to serial
-  type mediumint with extra auto_increment to serial
-  type bigint    with extra auto_increment to bigserial
-
-  type tinyint to boolean when (= 1 precision) using tinyint-to-boolean
-
-  type tinyint when unsigned to smallint   drop typemod
-  type smallint when unsigned to integer  drop typemod
-  type mediumint when unsigned to integer  drop typemod
-  type integer when unsigned to bigint    drop typemod
-  
-  type tinyint to smallint   drop typemod
-  type smallint to smallint  drop typemod
-  type mediumint to integer  drop typemod
-  type integer to integer    drop typemod
-  type bigint to bigint      drop typemod
-
-  type float to float        drop typemod
-  type double to double precision drop typemod
-
-  type numeric to numeric keep typemod
-  type decimal to decimal keep typemod
-
-Texts::
-
-  type char       to char keep typemod using remove-null-characters
-  type varchar    to varchar keep typemod using remove-null-characters
-  type tinytext   to text using remove-null-characters
-  type text       to text using remove-null-characters
-  type mediumtext to text using remove-null-characters
-  type longtext   to text using remove-null-characters
-
-Binary:
-
-  type binary     to bytea
-  type varbinary  to bytea
-  type tinyblob   to bytea
-  type blob       to bytea
-  type mediumblob to bytea
-  type longblob   to bytea
-
-Date::
-  
-  type datetime when default "0000-00-00 00:00:00" and not null
-    to timestamptz drop not null drop default
-	using zero-dates-to-null
-
-  type datetime when default "0000-00-00 00:00:00"
-    to timestamptz drop default
-	using zero-dates-to-null
-
-  type timestamp when default "0000-00-00 00:00:00" and not null
-    to timestamptz drop not null drop default
-	using zero-dates-to-null
-
-  type timestamp when default "0000-00-00 00:00:00"
-    to timestamptz drop default
-	using zero-dates-to-null
-
-  type date when default "0000-00-00" to date drop default
-	using zero-dates-to-null
-
-  type date to date
-  type datetime to timestamptz
-  type timestamp to timestamptz
-  type year to integer drop typemod
-
-Geometric::
-
-  type point to point using pgloader.transforms::convert-mysql-point
-
-Enum types are declared inline in MySQL and separately with a `CREATE TYPE`
-command in PostgreSQL, so each column of Enum Type is converted to a type
-named after the table and column names defined with the same labels in the
-same order.
-
-When the source type definition is not matched in the default casting rules
-nor in the casting rules provided in the command, then the type name with
-the typemod is used.
-

docs/_build/html/_sources/ref/sqlite.rst.txt

@@ -1,207 +0,0 @@
-Migrating a SQLite database to PostgreSQL
-=========================================
-
-This command instructs pgloader to load data from a SQLite file. Automatic
-discovery of the schema is supported, including build of the indexes.
-
-Here's an example::
-
-    load database
-         from sqlite:///Users/dim/Downloads/lastfm_tags.db
-         into postgresql:///tags
-
-     with include drop, create tables, create indexes, reset sequences
-
-      set work_mem to '16MB', maintenance_work_mem to '512 MB';
-
-The `sqlite` command accepts the following clauses and options.
-
-SQLite Database Source Specification: FROM
-------------------------------------------
-
-Path or HTTP URL to a SQLite file, might be a `.zip` file.
-
-SQLite Database Migration Options: WITH
----------------------------------------
-
-When loading from a `SQLite` database, the following options are
-supported:
-
-When loading from a `SQLite` database, the following options are
-supported, and the default *WITH* clause is: *no truncate*, *create
-tables*, *include drop*, *create indexes*, *reset sequences*, *downcase
-identifiers*, *encoding 'utf-8'*.
-
-  - *include drop*
-
-    When this option is listed, pgloader drops all the tables in the target
-    PostgreSQL database whose names appear in the SQLite database. This
-    option allows for using the same command several times in a row until
-    you figure out all the options, starting automatically from a clean
-    environment. Please note that `CASCADE` is used to ensure that tables
-    are dropped even if there are foreign keys pointing to them. This is
-    precisely what `include drop` is intended to do: drop all target tables
-    and recreate them.
-
-    Great care needs to be taken when using `include drop`, as it will
-    cascade to *all* objects referencing the target tables, possibly
-    including other tables that are not being loaded from the source DB.
-
-  - *include no drop*
-
-    When this option is listed, pgloader will not include any `DROP`
-    statement when loading the data.
-
-  - *truncate*
-
-    When this option is listed, pgloader issue the `TRUNCATE` command
-    against each PostgreSQL table just before loading data into it.
-
-  - *no truncate*
-
-    When this option is listed, pgloader issues no `TRUNCATE` command.
-
-  - *disable triggers*
-
-    When this option is listed, pgloader issues an `ALTER TABLE ... DISABLE
-    TRIGGER ALL` command against the PostgreSQL target table before copying
-    the data, then the command `ALTER TABLE ... ENABLE TRIGGER ALL` once the
-    `COPY` is done.
-
-    This option allows loading data into a pre-existing table ignoring
-    the *foreign key constraints* and user defined triggers and may
-    result in invalid *foreign key constraints* once the data is loaded.
-    Use with care.
-
-  - *create tables*
-
-    When this option is listed, pgloader creates the table using the meta
-    data found in the `SQLite` file, which must contain a list of fields
-    with their data type. A standard data type conversion from SQLite to
-    PostgreSQL is done.
-
-  - *create no tables*
-
-    When this option is listed, pgloader skips the creation of table before
-    loading data, target tables must then already exist.
-
-    Also, when using *create no tables* pgloader fetches the metadata
-    from the current target database and checks type casting, then will
-    remove constraints and indexes prior to loading the data and install
-    them back again once the loading is done.
-
-  - *create indexes*
-
-    When this option is listed, pgloader gets the definitions of all the
-    indexes found in the SQLite database and create the same set of index
-    definitions against the PostgreSQL database.
-
-  - *create no indexes*
-
-    When this option is listed, pgloader skips the creating indexes.
-
-  - *drop indexes*
-  
-    When this option is listed, pgloader drops the indexes in the target
-    database before loading the data, and creates them again at the end
-    of the data copy.
-
-  - *reset sequences*
-
-    When this option is listed, at the end of the data loading and after
-    the indexes have all been created, pgloader resets all the
-    PostgreSQL sequences created to the current maximum value of the
-    column they are attached to.
-
-  - *reset no sequences*
-
-    When this option is listed, pgloader skips resetting sequences after the
-    load.
-
-    The options *schema only* and *data only* have no effects on this
-    option.
-
-  - *schema only*
-
-    When this option is listed pgloader will refrain from migrating the data
-    over. Note that the schema in this context includes the indexes when the
-    option *create indexes* has been listed.
-
-  - *data only*
-
-    When this option is listed pgloader only issues the `COPY` statements,
-    without doing any other processing.
-
-  - *encoding*
-
-    This option allows to control which encoding to parse the SQLite text
-    data with. Defaults to UTF-8.
-
-SQLite Database Casting Rules
------------------------------
-    
-The command *CAST* introduces user-defined casting rules.
-
-The cast clause allows to specify custom casting rules, either to overload
-the default casting rules or to amend them with special cases.
-
-SQlite Database Partial Migrations
-----------------------------------
-
-INCLUDING ONLY TABLE NAMES LIKE
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table name patterns used to limit the
-tables to migrate to a sublist.
-
-Example::
-
- including only table names like 'Invoice%'
-
-EXCLUDING TABLE NAMES LIKE
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Introduce a comma separated list of table name patterns used to exclude
-table names from the migration. This filter only applies to the result of
-the *INCLUDING* filter.
-
-::
-  
-  excluding table names like 'appointments'
-
-Default SQLite Casting Rules
-----------------------------
-
-When migrating from SQLite the following Casting Rules are provided:
-
-Numbers::
-
-  type tinyint to smallint using integer-to-string
-  type integer to bigint   using integer-to-string
-
-  type float to float   using float-to-string
-  type real to real     using float-to-string
-  type double to double precision     using float-to-string
-  type numeric to numeric     using float-to-string
-
-Texts::
-
-  type character  to text drop typemod
-  type varchar    to text drop typemod
-  type nvarchar   to text drop typemod
-  type char       to text drop typemod
-  type nchar      to text drop typemod
-  type nvarchar   to text drop typemod
-  type clob       to text drop typemod
-
-Binary::
-
-  type blob       to bytea
-
-Date::
-
-  type datetime    to timestamptz using sqlite-timestamp-to-timestamp
-  type timestamp   to timestamptz using sqlite-timestamp-to-timestamp
-  type timestamptz to timestamptz using sqlite-timestamp-to-timestamp
-
-

docs/_build/html/_sources/ref/transforms.rst.txt

@@ -1,134 +0,0 @@
-Transformation Functions
-========================
-
-Some data types are implemented in a different enough way that a
-transformation function is necessary. This function must be written in
-`Common lisp` and is searched in the `pgloader.transforms` package.
-
-Some default transformation function are provided with pgloader, and you can
-use the `--load` command line option to load and compile your own lisp file
-into pgloader at runtime. For your functions to be found, remember to begin
-your lisp file with the following form::
-
-    (in-package #:pgloader.transforms)
-
-The provided transformation functions are:
-
-  - *zero-dates-to-null*
-
-    When the input date is all zeroes, return `nil`, which gets loaded as a
-    PostgreSQL `NULL` value.
-
-  - *date-with-no-separator*
-
-    Applies *zero-dates-to-null* then transform the given date into a format
-    that PostgreSQL will actually process::
-
-            In:  "20041002152952"
-            Out: "2004-10-02 15:29:52"
-
-  - *time-with-no-separator*
-
-    Transform the given time into a format that PostgreSQL will actually
-    process::
-
-            In:  "08231560"
-            Out: "08:23:15.60"
-
-  - *tinyint-to-boolean*
-
-    As MySQL lacks a proper boolean type, *tinyint* is often used to
-    implement that. This function transforms `0` to `'false'` and anything
-    else to `'true`'.
-
-  - *bits-to-boolean*
-
-    As MySQL lacks a proper boolean type, *BIT* is often used to implement
-    that. This function transforms 1-bit bit vectors from `0` to `f` and any
-    other value to `t`..
-
-  - *int-to-ip*
-
-    Convert an integer into a dotted representation of an ip4. ::
-
-        In:  18435761
-        Out: "1.25.78.177"
-
-  - *ip-range*
-
-    Converts a couple of integers given as strings into a range of ip4. ::
-
-            In:  "16825344" "16825599"
-            Out: "1.0.188.0-1.0.188.255"
-
-  - *convert-mysql-point*
-
-    Converts from the `astext` representation of points in MySQL to the
-    PostgreSQL representation. ::
-
-        In:  "POINT(48.5513589 7.6926827)"
-        Out: "(48.5513589,7.6926827)"
-
-  - *integer-to-string*
-
-    Converts a integer string or a Common Lisp integer into a string
-    suitable for a PostgreSQL integer. Takes care of quoted integers. ::
-
-            In:  "\"0\""
-            Out: "0"
-
-  - *float-to-string*
-
-    Converts a Common Lisp float into a string suitable for a PostgreSQL float::
-
-            In:  100.0d0
-            Out: "100.0"
-
-  - *set-to-enum-array*
-
-    Converts a string representing a MySQL SET into a PostgreSQL Array of
-    Enum values from the set. ::
-
-            In: "foo,bar"
-            Out: "{foo,bar}"
-
-  - *empty-string-to-null*
-
-    Convert an empty string to a null.
-
-  - *right-trim*
-
-    Remove whitespace at end of string.
-
-  - *remove-null-characters*
-
-    Remove `NUL` characters (`0x0`) from given strings.
-
-  - *byte-vector-to-bytea*
-
-    Transform a simple array of unsigned bytes to the PostgreSQL bytea Hex
-    Format representation as documented at
-    http://www.postgresql.org/docs/9.3/interactive/datatype-binary.html
-
-  - *sqlite-timestamp-to-timestamp*
-
-    SQLite type system is quite interesting, so cope with it here to produce
-    timestamp literals as expected by PostgreSQL. That covers year only on 4
-    digits, 0 dates to null, and proper date strings.
-
-  - *sql-server-uniqueidentifier-to-uuid*
-
-    The SQL Server driver receives data fo type uniqueidentifier as byte
-    vector that we then need to convert to an UUID string for PostgreSQL
-    COPY input format to process.
-
-  - *unix-timestamp-to-timestamptz*
-
-    Converts a unix timestamp (number of seconds elapsed since beginning of
-    1970) into a proper PostgreSQL timestamp format.
-
-  - *varbinary-to-string*
-
-    Converts binary encoded string (such as a MySQL `varbinary` entry) to a
-    decoded text, using the table's encoding that may be overloaded with the
-    *DECODING TABLE NAMES MATCHING* clause.

docs/_build/html/_sources/tutorial/csv.rst.txt

@@ -1,85 +0,0 @@
-Loading CSV Data with pgloader
-------------------------------
-
-CSV means *comma separated values* and is often found with quite varying
-specifications. pgloader allows you to describe those specs in its command.
-
-The Command
-^^^^^^^^^^^
-
-To load data with [pgloader](http://pgloader.io/) you need to define in a
-*command* the operations in some details. Here's our example for loading CSV
-data::
-
-     LOAD CSV
-          FROM 'path/to/file.csv' (x, y, a, b, c, d)
-          INTO postgresql:///pgloader?csv (a, b, d, c)
-     
-          WITH truncate,
-               skip header = 1,
-               fields optionally enclosed by '"',
-               fields escaped by double-quote,
-               fields terminated by ','
-     
-           SET client_encoding to 'latin1',
-               work_mem to '12MB',
-               standard_conforming_strings to 'on'
-     
-        BEFORE LOAD DO
-         $$ drop table if exists csv; $$,
-         $$ create table csv (
-             a bigint,
-             b bigint,
-             c char(2),
-             d text
-            );
-       $$;
-
-The Data
-^^^^^^^^
-
-This command allows loading the following CSV file content::
-
-    Header, with a © sign
-    "2.6.190.56","2.6.190.63","33996344","33996351","GB","United Kingdom"
-    "3.0.0.0","4.17.135.31","50331648","68257567","US","United States"
-    "4.17.135.32","4.17.135.63","68257568","68257599","CA","Canada"
-    "4.17.135.64","4.17.142.255","68257600","68259583","US","United States"
-    "4.17.143.0","4.17.143.15","68259584","68259599","CA","Canada"
-    "4.17.143.16","4.18.32.71","68259600","68296775","US","United States"
-
-Loading the data
-^^^^^^^^^^^^^^^^
-
-Here's how to start loading the data. Note that the ouput here has been
-edited so as to facilitate its browsing online::
-
-    $ pgloader csv.load
-    ... LOG Starting pgloader, log system is ready.
-    ... LOG Parsing commands from file "/Users/dim/dev/pgloader/test/csv.load"
-    
-           table name       read   imported     errors            time
-    -----------------  ---------  ---------  ---------  --------------
-          before load          2          2          0          0.039s
-    -----------------  ---------  ---------  ---------  --------------
-                  csv          6          6          0          0.019s
-    -----------------  ---------  ---------  ---------  --------------
-    Total import time          6          6          0          0.058s
-
-The result
-^^^^^^^^^^
-
-As you can see, the command described above is filtering the input and only
-importing some of the columns from the example data file. Here's what gets
-loaded in the PostgreSQL database::
-
-    pgloader# table csv;
-        a     |    b     | c  |       d        
-    ----------+----------+----+----------------
-     33996344 | 33996351 | GB | United Kingdom
-     50331648 | 68257567 | US | United States
-     68257568 | 68257599 | CA | Canada
-     68257600 | 68259583 | US | United States
-     68259584 | 68259599 | CA | Canada
-     68259600 | 68296775 | US | United States
-    (6 rows)

docs/_build/html/_sources/tutorial/dBase.rst.txt

@@ -1,56 +0,0 @@
-Loading dBase files with pgloader
----------------------------------
-
-The dBase format is still in use in some places as modern tools such as
-*Filemaker* and *Excel* offer some level of support for it. Speaking of
-support in modern tools, pgloader is right there on the list too!
-
-The Command
-^^^^^^^^^^^
-
-To load data with [pgloader](http://pgloader.io/) you need to define in a
-*command* the operations in some details. Here's our example for loading a
-dBase file, using a file provided by the french administration.
-
-You can find more files from them at the `Insee
-<http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement.asp>`_
-website.
-
-Here's our command::
-
-    LOAD DBF
-        FROM http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/historiq2013.zip
-        INTO postgresql:///pgloader
-        WITH truncate, create table
-         SET client_encoding TO 'latin1';
-
-Note that here pgloader will benefit from the meta-data information found in
-the dBase file to create a PostgreSQL table capable of hosting the data as
-described, then load the data.
-
-Loading the data
-^^^^^^^^^^^^^^^^
-
-Let's start the `pgloader` command with our `dbf-zip.load` command file::
-
-    $ pgloader dbf-zip.load
-    ... LOG Starting pgloader, log system is ready.
-    ... LOG Parsing commands from file "/Users/dim/dev/pgloader/test/dbf-zip.load"
-    ... LOG Fetching 'http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/historiq2013.zip'
-    ... LOG Extracting files from archive '//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//historiq2013.zip'
-    
-           table name       read   imported     errors            time
-    -----------------  ---------  ---------  ---------  --------------
-             download          0          0          0          0.167s
-              extract          0          0          0          1.010s
-     create, truncate          0          0          0          0.071s
-    -----------------  ---------  ---------  ---------  --------------
-         historiq2013       9181       9181          0          0.658s
-    -----------------  ---------  ---------  ---------  --------------
-    Total import time       9181       9181          0          1.906s
-
-We can see that `pgloader <http://pgloader.io>`_ did download the file from
-its HTTP URL location then *unziped* it before the loading itself.
-
-Note that the output of the command has been edited to facilitate its
-browsing online.

docs/_build/html/_sources/tutorial/fixed.rst.txt

@@ -1,96 +0,0 @@
-Loading Fixed Width Data File with pgloader
--------------------------------------------
-
-Some data providers still use a format where each column is specified with a
-starting index position and a given length. Usually the columns are
-blank-padded when the data is shorter than the full reserved range.
-
-The Command
-^^^^^^^^^^^
-
-To load data with [pgloader](http://pgloader.io/) you need to define in a
-*command* the operations in some details. Here's our example for loading
-Fixed Width Data, using a file provided by the US census.
-
-You can find more files from them at the
-[Census 2000 Gazetteer Files](http://www.census.gov/geo/maps-data/data/gazetteer2000.html).
-
-Here's our command::
-
-    LOAD ARCHIVE
-       FROM http://www.census.gov/geo/maps-data/data/docs/gazetteer/places2k.zip
-       INTO postgresql:///pgloader
-    
-       BEFORE LOAD DO
-         $$ drop table if exists places; $$,
-         $$ create table places
-           (
-              usps      char(2)  not null,
-              fips      char(2)  not null,
-              fips_code char(5),
-              loc_name  varchar(64)
-           );
-         $$
-         
-       LOAD FIXED
-            FROM FILENAME MATCHING ~/places2k.txt/
-                 WITH ENCODING latin1
-                 (
-                    usps           from   0 for  2,
-                    fips           from   2 for  2,
-                    fips_code      from   4 for  5,
-                    "LocationName" from   9 for 64 [trim right whitespace],
-                    p              from  73 for  9,
-                    h              from  82 for  9,
-                    land           from  91 for 14,
-                    water          from 105 for 14,
-                    ldm            from 119 for 14,
-                    wtm            from 131 for 14,
-                    lat            from 143 for 10,
-                    long           from 153 for 11
-                 )
-            INTO postgresql:///pgloader?places
-                 (
-    	        usps, fips, fips_code, "LocationName"
-                 );
-
-The Data
-^^^^^^^^
-
-This command allows loading the following file content, where we are only
-showing the first couple of lines::
-
-    AL0100124Abbeville city                                                       2987     1353      40301945        120383   15.560669    0.046480 31.566367 -85.251300
-    AL0100460Adamsville city                                                      4965     2042      50779330         14126   19.606010    0.005454 33.590411 -86.949166
-    AL0100484Addison town                                                          723      339       9101325             0    3.514041    0.000000 34.200042 -87.177851
-    AL0100676Akron town                                                            521      239       1436797             0    0.554750    0.000000 32.876425 -87.740978
-    AL0100820Alabaster city                                                      22619     8594      53023800        141711   20.472605    0.054715 33.231162 -86.823829
-    AL0100988Albertville city                                                    17247     7090      67212867        258738   25.951034    0.099899 34.265362 -86.211261
-    AL0101132Alexander City city                                                 15008     6855     100534344        433413   38.816529    0.167342 32.933157 -85.936008
-
-Loading the data
-^^^^^^^^^^^^^^^^
-
-Let's start the `pgloader` command with our `census-places.load` command file::
-
-    $ pgloader census-places.load
-    ... LOG Starting pgloader, log system is ready.
-    ... LOG Parsing commands from file "/Users/dim/dev/pgloader/test/census-places.load"
-    ... LOG Fetching 'http://www.census.gov/geo/maps-data/data/docs/gazetteer/places2k.zip'
-    ... LOG Extracting files from archive '//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//places2k.zip'
-    
-           table name       read   imported     errors            time
-    -----------------  ---------  ---------  ---------  --------------
-             download          0          0          0          1.494s
-              extract          0          0          0          1.013s
-          before load          2          2          0          0.013s
-    -----------------  ---------  ---------  ---------  --------------
-               places      25375      25375          0          0.499s
-    -----------------  ---------  ---------  ---------  --------------
-    Total import time      25375      25375          0          3.019s
-
-We can see that pgloader did download the file from its HTTP URL location
-then *unziped* it before the loading itself.
-
-Note that the output of the command has been edited to facilitate its
-browsing online.

docs/_build/html/_sources/tutorial/geolite.rst.txt

@@ -1,159 +0,0 @@
-Loading MaxMind Geolite Data with pgloader
-------------------------------------------
-
-`MaxMind <http://www.maxmind.com/>`_ provides a free dataset for
-geolocation, which is quite popular. Using pgloader you can download the
-lastest version of it, extract the CSV files from the archive and load their
-content into your database directly.
-
-The Command
-^^^^^^^^^^^
-
-To load data with pgloader you need to define in a *command* the operations
-in some details. Here's our example for loading the Geolite data::
-
-    /*
-     * Loading from a ZIP archive containing CSV files. The full test can be
-     * done with using the archive found at
-     * http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip
-     *
-     * And a very light version of this data set is found at
-     * http://pgsql.tapoueh.org/temp/foo.zip for quick testing.
-     */
-    
-    LOAD ARCHIVE
-       FROM http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip
-       INTO postgresql:///ip4r
-    
-       BEFORE LOAD DO
-         $$ create extension if not exists ip4r; $$,
-         $$ create schema if not exists geolite; $$,
-         $$ create table if not exists geolite.location
-           (
-              locid      integer primary key,
-              country    text,
-              region     text,
-              city       text,
-              postalcode text,
-              location   point,
-              metrocode  text,
-              areacode   text
-           );
-         $$,
-         $$ create table if not exists geolite.blocks
-           (
-              iprange    ip4r,
-              locid      integer
-           );
-         $$,
-         $$ drop index if exists geolite.blocks_ip4r_idx; $$,
-         $$ truncate table geolite.blocks, geolite.location cascade; $$
-    
-       LOAD CSV
-            FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
-                 WITH ENCODING iso-8859-1
-                 (
-                    locId,
-                    country,
-                    region     null if blanks,
-                    city       null if blanks,
-                    postalCode null if blanks,
-                    latitude,
-                    longitude,
-                    metroCode  null if blanks,
-                    areaCode   null if blanks
-                 )
-            INTO postgresql:///ip4r?geolite.location
-                 (
-                    locid,country,region,city,postalCode,
-                    location point using (format nil "(~a,~a)" longitude latitude),
-                    metroCode,areaCode
-                 )
-            WITH skip header = 2,
-                 fields optionally enclosed by '"',
-                 fields escaped by double-quote,
-                 fields terminated by ','
-    
-      AND LOAD CSV
-            FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
-                 WITH ENCODING iso-8859-1
-                 (
-                    startIpNum, endIpNum, locId
-                 )
-            INTO postgresql:///ip4r?geolite.blocks
-                 (
-                    iprange ip4r using (ip-range startIpNum endIpNum),
-                    locId
-                 )
-            WITH skip header = 2,
-                 fields optionally enclosed by '"',
-                 fields escaped by double-quote,
-                 fields terminated by ','
-    
-       FINALLY DO
-         $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;
-
-Note that while the *Geolite* data is using a pair of integers (*start*,
-*end*) to represent *ipv4* data, we use the very poweful `ip4r
-<https://github.com/RhodiumToad/ip4r>`_ PostgreSQL Extension instead.
-
-The transformation from a pair of integers into an IP is done dynamically by
-the pgloader process.
-
-Also, the location is given as a pair of *float* columns for the *longitude*
-and the *latitude* where PostgreSQL offers the
-`point <http://www.postgresql.org/docs/9.3/interactive/functions-geometry.html>`_
-datatype, so the pgloader command here will actually transform the data on
-the fly to use the appropriate data type and its input representation.
-
-Loading the data
-^^^^^^^^^^^^^^^^
-
-Here's how to start loading the data. Note that the ouput here has been
-edited so as to facilitate its browsing online::
-
-    $ pgloader archive.load
-    ... LOG Starting pgloader, log system is ready.
-    ... LOG Parsing commands from file "/Users/dim/dev/pgloader/test/archive.load"
-    ... LOG Fetching 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip'
-    ... LOG Extracting files from archive '//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//GeoLiteCity-latest.zip'
-    
-           table name       read   imported     errors            time
-    -----------------  ---------  ---------  ---------  --------------
-             download          0          0          0         11.592s
-              extract          0          0          0          1.012s
-          before load          6          6          0          0.019s
-    -----------------  ---------  ---------  ---------  --------------
-     geolite.location     470387     470387          0          7.743s
-       geolite.blocks    1903155    1903155          0         16.332s
-    -----------------  ---------  ---------  ---------  --------------
-              finally          1          1          0         31.692s
-    -----------------  ---------  ---------  ---------  --------------
-    Total import time    2373542    2373542          0        1m8.390s
-
-The timing of course includes the transformation of the *1.9 million* pairs
-of integer into a single *ipv4 range* each. The *finally* step consists of
-creating the *GiST* specialized index as given in the main command::
-
-    CREATE INDEX blocks_ip4r_idx ON geolite.blocks USING gist(iprange);
-
-That index will then be used to speed up queries wanting to find which
-recorded geolocation contains a specific IP address::
-
-    ip4r> select *
-            from      geolite.location l
-                 join geolite.blocks b using(locid)
-           where iprange >>= '8.8.8.8';
-           
-    -[ RECORD 1 ]------------------
-    locid      | 223
-    country    | US
-    region     | 
-    city       | 
-    postalcode | 
-    location   | (-97,38)
-    metrocode  | 
-    areacode   | 
-    iprange    | 8.8.8.8-8.8.37.255
-    
-    Time: 0.747 ms

docs/_build/html/_sources/tutorial/mysql.rst.txt

@@ -1,177 +0,0 @@
-Migrating from MySQL to PostgreSQL
-----------------------------------
-
-If you want to migrate your data over to `PostgreSQL
-<http://www.postgresql.org>`_ from MySQL then pgloader is the tool of
-choice!
-
-Most tools around are skipping the main problem with migrating from MySQL,
-which is to do with the type casting and data sanitizing that needs to be
-done. pgloader will not leave you alone on those topics.
-
-In a Single Command Line
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-As an example, we will use the f1db database from <http://ergast.com/mrd/>
-which which provides a historical record of motor racing data for
-non-commercial purposes. You can either use their API or download the whole
-database at `http://ergast.com/downloads/f1db.sql.gz
-<http://ergast.com/downloads/f1db.sql.gz>`_. Once you've done that load the
-database in MySQL::
-
-    $ mysql -u root
-    > create database f1db;
-    > source f1db.sql
-
-Now let's migrate this database into PostgreSQL in a single command line::
-
-    $ createdb f1db
-    $ pgloader mysql://root@localhost/f1db pgsql:///f1db
-
-Done! All with schema, table definitions, constraints, indexes, primary
-keys, *auto_increment* columns turned into *bigserial* , foreign keys,
-comments, and if you had some MySQL default values such as *ON UPDATE
-CURRENT_TIMESTAMP* they would have been translated to a `PostgreSQL before
-update trigger
-<https://www.postgresql.org/docs/current/static/plpgsql-trigger.html>`_
-automatically.
-
-::
-
-    $ pgloader mysql://root@localhost/f1db pgsql:///f1db
-    2017-06-16T08:56:14.064000+02:00 LOG Main logs in '/private/tmp/pgloader/pgloader.log'
-    2017-06-16T08:56:14.068000+02:00 LOG Data errors in '/private/tmp/pgloader/'
-    2017-06-16T08:56:19.542000+02:00 LOG report summary reset
-                   table name       read   imported     errors      total time
-    -------------------------  ---------  ---------  ---------  --------------
-              fetch meta data         33         33          0          0.365s 
-               Create Schemas          0          0          0          0.007s 
-             Create SQL Types          0          0          0          0.006s 
-                Create tables         26         26          0          0.068s 
-               Set Table OIDs         13         13          0          0.012s 
-    -------------------------  ---------  ---------  ---------  --------------
-      f1db.constructorresults      11011      11011          0          0.205s 
-                f1db.circuits         73         73          0          0.150s 
-            f1db.constructors        208        208          0          0.059s 
-    f1db.constructorstandings      11766      11766          0          0.365s 
-                 f1db.drivers        841        841          0          0.268s 
-                f1db.laptimes     413578     413578          0          2.892s 
-         f1db.driverstandings      31420      31420          0          0.583s 
-                f1db.pitstops       5796       5796          0          2.154s 
-                   f1db.races        976        976          0          0.227s 
-              f1db.qualifying       7257       7257          0          0.228s 
-                 f1db.seasons         68         68          0          0.527s 
-                 f1db.results      23514      23514          0          0.658s 
-                  f1db.status        133        133          0          0.130s 
-    -------------------------  ---------  ---------  ---------  --------------
-      COPY Threads Completion         39         39          0          4.303s 
-               Create Indexes         20         20          0          1.497s 
-       Index Build Completion         20         20          0          0.214s 
-              Reset Sequences          0         10          0          0.058s 
-                 Primary Keys         13         13          0          0.012s 
-          Create Foreign Keys          0          0          0          0.000s 
-              Create Triggers          0          0          0          0.001s 
-             Install Comments          0          0          0          0.000s 
-    -------------------------  ---------  ---------  ---------  --------------
-            Total import time     506641     506641          0          5.547s 
-
-You may need to have special cases to take care of tho, or views that you
-want to materialize while doing the migration. In advanced case you can use
-the pgloader command.
-
-The Command
-^^^^^^^^^^^
-
-To load data with pgloader you need to define in a *command* the operations
-in some details. Here's our example for loading the `MySQL Sakila Sample
-Database <http://dev.mysql.com/doc/sakila/en/>`_.
-
-Here's our command::
-
-    load database
-         from      mysql://root@localhost/sakila
-         into postgresql:///sakila
-    
-     WITH include drop, create tables, no truncate,
-          create indexes, reset sequences, foreign keys
-    
-      SET maintenance_work_mem to '128MB', work_mem to '12MB', search_path to 'sakila'
-    
-     CAST type datetime to timestamptz
-                    drop default drop not null using zero-dates-to-null,
-          type date drop not null drop default using zero-dates-to-null
-         
-     MATERIALIZE VIEWS film_list, staff_list
-    
-     -- INCLUDING ONLY TABLE NAMES MATCHING ~/film/, 'actor'
-     -- EXCLUDING TABLE NAMES MATCHING ~<ory>
-    
-     BEFORE LOAD DO
-     $$ create schema if not exists sakila; $$;
-
-Note that here pgloader will benefit from the meta-data information found in
-the MySQL database to create a PostgreSQL database capable of hosting the
-data as described, then load the data.
-
-In particular, some specific *casting rules* are given here, to cope with
-date values such as `0000-00-00` that MySQL allows and PostgreSQL rejects
-for not existing in our calendar. It's possible to add per-column casting
-rules too, which is useful is some of your `tinyint` are in fact `smallint`
-while some others are in fact `boolean` values.
-
-Finaly note that we are using the *MATERIALIZE VIEWS* clause of pgloader:
-the selected views here will be migrated over to PostgreSQL *with their
-contents*.
-
-It's possible to use the *MATERIALIZE VIEWS* clause and give both the name
-and the SQL (in MySQL dialect) definition of view, then pgloader creates the
-view before loading the data, then drops it again at the end.
-
-## Loading the data
-
-Let's start the `pgloader` command with our `sakila.load` command file::
-
-    $ pgloader sakila.load
-    ... LOG Starting pgloader, log system is ready.
-    ... LOG Parsing commands from file "/Users/dim/dev/pgloader/test/sakila.load"
-       <WARNING: table "xxx" does not exists have been edited away>
-    
-                table name       read   imported     errors            time
-    ----------------------  ---------  ---------  ---------  --------------
-               before load          1          1          0          0.007s
-           fetch meta data         45         45          0          0.402s
-              create, drop          0         36          0          0.208s
-    ----------------------  ---------  ---------  ---------  --------------
-                     actor        200        200          0          0.071s
-                   address        603        603          0          0.035s
-                  category         16         16          0          0.018s
-                      city        600        600          0          0.037s
-                   country        109        109          0          0.023s
-                  customer        599        599          0          0.073s
-                      film       1000       1000          0          0.135s
-                film_actor       5462       5462          0          0.236s
-             film_category       1000       1000          0          0.070s
-                 film_text       1000       1000          0          0.080s
-                 inventory       4581       4581          0          0.136s
-                  language          6          6          0          0.036s
-                   payment      16049      16049          0          0.539s
-                    rental      16044      16044          0          0.648s
-                     staff          2          2          0          0.041s
-                     store          2          2          0          0.036s
-                 film_list        997        997          0          0.247s
-                staff_list          2          2          0          0.135s
-    Index Build Completion          0          0          0          0.000s
-    ----------------------  ---------  ---------  ---------  --------------
-            Create Indexes         41         41          0          0.964s
-           Reset Sequences          0          1          0          0.035s
-              Foreign Keys         22         22          0          0.254s
-    ----------------------  ---------  ---------  ---------  --------------
-         Total import time      48272      48272          0          3.502s
-
-The *WARNING* messages we see here are expected as the PostgreSQL database
-is empty when running the command, and pgloader is using the SQL commands
-`DROP TABLE IF EXISTS` when the given command uses the `include drop`
-option.
-
-Note that the output of the command has been edited to facilitate its
-browsing online.

docs/_build/html/_sources/tutorial/quickstart.rst.txt

@@ -1,142 +0,0 @@
-PgLoader Quick Start
---------------------
-
-In simple cases, pgloader is very easy to use.
-
-CSV
-^^^
-
-Load data from a CSV file into a pre-existing table in your database::
-
-    pgloader --type csv                                   \
-             --field id --field field                     \
-             --with truncate                              \
-             --with "fields terminated by ','"            \
-             ./test/data/matching-1.csv                   \
-             postgres:///pgloader?tablename=matching
-
-In that example the whole loading is driven from the command line, bypassing
-the need for writing a command in the pgloader command syntax entirely. As
-there's no command though, the extra information needed must be provided on
-the command line using the `--type` and `--field` and `--with` switches.
-
-For documentation about the available syntaxes for the `--field` and
-`--with` switches, please refer to the CSV section later in the man page.
-
-Note also that the PostgreSQL URI includes the target *tablename*.
-
-Reading from STDIN
-^^^^^^^^^^^^^^^^^^
-
-File based pgloader sources can be loaded from the standard input, as in the
-following example::
-
-    pgloader --type csv                                         \
-             --field "usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong" \
-             --with "skip header = 1"                          \
-             --with "fields terminated by '\t'"                \
-             -                                                 \
-             postgresql:///pgloader?districts_longlat          \
-             < test/data/2013_Gaz_113CDs_national.txt
-
-The dash (`-`) character as a source is used to mean *standard input*, as
-usual in Unix command lines. It's possible to stream compressed content to
-pgloader with this technique, using the Unix pipe::
-
-    gunzip -c source.gz | pgloader --type csv ... - pgsql:///target?foo
-
-Loading from CSV available through HTTP
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The same command as just above can also be run if the CSV file happens to be
-found on a remote HTTP location::
-
-    pgloader --type csv                                                     \
-             --field "usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong" \
-             --with "skip header = 1"                                       \
-             --with "fields terminated by '\t'"                             \
-             http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt     \
-             postgresql:///pgloader?districts_longlat
-
-Some more options have to be used in that case, as the file contains a
-one-line header (most commonly that's column names, could be a copyright
-notice). Also, in that case, we specify all the fields right into a single
-`--field` option argument.
-
-Again, the PostgreSQL target connection string must contain the *tablename*
-option and you have to ensure that the target table exists and may fit the
-data. Here's the SQL command used in that example in case you want to try it
-yourself::
-
-    create table districts_longlat
-    (
-             usps        text,
-             geoid       text,
-             aland       bigint,
-             awater      bigint,
-             aland_sqmi  double precision,
-             awater_sqmi double precision,
-             intptlat    double precision,
-             intptlong   double precision
-    );
-
-Also notice that the same command will work against an archived version of
-the same data.
-
-Streaming CSV data from an HTTP compressed file
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Finally, it's important to note that pgloader first fetches the content from
-the HTTP URL it to a local file, then expand the archive when it's
-recognized to be one, and only then processes the locally expanded file.
-
-In some cases, either because pgloader has no direct support for your
-archive format or maybe because expanding the archive is not feasible in
-your environment, you might want to *stream* the content straight from its
-remote location into PostgreSQL. Here's how to do that, using the old battle
-tested Unix Pipes trick::
-
-    curl http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz \
-    | gunzip -c                                                        \
-    | pgloader --type csv                                              \
-               --field "usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong"
-               --with "skip header = 1"                                \
-               --with "fields terminated by '\t'"                      \
-               -                                                       \
-               postgresql:///pgloader?districts_longlat
-
-Now the OS will take care of the streaming and buffering between the network
-and the commands and pgloader will take care of streaming the data down to
-PostgreSQL.
-
-Migrating from SQLite
-^^^^^^^^^^^^^^^^^^^^^
-
-The following command will open the SQLite database, discover its tables
-definitions including indexes and foreign keys, migrate those definitions
-while *casting* the data type specifications to their PostgreSQL equivalent
-and then migrate the data over::
-
-    createdb newdb
-    pgloader ./test/sqlite/sqlite.db postgresql:///newdb
-
-Migrating from MySQL
-^^^^^^^^^^^^^^^^^^^^
-
-Just create a database where to host the MySQL data and definitions and have
-pgloader do the migration for you in a single command line::
-
-    createdb pagila
-    pgloader mysql://user@localhost/sakila postgresql:///pagila
-
-Fetching an archived DBF file from a HTTP remote location
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-It's possible for pgloader to download a file from HTTP, unarchive it, and
-only then open it to discover the schema then load the data::
-
-    createdb foo
-    pgloader --type dbf http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/historiq2013.zip postgresql:///foo
-
-Here it's not possible for pgloader to guess the kind of data source it's
-being given, so it's necessary to use the `--type` command line switch.

docs/_build/html/_sources/tutorial/sqlite.rst.txt

@@ -1,129 +0,0 @@
-Loading SQLite files with pgloader
-----------------------------------
-
-The SQLite database is a respected solution to manage your data with. Its
-embeded nature makes it a source of migrations when a projects now needs to
-handle more concurrency, which [PostgreSQL](http://www.postgresql.org/) is
-very good at. pgloader can help you there.
-
-In a Single Command Line
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-You can ::
-
-    $ createdb chinook
-    $ pgloader https://github.com/lerocha/chinook-database/raw/master/ChinookDatabase/DataSources/Chinook_Sqlite_AutoIncrementPKs.sqlite pgsql:///chinook
-
-Done! All with the schema, data, constraints, primary keys and foreign keys,
-etc. We also see an error with the Chinook schema that contains several
-primary key definitions against the same table, which is not accepted by
-PostgreSQL::
-
-    2017-06-20T16:18:59.019000+02:00 LOG Data errors in '/private/tmp/pgloader/'
-    2017-06-20T16:18:59.236000+02:00 LOG Fetching 'https://github.com/lerocha/chinook-database/raw/master/ChinookDatabase/DataSources/Chinook_Sqlite_AutoIncrementPKs.sqlite'
-    2017-06-20T16:19:00.664000+02:00 ERROR Database error 42P16: multiple primary keys for table "playlisttrack" are not allowed
-    QUERY: ALTER TABLE playlisttrack ADD PRIMARY KEY USING INDEX idx_66873_sqlite_autoindex_playlisttrack_1;
-    2017-06-20T16:19:00.665000+02:00 LOG report summary reset
-                 table name       read   imported     errors      total time
-    -----------------------  ---------  ---------  ---------  --------------
-                      fetch          0          0          0          0.877s 
-            fetch meta data         33         33          0          0.033s 
-             Create Schemas          0          0          0          0.003s 
-           Create SQL Types          0          0          0          0.006s 
-              Create tables         22         22          0          0.043s 
-             Set Table OIDs         11         11          0          0.012s 
-    -----------------------  ---------  ---------  ---------  --------------
-                      album        347        347          0          0.023s 
-                     artist        275        275          0          0.023s 
-                   customer         59         59          0          0.021s 
-                   employee          8          8          0          0.018s 
-                    invoice        412        412          0          0.031s 
-                      genre         25         25          0          0.021s 
-                invoiceline       2240       2240          0          0.034s 
-                  mediatype          5          5          0          0.025s 
-              playlisttrack       8715       8715          0          0.040s 
-                   playlist         18         18          0          0.016s 
-                      track       3503       3503          0          0.111s 
-    -----------------------  ---------  ---------  ---------  --------------
-    COPY Threads Completion         33         33          0          0.313s 
-             Create Indexes         22         22          0          0.160s 
-     Index Build Completion         22         22          0          0.027s 
-            Reset Sequences          0          0          0          0.017s 
-               Primary Keys         12          0          1          0.013s 
-        Create Foreign Keys         11         11          0          0.040s 
-            Create Triggers          0          0          0          0.000s 
-           Install Comments          0          0          0          0.000s 
-    -----------------------  ---------  ---------  ---------  --------------
-          Total import time      15607      15607          0          1.669s 
-
-You may need to have special cases to take care of tho. In advanced case you
-can use the pgloader command.
-
-The Command
-^^^^^^^^^^^
-
-To load data with [pgloader](http://pgloader.io/) you need to define in a
-*command* the operations in some details. Here's our command::
-
-    load database
-         from 'sqlite/Chinook_Sqlite_AutoIncrementPKs.sqlite'
-         into postgresql:///pgloader
-    
-     with include drop, create tables, create indexes, reset sequences
-    
-      set work_mem to '16MB', maintenance_work_mem to '512 MB';
-
-Note that here pgloader will benefit from the meta-data information found in
-the SQLite file to create a PostgreSQL database capable of hosting the data
-as described, then load the data.
-
-Loading the data
-^^^^^^^^^^^^^^^^
-
-Let's start the `pgloader` command with our `sqlite.load` command file::
-
-    $ pgloader sqlite.load
-    ... LOG Starting pgloader, log system is ready.
-    ... LOG Parsing commands from file "/Users/dim/dev/pgloader/test/sqlite.load"
-    ... WARNING Postgres warning: table "album" does not exist, skipping
-    ... WARNING Postgres warning: table "artist" does not exist, skipping
-    ... WARNING Postgres warning: table "customer" does not exist, skipping
-    ... WARNING Postgres warning: table "employee" does not exist, skipping
-    ... WARNING Postgres warning: table "genre" does not exist, skipping
-    ... WARNING Postgres warning: table "invoice" does not exist, skipping
-    ... WARNING Postgres warning: table "invoiceline" does not exist, skipping
-    ... WARNING Postgres warning: table "mediatype" does not exist, skipping
-    ... WARNING Postgres warning: table "playlist" does not exist, skipping
-    ... WARNING Postgres warning: table "playlisttrack" does not exist, skipping
-    ... WARNING Postgres warning: table "track" does not exist, skipping
-                table name       read   imported     errors            time
-    ----------------------  ---------  ---------  ---------  --------------
-          create, truncate          0          0          0          0.052s
-                     Album        347        347          0          0.070s
-                    Artist        275        275          0          0.014s
-                  Customer         59         59          0          0.014s
-                  Employee          8          8          0          0.012s
-                     Genre         25         25          0          0.018s
-                   Invoice        412        412          0          0.032s
-               InvoiceLine       2240       2240          0          0.077s
-                 MediaType          5          5          0          0.012s
-                  Playlist         18         18          0          0.008s
-             PlaylistTrack       8715       8715          0          0.071s
-                     Track       3503       3503          0          0.105s
-    index build completion          0          0          0          0.000s
-    ----------------------  ---------  ---------  ---------  --------------
-            Create Indexes         20         20          0          0.279s
-           reset sequences          0          0          0          0.043s
-    ----------------------  ---------  ---------  ---------  --------------
-      Total streaming time      15607      15607          0          0.476s
-
-We can see that `pgloader <http://pgloader.io>`_ did download the file from
-its HTTP URL location then *unziped* it before loading it.
-
-Also, the *WARNING* messages we see here are expected as the PostgreSQL
-database is empty when running the command, and pgloader is using the SQL
-commands `DROP TABLE IF EXISTS` when the given command uses the `include
-drop` option.
-
-Note that the output of the command has been edited to facilitate its
-browsing online.

docs/_build/html/_sources/tutorial/tutorial.rst.txt

@@ -1,10 +0,0 @@
-PgLoader Tutorial
-=================
-
-.. include:: quickstart.rst
-.. include:: csv.rst
-.. include:: fixed.rst
-.. include:: geolite.rst
-.. include:: dBase.rst
-.. include:: sqlite.rst
-.. include:: mysql.rst

docs/_build/html/_static/alabaster.css

@@ -1,689 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-@import url("basic.css");
-
-/* -- page layout ----------------------------------------------------------- */
-
-body {
-    font-family: 'goudy old style', 'minion pro', 'bell mt', Georgia, 'Hiragino Mincho Pro', serif;
-    font-size: 17px;
-    background-color: #fff;
-    color: #000;
-    margin: 0;
-    padding: 0;
-}
-
-
-div.document {
-    width: 940px;
-    margin: 30px auto 0 auto;
-}
-
-div.documentwrapper {
-    float: left;
-    width: 100%;
-}
-
-div.bodywrapper {
-    margin: 0 0 0 220px;
-}
-
-div.sphinxsidebar {
-    width: 220px;
-    font-size: 14px;
-    line-height: 1.5;
-}
-
-hr {
-    border: 1px solid #B1B4B6;
-}
-
-div.body {
-    background-color: #fff;
-    color: #3E4349;
-    padding: 0 30px 0 30px;
-}
-
-div.body > .section {
-    text-align: left;
-}
-
-div.footer {
-    width: 940px;
-    margin: 20px auto 30px auto;
-    font-size: 14px;
-    color: #888;
-    text-align: right;
-}
-
-div.footer a {
-    color: #888;
-}
-
-p.caption {
-    font-family: inherit;
-    font-size: inherit;
-}
-
-
-
-div.sphinxsidebar a {
-    color: #444;
-    text-decoration: none;
-    border-bottom: 1px dotted #999;
-}
-
-div.sphinxsidebar a:hover {
-    border-bottom: 1px solid #999;
-}
-
-div.sphinxsidebarwrapper {
-    padding: 18px 10px;
-}
-
-div.sphinxsidebarwrapper p.logo {
-    padding: 0;
-    margin: -10px 0 0 0px;
-    text-align: center;
-}
-
-div.sphinxsidebarwrapper h1.logo {
-    margin-top: -10px;
-    text-align: center;
-    margin-bottom: 5px;
-    text-align: left;
-}
-
-div.sphinxsidebarwrapper h1.logo-name {
-    margin-top: 0px;
-}
-
-div.sphinxsidebarwrapper p.blurb {
-    margin-top: 0;
-    font-style: normal;
-}
-
-div.sphinxsidebar h3,
-div.sphinxsidebar h4 {
-    font-family: 'Garamond', 'Georgia', serif;
-    color: #444;
-    font-size: 24px;
-    font-weight: normal;
-    margin: 0 0 5px 0;
-    padding: 0;
-}
-
-div.sphinxsidebar h4 {
-    font-size: 20px;
-}
-
-div.sphinxsidebar h3 a {
-    color: #444;
-}
-
-div.sphinxsidebar p.logo a,
-div.sphinxsidebar h3 a,
-div.sphinxsidebar p.logo a:hover,
-div.sphinxsidebar h3 a:hover {
-    border: none;
-}
-
-div.sphinxsidebar p {
-    color: #555;
-    margin: 10px 0;
-}
-
-div.sphinxsidebar ul {
-    margin: 10px 0;
-    padding: 0;
-    color: #000;
-}
-
-div.sphinxsidebar ul li.toctree-l1 > a {
-    font-size: 120%;
-}
-
-div.sphinxsidebar ul li.toctree-l2 > a {
-    font-size: 110%;
-}
-
-div.sphinxsidebar input {
-    border: 1px solid #CCC;
-    font-family: 'goudy old style', 'minion pro', 'bell mt', Georgia, 'Hiragino Mincho Pro', serif;
-    font-size: 1em;
-}
-
-div.sphinxsidebar hr {
-    border: none;
-    height: 1px;
-    color: #AAA;
-    background: #AAA;
-
-    text-align: left;
-    margin-left: 0;
-    width: 50%;
-}
-
-/* -- body styles ----------------------------------------------------------- */
-
-a {
-    color: #004B6B;
-    text-decoration: underline;
-}
-
-a:hover {
-    color: #6D4100;
-    text-decoration: underline;
-}
-
-div.body h1,
-div.body h2,
-div.body h3,
-div.body h4,
-div.body h5,
-div.body h6 {
-    font-family: 'Garamond', 'Georgia', serif;
-    font-weight: normal;
-    margin: 30px 0px 10px 0px;
-    padding: 0;
-}
-
-div.body h1 { margin-top: 0; padding-top: 0; font-size: 240%; }
-div.body h2 { font-size: 180%; }
-div.body h3 { font-size: 150%; }
-div.body h4 { font-size: 130%; }
-div.body h5 { font-size: 100%; }
-div.body h6 { font-size: 100%; }
-
-a.headerlink {
-    color: #DDD;
-    padding: 0 4px;
-    text-decoration: none;
-}
-
-a.headerlink:hover {
-    color: #444;
-    background: #EAEAEA;
-}
-
-div.body p, div.body dd, div.body li {
-    line-height: 1.4em;
-}
-
-div.admonition {
-    margin: 20px 0px;
-    padding: 10px 30px;
-    background-color: #EEE;
-    border: 1px solid #CCC;
-}
-
-div.admonition tt.xref, div.admonition code.xref, div.admonition a tt {
-    background-color: #FBFBFB;
-    border-bottom: 1px solid #fafafa;
-}
-
-div.admonition p.admonition-title {
-    font-family: 'Garamond', 'Georgia', serif;
-    font-weight: normal;
-    font-size: 24px;
-    margin: 0 0 10px 0;
-    padding: 0;
-    line-height: 1;
-}
-
-div.admonition p.last {
-    margin-bottom: 0;
-}
-
-div.highlight {
-    background-color: #fff;
-}
-
-dt:target, .highlight {
-    background: #FAF3E8;
-}
-
-div.warning {
-    background-color: #FCC;
-    border: 1px solid #FAA;
-}
-
-div.danger {
-    background-color: #FCC;
-    border: 1px solid #FAA;
-    -moz-box-shadow: 2px 2px 4px #D52C2C;
-    -webkit-box-shadow: 2px 2px 4px #D52C2C;
-    box-shadow: 2px 2px 4px #D52C2C;
-}
-
-div.error {
-    background-color: #FCC;
-    border: 1px solid #FAA;
-    -moz-box-shadow: 2px 2px 4px #D52C2C;
-    -webkit-box-shadow: 2px 2px 4px #D52C2C;
-    box-shadow: 2px 2px 4px #D52C2C;
-}
-
-div.caution {
-    background-color: #FCC;
-    border: 1px solid #FAA;
-}
-
-div.attention {
-    background-color: #FCC;
-    border: 1px solid #FAA;
-}
-
-div.important {
-    background-color: #EEE;
-    border: 1px solid #CCC;
-}
-
-div.note {
-    background-color: #EEE;
-    border: 1px solid #CCC;
-}
-
-div.tip {
-    background-color: #EEE;
-    border: 1px solid #CCC;
-}
-
-div.hint {
-    background-color: #EEE;
-    border: 1px solid #CCC;
-}
-
-div.seealso {
-    background-color: #EEE;
-    border: 1px solid #CCC;
-}
-
-div.topic {
-    background-color: #EEE;
-}
-
-p.admonition-title {
-    display: inline;
-}
-
-p.admonition-title:after {
-    content: ":";
-}
-
-pre, tt, code {
-    font-family: 'Consolas', 'Menlo', 'Deja Vu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
-    font-size: 0.9em;
-}
-
-.hll {
-    background-color: #FFC;
-    margin: 0 -12px;
-    padding: 0 12px;
-    display: block;
-}
-
-img.screenshot {
-}
-
-tt.descname, tt.descclassname, code.descname, code.descclassname {
-    font-size: 0.95em;
-}
-
-tt.descname, code.descname {
-    padding-right: 0.08em;
-}
-
-img.screenshot {
-    -moz-box-shadow: 2px 2px 4px #EEE;
-    -webkit-box-shadow: 2px 2px 4px #EEE;
-    box-shadow: 2px 2px 4px #EEE;
-}
-
-table.docutils {
-    border: 1px solid #888;
-    -moz-box-shadow: 2px 2px 4px #EEE;
-    -webkit-box-shadow: 2px 2px 4px #EEE;
-    box-shadow: 2px 2px 4px #EEE;
-}
-
-table.docutils td, table.docutils th {
-    border: 1px solid #888;
-    padding: 0.25em 0.7em;
-}
-
-table.field-list, table.footnote {
-    border: none;
-    -moz-box-shadow: none;
-    -webkit-box-shadow: none;
-    box-shadow: none;
-}
-
-table.footnote {
-    margin: 15px 0;
-    width: 100%;
-    border: 1px solid #EEE;
-    background: #FDFDFD;
-    font-size: 0.9em;
-}
-
-table.footnote + table.footnote {
-    margin-top: -15px;
-    border-top: none;
-}
-
-table.field-list th {
-    padding: 0 0.8em 0 0;
-}
-
-table.field-list td {
-    padding: 0;
-}
-
-table.field-list p {
-    margin-bottom: 0.8em;
-}
-
-/* Cloned from
- * https://github.com/sphinx-doc/sphinx/commit/ef60dbfce09286b20b7385333d63a60321784e68
- */
-.field-name {
-    -moz-hyphens: manual;
-    -ms-hyphens: manual;
-    -webkit-hyphens: manual;
-    hyphens: manual;
-}
-
-table.footnote td.label {
-    width: .1px;
-    padding: 0.3em 0 0.3em 0.5em;
-}
-
-table.footnote td {
-    padding: 0.3em 0.5em;
-}
-
-dl {
-    margin: 0;
-    padding: 0;
-}
-
-dl dd {
-    margin-left: 30px;
-}
-
-blockquote {
-    margin: 0 0 0 30px;
-    padding: 0;
-}
-
-ul, ol {
-    /* Matches the 30px from the narrow-screen "li > ul" selector below */
-    margin: 10px 0 10px 30px;
-    padding: 0;
-}
-
-pre {
-    background: #EEE;
-    padding: 7px 30px;
-    margin: 15px 0px;
-    line-height: 1.3em;
-}
-
-div.viewcode-block:target {
-    background: #ffd;
-}
-
-dl pre, blockquote pre, li pre {
-    margin-left: 0;
-    padding-left: 30px;
-}
-
-tt, code {
-    background-color: #ecf0f3;
-    color: #222;
-    /* padding: 1px 2px; */
-}
-
-tt.xref, code.xref, a tt {
-    background-color: #FBFBFB;
-    border-bottom: 1px solid #fff;
-}
-
-a.reference {
-    text-decoration: none;
-    border-bottom: 1px dotted #004B6B;
-}
-
-/* Don't put an underline on images */
-a.image-reference, a.image-reference:hover {
-    border-bottom: none;
-}
-
-a.reference:hover {
-    border-bottom: 1px solid #6D4100;
-}
-
-a.footnote-reference {
-    text-decoration: none;
-    font-size: 0.7em;
-    vertical-align: top;
-    border-bottom: 1px dotted #004B6B;
-}
-
-a.footnote-reference:hover {
-    border-bottom: 1px solid #6D4100;
-}
-
-a:hover tt, a:hover code {
-    background: #EEE;
-}
-
-
-@media screen and (max-width: 870px) {
-
-    div.sphinxsidebar {
-    	display: none;
-    }
-
-    div.document {
-       width: 100%;
-
-    }
-
-    div.documentwrapper {
-    	margin-left: 0;
-    	margin-top: 0;
-    	margin-right: 0;
-    	margin-bottom: 0;
-    }
-
-    div.bodywrapper {
-    	margin-top: 0;
-    	margin-right: 0;
-    	margin-bottom: 0;
-    	margin-left: 0;
-    }
-
-    ul {
-    	margin-left: 0;
-    }
-
-	li > ul {
-        /* Matches the 30px from the "ul, ol" selector above */
-		margin-left: 30px;
-	}
-
-    .document {
-    	width: auto;
-    }
-
-    .footer {
-    	width: auto;
-    }
-
-    .bodywrapper {
-    	margin: 0;
-    }
-
-    .footer {
-    	width: auto;
-    }
-
-    .github {
-        display: none;
-    }
-
-
-
-}
-
-
-
-@media screen and (max-width: 875px) {
-
-    body {
-        margin: 0;
-        padding: 20px 30px;
-    }
-
-    div.documentwrapper {
-        float: none;
-        background: #fff;
-    }
-
-    div.sphinxsidebar {
-        display: block;
-        float: none;
-        width: 102.5%;
-        margin: 50px -30px -20px -30px;
-        padding: 10px 20px;
-        background: #333;
-        color: #FFF;
-    }
-
-    div.sphinxsidebar h3, div.sphinxsidebar h4, div.sphinxsidebar p,
-    div.sphinxsidebar h3 a {
-        color: #fff;
-    }
-
-    div.sphinxsidebar a {
-        color: #AAA;
-    }
-
-    div.sphinxsidebar p.logo {
-        display: none;
-    }
-
-    div.document {
-        width: 100%;
-        margin: 0;
-    }
-
-    div.footer {
-        display: none;
-    }
-
-    div.bodywrapper {
-        margin: 0;
-    }
-
-    div.body {
-        min-height: 0;
-        padding: 0;
-    }
-
-    .rtd_doc_footer {
-        display: none;
-    }
-
-    .document {
-        width: auto;
-    }
-
-    .footer {
-        width: auto;
-    }
-
-    .footer {
-        width: auto;
-    }
-
-    .github {
-        display: none;
-    }
-}
-
-
-/* misc. */
-
-.revsys-inline {
-    display: none!important;
-}
-
-/* Make nested-list/multi-paragraph items look better in Releases changelog
- * pages. Without this, docutils' magical list fuckery causes inconsistent
- * formatting between different release sub-lists.
- */
-div#changelog > div.section > ul > li > p:only-child {
-    margin-bottom: 0;
-}
-
-/* Hide fugly table cell borders in ..bibliography:: directive output */
-table.docutils.citation, table.docutils.citation td, table.docutils.citation th {
-  border: none;
-  /* Below needed in some edge cases; if not applied, bottom shadows appear */
-  -moz-box-shadow: none;
-  -webkit-box-shadow: none;
-  box-shadow: none;
-}

docs/_build/html/_static/basic.css

@@ -1,643 +0,0 @@
-/*
- * basic.css
- * ~~~~~~~~~
- *
- * Sphinx stylesheet -- basic theme.
- *
- * :copyright: Copyright 2007-2017 by the Sphinx team, see AUTHORS.
- * :license: BSD, see LICENSE for details.
- *
- */
-
-/* -- main layout ----------------------------------------------------------- */
-
-div.clearer {
-    clear: both;
-}
-
-/* -- relbar ---------------------------------------------------------------- */
-
-div.related {
-    width: 100%;
-    font-size: 90%;
-}
-
-div.related h3 {
-    display: none;
-}
-
-div.related ul {
-    margin: 0;
-    padding: 0 0 0 10px;
-    list-style: none;
-}
-
-div.related li {
-    display: inline;
-}
-
-div.related li.right {
-    float: right;
-    margin-right: 5px;
-}
-
-/* -- sidebar --------------------------------------------------------------- */
-
-div.sphinxsidebarwrapper {
-    padding: 10px 5px 0 10px;
-}
-
-div.sphinxsidebar {
-    float: left;
-    width: 230px;
-    margin-left: -100%;
-    font-size: 90%;
-    word-wrap: break-word;
-    overflow-wrap : break-word;
-}
-
-div.sphinxsidebar ul {
-    list-style: none;
-}
-
-div.sphinxsidebar ul ul,
-div.sphinxsidebar ul.want-points {
-    margin-left: 20px;
-    list-style: square;
-}
-
-div.sphinxsidebar ul ul {
-    margin-top: 0;
-    margin-bottom: 0;
-}
-
-div.sphinxsidebar form {
-    margin-top: 10px;
-}
-
-div.sphinxsidebar input {
-    border: 1px solid #98dbcc;
-    font-family: sans-serif;
-    font-size: 1em;
-}
-
-div.sphinxsidebar #searchbox input[type="text"] {
-    width: 170px;
-}
-
-img {
-    border: 0;
-    max-width: 100%;
-}
-
-/* -- search page ----------------------------------------------------------- */
-
-ul.search {
-    margin: 10px 0 0 20px;
-    padding: 0;
-}
-
-ul.search li {
-    padding: 5px 0 5px 20px;
-    background-image: url(file.png);
-    background-repeat: no-repeat;
-    background-position: 0 7px;
-}
-
-ul.search li a {
-    font-weight: bold;
-}
-
-ul.search li div.context {
-    color: #888;
-    margin: 2px 0 0 30px;
-    text-align: left;
-}
-
-ul.keywordmatches li.goodmatch a {
-    font-weight: bold;
-}
-
-/* -- index page ------------------------------------------------------------ */
-
-table.contentstable {
-    width: 90%;
-    margin-left: auto;
-    margin-right: auto;
-}
-
-table.contentstable p.biglink {
-    line-height: 150%;
-}
-
-a.biglink {
-    font-size: 1.3em;
-}
-
-span.linkdescr {
-    font-style: italic;
-    padding-top: 5px;
-    font-size: 90%;
-}
-
-/* -- general index --------------------------------------------------------- */
-
-table.indextable {
-    width: 100%;
-}
-
-table.indextable td {
-    text-align: left;
-    vertical-align: top;
-}
-
-table.indextable ul {
-    margin-top: 0;
-    margin-bottom: 0;
-    list-style-type: none;
-}
-
-table.indextable > tbody > tr > td > ul {
-    padding-left: 0em;
-}
-
-table.indextable tr.pcap {
-    height: 10px;
-}
-
-table.indextable tr.cap {
-    margin-top: 10px;
-    background-color: #f2f2f2;
-}
-
-img.toggler {
-    margin-right: 3px;
-    margin-top: 3px;
-    cursor: pointer;
-}
-
-div.modindex-jumpbox {
-    border-top: 1px solid #ddd;
-    border-bottom: 1px solid #ddd;
-    margin: 1em 0 1em 0;
-    padding: 0.4em;
-}
-
-div.genindex-jumpbox {
-    border-top: 1px solid #ddd;
-    border-bottom: 1px solid #ddd;
-    margin: 1em 0 1em 0;
-    padding: 0.4em;
-}
-
-/* -- domain module index --------------------------------------------------- */
-
-table.modindextable td {
-    padding: 2px;
-    border-collapse: collapse;
-}
-
-/* -- general body styles --------------------------------------------------- */
-
-div.body p, div.body dd, div.body li, div.body blockquote {
-    -moz-hyphens: auto;
-    -ms-hyphens: auto;
-    -webkit-hyphens: auto;
-    hyphens: auto;
-}
-
-a.headerlink {
-    visibility: hidden;
-}
-
-h1:hover > a.headerlink,
-h2:hover > a.headerlink,
-h3:hover > a.headerlink,
-h4:hover > a.headerlink,
-h5:hover > a.headerlink,
-h6:hover > a.headerlink,
-dt:hover > a.headerlink,
-caption:hover > a.headerlink,
-p.caption:hover > a.headerlink,
-div.code-block-caption:hover > a.headerlink {
-    visibility: visible;
-}
-
-div.body p.caption {
-    text-align: inherit;
-}
-
-div.body td {
-    text-align: left;
-}
-
-.first {
-    margin-top: 0 !important;
-}
-
-p.rubric {
-    margin-top: 30px;
-    font-weight: bold;
-}
-
-img.align-left, .figure.align-left, object.align-left {
-    clear: left;
-    float: left;
-    margin-right: 1em;
-}
-
-img.align-right, .figure.align-right, object.align-right {
-    clear: right;
-    float: right;
-    margin-left: 1em;
-}
-
-img.align-center, .figure.align-center, object.align-center {
-  display: block;
-  margin-left: auto;
-  margin-right: auto;
-}
-
-.align-left {
-    text-align: left;
-}
-
-.align-center {
-    text-align: center;
-}
-
-.align-right {
-    text-align: right;
-}
-
-/* -- sidebars -------------------------------------------------------------- */
-
-div.sidebar {
-    margin: 0 0 0.5em 1em;
-    border: 1px solid #ddb;
-    padding: 7px 7px 0 7px;
-    background-color: #ffe;
-    width: 40%;
-    float: right;
-}
-
-p.sidebar-title {
-    font-weight: bold;
-}
-
-/* -- topics ---------------------------------------------------------------- */
-
-div.topic {
-    border: 1px solid #ccc;
-    padding: 7px 7px 0 7px;
-    margin: 10px 0 10px 0;
-}
-
-p.topic-title {
-    font-size: 1.1em;
-    font-weight: bold;
-    margin-top: 10px;
-}
-
-/* -- admonitions ----------------------------------------------------------- */
-
-div.admonition {
-    margin-top: 10px;
-    margin-bottom: 10px;
-    padding: 7px;
-}
-
-div.admonition dt {
-    font-weight: bold;
-}
-
-div.admonition dl {
-    margin-bottom: 0;
-}
-
-p.admonition-title {
-    margin: 0px 10px 5px 0px;
-    font-weight: bold;
-}
-
-div.body p.centered {
-    text-align: center;
-    margin-top: 25px;
-}
-
-/* -- tables ---------------------------------------------------------------- */
-
-table.docutils {
-    border: 0;
-    border-collapse: collapse;
-}
-
-table caption span.caption-number {
-    font-style: italic;
-}
-
-table caption span.caption-text {
-}
-
-table.docutils td, table.docutils th {
-    padding: 1px 8px 1px 5px;
-    border-top: 0;
-    border-left: 0;
-    border-right: 0;
-    border-bottom: 1px solid #aaa;
-}
-
-table.footnote td, table.footnote th {
-    border: 0 !important;
-}
-
-th {
-    text-align: left;
-    padding-right: 5px;
-}
-
-table.citation {
-    border-left: solid 1px gray;
-    margin-left: 1px;
-}
-
-table.citation td {
-    border-bottom: none;
-}
-
-/* -- figures --------------------------------------------------------------- */
-
-div.figure {
-    margin: 0.5em;
-    padding: 0.5em;
-}
-
-div.figure p.caption {
-    padding: 0.3em;
-}
-
-div.figure p.caption span.caption-number {
-    font-style: italic;
-}
-
-div.figure p.caption span.caption-text {
-}
-
-/* -- field list styles ----------------------------------------------------- */
-
-table.field-list td, table.field-list th {
-    border: 0 !important;
-}
-
-.field-list ul {
-    margin: 0;
-    padding-left: 1em;
-}
-
-.field-list p {
-    margin: 0;
-}
-
-.field-name {
-    -moz-hyphens: manual;
-    -ms-hyphens: manual;
-    -webkit-hyphens: manual;
-    hyphens: manual;
-}
-
-/* -- other body styles ----------------------------------------------------- */
-
-ol.arabic {
-    list-style: decimal;
-}
-
-ol.loweralpha {
-    list-style: lower-alpha;
-}
-
-ol.upperalpha {
-    list-style: upper-alpha;
-}
-
-ol.lowerroman {
-    list-style: lower-roman;
-}
-
-ol.upperroman {
-    list-style: upper-roman;
-}
-
-dl {
-    margin-bottom: 15px;
-}
-
-dd p {
-    margin-top: 0px;
-}
-
-dd ul, dd table {
-    margin-bottom: 10px;
-}
-
-dd {
-    margin-top: 3px;
-    margin-bottom: 10px;
-    margin-left: 30px;
-}
-
-dt:target, span.highlighted {
-    background-color: #fbe54e;
-}
-
-rect.highlighted {
-    fill: #fbe54e;
-}
-
-dl.glossary dt {
-    font-weight: bold;
-    font-size: 1.1em;
-}
-
-.optional {
-    font-size: 1.3em;
-}
-
-.sig-paren {
-    font-size: larger;
-}
-
-.versionmodified {
-    font-style: italic;
-}
-
-.system-message {
-    background-color: #fda;
-    padding: 5px;
-    border: 3px solid red;
-}
-
-.footnote:target  {
-    background-color: #ffa;
-}
-
-.line-block {
-    display: block;
-    margin-top: 1em;
-    margin-bottom: 1em;
-}
-
-.line-block .line-block {
-    margin-top: 0;
-    margin-bottom: 0;
-    margin-left: 1.5em;
-}
-
-.guilabel, .menuselection {
-    font-family: sans-serif;
-}
-
-.accelerator {
-    text-decoration: underline;
-}
-
-.classifier {
-    font-style: oblique;
-}
-
-abbr, acronym {
-    border-bottom: dotted 1px;
-    cursor: help;
-}
-
-/* -- code displays --------------------------------------------------------- */
-
-pre {
-    overflow: auto;
-    overflow-y: hidden;  /* fixes display issues on Chrome browsers */
-}
-
-span.pre {
-    -moz-hyphens: none;
-    -ms-hyphens: none;
-    -webkit-hyphens: none;
-    hyphens: none;
-}
-
-td.linenos pre {
-    padding: 5px 0px;
-    border: 0;
-    background-color: transparent;
-    color: #aaa;
-}
-
-table.highlighttable {
-    margin-left: 0.5em;
-}
-
-table.highlighttable td {
-    padding: 0 0.5em 0 0.5em;
-}
-
-div.code-block-caption {
-    padding: 2px 5px;
-    font-size: small;
-}
-
-div.code-block-caption code {
-    background-color: transparent;
-}
-
-div.code-block-caption + div > div.highlight > pre {
-    margin-top: 0;
-}
-
-div.code-block-caption span.caption-number {
-    padding: 0.1em 0.3em;
-    font-style: italic;
-}
-
-div.code-block-caption span.caption-text {
-}
-
-div.literal-block-wrapper {
-    padding: 1em 1em 0;
-}
-
-div.literal-block-wrapper div.highlight {
-    margin: 0;
-}
-
-code.descname {
-    background-color: transparent;
-    font-weight: bold;
-    font-size: 1.2em;
-}
-
-code.descclassname {
-    background-color: transparent;
-}
-
-code.xref, a code {
-    background-color: transparent;
-    font-weight: bold;
-}
-
-h1 code, h2 code, h3 code, h4 code, h5 code, h6 code {
-    background-color: transparent;
-}
-
-.viewcode-link {
-    float: right;
-}
-
-.viewcode-back {
-    float: right;
-    font-family: sans-serif;
-}
-
-div.viewcode-block:target {
-    margin: -1px -10px;
-    padding: 0 10px;
-}
-
-/* -- math display ---------------------------------------------------------- */
-
-img.math {
-    vertical-align: middle;
-}
-
-div.body div.math p {
-    text-align: center;
-}
-
-span.eqno {
-    float: right;
-}
-
-span.eqno a.headerlink {
-    position: relative;
-    left: 0px;
-    z-index: 1;
-}
-
-div.math:hover a.headerlink {
-    visibility: visible;
-}
-
-/* -- printout stylesheet --------------------------------------------------- */
-
-@media print {
-    div.document,
-    div.documentwrapper,
-    div.bodywrapper {
-        margin: 0 !important;
-        width: 100%;
-    }
-
-    div.sphinxsidebar,
-    div.related,
-    div.footer,
-    #top-link {
-        display: none;
-    }
-}

docs/_build/html/_static/custom.css

@@ -1 +0,0 @@
-/* This file intentionally left blank. */

docs/_build/html/_static/doctools.js

@@ -1,311 +0,0 @@
-/*
- * doctools.js
- * ~~~~~~~~~~~
- *
- * Sphinx JavaScript utilities for all documentation.
- *
- * :copyright: Copyright 2007-2017 by the Sphinx team, see AUTHORS.
- * :license: BSD, see LICENSE for details.
- *
- */
-
-/**
- * select a different prefix for underscore
- */
-$u = _.noConflict();
-
-/**
- * make the code below compatible with browsers without
- * an installed firebug like debugger
-if (!window.console || !console.firebug) {
-  var names = ["log", "debug", "info", "warn", "error", "assert", "dir",
-    "dirxml", "group", "groupEnd", "time", "timeEnd", "count", "trace",
-    "profile", "profileEnd"];
-  window.console = {};
-  for (var i = 0; i < names.length; ++i)
-    window.console[names[i]] = function() {};
-}
- */
-
-/**
- * small helper function to urldecode strings
- */
-jQuery.urldecode = function(x) {
-  return decodeURIComponent(x).replace(/\+/g, ' ');
-};
-
-/**
- * small helper function to urlencode strings
- */
-jQuery.urlencode = encodeURIComponent;
-
-/**
- * This function returns the parsed url parameters of the
- * current request. Multiple values per key are supported,
- * it will always return arrays of strings for the value parts.
- */
-jQuery.getQueryParameters = function(s) {
-  if (typeof s === 'undefined')
-    s = document.location.search;
-  var parts = s.substr(s.indexOf('?') + 1).split('&');
-  var result = {};
-  for (var i = 0; i < parts.length; i++) {
-    var tmp = parts[i].split('=', 2);
-    var key = jQuery.urldecode(tmp[0]);
-    var value = jQuery.urldecode(tmp[1]);
-    if (key in result)
-      result[key].push(value);
-    else
-      result[key] = [value];
-  }
-  return result;
-};
-
-/**
- * highlight a given string on a jquery object by wrapping it in
- * span elements with the given class name.
- */
-jQuery.fn.highlightText = function(text, className) {
-  function highlight(node, addItems) {
-    if (node.nodeType === 3) {
-      var val = node.nodeValue;
-      var pos = val.toLowerCase().indexOf(text);
-      if (pos >= 0 && !jQuery(node.parentNode).hasClass(className)) {
-        var span;
-        var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg");
-        if (isInSVG) {
-          span = document.createElementNS("http://www.w3.org/2000/svg", "tspan");
-        } else {
-          span = document.createElement("span");
-          span.className = className;
-        }
-        span.appendChild(document.createTextNode(val.substr(pos, text.length)));
-        node.parentNode.insertBefore(span, node.parentNode.insertBefore(
-          document.createTextNode(val.substr(pos + text.length)),
-          node.nextSibling));
-        node.nodeValue = val.substr(0, pos);
-        if (isInSVG) {
-          var bbox = span.getBBox();
-          var rect = document.createElementNS("http://www.w3.org/2000/svg", "rect");
-       	  rect.x.baseVal.value = bbox.x;
-          rect.y.baseVal.value = bbox.y;
-          rect.width.baseVal.value = bbox.width;
-          rect.height.baseVal.value = bbox.height;
-          rect.setAttribute('class', className);
-          var parentOfText = node.parentNode.parentNode;
-          addItems.push({
-              "parent": node.parentNode,
-              "target": rect});
-        }
-      }
-    }
-    else if (!jQuery(node).is("button, select, textarea")) {
-      jQuery.each(node.childNodes, function() {
-        highlight(this, addItems);
-      });
-    }
-  }
-  var addItems = [];
-  var result = this.each(function() {
-    highlight(this, addItems);
-  });
-  for (var i = 0; i < addItems.length; ++i) {
-    jQuery(addItems[i].parent).before(addItems[i].target);
-  }
-  return result;
-};
-
-/*
- * backward compatibility for jQuery.browser
- * This will be supported until firefox bug is fixed.
- */
-if (!jQuery.browser) {
-  jQuery.uaMatch = function(ua) {
-    ua = ua.toLowerCase();
-
-    var match = /(chrome)[ \/]([\w.]+)/.exec(ua) ||
-      /(webkit)[ \/]([\w.]+)/.exec(ua) ||
-      /(opera)(?:.*version|)[ \/]([\w.]+)/.exec(ua) ||
-      /(msie) ([\w.]+)/.exec(ua) ||
-      ua.indexOf("compatible") < 0 && /(mozilla)(?:.*? rv:([\w.]+)|)/.exec(ua) ||
-      [];
-
-    return {
-      browser: match[ 1 ] || "",
-      version: match[ 2 ] || "0"
-    };
-  };
-  jQuery.browser = {};
-  jQuery.browser[jQuery.uaMatch(navigator.userAgent).browser] = true;
-}
-
-/**
- * Small JavaScript module for the documentation.
- */
-var Documentation = {
-
-  init : function() {
-    this.fixFirefoxAnchorBug();
-    this.highlightSearchWords();
-    this.initIndexTable();
-    
-  },
-
-  /**
-   * i18n support
-   */
-  TRANSLATIONS : {},
-  PLURAL_EXPR : function(n) { return n === 1 ? 0 : 1; },
-  LOCALE : 'unknown',
-
-  // gettext and ngettext don't access this so that the functions
-  // can safely bound to a different name (_ = Documentation.gettext)
-  gettext : function(string) {
-    var translated = Documentation.TRANSLATIONS[string];
-    if (typeof translated === 'undefined')
-      return string;
-    return (typeof translated === 'string') ? translated : translated[0];
-  },
-
-  ngettext : function(singular, plural, n) {
-    var translated = Documentation.TRANSLATIONS[singular];
-    if (typeof translated === 'undefined')
-      return (n == 1) ? singular : plural;
-    return translated[Documentation.PLURALEXPR(n)];
-  },
-
-  addTranslations : function(catalog) {
-    for (var key in catalog.messages)
-      this.TRANSLATIONS[key] = catalog.messages[key];
-    this.PLURAL_EXPR = new Function('n', 'return +(' + catalog.plural_expr + ')');
-    this.LOCALE = catalog.locale;
-  },
-
-  /**
-   * add context elements like header anchor links
-   */
-  addContextElements : function() {
-    $('div[id] > :header:first').each(function() {
-      $('<a class="headerlink">\u00B6</a>').
-      attr('href', '#' + this.id).
-      attr('title', _('Permalink to this headline')).
-      appendTo(this);
-    });
-    $('dt[id]').each(function() {
-      $('<a class="headerlink">\u00B6</a>').
-      attr('href', '#' + this.id).
-      attr('title', _('Permalink to this definition')).
-      appendTo(this);
-    });
-  },
-
-  /**
-   * workaround a firefox stupidity
-   * see: https://bugzilla.mozilla.org/show_bug.cgi?id=645075
-   */
-  fixFirefoxAnchorBug : function() {
-    if (document.location.hash)
-      window.setTimeout(function() {
-        document.location.href += '';
-      }, 10);
-  },
-
-  /**
-   * highlight the search words provided in the url in the text
-   */
-  highlightSearchWords : function() {
-    var params = $.getQueryParameters();
-    var terms = (params.highlight) ? params.highlight[0].split(/\s+/) : [];
-    if (terms.length) {
-      var body = $('div.body');
-      if (!body.length) {
-        body = $('body');
-      }
-      window.setTimeout(function() {
-        $.each(terms, function() {
-          body.highlightText(this.toLowerCase(), 'highlighted');
-        });
-      }, 10);
-      $('<p class="highlight-link"><a href="javascript:Documentation.' +
-        'hideSearchWords()">' + _('Hide Search Matches') + '</a></p>')
-          .appendTo($('#searchbox'));
-    }
-  },
-
-  /**
-   * init the domain index toggle buttons
-   */
-  initIndexTable : function() {
-    var togglers = $('img.toggler').click(function() {
-      var src = $(this).attr('src');
-      var idnum = $(this).attr('id').substr(7);
-      $('tr.cg-' + idnum).toggle();
-      if (src.substr(-9) === 'minus.png')
-        $(this).attr('src', src.substr(0, src.length-9) + 'plus.png');
-      else
-        $(this).attr('src', src.substr(0, src.length-8) + 'minus.png');
-    }).css('display', '');
-    if (DOCUMENTATION_OPTIONS.COLLAPSE_INDEX) {
-        togglers.click();
-    }
-  },
-
-  /**
-   * helper function to hide the search marks again
-   */
-  hideSearchWords : function() {
-    $('#searchbox .highlight-link').fadeOut(300);
-    $('span.highlighted').removeClass('highlighted');
-  },
-
-  /**
-   * make the url absolute
-   */
-  makeURL : function(relativeURL) {
-    return DOCUMENTATION_OPTIONS.URL_ROOT + '/' + relativeURL;
-  },
-
-  /**
-   * get the current relative url
-   */
-  getCurrentURL : function() {
-    var path = document.location.pathname;
-    var parts = path.split(/\//);
-    $.each(DOCUMENTATION_OPTIONS.URL_ROOT.split(/\//), function() {
-      if (this === '..')
-        parts.pop();
-    });
-    var url = parts.join('/');
-    return path.substring(url.lastIndexOf('/') + 1, path.length - 1);
-  },
-
-  initOnKeyListeners: function() {
-    $(document).keyup(function(event) {
-      var activeElementType = document.activeElement.tagName;
-      // don't navigate when in search box or textarea
-      if (activeElementType !== 'TEXTAREA' && activeElementType !== 'INPUT' && activeElementType !== 'SELECT') {
-        switch (event.keyCode) {
-          case 37: // left
-            var prevHref = $('link[rel="prev"]').prop('href');
-            if (prevHref) {
-              window.location.href = prevHref;
-              return false;
-            }
-          case 39: // right
-            var nextHref = $('link[rel="next"]').prop('href');
-            if (nextHref) {
-              window.location.href = nextHref;
-              return false;
-            }
-        }
-      }
-    });
-  }
-};
-
-// quick alias for translations
-_ = Documentation.gettext;
-
-$(document).ready(function() {
-  Documentation.init();
-});

docs/_build/html/_static/pygments.css

@@ -1,69 +0,0 @@
-.highlight .hll { background-color: #ffffcc }
-.highlight  { background: #eeffcc; }
-.highlight .c { color: #408090; font-style: italic } /* Comment */
-.highlight .err { border: 1px solid #FF0000 } /* Error */
-.highlight .k { color: #007020; font-weight: bold } /* Keyword */
-.highlight .o { color: #666666 } /* Operator */
-.highlight .ch { color: #408090; font-style: italic } /* Comment.Hashbang */
-.highlight .cm { color: #408090; font-style: italic } /* Comment.Multiline */
-.highlight .cp { color: #007020 } /* Comment.Preproc */
-.highlight .cpf { color: #408090; font-style: italic } /* Comment.PreprocFile */
-.highlight .c1 { color: #408090; font-style: italic } /* Comment.Single */
-.highlight .cs { color: #408090; background-color: #fff0f0 } /* Comment.Special */
-.highlight .gd { color: #A00000 } /* Generic.Deleted */
-.highlight .ge { font-style: italic } /* Generic.Emph */
-.highlight .gr { color: #FF0000 } /* Generic.Error */
-.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
-.highlight .gi { color: #00A000 } /* Generic.Inserted */
-.highlight .go { color: #333333 } /* Generic.Output */
-.highlight .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */
-.highlight .gs { font-weight: bold } /* Generic.Strong */
-.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
-.highlight .gt { color: #0044DD } /* Generic.Traceback */
-.highlight .kc { color: #007020; font-weight: bold } /* Keyword.Constant */
-.highlight .kd { color: #007020; font-weight: bold } /* Keyword.Declaration */
-.highlight .kn { color: #007020; font-weight: bold } /* Keyword.Namespace */
-.highlight .kp { color: #007020 } /* Keyword.Pseudo */
-.highlight .kr { color: #007020; font-weight: bold } /* Keyword.Reserved */
-.highlight .kt { color: #902000 } /* Keyword.Type */
-.highlight .m { color: #208050 } /* Literal.Number */
-.highlight .s { color: #4070a0 } /* Literal.String */
-.highlight .na { color: #4070a0 } /* Name.Attribute */
-.highlight .nb { color: #007020 } /* Name.Builtin */
-.highlight .nc { color: #0e84b5; font-weight: bold } /* Name.Class */
-.highlight .no { color: #60add5 } /* Name.Constant */
-.highlight .nd { color: #555555; font-weight: bold } /* Name.Decorator */
-.highlight .ni { color: #d55537; font-weight: bold } /* Name.Entity */
-.highlight .ne { color: #007020 } /* Name.Exception */
-.highlight .nf { color: #06287e } /* Name.Function */
-.highlight .nl { color: #002070; font-weight: bold } /* Name.Label */
-.highlight .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */
-.highlight .nt { color: #062873; font-weight: bold } /* Name.Tag */
-.highlight .nv { color: #bb60d5 } /* Name.Variable */
-.highlight .ow { color: #007020; font-weight: bold } /* Operator.Word */
-.highlight .w { color: #bbbbbb } /* Text.Whitespace */
-.highlight .mb { color: #208050 } /* Literal.Number.Bin */
-.highlight .mf { color: #208050 } /* Literal.Number.Float */
-.highlight .mh { color: #208050 } /* Literal.Number.Hex */
-.highlight .mi { color: #208050 } /* Literal.Number.Integer */
-.highlight .mo { color: #208050 } /* Literal.Number.Oct */
-.highlight .sa { color: #4070a0 } /* Literal.String.Affix */
-.highlight .sb { color: #4070a0 } /* Literal.String.Backtick */
-.highlight .sc { color: #4070a0 } /* Literal.String.Char */
-.highlight .dl { color: #4070a0 } /* Literal.String.Delimiter */
-.highlight .sd { color: #4070a0; font-style: italic } /* Literal.String.Doc */
-.highlight .s2 { color: #4070a0 } /* Literal.String.Double */
-.highlight .se { color: #4070a0; font-weight: bold } /* Literal.String.Escape */
-.highlight .sh { color: #4070a0 } /* Literal.String.Heredoc */
-.highlight .si { color: #70a0d0; font-style: italic } /* Literal.String.Interpol */
-.highlight .sx { color: #c65d09 } /* Literal.String.Other */
-.highlight .sr { color: #235388 } /* Literal.String.Regex */
-.highlight .s1 { color: #4070a0 } /* Literal.String.Single */
-.highlight .ss { color: #517918 } /* Literal.String.Symbol */
-.highlight .bp { color: #007020 } /* Name.Builtin.Pseudo */
-.highlight .fm { color: #06287e } /* Name.Function.Magic */
-.highlight .vc { color: #bb60d5 } /* Name.Variable.Class */
-.highlight .vg { color: #bb60d5 } /* Name.Variable.Global */
-.highlight .vi { color: #bb60d5 } /* Name.Variable.Instance */
-.highlight .vm { color: #bb60d5 } /* Name.Variable.Magic */
-.highlight .il { color: #208050 } /* Literal.Number.Integer.Long */

docs/_build/html/_static/searchtools.js

@@ -1,761 +0,0 @@
-/*
- * searchtools.js_t
- * ~~~~~~~~~~~~~~~~
- *
- * Sphinx JavaScript utilities for the full-text search.
- *
- * :copyright: Copyright 2007-2017 by the Sphinx team, see AUTHORS.
- * :license: BSD, see LICENSE for details.
- *
- */
-
-
-/* Non-minified version JS is _stemmer.js if file is provided */ 
-/**
- * Porter Stemmer
- */
-var Stemmer = function() {
-
-  var step2list = {
-    ational: 'ate',
-    tional: 'tion',
-    enci: 'ence',
-    anci: 'ance',
-    izer: 'ize',
-    bli: 'ble',
-    alli: 'al',
-    entli: 'ent',
-    eli: 'e',
-    ousli: 'ous',
-    ization: 'ize',
-    ation: 'ate',
-    ator: 'ate',
-    alism: 'al',
-    iveness: 'ive',
-    fulness: 'ful',
-    ousness: 'ous',
-    aliti: 'al',
-    iviti: 'ive',
-    biliti: 'ble',
-    logi: 'log'
-  };
-
-  var step3list = {
-    icate: 'ic',
-    ative: '',
-    alize: 'al',
-    iciti: 'ic',
-    ical: 'ic',
-    ful: '',
-    ness: ''
-  };
-
-  var c = "[^aeiou]";          // consonant
-  var v = "[aeiouy]";          // vowel
-  var C = c + "[^aeiouy]*";    // consonant sequence
-  var V = v + "[aeiou]*";      // vowel sequence
-
-  var mgr0 = "^(" + C + ")?" + V + C;                      // [C]VC... is m>0
-  var meq1 = "^(" + C + ")?" + V + C + "(" + V + ")?$";    // [C]VC[V] is m=1
-  var mgr1 = "^(" + C + ")?" + V + C + V + C;              // [C]VCVC... is m>1
-  var s_v   = "^(" + C + ")?" + v;                         // vowel in stem
-
-  this.stemWord = function (w) {
-    var stem;
-    var suffix;
-    var firstch;
-    var origword = w;
-
-    if (w.length < 3)
-      return w;
-
-    var re;
-    var re2;
-    var re3;
-    var re4;
-
-    firstch = w.substr(0,1);
-    if (firstch == "y")
-      w = firstch.toUpperCase() + w.substr(1);
-
-    // Step 1a
-    re = /^(.+?)(ss|i)es$/;
-    re2 = /^(.+?)([^s])s$/;
-
-    if (re.test(w))
-      w = w.replace(re,"$1$2");
-    else if (re2.test(w))
-      w = w.replace(re2,"$1$2");
-
-    // Step 1b
-    re = /^(.+?)eed$/;
-    re2 = /^(.+?)(ed|ing)$/;
-    if (re.test(w)) {
-      var fp = re.exec(w);
-      re = new RegExp(mgr0);
-      if (re.test(fp[1])) {
-        re = /.$/;
-        w = w.replace(re,"");
-      }
-    }
-    else if (re2.test(w)) {
-      var fp = re2.exec(w);
-      stem = fp[1];
-      re2 = new RegExp(s_v);
-      if (re2.test(stem)) {
-        w = stem;
-        re2 = /(at|bl|iz)$/;
-        re3 = new RegExp("([^aeiouylsz])\\1$");
-        re4 = new RegExp("^" + C + v + "[^aeiouwxy]$");
-        if (re2.test(w))
-          w = w + "e";
-        else if (re3.test(w)) {
-          re = /.$/;
-          w = w.replace(re,"");
-        }
-        else if (re4.test(w))
-          w = w + "e";
-      }
-    }
-
-    // Step 1c
-    re = /^(.+?)y$/;
-    if (re.test(w)) {
-      var fp = re.exec(w);
-      stem = fp[1];
-      re = new RegExp(s_v);
-      if (re.test(stem))
-        w = stem + "i";
-    }
-
-    // Step 2
-    re = /^(.+?)(ational|tional|enci|anci|izer|bli|alli|entli|eli|ousli|ization|ation|ator|alism|iveness|fulness|ousness|aliti|iviti|biliti|logi)$/;
-    if (re.test(w)) {
-      var fp = re.exec(w);
-      stem = fp[1];
-      suffix = fp[2];
-      re = new RegExp(mgr0);
-      if (re.test(stem))
-        w = stem + step2list[suffix];
-    }
-
-    // Step 3
-    re = /^(.+?)(icate|ative|alize|iciti|ical|ful|ness)$/;
-    if (re.test(w)) {
-      var fp = re.exec(w);
-      stem = fp[1];
-      suffix = fp[2];
-      re = new RegExp(mgr0);
-      if (re.test(stem))
-        w = stem + step3list[suffix];
-    }
-
-    // Step 4
-    re = /^(.+?)(al|ance|ence|er|ic|able|ible|ant|ement|ment|ent|ou|ism|ate|iti|ous|ive|ize)$/;
-    re2 = /^(.+?)(s|t)(ion)$/;
-    if (re.test(w)) {
-      var fp = re.exec(w);
-      stem = fp[1];
-      re = new RegExp(mgr1);
-      if (re.test(stem))
-        w = stem;
-    }
-    else if (re2.test(w)) {
-      var fp = re2.exec(w);
-      stem = fp[1] + fp[2];
-      re2 = new RegExp(mgr1);
-      if (re2.test(stem))
-        w = stem;
-    }
-
-    // Step 5
-    re = /^(.+?)e$/;
-    if (re.test(w)) {
-      var fp = re.exec(w);
-      stem = fp[1];
-      re = new RegExp(mgr1);
-      re2 = new RegExp(meq1);
-      re3 = new RegExp("^" + C + v + "[^aeiouwxy]$");
-      if (re.test(stem) || (re2.test(stem) && !(re3.test(stem))))
-        w = stem;
-    }
-    re = /ll$/;
-    re2 = new RegExp(mgr1);
-    if (re.test(w) && re2.test(w)) {
-      re = /.$/;
-      w = w.replace(re,"");
-    }
-
-    // and turn initial Y back to y
-    if (firstch == "y")
-      w = firstch.toLowerCase() + w.substr(1);
-    return w;
-  }
-}
-
-
-
-/**
- * Simple result scoring code.
- */
-var Scorer = {
-  // Implement the following function to further tweak the score for each result
-  // The function takes a result array [filename, title, anchor, descr, score]
-  // and returns the new score.
-  /*
-  score: function(result) {
-    return result[4];
-  },
-  */
-
-  // query matches the full name of an object
-  objNameMatch: 11,
-  // or matches in the last dotted part of the object name
-  objPartialMatch: 6,
-  // Additive scores depending on the priority of the object
-  objPrio: {0:  15,   // used to be importantResults
-            1:  5,   // used to be objectResults
-            2: -5},  // used to be unimportantResults
-  //  Used when the priority is not in the mapping.
-  objPrioDefault: 0,
-
-  // query found in title
-  title: 15,
-  // query found in terms
-  term: 5
-};
-
-
-
-
-
-var splitChars = (function() {
-    var result = {};
-    var singles = [96, 180, 187, 191, 215, 247, 749, 885, 903, 907, 909, 930, 1014, 1648,
-         1748, 1809, 2416, 2473, 2481, 2526, 2601, 2609, 2612, 2615, 2653, 2702,
-         2706, 2729, 2737, 2740, 2857, 2865, 2868, 2910, 2928, 2948, 2961, 2971,
-         2973, 3085, 3089, 3113, 3124, 3213, 3217, 3241, 3252, 3295, 3341, 3345,
-         3369, 3506, 3516, 3633, 3715, 3721, 3736, 3744, 3748, 3750, 3756, 3761,
-         3781, 3912, 4239, 4347, 4681, 4695, 4697, 4745, 4785, 4799, 4801, 4823,
-         4881, 5760, 5901, 5997, 6313, 7405, 8024, 8026, 8028, 8030, 8117, 8125,
-         8133, 8181, 8468, 8485, 8487, 8489, 8494, 8527, 11311, 11359, 11687, 11695,
-         11703, 11711, 11719, 11727, 11735, 12448, 12539, 43010, 43014, 43019, 43587,
-         43696, 43713, 64286, 64297, 64311, 64317, 64319, 64322, 64325, 65141];
-    var i, j, start, end;
-    for (i = 0; i < singles.length; i++) {
-        result[singles[i]] = true;
-    }
-    var ranges = [[0, 47], [58, 64], [91, 94], [123, 169], [171, 177], [182, 184], [706, 709],
-         [722, 735], [741, 747], [751, 879], [888, 889], [894, 901], [1154, 1161],
-         [1318, 1328], [1367, 1368], [1370, 1376], [1416, 1487], [1515, 1519], [1523, 1568],
-         [1611, 1631], [1642, 1645], [1750, 1764], [1767, 1773], [1789, 1790], [1792, 1807],
-         [1840, 1868], [1958, 1968], [1970, 1983], [2027, 2035], [2038, 2041], [2043, 2047],
-         [2070, 2073], [2075, 2083], [2085, 2087], [2089, 2307], [2362, 2364], [2366, 2383],
-         [2385, 2391], [2402, 2405], [2419, 2424], [2432, 2436], [2445, 2446], [2449, 2450],
-         [2483, 2485], [2490, 2492], [2494, 2509], [2511, 2523], [2530, 2533], [2546, 2547],
-         [2554, 2564], [2571, 2574], [2577, 2578], [2618, 2648], [2655, 2661], [2672, 2673],
-         [2677, 2692], [2746, 2748], [2750, 2767], [2769, 2783], [2786, 2789], [2800, 2820],
-         [2829, 2830], [2833, 2834], [2874, 2876], [2878, 2907], [2914, 2917], [2930, 2946],
-         [2955, 2957], [2966, 2968], [2976, 2978], [2981, 2983], [2987, 2989], [3002, 3023],
-         [3025, 3045], [3059, 3076], [3130, 3132], [3134, 3159], [3162, 3167], [3170, 3173],
-         [3184, 3191], [3199, 3204], [3258, 3260], [3262, 3293], [3298, 3301], [3312, 3332],
-         [3386, 3388], [3390, 3423], [3426, 3429], [3446, 3449], [3456, 3460], [3479, 3481],
-         [3518, 3519], [3527, 3584], [3636, 3647], [3655, 3663], [3674, 3712], [3717, 3718],
-         [3723, 3724], [3726, 3731], [3752, 3753], [3764, 3772], [3774, 3775], [3783, 3791],
-         [3802, 3803], [3806, 3839], [3841, 3871], [3892, 3903], [3949, 3975], [3980, 4095],
-         [4139, 4158], [4170, 4175], [4182, 4185], [4190, 4192], [4194, 4196], [4199, 4205],
-         [4209, 4212], [4226, 4237], [4250, 4255], [4294, 4303], [4349, 4351], [4686, 4687],
-         [4702, 4703], [4750, 4751], [4790, 4791], [4806, 4807], [4886, 4887], [4955, 4968],
-         [4989, 4991], [5008, 5023], [5109, 5120], [5741, 5742], [5787, 5791], [5867, 5869],
-         [5873, 5887], [5906, 5919], [5938, 5951], [5970, 5983], [6001, 6015], [6068, 6102],
-         [6104, 6107], [6109, 6111], [6122, 6127], [6138, 6159], [6170, 6175], [6264, 6271],
-         [6315, 6319], [6390, 6399], [6429, 6469], [6510, 6511], [6517, 6527], [6572, 6592],
-         [6600, 6607], [6619, 6655], [6679, 6687], [6741, 6783], [6794, 6799], [6810, 6822],
-         [6824, 6916], [6964, 6980], [6988, 6991], [7002, 7042], [7073, 7085], [7098, 7167],
-         [7204, 7231], [7242, 7244], [7294, 7400], [7410, 7423], [7616, 7679], [7958, 7959],
-         [7966, 7967], [8006, 8007], [8014, 8015], [8062, 8063], [8127, 8129], [8141, 8143],
-         [8148, 8149], [8156, 8159], [8173, 8177], [8189, 8303], [8306, 8307], [8314, 8318],
-         [8330, 8335], [8341, 8449], [8451, 8454], [8456, 8457], [8470, 8472], [8478, 8483],
-         [8506, 8507], [8512, 8516], [8522, 8525], [8586, 9311], [9372, 9449], [9472, 10101],
-         [10132, 11263], [11493, 11498], [11503, 11516], [11518, 11519], [11558, 11567],
-         [11622, 11630], [11632, 11647], [11671, 11679], [11743, 11822], [11824, 12292],
-         [12296, 12320], [12330, 12336], [12342, 12343], [12349, 12352], [12439, 12444],
-         [12544, 12548], [12590, 12592], [12687, 12689], [12694, 12703], [12728, 12783],
-         [12800, 12831], [12842, 12880], [12896, 12927], [12938, 12976], [12992, 13311],
-         [19894, 19967], [40908, 40959], [42125, 42191], [42238, 42239], [42509, 42511],
-         [42540, 42559], [42592, 42593], [42607, 42622], [42648, 42655], [42736, 42774],
-         [42784, 42785], [42889, 42890], [42893, 43002], [43043, 43055], [43062, 43071],
-         [43124, 43137], [43188, 43215], [43226, 43249], [43256, 43258], [43260, 43263],
-         [43302, 43311], [43335, 43359], [43389, 43395], [43443, 43470], [43482, 43519],
-         [43561, 43583], [43596, 43599], [43610, 43615], [43639, 43641], [43643, 43647],
-         [43698, 43700], [43703, 43704], [43710, 43711], [43715, 43738], [43742, 43967],
-         [44003, 44015], [44026, 44031], [55204, 55215], [55239, 55242], [55292, 55295],
-         [57344, 63743], [64046, 64047], [64110, 64111], [64218, 64255], [64263, 64274],
-         [64280, 64284], [64434, 64466], [64830, 64847], [64912, 64913], [64968, 65007],
-         [65020, 65135], [65277, 65295], [65306, 65312], [65339, 65344], [65371, 65381],
-         [65471, 65473], [65480, 65481], [65488, 65489], [65496, 65497]];
-    for (i = 0; i < ranges.length; i++) {
-        start = ranges[i][0];
-        end = ranges[i][1];
-        for (j = start; j <= end; j++) {
-            result[j] = true;
-        }
-    }
-    return result;
-})();
-
-function splitQuery(query) {
-    var result = [];
-    var start = -1;
-    for (var i = 0; i < query.length; i++) {
-        if (splitChars[query.charCodeAt(i)]) {
-            if (start !== -1) {
-                result.push(query.slice(start, i));
-                start = -1;
-            }
-        } else if (start === -1) {
-            start = i;
-        }
-    }
-    if (start !== -1) {
-        result.push(query.slice(start));
-    }
-    return result;
-}
-
-
-
-
-/**
- * Search Module
- */
-var Search = {
-
-  _index : null,
-  _queued_query : null,
-  _pulse_status : -1,
-
-  init : function() {
-      var params = $.getQueryParameters();
-      if (params.q) {
-          var query = params.q[0];
-          $('input[name="q"]')[0].value = query;
-          this.performSearch(query);
-      }
-  },
-
-  loadIndex : function(url) {
-    $.ajax({type: "GET", url: url, data: null,
-            dataType: "script", cache: true,
-            complete: function(jqxhr, textstatus) {
-              if (textstatus != "success") {
-                document.getElementById("searchindexloader").src = url;
-              }
-            }});
-  },
-
-  setIndex : function(index) {
-    var q;
-    this._index = index;
-    if ((q = this._queued_query) !== null) {
-      this._queued_query = null;
-      Search.query(q);
-    }
-  },
-
-  hasIndex : function() {
-      return this._index !== null;
-  },
-
-  deferQuery : function(query) {
-      this._queued_query = query;
-  },
-
-  stopPulse : function() {
-      this._pulse_status = 0;
-  },
-
-  startPulse : function() {
-    if (this._pulse_status >= 0)
-        return;
-    function pulse() {
-      var i;
-      Search._pulse_status = (Search._pulse_status + 1) % 4;
-      var dotString = '';
-      for (i = 0; i < Search._pulse_status; i++)
-        dotString += '.';
-      Search.dots.text(dotString);
-      if (Search._pulse_status > -1)
-        window.setTimeout(pulse, 500);
-    }
-    pulse();
-  },
-
-  /**
-   * perform a search for something (or wait until index is loaded)
-   */
-  performSearch : function(query) {
-    // create the required interface elements
-    this.out = $('#search-results');
-    this.title = $('<h2>' + _('Searching') + '</h2>').appendTo(this.out);
-    this.dots = $('<span></span>').appendTo(this.title);
-    this.status = $('<p style="display: none"></p>').appendTo(this.out);
-    this.output = $('<ul class="search"/>').appendTo(this.out);
-
-    $('#search-progress').text(_('Preparing search...'));
-    this.startPulse();
-
-    // index already loaded, the browser was quick!
-    if (this.hasIndex())
-      this.query(query);
-    else
-      this.deferQuery(query);
-  },
-
-  /**
-   * execute search (requires search index to be loaded)
-   */
-  query : function(query) {
-    var i;
-    var stopwords = ["a","and","are","as","at","be","but","by","for","if","in","into","is","it","near","no","not","of","on","or","such","that","the","their","then","there","these","they","this","to","was","will","with"];
-
-    // stem the searchterms and add them to the correct list
-    var stemmer = new Stemmer();
-    var searchterms = [];
-    var excluded = [];
-    var hlterms = [];
-    var tmp = splitQuery(query);
-    var objectterms = [];
-    for (i = 0; i < tmp.length; i++) {
-      if (tmp[i] !== "") {
-          objectterms.push(tmp[i].toLowerCase());
-      }
-
-      if ($u.indexOf(stopwords, tmp[i].toLowerCase()) != -1 || tmp[i].match(/^\d+$/) ||
-          tmp[i] === "") {
-        // skip this "word"
-        continue;
-      }
-      // stem the word
-      var word = stemmer.stemWord(tmp[i].toLowerCase());
-      // prevent stemmer from cutting word smaller than two chars
-      if(word.length < 3 && tmp[i].length >= 3) {
-        word = tmp[i];
-      }
-      var toAppend;
-      // select the correct list
-      if (word[0] == '-') {
-        toAppend = excluded;
-        word = word.substr(1);
-      }
-      else {
-        toAppend = searchterms;
-        hlterms.push(tmp[i].toLowerCase());
-      }
-      // only add if not already in the list
-      if (!$u.contains(toAppend, word))
-        toAppend.push(word);
-    }
-    var highlightstring = '?highlight=' + $.urlencode(hlterms.join(" "));
-
-    // console.debug('SEARCH: searching for:');
-    // console.info('required: ', searchterms);
-    // console.info('excluded: ', excluded);
-
-    // prepare search
-    var terms = this._index.terms;
-    var titleterms = this._index.titleterms;
-
-    // array of [filename, title, anchor, descr, score]
-    var results = [];
-    $('#search-progress').empty();
-
-    // lookup as object
-    for (i = 0; i < objectterms.length; i++) {
-      var others = [].concat(objectterms.slice(0, i),
-                             objectterms.slice(i+1, objectterms.length));
-      results = results.concat(this.performObjectSearch(objectterms[i], others));
-    }
-
-    // lookup as search terms in fulltext
-    results = results.concat(this.performTermsSearch(searchterms, excluded, terms, titleterms));
-
-    // let the scorer override scores with a custom scoring function
-    if (Scorer.score) {
-      for (i = 0; i < results.length; i++)
-        results[i][4] = Scorer.score(results[i]);
-    }
-
-    // now sort the results by score (in opposite order of appearance, since the
-    // display function below uses pop() to retrieve items) and then
-    // alphabetically
-    results.sort(function(a, b) {
-      var left = a[4];
-      var right = b[4];
-      if (left > right) {
-        return 1;
-      } else if (left < right) {
-        return -1;
-      } else {
-        // same score: sort alphabetically
-        left = a[1].toLowerCase();
-        right = b[1].toLowerCase();
-        return (left > right) ? -1 : ((left < right) ? 1 : 0);
-      }
-    });
-
-    // for debugging
-    //Search.lastresults = results.slice();  // a copy
-    //console.info('search results:', Search.lastresults);
-
-    // print the results
-    var resultCount = results.length;
-    function displayNextItem() {
-      // results left, load the summary and display it
-      if (results.length) {
-        var item = results.pop();
-        var listItem = $('<li style="display:none"></li>');
-        if (DOCUMENTATION_OPTIONS.FILE_SUFFIX === '') {
-          // dirhtml builder
-          var dirname = item[0] + '/';
-          if (dirname.match(/\/index\/$/)) {
-            dirname = dirname.substring(0, dirname.length-6);
-          } else if (dirname == 'index/') {
-            dirname = '';
-          }
-          listItem.append($('<a/>').attr('href',
-            DOCUMENTATION_OPTIONS.URL_ROOT + dirname +
-            highlightstring + item[2]).html(item[1]));
-        } else {
-          // normal html builders
-          listItem.append($('<a/>').attr('href',
-            item[0] + DOCUMENTATION_OPTIONS.FILE_SUFFIX +
-            highlightstring + item[2]).html(item[1]));
-        }
-        if (item[3]) {
-          listItem.append($('<span> (' + item[3] + ')</span>'));
-          Search.output.append(listItem);
-          listItem.slideDown(5, function() {
-            displayNextItem();
-          });
-        } else if (DOCUMENTATION_OPTIONS.HAS_SOURCE) {
-          var suffix = DOCUMENTATION_OPTIONS.SOURCELINK_SUFFIX;
-          if (suffix === undefined) {
-            suffix = '.txt';
-          }
-          $.ajax({url: DOCUMENTATION_OPTIONS.URL_ROOT + '_sources/' + item[5] + (item[5].slice(-suffix.length) === suffix ? '' : suffix),
-                  dataType: "text",
-                  complete: function(jqxhr, textstatus) {
-                    var data = jqxhr.responseText;
-                    if (data !== '' && data !== undefined) {
-                      listItem.append(Search.makeSearchSummary(data, searchterms, hlterms));
-                    }
-                    Search.output.append(listItem);
-                    listItem.slideDown(5, function() {
-                      displayNextItem();
-                    });
-                  }});
-        } else {
-          // no source available, just display title
-          Search.output.append(listItem);
-          listItem.slideDown(5, function() {
-            displayNextItem();
-          });
-        }
-      }
-      // search finished, update title and status message
-      else {
-        Search.stopPulse();
-        Search.title.text(_('Search Results'));
-        if (!resultCount)
-          Search.status.text(_('Your search did not match any documents. Please make sure that all words are spelled correctly and that you\'ve selected enough categories.'));
-        else
-            Search.status.text(_('Search finished, found %s page(s) matching the search query.').replace('%s', resultCount));
-        Search.status.fadeIn(500);
-      }
-    }
-    displayNextItem();
-  },
-
-  /**
-   * search for object names
-   */
-  performObjectSearch : function(object, otherterms) {
-    var filenames = this._index.filenames;
-    var docnames = this._index.docnames;
-    var objects = this._index.objects;
-    var objnames = this._index.objnames;
-    var titles = this._index.titles;
-
-    var i;
-    var results = [];
-
-    for (var prefix in objects) {
-      for (var name in objects[prefix]) {
-        var fullname = (prefix ? prefix + '.' : '') + name;
-        if (fullname.toLowerCase().indexOf(object) > -1) {
-          var score = 0;
-          var parts = fullname.split('.');
-          // check for different match types: exact matches of full name or
-          // "last name" (i.e. last dotted part)
-          if (fullname == object || parts[parts.length - 1] == object) {
-            score += Scorer.objNameMatch;
-          // matches in last name
-          } else if (parts[parts.length - 1].indexOf(object) > -1) {
-            score += Scorer.objPartialMatch;
-          }
-          var match = objects[prefix][name];
-          var objname = objnames[match[1]][2];
-          var title = titles[match[0]];
-          // If more than one term searched for, we require other words to be
-          // found in the name/title/description
-          if (otherterms.length > 0) {
-            var haystack = (prefix + ' ' + name + ' ' +
-                            objname + ' ' + title).toLowerCase();
-            var allfound = true;
-            for (i = 0; i < otherterms.length; i++) {
-              if (haystack.indexOf(otherterms[i]) == -1) {
-                allfound = false;
-                break;
-              }
-            }
-            if (!allfound) {
-              continue;
-            }
-          }
-          var descr = objname + _(', in ') + title;
-
-          var anchor = match[3];
-          if (anchor === '')
-            anchor = fullname;
-          else if (anchor == '-')
-            anchor = objnames[match[1]][1] + '-' + fullname;
-          // add custom score for some objects according to scorer
-          if (Scorer.objPrio.hasOwnProperty(match[2])) {
-            score += Scorer.objPrio[match[2]];
-          } else {
-            score += Scorer.objPrioDefault;
-          }
-          results.push([docnames[match[0]], fullname, '#'+anchor, descr, score, filenames[match[0]]]);
-        }
-      }
-    }
-
-    return results;
-  },
-
-  /**
-   * search for full-text terms in the index
-   */
-  performTermsSearch : function(searchterms, excluded, terms, titleterms) {
-    var docnames = this._index.docnames;
-    var filenames = this._index.filenames;
-    var titles = this._index.titles;
-
-    var i, j, file;
-    var fileMap = {};
-    var scoreMap = {};
-    var results = [];
-
-    // perform the search on the required terms
-    for (i = 0; i < searchterms.length; i++) {
-      var word = searchterms[i];
-      var files = [];
-      var _o = [
-        {files: terms[word], score: Scorer.term},
-        {files: titleterms[word], score: Scorer.title}
-      ];
-
-      // no match but word was a required one
-      if ($u.every(_o, function(o){return o.files === undefined;})) {
-        break;
-      }
-      // found search word in contents
-      $u.each(_o, function(o) {
-        var _files = o.files;
-        if (_files === undefined)
-          return
-
-        if (_files.length === undefined)
-          _files = [_files];
-        files = files.concat(_files);
-
-        // set score for the word in each file to Scorer.term
-        for (j = 0; j < _files.length; j++) {
-          file = _files[j];
-          if (!(file in scoreMap))
-            scoreMap[file] = {}
-          scoreMap[file][word] = o.score;
-        }
-      });
-
-      // create the mapping
-      for (j = 0; j < files.length; j++) {
-        file = files[j];
-        if (file in fileMap)
-          fileMap[file].push(word);
-        else
-          fileMap[file] = [word];
-      }
-    }
-
-    // now check if the files don't contain excluded terms
-    for (file in fileMap) {
-      var valid = true;
-
-      // check if all requirements are matched
-      if (fileMap[file].length != searchterms.length)
-          continue;
-
-      // ensure that none of the excluded terms is in the search result
-      for (i = 0; i < excluded.length; i++) {
-        if (terms[excluded[i]] == file ||
-            titleterms[excluded[i]] == file ||
-            $u.contains(terms[excluded[i]] || [], file) ||
-            $u.contains(titleterms[excluded[i]] || [], file)) {
-          valid = false;
-          break;
-        }
-      }
-
-      // if we have still a valid result we can add it to the result list
-      if (valid) {
-        // select one (max) score for the file.
-        // for better ranking, we should calculate ranking by using words statistics like basic tf-idf...
-        var score = $u.max($u.map(fileMap[file], function(w){return scoreMap[file][w]}));
-        results.push([docnames[file], titles[file], '', null, score, filenames[file]]);
-      }
-    }
-    return results;
-  },
-
-  /**
-   * helper function to return a node containing the
-   * search summary for a given text. keywords is a list
-   * of stemmed words, hlwords is the list of normal, unstemmed
-   * words. the first one is used to find the occurrence, the
-   * latter for highlighting it.
-   */
-  makeSearchSummary : function(text, keywords, hlwords) {
-    var textLower = text.toLowerCase();
-    var start = 0;
-    $.each(keywords, function() {
-      var i = textLower.indexOf(this.toLowerCase());
-      if (i > -1)
-        start = i;
-    });
-    start = Math.max(start - 120, 0);
-    var excerpt = ((start > 0) ? '...' : '') +
-      $.trim(text.substr(start, 240)) +
-      ((start + 240 - text.length) ? '...' : '');
-    var rv = $('<div class="context"></div>').text(excerpt);
-    $.each(hlwords, function() {
-      rv = rv.highlightText(this, 'highlighted');
-    });
-    return rv;
-  }
-};
-
-$(document).ready(function() {
-  Search.init();
-});

docs/_build/html/_static/underscore-1.3.1.js

@@ -1,999 +0,0 @@
-//     Underscore.js 1.3.1
-//     (c) 2009-2012 Jeremy Ashkenas, DocumentCloud Inc.
-//     Underscore is freely distributable under the MIT license.
-//     Portions of Underscore are inspired or borrowed from Prototype,
-//     Oliver Steele's Functional, and John Resig's Micro-Templating.
-//     For all details and documentation:
-//     http://documentcloud.github.com/underscore
-
-(function() {
-
-  // Baseline setup
-  // --------------
-
-  // Establish the root object, `window` in the browser, or `global` on the server.
-  var root = this;
-
-  // Save the previous value of the `_` variable.
-  var previousUnderscore = root._;
-
-  // Establish the object that gets returned to break out of a loop iteration.
-  var breaker = {};
-
-  // Save bytes in the minified (but not gzipped) version:
-  var ArrayProto = Array.prototype, ObjProto = Object.prototype, FuncProto = Function.prototype;
-
-  // Create quick reference variables for speed access to core prototypes.
-  var slice            = ArrayProto.slice,
-      unshift          = ArrayProto.unshift,
-      toString         = ObjProto.toString,
-      hasOwnProperty   = ObjProto.hasOwnProperty;
-
-  // All **ECMAScript 5** native function implementations that we hope to use
-  // are declared here.
-  var
-    nativeForEach      = ArrayProto.forEach,
-    nativeMap          = ArrayProto.map,
-    nativeReduce       = ArrayProto.reduce,
-    nativeReduceRight  = ArrayProto.reduceRight,
-    nativeFilter       = ArrayProto.filter,
-    nativeEvery        = ArrayProto.every,
-    nativeSome         = ArrayProto.some,
-    nativeIndexOf      = ArrayProto.indexOf,
-    nativeLastIndexOf  = ArrayProto.lastIndexOf,
-    nativeIsArray      = Array.isArray,
-    nativeKeys         = Object.keys,
-    nativeBind         = FuncProto.bind;
-
-  // Create a safe reference to the Underscore object for use below.
-  var _ = function(obj) { return new wrapper(obj); };
-
-  // Export the Underscore object for **Node.js**, with
-  // backwards-compatibility for the old `require()` API. If we're in
-  // the browser, add `_` as a global object via a string identifier,
-  // for Closure Compiler "advanced" mode.
-  if (typeof exports !== 'undefined') {
-    if (typeof module !== 'undefined' && module.exports) {
-      exports = module.exports = _;
-    }
-    exports._ = _;
-  } else {
-    root['_'] = _;
-  }
-
-  // Current version.
-  _.VERSION = '1.3.1';
-
-  // Collection Functions
-  // --------------------
-
-  // The cornerstone, an `each` implementation, aka `forEach`.
-  // Handles objects with the built-in `forEach`, arrays, and raw objects.
-  // Delegates to **ECMAScript 5**'s native `forEach` if available.
-  var each = _.each = _.forEach = function(obj, iterator, context) {
-    if (obj == null) return;
-    if (nativeForEach && obj.forEach === nativeForEach) {
-      obj.forEach(iterator, context);
-    } else if (obj.length === +obj.length) {
-      for (var i = 0, l = obj.length; i < l; i++) {
-        if (i in obj && iterator.call(context, obj[i], i, obj) === breaker) return;
-      }
-    } else {
-      for (var key in obj) {
-        if (_.has(obj, key)) {
-          if (iterator.call(context, obj[key], key, obj) === breaker) return;
-        }
-      }
-    }
-  };
-
-  // Return the results of applying the iterator to each element.
-  // Delegates to **ECMAScript 5**'s native `map` if available.
-  _.map = _.collect = function(obj, iterator, context) {
-    var results = [];
-    if (obj == null) return results;
-    if (nativeMap && obj.map === nativeMap) return obj.map(iterator, context);
-    each(obj, function(value, index, list) {
-      results[results.length] = iterator.call(context, value, index, list);
-    });
-    if (obj.length === +obj.length) results.length = obj.length;
-    return results;
-  };
-
-  // **Reduce** builds up a single result from a list of values, aka `inject`,
-  // or `foldl`. Delegates to **ECMAScript 5**'s native `reduce` if available.
-  _.reduce = _.foldl = _.inject = function(obj, iterator, memo, context) {
-    var initial = arguments.length > 2;
-    if (obj == null) obj = [];
-    if (nativeReduce && obj.reduce === nativeReduce) {
-      if (context) iterator = _.bind(iterator, context);
-      return initial ? obj.reduce(iterator, memo) : obj.reduce(iterator);
-    }
-    each(obj, function(value, index, list) {
-      if (!initial) {
-        memo = value;
-        initial = true;
-      } else {
-        memo = iterator.call(context, memo, value, index, list);
-      }
-    });
-    if (!initial) throw new TypeError('Reduce of empty array with no initial value');
-    return memo;
-  };
-
-  // The right-associative version of reduce, also known as `foldr`.
-  // Delegates to **ECMAScript 5**'s native `reduceRight` if available.
-  _.reduceRight = _.foldr = function(obj, iterator, memo, context) {
-    var initial = arguments.length > 2;
-    if (obj == null) obj = [];
-    if (nativeReduceRight && obj.reduceRight === nativeReduceRight) {
-      if (context) iterator = _.bind(iterator, context);
-      return initial ? obj.reduceRight(iterator, memo) : obj.reduceRight(iterator);
-    }
-    var reversed = _.toArray(obj).reverse();
-    if (context && !initial) iterator = _.bind(iterator, context);
-    return initial ? _.reduce(reversed, iterator, memo, context) : _.reduce(reversed, iterator);
-  };
-
-  // Return the first value which passes a truth test. Aliased as `detect`.
-  _.find = _.detect = function(obj, iterator, context) {
-    var result;
-    any(obj, function(value, index, list) {
-      if (iterator.call(context, value, index, list)) {
-        result = value;
-        return true;
-      }
-    });
-    return result;
-  };
-
-  // Return all the elements that pass a truth test.
-  // Delegates to **ECMAScript 5**'s native `filter` if available.
-  // Aliased as `select`.
-  _.filter = _.select = function(obj, iterator, context) {
-    var results = [];
-    if (obj == null) return results;
-    if (nativeFilter && obj.filter === nativeFilter) return obj.filter(iterator, context);
-    each(obj, function(value, index, list) {
-      if (iterator.call(context, value, index, list)) results[results.length] = value;
-    });
-    return results;
-  };
-
-  // Return all the elements for which a truth test fails.
-  _.reject = function(obj, iterator, context) {
-    var results = [];
-    if (obj == null) return results;
-    each(obj, function(value, index, list) {
-      if (!iterator.call(context, value, index, list)) results[results.length] = value;
-    });
-    return results;
-  };
-
-  // Determine whether all of the elements match a truth test.
-  // Delegates to **ECMAScript 5**'s native `every` if available.
-  // Aliased as `all`.
-  _.every = _.all = function(obj, iterator, context) {
-    var result = true;
-    if (obj == null) return result;
-    if (nativeEvery && obj.every === nativeEvery) return obj.every(iterator, context);
-    each(obj, function(value, index, list) {
-      if (!(result = result && iterator.call(context, value, index, list))) return breaker;
-    });
-    return result;
-  };
-
-  // Determine if at least one element in the object matches a truth test.
-  // Delegates to **ECMAScript 5**'s native `some` if available.
-  // Aliased as `any`.
-  var any = _.some = _.any = function(obj, iterator, context) {
-    iterator || (iterator = _.identity);
-    var result = false;
-    if (obj == null) return result;
-    if (nativeSome && obj.some === nativeSome) return obj.some(iterator, context);
-    each(obj, function(value, index, list) {
-      if (result || (result = iterator.call(context, value, index, list))) return breaker;
-    });
-    return !!result;
-  };
-
-  // Determine if a given value is included in the array or object using `===`.
-  // Aliased as `contains`.
-  _.include = _.contains = function(obj, target) {
-    var found = false;
-    if (obj == null) return found;
-    if (nativeIndexOf && obj.indexOf === nativeIndexOf) return obj.indexOf(target) != -1;
-    found = any(obj, function(value) {
-      return value === target;
-    });
-    return found;
-  };
-
-  // Invoke a method (with arguments) on every item in a collection.
-  _.invoke = function(obj, method) {
-    var args = slice.call(arguments, 2);
-    return _.map(obj, function(value) {
-      return (_.isFunction(method) ? method || value : value[method]).apply(value, args);
-    });
-  };
-
-  // Convenience version of a common use case of `map`: fetching a property.
-  _.pluck = function(obj, key) {
-    return _.map(obj, function(value){ return value[key]; });
-  };
-
-  // Return the maximum element or (element-based computation).
-  _.max = function(obj, iterator, context) {
-    if (!iterator && _.isArray(obj)) return Math.max.apply(Math, obj);
-    if (!iterator && _.isEmpty(obj)) return -Infinity;
-    var result = {computed : -Infinity};
-    each(obj, function(value, index, list) {
-      var computed = iterator ? iterator.call(context, value, index, list) : value;
-      computed >= result.computed && (result = {value : value, computed : computed});
-    });
-    return result.value;
-  };
-
-  // Return the minimum element (or element-based computation).
-  _.min = function(obj, iterator, context) {
-    if (!iterator && _.isArray(obj)) return Math.min.apply(Math, obj);
-    if (!iterator && _.isEmpty(obj)) return Infinity;
-    var result = {computed : Infinity};
-    each(obj, function(value, index, list) {
-      var computed = iterator ? iterator.call(context, value, index, list) : value;
-      computed < result.computed && (result = {value : value, computed : computed});
-    });
-    return result.value;
-  };
-
-  // Shuffle an array.
-  _.shuffle = function(obj) {
-    var shuffled = [], rand;
-    each(obj, function(value, index, list) {
-      if (index == 0) {
-        shuffled[0] = value;
-      } else {
-        rand = Math.floor(Math.random() * (index + 1));
-        shuffled[index] = shuffled[rand];
-        shuffled[rand] = value;
-      }
-    });
-    return shuffled;
-  };
-
-  // Sort the object's values by a criterion produced by an iterator.
-  _.sortBy = function(obj, iterator, context) {
-    return _.pluck(_.map(obj, function(value, index, list) {
-      return {
-        value : value,
-        criteria : iterator.call(context, value, index, list)
-      };
-    }).sort(function(left, right) {
-      var a = left.criteria, b = right.criteria;
-      return a < b ? -1 : a > b ? 1 : 0;
-    }), 'value');
-  };
-
-  // Groups the object's values by a criterion. Pass either a string attribute
-  // to group by, or a function that returns the criterion.
-  _.groupBy = function(obj, val) {
-    var result = {};
-    var iterator = _.isFunction(val) ? val : function(obj) { return obj[val]; };
-    each(obj, function(value, index) {
-      var key = iterator(value, index);
-      (result[key] || (result[key] = [])).push(value);
-    });
-    return result;
-  };
-
-  // Use a comparator function to figure out at what index an object should
-  // be inserted so as to maintain order. Uses binary search.
-  _.sortedIndex = function(array, obj, iterator) {
-    iterator || (iterator = _.identity);
-    var low = 0, high = array.length;
-    while (low < high) {
-      var mid = (low + high) >> 1;
-      iterator(array[mid]) < iterator(obj) ? low = mid + 1 : high = mid;
-    }
-    return low;
-  };
-
-  // Safely convert anything iterable into a real, live array.
-  _.toArray = function(iterable) {
-    if (!iterable)                return [];
-    if (iterable.toArray)         return iterable.toArray();
-    if (_.isArray(iterable))      return slice.call(iterable);
-    if (_.isArguments(iterable))  return slice.call(iterable);
-    return _.values(iterable);
-  };
-
-  // Return the number of elements in an object.
-  _.size = function(obj) {
-    return _.toArray(obj).length;
-  };
-
-  // Array Functions
-  // ---------------
-
-  // Get the first element of an array. Passing **n** will return the first N
-  // values in the array. Aliased as `head`. The **guard** check allows it to work
-  // with `_.map`.
-  _.first = _.head = function(array, n, guard) {
-    return (n != null) && !guard ? slice.call(array, 0, n) : array[0];
-  };
-
-  // Returns everything but the last entry of the array. Especcialy useful on
-  // the arguments object. Passing **n** will return all the values in
-  // the array, excluding the last N. The **guard** check allows it to work with
-  // `_.map`.
-  _.initial = function(array, n, guard) {
-    return slice.call(array, 0, array.length - ((n == null) || guard ? 1 : n));
-  };
-
-  // Get the last element of an array. Passing **n** will return the last N
-  // values in the array. The **guard** check allows it to work with `_.map`.
-  _.last = function(array, n, guard) {
-    if ((n != null) && !guard) {
-      return slice.call(array, Math.max(array.length - n, 0));
-    } else {
-      return array[array.length - 1];
-    }
-  };
-
-  // Returns everything but the first entry of the array. Aliased as `tail`.
-  // Especially useful on the arguments object. Passing an **index** will return
-  // the rest of the values in the array from that index onward. The **guard**
-  // check allows it to work with `_.map`.
-  _.rest = _.tail = function(array, index, guard) {
-    return slice.call(array, (index == null) || guard ? 1 : index);
-  };
-
-  // Trim out all falsy values from an array.
-  _.compact = function(array) {
-    return _.filter(array, function(value){ return !!value; });
-  };
-
-  // Return a completely flattened version of an array.
-  _.flatten = function(array, shallow) {
-    return _.reduce(array, function(memo, value) {
-      if (_.isArray(value)) return memo.concat(shallow ? value : _.flatten(value));
-      memo[memo.length] = value;
-      return memo;
-    }, []);
-  };
-
-  // Return a version of the array that does not contain the specified value(s).
-  _.without = function(array) {
-    return _.difference(array, slice.call(arguments, 1));
-  };
-
-  // Produce a duplicate-free version of the array. If the array has already
-  // been sorted, you have the option of using a faster algorithm.
-  // Aliased as `unique`.
-  _.uniq = _.unique = function(array, isSorted, iterator) {
-    var initial = iterator ? _.map(array, iterator) : array;
-    var result = [];
-    _.reduce(initial, function(memo, el, i) {
-      if (0 == i || (isSorted === true ? _.last(memo) != el : !_.include(memo, el))) {
-        memo[memo.length] = el;
-        result[result.length] = array[i];
-      }
-      return memo;
-    }, []);
-    return result;
-  };
-
-  // Produce an array that contains the union: each distinct element from all of
-  // the passed-in arrays.
-  _.union = function() {
-    return _.uniq(_.flatten(arguments, true));
-  };
-
-  // Produce an array that contains every item shared between all the
-  // passed-in arrays. (Aliased as "intersect" for back-compat.)
-  _.intersection = _.intersect = function(array) {
-    var rest = slice.call(arguments, 1);
-    return _.filter(_.uniq(array), function(item) {
-      return _.every(rest, function(other) {
-        return _.indexOf(other, item) >= 0;
-      });
-    });
-  };
-
-  // Take the difference between one array and a number of other arrays.
-  // Only the elements present in just the first array will remain.
-  _.difference = function(array) {
-    var rest = _.flatten(slice.call(arguments, 1));
-    return _.filter(array, function(value){ return !_.include(rest, value); });
-  };
-
-  // Zip together multiple lists into a single array -- elements that share
-  // an index go together.
-  _.zip = function() {
-    var args = slice.call(arguments);
-    var length = _.max(_.pluck(args, 'length'));
-    var results = new Array(length);
-    for (var i = 0; i < length; i++) results[i] = _.pluck(args, "" + i);
-    return results;
-  };
-
-  // If the browser doesn't supply us with indexOf (I'm looking at you, **MSIE**),
-  // we need this function. Return the position of the first occurrence of an
-  // item in an array, or -1 if the item is not included in the array.
-  // Delegates to **ECMAScript 5**'s native `indexOf` if available.
-  // If the array is large and already in sort order, pass `true`
-  // for **isSorted** to use binary search.
-  _.indexOf = function(array, item, isSorted) {
-    if (array == null) return -1;
-    var i, l;
-    if (isSorted) {
-      i = _.sortedIndex(array, item);
-      return array[i] === item ? i : -1;
-    }
-    if (nativeIndexOf && array.indexOf === nativeIndexOf) return array.indexOf(item);
-    for (i = 0, l = array.length; i < l; i++) if (i in array && array[i] === item) return i;
-    return -1;
-  };
-
-  // Delegates to **ECMAScript 5**'s native `lastIndexOf` if available.
-  _.lastIndexOf = function(array, item) {
-    if (array == null) return -1;
-    if (nativeLastIndexOf && array.lastIndexOf === nativeLastIndexOf) return array.lastIndexOf(item);
-    var i = array.length;
-    while (i--) if (i in array && array[i] === item) return i;
-    return -1;
-  };
-
-  // Generate an integer Array containing an arithmetic progression. A port of
-  // the native Python `range()` function. See
-  // [the Python documentation](http://docs.python.org/library/functions.html#range).
-  _.range = function(start, stop, step) {
-    if (arguments.length <= 1) {
-      stop = start || 0;
-      start = 0;
-    }
-    step = arguments[2] || 1;
-
-    var len = Math.max(Math.ceil((stop - start) / step), 0);
-    var idx = 0;
-    var range = new Array(len);
-
-    while(idx < len) {
-      range[idx++] = start;
-      start += step;
-    }
-
-    return range;
-  };
-
-  // Function (ahem) Functions
-  // ------------------
-
-  // Reusable constructor function for prototype setting.
-  var ctor = function(){};
-
-  // Create a function bound to a given object (assigning `this`, and arguments,
-  // optionally). Binding with arguments is also known as `curry`.
-  // Delegates to **ECMAScript 5**'s native `Function.bind` if available.
-  // We check for `func.bind` first, to fail fast when `func` is undefined.
-  _.bind = function bind(func, context) {
-    var bound, args;
-    if (func.bind === nativeBind && nativeBind) return nativeBind.apply(func, slice.call(arguments, 1));
-    if (!_.isFunction(func)) throw new TypeError;
-    args = slice.call(arguments, 2);
-    return bound = function() {
-      if (!(this instanceof bound)) return func.apply(context, args.concat(slice.call(arguments)));
-      ctor.prototype = func.prototype;
-      var self = new ctor;
-      var result = func.apply(self, args.concat(slice.call(arguments)));
-      if (Object(result) === result) return result;
-      return self;
-    };
-  };
-
-  // Bind all of an object's methods to that object. Useful for ensuring that
-  // all callbacks defined on an object belong to it.
-  _.bindAll = function(obj) {
-    var funcs = slice.call(arguments, 1);
-    if (funcs.length == 0) funcs = _.functions(obj);
-    each(funcs, function(f) { obj[f] = _.bind(obj[f], obj); });
-    return obj;
-  };
-
-  // Memoize an expensive function by storing its results.
-  _.memoize = function(func, hasher) {
-    var memo = {};
-    hasher || (hasher = _.identity);
-    return function() {
-      var key = hasher.apply(this, arguments);
-      return _.has(memo, key) ? memo[key] : (memo[key] = func.apply(this, arguments));
-    };
-  };
-
-  // Delays a function for the given number of milliseconds, and then calls
-  // it with the arguments supplied.
-  _.delay = function(func, wait) {
-    var args = slice.call(arguments, 2);
-    return setTimeout(function(){ return func.apply(func, args); }, wait);
-  };
-
-  // Defers a function, scheduling it to run after the current call stack has
-  // cleared.
-  _.defer = function(func) {
-    return _.delay.apply(_, [func, 1].concat(slice.call(arguments, 1)));
-  };
-
-  // Returns a function, that, when invoked, will only be triggered at most once
-  // during a given window of time.
-  _.throttle = function(func, wait) {
-    var context, args, timeout, throttling, more;
-    var whenDone = _.debounce(function(){ more = throttling = false; }, wait);
-    return function() {
-      context = this; args = arguments;
-      var later = function() {
-        timeout = null;
-        if (more) func.apply(context, args);
-        whenDone();
-      };
-      if (!timeout) timeout = setTimeout(later, wait);
-      if (throttling) {
-        more = true;
-      } else {
-        func.apply(context, args);
-      }
-      whenDone();
-      throttling = true;
-    };
-  };
-
-  // Returns a function, that, as long as it continues to be invoked, will not
-  // be triggered. The function will be called after it stops being called for
-  // N milliseconds.
-  _.debounce = function(func, wait) {
-    var timeout;
-    return function() {
-      var context = this, args = arguments;
-      var later = function() {
-        timeout = null;
-        func.apply(context, args);
-      };
-      clearTimeout(timeout);
-      timeout = setTimeout(later, wait);
-    };
-  };
-
-  // Returns a function that will be executed at most one time, no matter how
-  // often you call it. Useful for lazy initialization.
-  _.once = function(func) {
-    var ran = false, memo;
-    return function() {
-      if (ran) return memo;
-      ran = true;
-      return memo = func.apply(this, arguments);
-    };
-  };
-
-  // Returns the first function passed as an argument to the second,
-  // allowing you to adjust arguments, run code before and after, and
-  // conditionally execute the original function.
-  _.wrap = function(func, wrapper) {
-    return function() {
-      var args = [func].concat(slice.call(arguments, 0));
-      return wrapper.apply(this, args);
-    };
-  };
-
-  // Returns a function that is the composition of a list of functions, each
-  // consuming the return value of the function that follows.
-  _.compose = function() {
-    var funcs = arguments;
-    return function() {
-      var args = arguments;
-      for (var i = funcs.length - 1; i >= 0; i--) {
-        args = [funcs[i].apply(this, args)];
-      }
-      return args[0];
-    };
-  };
-
-  // Returns a function that will only be executed after being called N times.
-  _.after = function(times, func) {
-    if (times <= 0) return func();
-    return function() {
-      if (--times < 1) { return func.apply(this, arguments); }
-    };
-  };
-
-  // Object Functions
-  // ----------------
-
-  // Retrieve the names of an object's properties.
-  // Delegates to **ECMAScript 5**'s native `Object.keys`
-  _.keys = nativeKeys || function(obj) {
-    if (obj !== Object(obj)) throw new TypeError('Invalid object');
-    var keys = [];
-    for (var key in obj) if (_.has(obj, key)) keys[keys.length] = key;
-    return keys;
-  };
-
-  // Retrieve the values of an object's properties.
-  _.values = function(obj) {
-    return _.map(obj, _.identity);
-  };
-
-  // Return a sorted list of the function names available on the object.
-  // Aliased as `methods`
-  _.functions = _.methods = function(obj) {
-    var names = [];
-    for (var key in obj) {
-      if (_.isFunction(obj[key])) names.push(key);
-    }
-    return names.sort();
-  };
-
-  // Extend a given object with all the properties in passed-in object(s).
-  _.extend = function(obj) {
-    each(slice.call(arguments, 1), function(source) {
-      for (var prop in source) {
-        obj[prop] = source[prop];
-      }
-    });
-    return obj;
-  };
-
-  // Fill in a given object with default properties.
-  _.defaults = function(obj) {
-    each(slice.call(arguments, 1), function(source) {
-      for (var prop in source) {
-        if (obj[prop] == null) obj[prop] = source[prop];
-      }
-    });
-    return obj;
-  };
-
-  // Create a (shallow-cloned) duplicate of an object.
-  _.clone = function(obj) {
-    if (!_.isObject(obj)) return obj;
-    return _.isArray(obj) ? obj.slice() : _.extend({}, obj);
-  };
-
-  // Invokes interceptor with the obj, and then returns obj.
-  // The primary purpose of this method is to "tap into" a method chain, in
-  // order to perform operations on intermediate results within the chain.
-  _.tap = function(obj, interceptor) {
-    interceptor(obj);
-    return obj;
-  };
-
-  // Internal recursive comparison function.
-  function eq(a, b, stack) {
-    // Identical objects are equal. `0 === -0`, but they aren't identical.
-    // See the Harmony `egal` proposal: http://wiki.ecmascript.org/doku.php?id=harmony:egal.
-    if (a === b) return a !== 0 || 1 / a == 1 / b;
-    // A strict comparison is necessary because `null == undefined`.
-    if (a == null || b == null) return a === b;
-    // Unwrap any wrapped objects.
-    if (a._chain) a = a._wrapped;
-    if (b._chain) b = b._wrapped;
-    // Invoke a custom `isEqual` method if one is provided.
-    if (a.isEqual && _.isFunction(a.isEqual)) return a.isEqual(b);
-    if (b.isEqual && _.isFunction(b.isEqual)) return b.isEqual(a);
-    // Compare `[[Class]]` names.
-    var className = toString.call(a);
-    if (className != toString.call(b)) return false;
-    switch (className) {
-      // Strings, numbers, dates, and booleans are compared by value.
-      case '[object String]':
-        // Primitives and their corresponding object wrappers are equivalent; thus, `"5"` is
-        // equivalent to `new String("5")`.
-        return a == String(b);
-      case '[object Number]':
-        // `NaN`s are equivalent, but non-reflexive. An `egal` comparison is performed for
-        // other numeric values.
-        return a != +a ? b != +b : (a == 0 ? 1 / a == 1 / b : a == +b);
-      case '[object Date]':
-      case '[object Boolean]':
-        // Coerce dates and booleans to numeric primitive values. Dates are compared by their
-        // millisecond representations. Note that invalid dates with millisecond representations
-        // of `NaN` are not equivalent.
-        return +a == +b;
-      // RegExps are compared by their source patterns and flags.
-      case '[object RegExp]':
-        return a.source == b.source &&
-               a.global == b.global &&
-               a.multiline == b.multiline &&
-               a.ignoreCase == b.ignoreCase;
-    }
-    if (typeof a != 'object' || typeof b != 'object') return false;
-    // Assume equality for cyclic structures. The algorithm for detecting cyclic
-    // structures is adapted from ES 5.1 section 15.12.3, abstract operation `JO`.
-    var length = stack.length;
-    while (length--) {
-      // Linear search. Performance is inversely proportional to the number of
-      // unique nested structures.
-      if (stack[length] == a) return true;
-    }
-    // Add the first object to the stack of traversed objects.
-    stack.push(a);
-    var size = 0, result = true;
-    // Recursively compare objects and arrays.
-    if (className == '[object Array]') {
-      // Compare array lengths to determine if a deep comparison is necessary.
-      size = a.length;
-      result = size == b.length;
-      if (result) {
-        // Deep compare the contents, ignoring non-numeric properties.
-        while (size--) {
-          // Ensure commutative equality for sparse arrays.
-          if (!(result = size in a == size in b && eq(a[size], b[size], stack))) break;
-        }
-      }
-    } else {
-      // Objects with different constructors are not equivalent.
-      if ('constructor' in a != 'constructor' in b || a.constructor != b.constructor) return false;
-      // Deep compare objects.
-      for (var key in a) {
-        if (_.has(a, key)) {
-          // Count the expected number of properties.
-          size++;
-          // Deep compare each member.
-          if (!(result = _.has(b, key) && eq(a[key], b[key], stack))) break;
-        }
-      }
-      // Ensure that both objects contain the same number of properties.
-      if (result) {
-        for (key in b) {
-          if (_.has(b, key) && !(size--)) break;
-        }
-        result = !size;
-      }
-    }
-    // Remove the first object from the stack of traversed objects.
-    stack.pop();
-    return result;
-  }
-
-  // Perform a deep comparison to check if two objects are equal.
-  _.isEqual = function(a, b) {
-    return eq(a, b, []);
-  };
-
-  // Is a given array, string, or object empty?
-  // An "empty" object has no enumerable own-properties.
-  _.isEmpty = function(obj) {
-    if (_.isArray(obj) || _.isString(obj)) return obj.length === 0;
-    for (var key in obj) if (_.has(obj, key)) return false;
-    return true;
-  };
-
-  // Is a given value a DOM element?
-  _.isElement = function(obj) {
-    return !!(obj && obj.nodeType == 1);
-  };
-
-  // Is a given value an array?
-  // Delegates to ECMA5's native Array.isArray
-  _.isArray = nativeIsArray || function(obj) {
-    return toString.call(obj) == '[object Array]';
-  };
-
-  // Is a given variable an object?
-  _.isObject = function(obj) {
-    return obj === Object(obj);
-  };
-
-  // Is a given variable an arguments object?
-  _.isArguments = function(obj) {
-    return toString.call(obj) == '[object Arguments]';
-  };
-  if (!_.isArguments(arguments)) {
-    _.isArguments = function(obj) {
-      return !!(obj && _.has(obj, 'callee'));
-    };
-  }
-
-  // Is a given value a function?
-  _.isFunction = function(obj) {
-    return toString.call(obj) == '[object Function]';
-  };
-
-  // Is a given value a string?
-  _.isString = function(obj) {
-    return toString.call(obj) == '[object String]';
-  };
-
-  // Is a given value a number?
-  _.isNumber = function(obj) {
-    return toString.call(obj) == '[object Number]';
-  };
-
-  // Is the given value `NaN`?
-  _.isNaN = function(obj) {
-    // `NaN` is the only value for which `===` is not reflexive.
-    return obj !== obj;
-  };
-
-  // Is a given value a boolean?
-  _.isBoolean = function(obj) {
-    return obj === true || obj === false || toString.call(obj) == '[object Boolean]';
-  };
-
-  // Is a given value a date?
-  _.isDate = function(obj) {
-    return toString.call(obj) == '[object Date]';
-  };
-
-  // Is the given value a regular expression?
-  _.isRegExp = function(obj) {
-    return toString.call(obj) == '[object RegExp]';
-  };
-
-  // Is a given value equal to null?
-  _.isNull = function(obj) {
-    return obj === null;
-  };
-
-  // Is a given variable undefined?
-  _.isUndefined = function(obj) {
-    return obj === void 0;
-  };
-
-  // Has own property?
-  _.has = function(obj, key) {
-    return hasOwnProperty.call(obj, key);
-  };
-
-  // Utility Functions
-  // -----------------
-
-  // Run Underscore.js in *noConflict* mode, returning the `_` variable to its
-  // previous owner. Returns a reference to the Underscore object.
-  _.noConflict = function() {
-    root._ = previousUnderscore;
-    return this;
-  };
-
-  // Keep the identity function around for default iterators.
-  _.identity = function(value) {
-    return value;
-  };
-
-  // Run a function **n** times.
-  _.times = function (n, iterator, context) {
-    for (var i = 0; i < n; i++) iterator.call(context, i);
-  };
-
-  // Escape a string for HTML interpolation.
-  _.escape = function(string) {
-    return (''+string).replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;').replace(/'/g, '&#x27;').replace(/\//g,'&#x2F;');
-  };
-
-  // Add your own custom functions to the Underscore object, ensuring that
-  // they're correctly added to the OOP wrapper as well.
-  _.mixin = function(obj) {
-    each(_.functions(obj), function(name){
-      addToWrapper(name, _[name] = obj[name]);
-    });
-  };
-
-  // Generate a unique integer id (unique within the entire client session).
-  // Useful for temporary DOM ids.
-  var idCounter = 0;
-  _.uniqueId = function(prefix) {
-    var id = idCounter++;
-    return prefix ? prefix + id : id;
-  };
-
-  // By default, Underscore uses ERB-style template delimiters, change the
-  // following template settings to use alternative delimiters.
-  _.templateSettings = {
-    evaluate    : /<%([\s\S]+?)%>/g,
-    interpolate : /<%=([\s\S]+?)%>/g,
-    escape      : /<%-([\s\S]+?)%>/g
-  };
-
-  // When customizing `templateSettings`, if you don't want to define an
-  // interpolation, evaluation or escaping regex, we need one that is
-  // guaranteed not to match.
-  var noMatch = /.^/;
-
-  // Within an interpolation, evaluation, or escaping, remove HTML escaping
-  // that had been previously added.
-  var unescape = function(code) {
-    return code.replace(/\\\\/g, '\\').replace(/\\'/g, "'");
-  };
-
-  // JavaScript micro-templating, similar to John Resig's implementation.
-  // Underscore templating handles arbitrary delimiters, preserves whitespace,
-  // and correctly escapes quotes within interpolated code.
-  _.template = function(str, data) {
-    var c  = _.templateSettings;
-    var tmpl = 'var __p=[],print=function(){__p.push.apply(__p,arguments);};' +
-      'with(obj||{}){__p.push(\'' +
-      str.replace(/\\/g, '\\\\')
-         .replace(/'/g, "\\'")
-         .replace(c.escape || noMatch, function(match, code) {
-           return "',_.escape(" + unescape(code) + "),'";
-         })
-         .replace(c.interpolate || noMatch, function(match, code) {
-           return "'," + unescape(code) + ",'";
-         })
-         .replace(c.evaluate || noMatch, function(match, code) {
-           return "');" + unescape(code).replace(/[\r\n\t]/g, ' ') + ";__p.push('";
-         })
-         .replace(/\r/g, '\\r')
-         .replace(/\n/g, '\\n')
-         .replace(/\t/g, '\\t')
-         + "');}return __p.join('');";
-    var func = new Function('obj', '_', tmpl);
-    if (data) return func(data, _);
-    return function(data) {
-      return func.call(this, data, _);
-    };
-  };
-
-  // Add a "chain" function, which will delegate to the wrapper.
-  _.chain = function(obj) {
-    return _(obj).chain();
-  };
-
-  // The OOP Wrapper
-  // ---------------
-
-  // If Underscore is called as a function, it returns a wrapped object that
-  // can be used OO-style. This wrapper holds altered versions of all the
-  // underscore functions. Wrapped objects may be chained.
-  var wrapper = function(obj) { this._wrapped = obj; };
-
-  // Expose `wrapper.prototype` as `_.prototype`
-  _.prototype = wrapper.prototype;
-
-  // Helper function to continue chaining intermediate results.
-  var result = function(obj, chain) {
-    return chain ? _(obj).chain() : obj;
-  };
-
-  // A method to easily add functions to the OOP wrapper.
-  var addToWrapper = function(name, func) {
-    wrapper.prototype[name] = function() {
-      var args = slice.call(arguments);
-      unshift.call(args, this._wrapped);
-      return result(func.apply(_, args), this._chain);
-    };
-  };
-
-  // Add all of the Underscore functions to the wrapper object.
-  _.mixin(_);
-
-  // Add all mutator Array functions to the wrapper.
-  each(['pop', 'push', 'reverse', 'shift', 'sort', 'splice', 'unshift'], function(name) {
-    var method = ArrayProto[name];
-    wrapper.prototype[name] = function() {
-      var wrapped = this._wrapped;
-      method.apply(wrapped, arguments);
-      var length = wrapped.length;
-      if ((name == 'shift' || name == 'splice') && length === 0) delete wrapped[0];
-      return result(wrapped, this._chain);
-    };
-  });
-
-  // Add all accessor Array functions to the wrapper.
-  each(['concat', 'join', 'slice'], function(name) {
-    var method = ArrayProto[name];
-    wrapper.prototype[name] = function() {
-      return result(method.apply(this._wrapped, arguments), this._chain);
-    };
-  });
-
-  // Start chaining a wrapped Underscore object.
-  wrapper.prototype.chain = function() {
-    this._chain = true;
-    return this;
-  };
-
-  // Extracts the result from a wrapped and chained object.
-  wrapper.prototype.value = function() {
-    return this._wrapped;
-  };
-
-}).call(this);

docs/_build/html/_static/underscore.js

@@ -1,31 +0,0 @@
-// Underscore.js 1.3.1
-// (c) 2009-2012 Jeremy Ashkenas, DocumentCloud Inc.
-// Underscore is freely distributable under the MIT license.
-// Portions of Underscore are inspired or borrowed from Prototype,
-// Oliver Steele's Functional, and John Resig's Micro-Templating.
-// For all details and documentation:
-// http://documentcloud.github.com/underscore
-(function(){function q(a,c,d){if(a===c)return a!==0||1/a==1/c;if(a==null||c==null)return a===c;if(a._chain)a=a._wrapped;if(c._chain)c=c._wrapped;if(a.isEqual&&b.isFunction(a.isEqual))return a.isEqual(c);if(c.isEqual&&b.isFunction(c.isEqual))return c.isEqual(a);var e=l.call(a);if(e!=l.call(c))return false;switch(e){case "[object String]":return a==String(c);case "[object Number]":return a!=+a?c!=+c:a==0?1/a==1/c:a==+c;case "[object Date]":case "[object Boolean]":return+a==+c;case "[object RegExp]":return a.source==
-c.source&&a.global==c.global&&a.multiline==c.multiline&&a.ignoreCase==c.ignoreCase}if(typeof a!="object"||typeof c!="object")return false;for(var f=d.length;f--;)if(d[f]==a)return true;d.push(a);var f=0,g=true;if(e=="[object Array]"){if(f=a.length,g=f==c.length)for(;f--;)if(!(g=f in a==f in c&&q(a[f],c[f],d)))break}else{if("constructor"in a!="constructor"in c||a.constructor!=c.constructor)return false;for(var h in a)if(b.has(a,h)&&(f++,!(g=b.has(c,h)&&q(a[h],c[h],d))))break;if(g){for(h in c)if(b.has(c,
-h)&&!f--)break;g=!f}}d.pop();return g}var r=this,G=r._,n={},k=Array.prototype,o=Object.prototype,i=k.slice,H=k.unshift,l=o.toString,I=o.hasOwnProperty,w=k.forEach,x=k.map,y=k.reduce,z=k.reduceRight,A=k.filter,B=k.every,C=k.some,p=k.indexOf,D=k.lastIndexOf,o=Array.isArray,J=Object.keys,s=Function.prototype.bind,b=function(a){return new m(a)};if(typeof exports!=="undefined"){if(typeof module!=="undefined"&&module.exports)exports=module.exports=b;exports._=b}else r._=b;b.VERSION="1.3.1";var j=b.each=
-b.forEach=function(a,c,d){if(a!=null)if(w&&a.forEach===w)a.forEach(c,d);else if(a.length===+a.length)for(var e=0,f=a.length;e<f;e++){if(e in a&&c.call(d,a[e],e,a)===n)break}else for(e in a)if(b.has(a,e)&&c.call(d,a[e],e,a)===n)break};b.map=b.collect=function(a,c,b){var e=[];if(a==null)return e;if(x&&a.map===x)return a.map(c,b);j(a,function(a,g,h){e[e.length]=c.call(b,a,g,h)});if(a.length===+a.length)e.length=a.length;return e};b.reduce=b.foldl=b.inject=function(a,c,d,e){var f=arguments.length>2;a==
-null&&(a=[]);if(y&&a.reduce===y)return e&&(c=b.bind(c,e)),f?a.reduce(c,d):a.reduce(c);j(a,function(a,b,i){f?d=c.call(e,d,a,b,i):(d=a,f=true)});if(!f)throw new TypeError("Reduce of empty array with no initial value");return d};b.reduceRight=b.foldr=function(a,c,d,e){var f=arguments.length>2;a==null&&(a=[]);if(z&&a.reduceRight===z)return e&&(c=b.bind(c,e)),f?a.reduceRight(c,d):a.reduceRight(c);var g=b.toArray(a).reverse();e&&!f&&(c=b.bind(c,e));return f?b.reduce(g,c,d,e):b.reduce(g,c)};b.find=b.detect=
-function(a,c,b){var e;E(a,function(a,g,h){if(c.call(b,a,g,h))return e=a,true});return e};b.filter=b.select=function(a,c,b){var e=[];if(a==null)return e;if(A&&a.filter===A)return a.filter(c,b);j(a,function(a,g,h){c.call(b,a,g,h)&&(e[e.length]=a)});return e};b.reject=function(a,c,b){var e=[];if(a==null)return e;j(a,function(a,g,h){c.call(b,a,g,h)||(e[e.length]=a)});return e};b.every=b.all=function(a,c,b){var e=true;if(a==null)return e;if(B&&a.every===B)return a.every(c,b);j(a,function(a,g,h){if(!(e=
-e&&c.call(b,a,g,h)))return n});return e};var E=b.some=b.any=function(a,c,d){c||(c=b.identity);var e=false;if(a==null)return e;if(C&&a.some===C)return a.some(c,d);j(a,function(a,b,h){if(e||(e=c.call(d,a,b,h)))return n});return!!e};b.include=b.contains=function(a,c){var b=false;if(a==null)return b;return p&&a.indexOf===p?a.indexOf(c)!=-1:b=E(a,function(a){return a===c})};b.invoke=function(a,c){var d=i.call(arguments,2);return b.map(a,function(a){return(b.isFunction(c)?c||a:a[c]).apply(a,d)})};b.pluck=
-function(a,c){return b.map(a,function(a){return a[c]})};b.max=function(a,c,d){if(!c&&b.isArray(a))return Math.max.apply(Math,a);if(!c&&b.isEmpty(a))return-Infinity;var e={computed:-Infinity};j(a,function(a,b,h){b=c?c.call(d,a,b,h):a;b>=e.computed&&(e={value:a,computed:b})});return e.value};b.min=function(a,c,d){if(!c&&b.isArray(a))return Math.min.apply(Math,a);if(!c&&b.isEmpty(a))return Infinity;var e={computed:Infinity};j(a,function(a,b,h){b=c?c.call(d,a,b,h):a;b<e.computed&&(e={value:a,computed:b})});
-return e.value};b.shuffle=function(a){var b=[],d;j(a,function(a,f){f==0?b[0]=a:(d=Math.floor(Math.random()*(f+1)),b[f]=b[d],b[d]=a)});return b};b.sortBy=function(a,c,d){return b.pluck(b.map(a,function(a,b,g){return{value:a,criteria:c.call(d,a,b,g)}}).sort(function(a,b){var c=a.criteria,d=b.criteria;return c<d?-1:c>d?1:0}),"value")};b.groupBy=function(a,c){var d={},e=b.isFunction(c)?c:function(a){return a[c]};j(a,function(a,b){var c=e(a,b);(d[c]||(d[c]=[])).push(a)});return d};b.sortedIndex=function(a,
-c,d){d||(d=b.identity);for(var e=0,f=a.length;e<f;){var g=e+f>>1;d(a[g])<d(c)?e=g+1:f=g}return e};b.toArray=function(a){return!a?[]:a.toArray?a.toArray():b.isArray(a)?i.call(a):b.isArguments(a)?i.call(a):b.values(a)};b.size=function(a){return b.toArray(a).length};b.first=b.head=function(a,b,d){return b!=null&&!d?i.call(a,0,b):a[0]};b.initial=function(a,b,d){return i.call(a,0,a.length-(b==null||d?1:b))};b.last=function(a,b,d){return b!=null&&!d?i.call(a,Math.max(a.length-b,0)):a[a.length-1]};b.rest=
-b.tail=function(a,b,d){return i.call(a,b==null||d?1:b)};b.compact=function(a){return b.filter(a,function(a){return!!a})};b.flatten=function(a,c){return b.reduce(a,function(a,e){if(b.isArray(e))return a.concat(c?e:b.flatten(e));a[a.length]=e;return a},[])};b.without=function(a){return b.difference(a,i.call(arguments,1))};b.uniq=b.unique=function(a,c,d){var d=d?b.map(a,d):a,e=[];b.reduce(d,function(d,g,h){if(0==h||(c===true?b.last(d)!=g:!b.include(d,g)))d[d.length]=g,e[e.length]=a[h];return d},[]);
-return e};b.union=function(){return b.uniq(b.flatten(arguments,true))};b.intersection=b.intersect=function(a){var c=i.call(arguments,1);return b.filter(b.uniq(a),function(a){return b.every(c,function(c){return b.indexOf(c,a)>=0})})};b.difference=function(a){var c=b.flatten(i.call(arguments,1));return b.filter(a,function(a){return!b.include(c,a)})};b.zip=function(){for(var a=i.call(arguments),c=b.max(b.pluck(a,"length")),d=Array(c),e=0;e<c;e++)d[e]=b.pluck(a,""+e);return d};b.indexOf=function(a,c,
-d){if(a==null)return-1;var e;if(d)return d=b.sortedIndex(a,c),a[d]===c?d:-1;if(p&&a.indexOf===p)return a.indexOf(c);for(d=0,e=a.length;d<e;d++)if(d in a&&a[d]===c)return d;return-1};b.lastIndexOf=function(a,b){if(a==null)return-1;if(D&&a.lastIndexOf===D)return a.lastIndexOf(b);for(var d=a.length;d--;)if(d in a&&a[d]===b)return d;return-1};b.range=function(a,b,d){arguments.length<=1&&(b=a||0,a=0);for(var d=arguments[2]||1,e=Math.max(Math.ceil((b-a)/d),0),f=0,g=Array(e);f<e;)g[f++]=a,a+=d;return g};
-var F=function(){};b.bind=function(a,c){var d,e;if(a.bind===s&&s)return s.apply(a,i.call(arguments,1));if(!b.isFunction(a))throw new TypeError;e=i.call(arguments,2);return d=function(){if(!(this instanceof d))return a.apply(c,e.concat(i.call(arguments)));F.prototype=a.prototype;var b=new F,g=a.apply(b,e.concat(i.call(arguments)));return Object(g)===g?g:b}};b.bindAll=function(a){var c=i.call(arguments,1);c.length==0&&(c=b.functions(a));j(c,function(c){a[c]=b.bind(a[c],a)});return a};b.memoize=function(a,
-c){var d={};c||(c=b.identity);return function(){var e=c.apply(this,arguments);return b.has(d,e)?d[e]:d[e]=a.apply(this,arguments)}};b.delay=function(a,b){var d=i.call(arguments,2);return setTimeout(function(){return a.apply(a,d)},b)};b.defer=function(a){return b.delay.apply(b,[a,1].concat(i.call(arguments,1)))};b.throttle=function(a,c){var d,e,f,g,h,i=b.debounce(function(){h=g=false},c);return function(){d=this;e=arguments;var b;f||(f=setTimeout(function(){f=null;h&&a.apply(d,e);i()},c));g?h=true:
-a.apply(d,e);i();g=true}};b.debounce=function(a,b){var d;return function(){var e=this,f=arguments;clearTimeout(d);d=setTimeout(function(){d=null;a.apply(e,f)},b)}};b.once=function(a){var b=false,d;return function(){if(b)return d;b=true;return d=a.apply(this,arguments)}};b.wrap=function(a,b){return function(){var d=[a].concat(i.call(arguments,0));return b.apply(this,d)}};b.compose=function(){var a=arguments;return function(){for(var b=arguments,d=a.length-1;d>=0;d--)b=[a[d].apply(this,b)];return b[0]}};
-b.after=function(a,b){return a<=0?b():function(){if(--a<1)return b.apply(this,arguments)}};b.keys=J||function(a){if(a!==Object(a))throw new TypeError("Invalid object");var c=[],d;for(d in a)b.has(a,d)&&(c[c.length]=d);return c};b.values=function(a){return b.map(a,b.identity)};b.functions=b.methods=function(a){var c=[],d;for(d in a)b.isFunction(a[d])&&c.push(d);return c.sort()};b.extend=function(a){j(i.call(arguments,1),function(b){for(var d in b)a[d]=b[d]});return a};b.defaults=function(a){j(i.call(arguments,
-1),function(b){for(var d in b)a[d]==null&&(a[d]=b[d])});return a};b.clone=function(a){return!b.isObject(a)?a:b.isArray(a)?a.slice():b.extend({},a)};b.tap=function(a,b){b(a);return a};b.isEqual=function(a,b){return q(a,b,[])};b.isEmpty=function(a){if(b.isArray(a)||b.isString(a))return a.length===0;for(var c in a)if(b.has(a,c))return false;return true};b.isElement=function(a){return!!(a&&a.nodeType==1)};b.isArray=o||function(a){return l.call(a)=="[object Array]"};b.isObject=function(a){return a===Object(a)};
-b.isArguments=function(a){return l.call(a)=="[object Arguments]"};if(!b.isArguments(arguments))b.isArguments=function(a){return!(!a||!b.has(a,"callee"))};b.isFunction=function(a){return l.call(a)=="[object Function]"};b.isString=function(a){return l.call(a)=="[object String]"};b.isNumber=function(a){return l.call(a)=="[object Number]"};b.isNaN=function(a){return a!==a};b.isBoolean=function(a){return a===true||a===false||l.call(a)=="[object Boolean]"};b.isDate=function(a){return l.call(a)=="[object Date]"};
-b.isRegExp=function(a){return l.call(a)=="[object RegExp]"};b.isNull=function(a){return a===null};b.isUndefined=function(a){return a===void 0};b.has=function(a,b){return I.call(a,b)};b.noConflict=function(){r._=G;return this};b.identity=function(a){return a};b.times=function(a,b,d){for(var e=0;e<a;e++)b.call(d,e)};b.escape=function(a){return(""+a).replace(/&/g,"&amp;").replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;").replace(/'/g,"&#x27;").replace(/\//g,"&#x2F;")};b.mixin=function(a){j(b.functions(a),
-function(c){K(c,b[c]=a[c])})};var L=0;b.uniqueId=function(a){var b=L++;return a?a+b:b};b.templateSettings={evaluate:/<%([\s\S]+?)%>/g,interpolate:/<%=([\s\S]+?)%>/g,escape:/<%-([\s\S]+?)%>/g};var t=/.^/,u=function(a){return a.replace(/\\\\/g,"\\").replace(/\\'/g,"'")};b.template=function(a,c){var d=b.templateSettings,d="var __p=[],print=function(){__p.push.apply(__p,arguments);};with(obj||{}){__p.push('"+a.replace(/\\/g,"\\\\").replace(/'/g,"\\'").replace(d.escape||t,function(a,b){return"',_.escape("+
-u(b)+"),'"}).replace(d.interpolate||t,function(a,b){return"',"+u(b)+",'"}).replace(d.evaluate||t,function(a,b){return"');"+u(b).replace(/[\r\n\t]/g," ")+";__p.push('"}).replace(/\r/g,"\\r").replace(/\n/g,"\\n").replace(/\t/g,"\\t")+"');}return __p.join('');",e=new Function("obj","_",d);return c?e(c,b):function(a){return e.call(this,a,b)}};b.chain=function(a){return b(a).chain()};var m=function(a){this._wrapped=a};b.prototype=m.prototype;var v=function(a,c){return c?b(a).chain():a},K=function(a,c){m.prototype[a]=
-function(){var a=i.call(arguments);H.call(a,this._wrapped);return v(c.apply(b,a),this._chain)}};b.mixin(b);j("pop,push,reverse,shift,sort,splice,unshift".split(","),function(a){var b=k[a];m.prototype[a]=function(){var d=this._wrapped;b.apply(d,arguments);var e=d.length;(a=="shift"||a=="splice")&&e===0&&delete d[0];return v(d,this._chain)}});j(["concat","join","slice"],function(a){var b=k[a];m.prototype[a]=function(){return v(b.apply(this._wrapped,arguments),this._chain)}});m.prototype.chain=function(){this._chain=
-true;return this};m.prototype.value=function(){return this._wrapped}}).call(this);

docs/_build/html/_static/websupport.js

@@ -1,808 +0,0 @@
-/*
- * websupport.js
- * ~~~~~~~~~~~~~
- *
- * sphinx.websupport utilities for all documentation.
- *
- * :copyright: Copyright 2007-2017 by the Sphinx team, see AUTHORS.
- * :license: BSD, see LICENSE for details.
- *
- */
-
-(function($) {
-  $.fn.autogrow = function() {
-    return this.each(function() {
-    var textarea = this;
-
-    $.fn.autogrow.resize(textarea);
-
-    $(textarea)
-      .focus(function() {
-        textarea.interval = setInterval(function() {
-          $.fn.autogrow.resize(textarea);
-        }, 500);
-      })
-      .blur(function() {
-        clearInterval(textarea.interval);
-      });
-    });
-  };
-
-  $.fn.autogrow.resize = function(textarea) {
-    var lineHeight = parseInt($(textarea).css('line-height'), 10);
-    var lines = textarea.value.split('\n');
-    var columns = textarea.cols;
-    var lineCount = 0;
-    $.each(lines, function() {
-      lineCount += Math.ceil(this.length / columns) || 1;
-    });
-    var height = lineHeight * (lineCount + 1);
-    $(textarea).css('height', height);
-  };
-})(jQuery);
-
-(function($) {
-  var comp, by;
-
-  function init() {
-    initEvents();
-    initComparator();
-  }
-
-  function initEvents() {
-    $(document).on("click", 'a.comment-close', function(event) {
-      event.preventDefault();
-      hide($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.vote', function(event) {
-      event.preventDefault();
-      handleVote($(this));
-    });
-    $(document).on("click", 'a.reply', function(event) {
-      event.preventDefault();
-      openReply($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.close-reply', function(event) {
-      event.preventDefault();
-      closeReply($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.sort-option', function(event) {
-      event.preventDefault();
-      handleReSort($(this));
-    });
-    $(document).on("click", 'a.show-proposal', function(event) {
-      event.preventDefault();
-      showProposal($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.hide-proposal', function(event) {
-      event.preventDefault();
-      hideProposal($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.show-propose-change', function(event) {
-      event.preventDefault();
-      showProposeChange($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.hide-propose-change', function(event) {
-      event.preventDefault();
-      hideProposeChange($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.accept-comment', function(event) {
-      event.preventDefault();
-      acceptComment($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.delete-comment', function(event) {
-      event.preventDefault();
-      deleteComment($(this).attr('id').substring(2));
-    });
-    $(document).on("click", 'a.comment-markup', function(event) {
-      event.preventDefault();
-      toggleCommentMarkupBox($(this).attr('id').substring(2));
-    });
-  }
-
-  /**
-   * Set comp, which is a comparator function used for sorting and
-   * inserting comments into the list.
-   */
-  function setComparator() {
-    // If the first three letters are "asc", sort in ascending order
-    // and remove the prefix.
-    if (by.substring(0,3) == 'asc') {
-      var i = by.substring(3);
-      comp = function(a, b) { return a[i] - b[i]; };
-    } else {
-      // Otherwise sort in descending order.
-      comp = function(a, b) { return b[by] - a[by]; };
-    }
-
-    // Reset link styles and format the selected sort option.
-    $('a.sel').attr('href', '#').removeClass('sel');
-    $('a.by' + by).removeAttr('href').addClass('sel');
-  }
-
-  /**
-   * Create a comp function. If the user has preferences stored in
-   * the sortBy cookie, use those, otherwise use the default.
-   */
-  function initComparator() {
-    by = 'rating'; // Default to sort by rating.
-    // If the sortBy cookie is set, use that instead.
-    if (document.cookie.length > 0) {
-      var start = document.cookie.indexOf('sortBy=');
-      if (start != -1) {
-        start = start + 7;
-        var end = document.cookie.indexOf(";", start);
-        if (end == -1) {
-          end = document.cookie.length;
-          by = unescape(document.cookie.substring(start, end));
-        }
-      }
-    }
-    setComparator();
-  }
-
-  /**
-   * Show a comment div.
-   */
-  function show(id) {
-    $('#ao' + id).hide();
-    $('#ah' + id).show();
-    var context = $.extend({id: id}, opts);
-    var popup = $(renderTemplate(popupTemplate, context)).hide();
-    popup.find('textarea[name="proposal"]').hide();
-    popup.find('a.by' + by).addClass('sel');
-    var form = popup.find('#cf' + id);
-    form.submit(function(event) {
-      event.preventDefault();
-      addComment(form);
-    });
-    $('#s' + id).after(popup);
-    popup.slideDown('fast', function() {
-      getComments(id);
-    });
-  }
-
-  /**
-   * Hide a comment div.
-   */
-  function hide(id) {
-    $('#ah' + id).hide();
-    $('#ao' + id).show();
-    var div = $('#sc' + id);
-    div.slideUp('fast', function() {
-      div.remove();
-    });
-  }
-
-  /**
-   * Perform an ajax request to get comments for a node
-   * and insert the comments into the comments tree.
-   */
-  function getComments(id) {
-    $.ajax({
-     type: 'GET',
-     url: opts.getCommentsURL,
-     data: {node: id},
-     success: function(data, textStatus, request) {
-       var ul = $('#cl' + id);
-       var speed = 100;
-       $('#cf' + id)
-         .find('textarea[name="proposal"]')
-         .data('source', data.source);
-
-       if (data.comments.length === 0) {
-         ul.html('<li>No comments yet.</li>');
-         ul.data('empty', true);
-       } else {
-         // If there are comments, sort them and put them in the list.
-         var comments = sortComments(data.comments);
-         speed = data.comments.length * 100;
-         appendComments(comments, ul);
-         ul.data('empty', false);
-       }
-       $('#cn' + id).slideUp(speed + 200);
-       ul.slideDown(speed);
-     },
-     error: function(request, textStatus, error) {
-       showError('Oops, there was a problem retrieving the comments.');
-     },
-     dataType: 'json'
-    });
-  }
-
-  /**
-   * Add a comment via ajax and insert the comment into the comment tree.
-   */
-  function addComment(form) {
-    var node_id = form.find('input[name="node"]').val();
-    var parent_id = form.find('input[name="parent"]').val();
-    var text = form.find('textarea[name="comment"]').val();
-    var proposal = form.find('textarea[name="proposal"]').val();
-
-    if (text == '') {
-      showError('Please enter a comment.');
-      return;
-    }
-
-    // Disable the form that is being submitted.
-    form.find('textarea,input').attr('disabled', 'disabled');
-
-    // Send the comment to the server.
-    $.ajax({
-      type: "POST",
-      url: opts.addCommentURL,
-      dataType: 'json',
-      data: {
-        node: node_id,
-        parent: parent_id,
-        text: text,
-        proposal: proposal
-      },
-      success: function(data, textStatus, error) {
-        // Reset the form.
-        if (node_id) {
-          hideProposeChange(node_id);
-        }
-        form.find('textarea')
-          .val('')
-          .add(form.find('input'))
-          .removeAttr('disabled');
-	var ul = $('#cl' + (node_id || parent_id));
-        if (ul.data('empty')) {
-          $(ul).empty();
-          ul.data('empty', false);
-        }
-        insertComment(data.comment);
-        var ao = $('#ao' + node_id);
-        ao.find('img').attr({'src': opts.commentBrightImage});
-        if (node_id) {
-          // if this was a "root" comment, remove the commenting box
-          // (the user can get it back by reopening the comment popup)
-          $('#ca' + node_id).slideUp();
-        }
-      },
-      error: function(request, textStatus, error) {
-        form.find('textarea,input').removeAttr('disabled');
-        showError('Oops, there was a problem adding the comment.');
-      }
-    });
-  }
-
-  /**
-   * Recursively append comments to the main comment list and children
-   * lists, creating the comment tree.
-   */
-  function appendComments(comments, ul) {
-    $.each(comments, function() {
-      var div = createCommentDiv(this);
-      ul.append($(document.createElement('li')).html(div));
-      appendComments(this.children, div.find('ul.comment-children'));
-      // To avoid stagnating data, don't store the comments children in data.
-      this.children = null;
-      div.data('comment', this);
-    });
-  }
-
-  /**
-   * After adding a new comment, it must be inserted in the correct
-   * location in the comment tree.
-   */
-  function insertComment(comment) {
-    var div = createCommentDiv(comment);
-
-    // To avoid stagnating data, don't store the comments children in data.
-    comment.children = null;
-    div.data('comment', comment);
-
-    var ul = $('#cl' + (comment.node || comment.parent));
-    var siblings = getChildren(ul);
-
-    var li = $(document.createElement('li'));
-    li.hide();
-
-    // Determine where in the parents children list to insert this comment.
-    for(i=0; i < siblings.length; i++) {
-      if (comp(comment, siblings[i]) <= 0) {
-        $('#cd' + siblings[i].id)
-          .parent()
-          .before(li.html(div));
-        li.slideDown('fast');
-        return;
-      }
-    }
-
-    // If we get here, this comment rates lower than all the others,
-    // or it is the only comment in the list.
-    ul.append(li.html(div));
-    li.slideDown('fast');
-  }
-
-  function acceptComment(id) {
-    $.ajax({
-      type: 'POST',
-      url: opts.acceptCommentURL,
-      data: {id: id},
-      success: function(data, textStatus, request) {
-        $('#cm' + id).fadeOut('fast');
-        $('#cd' + id).removeClass('moderate');
-      },
-      error: function(request, textStatus, error) {
-        showError('Oops, there was a problem accepting the comment.');
-      }
-    });
-  }
-
-  function deleteComment(id) {
-    $.ajax({
-      type: 'POST',
-      url: opts.deleteCommentURL,
-      data: {id: id},
-      success: function(data, textStatus, request) {
-        var div = $('#cd' + id);
-        if (data == 'delete') {
-          // Moderator mode: remove the comment and all children immediately
-          div.slideUp('fast', function() {
-            div.remove();
-          });
-          return;
-        }
-        // User mode: only mark the comment as deleted
-        div
-          .find('span.user-id:first')
-          .text('[deleted]').end()
-          .find('div.comment-text:first')
-          .text('[deleted]').end()
-          .find('#cm' + id + ', #dc' + id + ', #ac' + id + ', #rc' + id +
-                ', #sp' + id + ', #hp' + id + ', #cr' + id + ', #rl' + id)
-          .remove();
-        var comment = div.data('comment');
-        comment.username = '[deleted]';
-        comment.text = '[deleted]';
-        div.data('comment', comment);
-      },
-      error: function(request, textStatus, error) {
-        showError('Oops, there was a problem deleting the comment.');
-      }
-    });
-  }
-
-  function showProposal(id) {
-    $('#sp' + id).hide();
-    $('#hp' + id).show();
-    $('#pr' + id).slideDown('fast');
-  }
-
-  function hideProposal(id) {
-    $('#hp' + id).hide();
-    $('#sp' + id).show();
-    $('#pr' + id).slideUp('fast');
-  }
-
-  function showProposeChange(id) {
-    $('#pc' + id).hide();
-    $('#hc' + id).show();
-    var textarea = $('#pt' + id);
-    textarea.val(textarea.data('source'));
-    $.fn.autogrow.resize(textarea[0]);
-    textarea.slideDown('fast');
-  }
-
-  function hideProposeChange(id) {
-    $('#hc' + id).hide();
-    $('#pc' + id).show();
-    var textarea = $('#pt' + id);
-    textarea.val('').removeAttr('disabled');
-    textarea.slideUp('fast');
-  }
-
-  function toggleCommentMarkupBox(id) {
-    $('#mb' + id).toggle();
-  }
-
-  /** Handle when the user clicks on a sort by link. */
-  function handleReSort(link) {
-    var classes = link.attr('class').split(/\s+/);
-    for (var i=0; i<classes.length; i++) {
-      if (classes[i] != 'sort-option') {
-	by = classes[i].substring(2);
-      }
-    }
-    setComparator();
-    // Save/update the sortBy cookie.
-    var expiration = new Date();
-    expiration.setDate(expiration.getDate() + 365);
-    document.cookie= 'sortBy=' + escape(by) +
-                     ';expires=' + expiration.toUTCString();
-    $('ul.comment-ul').each(function(index, ul) {
-      var comments = getChildren($(ul), true);
-      comments = sortComments(comments);
-      appendComments(comments, $(ul).empty());
-    });
-  }
-
-  /**
-   * Function to process a vote when a user clicks an arrow.
-   */
-  function handleVote(link) {
-    if (!opts.voting) {
-      showError("You'll need to login to vote.");
-      return;
-    }
-
-    var id = link.attr('id');
-    if (!id) {
-      // Didn't click on one of the voting arrows.
-      return;
-    }
-    // If it is an unvote, the new vote value is 0,
-    // Otherwise it's 1 for an upvote, or -1 for a downvote.
-    var value = 0;
-    if (id.charAt(1) != 'u') {
-      value = id.charAt(0) == 'u' ? 1 : -1;
-    }
-    // The data to be sent to the server.
-    var d = {
-      comment_id: id.substring(2),
-      value: value
-    };
-
-    // Swap the vote and unvote links.
-    link.hide();
-    $('#' + id.charAt(0) + (id.charAt(1) == 'u' ? 'v' : 'u') + d.comment_id)
-      .show();
-
-    // The div the comment is displayed in.
-    var div = $('div#cd' + d.comment_id);
-    var data = div.data('comment');
-
-    // If this is not an unvote, and the other vote arrow has
-    // already been pressed, unpress it.
-    if ((d.value !== 0) && (data.vote === d.value * -1)) {
-      $('#' + (d.value == 1 ? 'd' : 'u') + 'u' + d.comment_id).hide();
-      $('#' + (d.value == 1 ? 'd' : 'u') + 'v' + d.comment_id).show();
-    }
-
-    // Update the comments rating in the local data.
-    data.rating += (data.vote === 0) ? d.value : (d.value - data.vote);
-    data.vote = d.value;
-    div.data('comment', data);
-
-    // Change the rating text.
-    div.find('.rating:first')
-      .text(data.rating + ' point' + (data.rating == 1 ? '' : 's'));
-
-    // Send the vote information to the server.
-    $.ajax({
-      type: "POST",
-      url: opts.processVoteURL,
-      data: d,
-      error: function(request, textStatus, error) {
-        showError('Oops, there was a problem casting that vote.');
-      }
-    });
-  }
-
-  /**
-   * Open a reply form used to reply to an existing comment.
-   */
-  function openReply(id) {
-    // Swap out the reply link for the hide link
-    $('#rl' + id).hide();
-    $('#cr' + id).show();
-
-    // Add the reply li to the children ul.
-    var div = $(renderTemplate(replyTemplate, {id: id})).hide();
-    $('#cl' + id)
-      .prepend(div)
-      // Setup the submit handler for the reply form.
-      .find('#rf' + id)
-      .submit(function(event) {
-        event.preventDefault();
-        addComment($('#rf' + id));
-        closeReply(id);
-      })
-      .find('input[type=button]')
-      .click(function() {
-        closeReply(id);
-      });
-    div.slideDown('fast', function() {
-      $('#rf' + id).find('textarea').focus();
-    });
-  }
-
-  /**
-   * Close the reply form opened with openReply.
-   */
-  function closeReply(id) {
-    // Remove the reply div from the DOM.
-    $('#rd' + id).slideUp('fast', function() {
-      $(this).remove();
-    });
-
-    // Swap out the hide link for the reply link
-    $('#cr' + id).hide();
-    $('#rl' + id).show();
-  }
-
-  /**
-   * Recursively sort a tree of comments using the comp comparator.
-   */
-  function sortComments(comments) {
-    comments.sort(comp);
-    $.each(comments, function() {
-      this.children = sortComments(this.children);
-    });
-    return comments;
-  }
-
-  /**
-   * Get the children comments from a ul. If recursive is true,
-   * recursively include childrens' children.
-   */
-  function getChildren(ul, recursive) {
-    var children = [];
-    ul.children().children("[id^='cd']")
-      .each(function() {
-        var comment = $(this).data('comment');
-        if (recursive)
-          comment.children = getChildren($(this).find('#cl' + comment.id), true);
-        children.push(comment);
-      });
-    return children;
-  }
-
-  /** Create a div to display a comment in. */
-  function createCommentDiv(comment) {
-    if (!comment.displayed && !opts.moderator) {
-      return $('<div class="moderate">Thank you!  Your comment will show up '
-               + 'once it is has been approved by a moderator.</div>');
-    }
-    // Prettify the comment rating.
-    comment.pretty_rating = comment.rating + ' point' +
-      (comment.rating == 1 ? '' : 's');
-    // Make a class (for displaying not yet moderated comments differently)
-    comment.css_class = comment.displayed ? '' : ' moderate';
-    // Create a div for this comment.
-    var context = $.extend({}, opts, comment);
-    var div = $(renderTemplate(commentTemplate, context));
-
-    // If the user has voted on this comment, highlight the correct arrow.
-    if (comment.vote) {
-      var direction = (comment.vote == 1) ? 'u' : 'd';
-      div.find('#' + direction + 'v' + comment.id).hide();
-      div.find('#' + direction + 'u' + comment.id).show();
-    }
-
-    if (opts.moderator || comment.text != '[deleted]') {
-      div.find('a.reply').show();
-      if (comment.proposal_diff)
-        div.find('#sp' + comment.id).show();
-      if (opts.moderator && !comment.displayed)
-        div.find('#cm' + comment.id).show();
-      if (opts.moderator || (opts.username == comment.username))
-        div.find('#dc' + comment.id).show();
-    }
-    return div;
-  }
-
-  /**
-   * A simple template renderer. Placeholders such as <%id%> are replaced
-   * by context['id'] with items being escaped. Placeholders such as <#id#>
-   * are not escaped.
-   */
-  function renderTemplate(template, context) {
-    var esc = $(document.createElement('div'));
-
-    function handle(ph, escape) {
-      var cur = context;
-      $.each(ph.split('.'), function() {
-        cur = cur[this];
-      });
-      return escape ? esc.text(cur || "").html() : cur;
-    }
-
-    return template.replace(/<([%#])([\w\.]*)\1>/g, function() {
-      return handle(arguments[2], arguments[1] == '%' ? true : false);
-    });
-  }
-
-  /** Flash an error message briefly. */
-  function showError(message) {
-    $(document.createElement('div')).attr({'class': 'popup-error'})
-      .append($(document.createElement('div'))
-               .attr({'class': 'error-message'}).text(message))
-      .appendTo('body')
-      .fadeIn("slow")
-      .delay(2000)
-      .fadeOut("slow");
-  }
-
-  /** Add a link the user uses to open the comments popup. */
-  $.fn.comment = function() {
-    return this.each(function() {
-      var id = $(this).attr('id').substring(1);
-      var count = COMMENT_METADATA[id];
-      var title = count + ' comment' + (count == 1 ? '' : 's');
-      var image = count > 0 ? opts.commentBrightImage : opts.commentImage;
-      var addcls = count == 0 ? ' nocomment' : '';
-      $(this)
-        .append(
-          $(document.createElement('a')).attr({
-            href: '#',
-            'class': 'sphinx-comment-open' + addcls,
-            id: 'ao' + id
-          })
-            .append($(document.createElement('img')).attr({
-              src: image,
-              alt: 'comment',
-              title: title
-            }))
-            .click(function(event) {
-              event.preventDefault();
-              show($(this).attr('id').substring(2));
-            })
-        )
-        .append(
-          $(document.createElement('a')).attr({
-            href: '#',
-            'class': 'sphinx-comment-close hidden',
-            id: 'ah' + id
-          })
-            .append($(document.createElement('img')).attr({
-              src: opts.closeCommentImage,
-              alt: 'close',
-              title: 'close'
-            }))
-            .click(function(event) {
-              event.preventDefault();
-              hide($(this).attr('id').substring(2));
-            })
-        );
-    });
-  };
-
-  var opts = {
-    processVoteURL: '/_process_vote',
-    addCommentURL: '/_add_comment',
-    getCommentsURL: '/_get_comments',
-    acceptCommentURL: '/_accept_comment',
-    deleteCommentURL: '/_delete_comment',
-    commentImage: '/static/_static/comment.png',
-    closeCommentImage: '/static/_static/comment-close.png',
-    loadingImage: '/static/_static/ajax-loader.gif',
-    commentBrightImage: '/static/_static/comment-bright.png',
-    upArrow: '/static/_static/up.png',
-    downArrow: '/static/_static/down.png',
-    upArrowPressed: '/static/_static/up-pressed.png',
-    downArrowPressed: '/static/_static/down-pressed.png',
-    voting: false,
-    moderator: false
-  };
-
-  if (typeof COMMENT_OPTIONS != "undefined") {
-    opts = jQuery.extend(opts, COMMENT_OPTIONS);
-  }
-
-  var popupTemplate = '\
-    <div class="sphinx-comments" id="sc<%id%>">\
-      <p class="sort-options">\
-        Sort by:\
-        <a href="#" class="sort-option byrating">best rated</a>\
-        <a href="#" class="sort-option byascage">newest</a>\
-        <a href="#" class="sort-option byage">oldest</a>\
-      </p>\
-      <div class="comment-header">Comments</div>\
-      <div class="comment-loading" id="cn<%id%>">\
-        loading comments... <img src="<%loadingImage%>" alt="" /></div>\
-      <ul id="cl<%id%>" class="comment-ul"></ul>\
-      <div id="ca<%id%>">\
-      <p class="add-a-comment">Add a comment\
-        (<a href="#" class="comment-markup" id="ab<%id%>">markup</a>):</p>\
-      <div class="comment-markup-box" id="mb<%id%>">\
-        reStructured text markup: <i>*emph*</i>, <b>**strong**</b>, \
-        <code>``code``</code>, \
-        code blocks: <code>::</code> and an indented block after blank line</div>\
-      <form method="post" id="cf<%id%>" class="comment-form" action="">\
-        <textarea name="comment" cols="80"></textarea>\
-        <p class="propose-button">\
-          <a href="#" id="pc<%id%>" class="show-propose-change">\
-            Propose a change &#9657;\
-          </a>\
-          <a href="#" id="hc<%id%>" class="hide-propose-change">\
-            Propose a change &#9663;\
-          </a>\
-        </p>\
-        <textarea name="proposal" id="pt<%id%>" cols="80"\
-                  spellcheck="false"></textarea>\
-        <input type="submit" value="Add comment" />\
-        <input type="hidden" name="node" value="<%id%>" />\
-        <input type="hidden" name="parent" value="" />\
-      </form>\
-      </div>\
-    </div>';
-
-  var commentTemplate = '\
-    <div id="cd<%id%>" class="sphinx-comment<%css_class%>">\
-      <div class="vote">\
-        <div class="arrow">\
-          <a href="#" id="uv<%id%>" class="vote" title="vote up">\
-            <img src="<%upArrow%>" />\
-          </a>\
-          <a href="#" id="uu<%id%>" class="un vote" title="vote up">\
-            <img src="<%upArrowPressed%>" />\
-          </a>\
-        </div>\
-        <div class="arrow">\
-          <a href="#" id="dv<%id%>" class="vote" title="vote down">\
-            <img src="<%downArrow%>" id="da<%id%>" />\
-          </a>\
-          <a href="#" id="du<%id%>" class="un vote" title="vote down">\
-            <img src="<%downArrowPressed%>" />\
-          </a>\
-        </div>\
-      </div>\
-      <div class="comment-content">\
-        <p class="tagline comment">\
-          <span class="user-id"><%username%></span>\
-          <span class="rating"><%pretty_rating%></span>\
-          <span class="delta"><%time.delta%></span>\
-        </p>\
-        <div class="comment-text comment"><#text#></div>\
-        <p class="comment-opts comment">\
-          <a href="#" class="reply hidden" id="rl<%id%>">reply &#9657;</a>\
-          <a href="#" class="close-reply" id="cr<%id%>">reply &#9663;</a>\
-          <a href="#" id="sp<%id%>" class="show-proposal">proposal &#9657;</a>\
-          <a href="#" id="hp<%id%>" class="hide-proposal">proposal &#9663;</a>\
-          <a href="#" id="dc<%id%>" class="delete-comment hidden">delete</a>\
-          <span id="cm<%id%>" class="moderation hidden">\
-            <a href="#" id="ac<%id%>" class="accept-comment">accept</a>\
-          </span>\
-        </p>\
-        <pre class="proposal" id="pr<%id%>">\
-<#proposal_diff#>\
-        </pre>\
-          <ul class="comment-children" id="cl<%id%>"></ul>\
-        </div>\
-        <div class="clearleft"></div>\
-      </div>\
-    </div>';
-
-  var replyTemplate = '\
-    <li>\
-      <div class="reply-div" id="rd<%id%>">\
-        <form id="rf<%id%>">\
-          <textarea name="comment" cols="80"></textarea>\
-          <input type="submit" value="Add reply" />\
-          <input type="button" value="Cancel" />\
-          <input type="hidden" name="parent" value="<%id%>" />\
-          <input type="hidden" name="node" value="" />\
-        </form>\
-      </div>\
-    </li>';
-
-  $(document).ready(function() {
-    init();
-  });
-})(jQuery);
-
-$(document).ready(function() {
-  // add comment anchors for all paragraphs that are commentable
-  $('.sphinx-has-comment').comment();
-
-  // highlight search words in search results
-  $("div.context").each(function() {
-    var params = $.getQueryParameters();
-    var terms = (params.q) ? params.q[0].split(/\s+/) : [];
-    var result = $(this);
-    $.each(terms, function() {
-      result.highlightText(this.toLowerCase(), 'highlighted');
-    });
-  });
-
-  // directly open comment window if requested
-  var anchor = document.location.hash;
-  if (anchor.substring(0, 9) == '#comment-') {
-    $('#ao' + anchor.substring(9)).click();
-    document.location.hash = '#s' + anchor.substring(9);
-  }
-});

docs/_build/html/bugreport.html

@@ -1,136 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Reporting Bugs &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    './',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="_static/jquery.js"></script>
-    <script type="text/javascript" src="_static/underscore.js"></script>
-    <script type="text/javascript" src="_static/doctools.js"></script>
-    <link rel="index" title="Index" href="genindex.html" />
-    <link rel="search" title="Search" href="search.html" />
-    <link rel="prev" title="Transformation Functions" href="ref/transforms.html" />
-   
-  <link rel="stylesheet" href="_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="reporting-bugs">
-<h1>Reporting Bugs<a class="headerlink" href="#reporting-bugs" title="Permalink to this headline">¶</a></h1>
-<p>pgloader is a software and as such contains bugs. Most bugs are easy to
-solve and taken care of in a short delay. For this to be possible though,
-bug reports need to follow those recommandations:</p>
-<blockquote>
-<div><ul class="simple">
-<li>include pgloader version,</li>
-<li>include problematic input and output,</li>
-<li>include a description of the output you expected,</li>
-<li>explain the difference between the ouput you have and the one you expected,</li>
-<li>include a self-reproducing test-case</li>
-</ul>
-</div></blockquote>
-<div class="section" id="test-cases-to-reproduce-bugs">
-<h2>Test Cases to Reproduce Bugs<a class="headerlink" href="#test-cases-to-reproduce-bugs" title="Permalink to this headline">¶</a></h2>
-<p>Use the <em>inline</em> source type to help reproduce a bug, as in the pgloader tests:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD CSV
-   FROM INLINE
-   INTO postgresql://dim@localhost/pgloader?public.&quot;HS&quot;
-
-   WITH truncate,
-        fields terminated by &#39;\t&#39;,
-        fields not enclosed,
-        fields escaped by backslash-quote,
-        quote identifiers
-
-    SET work_mem to &#39;128MB&#39;,
-        standard_conforming_strings to &#39;on&#39;,
-        application_name to &#39;my app name&#39;
-
- BEFORE LOAD DO
-  $$ create extension if not exists hstore; $$,
-  $$ drop table if exists &quot;HS&quot;; $$,
-  $$ CREATE TABLE &quot;HS&quot;
-     (
-        id     serial primary key,
-        kv     hstore
-     )
-  $$;
-
-
-1     email=&gt;foo@example.com,a=&gt;b
-2     test=&gt;value
-3     a=&gt;b,c=&gt;&quot;quoted hstore value&quot;,d=&gt;other
-4     baddata
-</pre></div>
-</div>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="ref/transforms.html" title="previous chapter">Transformation Functions</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="_sources/bugreport.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/genindex.html

@@ -1,88 +0,0 @@
-
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Index &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    './',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="_static/jquery.js"></script>
-    <script type="text/javascript" src="_static/underscore.js"></script>
-    <script type="text/javascript" src="_static/doctools.js"></script>
-    <link rel="index" title="Index" href="#" />
-    <link rel="search" title="Search" href="search.html" />
-   
-  <link rel="stylesheet" href="_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-
-<h1 id="index">Index</h1>
-
-<div class="genindex-jumpbox">
- 
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/index.html

@@ -1,198 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Welcome to pgloader’s documentation! &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    './',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="_static/jquery.js"></script>
-    <script type="text/javascript" src="_static/underscore.js"></script>
-    <script type="text/javascript" src="_static/doctools.js"></script>
-    <link rel="index" title="Index" href="genindex.html" />
-    <link rel="search" title="Search" href="search.html" />
-    <link rel="next" title="Introduction" href="intro.html" />
-   
-  <link rel="stylesheet" href="_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="welcome-to-pgloader-s-documentation">
-<h1>Welcome to pgloader’s documentation!<a class="headerlink" href="#welcome-to-pgloader-s-documentation" title="Permalink to this headline">¶</a></h1>
-<div class="toctree-wrapper compound">
-<p class="caption"><span class="caption-text">Table Of Contents:</span></p>
-<ul>
-<li class="toctree-l1"><a class="reference internal" href="intro.html">Introduction</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="intro.html#continuous-migration">Continuous Migration</a></li>
-<li class="toctree-l2"><a class="reference internal" href="intro.html#commands">Commands</a></li>
-<li class="toctree-l2"><a class="reference internal" href="intro.html#command-line">Command Line</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="tutorial/tutorial.html">PgLoader Tutorial</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="tutorial/tutorial.html#pgloader-quick-start">PgLoader Quick Start</a></li>
-<li class="toctree-l2"><a class="reference internal" href="tutorial/tutorial.html#loading-csv-data-with-pgloader">Loading CSV Data with pgloader</a></li>
-<li class="toctree-l2"><a class="reference internal" href="tutorial/tutorial.html#loading-fixed-width-data-file-with-pgloader">Loading Fixed Width Data File with pgloader</a></li>
-<li class="toctree-l2"><a class="reference internal" href="tutorial/tutorial.html#loading-maxmind-geolite-data-with-pgloader">Loading MaxMind Geolite Data with pgloader</a></li>
-<li class="toctree-l2"><a class="reference internal" href="tutorial/tutorial.html#loading-dbase-files-with-pgloader">Loading dBase files with pgloader</a></li>
-<li class="toctree-l2"><a class="reference internal" href="tutorial/tutorial.html#loading-sqlite-files-with-pgloader">Loading SQLite files with pgloader</a></li>
-<li class="toctree-l2"><a class="reference internal" href="tutorial/tutorial.html#migrating-from-mysql-to-postgresql">Migrating from MySQL to PostgreSQL</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="pgloader.html">PgLoader Reference Manual</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#arguments">Arguments</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#options">Options</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#batches-and-retry-behaviour">Batches And Retry Behaviour</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#a-note-about-performance">A Note About Performance</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#a-note-about-parallelism">A Note About Parallelism</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#source-formats">Source Formats</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#pgloader-commands-syntax">Pgloader Commands Syntax</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#templating-with-mustache">Templating with Mustache</a></li>
-<li class="toctree-l2"><a class="reference internal" href="pgloader.html#common-clauses">Common Clauses</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/csv.html">Loading CSV data</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/csv.html#csv-source-specification-from">CSV Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/csv.html#fields-specifications">Fields Specifications</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/csv.html#csv-loading-options-with">CSV Loading Options: WITH</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/fixed.html">Loading Fixed Cols File Formats</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/fixed.html#fixed-file-format-source-specification-from">Fixed File Format Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/fixed.html#fields-specifications">Fields Specifications</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/fixed.html#fixed-file-format-loading-options-with">Fixed File Format Loading Options: WITH</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/copy.html">Loading COPY Formatted Files</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/copy.html#copy-formatted-files-source-specification-from">COPY Formatted Files Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/copy.html#copy-formatted-file-options-with">COPY Formatted File Options: WITH</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/dbf.html">Loading DBF data</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/dbf.html#dbf-source-specification-from">DBF Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/dbf.html#dbf-loading-options-with">DBF Loading Options: WITH</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/ixf.html">Loading IXF Data</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/ixf.html#ixf-source-specification-from">IXF Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/ixf.html#ixf-loading-options-with">IXF Loading Options: WITH</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/archive.html">Loading From an Archive</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/archive.html#archive-source-specification-from">Archive Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/archive.html#archive-sub-commands">Archive Sub Commands</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/archive.html#archive-final-sql-commands">Archive Final SQL Commands</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/mysql.html">Migrating a MySQL Database to PostgreSQL</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-database-source-specification-from">MySQL Database Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-database-migration-options-with">MySQL Database Migration Options: WITH</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-database-casting-rules">MySQL Database Casting Rules</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-views-support">MySQL Views Support</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-partial-migration">MySQL Partial Migration</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-encoding-support">MySQL Encoding Support</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-schema-transformations">MySQL Schema Transformations</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#mysql-migration-limitations">MySQL Migration: limitations</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mysql.html#default-mysql-casting-rules">Default MySQL Casting Rules</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/sqlite.html">Migrating a SQLite database to PostgreSQL</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/sqlite.html#sqlite-database-source-specification-from">SQLite Database Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/sqlite.html#sqlite-database-migration-options-with">SQLite Database Migration Options: WITH</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/sqlite.html#sqlite-database-casting-rules">SQLite Database Casting Rules</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/sqlite.html#sqlite-database-partial-migrations">SQlite Database Partial Migrations</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/sqlite.html#default-sqlite-casting-rules">Default SQLite Casting Rules</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/mssql.html">Migrating a MS SQL Database to PostgreSQL</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="ref/mssql.html#ms-sql-database-source-specification-from">MS SQL Database Source Specification: FROM</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mssql.html#ms-sql-database-migration-options-with">MS SQL Database Migration Options: WITH</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mssql.html#ms-sql-database-casting-rules">MS SQL Database Casting Rules</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mssql.html#ms-sql-partial-migration">MS SQL Partial Migration</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mssql.html#ms-sql-schema-transformations">MS SQL Schema Transformations</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mssql.html#ms-sql-driver-setup-and-encoding">MS SQL Driver setup and encoding</a></li>
-<li class="toctree-l2"><a class="reference internal" href="ref/mssql.html#default-ms-sql-casting-rules">Default MS SQL Casting Rules</a></li>
-</ul>
-</li>
-<li class="toctree-l1"><a class="reference internal" href="ref/transforms.html">Transformation Functions</a></li>
-<li class="toctree-l1"><a class="reference internal" href="bugreport.html">Reporting Bugs</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="bugreport.html#test-cases-to-reproduce-bugs">Test Cases to Reproduce Bugs</a></li>
-</ul>
-</li>
-</ul>
-</div>
-</div>
-<div class="section" id="indices-and-tables">
-<h1>Indices and tables<a class="headerlink" href="#indices-and-tables" title="Permalink to this headline">¶</a></h1>
-<ul class="simple">
-<li><a class="reference internal" href="genindex.html"><span class="std std-ref">Index</span></a></li>
-<li><a class="reference internal" href="py-modindex.html"><span class="std std-ref">Module Index</span></a></li>
-<li><a class="reference internal" href="search.html"><span class="std std-ref">Search Page</span></a></li>
-</ul>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="#">Documentation overview</a><ul>
-      <li>Next: <a href="intro.html" title="next chapter">Introduction</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="_sources/index.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/intro.html

@@ -1,151 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Introduction &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    './',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="_static/jquery.js"></script>
-    <script type="text/javascript" src="_static/underscore.js"></script>
-    <script type="text/javascript" src="_static/doctools.js"></script>
-    <link rel="index" title="Index" href="genindex.html" />
-    <link rel="search" title="Search" href="search.html" />
-    <link rel="next" title="PgLoader Tutorial" href="tutorial/tutorial.html" />
-    <link rel="prev" title="Welcome to pgloader’s documentation!" href="index.html" />
-   
-  <link rel="stylesheet" href="_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="introduction">
-<h1>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h1>
-<p>pgloader loads data from various sources into PostgreSQL. It can
-transform the data it reads on the fly and submit raw SQL before and
-after the loading.  It uses the <cite>COPY</cite> PostgreSQL protocol to stream
-the data into the server, and manages errors by filling a pair of
-<em>reject.dat</em> and <em>reject.log</em> files.</p>
-<p>pgloader knows how to read data from different kind of sources:</p>
-<blockquote>
-<div><ul class="simple">
-<li>Files
-* CSV
-* Fixed Format
-* DBF</li>
-<li>Databases
-* SQLite
-* MySQL
-* MS SQL Server</li>
-</ul>
-</div></blockquote>
-<p>The level of automation provided by pgloader depends on the data source
-type. In the case of CSV and Fixed Format files, a full description of the
-expected input properties must be given to pgloader. In the case of a
-database, pgloader connects to the live service and knows how to fetch the
-metadata it needs directly from it.</p>
-<div class="section" id="continuous-migration">
-<h2>Continuous Migration<a class="headerlink" href="#continuous-migration" title="Permalink to this headline">¶</a></h2>
-<p>pgloader is meant to migrate a whole database in a single command line and
-without any manual intervention. The goal is to be able to setup a
-<em>Continuous Integration</em> environment as described in the <a class="reference external" href="http://mysqltopgsql.com/project/">Project
-Methodology</a> document of the <a class="reference external" href="http://mysqltopgsql.com/project/">MySQL to
-PostgreSQL</a> webpage.</p>
-<blockquote>
-<div><ol class="arabic simple">
-<li>Setup your target PostgreSQL Architecture</li>
-<li>Fork a Continuous Integration environment that uses PostgreSQL</li>
-<li>Migrate the data over and over again every night, from production</li>
-<li>As soon as the CI is all green using PostgreSQL, schedule the D-Day</li>
-<li>Migrate without suprise and enjoy!</li>
-</ol>
-</div></blockquote>
-<p>In order to be able to follow this great methodology, you need tooling to
-implement the third step in a fully automated way. That’s pgloader.</p>
-</div>
-<div class="section" id="commands">
-<h2>Commands<a class="headerlink" href="#commands" title="Permalink to this headline">¶</a></h2>
-<p>pgloader implements its own <em>Command Language</em>, a DSL that allows to specify
-every aspect of the data load and migration to implement. Some of the
-features provided in the language are only available for a specific source
-type.</p>
-</div>
-<div class="section" id="command-line">
-<h2>Command Line<a class="headerlink" href="#command-line" title="Permalink to this headline">¶</a></h2>
-<p>The pgloader command line accepts those two variants:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pgloader</span> <span class="p">[</span><span class="o">&lt;</span><span class="n">options</span><span class="o">&gt;</span><span class="p">]</span> <span class="p">[</span><span class="o">&lt;</span><span class="n">command</span><span class="o">-</span><span class="n">file</span><span class="o">&gt;</span><span class="p">]</span><span class="o">...</span>
-<span class="n">pgloader</span> <span class="p">[</span><span class="o">&lt;</span><span class="n">options</span><span class="o">&gt;</span><span class="p">]</span> <span class="n">SOURCE</span> <span class="n">TARGET</span>
-</pre></div>
-</div>
-<p>Either you have a <em>command-file</em> containing migration specifications in the
-pgloader <em>Command Language</em>, or you can give a <em>Source</em> for the data and a
-PostgreSQL database connection <em>Target</em> where to load the data into.</p>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="index.html" title="previous chapter">Welcome to pgloader’s documentation!</a></li>
-      <li>Next: <a href="tutorial/tutorial.html" title="next chapter">PgLoader Tutorial</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="_sources/intro.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/pgloader-usage-examples.html

@@ -1,237 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Pgloader Usage Examples &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    './',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="_static/jquery.js"></script>
-    <script type="text/javascript" src="_static/underscore.js"></script>
-    <script type="text/javascript" src="_static/doctools.js"></script>
-    <link rel="index" title="Index" href="genindex.html" />
-    <link rel="search" title="Search" href="search.html" />
-   
-  <link rel="stylesheet" href="_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="pgloader-usage-examples">
-<h1>Pgloader Usage Examples<a class="headerlink" href="#pgloader-usage-examples" title="Permalink to this headline">¶</a></h1>
-<p>Currently not included, because redundant with the tutorial.</p>
-<div class="section" id="usage-examples">
-<h2>Usage Examples<a class="headerlink" href="#usage-examples" title="Permalink to this headline">¶</a></h2>
-<p>Review the command line options and pgloader’s version:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pgloader</span> <span class="o">--</span><span class="n">help</span>
-<span class="n">pgloader</span> <span class="o">--</span><span class="n">version</span>
-</pre></div>
-</div>
-<div class="section" id="loading-from-a-complex-command">
-<h3>Loading from a complex command<a class="headerlink" href="#loading-from-a-complex-command" title="Permalink to this headline">¶</a></h3>
-<p>Use the command file as the pgloader command argument, pgloader will parse
-that file and execute the commands found in it:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pgloader</span> <span class="o">--</span><span class="n">verbose</span> <span class="o">./</span><span class="n">test</span><span class="o">/</span><span class="n">csv</span><span class="o">-</span><span class="n">districts</span><span class="o">.</span><span class="n">load</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="csv">
-<h3>CSV<a class="headerlink" href="#csv" title="Permalink to this headline">¶</a></h3>
-<p>Load data from a CSV file into a pre-existing table in your database, having
-pgloader guess the CSV properties (separator, quote and escape character):</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader ./test/data/matching-1.csv pgsql:///pgloader?tablename=matching
-</pre></div>
-</div>
-<p>Load data from a CSV file into a pre-existing table in your database, with
-expanded options:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                   \
-         --field id --field field                     \
-         --with truncate                              \
-         --with &quot;fields terminated by &#39;,&#39;&quot;            \
-         ./test/data/matching-1.csv                   \
-         postgres:///pgloader?tablename=matching
-</pre></div>
-</div>
-<p>In that example the whole loading is driven from the command line, bypassing
-the need for writing a command in the pgloader command syntax entirely. As
-there’s no command though, the extra inforamtion needed must be provided on
-the command line using the <cite>–type</cite> and <cite>–field</cite> and <cite>–with</cite> switches.</p>
-<p>For documentation about the available syntaxes for the <cite>–field</cite> and
-<cite>–with</cite> switches, please refer to the CSV section later in the man page.</p>
-<p>Note also that the PostgreSQL URI includes the target <em>tablename</em>.</p>
-</div>
-<div class="section" id="reading-from-stdin">
-<h3>Reading from STDIN<a class="headerlink" href="#reading-from-stdin" title="Permalink to this headline">¶</a></h3>
-<p>File based pgloader sources can be loaded from the standard input, as in the
-following example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                         \
-         --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot; \
-         --with &quot;skip header = 1&quot;                          \
-         --with &quot;fields terminated by &#39;\t&#39;&quot;                \
-         -                                                 \
-         postgresql:///pgloader?districts_longlat          \
-         &lt; test/data/2013_Gaz_113CDs_national.txt
-</pre></div>
-</div>
-<p>The dash (<cite>-</cite>) character as a source is used to mean <em>standard input</em>, as
-usual in Unix command lines. It’s possible to stream compressed content to
-pgloader with this technique, using the Unix pipe:</p>
-<blockquote>
-<div>gunzip -c source.gz | pgloader –type csv … - pgsql:///target?foo</div></blockquote>
-</div>
-<div class="section" id="loading-from-csv-available-through-http">
-<h3>Loading from CSV available through HTTP<a class="headerlink" href="#loading-from-csv-available-through-http" title="Permalink to this headline">¶</a></h3>
-<p>The same command as just above can also be run if the CSV file happens to be
-found on a remote HTTP location:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                                     \
-         --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot; \
-         --with &quot;skip header = 1&quot;                                       \
-         --with &quot;fields terminated by &#39;\t&#39;&quot;                             \
-         http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt     \
-         postgresql:///pgloader?districts_longlat
-</pre></div>
-</div>
-<p>Some more options have to be used in that case, as the file contains a
-one-line header (most commonly that’s column names, could be a copyright
-notice). Also, in that case, we specify all the fields right into a single
-<cite>–field</cite> option argument.</p>
-<p>Again, the PostgreSQL target connection string must contain the <em>tablename</em>
-option and you have to ensure that the target table exists and may fit the
-data. Here’s the SQL command used in that example in case you want to try it
-yourself:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">create</span> <span class="n">table</span> <span class="n">districts_longlat</span>
-<span class="p">(</span>
-         <span class="n">usps</span>        <span class="n">text</span><span class="p">,</span>
-         <span class="n">geoid</span>       <span class="n">text</span><span class="p">,</span>
-         <span class="n">aland</span>       <span class="n">bigint</span><span class="p">,</span>
-         <span class="n">awater</span>      <span class="n">bigint</span><span class="p">,</span>
-         <span class="n">aland_sqmi</span>  <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">awater_sqmi</span> <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">intptlat</span>    <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">intptlong</span>   <span class="n">double</span> <span class="n">precision</span>
-<span class="p">);</span>
-</pre></div>
-</div>
-<p>Also notice that the same command will work against an archived version of
-the same data, e.g.
-<a class="reference external" href="http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz">http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz</a>.</p>
-<p>Finally, it’s important to note that pgloader first fetches the content from
-the HTTP URL it to a local file, then expand the archive when it’s
-recognized to be one, and only then processes the locally expanded file.</p>
-<p>In some cases, either because pgloader has no direct support for your
-archive format or maybe because expanding the archive is not feasible in
-your environment, you might want to <em>stream</em> the content straight from its
-remote location into PostgreSQL. Here’s how to do that, using the old battle
-tested Unix Pipes trick:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>curl http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz \
-| gunzip -c                                                        \
-| pgloader --type csv                                              \
-           --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot;
-           --with &quot;skip header = 1&quot;                                \
-           --with &quot;fields terminated by &#39;\t&#39;&quot;                      \
-           -                                                       \
-           postgresql:///pgloader?districts_longlat
-</pre></div>
-</div>
-<p>Now the OS will take care of the streaming and buffering between the network
-and the commands and pgloader will take care of streaming the data down to
-PostgreSQL.</p>
-</div>
-<div class="section" id="migrating-from-sqlite">
-<h3>Migrating from SQLite<a class="headerlink" href="#migrating-from-sqlite" title="Permalink to this headline">¶</a></h3>
-<p>The following command will open the SQLite database, discover its tables
-definitions including indexes and foreign keys, migrate those definitions
-while <em>casting</em> the data type specifications to their PostgreSQL equivalent
-and then migrate the data over:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">newdb</span>
-<span class="n">pgloader</span> <span class="o">./</span><span class="n">test</span><span class="o">/</span><span class="n">sqlite</span><span class="o">/</span><span class="n">sqlite</span><span class="o">.</span><span class="n">db</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">newdb</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="migrating-from-mysql">
-<h3>Migrating from MySQL<a class="headerlink" href="#migrating-from-mysql" title="Permalink to this headline">¶</a></h3>
-<p>Just create a database where to host the MySQL data and definitions and have
-pgloader do the migration for you in a single command line:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">pagila</span>
-<span class="n">pgloader</span> <span class="n">mysql</span><span class="p">:</span><span class="o">//</span><span class="n">user</span><span class="nd">@localhost</span><span class="o">/</span><span class="n">sakila</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">pagila</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="fetching-an-archived-dbf-file-from-a-http-remote-location">
-<h3>Fetching an archived DBF file from a HTTP remote location<a class="headerlink" href="#fetching-an-archived-dbf-file-from-a-http-remote-location" title="Permalink to this headline">¶</a></h3>
-<p>It’s possible for pgloader to download a file from HTTP, unarchive it, and
-only then open it to discover the schema then load the data:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">foo</span>
-<span class="n">pgloader</span> <span class="o">--</span><span class="nb">type</span> <span class="n">dbf</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">insee</span><span class="o">.</span><span class="n">fr</span><span class="o">/</span><span class="n">fr</span><span class="o">/</span><span class="n">methodes</span><span class="o">/</span><span class="n">nomenclatures</span><span class="o">/</span><span class="n">cog</span><span class="o">/</span><span class="n">telechargement</span><span class="o">/</span><span class="mi">2013</span><span class="o">/</span><span class="n">dbf</span><span class="o">/</span><span class="n">historiq2013</span><span class="o">.</span><span class="n">zip</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">foo</span>
-</pre></div>
-</div>
-<p>Here it’s not possible for pgloader to guess the kind of data source it’s
-being given, so it’s necessary to use the <cite>–type</cite> command line switch.</p>
-</div>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="_sources/pgloader-usage-examples.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/archive.html

@@ -1,191 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading From an Archive &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Migrating a MySQL Database to PostgreSQL" href="mysql.html" />
-    <link rel="prev" title="Loading IXF Data" href="ixf.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-from-an-archive">
-<h1>Loading From an Archive<a class="headerlink" href="#loading-from-an-archive" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from one or more files contained
-in an archive. Currently the only supported archive format is <em>ZIP</em>, and the
-archive might be downloaded from an <em>HTTP</em> URL.</p>
-<p>Here’s an example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD ARCHIVE
-   FROM /Users/dim/Downloads/GeoLiteCity-latest.zip
-   INTO postgresql:///ip4r
-
-   BEFORE LOAD
-     DO $$ create extension if not exists ip4r; $$,
-        $$ create schema if not exists geolite; $$,
-
-     EXECUTE &#39;geolite.sql&#39;
-
-   LOAD CSV
-        FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
-             WITH ENCODING iso-8859-1
-             (
-                locId,
-                country,
-                region     null if blanks,
-                city       null if blanks,
-                postalCode null if blanks,
-                latitude,
-                longitude,
-                metroCode  null if blanks,
-                areaCode   null if blanks
-             )
-        INTO postgresql:///ip4r?geolite.location
-             (
-                locid,country,region,city,postalCode,
-                location point using (format nil &quot;(~a,~a)&quot; longitude latitude),
-                metroCode,areaCode
-             )
-        WITH skip header = 2,
-             fields optionally enclosed by &#39;&quot;&#39;,
-             fields escaped by double-quote,
-             fields terminated by &#39;,&#39;
-
-  AND LOAD CSV
-        FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
-             WITH ENCODING iso-8859-1
-             (
-                startIpNum, endIpNum, locId
-             )
-        INTO postgresql:///ip4r?geolite.blocks
-             (
-                iprange ip4r using (ip-range startIpNum endIpNum),
-                locId
-             )
-        WITH skip header = 2,
-             fields optionally enclosed by &#39;&quot;&#39;,
-             fields escaped by double-quote,
-             fields terminated by &#39;,&#39;
-
-   FINALLY DO
-     $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;
-</pre></div>
-</div>
-<p>The <cite>archive</cite> command accepts the following clauses and options.</p>
-<div class="section" id="archive-source-specification-from">
-<h2>Archive Source Specification: FROM<a class="headerlink" href="#archive-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Filename or HTTP URI where to load the data from. When given an HTTP URL the
-linked file will get downloaded locally before processing.</p>
-<p>If the file is a <cite>zip</cite> file, the command line utility <cite>unzip</cite> is used to
-expand the archive into files in <cite>$TMPDIR</cite>, or <cite>/tmp</cite> if <cite>$TMPDIR</cite> is unset
-or set to a non-existing directory.</p>
-<p>Then the following commands are used from the top level directory where the
-archive has been expanded.</p>
-</div>
-<div class="section" id="archive-sub-commands">
-<h2>Archive Sub Commands<a class="headerlink" href="#archive-sub-commands" title="Permalink to this headline">¶</a></h2>
-<blockquote>
-<div><ul>
-<li><p class="first">command [ <em>AND</em> command … ]</p>
-<p>A series of commands against the contents of the archive, at the moment
-only <cite>CSV</cite>,`’FIXED` and <cite>DBF</cite> commands are supported.</p>
-<p>Note that commands are supporting the clause <em>FROM FILENAME MATCHING</em>
-which allows the pgloader command not to depend on the exact names of
-the archive directories.</p>
-<p>The same clause can also be applied to several files with using the
-spelling <em>FROM ALL FILENAMES MATCHING</em> and a regular expression.</p>
-<p>The whole <em>matching</em> clause must follow the following rule:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">FROM</span> <span class="p">[</span> <span class="n">ALL</span> <span class="n">FILENAMES</span> <span class="o">|</span> <span class="p">[</span> <span class="n">FIRST</span> <span class="p">]</span> <span class="n">FILENAME</span> <span class="p">]</span> <span class="n">MATCHING</span>
-</pre></div>
-</div>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="archive-final-sql-commands">
-<h2>Archive Final SQL Commands<a class="headerlink" href="#archive-final-sql-commands" title="Permalink to this headline">¶</a></h2>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>FINALLY DO</em></p>
-<p>SQL Queries to run once the data is loaded, such as <cite>CREATE INDEX</cite>.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="ixf.html" title="previous chapter">Loading IXF Data</a></li>
-      <li>Next: <a href="mysql.html" title="next chapter">Migrating a MySQL Database to PostgreSQL</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/archive.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/copy.html

@@ -1,196 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading COPY Formatted Files &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Loading DBF data" href="dbf.html" />
-    <link rel="prev" title="Loading Fixed Cols File Formats" href="fixed.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-copy-formatted-files">
-<h1>Loading COPY Formatted Files<a class="headerlink" href="#loading-copy-formatted-files" title="Permalink to this headline">¶</a></h1>
-<p>This commands instructs pgloader to load from a file containing COPY TEXT
-data as described in the PostgreSQL documentation. Here’s an example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD COPY
-     FROM copy://./data/track.copy
-          (
-            trackid, track, album, media, genre, composer,
-            milliseconds, bytes, unitprice
-          )
-     INTO postgresql:///pgloader
-   TARGET TABLE track_full
-
-     WITH truncate
-
-      SET work_mem to &#39;14MB&#39;,
-          standard_conforming_strings to &#39;on&#39;
-
-BEFORE LOAD DO
-     $$ drop table if exists track_full; $$,
-     $$ create table track_full (
-          trackid      bigserial,
-          track        text,
-          album        text,
-          media        text,
-          genre        text,
-          composer     text,
-          milliseconds bigint,
-          bytes        bigint,
-          unitprice    numeric
-        );
-     $$;
-</pre></div>
-</div>
-<p>The <cite>COPY</cite> format command accepts the following clauses and options.</p>
-<div class="section" id="copy-formatted-files-source-specification-from">
-<h2>COPY Formatted Files Source Specification: FROM<a class="headerlink" href="#copy-formatted-files-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Filename where to load the data from. This support local files, HTTP URLs
-and zip files containing a single dbf file of the same name. Fetch such a
-zip file from an HTTP address is of course supported.</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>inline</em></p>
-<p>The data is found after the end of the parsed commands. Any number of
-empty lines between the end of the commands and the beginning of the
-data is accepted.</p>
-</li>
-<li><p class="first"><em>stdin</em></p>
-<p>Reads the data from the standard input stream.</p>
-</li>
-<li><p class="first"><em>FILENAMES MATCHING</em></p>
-<p>The whole <em>matching</em> clause must follow the following rule:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">[</span> <span class="n">ALL</span> <span class="n">FILENAMES</span> <span class="o">|</span> <span class="p">[</span> <span class="n">FIRST</span> <span class="p">]</span> <span class="n">FILENAME</span> <span class="p">]</span>
-<span class="n">MATCHING</span> <span class="n">regexp</span>
-<span class="p">[</span> <span class="n">IN</span> <span class="n">DIRECTORY</span> <span class="s1">&#39;...&#39;</span> <span class="p">]</span>
-</pre></div>
-</div>
-<p>The <em>matching</em> clause applies given <em>regular expression</em> (see above for
-exact syntax, several options can be used here) to filenames. It’s then
-possible to load data from only the first match of all of them.</p>
-<p>The optional <em>IN DIRECTORY</em> clause allows specifying which directory to
-walk for finding the data files, and can be either relative to where the
-command file is read from, or absolute. The given directory must exists.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="copy-formatted-file-options-with">
-<h2>COPY Formatted File Options: WITH<a class="headerlink" href="#copy-formatted-file-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>COPY</cite> file, the following options are supported:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>delimiter</em></p>
-<p>Takes a single character as argument, which must be found inside single
-quotes, and might be given as the printable character itself, the
-special value t to denote a tabulation character, or <cite>0x</cite> then an
-hexadecimal value read as the ASCII code for the character.</p>
-<p>This character is used as the <em>delimiter</em> when reading the data, in a
-similar way to the PostgreSQL <cite>COPY</cite> option.</p>
-</li>
-<li><p class="first"><em>null</em></p>
-<p>Takes a quoted string as an argument (quotes can be either double quotes
-or single quotes) and uses that string as the <cite>NULL</cite> representation in
-the data.</p>
-<p>This is similar to the <em>null</em> <cite>COPY</cite> option in PostgreSQL.</p>
-</li>
-<li><p class="first"><em>truncate</em></p>
-<p>When this option is listed, pgloader issues a <cite>TRUNCATE</cite> command against
-the PostgreSQL target table before reading the data file.</p>
-</li>
-<li><p class="first"><em>disable triggers</em></p>
-<p>When this option is listed, pgloader issues an <cite>ALTER TABLE … DISABLE
-TRIGGER ALL</cite> command against the PostgreSQL target table before copying
-the data, then the command <cite>ALTER TABLE … ENABLE TRIGGER ALL</cite> once the
-<cite>COPY</cite> is done.</p>
-<p>This option allows loading data into a pre-existing table ignoring the
-<em>foreign key constraints</em> and user defined triggers and may result in
-invalid <em>foreign key constraints</em> once the data is loaded. Use with
-care.</p>
-</li>
-<li><p class="first"><em>skip header</em></p>
-<p>Takes a numeric value as argument. Instruct pgloader to skip that many
-lines at the beginning of the input file.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="fixed.html" title="previous chapter">Loading Fixed Cols File Formats</a></li>
-      <li>Next: <a href="dbf.html" title="next chapter">Loading DBF data</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/copy.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/csv.html

@@ -1,316 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading CSV data &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Loading Fixed Cols File Formats" href="fixed.html" />
-    <link rel="prev" title="PgLoader Reference Manual" href="../pgloader.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-csv-data">
-<h1>Loading CSV data<a class="headerlink" href="#loading-csv-data" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from a <cite>CSV</cite> file. Here’s an
-example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">LOAD</span> <span class="n">CSV</span>
-   <span class="n">FROM</span> <span class="s1">&#39;GeoLiteCity-Blocks.csv&#39;</span> <span class="n">WITH</span> <span class="n">ENCODING</span> <span class="n">iso</span><span class="o">-</span><span class="mi">646</span><span class="o">-</span><span class="n">us</span>
-        <span class="n">HAVING</span> <span class="n">FIELDS</span>
-        <span class="p">(</span>
-           <span class="n">startIpNum</span><span class="p">,</span> <span class="n">endIpNum</span><span class="p">,</span> <span class="n">locId</span>
-        <span class="p">)</span>
-   <span class="n">INTO</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">//</span><span class="n">user</span><span class="nd">@localhost</span><span class="p">:</span><span class="mi">54393</span><span class="o">/</span><span class="n">dbname</span>
-        <span class="n">TARGET</span> <span class="n">TABLE</span> <span class="n">geolite</span><span class="o">.</span><span class="n">blocks</span>
-        <span class="n">TARGET</span> <span class="n">COLUMNS</span>
-        <span class="p">(</span>
-           <span class="n">iprange</span> <span class="n">ip4r</span> <span class="n">using</span> <span class="p">(</span><span class="n">ip</span><span class="o">-</span><span class="nb">range</span> <span class="n">startIpNum</span> <span class="n">endIpNum</span><span class="p">),</span>
-           <span class="n">locId</span>
-        <span class="p">)</span>
-   <span class="n">WITH</span> <span class="n">truncate</span><span class="p">,</span>
-        <span class="n">skip</span> <span class="n">header</span> <span class="o">=</span> <span class="mi">2</span><span class="p">,</span>
-        <span class="n">fields</span> <span class="n">optionally</span> <span class="n">enclosed</span> <span class="n">by</span> <span class="s1">&#39;&quot;&#39;</span><span class="p">,</span>
-        <span class="n">fields</span> <span class="n">escaped</span> <span class="n">by</span> <span class="n">backslash</span><span class="o">-</span><span class="n">quote</span><span class="p">,</span>
-        <span class="n">fields</span> <span class="n">terminated</span> <span class="n">by</span> <span class="s1">&#39;</span><span class="se">\t</span><span class="s1">&#39;</span>
-
-    <span class="n">SET</span> <span class="n">work_mem</span> <span class="n">to</span> <span class="s1">&#39;32 MB&#39;</span><span class="p">,</span> <span class="n">maintenance_work_mem</span> <span class="n">to</span> <span class="s1">&#39;64 MB&#39;</span><span class="p">;</span>
-</pre></div>
-</div>
-<p>The <cite>csv</cite> format command accepts the following clauses and options.</p>
-<div class="section" id="csv-source-specification-from">
-<h2>CSV Source Specification: FROM<a class="headerlink" href="#csv-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Filename where to load the data from. Accepts an <em>ENCODING</em> option. Use the
-<cite>–list-encodings</cite> option to know which encoding names are supported.</p>
-<p>The filename may be enclosed by single quotes, and could be one of the
-following special values:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>inline</em></p>
-<p>The data is found after the end of the parsed commands. Any number
-of empty lines between the end of the commands and the beginning of
-the data is accepted.</p>
-</li>
-<li><p class="first"><em>stdin</em></p>
-<p>Reads the data from the standard input stream.</p>
-</li>
-<li><p class="first"><em>FILENAMES MATCHING</em></p>
-<p>The whole <em>matching</em> clause must follow the following rule:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">[</span> <span class="n">ALL</span> <span class="n">FILENAMES</span> <span class="o">|</span> <span class="p">[</span> <span class="n">FIRST</span> <span class="p">]</span> <span class="n">FILENAME</span> <span class="p">]</span>
-<span class="n">MATCHING</span> <span class="n">regexp</span>
-<span class="p">[</span> <span class="n">IN</span> <span class="n">DIRECTORY</span> <span class="s1">&#39;...&#39;</span> <span class="p">]</span>
-</pre></div>
-</div>
-<p>The <em>matching</em> clause applies given <em>regular expression</em> (see above
-for exact syntax, several options can be used here) to filenames.
-It’s then possible to load data from only the first match of all of
-them.</p>
-<p>The optional <em>IN DIRECTORY</em> clause allows specifying which directory
-to walk for finding the data files, and can be either relative to
-where the command file is read from, or absolute. The given
-directory must exists.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="fields-specifications">
-<h2>Fields Specifications<a class="headerlink" href="#fields-specifications" title="Permalink to this headline">¶</a></h2>
-<p>The <em>FROM</em> option also supports an optional comma separated list of <em>field</em>
-names describing what is expected in the <cite>CSV</cite> data file, optionally
-introduced by the clause <cite>HAVING FIELDS</cite>.</p>
-<p>Each field name can be either only one name or a name following with
-specific reader options for that field, enclosed in square brackets and
-comma-separated. Supported per-field reader options are:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>terminated by</em></p>
-<blockquote>
-<div><p>See the description of <em>field terminated by</em> below.</p>
-<p>The processing of this option is not currently implemented.</p>
-</div></blockquote>
-</li>
-<li><p class="first"><em>date format</em></p>
-<p>When the field is expected of the date type, then this option allows
-to specify the date format used in the file.</p>
-<p>Date format string are template strings modeled against the
-PostgreSQL <cite>to_char</cite> template strings support, limited to the
-following patterns:</p>
-<blockquote>
-<div><ul class="simple">
-<li>YYYY, YYY, YY for the year part</li>
-<li>MM for the numeric month part</li>
-<li>DD for the numeric day part</li>
-<li>HH, HH12, HH24 for the hour part</li>
-<li>am, AM, a.m., A.M.</li>
-<li>pm, PM, p.m., P.M.</li>
-<li>MI for the minutes part</li>
-<li>SS for the seconds part</li>
-<li>MS for the milliseconds part (4 digits)</li>
-<li>US for the microseconds part (6 digits)</li>
-<li>unparsed punctuation signs: - . * # &#64; T / and space</li>
-</ul>
-</div></blockquote>
-<p>Here’s an example of a <em>date format</em> specification:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">column</span><span class="o">-</span><span class="n">name</span> <span class="p">[</span><span class="n">date</span> <span class="nb">format</span> <span class="s1">&#39;YYYY-MM-DD HH24-MI-SS.US&#39;</span><span class="p">]</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>null if</em></p>
-<blockquote>
-<div><p>This option takes an argument which is either the keyword <em>blanks</em>
-or a double-quoted string.</p>
-<p>When <em>blanks</em> is used and the field value that is read contains
-only space characters, then it’s automatically converted to an SQL
-<cite>NULL</cite> value.</p>
-<p>When a double-quoted string is used and that string is read as the
-field value, then the field value is automatically converted to an
-SQL <cite>NULL</cite> value.</p>
-</div></blockquote>
-</li>
-<li><p class="first"><em>trim both whitespace</em>, <em>trim left whitespace</em>, <em>trim right whitespace</em></p>
-<p>This option allows to trim whitespaces in the read data, either from
-both sides of the data, or only the whitespace characters found on
-the left of the streaing, or only those on the right of the string.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="csv-loading-options-with">
-<h2>CSV Loading Options: WITH<a class="headerlink" href="#csv-loading-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>CSV</cite> file, the following options are supported:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>truncate</em></p>
-<blockquote>
-<div><p>When this option is listed, pgloader issues a <cite>TRUNCATE</cite> command
-against the PostgreSQL target table before reading the data file.</p>
-</div></blockquote>
-</li>
-<li><p class="first"><em>drop indexes</em></p>
-<p>When this option is listed, pgloader issues <cite>DROP INDEX</cite> commands
-against all the indexes defined on the target table before copying
-the data, then <cite>CREATE INDEX</cite> commands once the <cite>COPY</cite> is done.</p>
-<p>In order to get the best performance possible, all the indexes are
-created in parallel and when done the primary keys are built again
-from the unique indexes just created. This two step process allows
-creating the primary key index in parallel with the other indexes,
-as only the <cite>ALTER TABLE</cite> command needs an <em>access exclusive lock</em>
-on the target table.</p>
-</li>
-<li><p class="first"><em>disable triggers</em></p>
-<p>When this option is listed, pgloader issues an <cite>ALTER TABLE …
-DISABLE TRIGGER ALL</cite> command against the PostgreSQL target table
-before copying the data, then the command <cite>ALTER TABLE … ENABLE
-TRIGGER ALL</cite> once the <cite>COPY</cite> is done.</p>
-<p>This option allows loading data into a pre-existing table ignoring
-the <em>foreign key constraints</em> and user defined triggers and may
-result in invalid <em>foreign key constraints</em> once the data is loaded.
-Use with care.</p>
-</li>
-<li><p class="first"><em>skip header</em></p>
-<p>Takes a numeric value as argument. Instruct pgloader to skip that
-many lines at the beginning of the input file.</p>
-</li>
-<li><p class="first"><em>csv header</em></p>
-<p>Use the first line read after <em>skip header</em> as the list of csv field
-names to be found in the CSV file, using the same CSV parameters as
-for the CSV data.</p>
-</li>
-<li><p class="first"><em>trim unquoted blanks</em></p>
-<p>When reading unquoted values in the <cite>CSV</cite> file, remove the blanks
-found in between the separator and the value. That behaviour is the
-default.</p>
-</li>
-<li><p class="first"><em>keep unquoted blanks</em></p>
-<p>When reading unquoted values in the <cite>CSV</cite> file, keep blanks found in
-between the separator and the value.</p>
-</li>
-<li><p class="first"><em>fields optionally enclosed by</em></p>
-<p>Takes a single character as argument, which must be found inside single
-quotes, and might be given as the printable character itself, the
-special value t to denote a tabulation character, the special value ‘
-to denote a single-quote, or <cite>0x</cite> then an hexadecimal value read as the
-ASCII code for the character.</p>
-<p>The following options specify the same enclosing character, a single quote:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">fields</span> <span class="n">optionally</span> <span class="n">enclosed</span> <span class="n">by</span> <span class="s1">&#39;</span><span class="se">\&#39;</span><span class="s1">&#39;</span>
-<span class="n">fields</span> <span class="n">optionally</span> <span class="n">enclosed</span> <span class="n">by</span> <span class="s1">&#39;0x27&#39;</span>
-</pre></div>
-</div>
-<p>This character is used as the quoting character in the <cite>CSV</cite> file,
-and defaults to double-quote.</p>
-</li>
-<li><p class="first"><em>fields not enclosed</em></p>
-<p>By default, pgloader will use the double-quote character as the
-enclosing character. If you have a CSV file where fields are not
-enclosed and are using double-quote as an expected ordinary
-character, then use the option <em>fields not enclosed</em> for the CSV
-parser to accept those values.</p>
-</li>
-<li><p class="first"><em>fields escaped by</em></p>
-<p>Takes either the special value <em>backslash-quote</em> or <em>double-quote</em>,
-or any value supported by the <em>fields terminated by</em> option (see
-below). This value is used to recognize escaped field separators
-when they are to be found within the data fields themselves.
-Defaults to <em>double-quote</em>.</p>
-</li>
-<li><p class="first"><em>csv escape mode</em></p>
-<p>Takes either the special value <em>quote</em> (the default) or <em>following</em>
-and allows the CSV parser to parse either only escaped field
-separator or any character (including CSV data) when using the
-<em>following</em> value.</p>
-</li>
-<li><p class="first"><em>fields terminated by</em></p>
-<p>Takes a single character as argument, which must be found inside
-single quotes, and might be given as the printable character itself,
-the special value t to denote a tabulation character, or <cite>0x</cite> then
-an hexadecimal value read as the ASCII code for the character.</p>
-<p>This character is used as the <em>field separator</em> when reading the
-<cite>CSV</cite> data.</p>
-</li>
-<li><p class="first"><em>lines terminated by</em></p>
-<p>Takes a single character as argument, which must be found inside
-single quotes, and might be given as the printable character itself,
-the special value t to denote a tabulation character, or <cite>0x</cite> then
-an hexadecimal value read as the ASCII code for the character.</p>
-<p>This character is used to recognize <em>end-of-line</em> condition when
-reading the <cite>CSV</cite> data.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="../pgloader.html" title="previous chapter">PgLoader Reference Manual</a></li>
-      <li>Next: <a href="fixed.html" title="next chapter">Loading Fixed Cols File Formats</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/csv.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/dbf.html

@@ -1,138 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading DBF data &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Loading IXF Data" href="ixf.html" />
-    <link rel="prev" title="Loading COPY Formatted Files" href="copy.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-dbf-data">
-<h1>Loading DBF data<a class="headerlink" href="#loading-dbf-data" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from a <cite>DBF</cite> file. Here’s an
-example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">LOAD</span> <span class="n">DBF</span>
-        <span class="n">FROM</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">insee</span><span class="o">.</span><span class="n">fr</span><span class="o">/</span><span class="n">fr</span><span class="o">/</span><span class="n">methodes</span><span class="o">/</span><span class="n">nomenclatures</span><span class="o">/</span><span class="n">cog</span><span class="o">/</span><span class="n">telechargement</span><span class="o">/</span><span class="mi">2013</span><span class="o">/</span><span class="n">dbf</span><span class="o">/</span><span class="n">reg2013</span><span class="o">.</span><span class="n">dbf</span>
-    <span class="n">INTO</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">//</span><span class="n">user</span><span class="nd">@localhost</span><span class="o">/</span><span class="n">dbname</span>
-    <span class="n">WITH</span> <span class="n">truncate</span><span class="p">,</span> <span class="n">create</span> <span class="n">table</span><span class="p">;</span>
-</pre></div>
-</div>
-<p>The <cite>dbf</cite> format command accepts the following clauses and options.</p>
-<div class="section" id="dbf-source-specification-from">
-<h2>DBF Source Specification: FROM<a class="headerlink" href="#dbf-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Filename where to load the data from. This support local files, HTTP URLs
-and zip files containing a single dbf file of the same name. Fetch such a
-zip file from an HTTP address is of course supported.</p>
-</div>
-<div class="section" id="dbf-loading-options-with">
-<h2>DBF Loading Options: WITH<a class="headerlink" href="#dbf-loading-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>DBF</cite> file, the following options are supported:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>truncate</em></p>
-<p>When this option is listed, pgloader issues a <cite>TRUNCATE</cite> command against
-the PostgreSQL target table before reading the data file.</p>
-</li>
-<li><p class="first"><em>disable triggers</em></p>
-<p>When this option is listed, pgloader issues an <cite>ALTER TABLE … DISABLE
-TRIGGER ALL</cite> command against the PostgreSQL target table before copying
-the data, then the command <cite>ALTER TABLE … ENABLE TRIGGER ALL</cite> once the
-<cite>COPY</cite> is done.</p>
-<p>This option allows loading data into a pre-existing table ignoring the
-<em>foreign key constraints</em> and user defined triggers and may result in
-invalid <em>foreign key constraints</em> once the data is loaded. Use with
-care.</p>
-</li>
-<li><p class="first"><em>create table</em></p>
-<p>When this option is listed, pgloader creates the table using the meta
-data found in the <cite>DBF</cite> file, which must contain a list of fields with
-their data type. A standard data type conversion from DBF to PostgreSQL
-is done.</p>
-</li>
-<li><p class="first"><em>table name</em></p>
-<p>This options expects as its value the possibly qualified name of the
-table to create.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="copy.html" title="previous chapter">Loading COPY Formatted Files</a></li>
-      <li>Next: <a href="ixf.html" title="next chapter">Loading IXF Data</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/dbf.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/fixed.html

@@ -1,270 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading Fixed Cols File Formats &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Loading COPY Formatted Files" href="copy.html" />
-    <link rel="prev" title="Loading CSV data" href="csv.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-fixed-cols-file-formats">
-<h1>Loading Fixed Cols File Formats<a class="headerlink" href="#loading-fixed-cols-file-formats" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from a text file containing
-columns arranged in a <em>fixed size</em> manner. Here’s an example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD FIXED
-     FROM inline
-          (
-           a from  0 for 10,
-           b from 10 for  8,
-           c from 18 for  8,
-           d from 26 for 17 [null if blanks, trim right whitespace]
-          )
-     INTO postgresql:///pgloader
-   TARGET TABLE fixed
-          (
-             a, b,
-             c time using (time-with-no-separator c),
-             d
-          )
-
-     WITH truncate
-
-      SET work_mem to &#39;14MB&#39;,
-          standard_conforming_strings to &#39;on&#39;
-
-BEFORE LOAD DO
-     $$ drop table if exists fixed; $$,
-     $$ create table fixed (
-         a integer,
-         b date,
-         c time,
-         d text
-        );
-     $$;
-
- 01234567892008052011431250firstline
-    01234562008052115182300left blank-padded
- 12345678902008052208231560another line
-  2345609872014092914371500
-  2345678902014092914371520
-</pre></div>
-</div>
-<p>The <cite>fixed</cite> format command accepts the following clauses and options.</p>
-<div class="section" id="fixed-file-format-source-specification-from">
-<h2>Fixed File Format Source Specification: FROM<a class="headerlink" href="#fixed-file-format-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Filename where to load the data from. Accepts an <em>ENCODING</em> option. Use the
-<cite>–list-encodings</cite> option to know which encoding names are supported.</p>
-<p>The filename may be enclosed by single quotes, and could be one of the
-following special values:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>inline</em></p>
-<blockquote>
-<div><p>The data is found after the end of the parsed commands. Any number
-of empty lines between the end of the commands and the beginning of
-the data is accepted.</p>
-</div></blockquote>
-</li>
-<li><p class="first"><em>stdin</em></p>
-<blockquote>
-<div><p>Reads the data from the standard input stream.</p>
-</div></blockquote>
-</li>
-<li><p class="first"><em>FILENAMES MATCHING</em></p>
-<p>The whole <em>matching</em> clause must follow the following rule:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">[</span> <span class="n">ALL</span> <span class="n">FILENAMES</span> <span class="o">|</span> <span class="p">[</span> <span class="n">FIRST</span> <span class="p">]</span> <span class="n">FILENAME</span> <span class="p">]</span>
-<span class="n">MATCHING</span> <span class="n">regexp</span>
-<span class="p">[</span> <span class="n">IN</span> <span class="n">DIRECTORY</span> <span class="s1">&#39;...&#39;</span> <span class="p">]</span>
-</pre></div>
-</div>
-<p>The <em>matching</em> clause applies given <em>regular expression</em> (see above
-for exact syntax, several options can be used here) to filenames.
-It’s then possible to load data from only the first match of all of
-them.</p>
-<p>The optional <em>IN DIRECTORY</em> clause allows specifying which directory
-to walk for finding the data files, and can be either relative to
-where the command file is read from, or absolute. The given
-directory must exists.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="fields-specifications">
-<h2>Fields Specifications<a class="headerlink" href="#fields-specifications" title="Permalink to this headline">¶</a></h2>
-<p>The <em>FROM</em> option also supports an optional comma separated list of <em>field</em>
-names describing what is expected in the <cite>FIXED</cite> data file.</p>
-<p>Each field name is composed of the field name followed with specific reader
-options for that field. Supported per-field reader options are the
-following, where only <em>start</em> and <em>length</em> are required.</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>start</em></p>
-<p>Position in the line where to start reading that field’s value. Can
-be entered with decimal digits or <cite>0x</cite> then hexadecimal digits.</p>
-</li>
-<li><p class="first"><em>length</em></p>
-<p>How many bytes to read from the <em>start</em> position to read that
-field’s value. Same format as <em>start</em>.</p>
-</li>
-</ul>
-</div></blockquote>
-<p>Those optional parameters must be enclosed in square brackets and
-comma-separated:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>terminated by</em></p>
-<blockquote>
-<div><p>See the description of <em>field terminated by</em> below.</p>
-<p>The processing of this option is not currently implemented.</p>
-</div></blockquote>
-</li>
-<li><p class="first"><em>date format</em></p>
-<p>When the field is expected of the date type, then this option allows
-to specify the date format used in the file.</p>
-<p>Date format string are template strings modeled against the
-PostgreSQL <cite>to_char</cite> template strings support, limited to the
-following patterns:</p>
-<blockquote>
-<div><ul class="simple">
-<li>YYYY, YYY, YY for the year part</li>
-<li>MM for the numeric month part</li>
-<li>DD for the numeric day part</li>
-<li>HH, HH12, HH24 for the hour part</li>
-<li>am, AM, a.m., A.M.</li>
-<li>pm, PM, p.m., P.M.</li>
-<li>MI for the minutes part</li>
-<li>SS for the seconds part</li>
-<li>MS for the milliseconds part (4 digits)</li>
-<li>US for the microseconds part (6 digits)</li>
-<li>unparsed punctuation signs: - . * # &#64; T / and space</li>
-</ul>
-</div></blockquote>
-<p>Here’s an example of a <em>date format</em> specification:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">column</span><span class="o">-</span><span class="n">name</span> <span class="p">[</span><span class="n">date</span> <span class="nb">format</span> <span class="s1">&#39;YYYY-MM-DD HH24-MI-SS.US&#39;</span><span class="p">]</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>null if</em></p>
-<p>This option takes an argument which is either the keyword <em>blanks</em>
-or a double-quoted string.</p>
-<p>When <em>blanks</em> is used and the field value that is read contains only
-space characters, then it’s automatically converted to an SQL <cite>NULL</cite>
-value.</p>
-<p>When a double-quoted string is used and that string is read as the
-field value, then the field value is automatically converted to an
-SQL <cite>NULL</cite> value.</p>
-</li>
-<li><p class="first"><em>trim both whitespace</em>, <em>trim left whitespace</em>, <em>trim right whitespace</em></p>
-<p>This option allows to trim whitespaces in the read data, either from
-both sides of the data, or only the whitespace characters found on
-the left of the streaing, or only those on the right of the string.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="fixed-file-format-loading-options-with">
-<h2>Fixed File Format Loading Options: WITH<a class="headerlink" href="#fixed-file-format-loading-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>FIXED</cite> file, the following options are supported:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>truncate</em></p>
-<p>When this option is listed, pgloader issues a <cite>TRUNCATE</cite> command
-against the PostgreSQL target table before reading the data file.</p>
-</li>
-<li><p class="first"><em>disable triggers</em></p>
-<p>When this option is listed, pgloader issues an <cite>ALTER TABLE …
-DISABLE TRIGGER ALL</cite> command against the PostgreSQL target table
-before copying the data, then the command <cite>ALTER TABLE … ENABLE
-TRIGGER ALL</cite> once the <cite>COPY</cite> is done.</p>
-<p>This option allows loading data into a pre-existing table ignoring
-the <em>foreign key constraints</em> and user defined triggers and may
-result in invalid <em>foreign key constraints</em> once the data is loaded.
-Use with care.</p>
-</li>
-<li><p class="first"><em>skip header</em></p>
-<p>Takes a numeric value as argument. Instruct pgloader to skip that
-many lines at the beginning of the input file.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="csv.html" title="previous chapter">Loading CSV data</a></li>
-      <li>Next: <a href="copy.html" title="next chapter">Loading COPY Formatted Files</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/fixed.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/ixf.html

@@ -1,149 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading IXF Data &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Loading From an Archive" href="archive.html" />
-    <link rel="prev" title="Loading DBF data" href="dbf.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-ixf-data">
-<h1>Loading IXF Data<a class="headerlink" href="#loading-ixf-data" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from an IBM <cite>IXF</cite> file. Here’s
-an example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD IXF
-    FROM data/nsitra.test1.ixf
-    INTO postgresql:///pgloader
-  TARGET TABLE nsitra.test1
-    WITH truncate, create table, timezone UTC
-
-  BEFORE LOAD DO
-   $$ create schema if not exists nsitra; $$,
-   $$ drop table if exists nsitra.test1; $$;
-</pre></div>
-</div>
-<p>The <cite>ixf</cite> format command accepts the following clauses and options.</p>
-<div class="section" id="ixf-source-specification-from">
-<h2>IXF Source Specification: FROM<a class="headerlink" href="#ixf-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Filename where to load the data from. This support local files, HTTP URLs
-and zip files containing a single ixf file of the same name. Fetch such a
-zip file from an HTTP address is of course supported.</p>
-</div>
-<div class="section" id="ixf-loading-options-with">
-<h2>IXF Loading Options: WITH<a class="headerlink" href="#ixf-loading-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>IXF</cite> file, the following options are supported:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>truncate</em></p>
-<p>When this option is listed, pgloader issues a <cite>TRUNCATE</cite> command against
-the PostgreSQL target table before reading the data file.</p>
-</li>
-<li><p class="first"><em>disable triggers</em></p>
-<p>When this option is listed, pgloader issues an <cite>ALTER TABLE … DISABLE
-TRIGGER ALL</cite> command against the PostgreSQL target table before copying
-the data, then the command <cite>ALTER TABLE … ENABLE TRIGGER ALL</cite> once the
-<cite>COPY</cite> is done.</p>
-<p>This option allows loading data into a pre-existing table ignoring the
-<em>foreign key constraints</em> and user defined triggers and may result in
-invalid <em>foreign key constraints</em> once the data is loaded. Use with
-care.</p>
-</li>
-<li><p class="first"><em>create table</em></p>
-<p>When this option is listed, pgloader creates the table using the meta
-data found in the <cite>DBF</cite> file, which must contain a list of fields with
-their data type. A standard data type conversion from DBF to PostgreSQL
-is done.</p>
-</li>
-<li><p class="first"><em>table name</em></p>
-<p>This options expects as its value the possibly qualified name of the
-table to create.</p>
-</li>
-<li><p class="first"><em>timezone</em></p>
-<p>This options allows to specify which timezone is used when parsing
-timestamps from an IXF file, and defaults to <em>UTC</em>. Expected values are
-either <cite>UTC</cite>, <cite>GMT</cite> or a single quoted location name such as
-<cite>‘Universal’</cite> or <cite>‘Europe/Paris’</cite>.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="dbf.html" title="previous chapter">Loading DBF data</a></li>
-      <li>Next: <a href="archive.html" title="next chapter">Loading From an Archive</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/ixf.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/mssql.html

@@ -1,236 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Migrating a MS SQL Database to PostgreSQL &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Transformation Functions" href="transforms.html" />
-    <link rel="prev" title="Migrating a SQLite database to PostgreSQL" href="sqlite.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="migrating-a-ms-sql-database-to-postgresql">
-<h1>Migrating a MS SQL Database to PostgreSQL<a class="headerlink" href="#migrating-a-ms-sql-database-to-postgresql" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from a MS SQL database.
-Automatic discovery of the schema is supported, including build of the
-indexes, primary and foreign keys constraints.</p>
-<p>Here’s an example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>load database
-     from mssql://user@host/dbname
-     into postgresql:///dbname
-
-including only table names like &#39;GlobalAccount&#39; in schema &#39;dbo&#39;
-
-set work_mem to &#39;16MB&#39;, maintenance_work_mem to &#39;512 MB&#39;
-
-before load do $$ drop schema if exists dbo cascade; $$;
-</pre></div>
-</div>
-<p>The <cite>mssql</cite> command accepts the following clauses and options.</p>
-<div class="section" id="ms-sql-database-source-specification-from">
-<h2>MS SQL Database Source Specification: FROM<a class="headerlink" href="#ms-sql-database-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Connection string to an existing MS SQL database server that listens and
-welcome external TCP/IP connection. As pgloader currently piggybacks on the
-FreeTDS driver, to change the port of the server please export the <cite>TDSPORT</cite>
-environment variable.</p>
-</div>
-<div class="section" id="ms-sql-database-migration-options-with">
-<h2>MS SQL Database Migration Options: WITH<a class="headerlink" href="#ms-sql-database-migration-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>MS SQL</cite> database, the same options as when loading a
-<cite>MySQL</cite> database are supported. Please refer to the MySQL section. The
-following options are added:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>create schemas</em></p>
-<p>When this option is listed, pgloader creates the same schemas as found
-on the MS SQL instance. This is the default.</p>
-</li>
-<li><p class="first"><em>create no schemas</em></p>
-<p>When this option is listed, pgloader refrains from creating any schemas
-at all, you must then ensure that the target schema do exist.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="ms-sql-database-casting-rules">
-<h2>MS SQL Database Casting Rules<a class="headerlink" href="#ms-sql-database-casting-rules" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="cast">
-<h3>CAST<a class="headerlink" href="#cast" title="Permalink to this headline">¶</a></h3>
-<p>The cast clause allows to specify custom casting rules, either to overload
-the default casting rules or to amend them with special cases.</p>
-<p>Please refer to the MySQL CAST clause for details.</p>
-</div>
-</div>
-<div class="section" id="ms-sql-partial-migration">
-<h2>MS SQL Partial Migration<a class="headerlink" href="#ms-sql-partial-migration" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="including-only-table-names-like">
-<h3>INCLUDING ONLY TABLE NAMES LIKE<a class="headerlink" href="#including-only-table-names-like" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table name patterns used to limit the
-tables to migrate to a sublist. More than one such clause may be used, they
-will be accumulated together.</p>
-<p>Example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">including</span> <span class="n">only</span> <span class="n">table</span> <span class="n">names</span> <span class="n">lile</span> <span class="s1">&#39;GlobalAccount&#39;</span> <span class="ow">in</span> <span class="n">schema</span> <span class="s1">&#39;dbo&#39;</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="excluding-table-names-like">
-<h3>EXCLUDING TABLE NAMES LIKE<a class="headerlink" href="#excluding-table-names-like" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table name patterns used to exclude
-table names from the migration. This filter only applies to the result of
-the <em>INCLUDING</em> filter.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">excluding</span> <span class="n">table</span> <span class="n">names</span> <span class="n">matching</span> <span class="s1">&#39;LocalAccount&#39;</span> <span class="ow">in</span> <span class="n">schema</span> <span class="s1">&#39;dbo&#39;</span>
-</pre></div>
-</div>
-</div>
-</div>
-<div class="section" id="ms-sql-schema-transformations">
-<h2>MS SQL Schema Transformations<a class="headerlink" href="#ms-sql-schema-transformations" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="alter-schema-rename-to">
-<h3>ALTER SCHEMA ‘…’ RENAME TO ‘…’<a class="headerlink" href="#alter-schema-rename-to" title="Permalink to this headline">¶</a></h3>
-<p>Allows to rename a schema on the flight, so that for instance the tables
-found in the schema ‘dbo’ in your source database will get migrated into the
-schema ‘public’ in the target database with this command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">alter</span> <span class="n">schema</span> <span class="s1">&#39;dbo&#39;</span> <span class="n">rename</span> <span class="n">to</span> <span class="s1">&#39;public&#39;</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="alter-table-names-matching-in-schema">
-<h3>ALTER TABLE NAMES MATCHING … IN SCHEMA ‘…’<a class="headerlink" href="#alter-table-names-matching-in-schema" title="Permalink to this headline">¶</a></h3>
-<p>See the MySQL explanation for this clause above. It works the same in the
-context of migrating from MS SQL, only with the added option to specify the
-name of the schema where to find the definition of the target tables.</p>
-<p>The matching is done in pgloader itself, with a Common Lisp regular
-expression lib, so doesn’t depend on the <em>LIKE</em> implementation of MS SQL,
-nor on the lack of support for regular expressions in the engine.</p>
-</div>
-</div>
-<div class="section" id="ms-sql-driver-setup-and-encoding">
-<h2>MS SQL Driver setup and encoding<a class="headerlink" href="#ms-sql-driver-setup-and-encoding" title="Permalink to this headline">¶</a></h2>
-<p>pgloader is using the <cite>FreeTDS</cite> driver, and internally expects the data to
-be sent in utf-8. To achieve that, you can configure the FreeTDS driver with
-those defaults, in the file <cite>~/.freetds.conf</cite>:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">[</span><span class="k">global</span><span class="p">]</span>
-    <span class="n">tds</span> <span class="n">version</span> <span class="o">=</span> <span class="mf">7.4</span>
-    <span class="n">client</span> <span class="n">charset</span> <span class="o">=</span> <span class="n">UTF</span><span class="o">-</span><span class="mi">8</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="default-ms-sql-casting-rules">
-<h2>Default MS SQL Casting Rules<a class="headerlink" href="#default-ms-sql-casting-rules" title="Permalink to this headline">¶</a></h2>
-<p>When migrating from MS SQL the following Casting Rules are provided:</p>
-<p>Numbers:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">tinyint</span> <span class="n">to</span> <span class="n">smallint</span>
-
-<span class="nb">type</span> <span class="nb">float</span> <span class="n">to</span> <span class="nb">float</span>   <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">real</span> <span class="n">to</span> <span class="n">real</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">double</span> <span class="n">to</span> <span class="n">double</span> <span class="n">precision</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">numeric</span> <span class="n">to</span> <span class="n">numeric</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">decimal</span> <span class="n">to</span> <span class="n">numeric</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">money</span> <span class="n">to</span> <span class="n">numeric</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">smallmoney</span> <span class="n">to</span> <span class="n">numeric</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-</pre></div>
-</div>
-<p>Texts:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">char</span>      <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">nchat</span>     <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">varchar</span>   <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">nvarchar</span>  <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">xml</span>       <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-</pre></div>
-</div>
-<p>Binary:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">binary</span>    <span class="n">to</span> <span class="n">bytea</span> <span class="n">using</span> <span class="n">byte</span><span class="o">-</span><span class="n">vector</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">bytea</span>
-<span class="nb">type</span> <span class="n">varbinary</span> <span class="n">to</span> <span class="n">bytea</span> <span class="n">using</span> <span class="n">byte</span><span class="o">-</span><span class="n">vector</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">bytea</span>
-</pre></div>
-</div>
-<p>Date:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">datetime</span>    <span class="n">to</span> <span class="n">timestamptz</span>
-<span class="nb">type</span> <span class="n">datetime2</span>   <span class="n">to</span> <span class="n">timestamptz</span>
-</pre></div>
-</div>
-<p>Others:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">bit</span> <span class="n">to</span> <span class="n">boolean</span>
-<span class="nb">type</span> <span class="n">hierarchyid</span> <span class="n">to</span> <span class="n">bytea</span>
-<span class="nb">type</span> <span class="n">geography</span> <span class="n">to</span> <span class="n">bytea</span>
-<span class="nb">type</span> <span class="n">uniqueidentifier</span> <span class="n">to</span> <span class="n">uuid</span> <span class="n">using</span> <span class="n">sql</span><span class="o">-</span><span class="n">server</span><span class="o">-</span><span class="n">uniqueidentifier</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">uuid</span>
-</pre></div>
-</div>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="sqlite.html" title="previous chapter">Migrating a SQLite database to PostgreSQL</a></li>
-      <li>Next: <a href="transforms.html" title="next chapter">Transformation Functions</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/mssql.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/mysql.html

@@ -1,639 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Migrating a MySQL Database to PostgreSQL &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Migrating a SQLite database to PostgreSQL" href="sqlite.html" />
-    <link rel="prev" title="Loading From an Archive" href="archive.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="migrating-a-mysql-database-to-postgresql">
-<h1>Migrating a MySQL Database to PostgreSQL<a class="headerlink" href="#migrating-a-mysql-database-to-postgresql" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from a database connection. The
-only supported database source is currently <em>MySQL</em>, and pgloader supports
-dynamically converting the schema of the source database and the indexes
-building.</p>
-<p>A default set of casting rules are provided and might be overloaded and
-appended to by the command.</p>
-<p>Here’s an example using as many options as possible, some of them even being
-defaults. Chances are you don’t need that complex a setup, don’t copy and
-paste it, use it only as a reference!</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD DATABASE
-     FROM      mysql://root@localhost/sakila
-     INTO postgresql://localhost:54393/sakila
-
- WITH include drop, create tables, create indexes, reset sequences,
-      workers = 8, concurrency = 1,
-      multiple readers per thread, rows per range = 50000
-
-  SET PostgreSQL PARAMETERS
-      maintenance_work_mem to &#39;128MB&#39;,
-      work_mem to &#39;12MB&#39;,
-      search_path to &#39;sakila, public, &quot;$user&quot;&#39;
-
-  SET MySQL PARAMETERS
-      net_read_timeout  = &#39;120&#39;,
-      net_write_timeout = &#39;120&#39;
-
- CAST type bigint when (= precision 20) to bigserial drop typemod,
-      type date drop not null drop default using zero-dates-to-null,
-      -- type tinyint to boolean using tinyint-to-boolean,
-      type year to integer
-
- MATERIALIZE VIEWS film_list, staff_list
-
- -- INCLUDING ONLY TABLE NAMES MATCHING ~/film/, &#39;actor&#39;
- -- EXCLUDING TABLE NAMES MATCHING ~&lt;ory&gt;
- -- DECODING TABLE NAMES MATCHING ~/messed/, ~/encoding/ AS utf8
- -- ALTER TABLE NAMES MATCHING &#39;film&#39; RENAME TO &#39;films&#39;
- -- ALTER TABLE NAMES MATCHING ~/_list$/ SET SCHEMA &#39;mv&#39;
-
- ALTER TABLE NAMES MATCHING ~/_list$/, &#39;sales_by_store&#39;, ~/sales_by/
-  SET SCHEMA &#39;mv&#39;
-
- ALTER TABLE NAMES MATCHING &#39;film&#39; RENAME TO &#39;films&#39;
- ALTER TABLE NAMES MATCHING ~/./ SET (fillfactor=&#39;40&#39;)
-
- ALTER SCHEMA &#39;sakila&#39; RENAME TO &#39;pagila&#39;
-
- BEFORE LOAD DO
-   $$ create schema if not exists pagila; $$,
-   $$ create schema if not exists mv;     $$,
-   $$ alter database sakila set search_path to pagila, mv, public; $$;
-</pre></div>
-</div>
-<p>The <cite>database</cite> command accepts the following clauses and options.</p>
-<div class="section" id="mysql-database-source-specification-from">
-<h2>MySQL Database Source Specification: FROM<a class="headerlink" href="#mysql-database-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Must be a connection URL pointing to a MySQL database.</p>
-<p>If the connection URI contains a table name, then only this table is
-migrated from MySQL to PostgreSQL.</p>
-<p>See the <cite>SOURCE CONNECTION STRING</cite> section above for details on how to write
-the connection string. The MySQL connection string accepts the same
-parameter <em>sslmode</em> as the PostgreSQL connection string, but the <em>verify</em>
-mode is not implemented (yet).</p>
-<p>Environment variables described in
-&lt;<a class="reference external" href="http://dev.mysql.com/doc/refman/5.0/en/environment-variables.html">http://dev.mysql.com/doc/refman/5.0/en/environment-variables.html</a>&gt; can be
-used as default values too. If the user is not provided, then it defaults to
-<cite>USER</cite> environment variable value. The password can be provided with the
-environment variable <cite>MYSQL_PWD</cite>. The host can be provided with the
-environment variable <cite>MYSQL_HOST</cite> and otherwise defaults to <cite>localhost</cite>. The
-port can be provided with the environment variable <cite>MYSQL_TCP_PORT</cite> and
-otherwise defaults to <cite>3306</cite>.</p>
-</div>
-<div class="section" id="mysql-database-migration-options-with">
-<h2>MySQL Database Migration Options: WITH<a class="headerlink" href="#mysql-database-migration-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>MySQL</cite> database, the following options are supported,
-and the default <em>WITH</em> clause is: <em>no truncate</em>, <em>create schema</em>, <em>create
-tables</em>, <em>include drop</em>, <em>create indexes</em>, <em>reset sequences</em>, <em>foreign
-keys</em>, <em>downcase identifiers</em>, <em>uniquify index names</em>.</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>include drop</em></p>
-<p>When this option is listed, pgloader drops all the tables in the target
-PostgreSQL database whose names appear in the MySQL database. This
-option allows for using the same command several times in a row until
-you figure out all the options, starting automatically from a clean
-environment. Please note that <cite>CASCADE</cite> is used to ensure that tables
-are dropped even if there are foreign keys pointing to them. This is
-precisely what <cite>include drop</cite> is intended to do: drop all target tables
-and recreate them.</p>
-<p>Great care needs to be taken when using <cite>include drop</cite>, as it will
-cascade to <em>all</em> objects referencing the target tables, possibly
-including other tables that are not being loaded from the source DB.</p>
-</li>
-<li><p class="first"><em>include no drop</em></p>
-<p>When this option is listed, pgloader will not include any <cite>DROP</cite>
-statement when loading the data.</p>
-</li>
-<li><p class="first"><em>truncate</em></p>
-<p>When this option is listed, pgloader issue the <cite>TRUNCATE</cite> command
-against each PostgreSQL table just before loading data into it.</p>
-</li>
-<li><p class="first"><em>no truncate</em></p>
-<p>When this option is listed, pgloader issues no <cite>TRUNCATE</cite> command.</p>
-</li>
-<li><p class="first"><em>disable triggers</em></p>
-<p>When this option is listed, pgloader issues an <cite>ALTER TABLE … DISABLE
-TRIGGER ALL</cite> command against the PostgreSQL target table before copying
-the data, then the command <cite>ALTER TABLE … ENABLE TRIGGER ALL</cite> once the
-<cite>COPY</cite> is done.</p>
-<p>This option allows loading data into a pre-existing table ignoring the
-<em>foreign key constraints</em> and user defined triggers and may result in
-invalid <em>foreign key constraints</em> once the data is loaded. Use with
-care.</p>
-</li>
-<li><p class="first"><em>create tables</em></p>
-<p>When this option is listed, pgloader creates the table using the meta
-data found in the <cite>MySQL</cite> file, which must contain a list of fields with
-their data type. A standard data type conversion from DBF to PostgreSQL
-is done.</p>
-</li>
-<li><p class="first"><em>create no tables</em></p>
-<p>When this option is listed, pgloader skips the creation of table before
-loading data, target tables must then already exist.</p>
-<p>Also, when using <em>create no tables</em> pgloader fetches the metadata from
-the current target database and checks type casting, then will remove
-constraints and indexes prior to loading the data and install them back
-again once the loading is done.</p>
-</li>
-<li><p class="first"><em>create indexes</em></p>
-<p>When this option is listed, pgloader gets the definitions of all the
-indexes found in the MySQL database and create the same set of index
-definitions against the PostgreSQL database.</p>
-</li>
-<li><p class="first"><em>create no indexes</em></p>
-<p>When this option is listed, pgloader skips the creating indexes.</p>
-</li>
-<li><p class="first"><em>drop indexes</em></p>
-<p>When this option is listed, pgloader drops the indexes in the target
-database before loading the data, and creates them again at the end
-of the data copy.</p>
-</li>
-<li><p class="first"><em>uniquify index names</em>, <em>preserve index names</em></p>
-<p>MySQL index names are unique per-table whereas in PostgreSQL index names
-have to be unique per-schema. The default for pgloader is to change the
-index name by prefixing it with <cite>idx_OID</cite> where <cite>OID</cite> is the internal
-numeric identifier of the table the index is built against.</p>
-<p>In somes cases like when the DDL are entirely left to a framework it
-might be sensible for pgloader to refrain from handling index unique
-names, that is achieved by using the <em>preserve index names</em> option.</p>
-<p>The default is to <em>uniquify index names</em>.</p>
-<p>Even when using the option <em>preserve index names</em>, MySQL primary key
-indexes named “PRIMARY” will get their names uniquified. Failing to do
-so would prevent the primary keys to be created again in PostgreSQL
-where the index names must be unique per schema.</p>
-</li>
-<li><p class="first"><em>drop schema</em></p>
-<p>When this option is listed, pgloader drops the target schema in the
-target PostgreSQL database before creating it again and all the objects
-it contains. The default behavior doesn’t drop the target schemas.</p>
-</li>
-<li><p class="first"><em>foreign keys</em></p>
-<p>When this option is listed, pgloader gets the definitions of all the
-foreign keys found in the MySQL database and create the same set of
-foreign key definitions against the PostgreSQL database.</p>
-</li>
-<li><p class="first"><em>no foreign keys</em></p>
-<p>When this option is listed, pgloader skips creating foreign keys.</p>
-</li>
-<li><p class="first"><em>reset sequences</em></p>
-<p>When this option is listed, at the end of the data loading and after the
-indexes have all been created, pgloader resets all the PostgreSQL
-sequences created to the current maximum value of the column they are
-attached to.</p>
-<p>The options <em>schema only</em> and <em>data only</em> have no effects on this
-option.</p>
-</li>
-<li><p class="first"><em>reset no sequences</em></p>
-<p>When this option is listed, pgloader skips resetting sequences after the
-load.</p>
-<p>The options <em>schema only</em> and <em>data only</em> have no effects on this
-option.</p>
-</li>
-<li><p class="first"><em>downcase identifiers</em></p>
-<p>When this option is listed, pgloader converts all MySQL identifiers
-(table names, index names, column names) to <em>downcase</em>, except for
-PostgreSQL <em>reserved</em> keywords.</p>
-<p>The PostgreSQL <em>reserved</em> keywords are determined dynamically by using
-the system function <cite>pg_get_keywords()</cite>.</p>
-</li>
-<li><p class="first"><em>quote identifiers</em></p>
-<p>When this option is listed, pgloader quotes all MySQL identifiers so
-that their case is respected. Note that you will then have to do the
-same thing in your application code queries.</p>
-</li>
-<li><p class="first"><em>schema only</em></p>
-<p>When this option is listed pgloader refrains from migrating the data
-over. Note that the schema in this context includes the indexes when the
-option <em>create indexes</em> has been listed.</p>
-</li>
-<li><p class="first"><em>data only</em></p>
-<p>When this option is listed pgloader only issues the <cite>COPY</cite> statements,
-without doing any other processing.</p>
-</li>
-<li><p class="first"><em>single reader per thread</em>, <em>multiple readers per thread</em></p>
-<p>The default is <em>single reader per thread</em> and it means that each
-MySQL table is read by a single thread as a whole, with a single
-<cite>SELECT</cite> statement using no <cite>WHERE</cite> clause.</p>
-<p>When using <em>multiple readers per thread</em> pgloader may be able to
-divide the reading work into several threads, as many as the
-<em>concurrency</em> setting, which needs to be greater than 1 for this
-option to kick be activated.</p>
-<p>For each source table, pgloader searches for a primary key over a
-single numeric column, or a multiple-column primary key index for
-which the first column is of a numeric data type (one of <cite>integer</cite>
-or <cite>bigint</cite>). When such an index exists, pgloader runs a query to
-find the <em>min</em> and <em>max</em> values on this column, and then split that
-range into many ranges containing a maximum of <em>rows per range</em>.</p>
-<p>When the range list we then obtain contains at least as many ranges
-than our concurrency setting, then we distribute those ranges to
-each reader thread.</p>
-<p>So when all the conditions are met, pgloader then starts as many
-reader thread as the <em>concurrency</em> setting, and each reader thread
-issues several queries with a <cite>WHERE id &gt;= x AND id &lt; y</cite>, where <cite>y -
-x = rows per range</cite> or less (for the last range, depending on the
-max value just obtained.</p>
-</li>
-<li><p class="first"><em>rows per range</em></p>
-<p>How many rows are fetched per <cite>SELECT</cite> query when using <em>multiple
-readers per thread</em>, see above for details.</p>
-</li>
-<li><p class="first"><em>SET MySQL PARAMETERS</em></p>
-<p>The <em>SET MySQL PARAMETERS</em> allows setting MySQL parameters using the
-MySQL <cite>SET</cite> command each time pgloader connects to it.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="mysql-database-casting-rules">
-<h2>MySQL Database Casting Rules<a class="headerlink" href="#mysql-database-casting-rules" title="Permalink to this headline">¶</a></h2>
-<p>The command <em>CAST</em> introduces user-defined casting rules.</p>
-<p>The cast clause allows to specify custom casting rules, either to overload
-the default casting rules or to amend them with special cases.</p>
-<p>A casting rule is expected to follow one of the forms:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="o">&lt;</span><span class="n">mysql</span><span class="o">-</span><span class="nb">type</span><span class="o">-</span><span class="n">name</span><span class="o">&gt;</span> <span class="p">[</span> <span class="o">&lt;</span><span class="n">guard</span><span class="o">&gt;</span> <span class="o">...</span> <span class="p">]</span> <span class="n">to</span> <span class="o">&lt;</span><span class="n">pgsql</span><span class="o">-</span><span class="nb">type</span><span class="o">-</span><span class="n">name</span><span class="o">&gt;</span> <span class="p">[</span> <span class="o">&lt;</span><span class="n">option</span><span class="o">&gt;</span> <span class="o">...</span> <span class="p">]</span>
-<span class="n">column</span> <span class="o">&lt;</span><span class="n">table</span><span class="o">-</span><span class="n">name</span><span class="o">&gt;.&lt;</span><span class="n">column</span><span class="o">-</span><span class="n">name</span><span class="o">&gt;</span> <span class="p">[</span> <span class="o">&lt;</span><span class="n">guards</span><span class="o">&gt;</span> <span class="p">]</span> <span class="n">to</span> <span class="o">...</span>
-</pre></div>
-</div>
-<p>It’s possible for a <em>casting rule</em> to either match against a MySQL data type
-or against a given <em>column name</em> in a given <em>table name</em>. That flexibility
-allows to cope with cases where the type <cite>tinyint</cite> might have been used as a
-<cite>boolean</cite> in some cases but as a <cite>smallint</cite> in others.</p>
-<p>The <em>casting rules</em> are applied in order, the first match prevents following
-rules to be applied, and user defined rules are evaluated first.</p>
-<p>The supported guards are:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>when unsigned</em></p>
-<p>The casting rule is only applied against MySQL columns of the source
-type that have the keyword <em>unsigned</em> in their data type definition.</p>
-<p>Example of a casting rule using a <em>unsigned</em> guard:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">smallint</span> <span class="n">when</span> <span class="n">unsigned</span> <span class="n">to</span> <span class="n">integer</span> <span class="n">drop</span> <span class="n">typemod</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>when default ‘value’</em></p>
-<p>The casting rule is only applied against MySQL columns of the source
-type that have given <em>value</em>, which must be a single-quoted or a
-double-quoted string.</p>
-</li>
-<li><p class="first"><em>when typemod expression</em></p>
-<p>The casting rule is only applied against MySQL columns of the source
-type that have a <em>typemod</em> value matching the given <em>typemod
-expression</em>. The <em>typemod</em> is separated into its <em>precision</em> and <em>scale</em>
-components.</p>
-<p>Example of a cast rule using a <em>typemod</em> guard:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">char</span> <span class="n">when</span> <span class="p">(</span><span class="o">=</span> <span class="n">precision</span> <span class="mi">1</span><span class="p">)</span> <span class="n">to</span> <span class="n">char</span> <span class="n">keep</span> <span class="n">typemod</span>
-</pre></div>
-</div>
-<p>This expression casts MySQL <cite>char(1)</cite> column to a PostgreSQL column of
-type <cite>char(1)</cite> while allowing for the general case <cite>char(N)</cite> will be
-converted by the default cast rule into a PostgreSQL type <cite>varchar(N)</cite>.</p>
-</li>
-<li><p class="first"><em>with extra auto_increment</em></p>
-<p>The casting rule is only applied against MySQL columns having the
-<em>extra</em> column <cite>auto_increment</cite> option set, so that it’s possible to
-target e.g. <cite>serial</cite> rather than <cite>integer</cite>.</p>
-<p>The default matching behavior, when this option isn’t set, is to match
-both columns with the extra definition and without.</p>
-<p>This means that if you want to implement a casting rule that target
-either <cite>serial</cite> or <cite>integer</cite> from a <cite>smallint</cite> definition depending on
-the <em>auto_increment</em> extra bit of information from MySQL, then you need
-to spell out two casting rules as following:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">smallint</span>  <span class="k">with</span> <span class="n">extra</span> <span class="n">auto_increment</span>
-  <span class="n">to</span> <span class="n">serial</span> <span class="n">drop</span> <span class="n">typemod</span> <span class="n">keep</span> <span class="n">default</span> <span class="n">keep</span> <span class="ow">not</span> <span class="n">null</span><span class="p">,</span>
-
-<span class="nb">type</span> <span class="n">smallint</span>
-  <span class="n">to</span> <span class="n">integer</span> <span class="n">drop</span> <span class="n">typemod</span> <span class="n">keep</span> <span class="n">default</span> <span class="n">keep</span> <span class="ow">not</span> <span class="n">null</span>
-</pre></div>
-</div>
-</li>
-</ul>
-</div></blockquote>
-<p>The supported casting options are:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>drop default</em>, <em>keep default</em></p>
-<p>When the option <em>drop default</em> is listed, pgloader drops any
-existing default expression in the MySQL database for columns of the
-source type from the <cite>CREATE TABLE</cite> statement it generates.</p>
-<p>The spelling <em>keep default</em> explicitly prevents that behaviour and
-can be used to overload the default casting rules.</p>
-</li>
-<li><p class="first"><em>drop not null</em>, <em>keep not null</em>, <em>set not null</em></p>
-<p>When the option <em>drop not null</em> is listed, pgloader drops any
-existing <cite>NOT NULL</cite> constraint associated with the given source
-MySQL datatype when it creates the tables in the PostgreSQL
-database.</p>
-<p>The spelling <em>keep not null</em> explicitly prevents that behaviour and
-can be used to overload the default casting rules.</p>
-<p>When the option <em>set not null</em> is listed, pgloader sets a <cite>NOT NULL</cite>
-constraint on the target column regardless whether it has been set
-in the source MySQL column.</p>
-</li>
-<li><p class="first"><em>drop typemod</em>, <em>keep typemod</em></p>
-<p>When the option <em>drop typemod</em> is listed, pgloader drops any
-existing <em>typemod</em> definition (e.g. <em>precision</em> and <em>scale</em>) from
-the datatype definition found in the MySQL columns of the source
-type when it created the tables in the PostgreSQL database.</p>
-<p>The spelling <em>keep typemod</em> explicitly prevents that behaviour and
-can be used to overload the default casting rules.</p>
-</li>
-<li><p class="first"><em>using</em></p>
-<p>This option takes as its single argument the name of a function to
-be found in the <cite>pgloader.transforms</cite> Common Lisp package. See above
-for details.</p>
-<p>It’s possible to augment a default cast rule (such as one that
-applies against <cite>ENUM</cite> data type for example) with a <em>transformation
-function</em> by omitting entirely the <cite>type</cite> parts of the casting rule,
-as in the following example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">column</span> <span class="nb">enumerate</span><span class="o">.</span><span class="n">foo</span> <span class="n">using</span> <span class="n">empty</span><span class="o">-</span><span class="n">string</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">null</span>
-</pre></div>
-</div>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="mysql-views-support">
-<h2>MySQL Views Support<a class="headerlink" href="#mysql-views-support" title="Permalink to this headline">¶</a></h2>
-<p>MySQL views support allows pgloader to migrate view as if they were base
-tables. This feature then allows for on-the-fly transformation from MySQL to
-PostgreSQL, as the view definition is used rather than the base data.</p>
-<div class="section" id="materialize-views">
-<h3>MATERIALIZE VIEWS<a class="headerlink" href="#materialize-views" title="Permalink to this headline">¶</a></h3>
-<p>This clause allows you to implement custom data processing at the data
-source by providing a <em>view definition</em> against which pgloader will query
-the data. It’s not possible to just allow for plain <cite>SQL</cite> because we want to
-know a lot about the exact data types of each column involved in the query
-output.</p>
-<p>This clause expect a comma separated list of view definitions, each one
-being either the name of an existing view in your database or the following
-expression:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>*name* `AS` `$$` *sql query* `$$`
-</pre></div>
-</div>
-<p>The <em>name</em> and the <em>sql query</em> will be used in a <cite>CREATE VIEW</cite> statement at
-the beginning of the data loading, and the resulting view will then be
-dropped at the end of the data loading.</p>
-</div>
-<div class="section" id="materialize-all-views">
-<h3>MATERIALIZE ALL VIEWS<a class="headerlink" href="#materialize-all-views" title="Permalink to this headline">¶</a></h3>
-<p>Same behaviour as <em>MATERIALIZE VIEWS</em> using the dynamic list of views as
-returned by MySQL rather than asking the user to specify the list.</p>
-</div>
-</div>
-<div class="section" id="mysql-partial-migration">
-<h2>MySQL Partial Migration<a class="headerlink" href="#mysql-partial-migration" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="including-only-table-names-matching">
-<h3>INCLUDING ONLY TABLE NAMES MATCHING<a class="headerlink" href="#including-only-table-names-matching" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table names or <em>regular expression</em> used
-to limit the tables to migrate to a sublist.</p>
-<p>Example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">including</span> <span class="n">only</span> <span class="n">table</span> <span class="n">names</span> <span class="n">matching</span> <span class="o">~/</span><span class="n">film</span><span class="o">/</span><span class="p">,</span> <span class="s1">&#39;actor&#39;</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="excluding-table-names-matching">
-<h3>EXCLUDING TABLE NAMES MATCHING<a class="headerlink" href="#excluding-table-names-matching" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table names or <em>regular expression</em> used
-to exclude table names from the migration. This filter only applies to the
-result of the <em>INCLUDING</em> filter.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">excluding</span> <span class="n">table</span> <span class="n">names</span> <span class="n">matching</span> <span class="o">~&lt;</span><span class="n">ory</span><span class="o">&gt;</span>
-</pre></div>
-</div>
-</div>
-</div>
-<div class="section" id="mysql-encoding-support">
-<h2>MySQL Encoding Support<a class="headerlink" href="#mysql-encoding-support" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="decoding-table-names-matching">
-<h3>DECODING TABLE NAMES MATCHING<a class="headerlink" href="#decoding-table-names-matching" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table names or <em>regular expressions</em>
-used to force the encoding to use when processing data from MySQL. If the
-data encoding known to you is different from MySQL’s idea about it, this is
-the option to use.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">decoding</span> <span class="n">table</span> <span class="n">names</span> <span class="n">matching</span> <span class="o">~/</span><span class="n">messed</span><span class="o">/</span><span class="p">,</span> <span class="o">~/</span><span class="n">encoding</span><span class="o">/</span> <span class="n">AS</span> <span class="n">utf8</span>
-</pre></div>
-</div>
-<p>You can use as many such rules as you need, all with possibly different
-encodings.</p>
-</div>
-</div>
-<div class="section" id="mysql-schema-transformations">
-<h2>MySQL Schema Transformations<a class="headerlink" href="#mysql-schema-transformations" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="alter-table-names-matching">
-<h3>ALTER TABLE NAMES MATCHING<a class="headerlink" href="#alter-table-names-matching" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table names or <em>regular expressions</em>
-that you want to target in the pgloader <em>ALTER TABLE</em> command. The only two
-available actions are <em>SET SCHEMA</em> and <em>RENAME TO</em>, both take a quoted
-string as parameter:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>ALTER TABLE NAMES MATCHING ~/_list$/, &#39;sales_by_store&#39;, ~/sales_by/
- SET SCHEMA &#39;mv&#39;
-
-ALTER TABLE NAMES MATCHING &#39;film&#39; RENAME TO &#39;films&#39;
-
-ALTER TABLE NAMES MATCHING ~/./ SET (fillfactor=&#39;40&#39;)
-</pre></div>
-</div>
-<p>You can use as many such rules as you need. The list of tables to be
-migrated is searched in pgloader memory against the <em>ALTER TABLE</em> matching
-rules, and for each command pgloader stops at the first matching criteria
-(regexp or string).</p>
-<p>No <em>ALTER TABLE</em> command is sent to PostgreSQL, the modification happens at
-the level of the pgloader in-memory representation of your source database
-schema. In case of a name change, the mapping is kept and reused in the
-<em>foreign key</em> and <em>index</em> support.</p>
-<p>The <em>SET ()</em> action takes effect as a <em>WITH</em> clause for the <cite>CREATE TABLE</cite>
-command that pgloader will run when it has to create a table.</p>
-</div>
-</div>
-<div class="section" id="mysql-migration-limitations">
-<h2>MySQL Migration: limitations<a class="headerlink" href="#mysql-migration-limitations" title="Permalink to this headline">¶</a></h2>
-<p>The <cite>database</cite> command currently only supports MySQL source database and has
-the following limitations:</p>
-<blockquote>
-<div><ul>
-<li><p class="first">Views are not migrated,</p>
-<p>Supporting views might require implementing a full SQL parser for the
-MySQL dialect with a porting engine to rewrite the SQL against
-PostgreSQL, including renaming functions and changing some constructs.</p>
-<p>While it’s not theoretically impossible, don’t hold your breath.</p>
-</li>
-<li><p class="first">Triggers are not migrated</p>
-<p>The difficulty of doing so is not yet assessed.</p>
-</li>
-<li><p class="first">Of the geometric datatypes, only the <cite>POINT</cite> database has been covered.
-The other ones should be easy enough to implement now, it’s just not
-done yet.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="default-mysql-casting-rules">
-<h2>Default MySQL Casting Rules<a class="headerlink" href="#default-mysql-casting-rules" title="Permalink to this headline">¶</a></h2>
-<p>When migrating from MySQL the following Casting Rules are provided:</p>
-<p>Numbers:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="nb">int</span> <span class="k">with</span> <span class="n">extra</span> <span class="n">auto_increment</span> <span class="n">to</span> <span class="n">serial</span> <span class="n">when</span> <span class="p">(</span><span class="o">&lt;</span> <span class="n">precision</span> <span class="mi">10</span><span class="p">)</span>
-<span class="nb">type</span> <span class="nb">int</span> <span class="k">with</span> <span class="n">extra</span> <span class="n">auto_increment</span> <span class="n">to</span> <span class="n">bigserial</span> <span class="n">when</span> <span class="p">(</span><span class="o">&lt;=</span> <span class="mi">10</span> <span class="n">precision</span><span class="p">)</span>
-<span class="nb">type</span> <span class="nb">int</span> <span class="n">to</span> <span class="nb">int</span>       <span class="n">when</span>  <span class="p">(</span><span class="o">&lt;</span> <span class="n">precision</span> <span class="mi">10</span><span class="p">)</span>
-<span class="nb">type</span> <span class="nb">int</span> <span class="n">to</span> <span class="n">bigint</span>    <span class="n">when</span>  <span class="p">(</span><span class="o">&lt;=</span> <span class="mi">10</span> <span class="n">precision</span><span class="p">)</span>
-<span class="nb">type</span> <span class="n">tinyint</span>   <span class="k">with</span> <span class="n">extra</span> <span class="n">auto_increment</span> <span class="n">to</span> <span class="n">serial</span>
-<span class="nb">type</span> <span class="n">smallint</span>  <span class="k">with</span> <span class="n">extra</span> <span class="n">auto_increment</span> <span class="n">to</span> <span class="n">serial</span>
-<span class="nb">type</span> <span class="n">mediumint</span> <span class="k">with</span> <span class="n">extra</span> <span class="n">auto_increment</span> <span class="n">to</span> <span class="n">serial</span>
-<span class="nb">type</span> <span class="n">bigint</span>    <span class="k">with</span> <span class="n">extra</span> <span class="n">auto_increment</span> <span class="n">to</span> <span class="n">bigserial</span>
-
-<span class="nb">type</span> <span class="n">tinyint</span> <span class="n">to</span> <span class="n">boolean</span> <span class="n">when</span> <span class="p">(</span><span class="o">=</span> <span class="mi">1</span> <span class="n">precision</span><span class="p">)</span> <span class="n">using</span> <span class="n">tinyint</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">boolean</span>
-
-<span class="nb">type</span> <span class="n">tinyint</span> <span class="n">when</span> <span class="n">unsigned</span> <span class="n">to</span> <span class="n">smallint</span>   <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">smallint</span> <span class="n">when</span> <span class="n">unsigned</span> <span class="n">to</span> <span class="n">integer</span>  <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">mediumint</span> <span class="n">when</span> <span class="n">unsigned</span> <span class="n">to</span> <span class="n">integer</span>  <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">integer</span> <span class="n">when</span> <span class="n">unsigned</span> <span class="n">to</span> <span class="n">bigint</span>    <span class="n">drop</span> <span class="n">typemod</span>
-
-<span class="nb">type</span> <span class="n">tinyint</span> <span class="n">to</span> <span class="n">smallint</span>   <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">smallint</span> <span class="n">to</span> <span class="n">smallint</span>  <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">mediumint</span> <span class="n">to</span> <span class="n">integer</span>  <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">integer</span> <span class="n">to</span> <span class="n">integer</span>    <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">bigint</span> <span class="n">to</span> <span class="n">bigint</span>      <span class="n">drop</span> <span class="n">typemod</span>
-
-<span class="nb">type</span> <span class="nb">float</span> <span class="n">to</span> <span class="nb">float</span>        <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">double</span> <span class="n">to</span> <span class="n">double</span> <span class="n">precision</span> <span class="n">drop</span> <span class="n">typemod</span>
-
-<span class="nb">type</span> <span class="n">numeric</span> <span class="n">to</span> <span class="n">numeric</span> <span class="n">keep</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">decimal</span> <span class="n">to</span> <span class="n">decimal</span> <span class="n">keep</span> <span class="n">typemod</span>
-</pre></div>
-</div>
-<p>Texts:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">char</span>       <span class="n">to</span> <span class="n">char</span> <span class="n">keep</span> <span class="n">typemod</span> <span class="n">using</span> <span class="n">remove</span><span class="o">-</span><span class="n">null</span><span class="o">-</span><span class="n">characters</span>
-<span class="nb">type</span> <span class="n">varchar</span>    <span class="n">to</span> <span class="n">varchar</span> <span class="n">keep</span> <span class="n">typemod</span> <span class="n">using</span> <span class="n">remove</span><span class="o">-</span><span class="n">null</span><span class="o">-</span><span class="n">characters</span>
-<span class="nb">type</span> <span class="n">tinytext</span>   <span class="n">to</span> <span class="n">text</span> <span class="n">using</span> <span class="n">remove</span><span class="o">-</span><span class="n">null</span><span class="o">-</span><span class="n">characters</span>
-<span class="nb">type</span> <span class="n">text</span>       <span class="n">to</span> <span class="n">text</span> <span class="n">using</span> <span class="n">remove</span><span class="o">-</span><span class="n">null</span><span class="o">-</span><span class="n">characters</span>
-<span class="nb">type</span> <span class="n">mediumtext</span> <span class="n">to</span> <span class="n">text</span> <span class="n">using</span> <span class="n">remove</span><span class="o">-</span><span class="n">null</span><span class="o">-</span><span class="n">characters</span>
-<span class="nb">type</span> <span class="n">longtext</span>   <span class="n">to</span> <span class="n">text</span> <span class="n">using</span> <span class="n">remove</span><span class="o">-</span><span class="n">null</span><span class="o">-</span><span class="n">characters</span>
-</pre></div>
-</div>
-<p>Binary:</p>
-<blockquote>
-<div>type binary     to bytea
-type varbinary  to bytea
-type tinyblob   to bytea
-type blob       to bytea
-type mediumblob to bytea
-type longblob   to bytea</div></blockquote>
-<p>Date:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">datetime</span> <span class="n">when</span> <span class="n">default</span> <span class="s2">&quot;0000-00-00 00:00:00&quot;</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">null</span>
-  <span class="n">to</span> <span class="n">timestamptz</span> <span class="n">drop</span> <span class="ow">not</span> <span class="n">null</span> <span class="n">drop</span> <span class="n">default</span>
-      <span class="n">using</span> <span class="n">zero</span><span class="o">-</span><span class="n">dates</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">null</span>
-
-<span class="nb">type</span> <span class="n">datetime</span> <span class="n">when</span> <span class="n">default</span> <span class="s2">&quot;0000-00-00 00:00:00&quot;</span>
-  <span class="n">to</span> <span class="n">timestamptz</span> <span class="n">drop</span> <span class="n">default</span>
-      <span class="n">using</span> <span class="n">zero</span><span class="o">-</span><span class="n">dates</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">null</span>
-
-<span class="nb">type</span> <span class="n">timestamp</span> <span class="n">when</span> <span class="n">default</span> <span class="s2">&quot;0000-00-00 00:00:00&quot;</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">null</span>
-  <span class="n">to</span> <span class="n">timestamptz</span> <span class="n">drop</span> <span class="ow">not</span> <span class="n">null</span> <span class="n">drop</span> <span class="n">default</span>
-      <span class="n">using</span> <span class="n">zero</span><span class="o">-</span><span class="n">dates</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">null</span>
-
-<span class="nb">type</span> <span class="n">timestamp</span> <span class="n">when</span> <span class="n">default</span> <span class="s2">&quot;0000-00-00 00:00:00&quot;</span>
-  <span class="n">to</span> <span class="n">timestamptz</span> <span class="n">drop</span> <span class="n">default</span>
-      <span class="n">using</span> <span class="n">zero</span><span class="o">-</span><span class="n">dates</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">null</span>
-
-<span class="nb">type</span> <span class="n">date</span> <span class="n">when</span> <span class="n">default</span> <span class="s2">&quot;0000-00-00&quot;</span> <span class="n">to</span> <span class="n">date</span> <span class="n">drop</span> <span class="n">default</span>
-      <span class="n">using</span> <span class="n">zero</span><span class="o">-</span><span class="n">dates</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">null</span>
-
-<span class="nb">type</span> <span class="n">date</span> <span class="n">to</span> <span class="n">date</span>
-<span class="nb">type</span> <span class="n">datetime</span> <span class="n">to</span> <span class="n">timestamptz</span>
-<span class="nb">type</span> <span class="n">timestamp</span> <span class="n">to</span> <span class="n">timestamptz</span>
-<span class="nb">type</span> <span class="n">year</span> <span class="n">to</span> <span class="n">integer</span> <span class="n">drop</span> <span class="n">typemod</span>
-</pre></div>
-</div>
-<p>Geometric:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">point</span> <span class="n">to</span> <span class="n">point</span> <span class="n">using</span> <span class="n">pgloader</span><span class="o">.</span><span class="n">transforms</span><span class="p">::</span><span class="n">convert</span><span class="o">-</span><span class="n">mysql</span><span class="o">-</span><span class="n">point</span>
-</pre></div>
-</div>
-<p>Enum types are declared inline in MySQL and separately with a <cite>CREATE TYPE</cite>
-command in PostgreSQL, so each column of Enum Type is converted to a type
-named after the table and column names defined with the same labels in the
-same order.</p>
-<p>When the source type definition is not matched in the default casting rules
-nor in the casting rules provided in the command, then the type name with
-the typemod is used.</p>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="archive.html" title="previous chapter">Loading From an Archive</a></li>
-      <li>Next: <a href="sqlite.html" title="next chapter">Migrating a SQLite database to PostgreSQL</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/mysql.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/sqlite.html

@@ -1,268 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Migrating a SQLite database to PostgreSQL &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Migrating a MS SQL Database to PostgreSQL" href="mssql.html" />
-    <link rel="prev" title="Migrating a MySQL Database to PostgreSQL" href="mysql.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="migrating-a-sqlite-database-to-postgresql">
-<h1>Migrating a SQLite database to PostgreSQL<a class="headerlink" href="#migrating-a-sqlite-database-to-postgresql" title="Permalink to this headline">¶</a></h1>
-<p>This command instructs pgloader to load data from a SQLite file. Automatic
-discovery of the schema is supported, including build of the indexes.</p>
-<p>Here’s an example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">load</span> <span class="n">database</span>
-     <span class="kn">from</span> <span class="nn">sqlite</span><span class="p">:</span><span class="o">///</span><span class="n">Users</span><span class="o">/</span><span class="n">dim</span><span class="o">/</span><span class="n">Downloads</span><span class="o">/</span><span class="n">lastfm_tags</span><span class="o">.</span><span class="n">db</span>
-     <span class="n">into</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">tags</span>
-
- <span class="k">with</span> <span class="n">include</span> <span class="n">drop</span><span class="p">,</span> <span class="n">create</span> <span class="n">tables</span><span class="p">,</span> <span class="n">create</span> <span class="n">indexes</span><span class="p">,</span> <span class="n">reset</span> <span class="n">sequences</span>
-
-  <span class="nb">set</span> <span class="n">work_mem</span> <span class="n">to</span> <span class="s1">&#39;16MB&#39;</span><span class="p">,</span> <span class="n">maintenance_work_mem</span> <span class="n">to</span> <span class="s1">&#39;512 MB&#39;</span><span class="p">;</span>
-</pre></div>
-</div>
-<p>The <cite>sqlite</cite> command accepts the following clauses and options.</p>
-<div class="section" id="sqlite-database-source-specification-from">
-<h2>SQLite Database Source Specification: FROM<a class="headerlink" href="#sqlite-database-source-specification-from" title="Permalink to this headline">¶</a></h2>
-<p>Path or HTTP URL to a SQLite file, might be a <cite>.zip</cite> file.</p>
-</div>
-<div class="section" id="sqlite-database-migration-options-with">
-<h2>SQLite Database Migration Options: WITH<a class="headerlink" href="#sqlite-database-migration-options-with" title="Permalink to this headline">¶</a></h2>
-<p>When loading from a <cite>SQLite</cite> database, the following options are
-supported:</p>
-<p>When loading from a <cite>SQLite</cite> database, the following options are
-supported, and the default <em>WITH</em> clause is: <em>no truncate</em>, <em>create
-tables</em>, <em>include drop</em>, <em>create indexes</em>, <em>reset sequences</em>, <em>downcase
-identifiers</em>, <em>encoding ‘utf-8’</em>.</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>include drop</em></p>
-<p>When this option is listed, pgloader drops all the tables in the target
-PostgreSQL database whose names appear in the SQLite database. This
-option allows for using the same command several times in a row until
-you figure out all the options, starting automatically from a clean
-environment. Please note that <cite>CASCADE</cite> is used to ensure that tables
-are dropped even if there are foreign keys pointing to them. This is
-precisely what <cite>include drop</cite> is intended to do: drop all target tables
-and recreate them.</p>
-<p>Great care needs to be taken when using <cite>include drop</cite>, as it will
-cascade to <em>all</em> objects referencing the target tables, possibly
-including other tables that are not being loaded from the source DB.</p>
-</li>
-<li><p class="first"><em>include no drop</em></p>
-<p>When this option is listed, pgloader will not include any <cite>DROP</cite>
-statement when loading the data.</p>
-</li>
-<li><p class="first"><em>truncate</em></p>
-<p>When this option is listed, pgloader issue the <cite>TRUNCATE</cite> command
-against each PostgreSQL table just before loading data into it.</p>
-</li>
-<li><p class="first"><em>no truncate</em></p>
-<p>When this option is listed, pgloader issues no <cite>TRUNCATE</cite> command.</p>
-</li>
-<li><p class="first"><em>disable triggers</em></p>
-<p>When this option is listed, pgloader issues an <cite>ALTER TABLE … DISABLE
-TRIGGER ALL</cite> command against the PostgreSQL target table before copying
-the data, then the command <cite>ALTER TABLE … ENABLE TRIGGER ALL</cite> once the
-<cite>COPY</cite> is done.</p>
-<p>This option allows loading data into a pre-existing table ignoring
-the <em>foreign key constraints</em> and user defined triggers and may
-result in invalid <em>foreign key constraints</em> once the data is loaded.
-Use with care.</p>
-</li>
-<li><p class="first"><em>create tables</em></p>
-<p>When this option is listed, pgloader creates the table using the meta
-data found in the <cite>SQLite</cite> file, which must contain a list of fields
-with their data type. A standard data type conversion from SQLite to
-PostgreSQL is done.</p>
-</li>
-<li><p class="first"><em>create no tables</em></p>
-<p>When this option is listed, pgloader skips the creation of table before
-loading data, target tables must then already exist.</p>
-<p>Also, when using <em>create no tables</em> pgloader fetches the metadata
-from the current target database and checks type casting, then will
-remove constraints and indexes prior to loading the data and install
-them back again once the loading is done.</p>
-</li>
-<li><p class="first"><em>create indexes</em></p>
-<p>When this option is listed, pgloader gets the definitions of all the
-indexes found in the SQLite database and create the same set of index
-definitions against the PostgreSQL database.</p>
-</li>
-<li><p class="first"><em>create no indexes</em></p>
-<p>When this option is listed, pgloader skips the creating indexes.</p>
-</li>
-<li><p class="first"><em>drop indexes</em></p>
-<p>When this option is listed, pgloader drops the indexes in the target
-database before loading the data, and creates them again at the end
-of the data copy.</p>
-</li>
-<li><p class="first"><em>reset sequences</em></p>
-<p>When this option is listed, at the end of the data loading and after
-the indexes have all been created, pgloader resets all the
-PostgreSQL sequences created to the current maximum value of the
-column they are attached to.</p>
-</li>
-<li><p class="first"><em>reset no sequences</em></p>
-<p>When this option is listed, pgloader skips resetting sequences after the
-load.</p>
-<p>The options <em>schema only</em> and <em>data only</em> have no effects on this
-option.</p>
-</li>
-<li><p class="first"><em>schema only</em></p>
-<p>When this option is listed pgloader will refrain from migrating the data
-over. Note that the schema in this context includes the indexes when the
-option <em>create indexes</em> has been listed.</p>
-</li>
-<li><p class="first"><em>data only</em></p>
-<p>When this option is listed pgloader only issues the <cite>COPY</cite> statements,
-without doing any other processing.</p>
-</li>
-<li><p class="first"><em>encoding</em></p>
-<p>This option allows to control which encoding to parse the SQLite text
-data with. Defaults to UTF-8.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-<div class="section" id="sqlite-database-casting-rules">
-<h2>SQLite Database Casting Rules<a class="headerlink" href="#sqlite-database-casting-rules" title="Permalink to this headline">¶</a></h2>
-<p>The command <em>CAST</em> introduces user-defined casting rules.</p>
-<p>The cast clause allows to specify custom casting rules, either to overload
-the default casting rules or to amend them with special cases.</p>
-</div>
-<div class="section" id="sqlite-database-partial-migrations">
-<h2>SQlite Database Partial Migrations<a class="headerlink" href="#sqlite-database-partial-migrations" title="Permalink to this headline">¶</a></h2>
-<div class="section" id="including-only-table-names-like">
-<h3>INCLUDING ONLY TABLE NAMES LIKE<a class="headerlink" href="#including-only-table-names-like" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table name patterns used to limit the
-tables to migrate to a sublist.</p>
-<p>Example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">including</span> <span class="n">only</span> <span class="n">table</span> <span class="n">names</span> <span class="n">like</span> <span class="s1">&#39;Invoice%&#39;</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="excluding-table-names-like">
-<h3>EXCLUDING TABLE NAMES LIKE<a class="headerlink" href="#excluding-table-names-like" title="Permalink to this headline">¶</a></h3>
-<p>Introduce a comma separated list of table name patterns used to exclude
-table names from the migration. This filter only applies to the result of
-the <em>INCLUDING</em> filter.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">excluding</span> <span class="n">table</span> <span class="n">names</span> <span class="n">like</span> <span class="s1">&#39;appointments&#39;</span>
-</pre></div>
-</div>
-</div>
-</div>
-<div class="section" id="default-sqlite-casting-rules">
-<h2>Default SQLite Casting Rules<a class="headerlink" href="#default-sqlite-casting-rules" title="Permalink to this headline">¶</a></h2>
-<p>When migrating from SQLite the following Casting Rules are provided:</p>
-<p>Numbers:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">tinyint</span> <span class="n">to</span> <span class="n">smallint</span> <span class="n">using</span> <span class="n">integer</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">integer</span> <span class="n">to</span> <span class="n">bigint</span>   <span class="n">using</span> <span class="n">integer</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-
-<span class="nb">type</span> <span class="nb">float</span> <span class="n">to</span> <span class="nb">float</span>   <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">real</span> <span class="n">to</span> <span class="n">real</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">double</span> <span class="n">to</span> <span class="n">double</span> <span class="n">precision</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-<span class="nb">type</span> <span class="n">numeric</span> <span class="n">to</span> <span class="n">numeric</span>     <span class="n">using</span> <span class="nb">float</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">string</span>
-</pre></div>
-</div>
-<p>Texts:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">character</span>  <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">varchar</span>    <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">nvarchar</span>   <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">char</span>       <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">nchar</span>      <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">nvarchar</span>   <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-<span class="nb">type</span> <span class="n">clob</span>       <span class="n">to</span> <span class="n">text</span> <span class="n">drop</span> <span class="n">typemod</span>
-</pre></div>
-</div>
-<p>Binary:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">blob</span>       <span class="n">to</span> <span class="n">bytea</span>
-</pre></div>
-</div>
-<p>Date:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">type</span> <span class="n">datetime</span>    <span class="n">to</span> <span class="n">timestamptz</span> <span class="n">using</span> <span class="n">sqlite</span><span class="o">-</span><span class="n">timestamp</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">timestamp</span>
-<span class="nb">type</span> <span class="n">timestamp</span>   <span class="n">to</span> <span class="n">timestamptz</span> <span class="n">using</span> <span class="n">sqlite</span><span class="o">-</span><span class="n">timestamp</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">timestamp</span>
-<span class="nb">type</span> <span class="n">timestamptz</span> <span class="n">to</span> <span class="n">timestamptz</span> <span class="n">using</span> <span class="n">sqlite</span><span class="o">-</span><span class="n">timestamp</span><span class="o">-</span><span class="n">to</span><span class="o">-</span><span class="n">timestamp</span>
-</pre></div>
-</div>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="mysql.html" title="previous chapter">Migrating a MySQL Database to PostgreSQL</a></li>
-      <li>Next: <a href="mssql.html" title="next chapter">Migrating a MS SQL Database to PostgreSQL</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/sqlite.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/ref/transforms.html

@@ -1,214 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Transformation Functions &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="Reporting Bugs" href="../bugreport.html" />
-    <link rel="prev" title="Migrating a MS SQL Database to PostgreSQL" href="mssql.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="transformation-functions">
-<h1>Transformation Functions<a class="headerlink" href="#transformation-functions" title="Permalink to this headline">¶</a></h1>
-<p>Some data types are implemented in a different enough way that a
-transformation function is necessary. This function must be written in
-<cite>Common lisp</cite> and is searched in the <cite>pgloader.transforms</cite> package.</p>
-<p>Some default transformation function are provided with pgloader, and you can
-use the <cite>–load</cite> command line option to load and compile your own lisp file
-into pgloader at runtime. For your functions to be found, remember to begin
-your lisp file with the following form:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">(</span><span class="ow">in</span><span class="o">-</span><span class="n">package</span> <span class="c1">#:pgloader.transforms)</span>
-</pre></div>
-</div>
-<p>The provided transformation functions are:</p>
-<blockquote>
-<div><ul>
-<li><p class="first"><em>zero-dates-to-null</em></p>
-<p>When the input date is all zeroes, return <cite>nil</cite>, which gets loaded as a
-PostgreSQL <cite>NULL</cite> value.</p>
-</li>
-<li><p class="first"><em>date-with-no-separator</em></p>
-<p>Applies <em>zero-dates-to-null</em> then transform the given date into a format
-that PostgreSQL will actually process:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span>  <span class="s2">&quot;20041002152952&quot;</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;2004-10-02 15:29:52&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>time-with-no-separator</em></p>
-<p>Transform the given time into a format that PostgreSQL will actually
-process:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span>  <span class="s2">&quot;08231560&quot;</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;08:23:15.60&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>tinyint-to-boolean</em></p>
-<p>As MySQL lacks a proper boolean type, <em>tinyint</em> is often used to
-implement that. This function transforms <cite>0</cite> to <cite>‘false’</cite> and anything
-else to <cite>‘true</cite>’.</p>
-</li>
-<li><p class="first"><em>bits-to-boolean</em></p>
-<p>As MySQL lacks a proper boolean type, <em>BIT</em> is often used to implement
-that. This function transforms 1-bit bit vectors from <cite>0</cite> to <cite>f</cite> and any
-other value to <cite>t</cite>..</p>
-</li>
-<li><p class="first"><em>int-to-ip</em></p>
-<p>Convert an integer into a dotted representation of an ip4.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span>  <span class="mi">18435761</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;1.25.78.177&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>ip-range</em></p>
-<p>Converts a couple of integers given as strings into a range of ip4.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span>  <span class="s2">&quot;16825344&quot;</span> <span class="s2">&quot;16825599&quot;</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;1.0.188.0-1.0.188.255&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>convert-mysql-point</em></p>
-<p>Converts from the <cite>astext</cite> representation of points in MySQL to the
-PostgreSQL representation.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span>  <span class="s2">&quot;POINT(48.5513589 7.6926827)&quot;</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;(48.5513589,7.6926827)&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>integer-to-string</em></p>
-<p>Converts a integer string or a Common Lisp integer into a string
-suitable for a PostgreSQL integer. Takes care of quoted integers.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span>  <span class="s2">&quot;</span><span class="se">\&quot;</span><span class="s2">0</span><span class="se">\&quot;</span><span class="s2">&quot;</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;0&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>float-to-string</em></p>
-<p>Converts a Common Lisp float into a string suitable for a PostgreSQL float:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span>  <span class="mf">100.0</span><span class="n">d0</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;100.0&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>set-to-enum-array</em></p>
-<p>Converts a string representing a MySQL SET into a PostgreSQL Array of
-Enum values from the set.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">In</span><span class="p">:</span> <span class="s2">&quot;foo,bar&quot;</span>
-<span class="n">Out</span><span class="p">:</span> <span class="s2">&quot;{foo,bar}&quot;</span>
-</pre></div>
-</div>
-</li>
-<li><p class="first"><em>empty-string-to-null</em></p>
-<p>Convert an empty string to a null.</p>
-</li>
-<li><p class="first"><em>right-trim</em></p>
-<p>Remove whitespace at end of string.</p>
-</li>
-<li><p class="first"><em>remove-null-characters</em></p>
-<p>Remove <cite>NUL</cite> characters (<cite>0x0</cite>) from given strings.</p>
-</li>
-<li><p class="first"><em>byte-vector-to-bytea</em></p>
-<p>Transform a simple array of unsigned bytes to the PostgreSQL bytea Hex
-Format representation as documented at
-<a class="reference external" href="http://www.postgresql.org/docs/9.3/interactive/datatype-binary.html">http://www.postgresql.org/docs/9.3/interactive/datatype-binary.html</a></p>
-</li>
-<li><p class="first"><em>sqlite-timestamp-to-timestamp</em></p>
-<p>SQLite type system is quite interesting, so cope with it here to produce
-timestamp literals as expected by PostgreSQL. That covers year only on 4
-digits, 0 dates to null, and proper date strings.</p>
-</li>
-<li><p class="first"><em>sql-server-uniqueidentifier-to-uuid</em></p>
-<p>The SQL Server driver receives data fo type uniqueidentifier as byte
-vector that we then need to convert to an UUID string for PostgreSQL
-COPY input format to process.</p>
-</li>
-<li><p class="first"><em>unix-timestamp-to-timestamptz</em></p>
-<p>Converts a unix timestamp (number of seconds elapsed since beginning of
-1970) into a proper PostgreSQL timestamp format.</p>
-</li>
-<li><p class="first"><em>varbinary-to-string</em></p>
-<p>Converts binary encoded string (such as a MySQL <cite>varbinary</cite> entry) to a
-decoded text, using the table’s encoding that may be overloaded with the
-<em>DECODING TABLE NAMES MATCHING</em> clause.</p>
-</li>
-</ul>
-</div></blockquote>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="mssql.html" title="previous chapter">Migrating a MS SQL Database to PostgreSQL</a></li>
-      <li>Next: <a href="../bugreport.html" title="next chapter">Reporting Bugs</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/ref/transforms.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/search.html

@@ -1,101 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Search &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    './',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="_static/jquery.js"></script>
-    <script type="text/javascript" src="_static/underscore.js"></script>
-    <script type="text/javascript" src="_static/doctools.js"></script>
-    <script type="text/javascript" src="_static/searchtools.js"></script>
-    <link rel="index" title="Index" href="genindex.html" />
-    <link rel="search" title="Search" href="#" />
-  <script type="text/javascript">
-    jQuery(function() { Search.loadIndex("searchindex.js"); });
-  </script>
-  
-  <script type="text/javascript" id="searchindexloader"></script>
-  
-   
-  <link rel="stylesheet" href="_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <h1 id="search-documentation">Search</h1>
-  <div id="fallback" class="admonition warning">
-  <script type="text/javascript">$('#fallback').hide();</script>
-  <p>
-    Please activate JavaScript to enable the search
-    functionality.
-  </p>
-  </div>
-  <p>
-    From here you can search these documents. Enter your search
-    words into the box below and click "search". Note that the search
-    function will automatically search for all of the words. Pages
-    containing fewer words won't appear in the result list.
-  </p>
-  <form action="" method="get">
-    <input type="text" name="q" value="" />
-    <input type="submit" value="search" />
-    <span id="search-progress" style="padding-left: 10px"></span>
-  </form>
-  
-  <div id="search-results">
-  
-  </div>
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/csv.html

@@ -1,169 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading CSV Data with pgloader &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-csv-data-with-pgloader">
-<h1>Loading CSV Data with pgloader<a class="headerlink" href="#loading-csv-data-with-pgloader" title="Permalink to this headline">¶</a></h1>
-<p>CSV means <em>comma separated values</em> and is often found with quite varying
-specifications. pgloader allows you to describe those specs in its command.</p>
-<div class="section" id="the-command">
-<h2>The Command<a class="headerlink" href="#the-command" title="Permalink to this headline">¶</a></h2>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our example for loading CSV
-data:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD CSV
-     FROM &#39;path/to/file.csv&#39; (x, y, a, b, c, d)
-     INTO postgresql:///pgloader?csv (a, b, d, c)
-
-     WITH truncate,
-          skip header = 1,
-          fields optionally enclosed by &#39;&quot;&#39;,
-          fields escaped by double-quote,
-          fields terminated by &#39;,&#39;
-
-      SET client_encoding to &#39;latin1&#39;,
-          work_mem to &#39;12MB&#39;,
-          standard_conforming_strings to &#39;on&#39;
-
-   BEFORE LOAD DO
-    $$ drop table if exists csv; $$,
-    $$ create table csv (
-        a bigint,
-        b bigint,
-        c char(2),
-        d text
-       );
-  $$;
-</pre></div>
-</div>
-</div>
-<div class="section" id="the-data">
-<h2>The Data<a class="headerlink" href="#the-data" title="Permalink to this headline">¶</a></h2>
-<p>This command allows loading the following CSV file content:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>Header, with a © sign
-&quot;2.6.190.56&quot;,&quot;2.6.190.63&quot;,&quot;33996344&quot;,&quot;33996351&quot;,&quot;GB&quot;,&quot;United Kingdom&quot;
-&quot;3.0.0.0&quot;,&quot;4.17.135.31&quot;,&quot;50331648&quot;,&quot;68257567&quot;,&quot;US&quot;,&quot;United States&quot;
-&quot;4.17.135.32&quot;,&quot;4.17.135.63&quot;,&quot;68257568&quot;,&quot;68257599&quot;,&quot;CA&quot;,&quot;Canada&quot;
-&quot;4.17.135.64&quot;,&quot;4.17.142.255&quot;,&quot;68257600&quot;,&quot;68259583&quot;,&quot;US&quot;,&quot;United States&quot;
-&quot;4.17.143.0&quot;,&quot;4.17.143.15&quot;,&quot;68259584&quot;,&quot;68259599&quot;,&quot;CA&quot;,&quot;Canada&quot;
-&quot;4.17.143.16&quot;,&quot;4.18.32.71&quot;,&quot;68259600&quot;,&quot;68296775&quot;,&quot;US&quot;,&quot;United States&quot;
-</pre></div>
-</div>
-</div>
-<div class="section" id="loading-the-data">
-<h2>Loading the data<a class="headerlink" href="#loading-the-data" title="Permalink to this headline">¶</a></h2>
-<p>Here’s how to start loading the data. Note that the ouput here has been
-edited so as to facilitate its browsing online:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader csv.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/csv.load&quot;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-      before load          2          2          0          0.039s
------------------  ---------  ---------  ---------  --------------
-              csv          6          6          0          0.019s
------------------  ---------  ---------  ---------  --------------
-Total import time          6          6          0          0.058s
-</pre></div>
-</div>
-</div>
-<div class="section" id="the-result">
-<h2>The result<a class="headerlink" href="#the-result" title="Permalink to this headline">¶</a></h2>
-<p>As you can see, the command described above is filtering the input and only
-importing some of the columns from the example data file. Here’s what gets
-loaded in the PostgreSQL database:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pgloader</span><span class="c1"># table csv;</span>
-    <span class="n">a</span>     <span class="o">|</span>    <span class="n">b</span>     <span class="o">|</span> <span class="n">c</span>  <span class="o">|</span>       <span class="n">d</span>
-<span class="o">----------+----------+----+----------------</span>
- <span class="mi">33996344</span> <span class="o">|</span> <span class="mi">33996351</span> <span class="o">|</span> <span class="n">GB</span> <span class="o">|</span> <span class="n">United</span> <span class="n">Kingdom</span>
- <span class="mi">50331648</span> <span class="o">|</span> <span class="mi">68257567</span> <span class="o">|</span> <span class="n">US</span> <span class="o">|</span> <span class="n">United</span> <span class="n">States</span>
- <span class="mi">68257568</span> <span class="o">|</span> <span class="mi">68257599</span> <span class="o">|</span> <span class="n">CA</span> <span class="o">|</span> <span class="n">Canada</span>
- <span class="mi">68257600</span> <span class="o">|</span> <span class="mi">68259583</span> <span class="o">|</span> <span class="n">US</span> <span class="o">|</span> <span class="n">United</span> <span class="n">States</span>
- <span class="mi">68259584</span> <span class="o">|</span> <span class="mi">68259599</span> <span class="o">|</span> <span class="n">CA</span> <span class="o">|</span> <span class="n">Canada</span>
- <span class="mi">68259600</span> <span class="o">|</span> <span class="mi">68296775</span> <span class="o">|</span> <span class="n">US</span> <span class="o">|</span> <span class="n">United</span> <span class="n">States</span>
-<span class="p">(</span><span class="mi">6</span> <span class="n">rows</span><span class="p">)</span>
-</pre></div>
-</div>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/csv.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/dBase.html

@@ -1,134 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading dBase files with pgloader &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-dbase-files-with-pgloader">
-<h1>Loading dBase files with pgloader<a class="headerlink" href="#loading-dbase-files-with-pgloader" title="Permalink to this headline">¶</a></h1>
-<p>The dBase format is still in use in some places as modern tools such as
-<em>Filemaker</em> and <em>Excel</em> offer some level of support for it. Speaking of
-support in modern tools, pgloader is right there on the list too!</p>
-<div class="section" id="the-command">
-<h2>The Command<a class="headerlink" href="#the-command" title="Permalink to this headline">¶</a></h2>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our example for loading a
-dBase file, using a file provided by the french administration.</p>
-<p>You can find more files from them at the <a class="reference external" href="http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement.asp">Insee</a>
-website.</p>
-<p>Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">LOAD</span> <span class="n">DBF</span>
-    <span class="n">FROM</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">insee</span><span class="o">.</span><span class="n">fr</span><span class="o">/</span><span class="n">fr</span><span class="o">/</span><span class="n">methodes</span><span class="o">/</span><span class="n">nomenclatures</span><span class="o">/</span><span class="n">cog</span><span class="o">/</span><span class="n">telechargement</span><span class="o">/</span><span class="mi">2013</span><span class="o">/</span><span class="n">dbf</span><span class="o">/</span><span class="n">historiq2013</span><span class="o">.</span><span class="n">zip</span>
-    <span class="n">INTO</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">pgloader</span>
-    <span class="n">WITH</span> <span class="n">truncate</span><span class="p">,</span> <span class="n">create</span> <span class="n">table</span>
-     <span class="n">SET</span> <span class="n">client_encoding</span> <span class="n">TO</span> <span class="s1">&#39;latin1&#39;</span><span class="p">;</span>
-</pre></div>
-</div>
-<p>Note that here pgloader will benefit from the meta-data information found in
-the dBase file to create a PostgreSQL table capable of hosting the data as
-described, then load the data.</p>
-</div>
-<div class="section" id="loading-the-data">
-<h2>Loading the data<a class="headerlink" href="#loading-the-data" title="Permalink to this headline">¶</a></h2>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>dbf-zip.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader dbf-zip.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/dbf-zip.load&quot;
-... LOG Fetching &#39;http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/historiq2013.zip&#39;
-... LOG Extracting files from archive &#39;//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//historiq2013.zip&#39;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-         download          0          0          0          0.167s
-          extract          0          0          0          1.010s
- create, truncate          0          0          0          0.071s
------------------  ---------  ---------  ---------  --------------
-     historiq2013       9181       9181          0          0.658s
------------------  ---------  ---------  ---------  --------------
-Total import time       9181       9181          0          1.906s
-</pre></div>
-</div>
-<p>We can see that <a class="reference external" href="http://pgloader.io">pgloader</a> did download the file from
-its HTTP URL location then <em>unziped</em> it before the loading itself.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/dBase.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/fixed.html

@@ -1,176 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading Fixed Width Data File with pgloader &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-fixed-width-data-file-with-pgloader">
-<h1>Loading Fixed Width Data File with pgloader<a class="headerlink" href="#loading-fixed-width-data-file-with-pgloader" title="Permalink to this headline">¶</a></h1>
-<p>Some data providers still use a format where each column is specified with a
-starting index position and a given length. Usually the columns are
-blank-padded when the data is shorter than the full reserved range.</p>
-<div class="section" id="the-command">
-<h2>The Command<a class="headerlink" href="#the-command" title="Permalink to this headline">¶</a></h2>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our example for loading
-Fixed Width Data, using a file provided by the US census.</p>
-<p>You can find more files from them at the
-[Census 2000 Gazetteer Files](<a class="reference external" href="http://www.census.gov/geo/maps-data/data/gazetteer2000.html">http://www.census.gov/geo/maps-data/data/gazetteer2000.html</a>).</p>
-<p>Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD ARCHIVE
-   FROM http://www.census.gov/geo/maps-data/data/docs/gazetteer/places2k.zip
-   INTO postgresql:///pgloader
-
-   BEFORE LOAD DO
-     $$ drop table if exists places; $$,
-     $$ create table places
-       (
-          usps      char(2)  not null,
-          fips      char(2)  not null,
-          fips_code char(5),
-          loc_name  varchar(64)
-       );
-     $$
-
-   LOAD FIXED
-        FROM FILENAME MATCHING ~/places2k.txt/
-             WITH ENCODING latin1
-             (
-                usps           from   0 for  2,
-                fips           from   2 for  2,
-                fips_code      from   4 for  5,
-                &quot;LocationName&quot; from   9 for 64 [trim right whitespace],
-                p              from  73 for  9,
-                h              from  82 for  9,
-                land           from  91 for 14,
-                water          from 105 for 14,
-                ldm            from 119 for 14,
-                wtm            from 131 for 14,
-                lat            from 143 for 10,
-                long           from 153 for 11
-             )
-        INTO postgresql:///pgloader?places
-             (
-            usps, fips, fips_code, &quot;LocationName&quot;
-             );
-</pre></div>
-</div>
-</div>
-<div class="section" id="the-data">
-<h2>The Data<a class="headerlink" href="#the-data" title="Permalink to this headline">¶</a></h2>
-<p>This command allows loading the following file content, where we are only
-showing the first couple of lines:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">AL0100124Abbeville</span> <span class="n">city</span>                                                       <span class="mi">2987</span>     <span class="mi">1353</span>      <span class="mi">40301945</span>        <span class="mi">120383</span>   <span class="mf">15.560669</span>    <span class="mf">0.046480</span> <span class="mf">31.566367</span> <span class="o">-</span><span class="mf">85.251300</span>
-<span class="n">AL0100460Adamsville</span> <span class="n">city</span>                                                      <span class="mi">4965</span>     <span class="mi">2042</span>      <span class="mi">50779330</span>         <span class="mi">14126</span>   <span class="mf">19.606010</span>    <span class="mf">0.005454</span> <span class="mf">33.590411</span> <span class="o">-</span><span class="mf">86.949166</span>
-<span class="n">AL0100484Addison</span> <span class="n">town</span>                                                          <span class="mi">723</span>      <span class="mi">339</span>       <span class="mi">9101325</span>             <span class="mi">0</span>    <span class="mf">3.514041</span>    <span class="mf">0.000000</span> <span class="mf">34.200042</span> <span class="o">-</span><span class="mf">87.177851</span>
-<span class="n">AL0100676Akron</span> <span class="n">town</span>                                                            <span class="mi">521</span>      <span class="mi">239</span>       <span class="mi">1436797</span>             <span class="mi">0</span>    <span class="mf">0.554750</span>    <span class="mf">0.000000</span> <span class="mf">32.876425</span> <span class="o">-</span><span class="mf">87.740978</span>
-<span class="n">AL0100820Alabaster</span> <span class="n">city</span>                                                      <span class="mi">22619</span>     <span class="mi">8594</span>      <span class="mi">53023800</span>        <span class="mi">141711</span>   <span class="mf">20.472605</span>    <span class="mf">0.054715</span> <span class="mf">33.231162</span> <span class="o">-</span><span class="mf">86.823829</span>
-<span class="n">AL0100988Albertville</span> <span class="n">city</span>                                                    <span class="mi">17247</span>     <span class="mi">7090</span>      <span class="mi">67212867</span>        <span class="mi">258738</span>   <span class="mf">25.951034</span>    <span class="mf">0.099899</span> <span class="mf">34.265362</span> <span class="o">-</span><span class="mf">86.211261</span>
-<span class="n">AL0101132Alexander</span> <span class="n">City</span> <span class="n">city</span>                                                 <span class="mi">15008</span>     <span class="mi">6855</span>     <span class="mi">100534344</span>        <span class="mi">433413</span>   <span class="mf">38.816529</span>    <span class="mf">0.167342</span> <span class="mf">32.933157</span> <span class="o">-</span><span class="mf">85.936008</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="loading-the-data">
-<h2>Loading the data<a class="headerlink" href="#loading-the-data" title="Permalink to this headline">¶</a></h2>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>census-places.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader census-places.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/census-places.load&quot;
-... LOG Fetching &#39;http://www.census.gov/geo/maps-data/data/docs/gazetteer/places2k.zip&#39;
-... LOG Extracting files from archive &#39;//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//places2k.zip&#39;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-         download          0          0          0          1.494s
-          extract          0          0          0          1.013s
-      before load          2          2          0          0.013s
------------------  ---------  ---------  ---------  --------------
-           places      25375      25375          0          0.499s
------------------  ---------  ---------  ---------  --------------
-Total import time      25375      25375          0          3.019s
-</pre></div>
-</div>
-<p>We can see that pgloader did download the file from its HTTP URL location
-then <em>unziped</em> it before the loading itself.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/fixed.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/geolite.html

@@ -1,239 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading MaxMind Geolite Data with pgloader &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-maxmind-geolite-data-with-pgloader">
-<h1>Loading MaxMind Geolite Data with pgloader<a class="headerlink" href="#loading-maxmind-geolite-data-with-pgloader" title="Permalink to this headline">¶</a></h1>
-<p><a class="reference external" href="http://www.maxmind.com/">MaxMind</a> provides a free dataset for
-geolocation, which is quite popular. Using pgloader you can download the
-lastest version of it, extract the CSV files from the archive and load their
-content into your database directly.</p>
-<div class="section" id="the-command">
-<h2>The Command<a class="headerlink" href="#the-command" title="Permalink to this headline">¶</a></h2>
-<p>To load data with pgloader you need to define in a <em>command</em> the operations
-in some details. Here’s our example for loading the Geolite data:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>/*
- * Loading from a ZIP archive containing CSV files. The full test can be
- * done with using the archive found at
- * http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip
- *
- * And a very light version of this data set is found at
- * http://pgsql.tapoueh.org/temp/foo.zip for quick testing.
- */
-
-LOAD ARCHIVE
-   FROM http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip
-   INTO postgresql:///ip4r
-
-   BEFORE LOAD DO
-     $$ create extension if not exists ip4r; $$,
-     $$ create schema if not exists geolite; $$,
-     $$ create table if not exists geolite.location
-       (
-          locid      integer primary key,
-          country    text,
-          region     text,
-          city       text,
-          postalcode text,
-          location   point,
-          metrocode  text,
-          areacode   text
-       );
-     $$,
-     $$ create table if not exists geolite.blocks
-       (
-          iprange    ip4r,
-          locid      integer
-       );
-     $$,
-     $$ drop index if exists geolite.blocks_ip4r_idx; $$,
-     $$ truncate table geolite.blocks, geolite.location cascade; $$
-
-   LOAD CSV
-        FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
-             WITH ENCODING iso-8859-1
-             (
-                locId,
-                country,
-                region     null if blanks,
-                city       null if blanks,
-                postalCode null if blanks,
-                latitude,
-                longitude,
-                metroCode  null if blanks,
-                areaCode   null if blanks
-             )
-        INTO postgresql:///ip4r?geolite.location
-             (
-                locid,country,region,city,postalCode,
-                location point using (format nil &quot;(~a,~a)&quot; longitude latitude),
-                metroCode,areaCode
-             )
-        WITH skip header = 2,
-             fields optionally enclosed by &#39;&quot;&#39;,
-             fields escaped by double-quote,
-             fields terminated by &#39;,&#39;
-
-  AND LOAD CSV
-        FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
-             WITH ENCODING iso-8859-1
-             (
-                startIpNum, endIpNum, locId
-             )
-        INTO postgresql:///ip4r?geolite.blocks
-             (
-                iprange ip4r using (ip-range startIpNum endIpNum),
-                locId
-             )
-        WITH skip header = 2,
-             fields optionally enclosed by &#39;&quot;&#39;,
-             fields escaped by double-quote,
-             fields terminated by &#39;,&#39;
-
-   FINALLY DO
-     $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;
-</pre></div>
-</div>
-<p>Note that while the <em>Geolite</em> data is using a pair of integers (<em>start</em>,
-<em>end</em>) to represent <em>ipv4</em> data, we use the very poweful <a class="reference external" href="https://github.com/RhodiumToad/ip4r">ip4r</a> PostgreSQL Extension instead.</p>
-<p>The transformation from a pair of integers into an IP is done dynamically by
-the pgloader process.</p>
-<p>Also, the location is given as a pair of <em>float</em> columns for the <em>longitude</em>
-and the <em>latitude</em> where PostgreSQL offers the
-<a class="reference external" href="http://www.postgresql.org/docs/9.3/interactive/functions-geometry.html">point</a>
-datatype, so the pgloader command here will actually transform the data on
-the fly to use the appropriate data type and its input representation.</p>
-</div>
-<div class="section" id="loading-the-data">
-<h2>Loading the data<a class="headerlink" href="#loading-the-data" title="Permalink to this headline">¶</a></h2>
-<p>Here’s how to start loading the data. Note that the ouput here has been
-edited so as to facilitate its browsing online:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader archive.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/archive.load&quot;
-... LOG Fetching &#39;http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip&#39;
-... LOG Extracting files from archive &#39;//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//GeoLiteCity-latest.zip&#39;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-         download          0          0          0         11.592s
-          extract          0          0          0          1.012s
-      before load          6          6          0          0.019s
------------------  ---------  ---------  ---------  --------------
- geolite.location     470387     470387          0          7.743s
-   geolite.blocks    1903155    1903155          0         16.332s
------------------  ---------  ---------  ---------  --------------
-          finally          1          1          0         31.692s
------------------  ---------  ---------  ---------  --------------
-Total import time    2373542    2373542          0        1m8.390s
-</pre></div>
-</div>
-<p>The timing of course includes the transformation of the <em>1.9 million</em> pairs
-of integer into a single <em>ipv4 range</em> each. The <em>finally</em> step consists of
-creating the <em>GiST</em> specialized index as given in the main command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">CREATE</span> <span class="n">INDEX</span> <span class="n">blocks_ip4r_idx</span> <span class="n">ON</span> <span class="n">geolite</span><span class="o">.</span><span class="n">blocks</span> <span class="n">USING</span> <span class="n">gist</span><span class="p">(</span><span class="n">iprange</span><span class="p">);</span>
-</pre></div>
-</div>
-<p>That index will then be used to speed up queries wanting to find which
-recorded geolocation contains a specific IP address:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">ip4r</span><span class="o">&gt;</span> <span class="n">select</span> <span class="o">*</span>
-        <span class="kn">from</span>      <span class="nn">geolite.location</span> <span class="n">l</span>
-             <span class="n">join</span> <span class="n">geolite</span><span class="o">.</span><span class="n">blocks</span> <span class="n">b</span> <span class="n">using</span><span class="p">(</span><span class="n">locid</span><span class="p">)</span>
-       <span class="n">where</span> <span class="n">iprange</span> <span class="o">&gt;&gt;=</span> <span class="s1">&#39;8.8.8.8&#39;</span><span class="p">;</span>
-
-<span class="o">-</span><span class="p">[</span> <span class="n">RECORD</span> <span class="mi">1</span> <span class="p">]</span><span class="o">------------------</span>
-<span class="n">locid</span>      <span class="o">|</span> <span class="mi">223</span>
-<span class="n">country</span>    <span class="o">|</span> <span class="n">US</span>
-<span class="n">region</span>     <span class="o">|</span>
-<span class="n">city</span>       <span class="o">|</span>
-<span class="n">postalcode</span> <span class="o">|</span>
-<span class="n">location</span>   <span class="o">|</span> <span class="p">(</span><span class="o">-</span><span class="mi">97</span><span class="p">,</span><span class="mi">38</span><span class="p">)</span>
-<span class="n">metrocode</span>  <span class="o">|</span>
-<span class="n">areacode</span>   <span class="o">|</span>
-<span class="n">iprange</span>    <span class="o">|</span> <span class="mf">8.8</span><span class="o">.</span><span class="mf">8.8</span><span class="o">-</span><span class="mf">8.8</span><span class="o">.</span><span class="mf">37.255</span>
-
-<span class="n">Time</span><span class="p">:</span> <span class="mf">0.747</span> <span class="n">ms</span>
-</pre></div>
-</div>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/geolite.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/mysql.html

@@ -1,246 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Migrating from MySQL to PostgreSQL &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="migrating-from-mysql-to-postgresql">
-<h1>Migrating from MySQL to PostgreSQL<a class="headerlink" href="#migrating-from-mysql-to-postgresql" title="Permalink to this headline">¶</a></h1>
-<p>If you want to migrate your data over to <a class="reference external" href="http://www.postgresql.org">PostgreSQL</a> from MySQL then pgloader is the tool of
-choice!</p>
-<p>Most tools around are skipping the main problem with migrating from MySQL,
-which is to do with the type casting and data sanitizing that needs to be
-done. pgloader will not leave you alone on those topics.</p>
-<div class="section" id="in-a-single-command-line">
-<h2>In a Single Command Line<a class="headerlink" href="#in-a-single-command-line" title="Permalink to this headline">¶</a></h2>
-<p>As an example, we will use the f1db database from &lt;<a class="reference external" href="http://ergast.com/mrd/">http://ergast.com/mrd/</a>&gt;
-which which provides a historical record of motor racing data for
-non-commercial purposes. You can either use their API or download the whole
-database at <a class="reference external" href="http://ergast.com/downloads/f1db.sql.gz">http://ergast.com/downloads/f1db.sql.gz</a>. Once you’ve done that load the
-database in MySQL:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ mysql -u root
-&gt; create database f1db;
-&gt; source f1db.sql
-</pre></div>
-</div>
-<p>Now let’s migrate this database into PostgreSQL in a single command line:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ createdb f1db
-$ pgloader mysql://root@localhost/f1db pgsql:///f1db
-</pre></div>
-</div>
-<p>Done! All with schema, table definitions, constraints, indexes, primary
-keys, <em>auto_increment</em> columns turned into <em>bigserial</em> , foreign keys,
-comments, and if you had some MySQL default values such as <em>ON UPDATE
-CURRENT_TIMESTAMP</em> they would have been translated to a <a class="reference external" href="https://www.postgresql.org/docs/current/static/plpgsql-trigger.html">PostgreSQL before
-update trigger</a>
-automatically.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader mysql://root@localhost/f1db pgsql:///f1db
-2017-06-16T08:56:14.064000+02:00 LOG Main logs in &#39;/private/tmp/pgloader/pgloader.log&#39;
-2017-06-16T08:56:14.068000+02:00 LOG Data errors in &#39;/private/tmp/pgloader/&#39;
-2017-06-16T08:56:19.542000+02:00 LOG report summary reset
-               table name       read   imported     errors      total time
--------------------------  ---------  ---------  ---------  --------------
-          fetch meta data         33         33          0          0.365s
-           Create Schemas          0          0          0          0.007s
-         Create SQL Types          0          0          0          0.006s
-            Create tables         26         26          0          0.068s
-           Set Table OIDs         13         13          0          0.012s
--------------------------  ---------  ---------  ---------  --------------
-  f1db.constructorresults      11011      11011          0          0.205s
-            f1db.circuits         73         73          0          0.150s
-        f1db.constructors        208        208          0          0.059s
-f1db.constructorstandings      11766      11766          0          0.365s
-             f1db.drivers        841        841          0          0.268s
-            f1db.laptimes     413578     413578          0          2.892s
-     f1db.driverstandings      31420      31420          0          0.583s
-            f1db.pitstops       5796       5796          0          2.154s
-               f1db.races        976        976          0          0.227s
-          f1db.qualifying       7257       7257          0          0.228s
-             f1db.seasons         68         68          0          0.527s
-             f1db.results      23514      23514          0          0.658s
-              f1db.status        133        133          0          0.130s
--------------------------  ---------  ---------  ---------  --------------
-  COPY Threads Completion         39         39          0          4.303s
-           Create Indexes         20         20          0          1.497s
-   Index Build Completion         20         20          0          0.214s
-          Reset Sequences          0         10          0          0.058s
-             Primary Keys         13         13          0          0.012s
-      Create Foreign Keys          0          0          0          0.000s
-          Create Triggers          0          0          0          0.001s
-         Install Comments          0          0          0          0.000s
--------------------------  ---------  ---------  ---------  --------------
-        Total import time     506641     506641          0          5.547s
-</pre></div>
-</div>
-<p>You may need to have special cases to take care of tho, or views that you
-want to materialize while doing the migration. In advanced case you can use
-the pgloader command.</p>
-</div>
-<div class="section" id="the-command">
-<h2>The Command<a class="headerlink" href="#the-command" title="Permalink to this headline">¶</a></h2>
-<p>To load data with pgloader you need to define in a <em>command</em> the operations
-in some details. Here’s our example for loading the <a class="reference external" href="http://dev.mysql.com/doc/sakila/en/">MySQL Sakila Sample
-Database</a>.</p>
-<p>Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>load database
-     from      mysql://root@localhost/sakila
-     into postgresql:///sakila
-
- WITH include drop, create tables, no truncate,
-      create indexes, reset sequences, foreign keys
-
-  SET maintenance_work_mem to &#39;128MB&#39;, work_mem to &#39;12MB&#39;, search_path to &#39;sakila&#39;
-
- CAST type datetime to timestamptz
-                drop default drop not null using zero-dates-to-null,
-      type date drop not null drop default using zero-dates-to-null
-
- MATERIALIZE VIEWS film_list, staff_list
-
- -- INCLUDING ONLY TABLE NAMES MATCHING ~/film/, &#39;actor&#39;
- -- EXCLUDING TABLE NAMES MATCHING ~&lt;ory&gt;
-
- BEFORE LOAD DO
- $$ create schema if not exists sakila; $$;
-</pre></div>
-</div>
-<p>Note that here pgloader will benefit from the meta-data information found in
-the MySQL database to create a PostgreSQL database capable of hosting the
-data as described, then load the data.</p>
-<p>In particular, some specific <em>casting rules</em> are given here, to cope with
-date values such as <cite>0000-00-00</cite> that MySQL allows and PostgreSQL rejects
-for not existing in our calendar. It’s possible to add per-column casting
-rules too, which is useful is some of your <cite>tinyint</cite> are in fact <cite>smallint</cite>
-while some others are in fact <cite>boolean</cite> values.</p>
-<p>Finaly note that we are using the <em>MATERIALIZE VIEWS</em> clause of pgloader:
-the selected views here will be migrated over to PostgreSQL <em>with their
-contents</em>.</p>
-<p>It’s possible to use the <em>MATERIALIZE VIEWS</em> clause and give both the name
-and the SQL (in MySQL dialect) definition of view, then pgloader creates the
-view before loading the data, then drops it again at the end.</p>
-<p>## Loading the data</p>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>sakila.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader sakila.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/sakila.load&quot;
-   &lt;WARNING: table &quot;xxx&quot; does not exists have been edited away&gt;
-
-            table name       read   imported     errors            time
-----------------------  ---------  ---------  ---------  --------------
-           before load          1          1          0          0.007s
-       fetch meta data         45         45          0          0.402s
-          create, drop          0         36          0          0.208s
-----------------------  ---------  ---------  ---------  --------------
-                 actor        200        200          0          0.071s
-               address        603        603          0          0.035s
-              category         16         16          0          0.018s
-                  city        600        600          0          0.037s
-               country        109        109          0          0.023s
-              customer        599        599          0          0.073s
-                  film       1000       1000          0          0.135s
-            film_actor       5462       5462          0          0.236s
-         film_category       1000       1000          0          0.070s
-             film_text       1000       1000          0          0.080s
-             inventory       4581       4581          0          0.136s
-              language          6          6          0          0.036s
-               payment      16049      16049          0          0.539s
-                rental      16044      16044          0          0.648s
-                 staff          2          2          0          0.041s
-                 store          2          2          0          0.036s
-             film_list        997        997          0          0.247s
-            staff_list          2          2          0          0.135s
-Index Build Completion          0          0          0          0.000s
-----------------------  ---------  ---------  ---------  --------------
-        Create Indexes         41         41          0          0.964s
-       Reset Sequences          0          1          0          0.035s
-          Foreign Keys         22         22          0          0.254s
-----------------------  ---------  ---------  ---------  --------------
-     Total import time      48272      48272          0          3.502s
-</pre></div>
-</div>
-<p>The <em>WARNING</em> messages we see here are expected as the PostgreSQL database
-is empty when running the command, and pgloader is using the SQL commands
-<cite>DROP TABLE IF EXISTS</cite> when the given command uses the <cite>include drop</cite>
-option.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/mysql.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/quickstart.html

@@ -1,218 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>PgLoader Quick Start &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="pgloader-quick-start">
-<h1>PgLoader Quick Start<a class="headerlink" href="#pgloader-quick-start" title="Permalink to this headline">¶</a></h1>
-<p>In simple cases, pgloader is very easy to use.</p>
-<div class="section" id="csv">
-<h2>CSV<a class="headerlink" href="#csv" title="Permalink to this headline">¶</a></h2>
-<p>Load data from a CSV file into a pre-existing table in your database:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                   \
-         --field id --field field                     \
-         --with truncate                              \
-         --with &quot;fields terminated by &#39;,&#39;&quot;            \
-         ./test/data/matching-1.csv                   \
-         postgres:///pgloader?tablename=matching
-</pre></div>
-</div>
-<p>In that example the whole loading is driven from the command line, bypassing
-the need for writing a command in the pgloader command syntax entirely. As
-there’s no command though, the extra information needed must be provided on
-the command line using the <cite>–type</cite> and <cite>–field</cite> and <cite>–with</cite> switches.</p>
-<p>For documentation about the available syntaxes for the <cite>–field</cite> and
-<cite>–with</cite> switches, please refer to the CSV section later in the man page.</p>
-<p>Note also that the PostgreSQL URI includes the target <em>tablename</em>.</p>
-</div>
-<div class="section" id="reading-from-stdin">
-<h2>Reading from STDIN<a class="headerlink" href="#reading-from-stdin" title="Permalink to this headline">¶</a></h2>
-<p>File based pgloader sources can be loaded from the standard input, as in the
-following example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                         \
-         --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot; \
-         --with &quot;skip header = 1&quot;                          \
-         --with &quot;fields terminated by &#39;\t&#39;&quot;                \
-         -                                                 \
-         postgresql:///pgloader?districts_longlat          \
-         &lt; test/data/2013_Gaz_113CDs_national.txt
-</pre></div>
-</div>
-<p>The dash (<cite>-</cite>) character as a source is used to mean <em>standard input</em>, as
-usual in Unix command lines. It’s possible to stream compressed content to
-pgloader with this technique, using the Unix pipe:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>gunzip -c source.gz | pgloader --type csv ... - pgsql:///target?foo
-</pre></div>
-</div>
-</div>
-<div class="section" id="loading-from-csv-available-through-http">
-<h2>Loading from CSV available through HTTP<a class="headerlink" href="#loading-from-csv-available-through-http" title="Permalink to this headline">¶</a></h2>
-<p>The same command as just above can also be run if the CSV file happens to be
-found on a remote HTTP location:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                                     \
-         --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot; \
-         --with &quot;skip header = 1&quot;                                       \
-         --with &quot;fields terminated by &#39;\t&#39;&quot;                             \
-         http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt     \
-         postgresql:///pgloader?districts_longlat
-</pre></div>
-</div>
-<p>Some more options have to be used in that case, as the file contains a
-one-line header (most commonly that’s column names, could be a copyright
-notice). Also, in that case, we specify all the fields right into a single
-<cite>–field</cite> option argument.</p>
-<p>Again, the PostgreSQL target connection string must contain the <em>tablename</em>
-option and you have to ensure that the target table exists and may fit the
-data. Here’s the SQL command used in that example in case you want to try it
-yourself:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">create</span> <span class="n">table</span> <span class="n">districts_longlat</span>
-<span class="p">(</span>
-         <span class="n">usps</span>        <span class="n">text</span><span class="p">,</span>
-         <span class="n">geoid</span>       <span class="n">text</span><span class="p">,</span>
-         <span class="n">aland</span>       <span class="n">bigint</span><span class="p">,</span>
-         <span class="n">awater</span>      <span class="n">bigint</span><span class="p">,</span>
-         <span class="n">aland_sqmi</span>  <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">awater_sqmi</span> <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">intptlat</span>    <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">intptlong</span>   <span class="n">double</span> <span class="n">precision</span>
-<span class="p">);</span>
-</pre></div>
-</div>
-<p>Also notice that the same command will work against an archived version of
-the same data.</p>
-</div>
-<div class="section" id="streaming-csv-data-from-an-http-compressed-file">
-<h2>Streaming CSV data from an HTTP compressed file<a class="headerlink" href="#streaming-csv-data-from-an-http-compressed-file" title="Permalink to this headline">¶</a></h2>
-<p>Finally, it’s important to note that pgloader first fetches the content from
-the HTTP URL it to a local file, then expand the archive when it’s
-recognized to be one, and only then processes the locally expanded file.</p>
-<p>In some cases, either because pgloader has no direct support for your
-archive format or maybe because expanding the archive is not feasible in
-your environment, you might want to <em>stream</em> the content straight from its
-remote location into PostgreSQL. Here’s how to do that, using the old battle
-tested Unix Pipes trick:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>curl http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz \
-| gunzip -c                                                        \
-| pgloader --type csv                                              \
-           --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot;
-           --with &quot;skip header = 1&quot;                                \
-           --with &quot;fields terminated by &#39;\t&#39;&quot;                      \
-           -                                                       \
-           postgresql:///pgloader?districts_longlat
-</pre></div>
-</div>
-<p>Now the OS will take care of the streaming and buffering between the network
-and the commands and pgloader will take care of streaming the data down to
-PostgreSQL.</p>
-</div>
-<div class="section" id="migrating-from-sqlite">
-<h2>Migrating from SQLite<a class="headerlink" href="#migrating-from-sqlite" title="Permalink to this headline">¶</a></h2>
-<p>The following command will open the SQLite database, discover its tables
-definitions including indexes and foreign keys, migrate those definitions
-while <em>casting</em> the data type specifications to their PostgreSQL equivalent
-and then migrate the data over:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">newdb</span>
-<span class="n">pgloader</span> <span class="o">./</span><span class="n">test</span><span class="o">/</span><span class="n">sqlite</span><span class="o">/</span><span class="n">sqlite</span><span class="o">.</span><span class="n">db</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">newdb</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="migrating-from-mysql">
-<h2>Migrating from MySQL<a class="headerlink" href="#migrating-from-mysql" title="Permalink to this headline">¶</a></h2>
-<p>Just create a database where to host the MySQL data and definitions and have
-pgloader do the migration for you in a single command line:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">pagila</span>
-<span class="n">pgloader</span> <span class="n">mysql</span><span class="p">:</span><span class="o">//</span><span class="n">user</span><span class="nd">@localhost</span><span class="o">/</span><span class="n">sakila</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">pagila</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="fetching-an-archived-dbf-file-from-a-http-remote-location">
-<h2>Fetching an archived DBF file from a HTTP remote location<a class="headerlink" href="#fetching-an-archived-dbf-file-from-a-http-remote-location" title="Permalink to this headline">¶</a></h2>
-<p>It’s possible for pgloader to download a file from HTTP, unarchive it, and
-only then open it to discover the schema then load the data:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">foo</span>
-<span class="n">pgloader</span> <span class="o">--</span><span class="nb">type</span> <span class="n">dbf</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">insee</span><span class="o">.</span><span class="n">fr</span><span class="o">/</span><span class="n">fr</span><span class="o">/</span><span class="n">methodes</span><span class="o">/</span><span class="n">nomenclatures</span><span class="o">/</span><span class="n">cog</span><span class="o">/</span><span class="n">telechargement</span><span class="o">/</span><span class="mi">2013</span><span class="o">/</span><span class="n">dbf</span><span class="o">/</span><span class="n">historiq2013</span><span class="o">.</span><span class="n">zip</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">foo</span>
-</pre></div>
-</div>
-<p>Here it’s not possible for pgloader to guess the kind of data source it’s
-being given, so it’s necessary to use the <cite>–type</cite> command line switch.</p>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/quickstart.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/sqlite.html

@@ -1,208 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>Loading SQLite files with pgloader &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="loading-sqlite-files-with-pgloader">
-<h1>Loading SQLite files with pgloader<a class="headerlink" href="#loading-sqlite-files-with-pgloader" title="Permalink to this headline">¶</a></h1>
-<p>The SQLite database is a respected solution to manage your data with. Its
-embeded nature makes it a source of migrations when a projects now needs to
-handle more concurrency, which [PostgreSQL](<a class="reference external" href="http://www.postgresql.org/">http://www.postgresql.org/</a>) is
-very good at. pgloader can help you there.</p>
-<div class="section" id="in-a-single-command-line">
-<h2>In a Single Command Line<a class="headerlink" href="#in-a-single-command-line" title="Permalink to this headline">¶</a></h2>
-<p>You can</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ createdb chinook
-$ pgloader https://github.com/lerocha/chinook-database/raw/master/ChinookDatabase/DataSources/Chinook_Sqlite_AutoIncrementPKs.sqlite pgsql:///chinook
-</pre></div>
-</div>
-<p>Done! All with the schema, data, constraints, primary keys and foreign keys,
-etc. We also see an error with the Chinook schema that contains several
-primary key definitions against the same table, which is not accepted by
-PostgreSQL:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mf">59.019000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">LOG</span> <span class="n">Data</span> <span class="n">errors</span> <span class="ow">in</span> <span class="s1">&#39;/private/tmp/pgloader/&#39;</span>
-<span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mf">59.236000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">LOG</span> <span class="n">Fetching</span> <span class="s1">&#39;https://github.com/lerocha/chinook-database/raw/master/ChinookDatabase/DataSources/Chinook_Sqlite_AutoIncrementPKs.sqlite&#39;</span>
-<span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">19</span><span class="p">:</span><span class="mf">00.664000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">ERROR</span> <span class="n">Database</span> <span class="n">error</span> <span class="mi">42</span><span class="n">P16</span><span class="p">:</span> <span class="n">multiple</span> <span class="n">primary</span> <span class="n">keys</span> <span class="k">for</span> <span class="n">table</span> <span class="s2">&quot;playlisttrack&quot;</span> <span class="n">are</span> <span class="ow">not</span> <span class="n">allowed</span>
-<span class="n">QUERY</span><span class="p">:</span> <span class="n">ALTER</span> <span class="n">TABLE</span> <span class="n">playlisttrack</span> <span class="n">ADD</span> <span class="n">PRIMARY</span> <span class="n">KEY</span> <span class="n">USING</span> <span class="n">INDEX</span> <span class="n">idx_66873_sqlite_autoindex_playlisttrack_1</span><span class="p">;</span>
-<span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">19</span><span class="p">:</span><span class="mf">00.665000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">LOG</span> <span class="n">report</span> <span class="n">summary</span> <span class="n">reset</span>
-             <span class="n">table</span> <span class="n">name</span>       <span class="n">read</span>   <span class="n">imported</span>     <span class="n">errors</span>      <span class="n">total</span> <span class="n">time</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-                  <span class="n">fetch</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.877</span><span class="n">s</span>
-        <span class="n">fetch</span> <span class="n">meta</span> <span class="n">data</span>         <span class="mi">33</span>         <span class="mi">33</span>          <span class="mi">0</span>          <span class="mf">0.033</span><span class="n">s</span>
-         <span class="n">Create</span> <span class="n">Schemas</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.003</span><span class="n">s</span>
-       <span class="n">Create</span> <span class="n">SQL</span> <span class="n">Types</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.006</span><span class="n">s</span>
-          <span class="n">Create</span> <span class="n">tables</span>         <span class="mi">22</span>         <span class="mi">22</span>          <span class="mi">0</span>          <span class="mf">0.043</span><span class="n">s</span>
-         <span class="n">Set</span> <span class="n">Table</span> <span class="n">OIDs</span>         <span class="mi">11</span>         <span class="mi">11</span>          <span class="mi">0</span>          <span class="mf">0.012</span><span class="n">s</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-                  <span class="n">album</span>        <span class="mi">347</span>        <span class="mi">347</span>          <span class="mi">0</span>          <span class="mf">0.023</span><span class="n">s</span>
-                 <span class="n">artist</span>        <span class="mi">275</span>        <span class="mi">275</span>          <span class="mi">0</span>          <span class="mf">0.023</span><span class="n">s</span>
-               <span class="n">customer</span>         <span class="mi">59</span>         <span class="mi">59</span>          <span class="mi">0</span>          <span class="mf">0.021</span><span class="n">s</span>
-               <span class="n">employee</span>          <span class="mi">8</span>          <span class="mi">8</span>          <span class="mi">0</span>          <span class="mf">0.018</span><span class="n">s</span>
-                <span class="n">invoice</span>        <span class="mi">412</span>        <span class="mi">412</span>          <span class="mi">0</span>          <span class="mf">0.031</span><span class="n">s</span>
-                  <span class="n">genre</span>         <span class="mi">25</span>         <span class="mi">25</span>          <span class="mi">0</span>          <span class="mf">0.021</span><span class="n">s</span>
-            <span class="n">invoiceline</span>       <span class="mi">2240</span>       <span class="mi">2240</span>          <span class="mi">0</span>          <span class="mf">0.034</span><span class="n">s</span>
-              <span class="n">mediatype</span>          <span class="mi">5</span>          <span class="mi">5</span>          <span class="mi">0</span>          <span class="mf">0.025</span><span class="n">s</span>
-          <span class="n">playlisttrack</span>       <span class="mi">8715</span>       <span class="mi">8715</span>          <span class="mi">0</span>          <span class="mf">0.040</span><span class="n">s</span>
-               <span class="n">playlist</span>         <span class="mi">18</span>         <span class="mi">18</span>          <span class="mi">0</span>          <span class="mf">0.016</span><span class="n">s</span>
-                  <span class="n">track</span>       <span class="mi">3503</span>       <span class="mi">3503</span>          <span class="mi">0</span>          <span class="mf">0.111</span><span class="n">s</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-<span class="n">COPY</span> <span class="n">Threads</span> <span class="n">Completion</span>         <span class="mi">33</span>         <span class="mi">33</span>          <span class="mi">0</span>          <span class="mf">0.313</span><span class="n">s</span>
-         <span class="n">Create</span> <span class="n">Indexes</span>         <span class="mi">22</span>         <span class="mi">22</span>          <span class="mi">0</span>          <span class="mf">0.160</span><span class="n">s</span>
- <span class="n">Index</span> <span class="n">Build</span> <span class="n">Completion</span>         <span class="mi">22</span>         <span class="mi">22</span>          <span class="mi">0</span>          <span class="mf">0.027</span><span class="n">s</span>
-        <span class="n">Reset</span> <span class="n">Sequences</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.017</span><span class="n">s</span>
-           <span class="n">Primary</span> <span class="n">Keys</span>         <span class="mi">12</span>          <span class="mi">0</span>          <span class="mi">1</span>          <span class="mf">0.013</span><span class="n">s</span>
-    <span class="n">Create</span> <span class="n">Foreign</span> <span class="n">Keys</span>         <span class="mi">11</span>         <span class="mi">11</span>          <span class="mi">0</span>          <span class="mf">0.040</span><span class="n">s</span>
-        <span class="n">Create</span> <span class="n">Triggers</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.000</span><span class="n">s</span>
-       <span class="n">Install</span> <span class="n">Comments</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.000</span><span class="n">s</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-      <span class="n">Total</span> <span class="kn">import</span> <span class="nn">time</span>      <span class="mi">15607</span>      <span class="mi">15607</span>          <span class="mi">0</span>          <span class="mf">1.669</span><span class="n">s</span>
-</pre></div>
-</div>
-<p>You may need to have special cases to take care of tho. In advanced case you
-can use the pgloader command.</p>
-</div>
-<div class="section" id="the-command">
-<h2>The Command<a class="headerlink" href="#the-command" title="Permalink to this headline">¶</a></h2>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">load</span> <span class="n">database</span>
-     <span class="kn">from</span> <span class="s1">&#39;sqlite/Chinook_Sqlite_AutoIncrementPKs.sqlite&#39;</span>
-     <span class="n">into</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">pgloader</span>
-
- <span class="k">with</span> <span class="n">include</span> <span class="n">drop</span><span class="p">,</span> <span class="n">create</span> <span class="n">tables</span><span class="p">,</span> <span class="n">create</span> <span class="n">indexes</span><span class="p">,</span> <span class="n">reset</span> <span class="n">sequences</span>
-
-  <span class="nb">set</span> <span class="n">work_mem</span> <span class="n">to</span> <span class="s1">&#39;16MB&#39;</span><span class="p">,</span> <span class="n">maintenance_work_mem</span> <span class="n">to</span> <span class="s1">&#39;512 MB&#39;</span><span class="p">;</span>
-</pre></div>
-</div>
-<p>Note that here pgloader will benefit from the meta-data information found in
-the SQLite file to create a PostgreSQL database capable of hosting the data
-as described, then load the data.</p>
-</div>
-<div class="section" id="loading-the-data">
-<h2>Loading the data<a class="headerlink" href="#loading-the-data" title="Permalink to this headline">¶</a></h2>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>sqlite.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader sqlite.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/sqlite.load&quot;
-... WARNING Postgres warning: table &quot;album&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;artist&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;customer&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;employee&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;genre&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;invoice&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;invoiceline&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;mediatype&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;playlist&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;playlisttrack&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;track&quot; does not exist, skipping
-            table name       read   imported     errors            time
-----------------------  ---------  ---------  ---------  --------------
-      create, truncate          0          0          0          0.052s
-                 Album        347        347          0          0.070s
-                Artist        275        275          0          0.014s
-              Customer         59         59          0          0.014s
-              Employee          8          8          0          0.012s
-                 Genre         25         25          0          0.018s
-               Invoice        412        412          0          0.032s
-           InvoiceLine       2240       2240          0          0.077s
-             MediaType          5          5          0          0.012s
-              Playlist         18         18          0          0.008s
-         PlaylistTrack       8715       8715          0          0.071s
-                 Track       3503       3503          0          0.105s
-index build completion          0          0          0          0.000s
-----------------------  ---------  ---------  ---------  --------------
-        Create Indexes         20         20          0          0.279s
-       reset sequences          0          0          0          0.043s
-----------------------  ---------  ---------  ---------  --------------
-  Total streaming time      15607      15607          0          0.476s
-</pre></div>
-</div>
-<p>We can see that <a class="reference external" href="http://pgloader.io">pgloader</a> did download the file from
-its HTTP URL location then <em>unziped</em> it before loading it.</p>
-<p>Also, the <em>WARNING</em> messages we see here are expected as the PostgreSQL
-database is empty when running the command, and pgloader is using the SQL
-commands <cite>DROP TABLE IF EXISTS</cite> when the given command uses the <cite>include
-drop</cite> option.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/sqlite.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>

docs/_build/html/tutorial/tutorial.html

@@ -1,893 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
-  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
-  <head>
-    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-    <title>PgLoader Tutorial &#8212; pgloader 3.4.1 documentation</title>
-    <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
-    <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
-    <script type="text/javascript">
-      var DOCUMENTATION_OPTIONS = {
-        URL_ROOT:    '../',
-        VERSION:     '3.4.1',
-        COLLAPSE_INDEX: false,
-        FILE_SUFFIX: '.html',
-        HAS_SOURCE:  true,
-        SOURCELINK_SUFFIX: '.txt'
-      };
-    </script>
-    <script type="text/javascript" src="../_static/jquery.js"></script>
-    <script type="text/javascript" src="../_static/underscore.js"></script>
-    <script type="text/javascript" src="../_static/doctools.js"></script>
-    <link rel="index" title="Index" href="../genindex.html" />
-    <link rel="search" title="Search" href="../search.html" />
-    <link rel="next" title="PgLoader Reference Manual" href="../pgloader.html" />
-    <link rel="prev" title="Introduction" href="../intro.html" />
-   
-  <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-  
-  
-  <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
-  </head>
-  <body>
-  
-
-    <div class="document">
-      <div class="documentwrapper">
-        <div class="bodywrapper">
-          <div class="body" role="main">
-            
-  <div class="section" id="pgloader-tutorial">
-<h1>PgLoader Tutorial<a class="headerlink" href="#pgloader-tutorial" title="Permalink to this headline">¶</a></h1>
-<div class="section" id="pgloader-quick-start">
-<h2>PgLoader Quick Start<a class="headerlink" href="#pgloader-quick-start" title="Permalink to this headline">¶</a></h2>
-<p>In simple cases, pgloader is very easy to use.</p>
-<div class="section" id="csv">
-<h3>CSV<a class="headerlink" href="#csv" title="Permalink to this headline">¶</a></h3>
-<p>Load data from a CSV file into a pre-existing table in your database:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                   \
-         --field id --field field                     \
-         --with truncate                              \
-         --with &quot;fields terminated by &#39;,&#39;&quot;            \
-         ./test/data/matching-1.csv                   \
-         postgres:///pgloader?tablename=matching
-</pre></div>
-</div>
-<p>In that example the whole loading is driven from the command line, bypassing
-the need for writing a command in the pgloader command syntax entirely. As
-there’s no command though, the extra information needed must be provided on
-the command line using the <cite>–type</cite> and <cite>–field</cite> and <cite>–with</cite> switches.</p>
-<p>For documentation about the available syntaxes for the <cite>–field</cite> and
-<cite>–with</cite> switches, please refer to the CSV section later in the man page.</p>
-<p>Note also that the PostgreSQL URI includes the target <em>tablename</em>.</p>
-</div>
-<div class="section" id="reading-from-stdin">
-<h3>Reading from STDIN<a class="headerlink" href="#reading-from-stdin" title="Permalink to this headline">¶</a></h3>
-<p>File based pgloader sources can be loaded from the standard input, as in the
-following example:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                         \
-         --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot; \
-         --with &quot;skip header = 1&quot;                          \
-         --with &quot;fields terminated by &#39;\t&#39;&quot;                \
-         -                                                 \
-         postgresql:///pgloader?districts_longlat          \
-         &lt; test/data/2013_Gaz_113CDs_national.txt
-</pre></div>
-</div>
-<p>The dash (<cite>-</cite>) character as a source is used to mean <em>standard input</em>, as
-usual in Unix command lines. It’s possible to stream compressed content to
-pgloader with this technique, using the Unix pipe:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>gunzip -c source.gz | pgloader --type csv ... - pgsql:///target?foo
-</pre></div>
-</div>
-</div>
-<div class="section" id="loading-from-csv-available-through-http">
-<h3>Loading from CSV available through HTTP<a class="headerlink" href="#loading-from-csv-available-through-http" title="Permalink to this headline">¶</a></h3>
-<p>The same command as just above can also be run if the CSV file happens to be
-found on a remote HTTP location:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>pgloader --type csv                                                     \
-         --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot; \
-         --with &quot;skip header = 1&quot;                                       \
-         --with &quot;fields terminated by &#39;\t&#39;&quot;                             \
-         http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt     \
-         postgresql:///pgloader?districts_longlat
-</pre></div>
-</div>
-<p>Some more options have to be used in that case, as the file contains a
-one-line header (most commonly that’s column names, could be a copyright
-notice). Also, in that case, we specify all the fields right into a single
-<cite>–field</cite> option argument.</p>
-<p>Again, the PostgreSQL target connection string must contain the <em>tablename</em>
-option and you have to ensure that the target table exists and may fit the
-data. Here’s the SQL command used in that example in case you want to try it
-yourself:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">create</span> <span class="n">table</span> <span class="n">districts_longlat</span>
-<span class="p">(</span>
-         <span class="n">usps</span>        <span class="n">text</span><span class="p">,</span>
-         <span class="n">geoid</span>       <span class="n">text</span><span class="p">,</span>
-         <span class="n">aland</span>       <span class="n">bigint</span><span class="p">,</span>
-         <span class="n">awater</span>      <span class="n">bigint</span><span class="p">,</span>
-         <span class="n">aland_sqmi</span>  <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">awater_sqmi</span> <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">intptlat</span>    <span class="n">double</span> <span class="n">precision</span><span class="p">,</span>
-         <span class="n">intptlong</span>   <span class="n">double</span> <span class="n">precision</span>
-<span class="p">);</span>
-</pre></div>
-</div>
-<p>Also notice that the same command will work against an archived version of
-the same data.</p>
-</div>
-<div class="section" id="streaming-csv-data-from-an-http-compressed-file">
-<h3>Streaming CSV data from an HTTP compressed file<a class="headerlink" href="#streaming-csv-data-from-an-http-compressed-file" title="Permalink to this headline">¶</a></h3>
-<p>Finally, it’s important to note that pgloader first fetches the content from
-the HTTP URL it to a local file, then expand the archive when it’s
-recognized to be one, and only then processes the locally expanded file.</p>
-<p>In some cases, either because pgloader has no direct support for your
-archive format or maybe because expanding the archive is not feasible in
-your environment, you might want to <em>stream</em> the content straight from its
-remote location into PostgreSQL. Here’s how to do that, using the old battle
-tested Unix Pipes trick:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>curl http://pgsql.tapoueh.org/temp/2013_Gaz_113CDs_national.txt.gz \
-| gunzip -c                                                        \
-| pgloader --type csv                                              \
-           --field &quot;usps,geoid,aland,awater,aland_sqmi,awater_sqmi,intptlat,intptlong&quot;
-           --with &quot;skip header = 1&quot;                                \
-           --with &quot;fields terminated by &#39;\t&#39;&quot;                      \
-           -                                                       \
-           postgresql:///pgloader?districts_longlat
-</pre></div>
-</div>
-<p>Now the OS will take care of the streaming and buffering between the network
-and the commands and pgloader will take care of streaming the data down to
-PostgreSQL.</p>
-</div>
-<div class="section" id="migrating-from-sqlite">
-<h3>Migrating from SQLite<a class="headerlink" href="#migrating-from-sqlite" title="Permalink to this headline">¶</a></h3>
-<p>The following command will open the SQLite database, discover its tables
-definitions including indexes and foreign keys, migrate those definitions
-while <em>casting</em> the data type specifications to their PostgreSQL equivalent
-and then migrate the data over:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">newdb</span>
-<span class="n">pgloader</span> <span class="o">./</span><span class="n">test</span><span class="o">/</span><span class="n">sqlite</span><span class="o">/</span><span class="n">sqlite</span><span class="o">.</span><span class="n">db</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">newdb</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="migrating-from-mysql">
-<h3>Migrating from MySQL<a class="headerlink" href="#migrating-from-mysql" title="Permalink to this headline">¶</a></h3>
-<p>Just create a database where to host the MySQL data and definitions and have
-pgloader do the migration for you in a single command line:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">pagila</span>
-<span class="n">pgloader</span> <span class="n">mysql</span><span class="p">:</span><span class="o">//</span><span class="n">user</span><span class="nd">@localhost</span><span class="o">/</span><span class="n">sakila</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">pagila</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="fetching-an-archived-dbf-file-from-a-http-remote-location">
-<h3>Fetching an archived DBF file from a HTTP remote location<a class="headerlink" href="#fetching-an-archived-dbf-file-from-a-http-remote-location" title="Permalink to this headline">¶</a></h3>
-<p>It’s possible for pgloader to download a file from HTTP, unarchive it, and
-only then open it to discover the schema then load the data:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">createdb</span> <span class="n">foo</span>
-<span class="n">pgloader</span> <span class="o">--</span><span class="nb">type</span> <span class="n">dbf</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">insee</span><span class="o">.</span><span class="n">fr</span><span class="o">/</span><span class="n">fr</span><span class="o">/</span><span class="n">methodes</span><span class="o">/</span><span class="n">nomenclatures</span><span class="o">/</span><span class="n">cog</span><span class="o">/</span><span class="n">telechargement</span><span class="o">/</span><span class="mi">2013</span><span class="o">/</span><span class="n">dbf</span><span class="o">/</span><span class="n">historiq2013</span><span class="o">.</span><span class="n">zip</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">foo</span>
-</pre></div>
-</div>
-<p>Here it’s not possible for pgloader to guess the kind of data source it’s
-being given, so it’s necessary to use the <cite>–type</cite> command line switch.</p>
-</div>
-</div>
-<div class="section" id="loading-csv-data-with-pgloader">
-<h2>Loading CSV Data with pgloader<a class="headerlink" href="#loading-csv-data-with-pgloader" title="Permalink to this headline">¶</a></h2>
-<p>CSV means <em>comma separated values</em> and is often found with quite varying
-specifications. pgloader allows you to describe those specs in its command.</p>
-<div class="section" id="the-command">
-<h3>The Command<a class="headerlink" href="#the-command" title="Permalink to this headline">¶</a></h3>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our example for loading CSV
-data:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD CSV
-     FROM &#39;path/to/file.csv&#39; (x, y, a, b, c, d)
-     INTO postgresql:///pgloader?csv (a, b, d, c)
-
-     WITH truncate,
-          skip header = 1,
-          fields optionally enclosed by &#39;&quot;&#39;,
-          fields escaped by double-quote,
-          fields terminated by &#39;,&#39;
-
-      SET client_encoding to &#39;latin1&#39;,
-          work_mem to &#39;12MB&#39;,
-          standard_conforming_strings to &#39;on&#39;
-
-   BEFORE LOAD DO
-    $$ drop table if exists csv; $$,
-    $$ create table csv (
-        a bigint,
-        b bigint,
-        c char(2),
-        d text
-       );
-  $$;
-</pre></div>
-</div>
-</div>
-<div class="section" id="the-data">
-<h3>The Data<a class="headerlink" href="#the-data" title="Permalink to this headline">¶</a></h3>
-<p>This command allows loading the following CSV file content:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>Header, with a © sign
-&quot;2.6.190.56&quot;,&quot;2.6.190.63&quot;,&quot;33996344&quot;,&quot;33996351&quot;,&quot;GB&quot;,&quot;United Kingdom&quot;
-&quot;3.0.0.0&quot;,&quot;4.17.135.31&quot;,&quot;50331648&quot;,&quot;68257567&quot;,&quot;US&quot;,&quot;United States&quot;
-&quot;4.17.135.32&quot;,&quot;4.17.135.63&quot;,&quot;68257568&quot;,&quot;68257599&quot;,&quot;CA&quot;,&quot;Canada&quot;
-&quot;4.17.135.64&quot;,&quot;4.17.142.255&quot;,&quot;68257600&quot;,&quot;68259583&quot;,&quot;US&quot;,&quot;United States&quot;
-&quot;4.17.143.0&quot;,&quot;4.17.143.15&quot;,&quot;68259584&quot;,&quot;68259599&quot;,&quot;CA&quot;,&quot;Canada&quot;
-&quot;4.17.143.16&quot;,&quot;4.18.32.71&quot;,&quot;68259600&quot;,&quot;68296775&quot;,&quot;US&quot;,&quot;United States&quot;
-</pre></div>
-</div>
-</div>
-<div class="section" id="loading-the-data">
-<h3>Loading the data<a class="headerlink" href="#loading-the-data" title="Permalink to this headline">¶</a></h3>
-<p>Here’s how to start loading the data. Note that the ouput here has been
-edited so as to facilitate its browsing online:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader csv.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/csv.load&quot;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-      before load          2          2          0          0.039s
------------------  ---------  ---------  ---------  --------------
-              csv          6          6          0          0.019s
------------------  ---------  ---------  ---------  --------------
-Total import time          6          6          0          0.058s
-</pre></div>
-</div>
-</div>
-<div class="section" id="the-result">
-<h3>The result<a class="headerlink" href="#the-result" title="Permalink to this headline">¶</a></h3>
-<p>As you can see, the command described above is filtering the input and only
-importing some of the columns from the example data file. Here’s what gets
-loaded in the PostgreSQL database:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pgloader</span><span class="c1"># table csv;</span>
-    <span class="n">a</span>     <span class="o">|</span>    <span class="n">b</span>     <span class="o">|</span> <span class="n">c</span>  <span class="o">|</span>       <span class="n">d</span>
-<span class="o">----------+----------+----+----------------</span>
- <span class="mi">33996344</span> <span class="o">|</span> <span class="mi">33996351</span> <span class="o">|</span> <span class="n">GB</span> <span class="o">|</span> <span class="n">United</span> <span class="n">Kingdom</span>
- <span class="mi">50331648</span> <span class="o">|</span> <span class="mi">68257567</span> <span class="o">|</span> <span class="n">US</span> <span class="o">|</span> <span class="n">United</span> <span class="n">States</span>
- <span class="mi">68257568</span> <span class="o">|</span> <span class="mi">68257599</span> <span class="o">|</span> <span class="n">CA</span> <span class="o">|</span> <span class="n">Canada</span>
- <span class="mi">68257600</span> <span class="o">|</span> <span class="mi">68259583</span> <span class="o">|</span> <span class="n">US</span> <span class="o">|</span> <span class="n">United</span> <span class="n">States</span>
- <span class="mi">68259584</span> <span class="o">|</span> <span class="mi">68259599</span> <span class="o">|</span> <span class="n">CA</span> <span class="o">|</span> <span class="n">Canada</span>
- <span class="mi">68259600</span> <span class="o">|</span> <span class="mi">68296775</span> <span class="o">|</span> <span class="n">US</span> <span class="o">|</span> <span class="n">United</span> <span class="n">States</span>
-<span class="p">(</span><span class="mi">6</span> <span class="n">rows</span><span class="p">)</span>
-</pre></div>
-</div>
-</div>
-</div>
-<div class="section" id="loading-fixed-width-data-file-with-pgloader">
-<h2>Loading Fixed Width Data File with pgloader<a class="headerlink" href="#loading-fixed-width-data-file-with-pgloader" title="Permalink to this headline">¶</a></h2>
-<p>Some data providers still use a format where each column is specified with a
-starting index position and a given length. Usually the columns are
-blank-padded when the data is shorter than the full reserved range.</p>
-<div class="section" id="id1">
-<h3>The Command<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our example for loading
-Fixed Width Data, using a file provided by the US census.</p>
-<p>You can find more files from them at the
-[Census 2000 Gazetteer Files](<a class="reference external" href="http://www.census.gov/geo/maps-data/data/gazetteer2000.html">http://www.census.gov/geo/maps-data/data/gazetteer2000.html</a>).</p>
-<p>Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD ARCHIVE
-   FROM http://www.census.gov/geo/maps-data/data/docs/gazetteer/places2k.zip
-   INTO postgresql:///pgloader
-
-   BEFORE LOAD DO
-     $$ drop table if exists places; $$,
-     $$ create table places
-       (
-          usps      char(2)  not null,
-          fips      char(2)  not null,
-          fips_code char(5),
-          loc_name  varchar(64)
-       );
-     $$
-
-   LOAD FIXED
-        FROM FILENAME MATCHING ~/places2k.txt/
-             WITH ENCODING latin1
-             (
-                usps           from   0 for  2,
-                fips           from   2 for  2,
-                fips_code      from   4 for  5,
-                &quot;LocationName&quot; from   9 for 64 [trim right whitespace],
-                p              from  73 for  9,
-                h              from  82 for  9,
-                land           from  91 for 14,
-                water          from 105 for 14,
-                ldm            from 119 for 14,
-                wtm            from 131 for 14,
-                lat            from 143 for 10,
-                long           from 153 for 11
-             )
-        INTO postgresql:///pgloader?places
-             (
-            usps, fips, fips_code, &quot;LocationName&quot;
-             );
-</pre></div>
-</div>
-</div>
-<div class="section" id="id2">
-<h3>The Data<a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h3>
-<p>This command allows loading the following file content, where we are only
-showing the first couple of lines:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">AL0100124Abbeville</span> <span class="n">city</span>                                                       <span class="mi">2987</span>     <span class="mi">1353</span>      <span class="mi">40301945</span>        <span class="mi">120383</span>   <span class="mf">15.560669</span>    <span class="mf">0.046480</span> <span class="mf">31.566367</span> <span class="o">-</span><span class="mf">85.251300</span>
-<span class="n">AL0100460Adamsville</span> <span class="n">city</span>                                                      <span class="mi">4965</span>     <span class="mi">2042</span>      <span class="mi">50779330</span>         <span class="mi">14126</span>   <span class="mf">19.606010</span>    <span class="mf">0.005454</span> <span class="mf">33.590411</span> <span class="o">-</span><span class="mf">86.949166</span>
-<span class="n">AL0100484Addison</span> <span class="n">town</span>                                                          <span class="mi">723</span>      <span class="mi">339</span>       <span class="mi">9101325</span>             <span class="mi">0</span>    <span class="mf">3.514041</span>    <span class="mf">0.000000</span> <span class="mf">34.200042</span> <span class="o">-</span><span class="mf">87.177851</span>
-<span class="n">AL0100676Akron</span> <span class="n">town</span>                                                            <span class="mi">521</span>      <span class="mi">239</span>       <span class="mi">1436797</span>             <span class="mi">0</span>    <span class="mf">0.554750</span>    <span class="mf">0.000000</span> <span class="mf">32.876425</span> <span class="o">-</span><span class="mf">87.740978</span>
-<span class="n">AL0100820Alabaster</span> <span class="n">city</span>                                                      <span class="mi">22619</span>     <span class="mi">8594</span>      <span class="mi">53023800</span>        <span class="mi">141711</span>   <span class="mf">20.472605</span>    <span class="mf">0.054715</span> <span class="mf">33.231162</span> <span class="o">-</span><span class="mf">86.823829</span>
-<span class="n">AL0100988Albertville</span> <span class="n">city</span>                                                    <span class="mi">17247</span>     <span class="mi">7090</span>      <span class="mi">67212867</span>        <span class="mi">258738</span>   <span class="mf">25.951034</span>    <span class="mf">0.099899</span> <span class="mf">34.265362</span> <span class="o">-</span><span class="mf">86.211261</span>
-<span class="n">AL0101132Alexander</span> <span class="n">City</span> <span class="n">city</span>                                                 <span class="mi">15008</span>     <span class="mi">6855</span>     <span class="mi">100534344</span>        <span class="mi">433413</span>   <span class="mf">38.816529</span>    <span class="mf">0.167342</span> <span class="mf">32.933157</span> <span class="o">-</span><span class="mf">85.936008</span>
-</pre></div>
-</div>
-</div>
-<div class="section" id="id3">
-<h3>Loading the data<a class="headerlink" href="#id3" title="Permalink to this headline">¶</a></h3>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>census-places.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader census-places.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/census-places.load&quot;
-... LOG Fetching &#39;http://www.census.gov/geo/maps-data/data/docs/gazetteer/places2k.zip&#39;
-... LOG Extracting files from archive &#39;//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//places2k.zip&#39;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-         download          0          0          0          1.494s
-          extract          0          0          0          1.013s
-      before load          2          2          0          0.013s
------------------  ---------  ---------  ---------  --------------
-           places      25375      25375          0          0.499s
------------------  ---------  ---------  ---------  --------------
-Total import time      25375      25375          0          3.019s
-</pre></div>
-</div>
-<p>We can see that pgloader did download the file from its HTTP URL location
-then <em>unziped</em> it before the loading itself.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-<div class="section" id="loading-maxmind-geolite-data-with-pgloader">
-<h2>Loading MaxMind Geolite Data with pgloader<a class="headerlink" href="#loading-maxmind-geolite-data-with-pgloader" title="Permalink to this headline">¶</a></h2>
-<p><a class="reference external" href="http://www.maxmind.com/">MaxMind</a> provides a free dataset for
-geolocation, which is quite popular. Using pgloader you can download the
-lastest version of it, extract the CSV files from the archive and load their
-content into your database directly.</p>
-<div class="section" id="id4">
-<h3>The Command<a class="headerlink" href="#id4" title="Permalink to this headline">¶</a></h3>
-<p>To load data with pgloader you need to define in a <em>command</em> the operations
-in some details. Here’s our example for loading the Geolite data:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>/*
- * Loading from a ZIP archive containing CSV files. The full test can be
- * done with using the archive found at
- * http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip
- *
- * And a very light version of this data set is found at
- * http://pgsql.tapoueh.org/temp/foo.zip for quick testing.
- */
-
-LOAD ARCHIVE
-   FROM http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip
-   INTO postgresql:///ip4r
-
-   BEFORE LOAD DO
-     $$ create extension if not exists ip4r; $$,
-     $$ create schema if not exists geolite; $$,
-     $$ create table if not exists geolite.location
-       (
-          locid      integer primary key,
-          country    text,
-          region     text,
-          city       text,
-          postalcode text,
-          location   point,
-          metrocode  text,
-          areacode   text
-       );
-     $$,
-     $$ create table if not exists geolite.blocks
-       (
-          iprange    ip4r,
-          locid      integer
-       );
-     $$,
-     $$ drop index if exists geolite.blocks_ip4r_idx; $$,
-     $$ truncate table geolite.blocks, geolite.location cascade; $$
-
-   LOAD CSV
-        FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
-             WITH ENCODING iso-8859-1
-             (
-                locId,
-                country,
-                region     null if blanks,
-                city       null if blanks,
-                postalCode null if blanks,
-                latitude,
-                longitude,
-                metroCode  null if blanks,
-                areaCode   null if blanks
-             )
-        INTO postgresql:///ip4r?geolite.location
-             (
-                locid,country,region,city,postalCode,
-                location point using (format nil &quot;(~a,~a)&quot; longitude latitude),
-                metroCode,areaCode
-             )
-        WITH skip header = 2,
-             fields optionally enclosed by &#39;&quot;&#39;,
-             fields escaped by double-quote,
-             fields terminated by &#39;,&#39;
-
-  AND LOAD CSV
-        FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
-             WITH ENCODING iso-8859-1
-             (
-                startIpNum, endIpNum, locId
-             )
-        INTO postgresql:///ip4r?geolite.blocks
-             (
-                iprange ip4r using (ip-range startIpNum endIpNum),
-                locId
-             )
-        WITH skip header = 2,
-             fields optionally enclosed by &#39;&quot;&#39;,
-             fields escaped by double-quote,
-             fields terminated by &#39;,&#39;
-
-   FINALLY DO
-     $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;
-</pre></div>
-</div>
-<p>Note that while the <em>Geolite</em> data is using a pair of integers (<em>start</em>,
-<em>end</em>) to represent <em>ipv4</em> data, we use the very poweful <a class="reference external" href="https://github.com/RhodiumToad/ip4r">ip4r</a> PostgreSQL Extension instead.</p>
-<p>The transformation from a pair of integers into an IP is done dynamically by
-the pgloader process.</p>
-<p>Also, the location is given as a pair of <em>float</em> columns for the <em>longitude</em>
-and the <em>latitude</em> where PostgreSQL offers the
-<a class="reference external" href="http://www.postgresql.org/docs/9.3/interactive/functions-geometry.html">point</a>
-datatype, so the pgloader command here will actually transform the data on
-the fly to use the appropriate data type and its input representation.</p>
-</div>
-<div class="section" id="id5">
-<h3>Loading the data<a class="headerlink" href="#id5" title="Permalink to this headline">¶</a></h3>
-<p>Here’s how to start loading the data. Note that the ouput here has been
-edited so as to facilitate its browsing online:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader archive.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/archive.load&quot;
-... LOG Fetching &#39;http://geolite.maxmind.com/download/geoip/database/GeoLiteCity_CSV/GeoLiteCity-latest.zip&#39;
-... LOG Extracting files from archive &#39;//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//GeoLiteCity-latest.zip&#39;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-         download          0          0          0         11.592s
-          extract          0          0          0          1.012s
-      before load          6          6          0          0.019s
------------------  ---------  ---------  ---------  --------------
- geolite.location     470387     470387          0          7.743s
-   geolite.blocks    1903155    1903155          0         16.332s
------------------  ---------  ---------  ---------  --------------
-          finally          1          1          0         31.692s
------------------  ---------  ---------  ---------  --------------
-Total import time    2373542    2373542          0        1m8.390s
-</pre></div>
-</div>
-<p>The timing of course includes the transformation of the <em>1.9 million</em> pairs
-of integer into a single <em>ipv4 range</em> each. The <em>finally</em> step consists of
-creating the <em>GiST</em> specialized index as given in the main command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">CREATE</span> <span class="n">INDEX</span> <span class="n">blocks_ip4r_idx</span> <span class="n">ON</span> <span class="n">geolite</span><span class="o">.</span><span class="n">blocks</span> <span class="n">USING</span> <span class="n">gist</span><span class="p">(</span><span class="n">iprange</span><span class="p">);</span>
-</pre></div>
-</div>
-<p>That index will then be used to speed up queries wanting to find which
-recorded geolocation contains a specific IP address:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">ip4r</span><span class="o">&gt;</span> <span class="n">select</span> <span class="o">*</span>
-        <span class="kn">from</span>      <span class="nn">geolite.location</span> <span class="n">l</span>
-             <span class="n">join</span> <span class="n">geolite</span><span class="o">.</span><span class="n">blocks</span> <span class="n">b</span> <span class="n">using</span><span class="p">(</span><span class="n">locid</span><span class="p">)</span>
-       <span class="n">where</span> <span class="n">iprange</span> <span class="o">&gt;&gt;=</span> <span class="s1">&#39;8.8.8.8&#39;</span><span class="p">;</span>
-
-<span class="o">-</span><span class="p">[</span> <span class="n">RECORD</span> <span class="mi">1</span> <span class="p">]</span><span class="o">------------------</span>
-<span class="n">locid</span>      <span class="o">|</span> <span class="mi">223</span>
-<span class="n">country</span>    <span class="o">|</span> <span class="n">US</span>
-<span class="n">region</span>     <span class="o">|</span>
-<span class="n">city</span>       <span class="o">|</span>
-<span class="n">postalcode</span> <span class="o">|</span>
-<span class="n">location</span>   <span class="o">|</span> <span class="p">(</span><span class="o">-</span><span class="mi">97</span><span class="p">,</span><span class="mi">38</span><span class="p">)</span>
-<span class="n">metrocode</span>  <span class="o">|</span>
-<span class="n">areacode</span>   <span class="o">|</span>
-<span class="n">iprange</span>    <span class="o">|</span> <span class="mf">8.8</span><span class="o">.</span><span class="mf">8.8</span><span class="o">-</span><span class="mf">8.8</span><span class="o">.</span><span class="mf">37.255</span>
-
-<span class="n">Time</span><span class="p">:</span> <span class="mf">0.747</span> <span class="n">ms</span>
-</pre></div>
-</div>
-</div>
-</div>
-<div class="section" id="loading-dbase-files-with-pgloader">
-<h2>Loading dBase files with pgloader<a class="headerlink" href="#loading-dbase-files-with-pgloader" title="Permalink to this headline">¶</a></h2>
-<p>The dBase format is still in use in some places as modern tools such as
-<em>Filemaker</em> and <em>Excel</em> offer some level of support for it. Speaking of
-support in modern tools, pgloader is right there on the list too!</p>
-<div class="section" id="id6">
-<h3>The Command<a class="headerlink" href="#id6" title="Permalink to this headline">¶</a></h3>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our example for loading a
-dBase file, using a file provided by the french administration.</p>
-<p>You can find more files from them at the <a class="reference external" href="http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement.asp">Insee</a>
-website.</p>
-<p>Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">LOAD</span> <span class="n">DBF</span>
-    <span class="n">FROM</span> <span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="o">.</span><span class="n">insee</span><span class="o">.</span><span class="n">fr</span><span class="o">/</span><span class="n">fr</span><span class="o">/</span><span class="n">methodes</span><span class="o">/</span><span class="n">nomenclatures</span><span class="o">/</span><span class="n">cog</span><span class="o">/</span><span class="n">telechargement</span><span class="o">/</span><span class="mi">2013</span><span class="o">/</span><span class="n">dbf</span><span class="o">/</span><span class="n">historiq2013</span><span class="o">.</span><span class="n">zip</span>
-    <span class="n">INTO</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">pgloader</span>
-    <span class="n">WITH</span> <span class="n">truncate</span><span class="p">,</span> <span class="n">create</span> <span class="n">table</span>
-     <span class="n">SET</span> <span class="n">client_encoding</span> <span class="n">TO</span> <span class="s1">&#39;latin1&#39;</span><span class="p">;</span>
-</pre></div>
-</div>
-<p>Note that here pgloader will benefit from the meta-data information found in
-the dBase file to create a PostgreSQL table capable of hosting the data as
-described, then load the data.</p>
-</div>
-<div class="section" id="id7">
-<h3>Loading the data<a class="headerlink" href="#id7" title="Permalink to this headline">¶</a></h3>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>dbf-zip.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader dbf-zip.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/dbf-zip.load&quot;
-... LOG Fetching &#39;http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/historiq2013.zip&#39;
-... LOG Extracting files from archive &#39;//private/var/folders/w7/9n8v8pw54t1gngfff0lj16040000gn/T/pgloader//historiq2013.zip&#39;
-
-       table name       read   imported     errors            time
------------------  ---------  ---------  ---------  --------------
-         download          0          0          0          0.167s
-          extract          0          0          0          1.010s
- create, truncate          0          0          0          0.071s
------------------  ---------  ---------  ---------  --------------
-     historiq2013       9181       9181          0          0.658s
------------------  ---------  ---------  ---------  --------------
-Total import time       9181       9181          0          1.906s
-</pre></div>
-</div>
-<p>We can see that <a class="reference external" href="http://pgloader.io">pgloader</a> did download the file from
-its HTTP URL location then <em>unziped</em> it before the loading itself.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-<div class="section" id="loading-sqlite-files-with-pgloader">
-<h2>Loading SQLite files with pgloader<a class="headerlink" href="#loading-sqlite-files-with-pgloader" title="Permalink to this headline">¶</a></h2>
-<p>The SQLite database is a respected solution to manage your data with. Its
-embeded nature makes it a source of migrations when a projects now needs to
-handle more concurrency, which [PostgreSQL](<a class="reference external" href="http://www.postgresql.org/">http://www.postgresql.org/</a>) is
-very good at. pgloader can help you there.</p>
-<div class="section" id="in-a-single-command-line">
-<h3>In a Single Command Line<a class="headerlink" href="#in-a-single-command-line" title="Permalink to this headline">¶</a></h3>
-<p>You can</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ createdb chinook
-$ pgloader https://github.com/lerocha/chinook-database/raw/master/ChinookDatabase/DataSources/Chinook_Sqlite_AutoIncrementPKs.sqlite pgsql:///chinook
-</pre></div>
-</div>
-<p>Done! All with the schema, data, constraints, primary keys and foreign keys,
-etc. We also see an error with the Chinook schema that contains several
-primary key definitions against the same table, which is not accepted by
-PostgreSQL:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mf">59.019000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">LOG</span> <span class="n">Data</span> <span class="n">errors</span> <span class="ow">in</span> <span class="s1">&#39;/private/tmp/pgloader/&#39;</span>
-<span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">18</span><span class="p">:</span><span class="mf">59.236000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">LOG</span> <span class="n">Fetching</span> <span class="s1">&#39;https://github.com/lerocha/chinook-database/raw/master/ChinookDatabase/DataSources/Chinook_Sqlite_AutoIncrementPKs.sqlite&#39;</span>
-<span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">19</span><span class="p">:</span><span class="mf">00.664000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">ERROR</span> <span class="n">Database</span> <span class="n">error</span> <span class="mi">42</span><span class="n">P16</span><span class="p">:</span> <span class="n">multiple</span> <span class="n">primary</span> <span class="n">keys</span> <span class="k">for</span> <span class="n">table</span> <span class="s2">&quot;playlisttrack&quot;</span> <span class="n">are</span> <span class="ow">not</span> <span class="n">allowed</span>
-<span class="n">QUERY</span><span class="p">:</span> <span class="n">ALTER</span> <span class="n">TABLE</span> <span class="n">playlisttrack</span> <span class="n">ADD</span> <span class="n">PRIMARY</span> <span class="n">KEY</span> <span class="n">USING</span> <span class="n">INDEX</span> <span class="n">idx_66873_sqlite_autoindex_playlisttrack_1</span><span class="p">;</span>
-<span class="mi">2017</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="n">T16</span><span class="p">:</span><span class="mi">19</span><span class="p">:</span><span class="mf">00.665000</span><span class="o">+</span><span class="mi">02</span><span class="p">:</span><span class="mi">00</span> <span class="n">LOG</span> <span class="n">report</span> <span class="n">summary</span> <span class="n">reset</span>
-             <span class="n">table</span> <span class="n">name</span>       <span class="n">read</span>   <span class="n">imported</span>     <span class="n">errors</span>      <span class="n">total</span> <span class="n">time</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-                  <span class="n">fetch</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.877</span><span class="n">s</span>
-        <span class="n">fetch</span> <span class="n">meta</span> <span class="n">data</span>         <span class="mi">33</span>         <span class="mi">33</span>          <span class="mi">0</span>          <span class="mf">0.033</span><span class="n">s</span>
-         <span class="n">Create</span> <span class="n">Schemas</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.003</span><span class="n">s</span>
-       <span class="n">Create</span> <span class="n">SQL</span> <span class="n">Types</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.006</span><span class="n">s</span>
-          <span class="n">Create</span> <span class="n">tables</span>         <span class="mi">22</span>         <span class="mi">22</span>          <span class="mi">0</span>          <span class="mf">0.043</span><span class="n">s</span>
-         <span class="n">Set</span> <span class="n">Table</span> <span class="n">OIDs</span>         <span class="mi">11</span>         <span class="mi">11</span>          <span class="mi">0</span>          <span class="mf">0.012</span><span class="n">s</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-                  <span class="n">album</span>        <span class="mi">347</span>        <span class="mi">347</span>          <span class="mi">0</span>          <span class="mf">0.023</span><span class="n">s</span>
-                 <span class="n">artist</span>        <span class="mi">275</span>        <span class="mi">275</span>          <span class="mi">0</span>          <span class="mf">0.023</span><span class="n">s</span>
-               <span class="n">customer</span>         <span class="mi">59</span>         <span class="mi">59</span>          <span class="mi">0</span>          <span class="mf">0.021</span><span class="n">s</span>
-               <span class="n">employee</span>          <span class="mi">8</span>          <span class="mi">8</span>          <span class="mi">0</span>          <span class="mf">0.018</span><span class="n">s</span>
-                <span class="n">invoice</span>        <span class="mi">412</span>        <span class="mi">412</span>          <span class="mi">0</span>          <span class="mf">0.031</span><span class="n">s</span>
-                  <span class="n">genre</span>         <span class="mi">25</span>         <span class="mi">25</span>          <span class="mi">0</span>          <span class="mf">0.021</span><span class="n">s</span>
-            <span class="n">invoiceline</span>       <span class="mi">2240</span>       <span class="mi">2240</span>          <span class="mi">0</span>          <span class="mf">0.034</span><span class="n">s</span>
-              <span class="n">mediatype</span>          <span class="mi">5</span>          <span class="mi">5</span>          <span class="mi">0</span>          <span class="mf">0.025</span><span class="n">s</span>
-          <span class="n">playlisttrack</span>       <span class="mi">8715</span>       <span class="mi">8715</span>          <span class="mi">0</span>          <span class="mf">0.040</span><span class="n">s</span>
-               <span class="n">playlist</span>         <span class="mi">18</span>         <span class="mi">18</span>          <span class="mi">0</span>          <span class="mf">0.016</span><span class="n">s</span>
-                  <span class="n">track</span>       <span class="mi">3503</span>       <span class="mi">3503</span>          <span class="mi">0</span>          <span class="mf">0.111</span><span class="n">s</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-<span class="n">COPY</span> <span class="n">Threads</span> <span class="n">Completion</span>         <span class="mi">33</span>         <span class="mi">33</span>          <span class="mi">0</span>          <span class="mf">0.313</span><span class="n">s</span>
-         <span class="n">Create</span> <span class="n">Indexes</span>         <span class="mi">22</span>         <span class="mi">22</span>          <span class="mi">0</span>          <span class="mf">0.160</span><span class="n">s</span>
- <span class="n">Index</span> <span class="n">Build</span> <span class="n">Completion</span>         <span class="mi">22</span>         <span class="mi">22</span>          <span class="mi">0</span>          <span class="mf">0.027</span><span class="n">s</span>
-        <span class="n">Reset</span> <span class="n">Sequences</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.017</span><span class="n">s</span>
-           <span class="n">Primary</span> <span class="n">Keys</span>         <span class="mi">12</span>          <span class="mi">0</span>          <span class="mi">1</span>          <span class="mf">0.013</span><span class="n">s</span>
-    <span class="n">Create</span> <span class="n">Foreign</span> <span class="n">Keys</span>         <span class="mi">11</span>         <span class="mi">11</span>          <span class="mi">0</span>          <span class="mf">0.040</span><span class="n">s</span>
-        <span class="n">Create</span> <span class="n">Triggers</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.000</span><span class="n">s</span>
-       <span class="n">Install</span> <span class="n">Comments</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mi">0</span>          <span class="mf">0.000</span><span class="n">s</span>
-<span class="o">-----------------------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">---------</span>  <span class="o">--------------</span>
-      <span class="n">Total</span> <span class="kn">import</span> <span class="nn">time</span>      <span class="mi">15607</span>      <span class="mi">15607</span>          <span class="mi">0</span>          <span class="mf">1.669</span><span class="n">s</span>
-</pre></div>
-</div>
-<p>You may need to have special cases to take care of tho. In advanced case you
-can use the pgloader command.</p>
-</div>
-<div class="section" id="id8">
-<h3>The Command<a class="headerlink" href="#id8" title="Permalink to this headline">¶</a></h3>
-<p>To load data with [pgloader](<a class="reference external" href="http://pgloader.io/">http://pgloader.io/</a>) you need to define in a
-<em>command</em> the operations in some details. Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">load</span> <span class="n">database</span>
-     <span class="kn">from</span> <span class="s1">&#39;sqlite/Chinook_Sqlite_AutoIncrementPKs.sqlite&#39;</span>
-     <span class="n">into</span> <span class="n">postgresql</span><span class="p">:</span><span class="o">///</span><span class="n">pgloader</span>
-
- <span class="k">with</span> <span class="n">include</span> <span class="n">drop</span><span class="p">,</span> <span class="n">create</span> <span class="n">tables</span><span class="p">,</span> <span class="n">create</span> <span class="n">indexes</span><span class="p">,</span> <span class="n">reset</span> <span class="n">sequences</span>
-
-  <span class="nb">set</span> <span class="n">work_mem</span> <span class="n">to</span> <span class="s1">&#39;16MB&#39;</span><span class="p">,</span> <span class="n">maintenance_work_mem</span> <span class="n">to</span> <span class="s1">&#39;512 MB&#39;</span><span class="p">;</span>
-</pre></div>
-</div>
-<p>Note that here pgloader will benefit from the meta-data information found in
-the SQLite file to create a PostgreSQL database capable of hosting the data
-as described, then load the data.</p>
-</div>
-<div class="section" id="id9">
-<h3>Loading the data<a class="headerlink" href="#id9" title="Permalink to this headline">¶</a></h3>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>sqlite.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader sqlite.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/sqlite.load&quot;
-... WARNING Postgres warning: table &quot;album&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;artist&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;customer&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;employee&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;genre&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;invoice&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;invoiceline&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;mediatype&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;playlist&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;playlisttrack&quot; does not exist, skipping
-... WARNING Postgres warning: table &quot;track&quot; does not exist, skipping
-            table name       read   imported     errors            time
-----------------------  ---------  ---------  ---------  --------------
-      create, truncate          0          0          0          0.052s
-                 Album        347        347          0          0.070s
-                Artist        275        275          0          0.014s
-              Customer         59         59          0          0.014s
-              Employee          8          8          0          0.012s
-                 Genre         25         25          0          0.018s
-               Invoice        412        412          0          0.032s
-           InvoiceLine       2240       2240          0          0.077s
-             MediaType          5          5          0          0.012s
-              Playlist         18         18          0          0.008s
-         PlaylistTrack       8715       8715          0          0.071s
-                 Track       3503       3503          0          0.105s
-index build completion          0          0          0          0.000s
-----------------------  ---------  ---------  ---------  --------------
-        Create Indexes         20         20          0          0.279s
-       reset sequences          0          0          0          0.043s
-----------------------  ---------  ---------  ---------  --------------
-  Total streaming time      15607      15607          0          0.476s
-</pre></div>
-</div>
-<p>We can see that <a class="reference external" href="http://pgloader.io">pgloader</a> did download the file from
-its HTTP URL location then <em>unziped</em> it before loading it.</p>
-<p>Also, the <em>WARNING</em> messages we see here are expected as the PostgreSQL
-database is empty when running the command, and pgloader is using the SQL
-commands <cite>DROP TABLE IF EXISTS</cite> when the given command uses the <cite>include
-drop</cite> option.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-<div class="section" id="migrating-from-mysql-to-postgresql">
-<h2>Migrating from MySQL to PostgreSQL<a class="headerlink" href="#migrating-from-mysql-to-postgresql" title="Permalink to this headline">¶</a></h2>
-<p>If you want to migrate your data over to <a class="reference external" href="http://www.postgresql.org">PostgreSQL</a> from MySQL then pgloader is the tool of
-choice!</p>
-<p>Most tools around are skipping the main problem with migrating from MySQL,
-which is to do with the type casting and data sanitizing that needs to be
-done. pgloader will not leave you alone on those topics.</p>
-<div class="section" id="id11">
-<h3>In a Single Command Line<a class="headerlink" href="#id11" title="Permalink to this headline">¶</a></h3>
-<p>As an example, we will use the f1db database from &lt;<a class="reference external" href="http://ergast.com/mrd/">http://ergast.com/mrd/</a>&gt;
-which which provides a historical record of motor racing data for
-non-commercial purposes. You can either use their API or download the whole
-database at <a class="reference external" href="http://ergast.com/downloads/f1db.sql.gz">http://ergast.com/downloads/f1db.sql.gz</a>. Once you’ve done that load the
-database in MySQL:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ mysql -u root
-&gt; create database f1db;
-&gt; source f1db.sql
-</pre></div>
-</div>
-<p>Now let’s migrate this database into PostgreSQL in a single command line:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ createdb f1db
-$ pgloader mysql://root@localhost/f1db pgsql:///f1db
-</pre></div>
-</div>
-<p>Done! All with schema, table definitions, constraints, indexes, primary
-keys, <em>auto_increment</em> columns turned into <em>bigserial</em> , foreign keys,
-comments, and if you had some MySQL default values such as <em>ON UPDATE
-CURRENT_TIMESTAMP</em> they would have been translated to a <a class="reference external" href="https://www.postgresql.org/docs/current/static/plpgsql-trigger.html">PostgreSQL before
-update trigger</a>
-automatically.</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader mysql://root@localhost/f1db pgsql:///f1db
-2017-06-16T08:56:14.064000+02:00 LOG Main logs in &#39;/private/tmp/pgloader/pgloader.log&#39;
-2017-06-16T08:56:14.068000+02:00 LOG Data errors in &#39;/private/tmp/pgloader/&#39;
-2017-06-16T08:56:19.542000+02:00 LOG report summary reset
-               table name       read   imported     errors      total time
--------------------------  ---------  ---------  ---------  --------------
-          fetch meta data         33         33          0          0.365s
-           Create Schemas          0          0          0          0.007s
-         Create SQL Types          0          0          0          0.006s
-            Create tables         26         26          0          0.068s
-           Set Table OIDs         13         13          0          0.012s
--------------------------  ---------  ---------  ---------  --------------
-  f1db.constructorresults      11011      11011          0          0.205s
-            f1db.circuits         73         73          0          0.150s
-        f1db.constructors        208        208          0          0.059s
-f1db.constructorstandings      11766      11766          0          0.365s
-             f1db.drivers        841        841          0          0.268s
-            f1db.laptimes     413578     413578          0          2.892s
-     f1db.driverstandings      31420      31420          0          0.583s
-            f1db.pitstops       5796       5796          0          2.154s
-               f1db.races        976        976          0          0.227s
-          f1db.qualifying       7257       7257          0          0.228s
-             f1db.seasons         68         68          0          0.527s
-             f1db.results      23514      23514          0          0.658s
-              f1db.status        133        133          0          0.130s
--------------------------  ---------  ---------  ---------  --------------
-  COPY Threads Completion         39         39          0          4.303s
-           Create Indexes         20         20          0          1.497s
-   Index Build Completion         20         20          0          0.214s
-          Reset Sequences          0         10          0          0.058s
-             Primary Keys         13         13          0          0.012s
-      Create Foreign Keys          0          0          0          0.000s
-          Create Triggers          0          0          0          0.001s
-         Install Comments          0          0          0          0.000s
--------------------------  ---------  ---------  ---------  --------------
-        Total import time     506641     506641          0          5.547s
-</pre></div>
-</div>
-<p>You may need to have special cases to take care of tho, or views that you
-want to materialize while doing the migration. In advanced case you can use
-the pgloader command.</p>
-</div>
-<div class="section" id="id12">
-<h3>The Command<a class="headerlink" href="#id12" title="Permalink to this headline">¶</a></h3>
-<p>To load data with pgloader you need to define in a <em>command</em> the operations
-in some details. Here’s our example for loading the <a class="reference external" href="http://dev.mysql.com/doc/sakila/en/">MySQL Sakila Sample
-Database</a>.</p>
-<p>Here’s our command:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>load database
-     from      mysql://root@localhost/sakila
-     into postgresql:///sakila
-
- WITH include drop, create tables, no truncate,
-      create indexes, reset sequences, foreign keys
-
-  SET maintenance_work_mem to &#39;128MB&#39;, work_mem to &#39;12MB&#39;, search_path to &#39;sakila&#39;
-
- CAST type datetime to timestamptz
-                drop default drop not null using zero-dates-to-null,
-      type date drop not null drop default using zero-dates-to-null
-
- MATERIALIZE VIEWS film_list, staff_list
-
- -- INCLUDING ONLY TABLE NAMES MATCHING ~/film/, &#39;actor&#39;
- -- EXCLUDING TABLE NAMES MATCHING ~&lt;ory&gt;
-
- BEFORE LOAD DO
- $$ create schema if not exists sakila; $$;
-</pre></div>
-</div>
-<p>Note that here pgloader will benefit from the meta-data information found in
-the MySQL database to create a PostgreSQL database capable of hosting the
-data as described, then load the data.</p>
-<p>In particular, some specific <em>casting rules</em> are given here, to cope with
-date values such as <cite>0000-00-00</cite> that MySQL allows and PostgreSQL rejects
-for not existing in our calendar. It’s possible to add per-column casting
-rules too, which is useful is some of your <cite>tinyint</cite> are in fact <cite>smallint</cite>
-while some others are in fact <cite>boolean</cite> values.</p>
-<p>Finaly note that we are using the <em>MATERIALIZE VIEWS</em> clause of pgloader:
-the selected views here will be migrated over to PostgreSQL <em>with their
-contents</em>.</p>
-<p>It’s possible to use the <em>MATERIALIZE VIEWS</em> clause and give both the name
-and the SQL (in MySQL dialect) definition of view, then pgloader creates the
-view before loading the data, then drops it again at the end.</p>
-<p>## Loading the data</p>
-<p>Let’s start the <cite>pgloader</cite> command with our <cite>sakila.load</cite> command file:</p>
-<div class="highlight-default"><div class="highlight"><pre><span></span>$ pgloader sakila.load
-... LOG Starting pgloader, log system is ready.
-... LOG Parsing commands from file &quot;/Users/dim/dev/pgloader/test/sakila.load&quot;
-   &lt;WARNING: table &quot;xxx&quot; does not exists have been edited away&gt;
-
-            table name       read   imported     errors            time
-----------------------  ---------  ---------  ---------  --------------
-           before load          1          1          0          0.007s
-       fetch meta data         45         45          0          0.402s
-          create, drop          0         36          0          0.208s
-----------------------  ---------  ---------  ---------  --------------
-                 actor        200        200          0          0.071s
-               address        603        603          0          0.035s
-              category         16         16          0          0.018s
-                  city        600        600          0          0.037s
-               country        109        109          0          0.023s
-              customer        599        599          0          0.073s
-                  film       1000       1000          0          0.135s
-            film_actor       5462       5462          0          0.236s
-         film_category       1000       1000          0          0.070s
-             film_text       1000       1000          0          0.080s
-             inventory       4581       4581          0          0.136s
-              language          6          6          0          0.036s
-               payment      16049      16049          0          0.539s
-                rental      16044      16044          0          0.648s
-                 staff          2          2          0          0.041s
-                 store          2          2          0          0.036s
-             film_list        997        997          0          0.247s
-            staff_list          2          2          0          0.135s
-Index Build Completion          0          0          0          0.000s
-----------------------  ---------  ---------  ---------  --------------
-        Create Indexes         41         41          0          0.964s
-       Reset Sequences          0          1          0          0.035s
-          Foreign Keys         22         22          0          0.254s
-----------------------  ---------  ---------  ---------  --------------
-     Total import time      48272      48272          0          3.502s
-</pre></div>
-</div>
-<p>The <em>WARNING</em> messages we see here are expected as the PostgreSQL database
-is empty when running the command, and pgloader is using the SQL commands
-<cite>DROP TABLE IF EXISTS</cite> when the given command uses the <cite>include drop</cite>
-option.</p>
-<p>Note that the output of the command has been edited to facilitate its
-browsing online.</p>
-</div>
-</div>
-</div>
-
-
-          </div>
-        </div>
-      </div>
-      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
-        <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
-  <li><a href="../index.html">Documentation overview</a><ul>
-      <li>Previous: <a href="../intro.html" title="previous chapter">Introduction</a></li>
-      <li>Next: <a href="../pgloader.html" title="next chapter">PgLoader Reference Manual</a></li>
-  </ul></li>
-</ul>
-</div>
-<div id="searchbox" style="display: none" role="search">
-  <h3>Quick search</h3>
-    <form class="search" action="../search.html" method="get">
-      <div><input type="text" name="q" /></div>
-      <div><input type="submit" value="Go" /></div>
-      <input type="hidden" name="check_keywords" value="yes" />
-      <input type="hidden" name="area" value="default" />
-    </form>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
-        </div>
-      </div>
-      <div class="clearer"></div>
-    </div>
-    <div class="footer">
-      &copy;2017, Dimitri Fontaine.
-      
-      |
-      Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-      
-      |
-      <a href="../_sources/tutorial/tutorial.rst.txt"
-          rel="nofollow">Page source</a>
-    </div>
-
-    
-
-    
-  </body>
-</html>