From f6b63ade0abbeb88e35c75e33dd2e01eccaa9c2f Mon Sep 17 00:00:00 2001 From: Nikhil Benesch Date: Wed, 12 Mar 2014 04:03:45 -0400 Subject: [PATCH] pgloader.1.md: remove trailing whitespace --- pgloader.1.md | 390 +++++++++++++++++++++++++------------------------- 1 file changed, 195 insertions(+), 195 deletions(-) diff --git a/pgloader.1.md b/pgloader.1.md index 3618638..e83c361 100644 --- a/pgloader.1.md +++ b/pgloader.1.md @@ -140,9 +140,9 @@ support only a subset of the general options and provide specific options. LOAD FROM [ WITH ] INTO - + [ WITH ] - + [ SET ] ; @@ -161,31 +161,31 @@ http://www.postgresql.org/docs/9.3/static/libpq-connect.html#LIBPQ-CONNSTRING. Where: - *user* - + Can contain any character, including colon (`:`) which must then be doubled (`::`) and at-sign (`@`) which must then be doubled (`@@`). - + - *password* - + Can contain any character, including that at sign (`@`) which must then be doubled (`@@`). To leave the password empty, when the *user* name ends with at at sign, you then have to use the syntax user:@. - + - *netloc* - + Can be either a hostname in dotted notation, or an ipv4, or an unix domain socket path. Empty is the default network location, under a system providing *unix domain socket* that method is prefered, otherwise the *netloc* default to `localhost`. - + It's possible to force the *unix domain socket* path by using the syntax `unix:/path/to/where/the/socket/file/is`, so to force a non default socket path and a non default port, you would have: - + postgresql://unix:/tmp:54321/dbname - *dbname* - + Should be a proper identifier (letter followed by a mix of letters, digits and the punctuation signs comma (`,`), dash (`-`) and underscore (`_`). @@ -198,12 +198,12 @@ Several clauses listed in the following accept *regular expressions* with the following input rules: - A regular expression begins with a tilde sign (`~`), - + - is then followed with an opening sign, - + - then any character is allowed and considered part of the regular expression, except for the closing sign, - + - then a closing sign is expected. The opening and closing sign are allowed by pair, here's the complete list @@ -230,7 +230,7 @@ Any command may contain comments, following those input rules: - the `--` delimiter begins a comment that ends with the end of the current line, - + - the delimiters `/*` and `*/` respectively start and end a comment, which can be found in the middle of a command or span several lines. @@ -256,7 +256,7 @@ example: fields optionally enclosed by '"', fields escaped by backslash-quote, fields terminated by '\t' - + SET work_mem to '32 MB', maintenance_work_mem to '64 MB'; The `csv` format command accepts the following clauses and options: @@ -269,7 +269,7 @@ The `csv` format command accepts the following clauses and options: The filename may be enclosed by single quotes, and could be one of the following special values: - + - *inline* The data is found after the end of the parsed commands. Any number @@ -277,53 +277,53 @@ The `csv` format command accepts the following clauses and options: the data is accepted. - *stdin* - + Reads the data from the standard input stream. The *FROM* option also supports an optional comma separated list of *field* names describing what is expected in the `CSV` data file. - + Each field name can be either only one name or a name following with specific reader options for that field. Supported per-field reader options are: - + - *terminated by* - + See the description of *field terminated by* below. - + The processing of this option is not currently implemented. - + - *date format* - + When the field is expected of the date type, then this option allows to specify the date format used in the file. The processing of this option is not currently implemented. - *null if* - + This option takes an argument which is either the keyword *blanks* or a double-quoted string. - + When *blanks* is used and the field value that is read contains only space characters, then it's automatically converted to an SQL `NULL` value. - + When a double-quoted string is used and that string is read as the field value, then the field value is automatically converted to an SQL `NULL` value. - *INTO* - + The PostgreSQL connection URI must contains the name of the target table where to load the data into. That table must have already been created in PostgreSQL, and the name might be schema qualified. - + Then *INTO* option also supports an optional comma separated list of target columns, which are either the name of an input *field* or the whitespace separated list of the target column name, its PostgreSQL data type and a *USING* expression. - + The *USING* expression can be any valid Common Lisp form and will be read with the current package set to `pgloader.transforms`, so that you can use functions defined in that package, such as functions loaded @@ -331,7 +331,7 @@ The `csv` format command accepts the following clauses and options: Each *USING* expression is compiled at runtime to native code, and will be called in a context such as: - + (destructuring-bind (field-name-1 field-name-2 ...) row (list column-name-1 @@ -343,21 +343,21 @@ The `csv` format command accepts the following clauses and options: custom code for that projection. - *WITH* - + When loading from a `CSV` file, the following options are supported: - + - *truncate* - + When this option is listed, pgloader issues a `TRUNCATE` command against the PostgreSQL target table before reading the data file. - + - *skip header* - + Takes a numeric value as argument. Instruct pgloader to skip that many lines at the beginning of the input file. - *trim unquoted blanks* - + When reading unquoted values in the `CSV` file, remove the blanks found in between the separator and the value. That behaviour is the default. @@ -368,17 +368,17 @@ The `csv` format command accepts the following clauses and options: between the separator and the value. - *fields optionally enclosed by* - + Takes a single character as argument, which must be found inside single quotes, and might be given as the printable character itself, the special value \t to denote a tabulation character, or `0x` then an hexadecimal value read as the ascii code for the character. - + This character is used as the quoting character in the `CSV` file, and defaults to double-quote. - *fields not enclosed* - + By default, pgloader will use the double-quote character as the enclosing character. If you have a CSV file where fields are not enclosed and are using double-quote as an expected ordinary @@ -386,12 +386,12 @@ The `csv` format command accepts the following clauses and options: parser to accept those values. - *fields escaped by* - + Takes either the special value *backslash-quote* or *double-quote*. This value is used to recognize escaped field separators when they are to be found within the data fields themselves. Defaults to *double-quote*. - + - *fields terminated by* Takes a single character as argument, which must be found inside @@ -413,11 +413,11 @@ The `csv` format command accepts the following clauses and options: reading the `CSV` data. - *SET* - + This clause allows to specify session parameters to be set for all the sessions opened by pgloader. It expects a list of parameter name, the equal sign, then the single-quoted value as a comma separated list. - + The names and values of the parameters are not validated by pgloader, they are given as-is to PostgreSQL. @@ -426,11 +426,11 @@ The `csv` format command accepts the following clauses and options: You can run SQL queries against the database before loading the data from the `CSV` file. Most common SQL queries are `CREATE TABLE IF NOT EXISTS` so that the data can be loaded. - + Each command must be *dollar-quoted*: it must begin and end with a double dollar sign, `$$`. Dollar-quoted queries are then comma separated. No extra punctuation is expected after the last SQL query. - + - *AFTER LOAD DO* Same format as *BEFORE LOAD DO*, the dollar-quoted queries found in that @@ -450,13 +450,13 @@ columns arranged in a *fixed size* manner. Here's an example: c time using (time-with-no-separator c), d ) - + WITH truncate - + SET client_encoding to 'latin1', work_mem to '14MB', standard_conforming_strings to 'on' - + BEFORE LOAD DO $$ drop table if exists fixed; $$, $$ create table fixed ( @@ -466,10 +466,10 @@ columns arranged in a *fixed size* manner. Here's an example: d text ); $$; - - 01234567892008052011431250firstline + + 01234567892008052011431250firstline 01234562008052115182300left blank-padded - 12345678902008052208231560another line + 12345678902008052208231560another line The `fixed` format command accepts the following clauses and options: @@ -481,7 +481,7 @@ The `fixed` format command accepts the following clauses and options: The filename may be enclosed by single quotes, and could be one of the following special values: - + - *inline* The data is found after the end of the parsed commands. Any number @@ -489,63 +489,63 @@ The `fixed` format command accepts the following clauses and options: the data is accepted. - *stdin* - + Reads the data from the standard input stream. The *FROM* option also supports an optional comma separated list of *field* names describing what is expected in the `FIXED` data file. - + Each field name is composed of the field name followed with specific reader options for that field. Supported per-field reader options are the following, where only *start* and *length* are required. - *start* - + Position in the line where to start reading that field's value. Can be entered with decimal digits or `0x` then hexadecimal digits. - *length* - + How many bytes to read from the *start* position to read that field's value. Same format as *start*. - *terminated by* - + See the description of *field terminated by* below. - + The processing of this option is not currently implemented. - + - *date format* - + When the field is expected of the date type, then this option allows to specify the date format used in the file. The processing of this option is not currently implemented. - *null if* - + This option takes an argument which is either the keyword *blanks* or a double-quoted string. - + When *blanks* is used and the field value that is read contains only space characters, then it's automatically converted to an SQL `NULL` value. - + When a double-quoted string is used and that string is read as the field value, then the field value is automatically converted to an SQL `NULL` value. - *INTO* - + The PostgreSQL connection URI must contains the name of the target table where to load the data into. That table must have already been created in PostgreSQL, and the name might be schema qualified. - + Then *INTO* option also supports an optional comma separated list of target columns, which are either the name of an input *field* or the whitespace separated list of the target column name, its PostgreSQL data type and a *USING* expression. - + The *USING* expression can be any valid Common Lisp form and will be read with the current package set to `pgloader.transforms`, so that you can use functions defined in that package, such as functions loaded @@ -553,7 +553,7 @@ The `fixed` format command accepts the following clauses and options: Each *USING* expression is compiled at runtime to native code, and will be called in a context such as: - + (destructuring-bind (field-name-1 field-name-2 ...) row (list column-name-1 @@ -565,25 +565,25 @@ The `fixed` format command accepts the following clauses and options: custom code for that projection. - *WITH* - + When loading from a `CSV` file, the following options are supported: - + - *truncate* - + When this option is listed, pgloader issues a `TRUNCATE` command against the PostgreSQL target table before reading the data file. - + - *skip header* - + Takes a numeric value as argument. Instruct pgloader to skip that many lines at the beginning of the input file. - *SET* - + This clause allows to specify session parameters to be set for all the sessions opened by pgloader. It expects a list of parameter name, the equal sign, then the single-quoted value as a comma separated list. - + The names and values of the parameters are not validated by pgloader, they are given as-is to PostgreSQL. @@ -592,11 +592,11 @@ The `fixed` format command accepts the following clauses and options: You can run SQL queries against the database before loading the data from the `CSV` file. Most common SQL queries are `CREATE TABLE IF NOT EXISTS` so that the data can be loaded. - + Each command must be *dollar-quoted*: it must begin and end with a double dollar sign, `$$`. Dollar-quoted queries are then comma separated. No extra punctuation is expected after the last SQL query. - + - *AFTER LOAD DO* Same format as *BEFORE LOAD DO*, the dollar-quoted queries found in that @@ -611,7 +611,7 @@ example: LOAD DBF FROM http://www.insee.fr/fr/methodes/nomenclatures/cog/telechargement/2013/dbf/reg2013.dbf INTO postgresql://user@localhost/dbname - WITH truncate, create table; + WITH truncate, create table; The `dbf` format command accepts the following clauses and options: @@ -622,37 +622,37 @@ The `dbf` format command accepts the following clauses and options: such a zip file from an HTTP address is of course supported. - *INTO* - + The PostgreSQL connection URI. If it doesn't have a table name in the target, then the name part of the filename will be used as a table name. - *WITH* - + When loading from a `DBF` file, the following options are supported: - + - *truncate* - + When this option is listed, pgloader issues a `TRUNCATE` command against the PostgreSQL target table before reading the data file. - + - *create table* - + When this option is listed, pgloader creates the table using the meta data found in the `DBF` file, which must contain a list of fields with their data type. A standard data type conversion from DBF to PostgreSQL is done. - + - *table name* - + This options expects as its value the possibly qualified name of the table to create. - *SET* - + This clause allows to specify session parameters to be set for all the sessions opened by pgloader. It expects a list of parameter name, the equal sign, then the single-quoted value as a comma separated list. - + The names and values of the parameters are not validated by pgloader, they are given as-is to PostgreSQL. @@ -667,7 +667,7 @@ Here's an example: LOAD ARCHIVE FROM /Users/dim/Downloads/GeoLiteCity-latest.zip INTO postgresql:///ip4r - + BEFORE LOAD DO $$ create extension if not exists ip4r; $$, $$ create schema if not exists geolite; $$, @@ -691,7 +691,7 @@ Here's an example: $$, $$ drop index if exists geolite.blocks_ip4r_idx; $$, $$ truncate table geolite.blocks, geolite.location cascade; $$ - + LOAD CSV FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/ WITH ENCODING iso-8859-1 @@ -716,7 +716,7 @@ Here's an example: fields optionally enclosed by '"', fields escaped by double-quote, fields terminated by ',' - + AND LOAD CSV FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/ WITH ENCODING iso-8859-1 @@ -732,7 +732,7 @@ Here's an example: fields optionally enclosed by '"', fields escaped by double-quote, fields terminated by ',' - + FINALLY DO $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$; @@ -742,11 +742,11 @@ The `archive` command accepts the following clauses and options: Filename or HTTP URI where to load the data from. When given an HTTP URL the linked file will get downloaded locally before processing. - + If the file is a `zip` file, the command line utility `unzip` is used to expand the archive into files in `$TMPDIR`, or `/tmp` if `$TMPDIR` is unset or set to a non-existing directory. - + Then the following commands are used from the top level directory where the archive has been expanded. @@ -760,13 +760,13 @@ The `archive` command accepts the following clauses and options: You can run SQL queries against the database before loading from the data files found in the archive. Most common SQL queries are `CREATE TABLE IF NOT EXISTS` so that the data can be loaded. - + Each command must be *dollar-quoted*: it must begin and end with a double dollar sign, `$$`. Queries are then comma separated. No extra punctuation is expected after the last SQL query. - command [ *AND* command ... ] - + A series of commands against the contents of the archive, at the moment only `CSV`,`'FIXED` and `DBF` commands are supported. @@ -776,13 +776,13 @@ The `archive` command accepts the following clauses and options: The same clause can also be applied to several files with using the spelling *FROM ALL FILENAMES MATCHING* and a regular expression. - + The whole *matching* clause must follow the following rule: - + FROM [ ALL FILENAMES | [ FIRST ] FILENAME ] MATCHING - *FINALLY DO* - + SQL Queries to run once the data is loaded, such as `CREATE INDEX`. ### LOAD MYSQL DATABASE @@ -800,13 +800,13 @@ Here's an example: LOAD DATABASE FROM mysql://root@localhost/sakila INTO postgresql://localhost:54393/sakila - + WITH include drop, create tables, create indexes, reset sequences - + SET maintenance_work_mem to '128MB', work_mem to '12MB', search_path to 'sakila' - + CAST type datetime to timestamptz drop default drop not null using zero-dates-to-null, type date drop not null drop default using zero-dates-to-null, -- type tinyint to boolean using tinyint-to-boolean, @@ -830,7 +830,7 @@ The `database` command accepts the following clauses and options: If the connection URI contains a table name, then only this table is migrated from MySQL to PostgreSQL. - + - *INTO* The target PostgreSQL connection URI. @@ -849,7 +849,7 @@ The `database` command accepts the following clauses and options: automatically from a clean environment. - *include no drop* - + When this option is listed, pgloader will not include any `DROP` statement when loading the data. @@ -857,9 +857,9 @@ The `database` command accepts the following clauses and options: When this option is listed, pgloader issue the `TRUNCATE` command against each PostgreSQL table just before loading data into it. - + - *no truncate* - + When this topion is listed, pgloader issues no `TRUNCATE` command. - *create tables* @@ -870,28 +870,28 @@ The `database` command accepts the following clauses and options: DBF to PostgreSQL is done. - *create no tables* - + When this option is listed, pgloader skips the creation of table before lading data, target tables must then already exist. - *create indexes* - + When this option is listed, pgloader gets the definitions of all the indexes found in the MySQL database and create the same set of index definitions against the PostgreSQL database. - *create no indexes* - + When this option is listed, pgloader skips the creating indexes. - *foreign keys* - + When this option is listed, pgloader gets the definitions of all the foreign keys found in the MySQL database and create the same set of foreign key definitions against the PostgreSQL database. - *no foreign keys* - + When this option is listed, pgloader skips creating foreign keys. - *reset sequences* @@ -900,12 +900,12 @@ The `database` command accepts the following clauses and options: the indexes have all been created, pgloader resets all the PostgreSQL sequences created to the current maximum value of the column they are attached to. - + The options *schema only* and *data only* have no effects on this option. - *reset no sequences* - + When this option is listed, pgloader skips resetting sequences after the load. @@ -917,24 +917,24 @@ The `database` command accepts the following clauses and options: When this option is listed, pgloader converts all MySQL identifiers (table names, index names, column names) to *downcase*, except for PostgreSQL *reserved* keywords. - + The PostgreSQL *reserved* keywords are determined dynamically by using the system function `pg_get_keywords()`. - *quote identifiers* - + When this option is listed, pgloader quotes all MySQL identifiers so that their case is respected. Note that you will then have to do the same thing in your application code queries. - + - *schema only* - + When this option is listed pgloader refrains from migrating the data over. Note that the schema in this context includes the indexes when the option *create indexes* has been listed. - + - *data only* - + When this option is listed pgloader only issues the `COPY` statements, without doing any other processing. @@ -943,17 +943,17 @@ The `database` command accepts the following clauses and options: This clause allows to specify session parameters to be set for all the sessions opened by pgloader. It expects a list of parameter name, the equal sign, then the single-quoted value as a comma separated list. - + The names and values of the parameters are not validated by pgloader, they are given as-is to PostgreSQL. - *CAST* - + The cast clause allows to specify custom casting rules, either to overload the default casting rules or to amend them with special cases. - + A casting rule is expected to follow one of the forms: - + type [ ... ] to [