From 6a7f3c2417edd241e2211148f14e90e9746b0668 Mon Sep 17 00:00:00 2001 From: Dimitri Fontaine Date: Sun, 29 Jun 2014 16:06:09 +0200 Subject: [PATCH] Add some docs about oversized batches, fix #86. --- pgloader.1 | 5 ++++- pgloader.1.md | 7 ++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/pgloader.1 b/pgloader.1 index 8accc42..782918c 100644 --- a/pgloader.1 +++ b/pgloader.1 @@ -385,7 +385,7 @@ the delimiters \fB/*\fR and \fB*/\fR respectively start and end a comment, which Any place where you could enter a \fIwhitespace\fR will accept a comment too\. . .SS "Batch behaviour options" -All pgloader commands have support for a \fIWITH\fR clause that allows for specifying options\. Some options are generic and accepted by all commands, such as the \fIbatch behaviour options\fR, and some options are specific to a data source kind, such as the CSV \fIskip header\fR options\. +All pgloader commands have support for a \fIWITH\fR clause that allows for specifying options\. Some options are generic and accepted by all commands, such as the \fIbatch behaviour options\fR, and some options are specific to a data source kind, such as the CSV \fIskip header\fR option\. . .P The global batch behaviour options are: @@ -416,6 +416,9 @@ Supporting more than a single batch being sent at a time is on the TODO list of .P Other options are specific to each input source, please refer to specific parts of the documentation for their listing and covering\. . +.P +A batch is then closed as soon as either the \fIbatch rows\fR or the \fIbatch size\fR threshold is crossed, whichever comes first\. In cases when a batch has to be closed because of the \fIbatch size\fR setting, a \fIdebug\fR level log message is printed with how many rows did fit in the \fIoversized\fR batch\. +. .SH "LOAD CSV" This command instructs pgloader to load data from a \fBCSV\fR file\. Here\'s an example: . diff --git a/pgloader.1.md b/pgloader.1.md index ecf8a2d..fa82db6 100644 --- a/pgloader.1.md +++ b/pgloader.1.md @@ -343,7 +343,7 @@ Any place where you could enter a *whitespace* will accept a comment too. All pgloader commands have support for a *WITH* clause that allows for specifying options. Some options are generic and accepted by all commands, such as the *batch behaviour options*, and some options are specific to a -data source kind, such as the CSV *skip header* options. +data source kind, such as the CSV *skip header* option. The global batch behaviour options are: @@ -376,6 +376,11 @@ The global batch behaviour options are: Other options are specific to each input source, please refer to specific parts of the documentation for their listing and covering. +A batch is then closed as soon as either the *batch rows* or the *batch +size* threshold is crossed, whichever comes first. In cases when a batch has +to be closed because of the *batch size* setting, a *debug* level log +message is printed with how many rows did fit in the *oversized* batch. + ## LOAD CSV This command instructs pgloader to load data from a `CSV` file. Here's an