mirror of
https://github.com/dimitri/pgloader.git
synced 2025-08-10 08:17:00 +02:00
25c79dfebc
The website is moving to pgloader.org and readthedocs.io is going to be integrated. Let's see what happens. The docs build fine locally with the sphinx tools and the docs/Makefile. Having separate files for the documentation should help ease the maintenance and add new topics, such as support for Common Lisp Hackers level docs, which are currently missing.
270 lines
11 KiB
HTML
270 lines
11 KiB
HTML
|
||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
||
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||
|
||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||
<title>Loading Fixed Cols File Formats — pgloader 3.4.1 documentation</title>
|
||
<link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
|
||
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
|
||
<script type="text/javascript">
|
||
var DOCUMENTATION_OPTIONS = {
|
||
URL_ROOT: '../',
|
||
VERSION: '3.4.1',
|
||
COLLAPSE_INDEX: false,
|
||
FILE_SUFFIX: '.html',
|
||
HAS_SOURCE: true,
|
||
SOURCELINK_SUFFIX: '.txt'
|
||
};
|
||
</script>
|
||
<script type="text/javascript" src="../_static/jquery.js"></script>
|
||
<script type="text/javascript" src="../_static/underscore.js"></script>
|
||
<script type="text/javascript" src="../_static/doctools.js"></script>
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="next" title="Loading COPY Formatted Files" href="copy.html" />
|
||
<link rel="prev" title="Loading CSV data" href="csv.html" />
|
||
|
||
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
|
||
|
||
|
||
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
|
||
|
||
</head>
|
||
<body>
|
||
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
<div class="body" role="main">
|
||
|
||
<div class="section" id="loading-fixed-cols-file-formats">
|
||
<h1>Loading Fixed Cols File Formats<a class="headerlink" href="#loading-fixed-cols-file-formats" title="Permalink to this headline">¶</a></h1>
|
||
<p>This command instructs pgloader to load data from a text file containing
|
||
columns arranged in a <em>fixed size</em> manner. Here’s an example:</p>
|
||
<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD FIXED
|
||
FROM inline
|
||
(
|
||
a from 0 for 10,
|
||
b from 10 for 8,
|
||
c from 18 for 8,
|
||
d from 26 for 17 [null if blanks, trim right whitespace]
|
||
)
|
||
INTO postgresql:///pgloader
|
||
TARGET TABLE fixed
|
||
(
|
||
a, b,
|
||
c time using (time-with-no-separator c),
|
||
d
|
||
)
|
||
|
||
WITH truncate
|
||
|
||
SET work_mem to '14MB',
|
||
standard_conforming_strings to 'on'
|
||
|
||
BEFORE LOAD DO
|
||
$$ drop table if exists fixed; $$,
|
||
$$ create table fixed (
|
||
a integer,
|
||
b date,
|
||
c time,
|
||
d text
|
||
);
|
||
$$;
|
||
|
||
01234567892008052011431250firstline
|
||
01234562008052115182300left blank-padded
|
||
12345678902008052208231560another line
|
||
2345609872014092914371500
|
||
2345678902014092914371520
|
||
</pre></div>
|
||
</div>
|
||
<p>The <cite>fixed</cite> format command accepts the following clauses and options.</p>
|
||
<div class="section" id="fixed-file-format-source-specification-from">
|
||
<h2>Fixed File Format Source Specification: FROM<a class="headerlink" href="#fixed-file-format-source-specification-from" title="Permalink to this headline">¶</a></h2>
|
||
<p>Filename where to load the data from. Accepts an <em>ENCODING</em> option. Use the
|
||
<cite>–list-encodings</cite> option to know which encoding names are supported.</p>
|
||
<p>The filename may be enclosed by single quotes, and could be one of the
|
||
following special values:</p>
|
||
<blockquote>
|
||
<div><ul>
|
||
<li><p class="first"><em>inline</em></p>
|
||
<blockquote>
|
||
<div><p>The data is found after the end of the parsed commands. Any number
|
||
of empty lines between the end of the commands and the beginning of
|
||
the data is accepted.</p>
|
||
</div></blockquote>
|
||
</li>
|
||
<li><p class="first"><em>stdin</em></p>
|
||
<blockquote>
|
||
<div><p>Reads the data from the standard input stream.</p>
|
||
</div></blockquote>
|
||
</li>
|
||
<li><p class="first"><em>FILENAMES MATCHING</em></p>
|
||
<p>The whole <em>matching</em> clause must follow the following rule:</p>
|
||
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="p">[</span> <span class="n">ALL</span> <span class="n">FILENAMES</span> <span class="o">|</span> <span class="p">[</span> <span class="n">FIRST</span> <span class="p">]</span> <span class="n">FILENAME</span> <span class="p">]</span>
|
||
<span class="n">MATCHING</span> <span class="n">regexp</span>
|
||
<span class="p">[</span> <span class="n">IN</span> <span class="n">DIRECTORY</span> <span class="s1">'...'</span> <span class="p">]</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>The <em>matching</em> clause applies given <em>regular expression</em> (see above
|
||
for exact syntax, several options can be used here) to filenames.
|
||
It’s then possible to load data from only the first match of all of
|
||
them.</p>
|
||
<p>The optional <em>IN DIRECTORY</em> clause allows specifying which directory
|
||
to walk for finding the data files, and can be either relative to
|
||
where the command file is read from, or absolute. The given
|
||
directory must exists.</p>
|
||
</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
</div>
|
||
<div class="section" id="fields-specifications">
|
||
<h2>Fields Specifications<a class="headerlink" href="#fields-specifications" title="Permalink to this headline">¶</a></h2>
|
||
<p>The <em>FROM</em> option also supports an optional comma separated list of <em>field</em>
|
||
names describing what is expected in the <cite>FIXED</cite> data file.</p>
|
||
<p>Each field name is composed of the field name followed with specific reader
|
||
options for that field. Supported per-field reader options are the
|
||
following, where only <em>start</em> and <em>length</em> are required.</p>
|
||
<blockquote>
|
||
<div><ul>
|
||
<li><p class="first"><em>start</em></p>
|
||
<p>Position in the line where to start reading that field’s value. Can
|
||
be entered with decimal digits or <cite>0x</cite> then hexadecimal digits.</p>
|
||
</li>
|
||
<li><p class="first"><em>length</em></p>
|
||
<p>How many bytes to read from the <em>start</em> position to read that
|
||
field’s value. Same format as <em>start</em>.</p>
|
||
</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
<p>Those optional parameters must be enclosed in square brackets and
|
||
comma-separated:</p>
|
||
<blockquote>
|
||
<div><ul>
|
||
<li><p class="first"><em>terminated by</em></p>
|
||
<blockquote>
|
||
<div><p>See the description of <em>field terminated by</em> below.</p>
|
||
<p>The processing of this option is not currently implemented.</p>
|
||
</div></blockquote>
|
||
</li>
|
||
<li><p class="first"><em>date format</em></p>
|
||
<p>When the field is expected of the date type, then this option allows
|
||
to specify the date format used in the file.</p>
|
||
<p>Date format string are template strings modeled against the
|
||
PostgreSQL <cite>to_char</cite> template strings support, limited to the
|
||
following patterns:</p>
|
||
<blockquote>
|
||
<div><ul class="simple">
|
||
<li>YYYY, YYY, YY for the year part</li>
|
||
<li>MM for the numeric month part</li>
|
||
<li>DD for the numeric day part</li>
|
||
<li>HH, HH12, HH24 for the hour part</li>
|
||
<li>am, AM, a.m., A.M.</li>
|
||
<li>pm, PM, p.m., P.M.</li>
|
||
<li>MI for the minutes part</li>
|
||
<li>SS for the seconds part</li>
|
||
<li>MS for the milliseconds part (4 digits)</li>
|
||
<li>US for the microseconds part (6 digits)</li>
|
||
<li>unparsed punctuation signs: - . * # @ T / and space</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
<p>Here’s an example of a <em>date format</em> specification:</p>
|
||
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">column</span><span class="o">-</span><span class="n">name</span> <span class="p">[</span><span class="n">date</span> <span class="nb">format</span> <span class="s1">'YYYY-MM-DD HH24-MI-SS.US'</span><span class="p">]</span>
|
||
</pre></div>
|
||
</div>
|
||
</li>
|
||
<li><p class="first"><em>null if</em></p>
|
||
<p>This option takes an argument which is either the keyword <em>blanks</em>
|
||
or a double-quoted string.</p>
|
||
<p>When <em>blanks</em> is used and the field value that is read contains only
|
||
space characters, then it’s automatically converted to an SQL <cite>NULL</cite>
|
||
value.</p>
|
||
<p>When a double-quoted string is used and that string is read as the
|
||
field value, then the field value is automatically converted to an
|
||
SQL <cite>NULL</cite> value.</p>
|
||
</li>
|
||
<li><p class="first"><em>trim both whitespace</em>, <em>trim left whitespace</em>, <em>trim right whitespace</em></p>
|
||
<p>This option allows to trim whitespaces in the read data, either from
|
||
both sides of the data, or only the whitespace characters found on
|
||
the left of the streaing, or only those on the right of the string.</p>
|
||
</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
</div>
|
||
<div class="section" id="fixed-file-format-loading-options-with">
|
||
<h2>Fixed File Format Loading Options: WITH<a class="headerlink" href="#fixed-file-format-loading-options-with" title="Permalink to this headline">¶</a></h2>
|
||
<p>When loading from a <cite>FIXED</cite> file, the following options are supported:</p>
|
||
<blockquote>
|
||
<div><ul>
|
||
<li><p class="first"><em>truncate</em></p>
|
||
<p>When this option is listed, pgloader issues a <cite>TRUNCATE</cite> command
|
||
against the PostgreSQL target table before reading the data file.</p>
|
||
</li>
|
||
<li><p class="first"><em>disable triggers</em></p>
|
||
<p>When this option is listed, pgloader issues an <cite>ALTER TABLE …
|
||
DISABLE TRIGGER ALL</cite> command against the PostgreSQL target table
|
||
before copying the data, then the command <cite>ALTER TABLE … ENABLE
|
||
TRIGGER ALL</cite> once the <cite>COPY</cite> is done.</p>
|
||
<p>This option allows loading data into a pre-existing table ignoring
|
||
the <em>foreign key constraints</em> and user defined triggers and may
|
||
result in invalid <em>foreign key constraints</em> once the data is loaded.
|
||
Use with care.</p>
|
||
</li>
|
||
<li><p class="first"><em>skip header</em></p>
|
||
<p>Takes a numeric value as argument. Instruct pgloader to skip that
|
||
many lines at the beginning of the input file.</p>
|
||
</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper"><div class="relations">
|
||
<h3>Related Topics</h3>
|
||
<ul>
|
||
<li><a href="../index.html">Documentation overview</a><ul>
|
||
<li>Previous: <a href="csv.html" title="previous chapter">Loading CSV data</a></li>
|
||
<li>Next: <a href="copy.html" title="next chapter">Loading COPY Formatted Files</a></li>
|
||
</ul></li>
|
||
</ul>
|
||
</div>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3>Quick search</h3>
|
||
<form class="search" action="../search.html" method="get">
|
||
<div><input type="text" name="q" /></div>
|
||
<div><input type="submit" value="Go" /></div>
|
||
<input type="hidden" name="check_keywords" value="yes" />
|
||
<input type="hidden" name="area" value="default" />
|
||
</form>
|
||
</div>
|
||
<script type="text/javascript">$('#searchbox').show(0);</script>
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
<div class="footer">
|
||
©2017, Dimitri Fontaine.
|
||
|
||
|
|
||
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
|
||
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
|
||
|
||
|
|
||
<a href="../_sources/ref/fixed.rst.txt"
|
||
rel="nofollow">Page source</a>
|
||
</div>
|
||
|
||
|
||
|
||
|
||
</body>
|
||
</html> |