mirror of
https://github.com/dimitri/pgloader.git
synced 2025-08-10 08:17:00 +02:00
25c79dfebc
The website is moving to pgloader.org and readthedocs.io is going to be integrated. Let's see what happens. The docs build fine locally with the sphinx tools and the docs/Makefile. Having separate files for the documentation should help ease the maintenance and add new topics, such as support for Common Lisp Hackers level docs, which are currently missing.
191 lines
7.6 KiB
HTML
191 lines
7.6 KiB
HTML
|
||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
||
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||
|
||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||
<title>Loading From an Archive — pgloader 3.4.1 documentation</title>
|
||
<link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
|
||
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
|
||
<script type="text/javascript">
|
||
var DOCUMENTATION_OPTIONS = {
|
||
URL_ROOT: '../',
|
||
VERSION: '3.4.1',
|
||
COLLAPSE_INDEX: false,
|
||
FILE_SUFFIX: '.html',
|
||
HAS_SOURCE: true,
|
||
SOURCELINK_SUFFIX: '.txt'
|
||
};
|
||
</script>
|
||
<script type="text/javascript" src="../_static/jquery.js"></script>
|
||
<script type="text/javascript" src="../_static/underscore.js"></script>
|
||
<script type="text/javascript" src="../_static/doctools.js"></script>
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="next" title="Migrating a MySQL Database to PostgreSQL" href="mysql.html" />
|
||
<link rel="prev" title="Loading IXF Data" href="ixf.html" />
|
||
|
||
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
|
||
|
||
|
||
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
|
||
|
||
</head>
|
||
<body>
|
||
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
<div class="body" role="main">
|
||
|
||
<div class="section" id="loading-from-an-archive">
|
||
<h1>Loading From an Archive<a class="headerlink" href="#loading-from-an-archive" title="Permalink to this headline">¶</a></h1>
|
||
<p>This command instructs pgloader to load data from one or more files contained
|
||
in an archive. Currently the only supported archive format is <em>ZIP</em>, and the
|
||
archive might be downloaded from an <em>HTTP</em> URL.</p>
|
||
<p>Here’s an example:</p>
|
||
<div class="highlight-default"><div class="highlight"><pre><span></span>LOAD ARCHIVE
|
||
FROM /Users/dim/Downloads/GeoLiteCity-latest.zip
|
||
INTO postgresql:///ip4r
|
||
|
||
BEFORE LOAD
|
||
DO $$ create extension if not exists ip4r; $$,
|
||
$$ create schema if not exists geolite; $$,
|
||
|
||
EXECUTE 'geolite.sql'
|
||
|
||
LOAD CSV
|
||
FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
|
||
WITH ENCODING iso-8859-1
|
||
(
|
||
locId,
|
||
country,
|
||
region null if blanks,
|
||
city null if blanks,
|
||
postalCode null if blanks,
|
||
latitude,
|
||
longitude,
|
||
metroCode null if blanks,
|
||
areaCode null if blanks
|
||
)
|
||
INTO postgresql:///ip4r?geolite.location
|
||
(
|
||
locid,country,region,city,postalCode,
|
||
location point using (format nil "(~a,~a)" longitude latitude),
|
||
metroCode,areaCode
|
||
)
|
||
WITH skip header = 2,
|
||
fields optionally enclosed by '"',
|
||
fields escaped by double-quote,
|
||
fields terminated by ','
|
||
|
||
AND LOAD CSV
|
||
FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
|
||
WITH ENCODING iso-8859-1
|
||
(
|
||
startIpNum, endIpNum, locId
|
||
)
|
||
INTO postgresql:///ip4r?geolite.blocks
|
||
(
|
||
iprange ip4r using (ip-range startIpNum endIpNum),
|
||
locId
|
||
)
|
||
WITH skip header = 2,
|
||
fields optionally enclosed by '"',
|
||
fields escaped by double-quote,
|
||
fields terminated by ','
|
||
|
||
FINALLY DO
|
||
$$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;
|
||
</pre></div>
|
||
</div>
|
||
<p>The <cite>archive</cite> command accepts the following clauses and options.</p>
|
||
<div class="section" id="archive-source-specification-from">
|
||
<h2>Archive Source Specification: FROM<a class="headerlink" href="#archive-source-specification-from" title="Permalink to this headline">¶</a></h2>
|
||
<p>Filename or HTTP URI where to load the data from. When given an HTTP URL the
|
||
linked file will get downloaded locally before processing.</p>
|
||
<p>If the file is a <cite>zip</cite> file, the command line utility <cite>unzip</cite> is used to
|
||
expand the archive into files in <cite>$TMPDIR</cite>, or <cite>/tmp</cite> if <cite>$TMPDIR</cite> is unset
|
||
or set to a non-existing directory.</p>
|
||
<p>Then the following commands are used from the top level directory where the
|
||
archive has been expanded.</p>
|
||
</div>
|
||
<div class="section" id="archive-sub-commands">
|
||
<h2>Archive Sub Commands<a class="headerlink" href="#archive-sub-commands" title="Permalink to this headline">¶</a></h2>
|
||
<blockquote>
|
||
<div><ul>
|
||
<li><p class="first">command [ <em>AND</em> command … ]</p>
|
||
<p>A series of commands against the contents of the archive, at the moment
|
||
only <cite>CSV</cite>,`’FIXED` and <cite>DBF</cite> commands are supported.</p>
|
||
<p>Note that commands are supporting the clause <em>FROM FILENAME MATCHING</em>
|
||
which allows the pgloader command not to depend on the exact names of
|
||
the archive directories.</p>
|
||
<p>The same clause can also be applied to several files with using the
|
||
spelling <em>FROM ALL FILENAMES MATCHING</em> and a regular expression.</p>
|
||
<p>The whole <em>matching</em> clause must follow the following rule:</p>
|
||
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">FROM</span> <span class="p">[</span> <span class="n">ALL</span> <span class="n">FILENAMES</span> <span class="o">|</span> <span class="p">[</span> <span class="n">FIRST</span> <span class="p">]</span> <span class="n">FILENAME</span> <span class="p">]</span> <span class="n">MATCHING</span>
|
||
</pre></div>
|
||
</div>
|
||
</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
</div>
|
||
<div class="section" id="archive-final-sql-commands">
|
||
<h2>Archive Final SQL Commands<a class="headerlink" href="#archive-final-sql-commands" title="Permalink to this headline">¶</a></h2>
|
||
<blockquote>
|
||
<div><ul>
|
||
<li><p class="first"><em>FINALLY DO</em></p>
|
||
<p>SQL Queries to run once the data is loaded, such as <cite>CREATE INDEX</cite>.</p>
|
||
</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper"><div class="relations">
|
||
<h3>Related Topics</h3>
|
||
<ul>
|
||
<li><a href="../index.html">Documentation overview</a><ul>
|
||
<li>Previous: <a href="ixf.html" title="previous chapter">Loading IXF Data</a></li>
|
||
<li>Next: <a href="mysql.html" title="next chapter">Migrating a MySQL Database to PostgreSQL</a></li>
|
||
</ul></li>
|
||
</ul>
|
||
</div>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3>Quick search</h3>
|
||
<form class="search" action="../search.html" method="get">
|
||
<div><input type="text" name="q" /></div>
|
||
<div><input type="submit" value="Go" /></div>
|
||
<input type="hidden" name="check_keywords" value="yes" />
|
||
<input type="hidden" name="area" value="default" />
|
||
</form>
|
||
</div>
|
||
<script type="text/javascript">$('#searchbox').show(0);</script>
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
<div class="footer">
|
||
©2017, Dimitri Fontaine.
|
||
|
||
|
|
||
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
|
||
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
|
||
|
||
|
|
||
<a href="../_sources/ref/archive.rst.txt"
|
||
rel="nofollow">Page source</a>
|
||
</div>
|
||
|
||
|
||
|
||
|
||
</body>
|
||
</html> |