mirror of
https://github.com/dimitri/pgloader.git
synced 2025-08-10 00:07:00 +02:00
25c79dfebc
The website is moving to pgloader.org and readthedocs.io is going to be integrated. Let's see what happens. The docs build fine locally with the sphinx tools and the docs/Makefile. Having separate files for the documentation should help ease the maintenance and add new topics, such as support for Common Lisp Hackers level docs, which are currently missing.
151 lines
6.6 KiB
HTML
151 lines
6.6 KiB
HTML
|
||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
||
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||
|
||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||
<title>Introduction — pgloader 3.4.1 documentation</title>
|
||
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
|
||
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
|
||
<script type="text/javascript">
|
||
var DOCUMENTATION_OPTIONS = {
|
||
URL_ROOT: './',
|
||
VERSION: '3.4.1',
|
||
COLLAPSE_INDEX: false,
|
||
FILE_SUFFIX: '.html',
|
||
HAS_SOURCE: true,
|
||
SOURCELINK_SUFFIX: '.txt'
|
||
};
|
||
</script>
|
||
<script type="text/javascript" src="_static/jquery.js"></script>
|
||
<script type="text/javascript" src="_static/underscore.js"></script>
|
||
<script type="text/javascript" src="_static/doctools.js"></script>
|
||
<link rel="index" title="Index" href="genindex.html" />
|
||
<link rel="search" title="Search" href="search.html" />
|
||
<link rel="next" title="PgLoader Tutorial" href="tutorial/tutorial.html" />
|
||
<link rel="prev" title="Welcome to pgloader’s documentation!" href="index.html" />
|
||
|
||
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
|
||
|
||
|
||
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
|
||
|
||
</head>
|
||
<body>
|
||
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
<div class="body" role="main">
|
||
|
||
<div class="section" id="introduction">
|
||
<h1>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h1>
|
||
<p>pgloader loads data from various sources into PostgreSQL. It can
|
||
transform the data it reads on the fly and submit raw SQL before and
|
||
after the loading. It uses the <cite>COPY</cite> PostgreSQL protocol to stream
|
||
the data into the server, and manages errors by filling a pair of
|
||
<em>reject.dat</em> and <em>reject.log</em> files.</p>
|
||
<p>pgloader knows how to read data from different kind of sources:</p>
|
||
<blockquote>
|
||
<div><ul class="simple">
|
||
<li>Files
|
||
* CSV
|
||
* Fixed Format
|
||
* DBF</li>
|
||
<li>Databases
|
||
* SQLite
|
||
* MySQL
|
||
* MS SQL Server</li>
|
||
</ul>
|
||
</div></blockquote>
|
||
<p>The level of automation provided by pgloader depends on the data source
|
||
type. In the case of CSV and Fixed Format files, a full description of the
|
||
expected input properties must be given to pgloader. In the case of a
|
||
database, pgloader connects to the live service and knows how to fetch the
|
||
metadata it needs directly from it.</p>
|
||
<div class="section" id="continuous-migration">
|
||
<h2>Continuous Migration<a class="headerlink" href="#continuous-migration" title="Permalink to this headline">¶</a></h2>
|
||
<p>pgloader is meant to migrate a whole database in a single command line and
|
||
without any manual intervention. The goal is to be able to setup a
|
||
<em>Continuous Integration</em> environment as described in the <a class="reference external" href="http://mysqltopgsql.com/project/">Project
|
||
Methodology</a> document of the <a class="reference external" href="http://mysqltopgsql.com/project/">MySQL to
|
||
PostgreSQL</a> webpage.</p>
|
||
<blockquote>
|
||
<div><ol class="arabic simple">
|
||
<li>Setup your target PostgreSQL Architecture</li>
|
||
<li>Fork a Continuous Integration environment that uses PostgreSQL</li>
|
||
<li>Migrate the data over and over again every night, from production</li>
|
||
<li>As soon as the CI is all green using PostgreSQL, schedule the D-Day</li>
|
||
<li>Migrate without suprise and enjoy!</li>
|
||
</ol>
|
||
</div></blockquote>
|
||
<p>In order to be able to follow this great methodology, you need tooling to
|
||
implement the third step in a fully automated way. That’s pgloader.</p>
|
||
</div>
|
||
<div class="section" id="commands">
|
||
<h2>Commands<a class="headerlink" href="#commands" title="Permalink to this headline">¶</a></h2>
|
||
<p>pgloader implements its own <em>Command Language</em>, a DSL that allows to specify
|
||
every aspect of the data load and migration to implement. Some of the
|
||
features provided in the language are only available for a specific source
|
||
type.</p>
|
||
</div>
|
||
<div class="section" id="command-line">
|
||
<h2>Command Line<a class="headerlink" href="#command-line" title="Permalink to this headline">¶</a></h2>
|
||
<p>The pgloader command line accepts those two variants:</p>
|
||
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">pgloader</span> <span class="p">[</span><span class="o"><</span><span class="n">options</span><span class="o">></span><span class="p">]</span> <span class="p">[</span><span class="o"><</span><span class="n">command</span><span class="o">-</span><span class="n">file</span><span class="o">></span><span class="p">]</span><span class="o">...</span>
|
||
<span class="n">pgloader</span> <span class="p">[</span><span class="o"><</span><span class="n">options</span><span class="o">></span><span class="p">]</span> <span class="n">SOURCE</span> <span class="n">TARGET</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>Either you have a <em>command-file</em> containing migration specifications in the
|
||
pgloader <em>Command Language</em>, or you can give a <em>Source</em> for the data and a
|
||
PostgreSQL database connection <em>Target</em> where to load the data into.</p>
|
||
</div>
|
||
</div>
|
||
|
||
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper"><div class="relations">
|
||
<h3>Related Topics</h3>
|
||
<ul>
|
||
<li><a href="index.html">Documentation overview</a><ul>
|
||
<li>Previous: <a href="index.html" title="previous chapter">Welcome to pgloader’s documentation!</a></li>
|
||
<li>Next: <a href="tutorial/tutorial.html" title="next chapter">PgLoader Tutorial</a></li>
|
||
</ul></li>
|
||
</ul>
|
||
</div>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3>Quick search</h3>
|
||
<form class="search" action="search.html" method="get">
|
||
<div><input type="text" name="q" /></div>
|
||
<div><input type="submit" value="Go" /></div>
|
||
<input type="hidden" name="check_keywords" value="yes" />
|
||
<input type="hidden" name="area" value="default" />
|
||
</form>
|
||
</div>
|
||
<script type="text/javascript">$('#searchbox').show(0);</script>
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
<div class="footer">
|
||
©2017, Dimitri Fontaine.
|
||
|
||
|
|
||
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.5</a>
|
||
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
|
||
|
||
|
|
||
<a href="_sources/intro.rst.txt"
|
||
rel="nofollow">Page source</a>
|
||
</div>
|
||
|
||
|
||
|
||
|
||
</body>
|
||
</html> |