From dbc96999406fe14fc8c8bc11d2cf23c8eba7a8ae Mon Sep 17 00:00:00 2001
From: Dimitri Fontaine <dim@tapoueh.org>
Date: Mon, 26 May 2014 00:04:50 +0200
Subject: [PATCH] Review the TODO list.

---
 TODO.md | 93 +++++++++++++++++++++++++++++----------------------------
 1 file changed, 48 insertions(+), 45 deletions(-)

diff --git a/TODO.md b/TODO.md
index 32dcb0f..d36d47c 100644
--- a/TODO.md
+++ b/TODO.md
@@ -4,76 +4,79 @@ Some notes about what I intend to be working on next. You can sponsor any
 and all of those ideas if you actually need them today, and you can also
 sponsor new ideas not on the list yet.
 
+## New Features
+
+### Filtering
+
+Add commands to pick different target tables depending on the data found
+when reading from the source.
+
 ## Data Formats
 
-## MySQL Support
+### CSV
+
+  - see about schema discovery (column names and types)
+
+### JSON
+
+Propose to load JSON either in a "document" column, or to normalize it by
+applying some advanced filtering.
+
+Implement PostgreSQL JSON operators and functions in pgloader to help setup
+the normalisation steps:
+[PostgreSQL JSON Functions and Operators](http://www.postgresql.org/docs/9.3/interactive/functions-json.html).
+
+### XML
+
+Add an XML reader to load XML documents into the database as a column value,
+and XSLT capabilities to normalize the XML contents into a proper relational
+model.
+
+### Other databases
+
+Add support for full data and schema migrations for the following:
+
+  - SQL Server
+  - Sybase
+  - Oracle
+
+## Database support
+
+### MySQL Support
 
   - MATERIALIZE ALL VIEWS
   - Convert SQL dialect for SQL views
   - Triggers and Stored Procedures
+  
+### SQLite support
+
+  - implement CAST rules support
+
+## Compat
+
+  - add parsing for SQL*Loader file format
 
 ## Other
 
-### binary distribution
-
-  - prepare an all-included binary for several platforms
-  
-### docs
-
-  - host a proper website for the tool, with use cases and a tutorial
-
 ### error management
 
-  - error management with a local buffer (done)
-  - error reporting (done)
   - add input line number to log file?
 
 ### data output
 
   - PostgreSQL COPY Text format output for any supported input
 
-#### convenience
-
-  - automatic creation of schema even when loading from text files
-  - pre-fetch some rows to guesstimate data types?
-
-#### performances
+### performances
 
   - some more parallelizing options
   - support for partitionning in pgloader itself
 
-#### reformating
-
-Data reformating is now going to have to happen in Common Lisp mostly, maybe
-offer some other languages (cl-awk etc).
-
-  - raw reformating, before rows are split
-
-Have a try at something approaching:
-
-    WITH data AS (
-		COPY FROM ...
-		RETURNING x, y
-	)
-	SELECT foo(x), bar(y)
-	  FROM data
-	 WHERE ...
-
-A part of that needs to happen client-side, another part server-side, and
-the grammar has to make it clear what happens where. Maybe add a WHERE
-clause to the `COPY` or `LOAD` grammar for the client.
-
-#### filtering
-
-Add commands to pick different target tables depending on the data found
-when reading from the source.
-
-#### UI
+### UI
 
   - add a web controler with pretty monitoring
   - launch new jobs from the web controler
 
-#### crazy ideas
+### crazy ideas
 
   - MySQL replication, reading from the binlog directly
   - plproxy (re-)sharding support