mirror of
https://github.com/dimitri/pgloader.git
synced 2025-08-10 08:17:00 +02:00
Review the TODO list.
This commit is contained in:
parent
24c0580e11
commit
dbc9699940
93
TODO.md
93
TODO.md
@ -4,76 +4,79 @@ Some notes about what I intend to be working on next. You can sponsor any
|
||||
and all of those ideas if you actually need them today, and you can also
|
||||
sponsor new ideas not on the list yet.
|
||||
|
||||
## New Features
|
||||
|
||||
### Filtering
|
||||
|
||||
Add commands to pick different target tables depending on the data found
|
||||
when reading from the source.
|
||||
|
||||
## Data Formats
|
||||
|
||||
## MySQL Support
|
||||
### CSV
|
||||
|
||||
- see about schema discovery (column names and types)
|
||||
|
||||
### JSON
|
||||
|
||||
Propose to load JSON either in a "document" column, or to normalize it by
|
||||
applying some advanced filtering.
|
||||
|
||||
Implement PostgreSQL JSON operators and functions in pgloader to help setup
|
||||
the normalisation steps:
|
||||
[PostgreSQL JSON Functions and Operators](http://www.postgresql.org/docs/9.3/interactive/functions-json.html).
|
||||
|
||||
### XML
|
||||
|
||||
Add an XML reader to load XML documents into the database as a column value,
|
||||
and XSLT capabilities to normalize the XML contents into a proper relational
|
||||
model.
|
||||
|
||||
### Other databases
|
||||
|
||||
Add support for full data and schema migrations for the following:
|
||||
|
||||
- SQL Server
|
||||
- Sybase
|
||||
- Oracle
|
||||
|
||||
## Database support
|
||||
|
||||
### MySQL Support
|
||||
|
||||
- MATERIALIZE ALL VIEWS
|
||||
- Convert SQL dialect for SQL views
|
||||
- Triggers and Stored Procedures
|
||||
|
||||
### SQLite support
|
||||
|
||||
- implement CAST rules support
|
||||
|
||||
## Compat
|
||||
|
||||
- add parsing for SQL*Loader file format
|
||||
|
||||
## Other
|
||||
|
||||
### binary distribution
|
||||
|
||||
- prepare an all-included binary for several platforms
|
||||
|
||||
### docs
|
||||
|
||||
- host a proper website for the tool, with use cases and a tutorial
|
||||
|
||||
### error management
|
||||
|
||||
- error management with a local buffer (done)
|
||||
- error reporting (done)
|
||||
- add input line number to log file?
|
||||
|
||||
### data output
|
||||
|
||||
- PostgreSQL COPY Text format output for any supported input
|
||||
|
||||
#### convenience
|
||||
|
||||
- automatic creation of schema even when loading from text files
|
||||
- pre-fetch some rows to guesstimate data types?
|
||||
|
||||
#### performances
|
||||
### performances
|
||||
|
||||
- some more parallelizing options
|
||||
- support for partitionning in pgloader itself
|
||||
|
||||
#### reformating
|
||||
|
||||
Data reformating is now going to have to happen in Common Lisp mostly, maybe
|
||||
offer some other languages (cl-awk etc).
|
||||
|
||||
- raw reformating, before rows are split
|
||||
|
||||
Have a try at something approaching:
|
||||
|
||||
WITH data AS (
|
||||
COPY FROM ...
|
||||
RETURNING x, y
|
||||
)
|
||||
SELECT foo(x), bar(y)
|
||||
FROM data
|
||||
WHERE ...
|
||||
|
||||
A part of that needs to happen client-side, another part server-side, and
|
||||
the grammar has to make it clear what happens where. Maybe add a WHERE
|
||||
clause to the `COPY` or `LOAD` grammar for the client.
|
||||
|
||||
#### filtering
|
||||
|
||||
Add commands to pick different target tables depending on the data found
|
||||
when reading from the source.
|
||||
|
||||
#### UI
|
||||
### UI
|
||||
|
||||
- add a web controler with pretty monitoring
|
||||
- launch new jobs from the web controler
|
||||
|
||||
#### crazy ideas
|
||||
### crazy ideas
|
||||
|
||||
- MySQL replication, reading from the binlog directly
|
||||
- plproxy (re-)sharding support
|
||||
|
Loading…
Reference in New Issue
Block a user