Loading csv files to a database has always been a challenging task, without the database tables already created one has to go through the tiring process of manually inspecting the CSVs and creating the DDL queries to create the tables before the data can be loaded.
Again, when dealing with huge csv files, the process might take a very long time to finish.
This is where csv2sql come into the picture.
Csv2sql is a blazing fast tool written in Elixir for loading csv files into the database.
Csv2sql can automatically infer the basic types of data stored in your csv file and create the database tables and load the data all with one single command.
Got a directory full of large csvs that needs to be loaded to a database?
With csv2sql it’s as simple as running a simple command like..
./csv2sql --source-csv-directory "/home/user/Desktop/csvs" --db-connection-string "username:password@localhost/my_database"
That’s all ! Csv2Sql will analyze the files and insert them in to inferred data tables.
After it is done, it will give you a summary of the results.
Is it fast ?
Yes, csv2sql is made using elixir which makes the most of your cpu power by processing the csvs parallely thus greatly reducing the overall time taken, so it’s perfect for huge csv files.
Dependencies ?
Csv2sql only require erlang to be installed, that’s it.
Find the csv2sql executable here.
Finally if you want to know how it works or if you are interested in elixir or genservers, find the repository here, also this project is open source and all your contributions are welcome.
Good Day !