Querying CSV Files with SQL using ‘q’ Presented by Simon Frank

Context Working with data often means dealing with delimited files (CSV, TSV, etc…) Import delimited file(s) into data model Export data from data model into delimited file(s)

Some Common Challenges File not in correct format Column order screwed up Incorrect formatting Categorical values not correctly mapped Data contains duplicates Data incomplete / different from the previous file And so on…

Cry Me A River, Buddy Stop complaining and deal with it! Fix it with your text editor Edit it in Excel Ever heard of Grep, Awk or Sed? What about a.Net console app or a Python script? BCP it into SQL Server and query it from there

Finally – The Point Dealing with import and export files suck We have more interesting things to do Quick and easy to finish the job with SQL We already know SQL!

Enter q Command line tool to run SQL queries on delimited files Implementation: In-memory table Supports all sqlite3 query constructs

DEMO

FREEDOM !