© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 1 HiVertica Capstone Project University of Pittsburgh January 11, 2013 Stephen Walkauskas, Architect, Data Management, Vertica

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2 Contact info Stephen Walkauskas

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3 Vertica culture

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4 What Is Vertica SQL Database for Real-time Analytics Runs on x86 hardware MPP Columnar Architecture – scales to PBs! Reduced footprint via Advanced Compression Extensible analytics capabilities Easy to setup and use Elastic - grow/shrink as needed Extensive Ecosystem of analytic tools Speed Scale Simplicity

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5 Map/Reduce

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice HQL SELECT a.val1, a.val2, b.val, c.val FROM a JOIN b ON (a.key = b.key) LEFT OUTER JOIN c ON (a.key = c.key)

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7 HiVertica

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8 HiVertica a) Write code to read Hive / HCatalog meta-data and generate DDL to create corresponding external tables (ETs) in a Vertica DB. b) Configure ETs with files referenced by the corresponding Hive tables. Vertica ships a connector to source files from hdfs. Using this connector the aforementioned ETs can be used to query data in Hive (assuming data is in a format Vertica can parse).

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9 HiVertica c) Vertica supports User Defined Parsers (you can write your own csv parser if you’re so inclined). RCFile is commonly used to store data in Hive. It would be useful to be able to parse that format in a Vertica UDParser. d)Find that place in Hive where it compiles HQL into M/R jobs and instead rename the HQL to SQL and, leveraging the above features, send the query to Vertica instead. The two systems are not 100%; we can tweak them to shrink the feature gap.

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10 Thanks!