Accelerating Business Velocity at Etsy Chris Bohn “CB” Senior Database Engineer 1 Cop©yCrigohpty2ri0g1h4t 2H0e1w4leHtet-wPlaecttk-aPradcDkaervdeDloepvmeelonpt mCeonmtpCaonmy,pLa.nPy., LT.hPe. inTfhoerminaftoiormn actoionntacinoendtahineeredinheisresinubisjescut btojeccht atongcehawnigtheowutitnhootuicten.otice. ©

Etsy.com Leading online etailer of handmade artisan crafts Sales of $2.3 Billion in 2015, high growth rate 24 million active buyers, 1.6 million active shops Sales in 90% of the world's countries

Etsy Production Data Architecture Lookup Index EtsyORM MySQL shards PostgreSQL Master MySQL Ticker Server

The Etsy BI Problem MySQL shards PostgreSQL master Putting all production back to one BI server simply recreated our earlier scaling problem:

Business Intelligence Use Cases Repository of clickstream data Data analyst ad-hoc queries A/B test analysis Financial report generation Monitoring dashboards

The BI Replacement Search Key Requirements: Unlimited scale Non-disruptive replication from master Postgres and sharded MySQL relational databases Easy replication of Hadoop data Easily adapt existing queries from legacy Postgres

Evaluating Analytic Databases Reviewed all major players with specifically designed fast aggregation and analytic queries Narrowed the list of analytic database systems of contenders Requirements : Commodity hardware Easy scalability by adding additional nodes Reasonable cost Trusted partner

The Vertica Solution Etsy selected Vertica: Outstanding analytic query performance Rich analytic function support, including windowing functions Runs on commodity hardware Enabled in-house replication solutions SQL language very compatible with SQL92 Cost efective

Replication to Vertica Data replication was essential - Vertica made it easy: Vertica derived from Postgres (uses similar COPY command) Etsy created its own replication toolset called “Schlep” that pipes data into Vertica using the COPY command Very high performance bulk data loading Good trickle load performance once tuned

Vertica User Benefits Very high compatibility with existing queries written for Postgres BI server; most could run on Vertica unchanged Excellent SQL language support provided very fast time to productivity for our analysts Excellent ODBC support, which allowed custom query management tools

Vertica: Indispensable to Etsy Data Stack Used by data analysts for almost all ad-hoc queries correlating clickstream data with production data; excels for A/B test analysis Enables deep financial analysis - our accountants spend less time with auditors Powers dashboards that let Etsy monitor its operations Enables more detailed fraud analysis more quickly

Vertica Performance Exceeded Expectations Queries that took several days on the legacy Postgres BI server now complete in minutes All clickstream data and all A/B tests are analyzed in Vertica with SQL Vertica enables deep financial analysis Hadoop is hard; replaced many Hadoop jobs with Vertica queries

Vertica in Use Use internal database and operations staf to integrate Vertica Client vsql tool easy to use, similar to Postgres psql tool Excellent support for monitoring/graphing (use nagios & ganglia) Easy installation Excellent, responsive support from HP

Vertica (Good) Surprises Started with 10 TB license on 5 nodes Internal demand led to renew license at 130 TB with 20 nodes Allowed deep financial analysis previously not possible Able to run many A/B tests because Vertica analytic functions make all that huge data accessible Works in concert with Hadoop; always try to see if a Vertica query can replace a map reduce job

Summary of Benefits Vertica has become an essential technology at Etsy because it has greatly accelerated our business velocity Quicker turnaround for analysts Analysts know & love SQL, much more efficient than Hadoop jobs Tamed our huge clickstream data Provides deeper insight more quickly into our business Cost effective, backed by HPE

Questions? Thank You for Your Time! Chris Bohn “CB” -