data & analytics beyond dashboards moshirm@amazon.fr

Data on AWS AWS platform offers Storage Databases Datawarehouses Bi Tools Hadoop as a service Machine Learning as a service ETL as a service Natural Language Processing

Data on AWS Well, AWS it offers a little more : more than Storage ………........ S3 is called by our customers « The Cloud Datwarehouse  » Databases choice …………………isv, oss, sql, nosql, aurora, with backups monitoring, replication included trusted Datawarehouses ……..The same DW which powers Amazon : Redshift, $1k /TB/Year integrated Bi Tools …….……….. In memory processing which integrates with every data source on AWS mature Hadoop ……………….…. The first hadoop as a service on the market simple ML as a service ………….The same algorithms which powers Amazon zero admin ETL …………………... Serverless spark based ETL successful NLP ………………………The same which powers Amazon Alexa GPU for all ………………………. New instance types with pluggable GPU Deep learning OSS ……………… AWS contributes to Mxnet, an open source DL framework

Data on AWS And we innovate (a lot): This Year (and counting) Last year … Kinesis Analytics : select * from stream Quicksight : Bi as a service Firehose : stream directly to your database This Year (and counting) Athena : Serverless Presto (in EU!) Glue : Spark as a service for ETL (in preview) Spectrum : select * from S3 / use Redshift to query S3 with

Data on AWS The single thing you should remember today Amazon S3 is a big opportunity for any data practitionner Before you ask, yes, it’s secured and data are encrypted You can replace HDFS completely and save moeny and sleep EMRFS = HDFS api on top of S3, so Spark, Hive, Presto sees it as HDFS You can now query data in S3 directly from Redshift It has monitoring/auditing/versionning/lifecycle management done It replicates data for you for $0.023/GB/m

Data on AWS But Data Locality ?! Well, seems like some of our customers find S3 performances & costs just fine

Data on AWS But what about Data Locality ?! https://www.youtube.com/watch?v=N7_u1b18yGg

Data on AWS But what about Data Locality ?! https://www.youtube.com/watch?v=N7_u1b18yGg

Data on AWS Catch this opportunity to share, process, expose, analyze your data Some reasons not to : You like to configure HADOOP replication  and patch YARN on your weekends You thought AWS was just selling excessive capacity You might think Amazon is going to take their data, in which case look at this

Data on AWS Security ? https://aws.amazon.com/blogs/big-data/low-latency-access-on-trillions-of-records-finras-architecture-using-apache-hbase-on-amazon-emr-with-amazon-s3/

Ok, we got it ! Did I mention you pay everything as a service ?! And you can start today for free

So, what can we do with all this ? be Creative!

So, what can we do with all this ? Cloud = 0 * low value tasks + primitives primitives

So, what can we do with all this ? Kinesis Dybamodb API Gateway S3 EC2 Codepipeline AWS ML Quicksight Memcache Redshift EMR Glue Spectrum ECS Aurora Mysql, Postgres Oracle/MSSQL Redis 20+ data services on aws

So, what can we do with all this ? Traditional BI (for the past 20 years)

So, what can we do with all this ? New data ingestion ? streams Streams processing Hydrate API

So, what can we do with all this ? Lambda Source Kinesis Firehose Redshift Clients Dynamodb

So, what can we do with all this ? Are people using dashboards ? Business analysts might use dashboard But no-tech savvy users dont use dashboards

So, what can we do with all this ? Ok, what is the N° 1requirement after we deployed a BI tool to production ? Export to Excel !

So, what can we do with all this ? What people want from the data ? We [data engineers/scientists/analysts] care about data Bu people cares about ANSWERS TO QUESTIONS ACTIONS THEY CAN TAKE

my-data x https://jsdev.com/ mydata Catalog Collections Bots Query Apps Blog APIs Help Profile Logout Bots Sales Bot me 11:35 How many sales where made this week online ? Salesbot 11:35 I looked at the data and I see that 12345 sales were made this week, 13% increase across last week me 11:35 Alert me when number sales decrease Salesbot 11:35 ok, will add this alert for you whenever number of sales per week decrease How many sales where made this week online ? Alert me when number sales decrease send

Can we build this on AWS ? User age country Order user product Product department aisle Department aisle Aisle

Can we build this on AWS ? Demo

Can we build this on AWS ? Well, yes The easy part : The funny part Stream data coming from operation systems to S3 Use Redshift Spectrum to query data The funny part Plug a LEX powered chatbot to Slack Process request with Lambda

Can we build this on AWS ? Sales Bot

Can we build this on AWS ? This is what it takes : Redshift Lex Lambda

Conclusions Amazon S3 is an opportunity to make your data work (as Airbnb, Netflix, Philips, FINRA, NTTDocommo etc. recognize…) AWS has a lot to offer for you next idea : AWS Data Services : more than 20 services from SQL to Hadoop including ML, Streaming AWS Platform and the magic integration between service can turn YOUR IDEA into reality AWS Free Tier can help you get started quickly, give it !