Big Data Analytics Module 4 – Data Mining and Predictive Analytics Including Mahout Saptak Sen, Microsoft Bill Ramos, Advaiya

Overview of predictive analytics & data mining How Microsoft supports predictive analytics How Mahout fits into the picture Demos Agenda

Data Mining

Recommenda- tion engines Advertising analysis Weather forecasting for business planning Social network analysis IT infrastructure and web app optimization Legal discovery and document archiving Pricing analysis Fraud detection Churn analysis Equipment monitoring Location-based tracking and services Personalized Insurance Predictive analytics should address the likelihood of something happening in the future, even if it is just an instant later*

Rich data mining algorithms, for clustering, classification, forecasting through time series analysis, and more Rich developer experience

Ease of use through Excel Rich data mining algorithms for clustering, prediction, forecasting, market basket analysis, and more Scalable through integration with SSAS

MenuData Mining Analyze Key InfluencersNaïve Bayes Detect CategoriesClustering Fill From ExampleLogical Regression ForecastTime Series Highlight ExceptionsClustering Scenario Analysis – Goal SeekLogical Regression Scenario Analysis – What IfLogical Regression Prediction CalculatorLogical Regression Shopping Basket AnalysisAssociation Rules

Windows Azure HDInsight Microsoft Excel (Mining Add-in) Microsoft Excel Excel Data Mining Add-in Serving LayerSpeed LayerBatch Layer Flat files (.txt,.dat,.xlsx, etc.)

Mahout

Scalable machine learning algorithms on Hadoop platform Algorithms for clustering, classification, and batch-based collaborative filtering using the map/reduce paradigm Supports a wide range of use cases—from spam filtering, to fraud detection, to recommendations for books or movies ClusteringRecommenders Vector Similarity Pattern Mining Classification RegressionGenetic Dimension Reduction Matrices Collocations

Flat files (.txt,.dat,.xlsx, etc.) Running Mahout job on Hadoop Command Window to get output file Convert to Mahout input Hadoop Command Window Output file Serving LayerSpeed LayerBatch Layer Windows Azure HDInsight HDInsight Consoles

Questions?