Ch 7. What Makes a Great Analysis? Taming The Big Data Tidal Wave 31 May 2012 SNU IDB Lab. Sengyu Rim

Outline  Criterions for a Good Analysis  Frame the Problem Correctly  Making Inferences 2

Criterions for a Good Analysis(1/7) What is Reporting?  Reporting isn’t equal to analysis – Many organizations mistakenly equate reporting with analysis  A reporting environment(business intelligence environment) – Select the reports they want to run – Get the reports executed – View the results 3 report Provide data Predefined form Inflexible

Criterions for a Good Analysis(2/7) What is Analysis?  An analysis is an interactive process of – Tackling problem – Finding the data required – Analyze the data – Interpret the results 4 Analysis Provide answer Customized Flexible

Criterions for a Good Analysis(3/7) Comparison between Reporting and Analysis  Summary of Analysis versus Reporting 5 ReportingAnalysis Provides dataProvides answers Provides what is asked forProvides what is needed Is typically standardizedIs typically customized Does not involve a personInvolves a person Is fairy inflexibleIs extremely flexible

Criterions for a Good Analysis(4/7) G.R.E.A.T criteria  G.R.E.A.T criteria will add value to analysis 6 G R E A T Guided-guided by a business need Relevant-relevant to the business Explainable-analysis needs to be explained effectively Actionable-a great analysis will be actionable Timely-analysis will be delivered in a timely fashion

Criterions for a Good Analysis(5/7) What are Core Analytics?  Core analytics tend to ask simple questions and provide simple answers – What happened – When it happened – What the impact was 7 Sales Promotion 1.How many subscribers signed up? 2.How did the sing- ups occur everyday? 3.How much money did the new subscribers bring in?

Criterions for a Good Analysis(6/7) What are Advanced Analytics?  Advanced analytics go further than core analytics – What caused it to happen – What can be done in the future 8 Customer Web Activity 1. Identify the relationship between browsing and sales 2. Formulate strategy for marketing

Criterions for a Good Analysis(7/7) Cherry Picking  Sometimes the gut feelings of executives conflict with analysis  One of the worst abuses is to cherry pick results  Cherry picking – Use the analytics when the results serve your purpose – Ignore the findings when the results conflict with the original plan 9

Outline  Criterions for a Good Analysis  Frame the Problem Correctly  Making Inferences 10

Frame the Problem Correctly(1/6) How to Frame the Problem?  Great analysis starts with framing the problem correctly – Assess the data correctly – Develop a solid analysis plan – Technical and practical considerations should be taken into account  Framing the problem is the most important step of an analysis 11

Frame the Problem Correctly(2/6) Statistical Significance  Statistical significance – Used to evaluate the parameter estimates  A statistical significance will validate the conclusions 12

Frame the Problem Correctly(3/6) Never Take Shortcuts  Ensure you have all the data you need – Given the part of the story, conclusions may be completely wrong  Who has the higher average batting? 13 Season TomJoeWinner Joe Joe Joe Joe Joe Year Tom Avg Tom At Bats Tom Hits Joe Avg Joe At Bats Joe Hits winner Joe Joe Joe Joe Joe Total Tom

Frame the Problem Correctly(4/6) Business Importance  Statistical significance should match the business perspective – What are the costs to make the recommended changes? – How much additional revenue might be generated? – Is the new approach consistent with the overall strategy? – Are the new changes executable? 14 Statistical significance Business importance

Frame the Problem Correctly(5/6) Samples versus Population  Using today’s scalable systems, it’s possible to work with an entire population – With big data, we have enough data for a sufficient sample  When a sampling process is needed, it needs to be done correctly – The bigger sample is made, the tighter the margin of the error – Sample size should be suitable for the problem 15

Frame the Problem Correctly(6/6) All Data is Needed  Any given problem may require only a small sample of the data  Different samples require different data – Entire data should be kept 16 s1 s2 s4 s7 s8 s6 s5 s3 s9 s10

Outline  Criterions for a Good Analysis  Frame the Problem Correctly  Making Inferences 17

Making Inferences  To produce a great analysis, it is necessary to infer potential actions – Make initial inferences based on analysis 18