Big Data Big Deal or Big Distraction? Agenda: 1.What is Big Data? 2.Why YOU Should Care About (Big) Data? 3.A Brief Introduction to Big Data Econometrics

Internet of People

Today computers… are almost wholly dependent on human beings for information -- by typing, pressing a record button, taking a digital picture or scanning a bar code... The problem is, people have limited time, attention and accuracy—all of which means they are not very good at capturing data about things in the real world… Kevin Ashton, 'That 'Internet of Things' Thing', RFID Journal, July 22, 2009RFID Journal The Problem With People

Internet of Things

What do people want? But remember Jobs… Where are the people we want? Customization, add placement What will sales be? Predicting the future… Are my ads working? : The ATTRIBUTION problem

+ “…the Publicis CEO noted that "the communication and marketing landscape has undergone dramatic changes in recent years, including the exponential development of new media giants, the explosion of Big Data, blurring of the roles of all players and profound changes in consumer behavior.“ WSJ 7/28/13 “…a $35.1 billion cross-border linkup that shows how Big Data is making Madison Avenue look more like Wall Street.” WSJ 7/28/13

Creative vs Analytical

A Brief Introduction to Big Data Econometrics A.What can we do with data? B.Correlation vs. Causation C. Types of data i. Cross section ii. Time series iii. Panel D. Fit, overfit, validation E. Tools of the trade i. Regression, logit, probit ii. Trees & Forests iii. Baysean simulation

1.Prediction 2.Summarization 3.Estimation 4.Hypothesis Testing

Is Marriage Good for Your Health? Tara Parker-Pope, 4/14/10 Contemporary studies, for instance, have shown that married people are less likely to get pneumonia, have surgery, develop cancer or have heart attacks. A group of Swedish researchers has found that being married or cohabiting at midlife is associated with a lower risk for dementia. A study of two dozen causes of death in the Netherlands found that in virtually every category, ranging from violent deaths like homicide and car accidents to certain forms of cancer, the unmarried were at far higher risk than the married. Correlation vs. Causation

What can get in the way of determining CAUSATION? ENDOGENEITY 1. Reverse causality (also selection bias): healthier people are more likely to get married 2. Unobservable characteristics such as time preference, aptitude, genetics

Counterfactual 1. What would happen if we change the “cause”? 2. Is there a plausible alternative explanation? What would sales have been if the ad did not run? What would people do if they did not use Google? What would people buy if the weather was warmer?

Cross-section Data: Lots of observations at one point in time.

Time Series Data: One observation over time.

Panel Data: Multiple observations of the same thing over time.

Fit, Overfit, Validation and Out of Sample Prediction

Linear RegressionLogit/Probit Regression Book: On-line lectures:

Trees and Forests

“Uninformative” Prior Probability Gather Data Conditional probability of observing data “Updated” Probability Bayesian Statistics With BIG DATA we can repeat this process over and over again with multiple models to get better predictions!

Corsea Machine Learning by Andrew Ng An Introduction to Statistical Learning Book: bcf.usc.edu/~gareth/ISL/ bcf.usc.edu/~gareth/ISL/ Lecture videos & problem sets:

Feeling a bit overwhelmed?

“…it’s no wonder that the latest fad in the business world is Big Data … Big data can be an extraordinary tool, helping to gather new information about our behavior and preferences. What it can’t explain is why we do what we do.” WSJ 3/22/14