1. Statistics and Data Analysis
Professor William Greene
Stern School of Business
IOMS Department
Department of Economics

2. Statistics and Data Analysis
Part 1 – Data Presentation
Telling the story statistically

3. Samples are surprisingly small
> 1010 Observations
> Telephone sample
> Sampling error

4. What Does it Mean?
Slightly more than one-third of Americans have a favorable opinion of the Democratic-led Congress, a poll said Wednesday.
The Pew Research Center for the People & the Press said the 37% expressing a positive opinion represents a decline of 13 points since April.
The favorable percentage is one of the lowest in more than two decades of Pew surveys – if not the lowest, the poll said. The previous low was 40% in January, but the result is not statistically significant because of the margin of error.
(USA Today)
We will develop the idea of the “margin of error” and how it is computed.

5. Really? The following was taken from
An msnbc.com guide to presidential polls
Why results, samples and methodology vary from survey to survey
WASHINGTON - A poll is a small sample of some larger number, an estimate of something about that larger number. For instance, what percentage of people reports that they will cast their ballots for a particular candidate in an election? A sample reflects the larger number from which it is drawn. Let’s say you had a perfectly mixed barrel of 1,000 tennis balls, of which 700 are white and 300 orange. You do your sample by scooping up just 50 of those tennis balls. If your barrel was perfectly mixed, you wouldn’t need to count all 1,000 tennis balls — your sample would tell you that 30 percent of the balls were orange.
Your sample might tell you that approximately 30 percent of the balls were orange.

6. The Visual Data Do Tell the Story: Napoleon’s March to and from Moscow

7. 40 Informative Data Table Life Expectancy: Highest 15 Countries, 2010
Disability Adjusted Life Expectancy 40

8. A Dynamic Picture

9. Bar Charts vs. Data Tables

10. Probability of Survival to Age 50, Female at Birth U. S
Probability of Survival to Age 50, Female at Birth U.S. and 20 Other Wealthy Countries
It is possible to be misled by a presentation such as this one. Note the vertical axis.
What does this graph tell you? What do the probabilities mean? Are the differences meaningful?

12. Does living longer make people happier
Does living longer make people happier? Or do people live longer because they are happier?

13. Does the Picture Tell the Story?
This is the only graphic in the article. The article compares default rates on VA vs. FHA mortgages. Is there anything wrong with this picture? The very technical looking graph/table is unrelated to the article.
New York Times, Page RE1, July 24, 2014

14. Data Presentation Agenda
Data Types: Cross Section and Time Series
Summarizing Data Graphically
Pie chart, bar chart
Box plot, histogram
Summarizing Data with Descriptive Statistics
Central tendency
Spread
Distribution (shape)

15. Data = A Set of Facts A picture of some aspect of the world
Pizza Sales by Type
What do the data tell you?
How can you use the information?
What additional information would make these data (more) informative?

16. Data Types and Measurement
Quantitative
Discrete = count: Number of car accidents by city by time
Continuous = quantitative measurement: Housing prices
Qualitative
Categorical: Shopping mall, car brand, trip mode
Ordinal: Survey data on attitudes; “How do you feel about…?”
Strongly disagree  Disagree  Neutral  Agree  Strongly agree
Moody’s bond ratings: Aaa, Aa, A, Bbb, Bb, B, and so on.
Frameworks
Cross section
Time series

17. Discrete, Count Data, Time Series

18. Continuous Quantitative Data Housing Prices and Incomes

19. Unordered Qualitative Data Travel Mode Between Sydney and Melbourne by 210 Travelers

20. Ordered Qualitative Data German Health Satisfaction Survey; 27,326 individuals. On a scale from 0 to 10, how do you feel about your health?

21. Aggregated Data May Be Easier to Understand
(7-8)
(4-6)
(9-10)
(0-3)
Bad Fair Good Excellent

22. Ordered Qualitative Outcomes
Bond Ratings Movie Ratings
Arithmetic Mean may not be meaningful.
(a) Ordinal measure – rankings
(b) Look at that distribution!

23. A Problem with Ordered Survey Response Data
61 Stern Students’ Ranking of Subway Safety (1994)*
Safety
Count
Percent
Cum Pct 1 17
27.87 2 15
24.59
52.46 3
80.33 4 10
16.39
96.72 5
3.28
100.00
Very Unsatisfactory
Unsatisfactory OK
Satisfactory
Very Satisfactory
There is no objective meaning to “3” on some standard scale. Does everyone’s “1” or “2” or “3” … mean the same thing?
* Jeff Simonoff: Data Presentation and Summary, pp. 3-4

24. Cross Section Data Housing Prices and Incomes

25. Time Series Data: Oil Price
Graph is much more useful and informative than a table for time series data.

26. Representing Data In raw form Transformed to a visual form
Summarized graphically
Summarized statistically

27. Pie Chart vs. Frequency Table
Pizza Pies Sold, by Type
Same Information. Which is more useful for your audience?

28. Data Representation: Bar Chart vs. Pie Chart
BAR CHART PIE CHART
Same data. Which is easier to understand?

29. Table vs. Bar Chart (or both)
2013 data. Source: Bloomberg

30. 2013 Valuation of U.S. Sports Teams These figures reveal a league strategy.
Football
Baseball

31. A Box Plot Describes the Distribution of Values in a Set of Data
Hawaii
Box and Whisker Plot for House Price Listings

32. Raw Data on Housing Prices and Incomes

33. Making a Box Plot for Per Capita Income
Maximum=31136
3rd Quartile = 24933
Median =22610
Interquartile Range = IQR = = 3256
1st Quartile = 21677
Minimum=17043

34. Box and Whisker Plot = extreme observations
What is an outlier? Why do we believe a particular point is an outlier?
Outliers
Smaller of (Maximum, Median IQR
75th Percentile
Interquartile range=IQR
Median
25th Percentile
Larger of (Minimum, Median – 1.5 IQR

35. Histogram for House Price Listings
A histogram describes the sample data and suggests the nature of the underlying data generating process. Note the “skewness” of the distribution of listings.

36. Distribution of House Price Listings
… shows up in the box and whisker plot. Note the long whisker at the top of the figure.
Asymmetry (skewness) in the histogram of listing prices…

37. House Price Listings and Per Capita Incomes. States.
Regression and Correlation. Are these two variables correlated?
r = .48
How to describe/summarize them.
How to explain the variation across states
How to determine if there is any correlation between the two variables.

38. Big Data: Netflix Cinematch Rating/Recommendation System

39. Summary What story does the data presentation tell?
Data in raw form tell no story.
Visual representation of data tells something about the data
The representation of the data may reveal something about the underlying process that the data measure.
What tool is most informative?
Reduction to a small number of features
Visual displays of data
Data Table – Organizing the data is often a good start.
Pie chart
Box and whisker plots
Bar charts
Histograms
Time series plots
“There are lies, damned lies and statistics.” (Benjamin Disraeli)