Module 3: Using Statistics for Benchmarking and Comparison Common Measures Training Chelmsford, MA September 28, 2006.

Slides:

Advertisements

Similar presentations

Designing an impact evaluation: Randomization, statistical power, and some more fun…

Advertisements

Topics

SAMPLE DESIGN: HOW MANY WILL BE IN THE SAMPLE—DESCRIPTIVE STUDIES ?

November 10, 2010DSTS meeting, Copenhagen1 Power and sample size calculations Michael Væth, University of Aarhus Introductory remarks Two-sample problem.

CONFIDENCE INTERVALS n point estimate: estimate exact value – precise – likely to be wrong n interval estimate: range of values – less precise – less.

Module 2: Introduction to ERP Statistical Concepts and Tools Common Measures Training Chelmsford, MA September 28, 2006.

10-1 Introduction 10-2 Inference for a Difference in Means of Two Normal Distributions, Variances Known Figure 10-1 Two independent populations.

Mean for sample of n=10 n = 10: t = 1.361df = 9Critical value = Conclusion: accept the null hypothesis; no difference between this sample.

PSY 307 – Statistics for the Behavioral Sciences

Statistics 101 Class 9. Overview Last class Last class Our FAVORATE 3 distributions Our FAVORATE 3 distributions The one sample Z-test The one sample.

CONFIDENCE INTERVALS What is the Purpose of a Confidence Interval?

Incidence, Prevalence and 95% Confidence Intervals Tom Walker and Matt Newman.

Chapter 11: Inference for Distributions

Stat 217 – Day 20 Comparing Two Proportions The judge asked the statistician if she promised to tell the truth, the whole truth, and nothing but the truth?

Chapter 9 Hypothesis Testing II. Chapter Outline  Introduction  Hypothesis Testing with Sample Means (Large Samples)  Hypothesis Testing with Sample.

Today Concepts underlying inferential statistics

5-3 Inference on the Means of Two Populations, Variances Unknown

1 Copyright © 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved. Section 7.2 Estimating a Population Proportion Objective Find the confidence.

Chapter 9 Hypothesis Testing II. Chapter Outline  Introduction  Hypothesis Testing with Sample Means (Large Samples)  Hypothesis Testing with Sample.

Module 1: Why Use Statistics? An Introduction to the ERP Case Study Steve DeGabriele (Massachusetts DEP) For the Workshop: Using Essential Statistics for.

Copyright © 2012 by Nelson Education Limited. Chapter 8 Hypothesis Testing II: The Two-Sample Case 8-1.

Module 4: Establishing the Universe Common Measures Training Chelmsford, MA September 28, 2006.

Academic Viva POWER and ERROR T R Wilson. Impact Factor Measure reflecting the average number of citations to recent articles published in that journal.

T tests comparing two means t tests comparing two means.

When estimating the mean, how large a sample must be used in order to assure a given level of confidence? Use the formula:

1 Math 10 Part 5 Slides Confidence Intervals © Maurice Geraghty, 2009.

Comparing Two Proportions

Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 8-1 Confidence Interval Estimation.

Chapter 9 Hypothesis Testing II: two samples Test of significance for sample means (large samples) The difference between “statistical significance” and.

PARAMETRIC STATISTICAL INFERENCE

STA291 Statistical Methods Lecture 24. Comparing Two Proportions Sample of 25, year-olds: Men: 84.9% diploma rate Women: 88.1% diploma rate Are.

Module 2: Statistical Approaches to Assess the Current Performance of a Group Michael Crow (Independent Consultant to U.S. EPA) For the Workshop: Using.

LECTURE 19 THURSDAY, 14 April STA 291 Spring

1 Things That May Affect Estimates from the American Community Survey.

AP STATISTICS LESSON COMPARING TWO PROPORTIONS.

Variables Instrumentales Econometría UDESA. Example Wooldridge The estimate for β 1 implies an almost 11% return for another year of education.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide

1 ConceptsDescriptionHypothesis TheoryLawsModel organizesurprise validate formalize The Scientific Method.

5-4-1 Unit 4: Sampling approaches After completing this unit you should be able to: Outline the purpose of sampling Understand key theoretical.

Copyright © 2010 Pearson Education, Inc. Chapter 22 Comparing Two Proportions.

T T Population Sample Size Calculations Purpose Allows the analyst to analyze the sample size necessary to conduct "statistically significant"

Aron, Aron, & Coups, Statistics for the Behavioral and Social Sciences: A Brief Course (3e), © 2005 Prentice Hall Chapter 6 Hypothesis Tests with Means.

Module 6: Working with Secondary Data Common Measures Training Chelmsford, MA September 28, 2006.

T tests comparing two means t tests comparing two means.

Various Topics of Interest to the Inquiring Orthopedist Richard Gerkin, MD, MS BGSMC GME Research.

Comparing Two Proportions. AP Statistics Chap 13-2 Two Population Proportions The point estimate for the difference is p 1 – p 2 Population proportions.

Week 7 Chapter 6 - Introduction to Inferential Statistics: Sampling and the Sampling Distribution & Chapter 7 – Estimation Procedures.

1 Probability and Statistics Confidence Intervals.

Sampling Distribution (a.k.a. “Distribution of Sample Outcomes”) – Based on the laws of probability – “OUTCOMES” = proportions, means, test statistics.

AP Statistics Section 10.1 C Determining Necessary Sample Size.

T tests comparing two means t tests comparing two means.

Sample Size Mahmoud Alhussami, DSc., PhD. Sample Size Determination Is the act of choosing the number of observations or replicates to include in a statistical.

Ex St 801 Statistical Methods Inference about a Single Population Mean (CI)

Common Measures Project Introductory Measurement and Quality Training Course Chelmsford, MA September 28, 2006.

Probability in Sampling. Key Concepts l Statistical terms in sampling l Sampling error l The sampling distribution.

Oneway ANOVA comparing 3 or more means. Overall Purpose A Oneway ANOVA is used to compare three or more average scores. A Oneway ANOVA is used to compare.

Correlation and Linear Regression

Comparing Two Proportions

Inferences On Two Samples

Hypothesis Tests for Two Population Proportions

Bootstrap Confidence Intervals using Percentiles

Introduction to Inference Confidence Intervals

SAMPLE DESIGN: HOW MANY WILL BE IN THE SAMPLE—DESCRIPTIVE STUDIES ?

Chapter 10: Basics of Confidence Intervals

Chapter 13 - Confidence Intervals - The Basics

CHAPTER 10 Comparing Two Populations or Groups

AP Statistics Chapter 12 Notes.

Chapter Outline Inferences About the Difference Between Two Population Means: s 1 and s 2 Known.

Inference for Proportions

Chapter 14 - Confidence Intervals: The Basics

Presentation transcript:

Module 3: Using Statistics for Benchmarking and Comparison Common Measures Training Chelmsford, MA September 28, 2006

Comparison: Two Questions Comparing two samples of one population, over time Are we confident performance changed? How much did performance change? Comparing two samples from two different states Are we confident performance is different? How much difference is there?

Key Concepts for Comparison "Statistical Significance" / Significant Difference Confidence Interval of a Difference Minimum Detectable Difference

“Statistical Significance” Important ERP Question: Did facility performance change between the baseline and the post-certification inspections? Population improvement? If we observed an improvement between the first sample and the second sample, are we confident there was an improvement in the overall population? Remember, both the first and second samples’ point estimates have error associated with them

“Statistical Significance” “Statistical significance” means we are confident there was an improvement in the population Other differences. Concept also applies when comparing results from two states Are they significantly different?

Understanding Differences Difference = proportion or quantity. E.g., Proportion: 40% compliance in Round 1 60% compliance in Round 2 Difference = 20% Mean: 300 pounds average waste generation in State A 400 pounds average waste generation in State B Difference = 100 pounds Outcome measures, indicator scores and certification accuracy can have differences, too

Confidence Intervals for Differences Just like single sample observations, observed differences have error associated with them Confidence interval reflects that error. E.g., We are 95% confident that compliance increased 20% (+/-7%) We are 95% confident that average waste generation is pounds lower in State A than in State B

Is the Difference Significant? Two ways to understand if the difference is significant Formal hypothesis test Less flexible Test must be defined pre-sample Or confidence interval of the difference…

Confidence Interval of the Difference Confidence interval of the difference contains zero? Difference is 10% +/- 7% (95% conf.) 95% confidence that difference is between 3% and 17% (excludes zero) Difference is statistically significant Difference is 10% +/- 11% (95% conf.) 95% confidence that difference is between - 1% and 21% (could be zero) Difference is not statistically significant

Minimum Detectable Difference = How large a change in performance you need to see for it to be “statistically significant” Important concept in sample planning Depends on population size and sample size for each sample Smaller detectable difference = larger sample size

Minimum Detectable Difference Think about what level of change would drive decision-making Worthwhile to design samples to detect a 2% change? Is it different for practical purposes? Would you do anything differently than if you thought there was no change?

Tour of the Spreadsheet Tools Two-sample sheets in Sample Planner and Results Analyzer.

For more information… Contact Michael Crow Phone: