Statistical and Practical Significance

Slides:

Advertisements

Similar presentations

Chapter 16 Inferential Statistics

Advertisements

Hypothesis Testing Steps in Hypothesis Testing:

Hypothesis Testing making decisions using sample data.

Inferential Statistics

Statistical and Practical Significance Advanced Statistics Petr Soukup.

Empirical Analysis Doing and interpreting empirical work.

Chapter Seventeen HYPOTHESIS TESTING

10-1 Introduction 10-2 Inference for a Difference in Means of Two Normal Distributions, Variances Known Figure 10-1 Two independent populations.

Hypothesis Testing Steps of a Statistical Significance Test. 1. Assumptions Type of data, form of population, method of sampling, sample size.

Sample size computations Petter Mostad

9-1 Hypothesis Testing Statistical Hypotheses Statistical hypothesis testing and confidence interval estimation of parameters are the fundamental.

Stat Day 16 Observations (Topic 16 and Topic 14)

BCOR 1020 Business Statistics Lecture 22 – April 10, 2008.

BCOR 1020 Business Statistics Lecture 28 – May 1, 2008.

Chapter 9 Hypothesis Testing 9.4 Testing a Hypothesis about a Population Proportion.

4-1 Statistical Inference The field of statistical inference consists of those methods used to make decisions or draw conclusions about a population.

BCOR 1020 Business Statistics Lecture 20 – April 3, 2008.

BCOR 1020 Business Statistics

Chapter 9 Hypothesis Testing II. Chapter Outline  Introduction  Hypothesis Testing with Sample Means (Large Samples)  Hypothesis Testing with Sample.

Chapter 9 Hypothesis Testing II. Chapter Outline  Introduction  Hypothesis Testing with Sample Means (Large Samples)  Hypothesis Testing with Sample.

Simple Linear Regression Analysis

Chapter 9 Title and Outline 1 9 Tests of Hypotheses for a Single Sample 9-1 Hypothesis Testing Statistical Hypotheses Tests of Statistical.

Choosing Statistical Procedures

AM Recitation 2/10/11.

1 © Lecture note 3 Hypothesis Testing MAKE HYPOTHESIS ©

Chapter 8 Inferences Based on a Single Sample: Tests of Hypothesis.

1/2555 สมศักดิ์ ศิวดำรงพงศ์

Comparing Two Proportions

Statistical Inference Decision Making (Hypothesis Testing) Decision Making (Hypothesis Testing) A formal method for decision making in the presence of.

Chapter 9 Hypothesis Testing II: two samples Test of significance for sample means (large samples) The difference between “statistical significance” and.

10-1 Introduction 10-2 Inference for a Difference in Means of Two Normal Distributions, Variances Known Figure 10-1 Two independent populations.

9-1 Hypothesis Testing Statistical Hypotheses Definition Statistical hypothesis testing and confidence interval estimation of parameters are.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide

Statistical Hypotheses & Hypothesis Testing. Statistical Hypotheses There are two types of statistical hypotheses. Null Hypothesis The null hypothesis,

4 Hypothesis & Testing. CHAPTER OUTLINE 4-1 STATISTICAL INFERENCE 4-2 POINT ESTIMATION 4-3 HYPOTHESIS TESTING Statistical Hypotheses Testing.

S-012 Testing statistical hypotheses The CI approach The NHST approach.

Assessing Hypothesis Testing Fit Indices

Elementary statistics for foresters Lecture 5 Socrates/Erasmus WAU Spring semester 2005/2006.

1 9 Tests of Hypotheses for a Single Sample. © John Wiley & Sons, Inc. Applied Statistics and Probability for Engineers, by Montgomery and Runger. 9-1.

1 CHAPTER 4 CHAPTER 4 WHAT IS A CONFIDENCE INTERVAL? WHAT IS A CONFIDENCE INTERVAL? confidence interval A confidence interval estimates a population parameter.

Chapter 9: Testing Hypotheses Overview Research and null hypotheses One and two-tailed tests Type I and II Errors Testing the difference between two means.

Hypothesis Testing Errors. Hypothesis Testing Suppose we believe the average systolic blood pressure of healthy adults is normally distributed with mean.

T Test for Two Independent Samples. t test for two independent samples Basic Assumptions Independent samples are not paired with other observations Null.

Chapter 22 Comparing Two Proportions.  Comparisons between two percentages are much more common than questions about isolated percentages.  We often.

PEP-PMMA Training Session Statistical inference Lima, Peru Abdelkrim Araar / Jean-Yves Duclos 9-10 June 2007.

CHAPTER 7: TESTING HYPOTHESES Leon-Guerrero and Frankfort-Nachmias, Essentials of Statistics for a Diverse Society.

McGraw-Hill/Irwin © 2003 The McGraw-Hill Companies, Inc.,All Rights Reserved. Part Four ANALYSIS AND PRESENTATION OF DATA.

1 Basics of Inferential Statistics Mark A. Weaver, PhD Family Health International Office of AIDS Research, NIH ICSSC, FHI Lucknow, India, March 2010.

Part Four ANALYSIS AND PRESENTATION OF DATA

STAT 312 Chapter 7 - Statistical Intervals Based on a Single Sample

STA 291 Spring 2010 Lecture 18 Dustin Lueker.

Hypothesis Testing II: The Two-sample Case

Statistics in SPSS Lecture 7

MATH 2311 Section 8.2.

Hypothesis Theory PhD course.

Comparing Two Proportions

Hypothesis Tests for a Population Mean in Practice

P-value Approach for Test Conclusion

Chapter 6 Making Sense of Statistical Significance: Decision Errors, Effect Size and Statistical Power Part 1: Sept. 18, 2014.

9 Tests of Hypotheses for a Single Sample CHAPTER OUTLINE

Section 12.2: Tests about a Population Proportion

Comparing Two Proportions

CHAPTER 12 Inference for Proportions

Statistics in SPSS Lecture 6

STA 291 Spring 2008 Lecture 18 Dustin Lueker.

CHAPTER 12 Inference for Proportions

AP Statistics Chapter 12 Notes.

STA 291 Summer 2008 Lecture 18 Dustin Lueker.

Statistical Power.

Presentation transcript:

Statistical and Practical Significance Advanced Statistics Petr Soukup

Outline Reminder of statistical significance Limits of statistical significance Misuses of statistical significance Alternatives to statistical significance Practical significance Effect sizes

REMINDER OF STATISTICAL SIGNIFICANCE (NHST)

Hypotheses and tests Tested hypothesis in experiments (Fisher, 1925) Null and alternative hypothesis (NHST) (Neyman&Pearson, 1937) Common tests - t-tests, analysis of variance, analysis of covariance, correlation analysis etc.

Definition of statistical significance Decision True status H0 H1 OK (P=1- α) Type I error (P= α) Type II error (P= β) OK (P= 1-β) Test Power Definition: Conditional probability, that our sample can be drawn from population in which null hypothesis is valid (α). Statistical significance is P(D/H0) and not P(H0/D)

LIMITS OF NHST

Assumptions for classical NHST Big big probability samples from infinite or very big finite populations Three assumptions: Big (infinite) population (at least 100times bigger than the sample) Probability sampling (all units same probability of selection) Big sample (> 30-50 units)

LIMITS OF NHST 1. data from censuses 2. data from non-probability samples 3. data from small samples 4. data based on sample that are big proportion of the basic population 5. big data samples from merged (internationally or by time) files

Beyond the limits of NHST in CSR* N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research *CSR-Czech sociological review

MISUSES OF NHST

Objections against NHST (Misuses of NHST) a) Insufficient statement about population, b) null hypotheses are unreal (nill null), c) mechanical usage of classical 5% statistical significance (asterisks, stepwise methods, best models etc.), d) statistical significant doesn’t mean important, e) publishing only statistical significant results (file drawer problem).

Misuses of NHST in CSR* *CSR-Czech sociological review N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research *CSR-Czech sociological review

Conference examples (P<0,01)

Conference examples (***)

Conference examples (*** and stepwise)

ALTERNATIVES TO NHST

Some alternatives to statistical significance a) Confidence Intervals (Problems for r, formulas, regression etc.) b) Test power (quite good in sociology), c) Estimate of minimum sample size & What if strategy, d) Comparison of models via information criterias (AIC, BIC) e) Bayesian approach

PRACTICAL SIGNIFICANCE

Practical significance - terminology b) Substantive significance c) Logical significance d) Scientific significance sometimes also: e) result importance or f) result meaningfulness

How to measure Practical sig.? History - Absolute and relative approach Example: Income differencies Absolute and relative difference

How to measure Practical sig.? Effect sizes – measures of practical significance Some well known: Cohen d Hayes ω But also R2, r, C, Fisher η2 are effect sizes Problem: Sometimes published but not interpreted

OTHER SIGNIFICANCES

Special significances Economic significance Clinical significance Etc.

CONCLUSION? Statistical significance is: LIMITED MISUSED BUT NOT BAD Substantive significance is: NOT OFTEN USED BUT NECESSARY