Power and Sample Size Boulder 2004 Benjamin Neale Shaun Purcell

To Be Accomplished Introduce Concept of Power via Correlation Coefficient (ρ) Example Identify Relevant Factors Contributing to Power Practical: Power Analysis for Univariate Twin Model How to use Mx for Power

Simple example Investigate the linear relationship (  between two random variables X and Y:  =0 vs.  0 (correlation coefficient). draw a sample, measure X,Y calculate the measure of association  (Pearson product moment corr. coeff.) test whether   0.

How to Test   0 assumed the data are normally distributed defined a null-hypothesis (  = 0) chosen  level (usually.05) utilized the (null) distribution of the test statistic associated with  =0 t=   [(N-2)/(1-  2 )]

How to Test   0 Sample N=40 r=.303, t=1.867, df=38, p=.06 α=.05 As p > α, we fail to reject  = 0 have we drawn the correct conclusion?

 = type I error rate probability of deciding  0 (while in truth  =0)  is often chosen to equal.05...why? DOGMA

N=40, r=0, nrep=1000 – central t(38),  =0.05 (critical value 2.04)

Observed non-null distribution (  =.2) and null distribution

In 23% of tests of  =0, |t|>2.024 (  =0.05), and thus draw the correct conclusion that of rejecting  0. The probability of rejecting the null- hypothesis (  =0) correctly is 1- , or the power, when a true effect exists

Hypothesis Testing Correlation Coefficient hypotheses: –h o (null hypothesis) is ρ=0 –h a (alternative hypothesis) is ρ ≠ 0 Two-sided test, where ρ > 0 or ρ < 0 are one-sided Null hypothesis usually assumes no effect Alternative hypothesis is the idea being tested

Summary of Possible Results H-0 trueH-0 false accept H-01-  reject H-0  1-   =type 1 error rate  =type 2 error rate 1-  =statistical power

Rejection of H 0 Non-rejection of H 0 H 0 true H A true Nonsignificant result (1-  ) Type II error at rate  Significant result (1-  ) Type I error at rate 

Power The probability of rejection of a false null-hypothesis depends on: –the significance criterion (  ) –the sample size (N) –the effect size (NCP) “The probability of detecting a given effect size in a population from a sample of size N, using significance criterion  ”

P(T)P(T) T alpha 0.05 Sampling distribution if H A were true Sampling distribution if H 0 were true   POWER = 1 -  Standard Case Effect Size (NCP)

P(T)P(T) T alpha 0.1 Sampling distribution if H A were true Sampling distribution if H 0 were true POWER = 1 -   Impact of Less Cons. alpha  

P(T)P(T) T alpha 0.01 Sampling distribution if H A were true Sampling distribution if H 0 were true POWER = 1 -  Impact of More Cons. alpha  

P(T)P(T) T alpha 0.05   Increased Sample Size Sampling distribution if H A were true Sampling distribution if H 0 were true POWER = 1 - 

P(T)P(T) T alpha 0.05 Sampling distribution if H A were true Sampling distribution if H 0 were true   POWER = 1 -  Increase in Effect Size Effect Size (NCP)↑

Effects on Power Recap Larger Effect Size Larger Sample Size Alpha Level shifts Type of Data: –Binary, Ordinal, Continuous

When To Do Power Calcs? Generally study planning stages of study Occasionally with negative result No need if significance is achieved Computed to determine chances of success

Power Calculations Empirical Attempt to Grasp the NCP from Null Simulate Data under theorized model Calculate Statistics and Perform Test Given α, how many tests p < α Power = (#hits)/(#tests)

Practical: Empirical Power 1 We will Simulate Data under a model online We will run an ACE model, and test for C We will then submit our data and Shaun will collate it for us While he’s collating, we’ll talk about theoretical power calculations

Practical: Empirical Power 2 First get F:\ben\2004\ace.mx and put it into your directory We will paste our simulated data into this script, so open it now in preparation, and note both places where we must paste in the data Note that you will have to fit the ACE model and then fit the AE submodel

Practical: Empirical Power 3 Simulation Conditions –30% A 2 20% C 2 50% E 2 –Input: –A C of E of –350 MZ 350 DZ –Simulate and Space Delimited at – or click here in slide show modehere –Click submit after filling in the fields and you will get a page of data

Practical: Empirical Power 4 With the data page, use control-a to select the data, control-c to copy, and in Mx control-v to paste in both the MZ and DZ groups. Run the ace.mx script with the data pasted in and modify it to run the AE model. Report the A, C, and E estimates of the first model, and the A and E estimates of the second model as well as both the -2log-likelihoods on the webpage or click here in slide show mode here

Practical: Empirical Power 5 Once all of you have submitted your results we will take a look at the theoretical power calculation, using Mx. Once we have finished with the theory Shaun will show us the empirical distribution that we generated today

Theoretical Power Calculations Based on Stats, rather than Simulations Can be calculated by hand sometimes, but Mx does it for us Note that sample size and alpha-level are the only things we can change, but can assume different effect sizes Mx gives us the relative power levels at the alpha specified for different sample sizes

Theoretical Power Calculations We will use the power.mx script to look at the sample size necessary for different power levels In Mx, power calculations can be computed in 2 ways: –Using Covariance Matrices (We Do This One) –Requiring an initial dataset to generate a likelihood so that we can use a chi-square test

Power.mx 1 ! Simulate the data ! 30% additive genetic ! 20% common environment ! 50% nonshared environment #NGroups 3 G1: model parameters Calculation Begin Matrices; X lower 1 1 fixed Y lower 1 1 fixed Z lower 1 1 fixed End Matrices; Matrix X Matrix Y Matrix Z Begin Algebra; A = X*X' ; C = Y*Y' ; E = Z*Z' ; End Algebra; End

Power.mx 2 G2: MZ twin pairs Calculation Matrices = Group 1 Covariances A+C+E|A+C _ A+C|A+C+E / Options MX%E=mzsim.cov End G3: DZ twin pairs Calculation Matrices = Group 1 H Full 1 1 Covariances _ / Matrix H 0.5 Options MX%E=dzsim.cov End

Power.mx 3 ! Second part of script ! Fit the wrong model to the simulated data ! to calculate power #NGroups 3 G1 : model parameters Calculation Begin Matrices; X lower 1 1 free Y lower 1 1 fixed Z lower 1 1 free End Matrices; Begin Algebra; A = X*X' ; C = Y*Y' ; E = Z*Z' ; End Algebra; End

Power.mx 4 G2 : MZ twins Data NInput_vars=2 NObservations=350 CMatrix Full File=mzsim.cov Matrices= Group 1 Covariances A+C+E|A+C _ A+C | A+C+E / Option RSiduals End G3 : DZ twins Data NInput_vars=2 NObservations=350 CMatrix Full File=dzsim.cov Matrices= Group 1 H Full 1 1 Covariances _ | A+C+E / Matix H 0.5 Option RSiduals ! Power for alpha = 0.05 and 1 df Option Power= 0.05,1 End