1. Statistical Power and Meta-analysis
Pak Sham, Ben Neale, Meike Bartels
International Workshop on Statistical Genetic Methods for Human Complex Traits
March 3, 2015

2. What is statistical power?
Statistical power is the probability of rejecting the null hypothesis in a statistical test, when the alternative hypothesis is true
The concept of statistical power was introduced by Neyman and Pearson, extending Fisher’s work on significance testing

3. Error rates
The type 1 error rate (α) is the probability of rejecting the null hypothesis when it is true
The type 2 error rate (β) is the probability of NOT rejecting the null hypothesis when the alternative hypothesis is true (power = 1-β)
The false positive report rate is the probability that a null hypothesis is true when a significant result occurs

4. Why calculate power?
To determine if a study has a reasonable chance of success (detecting a true effect)
To design a study that has greatest power given available resources
To determine the minimum required sample size in a study
Usually obligatory in grant proposals

5. Both type 1 and type 2 error rates affect the false positive rate
1000 Tests H0 H1 10
990 NS S NS S
990
10(1-)
What is the false positive report rate?

6. Bayes’ theorem α = type 1 error rate β = type 2 error rate
π0 = prior probability of null hypothesis

7. Candidate gene studies
Hirschhorn et al. 2002: Reviewed 166 putative single allelic association with 2 or more replication attempts:
6 reliably replicated (≥75% positive replications)
97 with at least 1 replication
63 with no subsequent replications
Other such surveys reached similar conclusions (Ioannidis 2003; Ioannidis et al. 2003; Lohmueller et al. 2003)
Probably caused by widespread multiple testing, inadequate control of type 1 error rate, and Inadequate statistical power

8. Power calculation “triad”
Effect size
Sample size
Power
Given any two, calculate the third

9. Defining the effect
The impact of the predictor variable (e.g. genotype) on the outcome variable (e.g. disease risk).
Example:

10. Odds Ratio vs Risk Ratio
Conversion between OR and RR requires knowledge of population risk (see So et al, Genetic Epi, 2011)

11. Quantitative trait
For trait X: Mean trait difference = E(X|G1) – E(X|G0)

12. Some complications A biallelic locus has 3 possible genotypes
Comparing each genotype (AA and AB) to a reference genotype (BB) gives raise to 2 effect sizes
Simplification to a single effect size only possible under particular model assumptions
Dominant model: AA and AB have same effect
Recessive model: AB has no effect
Multiplicative model: the effect of AA equals the square of the effect for AB
Additive model: the effect of AA equals twice the effect of AB

13. How to set the effect size?
Replication studies
Take effect size of original study (possibly with adjustment for winner’s curse if original study involved extensive multiple testing)
Original studies
Take typical effect sizes found by previous studies of similar phenotypes and similar genetic variants
Take smallest effect size considered to be scientifically interesting
Often desirable to consider a range of plausible effect sizes and present results in tables or graphs

14. Winner’s curse
Suppose 100 independent SNPs on a SNP chip each has 1% power to reach critical genome-wide significance
The probability that at least one SNP achieves genome-wide significance is 1-(0.99)100 ≈ The estimated effect size of the most significant SNP will also be much greater than its true effect size
A replication study with identical design and sample size has only a 1% chance of replicating a particular SNP at the same genome-wide level of significance.

15. Allele frequency
Power is also influenced by the variance of the independent variable
For a locus with allele frequency p, coded as 0, 1, 2 (additive model) and in Hardy-Weinberg equilibrium, the population variance of the genotype is 2p(1-p)

16. Sample size plot
OR=1.2
1.3
1.5
2.0
Wang et al, (2005)

17. Power calculation via NCP
Sample size
Effect size
NCP
Power
Allele frequency α

18. NCP of chi-squared test
If Z~N(,1), then Z2 ~ 2(df=1, NCP=2)
df = degrees of freedom
NCP = Non-Centrality Parameter
Mean of Z2 = df + NCP
Nice properties of NCP for direct association
NCP  sample size
NCP  effect size of genotype (function of)
NCP  variance of genotype

19. NCP determines power
Rejection of H0

20. NCP determines power

21. Linear regression
Y = α + βX + ε H0: β=0, usually t-test or F-test In large samples, t ≈ Normal, F ≈ Chi-squared Directly related to the proportion of variance explained by QTL, when the residual variance is close to the trait variance

22. Variance Explained Pawitan Y et al, PLOSone, 2009
For binary disease trait determined by liability-threshold model, the proportion of variance in liability explained by a SNP with allele frequency p and allelic odds ratio θ is approximately
This. together with N, determine the NCP for simple random samples of the population
Note the regression coefficient in a logistic regression represents ln(OR)
Pawitan Y et al, PLOSone, 2009
So HC et al, Genet Epi, 2011

23. Indirect association
If a direct association study of a causal SNP would provide an NCP of 
Then an indirect association study of a SNP in LD with the causal SNP has NCP of R2
i.e. sample size to achieve the same power is increased by a factor of 1/ R2
Sham et al, Am J Hum Genet 2000

24. Selecting extremes
Selecting individuals with extreme (very low or very high) phenotypic values for genotyping can improve study efficiency
NCPS / NCPP = VarS / VarP

25. Repeated measurements
If the quantitative trait has test-retest correlation r, then taking the average of k independent measurements reduces the measurement variance from 1-r to (1-r)/k.
The variance explained by a causal locus, and therefore the NCP, increases by a factor of

26. Genetic Power Calculator (GPC)
Purcell, Cherny and Sham, Bioinformatics, 2003

27. Exercise 1 Candidate gene case-control study Disease prevalence 2%
Genotype risk ratio Aa = 2, genotype risk ratio AA = 4
Frequency of high risk disease allele = 0.05
Frequency of associated marker allele = 0.1
Linkage disequilibrium D-Prime = 0.8
Sample size: 500 cases, 500 controls
Type 1 error rate: 0.01
Calculate
Marker allele frequencies in cases and controls, NCP, Power

28. Exercise 2 For a discrete trait TDT study
Assumptions: same as in Exercise 1
Sample size: 500 parent-offspring trios
Type 1 error rate: 0.01
Calculate:
Ratio of transmission of marker alleles from heterozygous parents, NCP, Power

29. Exercise 3 For the same assumptions as Exercise 1
Find the type 1 error rates that correspond to 80% power for sample sizes of
500 cases, 500 controls
1000 cases, 1000 controls
2000 cases, 2000 controls

30. Answer to Exercise 1
High risk allele frequencies in cases and controls are 13.43% and 9.93%, respectively,
NCP = 5.933
Power =

31. Answer to Exercise 2
Ratio of transmission of high-risk and low-risk alleles from heterozygous parents to affected children is :0.1731
NCP = 5.667
Power =

32. Answer to Exercise 3 Sample size NCP Critical 2 Critical  500 5.933
.111
1000
11.866
.00924
2000
23.732
NCPs obtained by multiplication of NCP from Ex 1
Critical 2 obtained from inverse non-central chi-square distribution function
Critical  obtained from chi-square distribution function (with NCP set to 0)

33. Meta-analysis Combine data from multiple studies to
increase statistical power
obtain more precise effect size estimates
Uses summary statistics from each study, rather than raw data
estimates (β) + standard error
test statistics (Z) + sample size
p-values

34. Steps in meta-analysis
Identify relevant studies
Obtain agreement for participation
Ensure uniformity in phenotype definition, marker set (imputation), analysis method, file format for summary statistics
Share summary statistics files
Combine the summary statistics of studies to give “meta-statistics” (Z, β, p, etc)
Look for evidence of heterogeneity in effect size
Check for signs of publication bias

35. Phenotype definition
Make sure phenotypes have same definition in different studies.
If not, use Z-combination, p-combination, or re-scale β’s and their standard errors before combining

36. Strand orientation ATCTGGT[A/C]CTCCAT TAGACCA[T/G]GAGGTA
A is equivalent to T
C is equivalent to G
No ambiguity

37. Strand orientation The annoying problem: ATCTGGT[A/T]CTCCAT
TAGACCA[T/A]GAGGTA
Allele A in one study may be labeled as T in another
G/C SNPs have the same problem

38. Strand orientation Two ambiguous SNP types A/T and G/C
Flip alleles if probe sequence is complementary to reference sequence (the + strand):
Further checks
Allele frequencies [know your population]
LD [if you have raw data]
Directionless combination

39. Weighted β
where

40. Weighted Z
where
The test statistics Zi can be obtained from two-tailed p-values and the direction of effect, or one-tailed p-values, using the inverse normal distribution function

41. Directionless combination
Fisher’s method:
Sum of χ2’s
Correlation of p-values from the methods ~ 0.99
Chi-squared statistics can be weighted by sample size

42. Test for Heterogeneity Cochran’s Q

43. Publication bias: funnel plot
Positive studies easier to publish than negative studies, especially of sample size is modest or small
Does not affect GWAS meta-analyses!

44. Why not random effects?
Random effects (RE) meta-analysis allows β to differ in different populations
In contrast, fixed effects (FE) meta-analysis assumes the same β across populations
Since variation of β is likely to exist, why is FE meta-analysis preferred?
H0: β = 0 for all populations
H0: E(β) = 0 across populations
The first H0 is more appropriate, but the RE model is designed to test the second H0
Han & Eskin, 2011, AJHG

45. METAL Practical You have ran a GWA analysis.
We will take the results file and a second results file to run a meta-analysis using METAL
Copy files from faculty/meike/2015/metal-practical to your own folder

46. Documentation can be found at the metal wiki:
Documentation can be found at the metal wiki:

47. METAL Metal is flexible
By default, METAL combined p-values across studies (sample size, direction of effect)
Alternative, standard error based weights (but beta and standard error use same units in all studies)

48. METAL Requires results files ‘Driver’ file Describes the input files
Defines meta-analysis strategy
Names output file

49. Steps Check format of results files Prepare driver file Run metal
Ensure all necessary columns are available
Modify files to include all information
Prepare driver file
Ensure headers match description
Crosscheck each results file matches Process name
Run metal

50. Results Files Previously asked for standard columns in SOP
Current procedure is to upload complete files (results and info files)

51. INPUT FILES We will use the GWAs results (results1.txt)
We will use a second set of results (results2.txt)

52. Columns METAL uses SNP OR SE [for standard error meta-analysis]
P-value [for Z-score meta-analysis]
If we had two samples of different sizes we would have to add an N/weight column

53. Meta-analysis running
We will run meta-analysis based on effect size and on test statistic
For the weights of test statistic, I’ve assumed that the sample sizes are the same
METAL defaults to weight of 1 when no weight column is supplied

54. Step 2: driver file: meta_run_file
# PERFORM META-ANALYSIS based on effect size and on test statistic
# Loading in the input files with results from the participating samples
# Note: Order of samples is …[sample size, alphabetic order,..]
# Phenotype is ..
# MB March 2015
MARKER SNP
ALLELE A1 A2
PVALUE P
EFFECT log(OR)
STDERR SE specifies column names
PROCESS results1.txt
PROCESS results2.txt processes two results files
OUTFILE meta_res_Z .txt Output file naming
ANALYZE Conducts Z-based meta-analysis from test statistic
CLEAR Clears workspace
SCHEME STDERR Changes meta-analysis scheme to beta + SE
OUTFILE meta_res_SE .txt Output file naming
ANALYZE Conducts effect size meta-analysis

55. Larger Consortia # PERFORM META-ANALYSIS on P-values
# Loading in the inputfiles with results from the participating samples
# Note: Order of samples is alpahabetic
# Phenotype is WB
# 1. AGES_HAP
MARKER SNPID
ALLELE coded_all noncoded_all
EFFECT Beta
PVALUE Pval
WEIGHT n_total
GENOMICCONTROL ON
COLUMNCOUNTING LENIENT
PROCESS AGES_HAP.txt
# 2. ALSPAC_HAP
PROCESS ALSPAC_HAP.txt
AND SO ON (in this case 40 files)

56. Running metal metal < metal_run_file metal is the command
metal_run_file is the driver file
This will output information on the running of METAL things to standard out [the terminal]
It will spawn 4 files:
2 results files: meta_res_Z1.txt + meta_res_SE1.txt
2 info files: meta_res_Z1.txt.info + meta_res_SE1.txt.info

57. Output you’ll see Overview of METAL commands Any errors
And your best hit from meta-analysis

58. To load into Haploview We have to change the header
In the same directory run:
./reformat.sh
This changes 1st column name to SNP
We can then load the meta-analysis results files into haploview
Same as before but load in the meta_res_Z1.txt
Make sure to include gwas-example.bim

60. Plot

62. Example: Wellbeing
I presented a single cohort GWAs on WB at the BGA conference in 2011 (n=3089)
Social Science Genetic Association Consortium (SSGAC)
SOP has been sent out FEB 2012
Only aware of a couple of cohorts with WB data
DEC cohorts agreed to participate (PA: 90K)
DEC cohort (PA: 137K, LS: 92K, WB: 172K)