1. 6/10/20151 Microarray Data Analysis

2. 6/10/20152 Copyright notice Many of the images in this power point presentation of other people. The Copyright belong to the original authors. Thanks!

3. 6/10/20153 Gene Expression Matrix After image processing, obtain a data matrixThe final gene expression matrix (on the right) is needed for higher level analysis and mining. Samples Genes Gene expression levels Images Spots  Spot/Image quantiations

4. 6/10/20154 Missing data in microarray Randomly missing values the fact that the value is missing is independent of its value methods are available for dealing with randomly missing data Non-randomly missing values: the fact that the value is missing is dependent on its value –(i.e. the value is missing because it is low expression, or the value is missing because it is high expression) available methods do not adequately deal with the situation of non-randomly missing data

5. 6/10/20155 Missing data in microarray Randomly missing data: –spotting problems –dust –finger prints –poor hybridization –inadequate resolution –fabrication errors (e.g. scratches) –image corruption –omission of suspect values * * could also be non-random Non-randomly missing data: low expression e.g. background exceeds signal censored data Arrays max observable intensity Expression

6. 6/10/20156 Dealing with missing data The problem: –many analyses require complete data matrices classification algorithms clustering algorithms dimension-reduction methods Solutions: –remove all genes (rows) and arrays (columns) with missing values –estimate missing values

7. 6/10/20157 Imputation methods Naive approaches –missing values = row (gene) average –missing values = column (array) average Smarter approaches have been proposed: –K-nearest neighbors –regression-based methods –singular value decomposition like principal components for matrices with unequal numbers of rows and columns

8. 6/10/20158 K-Nearest Neighbors (KNN) ArraysExpression ? randomly missing datum chose k genes that are most similar to the gene with the missing value (MV) estimate MV as the weighted mean of the neighbors considerations: –number of neighbors –distance metric –normalization step

9. 6/10/20159 KNN - considerations parameter k –10 usually works (5-15) distance metric –euclidean distance –correlation-based distance Arrays Expression ?

10. 6/10/201510 Ordinary Least Squares (OLS) regression-based approach also uses k-neighbors algorithm: –choose k neighbors (euclidean or correlation; normalize or not) –the gene with the MV is regressed over the neighbor genes (one at a time, i.e. simple regression) –for each neighbor, MV is predicted from the regression model –MV is imputed as the weighed average of the k predictions

11. 6/10/201511 Singular Value Decomposition (SVD) goal: –use the strongest patterns of correlation within the data matrix to estimate algorithm –set MVs to row average (need a starting point) –decompose expression matrix in orthogonal components, “eigengenes”. –use the proportion, p, of eigengenes corresponding to largest eigenvalues to reconstruct the MVs from the original matrix (i.e. improve your estimate) –use EM approach to iteratively improve estimates of MVs until convergence

12. 6/10/201512 Other Imputation Methods: Local Singular Value Decomposition (LSVD) –combines KNN and SVD –algorithm: start with a n genes x m arrays matrix select k neighbor genes (euclidean or correlation; normalize or not) perform SVD on the k x m array matrix Partial Least Squares (PLS) regression –uses all genes and available data from target gene Factor Analysis (FA) regression

13. 6/10/201513 Which imputation method to use? KNN is the most widely-used; current standard many alternative choices: OLS, SVD, LSVD, PLS, (FA) algorithms require user-supplied parameters: k, p, distance metric, etc. No set of rules for choosing which method to use

14. 6/10/201514 Characteristics of data that may affect choice of imputation method dimensionality percentage of values missing experimental design (time series, case/control, etc.) entropy - patterns of correlation in data others?

15. 6/10/201515 Data Analysis Determine differential gene expression Identify up- and down-regulated genes Gene lists produced using Factor 2 Rule, t-test based methods Co-regulation of genes Clustering algorithms Identify genes that regulate other genes Networks (e.g. Bayesian)

16. 6/10/201516 Methods to Decide Differential Expression Compare treatment to the control –The fold approach –The t-test –Variations of the t-test SAM: significance analysis of microarraysSAM Compare several treatments –ANOVA: analysis of variance –MAANOVA: http://www.jax.org/staff/churchill/labsite/software/anov a/index.html http://www.jax.org/staff/churchill/labsite/software/anov a/index.html

17. 6/10/201517 Fold Change Measure ratios of gene expression levels. Ratio = T i /C i. Ratio of measured treatment intensity to control intensity for the i th spot The log 2 ratio treats up and down regulated genes equally –e.g. when looking for genes with more than 2 fold variation in expression

18. 6/10/201518 The Fold Approach In northern analysis, a 2-fold change can be seen with bare eyes Thus biologists tend to use 2-fold as the threshold of differential expression mean(x 1, x 2 ) > 1 mean(x 1, x 2 ) < -1

19. 6/10/201519 Illustration of the benefit of using Log ratios

20. 6/10/201520 Two-fold up-regulation Problems with this approach: –Only identifies most changed genes. –Also identifies noise and highly variable genes. –Ratio is unstable when the denominator is small.

21. 6/10/201521 Ratios are unstable Initial measurements: 30/60 = 0.5 500/1000 = 0.5 Add random noise (+15 numerator and -15 denominator): 45/45 = 1.0 515/985 = 0.52

22. 6/10/201522 Types of tests Standard t-test assumes the samples are drawn from normal distributions with equal variance and different means. Welch’s t-test allows for different variances between classes. Mann-Whitney (Wilcoxon) converts the data to ranks, and does not assume a particular distribution. Permutation test computes the t-statistic for many random permutations of the labels.

23. 6/10/201523 The Student’s t-test For sample sizes less than 30 we have to make use of a t-distribution We make use of this distribution in the two-sample Students t-test. This test is used to test whether two samples come from distributions with the same means. The samples are assumed to come from Gaussian (normal) distributions. The two samples must have similar dispersions

24. 6/10/201524 The student’s t distribution The students t distribution –is mound shaped –is symmetrical about zero –is more widely dispersed than the standard normal distribution –it’s actual shape is dependent on the sample size different t distributions are identified by their degrees of freedom (df), where df = n-1

25. 6/10/201525 The student’s t distribution (cont.) df=120 (=z) df=30 df=15 EG’s (not to scale)

26. Mean and Median The mean is the most common measure of the location of a set of points. However, the mean is very sensitive to outliers. Thus, the median or a trimmed mean is also commonly used. 6/10/201526

27. Range and Variance Range is the difference between the max and min The variance or standard deviation s x is the most common measure of the spread of a set of points. Because of outliers, other measures are often used. 6/10/201527

28. 6/10/201528 Statistical Analysis control group mean treatment group mean Is there a difference?

29. 6/10/201529 What does difference mean? medium variability high variability low variability The mean difference is the same for all three cases

30. 6/10/201530 What does difference mean? medium variability high variability low variability Which one shows the greatest difference?

31. 6/10/201531 What does difference mean? a statistical difference is a function of the difference between means relative to the variability a small difference between means with large variability could be due to chance like a signal-to-noise ratio low variability Which one shows the greatest difference?

32. 6/10/201532 So we estimate low variability signal noise difference between group means variability of groups = X T - X C SE(X T - X C ) = = t-value __ __

33. 6/10/201533 Probability - p With t we check the probability Reject or do not reject Null hypothesis You reject if p < 0.05 or less Difference between means (groups) is more & more significant if p is less & less

34. 6/10/201534 Important notes on two sample comparisons Type I errors (false positive)– we accept a difference is real when it is not (at the 95% confidence level we are, of course, wrong 5% of the time) –We can increase the significance level to decrease these errors Type II errors (false negative)– if we increase our significance level we risk missing some real differences by making our testing too stringent. Convention is we should reduce Type I errors and be conservative Both can be minimised by increasing the sample size

35. 6/10/201535 Paired and unpaired tests There are different formulas for the T-test depending on whether we have paired or unpaired data –Paired – making observations of N individuals in two different situations In this situation we can consider the difference for each individual rather than calculate separate means and SEs for the two effects –Unpaired – Two separate samples drawn from the same parent population Can have different sample sizes

36. 6/10/201536 Tails Two-tailed: Do set A and set B come from different distributions? One-tailed: Does set A come from a distribution with larger mean than set B? This corresponds to finding differentially regulated genes versus finding up-regulated genes.

37. 6/10/201537 Selecting genes with a t-test μ i = mean expression value in class i n i = number of examples in class i v = pooled variance across both classes http://mathworld.wolfram.com/Studentst-Distribution.html Zar. Biostatistical Analysis. 1999.

38. 6/10/201538 Standard T Test: An example Observed gene expression values: Treatment A: 0.45 0.57 1.02 0.97 Treatment B: 1.50 2.07 0.51 1.63 Compute mean: mean (A) = 3.01 / 4 = 0.7525 mean (B) = 5.71 / 4 = 1.4275

39. 6/10/201539 Pooled variance The standard t-test assumes samples are drawn from distributions with the same variance. Pooled variance = (SS 1 + SS 1 ) / (n 1 + n 2 - 2) = (0.243675 + 1.300875) / (4 + 4 - 2) = 0.2574 SS: variance

40. 6/10/201540 Selecting genes with a t-test t = (0.7525 - 1.4275) / sqrt(0.2574/4 + 0.2574/4) = 1.8815

41. 6/10/201541 If the Sample Variances are Unlikely to be Equal Use Welch’s t-testdegrees of freedom where

42. 6/10/201542 Welch’s approximation t = 1.8815 Welch’s = |0.7525 - 1.4275| / sqrt(0.08089/4 + 0.43363/4) = 1.8821 t-testWelch’s

43. 6/10/201543 Degrees of freedom For the t-test, dof = n 1 + n 2 - 2. For Welch’s approximation, it is not so simple. Let A i = var i / n i. Then

44. 6/10/201544 Non-parametric p-value The t-test assumes the t-distribution –a parametric method –compute the test statistics –use the t pdf to determine the p-value A non-parametric method –data are labeled as X and Y –compute the test statistics with true labels –randomly permute the individual labels 10000 times, and compute the test statistics –find the rank of the true test statistics among the test statistics of random permutations –for example, if there are 10 permutations with test statistics larger than the true test statistics, then the p-value is 0.001

45. 6/10/201545 Mann-Whitney u-test Mann-Whitney, also known as Wilcoxon, is a non-parametric test. Begin by converting to ranks: Treatment A: 0.45 0.57 1.02 0.97 Treatment B: 1.50 2.07 0.51 1.63 Treatment A: 1 3 5 4 Treatment B: 6 8 2 7

46. 6/10/201546 Mann-Whitney u statistic The u statistic is where R i is the sum of the ranks in class i. U = 16 + 10 - 13 = 13

47. 6/10/201547 Permutation test

48. 6/10/201548 Cost-benefits analysis t-test assumes both samples are drawn from the same normal distribution. Welch’s approximation allows the samples to be drawn from different normals. Mann-Whitney makes no assumption about the distribution. The tests, as listed, yield decreasing power. The permutation test gives the most flexibility in choosing a test statistic that reflects prior knowledge, but it can be computationally expensive for small p-values.

49. 6/10/201549 Multiple testing correction On an array of 10,000 spots, a p-value of 0.0001 may not be significant. For significance of 0.05 with 10,000 spots, you need a p-value of 5  10 -6.

50. 6/10/201550 Family-wise Error-rate FWER Chance of any false positives Assume 0.01 significance level for one gene Multiply by the number of genes Many false positives Bonferroni correction: divide 0.01 by the number of genes Bonferroni is conservative because it assumes that all genes are independent.

51. 6/10/201551 Types of errors False positive (Type I error): the experiment indicates that the gene has changed, but it actually has not. False negative (Type II error): the gene has changed, but the experiment failed to indicate the change. Typically, researchers are more concerned about false positives. Without doing many (expensive) replicates, there will always be many false negatives.

52. 6/10/201552 False discovery rate The false discovery rate (FDR) is the percentage of genes above a given position in the ranked list that are expected to be false positives. False positive rate: percentage of non-differentially expressed genes that are flagged. False discovery rate: percentage of flagged genes that are not differentially expressed. 5 FP 13 TP 33 TN 5 FN FDR = FP / (FP + TP) = 5/18 = 27.8% FPR = FP / (FP + TN) = 5/38 = 13.2%

53. 6/10/201553 Bonferroni vs. FDR Bonferroni controls the family-wise error rate; i.e., the probability of at least one false positive. FDR is the proportion of false positives among the genes that are flagged as differentially expressed.

54. 6/10/201554 Controlling the FDR Order the unadjusted p-values p 1  p 2  …  p m. To control FDR at level α, Reject the null hypothesis for j = 1, …, j*. This approach is conservative if many genes are differentially expressed. (Benjamini & Hochberg, 1995)

55. 6/10/201555 q-value The p-value for a particular gene G is the probability that a randomly generated expression profile would be as or more extremely differentially expressed. The q-value for a particular gene G is the proportion of false positives among all genes that are as or more extremely differentially expressed. Equivalently, the q-value is the minimal FDR at which this gene appears significant.

56. 6/10/201556 Q-value software http://faculty.washington.edu/~jstorey/qvalue/

57. 6/10/201557 SAM Significance analysis of microarrays applied to the ionizing radiation response Virginia Goss Tusher, Robert Tibshirani, and Gilbert Chu Proc. Natl. Acad. Sci. USA, Vol. 98, Issue 9, 5116- 5121, April 24, 2001

58. 6/10/201558 Abstract Method for gene filtering: find genes change that significantly across samples Significance Analysis of Microarrays (SAM) assigns a score to each gene on the basis of change in gene expression relative to the standard deviation of repeated measurements. For genes with scores greater than an adjustable threshold, SAM uses permutations of the repeated measurements to estimate the percentage of genes identified by chance, the false discovery rate (FDR).

59. 6/10/201559 Introduction Suitable for oligo, cDNA, protein arrays Does not normalize the data! Challenge: –methods based on conventional t tests provide the probability (P) that a difference in gene expression occurred by chance. For an array with 10000 genes, a significance level of alpha = 0.01 would identify 100 genes by chance. –Experiments are expensive.

60. 6/10/201560 Introduction Solution based on SAM: –assimilate a set of gene-specific t tests. Each gene is assigned a score on the basis of its change in gene expression relative to the standard deviation of repeated measurements for that gene. –Instead of more replicates, generate permutations of the data (mix the labels) Genes with scores greater than a threshold are deemed potentially significant. The percentage of such genes identified by chance is the false discovery rate (FDR). To estimate the FDR, nonsense genes are identified by analyzing permutations of the measurements. The threshold can be adjusted to identify smaller or larger sets of genes, and FDRs are calculated for each set. To demonstrate its utility, SAM was used to analyze a biologically important problem: the transcriptional response of lymphoblastoid cells to ionizing radiation (IR).

61. 6/10/201561 Motivating Experiment Human Cell Lines Treatment Irradiated (I)Unirradiated (U) 1 One RNA sample for each combination of cell line and treatment 2

62. 6/10/201562 Motivating Experiment Human Cell Lines Treatment Irradiated (I)Unirradiated (U) 1 U1A U1B U2A U2B I1A I1B I2A I2B After labeling, each RNA sample was split into two aliquots denoted A and B. 2

63. 6/10/201563 Motivating Experiment Human Cell Lines Treatment Irradiated (I)Unirradiated (U) 1 U1A U1B U2A U2B I1A I1B I2A I2B 8 GeneChips, one for each sample, were used to obtain measures of expression. 2

64. 6/10/201564 First glance at the data Linear Scatter plot of gene expressionCube root scatter plot of gene expression

65. 6/10/201565 How to find the significant changes? Naïve method Cube root scatter plot of average gene expression from the four hybridizations with uninduced cells (avg xU) and induced cells 4 h after exposure to 5 Gy of IR (avg xI). Some of the genes that responded to IR are indicated by arrows.

66. 6/10/201566 Test Statistic for the i th Gene d(i) = x I (i) – x U (i) - - s(i)+s 0 Average of 4 normalized measures from irradiated samples Average of 4 normalized measures from unirradiated samples The usual standard deviation in the denominator of a two-sample t-stat A constant common to all genes that is added to make variation in d(i) similar across genes of all intensity levels

67. 6/10/201567 Selecting the constant s 0 At low expression levels, variance in d(i) can be high because of small values of s(i). To stabilize the variance of d(i) across genes, a small positive constant s 0 was used in the denominator of the test statistic. “The coefficient of variation of d(i) was computed as a function of s(i) in moving windows across the data. The value for s 0 was chosen to minimize the coefficient of variation.” s 0 was chosen to be 3.3 for the ionizing radiation data.

68. 6/10/201568 More Detail on Selecting s 0 The d(i) are separated into approximately 100 groups. The 1% of the d(i) values with the smallest s(i) values are placed in the first group, the 1% of the d(i) values with the next smallest s(i) are placed in the second group, etc. The median absolute deviation (MAD) of the d(i) values is computed separately for each group. The coefficient of variation (CV) of these 100 MAD values is computed.

69. 6/10/201569 More Detail on Selecting s 0 (continued) This process is repeated for values of s 0 equal to the minimum of s(i) over i, the 5 th percentile of the s(i) values, the 10 th percentile of the s(i) values,..., the 95 th percentile of the s(i) values, and the maximum of the s(i) values. The value of s 0 that minimizes the CV of the 100 MAD values over candidate s 0 described above is selected as the constant s 0.

70. 6/10/201570 Balancing the Permutations There are differences between the two cell lines. Balanced permutations- to minimize the effects of these differences A permutation is balanced if each group of four experiments contained two experiments from line 1 and two from line 2. There are 36 balanced permutations.

71. 6/10/201571 Example Permutations Human Cell Lines Treatment Irradiated (I)Unirradiated (U) 1 I1A I1B U1A U1B I2A I2B U2A U2B 2

72. 6/10/201572 Scatter plots of relative difference in gene expression d(i) vs. genespecific scatter s(i).

73. 6/10/201573 A Permutation Procedure for Assessing Significance 1.The irradiated and unirradiated GeneChips were shuffled within each cell line. 2.The d(i) statistic was computed for each gene and ordered across genes from smallest to largest to obtain d 1 (1)<d 1 (2)< <d 1 (g) where g denotes the number of genes. 3.Steps 1 and 2 were repeated for all possible data permutations described in step 1 to obtain d p (1)<d p (2)< <d p (g) for p=1,...,36.... 4242 4242

74. 6/10/201574 A Permutation Procedure for Assessing Significance (continued) 4.For each i, d 1 (i),...,d 36 (i) were averaged to obtain d E (i), the “expected relative difference.” 5.The original d(i) statistics were also sorted so that d(1)<d(2)< <d(g). 6.Genes for which | d(i) – d E (i) | >  were declared significant, where  is a user specified cutoff for significance....

75. 6/10/201575 Example

76. 6/10/201576 Plot of Observed vs. “Expected” Test Statistics d(i) d E (i) Points for genes with evidence of induction Points for genes with evidence of repression 2 

77. 6/10/201577 Plot of d(i) vs. log 10 s(i) for the Ionizing Radiation Data d(i) log 10 s(i) 24 induced genes 22 repressed genes

78. 6/10/201578 Estimating FDR for a Selected  1.Find the smallest d(i) among those d(i) for which d(i) – d E (i) >  and call it d up. 2.Find the largest d(i) among those d(i) for which d(i) - d E (i) < -  and call it d down. 3.For each permuted data set, find the number of genes with d(i) >= d up or d(i) <= d down and denote these counts by n 1,...,n 36. 4.FDR is estimated by n / n where n is the average of n 1,...,n 36 and n is the number of genes identified as significant in the original data. --

79. 6/10/201579 FDR cont’d Note: Cutoffs are asymmetric

80. 6/10/201580 Counts of Genes beyond the Threshold For Each Permutation 1 45 2 5 3 2 4 3 5 4 6 11 7 8 8 5 9 1 10 1 11 3 12 4 13 4 14 1 15 3 16 9 17 12 18 31 19 31 20 12 21 9 22 3 23 1 24 4 25 4 26 2 27 1 28 1 29 5 30 9 31 11 32 4 33 3 34 2 35 5 36 46 Perm Count

81. 6/10/201581 Mean Count = 8.472 FDR Estimate = 8.472/46 = 18.4% 1 45 2 5 3 2 4 3 5 4 6 11 7 8 8 5 9 1 10 1 11 3 12 4 13 4 14 1 15 3 16 9 17 12 18 31 19 31 20 12 21 9 22 3 23 1 24 4 25 4 26 2 27 1 28 1 29 5 30 9 31 11 32 4 33 3 34 2 35 5 36 46 Perm Count

82. 6/10/201582 How to choose Δ? Omitting s 0 caused higher FDR.

83. 6/10/201583 Plot of Observed vs. “Expected” Test Statistics d(i) d E (i) -4.073859 4.054688

84. 6/10/201584 Plot of d(i) vs. log 10 s(i) for the Ionizing Radiation Data d(i) log 10 s(i) -4.073859 4.054688

85. 6/10/201585 Same Plot for One of the Permuted Data Sets d(i) -4.073859 4.054688 log 10 s(i) only 5 genes beyond thresholds compared to 46 for original data

86. 6/10/201586 SAM vs. R fold R-fold Method: Gene i is significant if r(i)>R or r(i)<1/R FDR 73%-84% - Unacceptable. Pairwise fold change: At least 12 out of 16 pairings satisfying the criteria. FDR 60%-71% - Unacceptable. Why doesn’t it work?

87. 6/10/201587 Fold-change, SAM- Validation

88. 6/10/201588

89. 6/10/201589 SAM vs. Multiple t-Tests Trying to keep the FDR or FWER (Family– wise error rate). Why doesn’t it work? FWER- too stringent (Bonferroni, Westfall and Young) FDR- too granular (Benjamini and Hochberg) SAM does not assume normal distribution of the data SAM works effectively even with small sample size.

90. 6/10/201590 Conclusion SAM SAM is a method for identifying genes on a microarray with statistically significant changes in expression. SAM provides an estimate of the FDR for each value of the tuning parameter. The estimated FDR is computed from permutations of the data. SAM can be generalized to other types of experiments and outcomes by redefining d(i) http://www-stat- class.stanford.edu/SAM/SAMServlet.

91. 6/10/201591 ANOVA The t-test and its variants only work when there are two sample pools. Analysis of variance (ANOVA) is a general technique for handling multiple variables, with replicates. A tutorial is available here: http://cran.at.r-project.org/doc/contrib/Faraway-PRA.pdf

92. 6/10/201592 A simple experiment Measure response to a drug treatment in two different mouse strains. Repeat each measurement five times. Total experiment = 2 strains * 2 treatments * 5 repetitions = 20 arrays If you look for treatment effects using a t- test, then you ignore the strain effects.

93. 6/10/201593 ANOVA lingo Factor: a variable that is under the control of the experimenter (strain, treatment). Level: a possible value of a factor (drug, no drug). Main effect: an effect that involves only one factor. Interaction effect: an effect that involves two or more factors simultaneously. Balanced design: an experiment in which each factor and level is measured an equal number of times.

94. 6/10/201594 Two-factor design

95. 6/10/201595 Fixed and random effects Fixed effect: a factor for which the levels would be repeated exactly if the experiment were repeated. Random effect: a term for which the levels would not repeat in a replicated experiment. In the simple experiment, treatment and strain are fixed effects, and we include a random effect to account for biological and experimental variability.

96. 6/10/201596 ANOVA model  is the mean expression level of the gene. T and S are main effects (treatment, strain) with n and m levels, respectively. TS is an interaction effect. p is the number of replicates per group.  represents random error (to be minimized).

97. 6/10/201597 ANOVA steps For each gene on the array –Fit the parameters T and S, minimizing . –Test T, S and TS for difference from zero, yielding three F statistics. –Convert the F statistics into p-values.

98. 6/10/201598 ANOVA assumptions For a given gene, the random error terms are independent, normally distributed and have uniform variance. The main effects and their interactions are linear.

99. 6/10/201599 Summary Individual measurements from microarray experiments are not trustworthy. Repetition or independent verification (e.g., RT- PCR) are the best means of verification. For simple designs, use Welch’s approximation of the t-test. For complex designs, use ANOVA. Correct for multiple comparisons using FDR and q-values.

100. 6/10/2015100 Bioconductor Bioconductor is an open source project to design and provide high quality software and documentation for bioinformatics. Current focus: microarrays and gene (transcript) annotation Most of the early developments are in the form of R packages. Open to (your?) contributions Software and documentation are available from www.bioconductor.org.www.bioconductor.org

101. 6/10/2015101 Bioconductor packages General infrastructure –Biobase –annotate, AnnBuilder –tkWidgets Pre-processing for Affymetrix data –affy. Pre-processing for cDNA data –marrayClasses, marrayInput, marrayNorm, marrayPlots. Differential expression –edd, genefilter, multtest, ROC. etc.