First approach - repeating a simple analysis for each gene separately - 30k times Assume we have two experimental conditions (j=1,2) We measure expression of all genes n times under both experimental conditions (n two- channel microarrays) For a specific gene (focusing on a single gene) x ij = i th measurement under condition j Statistical models for expression measurements under two different Identifying Differentially Expressed Genes  1,  2,  are unknown model parameters -  j represents the average expression measurement in the large number of replicated experiments,  represents the variability of measurements Question if the gene is differentially expressed corresponds to assessing if  1   2 Strength of evidence in the observed data that this is the case is expressed in terms of a p- value

Estimate the model parameters based on the data P-value Calculating t-statistic which summarizes information about our hypothesis of interest (  1   2 ) Establishing the null-distribution of the t-statistic (the distribution assuming the “null- hypothesis” that  1 =  2 ) The “null-distribution” in this case turns out to be the t-distribution with n 1 +n 2 -2 degrees of freedom P-value is the probability of observing as extreme or more extreme value under the “null- distribution” as it was calculated from the data (t * )

t-distribution Number of experimental replicates affects the precision at two levels 1.Everything else being equal, increase in sample size increases the t * 2.Everything else being equal, increase in sample size “shrinks” the “null-distribution” Suppose that t * =3. What is the difference in p-values depending on the sample size alone. p-value = 0.2 p-value = 0.1 p-value = 0.01 p-value = 0.003

Performing t-test > load(url(" > ls() [1] "SimpleData" > SimpleData[1:5,] Name Ctl Nic Nic.1 Nic.2 Ctl.1 Ctl.2 1 D X M U X > > Nic<-grep("Nic",dimnames(SimpleData)[[2]]) > Ctl<-grep("Ctl",dimnames(SimpleData)[[2]]) > Nic [1] > Ctl [1] > SimpleData[1,Nic] Nic Nic.1 Nic > SimpleData[1,Ctl] Ctl Ctl.1 Ctl

Performing t-test > MNic<-mean(unlist(SimpleData[1,Nic])) > MNic [1] > MCtl<-mean(unlist(SimpleData[1,Ctl])) > MCtl [1] > VNic<-var(unlist(SimpleData[1,Nic])) > VNic > VCtl<-var(unlist(SimpleData[1,Ctl])) > VCtl > NNic<-sum(!is.na(SimpleData[1,Nic])) > NNic [1] 3 > NCtl<-sum(!is.na(SimpleData[1,Ctl])) > NCtl [1] 3 > VNicCtl<-(((NNic-1)*VNic)+((NCtl-1)*VCtl))/(NNic+NCtl-2) > VNicCtl > DF<-NNic+NCtl-2 > DF [1] 4 > TStat<-abs(MNic-MCtl)/((VNicCtl*((1/NNic)+(1/NCtl)))^0.5) > TStat > TPvalue<-2*pt(TStat,DF,lower.tail=FALSE) > TPvalue > >t.test(SimpleData[1,Nic],SimpleData[1,Ctl],var.equal=TRUE) Two Sample t-test data: LSimpleData[1, W] and LSimpleData[1, C] t = , df = 10, p-value = alternative hypothesis: true difference in means is not equal to 0 95 percent confidence interval: sample estimates: mean of x mean of y source(" source("

Statistical Inference and Statistical Significance – P-value Statistical Inference consists of drawing conclusions about the measured phenomenon (e.g. gene expression) in terms of probabilistic statements based on observed data. P-value is one way of doing this. P-value is NOT the probability of null hypothesis being true. Rigorous interpretation of p-value is tricky. It was introduced to measure the level of evidence against the “null-hypothesis” or better to say in favor of a “positive experimental finding” In this context p-value of could be interpreted as a stronger evidence than the p- value of 0.01 Establishing Statistical Significance (is a difference in expression level statistically significant or not) requires that we establish “cut-off” points for our “measure of significance” (p-value) For various historic reasons the cut-off 0.05 is generally used to establish “statistical significance”. It’s a rather arbitrary cut-off, but it is taken as a gold standard Originally the p-value was introduced as a descriptive measure to be used in conjuction with other criteria to judge the strength of evidence one way or another

Statistical Inference and Statistical Significance-Hypothesis Testing The 5% cut-off points comes from the Hypothesis testing world In this world the exact magnitude of p-value does not matter. It only matters if it is smaller than the pre-specified statistical significance cut-off (  ). The null hypothesis is rejected in favor of the alternative hypothesis at a significance level of  = 0.05 if p-value<0.05 Type I error is committed when the null-hypothesis is falsely rejected Type II error is committed when the null-hypothesis is not rejected but it is false By following this “decision making scheme” you will on average falsely reject 5% of null- hypothesis If such a “decision making scheme” is adopted to identify differentially expressed genes on a microarray, 5% of non-differentially expressed genes will be falsely implicated as differentially expressed. Family-wise Type I Error is committed if any of a set of null hypothesis is falsely rejected Establishing statistical significance is a necessary but not sufficient step in assuring the “reproducibility” of a scientific finding – Important point that will be further discussed when we start talking about issues in experimental design The other essential ingredient is a “representative sample” from the “population of interest” This is still a murky point in molecular biology experimentation

For a specific gene x ij = i th measurement under condition j, i=1,…,6; j=1,2 Is a Specific Gene Differentially Expressed Differential expression   1   2 Statistical Model of observed data Estimate the model parameters based on the data Calculating t-statistic t*t* -t * Calculating p-value based on the “null distribution” of the t-statistic assuming  1 =  2

How do we perform t-test for 30,000 at once How do we handle results, present data and results What is significant How to compare different approaches to normalization of the data and the statistical analysis of results Ideally, we would like to maximize our ability to identify truly differentially expressed genes and minimize the falsely implicated genes. Doing it by hand (by R) first Using Bioconductor Genome-wide analysis

Calculating t-test for 30,000 genes at a time > load(url(" > ls() [1] "SimpleData" > SimpleData[1:5,] Name Ctl Nic Nic.1 Nic.2 Ctl.1 Ctl.2 1 D X M U X > > Nic<-grep("Nic",dimnames(SimpleData)[[2]]) > Ctl<-grep("Ctl",dimnames(SimpleData)[[2]]) > Nic [1] > Ctl [1] > SimpleData[1,Nic] Nic Nic.1 Nic > SimpleData[1,Ctl] Ctl Ctl.1 Ctl

Calculating t-test for 30,000 genes at a time Calculating t-tests : source(" > MNic<-apply(SimpleData[,Nic],1,mean,na.rm=TRUE) > VNic<-apply(SimpleData[,Nic],1,var,na.rm=TRUE) > MCtl<-apply(SimpleData[,Ctl],1,mean,na.rm=TRUE) > VCtl<-apply(SimpleData[,Ctl],1,var,na.rm=TRUE) > NNic<-apply(!is.na(SimpleData[,Nic]),1,sum,na.rm=TRUE) > NCtl<-apply(!is.na(SimpleData[,Ctl]),1,sum,na.rm=TRUE) > > VNicCtl<-(((NNic-1)*VNic)+((NCtl-1)*VCtl))/(NCtl+NNic-2) > > DF<-NNic+NCtl-2 > > TStat<-abs(MNic-MCtl)/((VNicCtl*((1/NNic)+(1/NCtl)))^0.5) > TPvalue<-2*pt(TStat,DF,lower.tail=FALSE) > TStat[1] > TPvalue[1]

Calculating t-test for 30,000 genes at a time Calculating t-tests : source(" > par(mfrow=c(2,2)) > > plot((MNic-MCtl),-log(TPvalue,base=10),type="p",main="Vulcano Plot",xlab="Mean Difference",ylab="-log10(p-value)") > grid(nx = NULL, ny = NULL, col = "lightgray", lty = "dotted",lwd = NULL, equilogs = TRUE) > > plot(VNicCtl^0.5,-log(TPvalue,base=10),type="p",main="Signficance vs Variability",xlab="Standard Deviation",ylab="-log10(p- value)") > grid(nx = NULL, ny = NULL, col = "lightgray", lty = "dotted",lwd = NULL, equilogs = TRUE) > > plot((MNic+MCtl)/2,-log(TPvalue,base=10),type="p",main="p-values vs Average Expression",xlab="Average Expression",ylab="- log10(p-value)") > grid(nx = NULL, ny = NULL, col = "lightgray", lty = "dotted",lwd = NULL, equilogs = TRUE) > > plot((MNic+MCtl)/2,(MNic-MCtl),type="p",main="Differences vs Average Expression",xlab="Average Expression",ylab="Mean Difference") > grid(nx = NULL, ny = NULL, col = "lightgray", lty = "dotted",lwd = NULL, equilogs = TRUE) >

source(" Displaying results – Scatter Plots

Annotating Significant Genes Calculating t-tests : source(" > SigGenes<-(TPvalue<0.001) > sum(SigGenes) [1] 7 > SimpleData[SigGenes,] Name Ctl Nic Nic.1 Nic.2 Ctl.1 Ctl.2 34 M AK AF NM NM AF AF >

Annotating Significant Genes Calculating t-tests : source(" > library(annotate) > library(mouseLLMappings) > > locuslinkByID("13107") [1] " > > ACC2LL <- as.list(mouseLLMappingsACCNUM2LL) > ACC2LL["M77497"] $M77497 [1] > SigGenesLL<-ACC2LL[as.character(SimpleData[SigGenes,"Name"])]

Annotating Significant Genes Calculating t-tests : source(" > SigGenesLL<-ACC2LL[as.character(SimpleData[SigGenes,"Name"])] > SigGenesLL $M77497 [1] $AK [1] $"NA" NULL $"NA" NULL $"NA" NULL $AF [1] $"NA" NULL > locuslinkByID(unlist(SigGenesLL)) [1] " >