More On Preprocessing Javier Cabrera

Outline 1.Transform the data into a scale suitable for analysis. 2.Remove the effects of systematic and obfuscating sources of variation. 3.Identify discrepant observations.

Outline Preprocessing => Quality of downstream analyses log transformation, X  log(X) The variation of logged intensities may be less dependent on magnitude, Logs reduces the skewness of highly skewed distributions. Taking logs improves variance estimation. 2. Other Transformations Power transformations (X  X  for some  =1/2, 1/3 or other) Amaratunga and Cabrera (2000), Tusher et al (2001) 3. Variance stabilizing transformations X  log(X+c) : Symmetrizing the spot intensity data and stabilizing their variances.

Transformations 4. Rocke and Durbin (2001) arrays with replicate spots. Analogy: models used for estimating concentration of analyte:X =  +  e  +   mean background,  true expression level;  and  normally distributed error (   2,   2 ) 5. Durbin et al (2002) generalized log transformation: - ,   2 and   2 must be estimated.

Power Transformations  must be estimated. -Three criteria: - Equal variances: CV ( gene variances) Low skewness: mean( skewness) No Mean Variance correlation: correlation between mean and variance

Example 1: Tissue Data Tissue data: 3 treatments applied to mice tissue. (A,B,C) Arrays: Treatment A: 11 Treatment B: 11 Treatment C: 19 Genes: 3487 genes. Gene expression matrix X: Dim(X)=100x41 treatA.1 treatA.2 treatA.3 treatA.4 treatA.5 treatA.6 treatA.7 treatA.8 treatA.9 treatA.10 treatA.11 treatB.12 treatB >

Power Trans (X-3.60 ) -0.4 Quantile Normalized Raw Data Equal 75pctl Log Transformed

Gene selection for classification - Left panel: PC2 vs PC1 plot log transformation - Right panel: PC2 vs PC1 plot power transformation

Example 2: Khan et al (2001): 4 types of small round blue cell tumors (SRBC) - Neuroblastoma (NB) - Rhabdomyosarcoma (RMS) - Ewing family of tumors (EWS) - Burkitt lymphomas (BL) Training set= 63 (23 EWS, 20 RMS, 12 NB, 8 BL) Testing set= 25 (6 EWS, 5 RMS, 6 NB, 3 BL, 5 ot) Genes: Of 6567 initial genes, 2308 genes were selected because they showed minimal expression Subset A: Cells: 23 EWS and 20 RMS from training set. 100 most significant genes after performing a t-test. Gene expression matrix X: Dim(X)=100x43 EWS.T1 EWS.T2 EWS.T3 EWS.T4 EWS.T6 EWS.T7 EWS.T9 EWS.T11 EWS.T12 EWS.T13 EWS.T14 EWS.T15 EWS.T19 EWS.C8 EWS.C3 EWS.C2 EWS.C4 EWS.C6 EWS.C

Power Trans -(X-0.66 ) Quantile Normalized Raw Data Equal 75pctl Log Transformed

Judging the success of a normalization {Y g1 } and {Y g2 }. Successful workflow =>Arrays are monotonically related to each other. Pearson’s correlation coefficient: measures linearity rather than agreement. Concordance correlation coefficient :

Judging the success of a normalization {Y g1 } and {Y g2 }. Successful workflow =>Arrays are monotonically related to each other. -Spearman’s rank correlation coefficient: R gi is the rank of Y gi when the {Y gi } are ranked from 1 to G.

Concordance Map Image Plot of Concordance Correlations: X44 X45 X46 X47 X48 X49 X50 X X X X X X X

Concordance Map Image Plot of Concordance Correlations: X44 X45 X46 X47 X48 X49 X50 X X X X X X X

Linear correlation Standard Normal t dist, df=6 t dist, df=2

correlation 1. If the distributional properties of the values change substantially during a normalization (e.g., the skewness is decreased), it is possible that the concordance correlation coefficients might increase, but this may only be an artificial improvement. 2.For microarrays that have been normalized by equating all the quantiles, the concordance correlation coefficient will be equal to Pearson’s correlation coefficient. This is because, after such a normalization, the quantiles of both samples are identical and, therefore, both means are equal and both variances are equal too 3.Spearman’s rank correlation coefficient is equal to (a) Pearson’s correlation coefficient calculated on the ranks of the data (b) the concordance correlation coefficient calculated on the ranks of the data.