Presentation is loading. Please wait.

Presentation is loading. Please wait.

Advanced Methods of Data Analysis 9:00 - 10:00CTWC 10:00 - 11:00 CTWC exercise 11:00 – 11:30 Break 11:30 - 12:00 SPIN 12:00 - 13:00 SPIN exercise Course.

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Advanced Methods of Data Analysis 9:00 - 10:00CTWC 10:00 - 11:00 CTWC exercise 11:00 – 11:30 Break 11:30 - 12:00 SPIN 12:00 - 13:00 SPIN exercise Course."— Presentation transcript:

1 Advanced Methods of Data Analysis 9:00 - 10:00CTWC 10:00 - 11:00 CTWC exercise 11:00 – 11:30 Break 11:30 - 12:00 SPIN 12:00 - 13:00 SPIN exercise Course on Microarray Data Acquisition and Analysis Weizmann Institute of Science 16 May 2007 Presented by Tal Shay & Yuval Tabach Weizmann Institute of Science Rehovot, Israel

2 Coupled Two-Way Clustering CTWC Gad Getz, Erel Levine, and Eytan Domany Coupled two-way clustering analysis of gene microarray data PNAS 97: 12079-12084 Course on Microarray Data Acquisition and Analysis Weizmann Institute of Science 16 May 2007 Presented by Tal Shay & Yuval Tabach Weizmann Institute of Science Rehovot, Israel

3 Talk Aim Guide how to use the CTWC server to properly analyze micro-array data.

4 Motivation Micro-array experiments generate millions of numbers containing a lot of biological information. The problem: Very complicated data contain large amount of noise. How to unravel the biological information which is masked by a mess of irrelevant information. CTWC is a simple heuristic clustering procedure that was developed especially to cope with micro-array data.

5 Talk Outline Preprocessing and filtering Clustering of Genes and Conditions Super-Paramagnetic Clustering (SPC) Coupled Two-Way Clustering (CTWC) CTWC server Exercise

6 Gene Expression Matrix – CTWC format Sample3Sample2Sample1Name DB_NAME E 13 E 12 E 11 Gene1Acc1 E 23 E 22 E 21 Gene2Acc2 E 33 E 32 E 31 Gene3Acc3 The DB_NAME is used to link genes to a database

7 Visualization of Expression Matrix Column = chip (=sample) Row = probeset Color = expression level genes samples

8 Preprocessing Initial Expression Matrix genes samples 1.Select variable genes 2.Standardize

9 Preprocessing 1000 probesets with highest standard deviation genes samples 1.Select variable genes 2.Standardize

10 Preprocessing genes samples 1.Select variable genes 2.Standardize 1000 probesets with highest standard deviation, standardized

11 Talk Outline Preprocessing and filtering Clustering of Genes and Conditions Super-Paramagnetic Clustering (SPC) Coupled Two-Way Clustering (CTWC) CTWC server Exercise

12 What questions can we ask? Which genes are expressed differently in two known types of samples? What is the minimal set of genes needed to distinguish one type of samples from the others? Which genes behave similarly in the experiments? How many different types of samples are there? Supervised Methods Hypothesis Testing (use predefined labels) Supervised Methods Hypothesis Testing (use predefined labels) Unsupervised Methods Exploratory Analysis (use only the data)

13 All genes Filtering Clustering samples genes Clustering – unsupervised analysis Low variation genes 1 2 3 High variation genes 3 clusters, each contains highly correlated genes

14 Goal A: Find groups of genes that have correlated expression profiles. These genes are believed to belong to the same biological process and might be co-regulated. Learn on the biology, infer function Goal B: Divide conditions to groups with similar gene expression profiles. Examples: Find sub-types of a disease, group or drugs according to their effect Unsupervised Analysis Clustering Methods

15 Giraffe DEFINITION OF THE CLUSTERING PROBLEM

16 CLUSTER ANALYSIS YIELDS DENDROGRAM Dendrogram1 T (RESOLUTION) How many clusters we have ? The answer depends on the resolution

17 Giraffe + Okapi BUT WHAT ABOUT THE OKAPI?

18 Clustering problem definition Input: N data points, X i, i=1,2,…,N in a D dimensional space. Goal: Find “natural” groups (clusters) of points. Points that belong to the same cluster – are “more similar”

19 Clustering is not well defined Similarity: which points should be considered close? Clustering method: –Resolution: specify/hierarchical results –Shape of clusters: general, spherical.

20 5 24 13 Agglomerative Hierarchical Clustering 3 1 4 2 5 Distance between joined clusters Dendrogram

21 Need to define the distance between the new cluster and the other clusters. Single Linkage: Distance between closest pair. Complete Linkage: Distance between farthest pair. Average Linkage: Average distance between all pairs or distance between cluster centers Need to define the distance between the new cluster and the other clusters. Single Linkage: Distance between closest pair. Complete Linkage: Distance between farthest pair. Average Linkage: Average distance between all pairs or distance between cluster centers Single Linkage Average Linkage Conclusion: The clustering result depends on the method we are using

22 Agglomerative Hierarchical Clustering Results depend on distance update method –Single Linkage: elongated clusters –Average Linkage: sphere-like clusters Greedy iterative process NOT robust against noise Not always finds the “natural” clusters.

23 Stop … think We want to identify the real (“natural”) clusters. We should have a reliability parameter that will help us to distinguish between significant and non-significant clusters.

24 Talk Outline Preprocessing and filtering Clustering of Genes and Conditions Super-Paramagnetic Clustering (SPC) Coupled Two-Way Clustering (CTWC) CTWC server Exercise

25 Super-Paramagnetic Clustering (SPC) M.Blatt, S.Weisman and E.Domany (1996) Neural Computation The idea behind SPC is based on the physical properties of dilute magnets. Calculating correlation between magnet orientations at different temperatures (T). T=Low Small elements, Spins

26 The idea behind SPC is based on the physical properties of dilute magnets. Calculating correlation between magnet orientations at different temperatures (T). T=High Super-Paramagnetic Clustering (SPC) M.Blatt, S.Weisman and E.Domany (1996) Neural Computation

27 The idea behind SPC is based on the physical properties of dilute magnets. Calculating correlation between magnet orientations at different temperatures (T). T=Intermediate

28 T=High Phases of the Inhomogeneous Potts Ferromagnet T=Low T=Intermediate Ferro Para Super-Para

29 Super-Paramagnetic Clustering (SPC) T=Low T=High T=Low T=Intermediate

30 The algorithm simulates the magnets behavior at a range of temperatures and decides which interactions to break. The temperature (T) controls the resolution Super-Paramagnetic Clustering (SPC) Example: N=4800 points in D=2

31 Identify the stable clusters T=16

32 Same data - Average Linkage

33 Advantages of SPC Scans all resolutions (T) Robust against noise and initialization - calculates collective correlations. Identifies “natural” and stable clusters (  T) No need to pre-specify number of clusters Clusters can be any shape

34 Inside SPC: dendrogam and stable clusters T 10 22 24 26 28 Min Cluster Size: 3 Stable Delta T: 14 Ignore dropout: 1

35 GenesSamples CTWC server - Setting the SPC parameters

36 Talk Outline Preprocessing and filtering Clustering of Genes and Conditions Super-Paramagnetic Clustering (SPC) Coupled Two-Way Clustering (CTWC) CTWC server Exercise

37 Back to gene expression data 2 Goals: Cluster Genes and Conditions 2 independent clustering: –Genes represented as vectors of expression in all conditions –Conditions are represented as vectors of expression of all genes

38 1. Identify tissue classes (tumor/normal) First clustering - Experiments D = 2000

39 2. Find Differentiating And Correlated Genes Second Clustering - Genes D = 62 genes samples

40 Two-way clustering S1(G1) G1(S1) TWO-WAY CLUSTERING:

41 TWO-WAY CLUSTERING: Two way clustering-ordered S1(G1) G1(S1)

42 Football Song A Song B

43 Coupled Two-Way Clustering (CTWC) G. Getz, E. Levine and E. Domany (2000) PNAS Philosophy: Only a small subset of genes play a role in a particular biological process; the other genes introduce noise, which may mask the signal of the important players. Only a subset of the samples exhibit the expression patterns of interest. New Goal: Use subsets of genes to study subsets of samples (and vice versa) A non-trivial task – exponential number of subsets. CTWC is a heuristic to solve this problem.

44 Inside CTWC: Iterations DepthGenesSamples InitG1S1 1G1(S1)G2,G3,…G5S1(G1)S2,S3 2G1(S2) G1(S3) G6,G7,….G13 G14,…G21 S1(G2) … S1(G5) S4,S5,S6 S10,S11 None 3G2(S1)…G2(S3) … G5(S1)…G5(S3) G22… … …G97 S2(G1)…S2(G5) S3(G1)…S3(G5) S12,… …S51 4G1(S4) … G1(S11) G98,..G105 … G151,..G160 S1(G6) … S1(G21) S52,... S67 5G2(S4)...G2(S11) … G5(S4)...G5(S11) G161… … …G216 S2(G6)...S2(G21) S3(G6)…S3(G21) S68… …S113 Two-way clustering

45 E-mail notification CTWC server - Setting the coupled two-way clustering parameters

46 COUPLED TWO-WAY CLUSTERING OF COLON CANCER: TISSUES tissues 1 G4 G12 S1(G4) S1(G12)

47 COUPLED TWO-WAY CLUSTERING OF COLON CANCER: TISSUES CTWC colon cancer - tissues S1(G4) S1(G12) S17

48 colon cancer carcinoma +adenoma What kind of results do you wish to find ? type A /type B distance matrix

49 Talk Outline Preprocessing and filtering Clustering of Genes and Conditions Super-Paramagnetic Clustering (SPC) Coupled Two-Way Clustering (CTWC) CTWC server Exercise

50 CTWC software Web interface –ctwc.weizmann.ac.il –ctwc.bioz.unibas.ch Standalone – Write to Assif.Yitzhaky@weizmann.ac.il

51 CTWC standalone

52 Sample Labels Given as a binary file For a cluster Gx, label L with values L1 and L2: Purity(C1, L1) – how much of C1 is composed of L1? Efficiency(C1, L1) – how much of L1 is contained in of C1? #L1 in C |L1| #L1 in C |C1|

53 Biological Work Literature search for information on interesting genes. Annotation analysis: classify the genes according to their function. Find whether there is a common function or biological meaning for clusters of interest. Find what is in common with sets of experiments/conditions. Genomics analysis: search for common regulatory signal upstream of the genes Design next experiment – get more data to validate result. Remember : most of your work is starting here - understanding the biology behind your results

54 Summary Clustering methods are used to –find genes from the same biological process –group the experiments to similar conditions Focusing on subsets of the genes and conditions can unravel structure that is masked when using all genes and conditions ctwc.weizmann.ac.il or Assif.Yitzhaky@weizmann.ac.il

55 Exercise - Course Experiment NT48hr72hr96hr D8D8_NT_s_1b D8_NT_c_1a D8_NT_c_2 D8_48h_s_1b D8_48h_c_1a D8_48h_c_2 D8_72h_s_1b D8_72h_c_1a D8_96h_s_1b D8_96h_c_1a D8_96h_c_2 D11D11_NT_s_2 D11_NT_c_1a D11_NT_c_1b D11_48h_c_1a D11_48h_c_1b D11_72h_s_2 D11_72h_c_1a D11_72h_c_1b D11_96h_c_1a D11_96h_c_1b On time 0 a treatment is given. For D8, treatment suppresses mutp53. For D11, treatment does not.

56 The Data Save and backup the CEL files!

57 R Code – From CEL to ECXEL > library(affy) > A = ReadAffy() > rma_data = rma(A) > write.exprs(rma_data, file='rma_expression.txt') > mas5_data = mas5(A) > write.exprs(mas5_data, file = 'mas5_expression') > mas5_calls = mas5calls(A) > write.exprs(mas5_calls, file = 'mas5_detection')

58 The EXCEL Filter the genes – do not cluster all probesets on the chip!

59 Edit the EXCEL for CTWC Title #1: U133_AFFX Title #2: NAME Column #2: Probeset info Make the chip names clear!

60 Samples distance matrix

Download ppt "Advanced Methods of Data Analysis 9:00 - 10:00CTWC 10:00 - 11:00 CTWC exercise 11:00 – 11:30 Break 11:30 - 12:00 SPIN 12:00 - 13:00 SPIN exercise Course."

Similar presentations

Basic Gene Expression Data Analysis--Clustering

Basic Gene Expression Data Analysis--Clustering
Hierarchical Clustering. Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram – A tree-like diagram that.

Hierarchical Clustering. Produces a set of nested clusters organized as a hierarchical tree Can be visualized as a dendrogram – A tree-like diagram that.
Supervised and unsupervised analysis of gene expression data Bing Zhang Department of Biomedical Informatics Vanderbilt University

Supervised and unsupervised analysis of gene expression data Bing Zhang Department of Biomedical Informatics Vanderbilt University
D ISCOVERING REGULATORY AND SIGNALLING CIRCUITS IN MOLECULAR INTERACTION NETWORK Ideker Bioinformatics 2002 Presented by: Omrit Zemach April 3 2013 Seminar.

D ISCOVERING REGULATORY AND SIGNALLING CIRCUITS IN MOLECULAR INTERACTION NETWORK Ideker Bioinformatics 2002 Presented by: Omrit Zemach April Seminar.
Principal Component Analysis (PCA) for Clustering Gene Expression Data K. Y. Yeung and W. L. Ruzzo.

Principal Component Analysis (PCA) for Clustering Gene Expression Data K. Y. Yeung and W. L. Ruzzo.
BASIC METHODOLOGIES OF ANALYSIS: SUPERVISED ANALYSIS: HYPOTHESIS TESTING USING CLINICAL INFORMATION (MLL VS NO TRANS.) IDENTIFY DIFFERENTIATING GENES Basic.

BASIC METHODOLOGIES OF ANALYSIS: SUPERVISED ANALYSIS: HYPOTHESIS TESTING USING CLINICAL INFORMATION (MLL VS NO TRANS.) IDENTIFY DIFFERENTIATING GENES Basic.
UNSUPERVISED ANALYSIS GOAL A: FIND GROUPS OF GENES THAT HAVE CORRELATED EXPRESSION PROFILES. THESE GENES ARE BELIEVED TO BELONG TO THE SAME BIOLOGICAL.

UNSUPERVISED ANALYSIS GOAL A: FIND GROUPS OF GENES THAT HAVE CORRELATED EXPRESSION PROFILES. THESE GENES ARE BELIEVED TO BELONG TO THE SAME BIOLOGICAL.
The Broad Institute of MIT and Harvard Clustering.

The Broad Institute of MIT and Harvard Clustering.
Microarray technology and analysis of gene expression data Hillevi Lindroos.

Microarray technology and analysis of gene expression data Hillevi Lindroos.
Clustering II.

Clustering II.
Mutual Information Mathematical Biology Seminar 23.5.2005.

Mutual Information Mathematical Biology Seminar
C E N T R F O R I N T E G R A T I V E B I O I N F O R M A T I C S V U E Lecture 9 Clustering Algorithms Bioinformatics Data Analysis and Tools.

C E N T R F O R I N T E G R A T I V E B I O I N F O R M A T I C S V U E Lecture 9 Clustering Algorithms Bioinformatics Data Analysis and Tools.
Www.bioalgorithms.infoAn Introduction to Bioinformatics Algorithms Clustering.

Introduction to Bioinformatics Algorithms Clustering.
MOPAC: Motif-finding by Preprocessing and Agglomerative Clustering from Microarrays Thomas R. Ioerger 1 Ganesh Rajagopalan 1 Debby Siegele 2 1 Department.

MOPAC: Motif-finding by Preprocessing and Agglomerative Clustering from Microarrays Thomas R. Ioerger 1 Ganesh Rajagopalan 1 Debby Siegele 2 1 Department.
Clustering (Part II) 10/07/09. Outline Affinity propagation Quality evaluation.

Clustering (Part II) 10/07/09. Outline Affinity propagation Quality evaluation.
DNA Arrays …DNA systematically arrayed at high density, –virtual genomes for expression studies, RNA hybridization to DNA for expression studies, –comparative.

DNA Arrays …DNA systematically arrayed at high density, –virtual genomes for expression studies, RNA hybridization to DNA for expression studies, –comparative.
1 MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING By Kaan Tariman M.S. in Computer Science CSCI 8810 Course Project.

1 MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING By Kaan Tariman M.S. in Computer Science CSCI 8810 Course Project.
Introduction to Bioinformatics - Tutorial no. 12

Introduction to Bioinformatics - Tutorial no. 12
Gene Expression 1. Methods –Unsupervised Clustering Hierarchical clustering K-means clustering Expression data –GEO –UCSC EPCLUST 2.

Gene Expression 1. Methods –Unsupervised Clustering Hierarchical clustering K-means clustering Expression data –GEO –UCSC EPCLUST 2.
 Goal A: Find groups of genes that have correlated expression profiles. These genes are believed to belong to the same biological process and/or are co-regulated.

 Goal A: Find groups of genes that have correlated expression profiles. These genes are believed to belong to the same biological process and/or are co-regulated.

Similar presentations

Ads by Google