Download presentation
Presentation is loading. Please wait.
1
EGASP 2005 Evaluation Protocol
Paul Flicek EBI
2
Basics The evaluations are probably wrong GTF is not standard
There are hidden assumptions Filters, overlaps, clusters Terminology varies Genes, exons, etc. EGASP 2005 Evaluations
3
Evaluation Measures Exons and introns Transcript Gene Sensitivity (Sn)
Specificity (Sp) Exon length Exons per transcript Transcript Sn / Sp Overlap Gene EGASP 2005 Evaluations
4
Definitions EGASP 2005 Evaluations
5
Definitions Positive Transcript Positive Gene
Correct translation start Correct translation stop Every splice site correct Positive Gene At least one positive transcript EGASP 2005 Evaluations
6
Examples Annotation Trans Sn = 0.5 Trans Sp = 1.0 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
7
Examples Annotation Trans Sn = 0.5 Trans Sp = 1.0 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
8
Examples Annotation Trans Sn = 0.0 Trans Sp = 0.0 Gene Sn = 0.0
Gene Sp = 0.0 Prediction EGASP 2005 Evaluations
9
Examples Annotation Trans Sn = 1.0 Trans Sp = 1.0 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
10
Examples Annotation Trans Sn = 0.5 Trans Sp = 0.5 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
11
Examples Annotation Trans Sn = 1.0 Trans Sp = 0.67 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
12
The winners are… (there are clear trends)
The most successful programs use expressed sequences Programs using evolutionary conservation are more successful than those that do not Exon and nucleotide measures are similar We are improving EGASP 2005 Evaluations
13
Spear Catching Time EGASP 2005 Evaluations
15
EGASP 2005 Evaluations Block 1
Paul Flicek EBI Expressed Sequence Methods
16
Nucleotide EGASP 2005 Evaluations
17
Exon EGASP 2005 Evaluations
18
Intron EGASP 2005 Evaluations
19
Gene EGASP 2005 Evaluations
20
Number of Genes 1027 1389 EGASP 2005 Evaluations
21
Unique Exons EGASP 2005 Evaluations
22
Summary EGASP 2005 Evaluations
24
EGASP 2005 Evaluations Block 2
Paul Flicek EBI Evolutionary Conservation (Dual/Multiple Genome) Methods
25
Summary EGASP 2005 Evaluations
27
EGASP 2005 Evaluations Block 3a
Paul Flicek EBI Ab initio (single genome) and Exon only Methods
28
Summary EGASP 2005 Evaluations
30
EGASP 2005 Evaluations Block 3b
Paul Flicek EBI Open (Any) Methods
31
Summary EGASP 2005 Evaluations
Similar presentations
