Download presentation
Presentation is loading. Please wait.
1
EGASP 2005 Evaluation Protocol
Paul Flicek EBI
2
Basics The evaluations are probably wrong GTF is not standard
There are hidden assumptions Filters, overlaps, clusters Terminology varies Genes, exons, etc. EGASP 2005 Evaluations
3
Evaluation Measures Exons and introns Transcript Gene Sensitivity (Sn)
Specificity (Sp) Exon length Exons per transcript Transcript Sn / Sp Overlap Gene EGASP 2005 Evaluations
4
Definitions EGASP 2005 Evaluations
5
Definitions Positive Transcript Positive Gene
Correct translation start Correct translation stop Every splice site correct Positive Gene At least one positive transcript EGASP 2005 Evaluations
6
Examples Annotation Trans Sn = 0.5 Trans Sp = 1.0 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
7
Examples Annotation Trans Sn = 0.5 Trans Sp = 1.0 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
8
Examples Annotation Trans Sn = 0.0 Trans Sp = 0.0 Gene Sn = 0.0
Gene Sp = 0.0 Prediction EGASP 2005 Evaluations
9
Examples Annotation Trans Sn = 1.0 Trans Sp = 1.0 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
10
Examples Annotation Trans Sn = 0.5 Trans Sp = 0.5 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
11
Examples Annotation Trans Sn = 1.0 Trans Sp = 0.67 Gene Sn = 1.0
Gene Sp = 1.0 Prediction EGASP 2005 Evaluations
12
The winners are… (there are clear trends)
The most successful programs use expressed sequences Programs using evolutionary conservation are more successful than those that do not Exon and nucleotide measures are similar We are improving EGASP 2005 Evaluations
13
Spear Catching Time EGASP 2005 Evaluations
15
EGASP 2005 Evaluations Block 1
Paul Flicek EBI Expressed Sequence Methods
16
Nucleotide EGASP 2005 Evaluations
17
Exon EGASP 2005 Evaluations
18
Intron EGASP 2005 Evaluations
19
Transcript EGASP 2005 Evaluations
20
Gene EGASP 2005 Evaluations
21
Transcript Overlap EGASP 2005 Evaluations
22
Average Exon Length EGASP 2005 Evaluations
23
Exons per transcript EGASP 2005 Evaluations
24
Number of Genes 1027 1389 EGASP 2005 Evaluations
25
Unique Exons EGASP 2005 Evaluations
26
Summary EGASP 2005 Evaluations
28
EGASP 2005 Evaluations Block 2
Paul Flicek EBI Evolutionary Conservation (Dual/Multiple Genome) Methods
29
Nucleotide EGASP 2005 Evaluations
30
Exon EGASP 2005 Evaluations
31
Intron EGASP 2005 Evaluations
32
Transcript EGASP 2005 Evaluations
33
Gene EGASP 2005 Evaluations
34
Transcript Overlap EGASP 2005 Evaluations
35
Average Exon Length EGASP 2005 Evaluations
36
Exons per transcript EGASP 2005 Evaluations
37
Number of Genes 1027 1389 EGASP 2005 Evaluations
38
Unique Exons EGASP 2005 Evaluations
39
Summary EGASP 2005 Evaluations
41
EGASP 2005 Evaluations Block 3a
Paul Flicek EBI Ab initio (single genome) and Exon only Methods
42
Nucleotide EGASP 2005 Evaluations
43
Exon EGASP 2005 Evaluations
44
Intron EGASP 2005 Evaluations
45
Transcript EGASP 2005 Evaluations
46
Gene EGASP 2005 Evaluations
47
Transcript Overlap EGASP 2005 Evaluations
48
Average Exon Length EGASP 2005 Evaluations
49
Exons per transcript EGASP 2005 Evaluations
50
Number of Genes 1027 1389 EGASP 2005 Evaluations
51
Unique Exons EGASP 2005 Evaluations
52
Summary EGASP 2005 Evaluations
54
EGASP 2005 Evaluations Block 3b
Paul Flicek EBI Open (Any) Methods
55
Nucleotide EGASP 2005 Evaluations
56
Exon EGASP 2005 Evaluations
57
Intron EGASP 2005 Evaluations
58
Transcript EGASP 2005 Evaluations
59
Gene EGASP 2005 Evaluations
60
Transcript Overlap EGASP 2005 Evaluations
61
Average Exon Length EGASP 2005 Evaluations
62
Exons per transcript EGASP 2005 Evaluations
63
Number of Genes 1027 1389 EGASP 2005 Evaluations
64
Unique Exons EGASP 2005 Evaluations
65
Summary EGASP 2005 Evaluations
Similar presentations
