Download presentation
Presentation is loading. Please wait.
Published byAnnabella Mills Modified over 9 years ago
1
Protein Identification via Database searching Attila Kertész-Farkas kfattila@icgeb.org Protein Structure and Bioinformatics Group, ICGEB, Trieste
2
Mass Spectra analysis Biological sample Results report
3
Mass Spectra analysis Biological sample Results report
4
Computational analysis of MS/MS Two approaches: – De novo sequencing – Database searching based – Hybrid
5
De novo sequencing
6
– can identify new peptides and proteins – Able to discover (new) PTMs – Independent of protein databases – Requires MS/MS data of good quality – No statistics based validation
7
Database searching-based MS/MS tandem mass spectra identification Pipeline Input data Peptide assignment Validation Protein inference Quantitation Interpretation
8
Database searching-based MS/MS tandem mass spectra identification Pipeline Input data Peptide assignment Validation Protein inference Quantitation Interpretation
9
Database searching-based MS/MS tandem mass spectra identification Pipeline Input data Peptide identification Validation Protein inference Quantitation Interpretation Data formatsDatabase searching Statistical methods for validations Protein assembling
10
Mass spectrum: – Histogram of the mass over charge of the observed fragment ions. – Spectrum normalization. Usually intensity is scaled to [0,100] interval. Input data Peptide assignment Validation Protein inference Quantitation Interpretation
11
Most common formats are the mzXML, MGF and DAT, Input data Peptide assignment Validation Protein inference Quantitation Interpretation
12
MGF file format Input data Peptide assignment Validation Protein inference Quantitation Interpretation
13
.mzXML Input data Peptide assignment Validation Protein inference Quantitation Interpretation
14
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 1.2 Scores: 1.2 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
15
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 1. 2 2.1 Scores: 1. 2 2.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
16
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 3.4 1.2 2.1 Scores: 3.4 1.2 2.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
17
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 3.4 1.2 2.1 4.1 Scores: 3.4 1.2 2.1 4.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
18
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 3.4 1.2 2.1 4.1 5.1 Scores: 3.4 1.2 2.1 4.1 5.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
19
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 3.4 1.2 2.2 2.1 4.1 5.1 Scores: 3.4 1.2 2.2 2.1 4.1 5.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
20
>IPI:IPI00000045.1|SWISS-PROT:P18510-1 MEICRGLRSHLITLLLFLFHSETICRPSGRKSSKMQAFRIWDVNQKTFYLRNNQLVAGYLQGPNVN LEEKIDVVPIEPHALFLGIHGGKMCLSCVKSGDETRLQLEAVNITDLSENRKQDKRFAFIRSDSGPTT SFESAACPGWFLCTAMEADQPVSLTNMPDEGVMVTKFYFQEDE Input data Experimental Spectra Scores: 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Scores: 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
21
>IPI:IPI00000045.1|SWISS-PROT:P18510-1 MEICRGLRSHLITLLLFLFHSETICRPSGRKSSKMQAFRIWDVNQKTFYLRNNQLVAGYLQGPNVN LEEKIDVVPIEPHALFLGIHGGKMCLSCVKSGDETRLQLEAVNITDLSENRKQDKRFAFIRSDSGPTT SFESAACPGWFLCTAMEADQPVSLTNMPDEGVMVTKFYFQEDE Input data Experimental Spectra Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
22
Input data Experimental Spectra Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Protein sequence DB Score: 32 Peptide: SHLITLLLFLFHSETICR
23
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
24
Input data Experimental Spectra Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR
25
Input data Experimental Spectra Scores: 11.3 6.3 9.3 3.3 1.3 4.2 7.2 13.2 1.1 10.1 Scores: 11.3 6.3 9.3 3.3 1.3 4.2 7.2 13.2 1.1 10.1 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR
26
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Input data Peptide assignment Validation Protein inference Quantitation Interpretation Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK
27
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 1.2 Scores: 1.2 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB
28
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 1.2 Scores: 1.2 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB 1.
29
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 1.2 Scores: 1.2 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: Protein sequence DB 1. 2.
30
Shared Peak Count (SPC) This is the number of the peaks in the theoretical spectrum that are matched to peaks in the experimental spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0
31
Shared Peak Count (SPC) This is the number of the peaks in the theoretical spectrum that are matched to peaks in the experimental spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 SPC = 7
32
Inner product (I) This is the sum of the intensities of the peaks in the experimental spectrum that match to peaks in the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0
33
Inner product (I) This is the sum of the intensities of the peaks in the experimental spectrum that match to peaks in the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 I = 3.5
34
Hyperscore: H = I*N b !*N y ! I is the sum of the intensity of the matched peaks N b, (resp. N y ) is the number of the matched b (resp. y ) peaks in the theoretical spectrum ! is the factorial function. Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 bbbb b yyyy y
35
Hyperscore: H = I*N b !*N y ! - I is the sum of the intensity of the matched peaks - N b, (resp. N y ) is the number of the matched b (resp. y ) peaks in the theoretical spectrum - ! is the factorial function. Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 bbbb b yyyy y H = 3.2*3!*4! = 3.2*6*24 = 460.8
36
Xcorr q is the query spectrum t is the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0
37
Xcorr q is the query spectrum t is the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 I(q,t)=3.2
38
Xcorr q is the query spectrum t is the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 I(q,t)=3.2 I(q,t[-75])=
39
Xcorr q is the query spectrum t is the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 I(q,t)=3.2 I(q,t[-32])=
40
Xcorr q is the query spectrum t is the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 I(q,t)=3.2 I(q,t[0])=
41
Xcorr q is the query spectrum t is the theoretical spectrum Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison: 1. 100% 0% 1 0 I(q,t)=3.2 I(q,t[32])= And so on.
42
Protein Sequence Databases – Completeness: Complete Longer searching time – Redundancy: Sequence variations can be found Redundant database can mess up the statistics – Quality of sequence annotation Protein sequence DB 2. Input data Peptide assignment Validation Protein inference Quantitation Interpretation
43
Entrez Protein DB – http://www.ncbi.nlm.nih.gov/sites/entrez?db=protein http://www.ncbi.nlm.nih.gov/sites/entrez?db=protein – Most complete, redundant Reference Sequence (RefSeq) and UniProt (Swiss-Prot and TrEMBL) – http://www.ncbi.nlm.nih.gov/RefSeq/ http://www.ncbi.nlm.nih.gov/RefSeq/ – http://www.uniprot.org/ http://www.uniprot.org/ – Well annotated, non-redundant International Protein Index (IPI) – http://www.ebi.ac.uk/IPI/IPIhelp.html http://www.ebi.ac.uk/IPI/IPIhelp.html – Represents a good balance between redundancy and completeness. – Contains cross-reference to Ensemble, UniProt, RefSeq. Sequences from a single genome – Difficult to obtain good statistics on small datasats. Protein sequence DB 2. Input data Peptide assignment Validation Protein inference Quantitation Interpretation
44
Protein sequence DB 2. Input data Peptide assignment Validation Protein inference Quantitation Interpretation Taxonomy Allows searches to be limited to entries from particular species or groups of species. Speed up a search, and ensures that the hit list will only contain entries from the selected species. For non-redundant databases, a single entry may represent identical sequences from multiple species. The accession string and title text from the FASTA entry, listed on the master results page, will usually describe just one of these entries. To see the equivalent entries, and to explore their taxonomy, follow the accession number link in the results list to the Protein View. If the hit is from a non-redundant database, and represents multiple entries with identical sequences, the Protein View will include links to NCBI Entrez and the NCBI Taxonomy Browser for all equivalent entries.
45
Run time Database search has to enumerate all peptides and compare them to all experimental spectra. This can be slow with large protein sequence databases especially when slow scoring function is applied, like Xcorr. Input data Peptide assignment Validation Protein inference Quantitation Interpretation
46
Speedup techniques Fast database indexing – Fast implementation of sequence indexing in the database Parent mass check – PTMs can be lost Sequest’s preliminary score Tag-based filtering (de novo hybrid) – Increases the specificity(or sensitivity) Input data Peptide assignment Validation Protein inference Quantitation Interpretation
47
Advanced database indexing – Better implementation of the sequence indexing – Better representation of protein sequences. Input data Peptide assignment Validation Protein inference Quantitation Interpretation
48
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: 1.2 Scores: 1.2 Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison Protein sequence DB Parent mass check
49
>IPI:IPI00000044.1|SWISS-PROT:P01127 MNRCWALFLSLCCYLRLVSAEGDPIPEELYEMLSDHSIRSFDDLQRLLHGDPGEEDKAELDLNMTR SHSGGELESLARGRRSLGSLTIAEPAMIAECKTRTEVFEISRRLIDRTNANFLVWPPCVEVQRCSGC CNNRNVQCRPTQVQLRPVQVRKIEIVRKKPIFKKATVTLEDHLACKCETVAAARPVTRSPGGSQE QRAKTPQTRVTIRTVRVRRPPKGKHRKFKHTHDKTALKETLGA Input data Experimental Spectra Scores: Input data Peptide assignment Validation Protein inference Quantitation Interpretation Spectra comparison Protein sequence DB Parent mass check
50
Fast prescoring (used in SEQUEST) So called Sp score: R(q,t) is the maximum number of consecutive matched b-y ions. Input data Peptide assignment Validation Protein inference Quantitation Interpretation 100% 0% 1 0 S p =3.2*7*(1+0.0075*4)/10=2.3072 SEQUEST selects the top 500 scoring peptides, scored by S p, and rescores them using the Xcorr.
51
Sequence tag based filtering Extract short amino acid tags from the experimental spectra, Using spectrum graph, where nodes are the peaks, masses which differ by the mass of an amino acid are linked by an edge. Input data Peptide assignment Validation Protein inference Quantitation Interpretation
52
W R A C V G E K D W Q P T L T Input data Peptide assignment Validation Protein inference Quantitation Interpretation
53
W R A C V G E K D W L P T L T TAG Prefix Mass AVG 0.0 WTD 120.2 PET 211.4 Generates short peptide sequence tags from the spectrum, and uses these tags to filter the protein sequence database. Tags make database search much faster, analogous to the way that BLAST’s filter speeds up sequence search. Input data Peptide assignment Validation Protein inference Quantitation Interpretation
54
Tag-based filtering MDHPEDESHSEK QDDEEALARLEEIK SIEAKLTLR QNNLNPERPDSAYLR LKQINEEQREGLR FVSEAVTAICEAK SSDIQAAVQICSLLHQR EFSASLTQGLLK SAEDLEADK MDHPEDESHSEK QDD EEA LARLEEIK SIEAKLTLR QNNLNPERPDSAYLR LKQIN EEQ REGLR FVSEAVTAICEAK SSDIQAAVQICSLLHQR EFSAS LTQ GLLK SAEDLEADK Input data Peptide assignment Validation Protein inference Quantitation Interpretation
55
Summary Experimental spectra are compared to protein sequence database. Scoring function, Protein Database, Speedup techniques, Input data Peptide assignment Validation Protein inference Quantitation Interpretation
56
Validation Input data Peptide assignment Validation Protein inference Quantitation Interpretation
57
Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation
58
Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation
59
Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation
60
Scores: 11.3 6.3 9.3 3.3 1.3 4.2 7.2 13.2 1.1 10.1 Scores: 11.3 6.3 9.3 3.3 1.3 4.2 7.2 13.2 1.1 10.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation
61
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation
62
Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation How can peptide assignments be approved or rejected automatically? Why is it necessary?
63
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Human judgment is biased and can be unreliable, Millions of spectra per day, Very difficult by looking at the spectrum visually. Why is it necessary to do it automatically?
64
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Two computational approaches: Relative score probability based scoring
65
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Relative score: SEQUEST: delta score
66
Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Scores: 15.32 3.4 14.3 1.2 2.2 7.2 2.1 4.1 9.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875
67
Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.4 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0
68
Scores: 11.3 6.3 9.3 3.3 1.3 4.2 7.2 13.2 1.1 10.1 Scores: 11.3 6.3 9.3 3.3 1.3 4.2 7.2 13.2 1.1 10.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0
69
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0 Cn=(15-4)/15=0.733
70
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0 Cn=(15-4)/15=0.733 Keep the peptide assignment that exceeds a certain limit.
71
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0 Cn=(15-4)/15=0.733 Keep the peptide assignment that exceeds a certain limit.
72
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0 Cn=(15-4)/15=0.733 Keep the peptide assignment that exceeds a certain limit.
73
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0 Cn=(15-4)/15=0.733 Keep the peptide assignment that exceeds a certain limit.
74
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0 Cn=(15-4)/15=0.733 Keep the peptide assignment that exceeds a certain limit.
75
Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Input data Experimental Spectra Protein sequence DB Score: 4 Peptide: AELDLNMTR Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 3 Peptide: MEICRGLR Score: 15 Peptide: LLHGDPGEEDK Score: 4 Peptide: MDHPEDESHSEK Score: 5 Peptide: SAEDLEADK Score: 3 Peptide: SIEAKLTLR Input data Peptide assignment Validation Protein inference Quantitation Interpretation Cn=(32-4)/32=0.875 Cn=(4-4)/4=0 Cn=(3-3)/3=0 Cn=(15-4)/15=0.733 Keep the peptide assignment that exceeds a certain limit.
76
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Probability based peptide assignment validation: Compute the statistical significance of the score. The statistical significance of a score s is the probability of observing a random score x that is higher or equal that the score s, formally P(s <= x). This probability is called the p-value. 3 approaches: 1. using analytical functions, 2. Fitting a distribution of the sample of random scores. 3. non-parametric approach. Compute the probability that the peptide assignment with the corresponding score is correct.
77
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Probability based peptide assignment validation: The probability based approach means, very loosely speaking, how far the score is from the random.
78
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Probability based peptide assignment validation: Random score is a score obtained by a comparison between a randomly selected experimental and a randomly selected theoretical spectrum. This random score has a probability density distribution, and it depends on the scoring functions. As a null hypothesis.
79
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Probability based peptide assignment validation: The distribution depends on the scoring function. Random matches caused by match with noise
80
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Probability based peptide assignment validation: 1. Analytical function. Depends on the scoring function. And the parameters are calculated from the spectra to be compared. 1. In the case of the SPC scoring function, the distribution of the random scores can be modeled with hyper geometrical distribution. 2. In the case of the inner product scoring function, the random scores can be modeled with normal distirbution.
81
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Probability based approach: Build a histogram of the scores that were obtained during the comparison. Fit a known distribution function, and use this for calculation of the p-value of the top score.
82
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Probability based approach: Decoy approach. Make a dummy dataset, big enough to obtain solid statistics. Decoy dataset can be made by: 1.random shuffling 2.Markov-chain generated amino acid sequences 3.more typically, by simply reversing the sequence of proteins in the database. Sometimes it is called reverse database. No correct matches are expected from the decoy dataset, so the scores obtained on Decoy dataset are used for excellent estimate of random distribution.
83
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Spectra comparison: Protein sequence DB Input data Experimental Spectra >IPI:IPI00000045.1|SWISS-PROT:P18510-1 MEICRGLRSHLITLLLFLFHSETICRPSGRKSSKMQAFRIWDVNQKTFYLRNNQLVAGYLQGPNVN LEEKIDVVPIEPHALFLGIHGGKMCLSCVKSGDETRLQLEAVNITDLSENRKQDKRFAFIRSDSGPTT SFESAACPGWFLCTAMEADQPVSLTNMPDEGVMVTKFYFQEDE Decoy Protein sequence DB
84
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Spectra comparison: Protein sequence DB Input data Experimental Spectra >Decoy_protein_sequence_1 EDEQFYFKTVMVGEDPMNTRLSVPQDAEMATCLFWGPCAASEFSTTPGSDSRIFAFRKDQKRNE SLDTINVAELQLRTEDGSKVCSLCMKGGHIGLFLAHPEIPVVDIKEELNVNPGQLYGAVLQNNRLYF TKQNVDWIRFAQMKSSKRGSPRCITESHFLFLLLTILHSRLGRCIEM Decoy Protein sequence DB Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1
85
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Protein sequence DB Input data Experimental Spectra Decoy Protein sequence DB Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 Can provide more accurate random distribution model. Doubles the execution time. Frequently applied approach!
86
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Protein sequence DB Input data Experimental Spectra Decoy Protein sequence DB Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 Non-parametric approach. Instead of fitting probability density function to the histogram: Calculate the percentage of the scores on the decoy dataset, equal or higher score than the actual top score.
87
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Protein sequence DB Decoy Protein sequence DB Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 False Positive Rate (FPR), the probability of labelling a random score significant (area B in the figure). A FPR of 0.01 means that 1% of the random scores are labelled significant. E-value: The E-value of a query is the expected number for finding a database element with random score greater than or equal to the query hit s on a database of n data. For instance, an E-value of 10 -2 means that the score h is expected to occur by chance only once in 100 independent similarity searches over the database. If the E-value is 10, then ten random hits with score greater or equal to h are expected within a single similarity search.
88
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Scores: 13.15 6.4 1.4 9.3 4.3 3.2 7.2 11.2 8.1 10.1 2.1 5.1 12.1 Protein sequence DB Decoy Protein sequence DB Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 Decoy Scores: 5.4 3.4 4.4 10.3 8.3 7.3 2.2 6.2 1.2 12.1 9.1 11.1 False Discovery Rate, the ratio of random scores within significant scores, formally FDR= A /( A + B). The FDR = 0.01 means the 1% of the scores labelled significant are actually observed by chance. FDR is often used to control the ratio of the false positives. The threshold T can be set to keep the FDR under a certain level, typical levels are 0.01 or 0.05, i.e experimenters set thresholds to allow 1% or 5% of false positives. The lower the FDR the more true (non-random) similarity hits are lost. Decoy dataset is used to calculate the FDR.
89
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Summary: 1.Peptide assignment has to be validated. 2.Relative scoring or probability based scoring can be applied. 3.False positives (false assignments) can be kept under a certain level.
90
Protein Inference Input data Peptide assignment Validation Protein inference Quantitation Interpretation
91
Take the peptides that passed the validation. This section is about to infer the proteins that could produces these peptides. The task is not trivial. Input data Peptide assignment Validation Protein inference Quantitation Interpretation Input data Experimental Spectra Score: 32 Peptide: SHLITLLLFLFHSETICR Score: 15 Peptide: LLHGDPGEEDK
92
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Peptides: MDHPEDESHSEK QDDEEALARLEEIK SIETLR QNNLNPERPDSAYLR LKQINEEQREGLR FVSEAVTAICEAK SSDIQAAVQICSLLHQR EFSASLTQGLLK SAEDLEADK Proteins:
93
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Peptides: MDHPEDESHSEK QDDEEALARLEEIK SIETLR QNNLNPERPDSAYLR LKQINEEQREGLR FVSEAVTAICEAK SSDIQAAVQICSLLHQR EFSASLTQGLLK SAEDLEADK Proteins:
94
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Peptides: MDHPEDESHSEK QDDEEALARLEEIK SIETLR QNNLNPERPDSAYLR LKQINEEQREGLR FVSEAVTAICEAK SSDIQAAVQICSLLHQR EFSASLTQGLLK SAEDLEADK Proteins:
95
Input data Peptide assignment Validation Protein inference Quantitation Interpretation By Occam’s razor, the Protein A should be preferred. Protein A, B ad C can be homologous proteins
96
Input data Peptide assignment Validation Protein inference Quantitation Interpretation Many models have been develop to cope with to this problem. Statistical based, Graph theory and spectral Network based. Well-known method ProteinProphet.
97
Summary Input data Peptide identification Validation Protein inference Quantitation Interpretation Data formatsDatabase searching Statistical methods for validations Protein assembling
98
Database Searching – Simple and straightforward – Has a limited search space. – Completeness – Statistical analysis can be carried out. – Has a limited search space. Limited to the database. – Enumerating all candidates is too slow, particularly when modifications and non-tryptic peptides must be considered. (A modern instrument produces million spectra per day) Input data Peptide assignment Validation Protein inference Quantitation Interpretation
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.