Presentation is loading. Please wait.

Presentation is loading. Please wait.

Biosequence Similarity Search on the Mercury System Praveen Krishnamurthy, Jeremy Buhler, Roger Chamberlain, Mark Franklin, Kwame Gyang, and Joseph Lancaster.

Published byJoanna Price Modified over 9 years ago

Similar presentations

Presentation on theme: "Biosequence Similarity Search on the Mercury System Praveen Krishnamurthy, Jeremy Buhler, Roger Chamberlain, Mark Franklin, Kwame Gyang, and Joseph Lancaster."— Presentation transcript:

1 Biosequence Similarity Search on the Mercury System Praveen Krishnamurthy, Jeremy Buhler, Roger Chamberlain, Mark Franklin, Kwame Gyang, and Joseph Lancaster Department of Computer Science and Engineering, Washington University in Saint Louis, MO Supported by an NIH STTR Grant & NSF Grants DBI-0237902, ITR-0313203, CCR-0217334

2 Slide # 2Washington University in St. Louis Outline Overview of BLAST Overview of the Mercury system Description of BLASTN algorithm Algorithmic changes to BLASTN Improvement in performance Related work Conclusion

3 Slide # 3Washington University in St. Louis Basic Local Alignment Search Tool Biosequence comparison software –Query sequence (new genome) to large database of known biosequences Look for similar regions Exponential growth of genomic databases –Longer time for searches to complete –Solutions Perform comparison over multiple machines Specialized hardware - Our Approach

4 Slide # 4Washington University in St. Louis The Mercury System

5 Slide # 5Washington University in St. Louis The Mercury System Proximity to disk –Simple operations performed close to disk Avoids CPU use –400 Mbytes/s throughput from the disk Concurrent Independent operation –Does not use processor cache cycles, memory or I/O buses Reconfigurable logic –Logic can be tuned to the particular need of the application

6 Slide # 6Washington University in St. Louis BLASTN –Both the query and the database are long DNA strings –Consist of {A, C, T, G} and some unknowns Each stage processes lesser data The stages become more computationally expensive

7 Slide # 7Washington University in St. Louis BLASTN - Terminology … ACTGTGTTTCACTGACGGGTGT … … CTGTGTCCCCAACACTGCTGACGTAGAATCGTGTAG … Query Database ‘w-mer’ is a sequence of ‘w’ consecutive bases

8 Slide # 8Washington University in St. Louis BLASTN - Pipeline - Stage 1 Matches each ‘11-mer’ in query to database –Exact string matching 83% of overall time is spent in this stage Filters 92% of data entering this stage –Only 8% of data proceeds to the next stage

9 Slide # 9Washington University in St. Louis BLASTN - Pipeline - Stage 2 Extends the matches from stage 1 … ACTGTGTTTCACTGACGGGTGT … … GTGTCCCCAACATTTCACTGACGAGAATCGTGTAG …

10 Slide # 10Washington University in St. Louis BLASTN - Pipeline - Stage 2 Extends the matches from stage 1 –Allows mismatches of individual bases –Does not allow gaps in either the query or the database –Match score should be higher than threshold to proceed 16% of pipeline time is spent is this stage Only 2/100,000 of data entering this stage proceeds to the next stage

11 Slide # 11Washington University in St. Louis BLASTN - Pipeline - Stage 3 Extends the matches from stage 2 … ACCACTGTTTCACTGACG_GA_T_GT … … CTGTGTCCCCAC_GTTTCACTGACGAGAATCGTGTAG …

12 Slide # 12Washington University in St. Louis BLASTN - Pipeline - Stage 3 Extends the matches from stage 2 –Scores matches with Gaps inserted in both the sequences –Smith-Waterman dynamic programming algorithm <1% of pipeline time is spent is this stage

13 Slide # 13Washington University in St. Louis NCBI - BLASTN Stage 1 (word matching) is implemented as a lookup table –Efficient only for certain word lengths (w= 11) Performance degrades dramatically for larger query sizes Query Size 10 Kbases 100 Kbases 1 MbasesUnits Throughput66.98.760.657Mbases/s Pentium-4 2.6GHz 1Gbyte RAM

14 Slide # 14Washington University in St. Louis Firmware implementation - Stage 1 Bloom Filters Hash Lookup Redundancy Eliminator Eliminates false-positives from Bloom filters, obtain offset in query Discards matches that are close to one another Matches ‘11-mers’ to query, but generates false-positives

15 Slide # 15Washington University in St. Louis Bloom filters operation ‘11-mer’ K Hash Functions Programming the query into the bloom filter (processing query) ‘m-bit’ vector 1 1 1 query

16 Slide # 16Washington University in St. Louis Bloom filters operation database K Hash Functions Finding matches in the database ‘m-bit’ vector ? ? ? 1: Potential match 0: Not a match ‘11-mer’

17 Slide # 17Washington University in St. Louis Bloom filters operation K Hash Functions Finding matches in the database ‘m-bit’ vector ? ? ? 1*: Potential match 0: Not a match * False positives are eliminated using a hash table database ‘11-mer’

18 Slide # 18Washington University in St. Louis Bloom filter performance

19 Slide # 19Washington University in St. Louis Performance analysis Firmware Vs. Software Stage 1 Query Size 10 Kbases 100 Kbases 1 Mbases Units NCBI BLASTN 77.410.50.771Mbases/s Mercury BLASTN 800 80Mbases/s Speedup10.376.1104

20 Slide # 20Washington University in St. Louis Overall system throughput Query Size 10 Kbases 100 Kbases 1 MbasesUnits NCBI BLASTN 67.08.760.657Mbases/s Mercury BLASTN 49752.64.47Mbases/s Speedup7.426.016.80 Tput overall = min (Tput 1, Tput (2&3) )

21 Slide # 21Washington University in St. Louis Stage 2 in firmware - Throughput

22 Slide # 22Washington University in St. Louis Stage 2 in firmware - Speedup

23 Slide # 23Washington University in St. Louis Related work Hardware based commercial systems –Paracel GeneMatcher TM, used ASIC, and hence is inflexible –RDisk, FPGA based system with throughput of 60 Mbases/s for stage 1 High-end commercial system –Paracel BLASTMachine2 TM, 32 CPU linux cluster 2.93 Mbases/s for 2.8 Mbase query 2 times faster than 1-node Mercury BLASTN –TimeLogic DeCypherBLAST TM, FPGA based 213 Kbases/s for a 16 Mbase query Comparable to 1-node Mercury BLASTN

24 Slide # 24Washington University in St. Louis Conclusion BLASTN on the Mercury system –Bloom filters to improve performance of stage 1 Efficient hash functions in hardware –7x improvement in speed with only stage 1 firmware –>50x speedup with stage 2 implemented in firmware Future work –Algorithmic changes to stage 2 Efficient use of hardware capabilities –Other apps BLASTP, BLASTX etc.

25 Slide # 25Washington University in St. Louis Thank you

Download ppt "Biosequence Similarity Search on the Mercury System Praveen Krishnamurthy, Jeremy Buhler, Roger Chamberlain, Mark Franklin, Kwame Gyang, and Joseph Lancaster."

Similar presentations

SE-292 High Performance Computing

SE-292 High Performance Computing
Indexing DNA Sequences Using q-Grams

Indexing DNA Sequences Using q-Grams
SE-292 High Performance Computing Memory Hierarchy R. Govindarajan

SE-292 High Performance Computing Memory Hierarchy R. Govindarajan
A Scalable and Reconfigurable Search Memory Substrate for High Throughput Packet Processing Sangyeun Cho and Rami Melhem Dept. of Computer Science University.

A Scalable and Reconfigurable Search Memory Substrate for High Throughput Packet Processing Sangyeun Cho and Rami Melhem Dept. of Computer Science University.
Alignment methods Introduction to global and local sequence alignment methods Global : Needleman-Wunch Local : Smith-Waterman Database Search BLAST FASTA.

Alignment methods Introduction to global and local sequence alignment methods Global : Needleman-Wunch Local : Smith-Waterman Database Search BLAST FASTA.
Improved Alignment of Protein Sequences Based on Common Parts David Hoksza Charles University in Prague Department of Software Engineering Czech Republic.

Improved Alignment of Protein Sequences Based on Common Parts David Hoksza Charles University in Prague Department of Software Engineering Czech Republic.
BLAST Sequence alignment, E-value & Extreme value distribution.

BLAST Sequence alignment, E-value & Extreme value distribution.
Multithreaded FPGA Acceleration of DNA Sequence Mapping Edward Fernandez, Walid Najjar, Stefano Lonardi, Jason Villarreal UC Riverside, Department of Computer.

Multithreaded FPGA Acceleration of DNA Sequence Mapping Edward Fernandez, Walid Najjar, Stefano Lonardi, Jason Villarreal UC Riverside, Department of Computer.
1 ALAE: Accelerating Local Alignment with Affine Gap Exactly in Biosequence Databases Xiaochun Yang, Honglei Liu, Bin Wang Northeastern University, China.

1 ALAE: Accelerating Local Alignment with Affine Gap Exactly in Biosequence Databases Xiaochun Yang, Honglei Liu, Bin Wang Northeastern University, China.
Bioinformatics Unit 1: Data Bases and Alignments Lecture 2: “Homology” Searches and Sequence Alignments.

Bioinformatics Unit 1: Data Bases and Alignments Lecture 2: “Homology” Searches and Sequence Alignments.
Local alignments Seq X: Seq Y:. Local alignment  What’s local? –Allow only parts of the sequence to match –Results in High Scoring Segments –Locally.

Local alignments Seq X: Seq Y:. Local alignment  What’s local? –Allow only parts of the sequence to match –Results in High Scoring Segments –Locally.
A Grid implementation of the sliding window algorithm for protein similarity searches facilitates whole proteome analysis on continuously updated databases.

A Grid implementation of the sliding window algorithm for protein similarity searches facilitates whole proteome analysis on continuously updated databases.
Extensible Processors. 2 ASIP Gain performance by:  Specialized hardware for the whole application (ASIC). −  Almost no flexibility. −High cost.  Use.

Extensible Processors. 2 ASIP Gain performance by:  Specialized hardware for the whole application (ASIC). −  Almost no flexibility. −High cost.  Use.
1 An Evolution of Pattern Matching within Network Intrusion Detection Systems Erik Anderson 9 November 2006.

1 An Evolution of Pattern Matching within Network Intrusion Detection Systems Erik Anderson 9 November 2006.
Designing Multiple Simultaneous Seeds for DNA Similarity Search Yanni Sun, Jeremy Buhler Washington University in Saint Louis.

Designing Multiple Simultaneous Seeds for DNA Similarity Search Yanni Sun, Jeremy Buhler Washington University in Saint Louis.
Database searching. Purposes of similarity search Function prediction by homology (in silico annotation) Function prediction by homology (in silico annotation)

Database searching. Purposes of similarity search Function prediction by homology (in silico annotation) Function prediction by homology (in silico annotation)
Overview of sequence database searching techniques and multiple alignment May 1, 2001 Quiz on May 3-Dynamic programming- Needleman-Wunsch method Learning.

Overview of sequence database searching techniques and multiple alignment May 1, 2001 Quiz on May 3-Dynamic programming- Needleman-Wunsch method Learning.
Sequence Alignment Bioinformatics. Sequence Comparison Problem: Given two sequences S & T, are S and T similar? Need to establish some notion of similarity.

Sequence Alignment Bioinformatics. Sequence Comparison Problem: Given two sequences S & T, are S and T similar? Need to establish some notion of similarity.
Pairwise Sequence Alignment Part 2. Outline Global alignments-continuation Local versus Global BLAST algorithms Evaluating significance of alignments.

Pairwise Sequence Alignment Part 2. Outline Global alignments-continuation Local versus Global BLAST algorithms Evaluating significance of alignments.
Cluster Computer For Bioinformatics Applications Nile University, Bioinformatics Group. Hisham Adel 2008.

Cluster Computer For Bioinformatics Applications Nile University, Bioinformatics Group. Hisham Adel 2008.

Similar presentations

Ads by Google