1. Bioinformatics Programming 1 EE, NCKU Tien-Hao Chang (Darby Chang)

2. Regular Expression 2

3. http://rp1.monday.vip.tw1.yahoo.net/res/gdsale/st_pic/0469/st-469571-1.jpg 3

4. Text patterns and matches A regular expression, or regex for short, is a pattern describing a certain amount of text In this slide, regular expressions are highlighted as regex –it is the most basic pattern, simply matching the literal text regex (highlighted in this slide) I will use the term “string” to indicate the text that I am applying the regular expression to and will be highlighted as string 4

5. Literal characters The most basic regular expression consists of a single literal character, ex: a –match the first occurrence of that character in the string –on Jack is a boy Jack is a boy, not Jack is a boy In this slide, I’ll use a shorter notation sometimes –a: Jack is a boy Eleven characters with special meanings: –[ \ ^ $. | ? * + ( )–[ \ ^ $. | ? * + ( ) –metacharacters –escape metacharacters with a backslash use 1\+1=2 to match 1+1=2 5

6. Character classes/sets Match only one out of several characters –to match an a or an e, use [ae] –you could use this in gr[ae]y to match gray or grey –a character class matches only a single character –gr[ae]y will not match graay or graey –the order does not matter Use a hyphen to specify a range of characters –[0-9] matches a single digit between 0 and 9 –combine ranges and single characters [0-9a-fA-F] –combine ranges and single characters [0-9a-fxA-FX] A caret after the opening square bracket negates the class –q[^x] matches qu in question but does not match Iraq since there is no character after the q for the negated character class to match 6

7. Shorthand character classes \d matches a single character that is a digit \w matches a word character –alphanumeric characters plus underscore \s matches a whitespace character –includes tabs and line breaks –\S not \s The actual characters matched by the shorthands depends on the software you’re using –$ man perlre 7

8. Non-printable characters Use special character sequences to put non-printable characters –\t for tab (ASCII 0x09) –\r for carriage return (0x0D) –\n for line feed (0x0A) Remember that Windows text files use \r\n to terminate lines, while UNIX text files use \n Use \xFF to match a specify character by its hexadecimal index in the character set –\xA9 matches the copyright symbol \uFFFF for a Unicode character (if supported) –\u20A0 matches the euro currency sign 8

9. The dot The dot,., matches (almost) any character The dot matches a single character, except line break characters –a short for [^\n] –gr.y matches gray, grey, gr%y, etc Most regex engines have a “dot matches all” or “single line” mode that makes the dot match any single character, including line breaks 9

10. Anchors Anchors do not match any characters but match a position –^ matches at the start of the string –$ matches at the end of the string Most regex engines have a “multi-line” mode that makes ^ match after any line break, and $ before any line break –b$ matches only bob \b matches at a word boundary –a word boundary is a position between a character that can be matched by \w and a character that cannot be matched by \w –\b also matches at the start and/or end of the string if the first and/or last characters in the string are word characters –\B matches at every position where \b cannot match –\bis\b: This island is beautiful 10

11. Alternation Alternation is the regular expression equivalent of “or” –cat|dog: About cats and dogs You can add as many alternatives as you want –cat|dog|mouse|fish 11

12. Repetition ? makes the preceding token in the regular expression optional –colou?r matches colour or color * matches the preceding token zero or more times + matches the preceding token once or more – matches an HTML tag without any attributes – is easier to write but matches invalid tags such as {} specifies a specific amount of repetition –\b[1-9][0-9]{3}\b matches 1000–9999 –\b[1-9][0-9]{2,4}\b matches 100–99999 12

13. Greedy and lazy repetition The repetition operators or quantifiers are greedy They will expand the match as far as they can, and only give back if they must to satisfy the remainder of the regex – : This is a first test Place a question mark after the quantifier to make it lazy, i.e., stop matching as soon as possible – : This is a first test A better solution is to use ]+> to quickly match an HTML tag without regard to attributes –the negated character class is more specific than the dot, which helps the regex engine find matches quickly 13

14. Grouping and backreferences Place round brackets, (), around multiple tokens to group them together –you can then apply a quantifier to the group –Set(Value)? matches Set or SetValue Round brackets create a capturing group –the above example has one group –how to access the group’s contents depends on the software or programming language you’re using Group zero always contains the entire regex match –Set(Value)?: SetValue, then $0 = SetValue, $1 = Value –Set(Value)?: Set, then or $0 = Set, $1 is nothing Use the special syntax Set(?:Value)? to group tokens without creating a capturing group –more efficient if you don’t need the contents 14

15. Look-around Look-around is a special kind of group The tokens inside the group are matched normally, but then the regex engine makes the group give up its match and keeps only the result Look-around matches a position, just like anchors –q(?=u) matches question, but not Iraq (?=u) match at each position in the string before a u u is not part of the overall regex match positive look-ahead –q(?!u) matches Iraq but not question negative look-ahead –(?<=a)b matches abc positive look-behind –(?<!a)b fails to match abc negative look-behind 15

16. Reference Regular Expression Quick Start –http://www.regular- expressions.info/quickstart.htmlhttp://www.regular- expressions.info/quickstart.html 16

17. We have done a lot of exercises http://www.pdqmailingservices.com/db5/00409/pdqmailingservices.com/_uimages/MessyDesk.jpg 17

18. Now, let’s Talk about Bioinformatics programming in real cases 18

19. Sequence alignment We have learnt/implemented it twice –dynamic programming –longest common sub-string/sub-sequence –sequence alignment DNA/protein sequence residue substitution We know that –time complexity is O(n2) –backtracking, alternative alignments… That’s all? –No! There are always better algorithms. That’s why we always have new papers to read. Theoretical Applicative 19

20. Sequence alignment Some advanced ideas When is this point never considered? Band alignment Arbitrary region 20

21. 21 Seq = AGATCGAT 12345678 AAA AAC. AGA 1. ATC 3. CGA 5. GAT 2 6 TCG 4. TTT The state-of-the-art solutions: seeding and extension.

22. This is not 22 Bioinformatics algorithm

23. Protein clustering 23 Ina FASTA file and an integer k Outk clusters of proteins Requirement - invoke BLAST - complexity/teamwork report - using Perl would be the best Bonus - k-means algorithm - invoke clustering package

24. Deadline 24 2010/5/4 23:59 Zip your code, step-by-step README, complexity analyses and anything worthy extra credit. Email to darby@ee.ncku.edu.tw. darby@ee.ncku.edu.tw

25. BLAST Download protein sequence from UniProt –$ wget -o ytf.fa 'http://www.uniprot.org/uniprot/?query=Saccharomyces+cerevisiae+transc ription+factor+AND+reviewed%3ayes&force=yes&format=fasta‘http://www.uniprot.org/uniprot/?query=Saccharomyces+cerevisiae+transc ription+factor+AND+reviewed%3ayes&force=yes&format=fasta A Unix tip using grep and regular expression –$ grep '^>' ytf.fa | wc –l # how many sequences –$ grep -c '^>' ytf.fa # a better version Download BLAST from NCBI –http://blast.ncbi.nlm.nih.gov/http://blast.ncbi.nlm.nih.gov/ –I prefer this version ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/LATEST/blast-2.2.23- ia32-linux.tar.gz ftp://ftp.ncbi.nlm.nih.gov/blast/executables/release/LATEST/blast-2.2.23- ia32-linux.tar.gz Execution –$ format db –i ytf.fa # building indices –$ blastall -d ytf.fa -i ytf.fa -p blastp > ytf.bo # default output –$ blastall -d ytf.fa -i ytf.fa -m 6 -p blastp > ytf.bo # tabular output 25