Recuperació de la informació Modern Information Retrieval (1999) Ricardo-Baeza Yates and Berthier Ribeiro-Neto Flexible Pattern Matching in Strings (2002) Gonzalo Navarro and Mathieu Raffinot Algorithms on strings (2001) M. Crochemore, C. Hancart and T. Lecroq

String Matching String matching: definition of the problem (text,pattern) depends on what we have: text or patterns Exact matching: Approximate matching: 1 pattern ---> The algorithm depends on |p| and |  | k patterns ---> The algorithm depends on k, |p| and |  | The text ----> Data structure for the text (suffix tree,...) The patterns ---> Data structures for the patterns Dynamic programming Sequence alignment (pairwise and multiple) Extensions Regular Expressions Probabilistic search: Sequence assembly: hash algorithm Hidden Markov Models

Regular expression A regular expression ℛ is a string on the set of simbols Σ U { ε, |, ·, *, (, ) } which is recursively defined as: ε (empty character) is a regular expression A character of Σ is a regular expression ( ℛ ) is a regular expression ℛ 1 · ℛ 2 is a regular expression ℛ * is a regular expression ℛ 1 | ℛ 2 is a regular expression

Regular lenguage The lenguage defined by a regular expression ℛ is the set of strings generated by ℛ. The problem of searching for a regular expression in the text T is to find all the factors in T that belong to the lenguage.

Methods Regular expression NFA Strings found DFA Search with deterministic finit automata Search with bit-parallel Thompson automata Parse tree

Methods Regular expression NFA Strings found Search with bit-parallel Thompson automata Parse tree DFA Search with deterministic finit automata

Search with a deterministic finit automata Given the regular expression bb*(b|b*a) the NFA is As it’s not possible to spell the text out the NFA, the NFA is transformed into a DFA … And the search process… What is the cost? b 1 0 b b a 3 2 b b 1 0 b a a 3 12

Search example with DFA Given the regular expression bb*(b|b*a) and the NFA: The search on the text:b b b a a b a a b b … b b 1 0 b a a 3 12 …

Methods Regular expression NFA Strings found DFA Search with deterministic finit automata Parse tree Search with bit-parallel Thompson automata

Parse tree Is a tree such that: - internal nodes are labeled by operators - leaves are labeled by characters of Σ and ε ( ℛ ) ℛ 1 · ℛ 2 ℛ * ℛ * ℛ 1 | ℛ 2 ℛ. ℛ 1 ℛ 2 | ℛ *

Parse tree: example Given the regular expression bb*(b|b*a) the parse tree is: a b* b. | b. * b

NFA (Thompson automaton) From the regular expression or from the parse tree we define the automaton: For a character a of Σ: a. ℛ 1 ℛ 2 ℛ * | ε ε ε ε ε ε ε

Thompsom automaton construction b a b* b | b. * b bb*(b|b*a) b a b b.

NFA: ε-closure (states ε-equivalents) a b b b b ε bb*(b|b*a) 1, 2, 4, 5, 6, 8, 10 5, 6, 8 9, 12 6, 7, 8 4, 5, 6, 8, 10 2, 3, 4, 5, 6, 8,10 11, 12

Bit-parallel Thompsom algorithm bb*(b|b*a) ε 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 a b b b b B a b Text: ababbbaab The bit-vector D mark the active states: at the begining D At every step we shift to the right followed by an “and” operator with the mask of the last read character… D (a) …and the ε-closure extension of active states. -> The masks are

Bit-parallel Thompsom algorithm ε 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (a) D >

Bit-parallel Thompsom algorithm ε 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (a) (b) D >

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab ->

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (a)

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (a) D

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (a) > D

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (a) > (b) D

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (a) > (b) D

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab ->

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (b)

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (b) D

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (b) > D

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (b) > (b) D

Bit-parallel Thompsom algorithm E 1 1, 2, 4, 5, 6, 8, , 3, 4, 5, 6, 8,10 4 4, 5, 6, 8, , 6, 8 7 6, 7, 8 9 9, , 12 bb*(b|b*a) a b b b b B a b D Text: ababbbaab -> (b) > (b) D