1. Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations
Shuhei Denzumi1, Ryo Yoshinaka2, 1, Shin-ichi Minato1,2, and Hiroki Arimura1
1) Hokkaido University 2) JST ERATO Minato Discrete Structure Manipulation System Project
My name is Shuhei Denzumi.

2. Background Researches on string processing become active. Data mining
Massive online data: The internet and sensing networks.
String matching and string mining problems.
Data mining
Input data should be represented in compact form
Computation under compressed structure is needed
Input
Back ground.
Researches on string processing become more active recently.
The internet and sensing network provide massive online data every day.
String matching and string mining are required to handle these data.
Then, efficient data structures are needed to solve these problems.
We use data structures to represent data compactly and execute computation under compressed form.
Data Structure
Result
Input
Compress
Operation
Input
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

3. Manipulatable & Compact
Manipulatable Compact data structure
Represent data in compressed form
Have operations to manipulate data in compacted style
Get much attention for recent years
Binary Decision Diagram (BDD)
LSI area
Deterministic Finite Automata (DFA)
Natural Language Processing area
Data Structure
Input
Compaction
D 1
In many types of data structures, we forcus manipulatable compact datastructure.
This data structure not only represent data in compact form, but also have efficient method to compute operation between multiple different structures.
For instance of manipulatable compact data structure, Binary Decision Diagram and Deterministic Finite Automata exist.
Binary Decision Diagram, BDD, is widely used in LSI designing area.
DFA is applied to natural language processing.
Operation
D 3
D 2
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

4. Sequence Binary Decision Diagram
What is Sequence BDD?
Sequence Binary Decision Diagram (SeqBDD, SDD).
Loekito, Bailey, and Pei (2009)
Graph structure
Represent finite sets of strings with finite length
SDD’s basic properties are unknown
Minimization
Size complexity
Operation time
Application
Data mining
Graph mining
Human genome sequencing
Text …
Sequence Binary Decision Diagram
Sequence Binary Decision Diagram is a new manipulatable compact data structure.
It is porposed by Leokito, et.al in 2009.
The SDD represent finite sets of strings with finite length.
The basic properties of SDD are still unknown, because it is new.
We think that SDD is applicable to data mining, enumerating and bioinformatics area.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

5. Family of BDDs Compact representation for discrete structure
With rich algebraic operations
BDD [Bryant 1986]
Boolean functions
xy ∨ yz ∨ zx
￢xyz ∨ x￢yz ∨ xy￢z
ZDD [Minato 1993]
Sets of combinations
{{a}, {b}, {a, b}}
{{a}, {b}, {c}, {a, b, c}}
SDD [Loekito, et.al 2009]
Sets of strings
{a, b, ab, bab, abbab}
{abc, acb, bac, bca}
The SDD is a new member in the BDD family.
Respective members of the family represents and different tyeps of discrete structures.
The BDD, the original of the family, manipulate boolean functions.
The ZDD, manipulates sets of combinations.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

6. Result Relationship to Acyclic Deterministic Finite Automata (ADFA)
Translation from an SDD to an ADFA and vice versa
An SDD is never larger than an ADFA
An SDD can be |Σ| times smaller than an ADFA
Computational complexity of binary set operations
Generalize eight set operations
Tight analysis on time complexity for binary set operation algorithm
Experimental results
SDDs can be smaller than ADFAs
Binary operation time
Our results can be devided into two parts.
First, we study the relationship between SDD and ADFA.
We proved that SDD can be alphabet size times smaller than ADFA.
In second part, we analyzed time complexity of bianry set operations on SDDs.
We also conducted some experiments.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

7. Preliminary
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

8. Definition Σ: alphabet (totally ordered by ≺)
Internal node: , , , , 1/0 - terminal node: /
1/0 - edge: /
SDD: directed acyclic graph
Internal node S, τ(S) ↦ 〈S.lab, S.1, S.0〉
S.lab: label
S.1: 1-child
S.0: 0-child
Ordering rule
N.lab ≺ (N.0).lab a b … z 1
S.0
S.lab S
S.1 1 a b c
Let sigma be an alphabet with total order.
SDD is directed acyclic graph structure.
It composed from internal nodes and 0 and 1 terminal node.
Internal nodes have exactly two edges, 1-edge and 0-edge.
1-edge are denoted by solid line, and 0-edge is denoted by broken line.
SDD internal nodes are repsesented by a tiple tau(S) = <S.lab, S.1, and S>0>
S.Lab is a letter labelling the node.
S.1 is the node pointed by 1-edge by S.
S.0 is the node pointed by 0-edge by S.
We call them as 1-child and 0-child respectively.
Internal nodes follow ordering rule.
The latter labeling a node must be smaller than the letter labeling the 0-child of the node. a b z … ≺
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

9. Semantics L(N): set of strings N represents L( ) = {ε} L( ) = {}
L(N) = N.lab・L(N.1) ∪ L(N.0)
A path from the root to the 1-terminal node represent a string. 1 a b
{ε} {}
{b}
{a, b}
{bb}
{aa, ab, bb} 1 a b
{ε} {}
{b}
{a, b}
{bb}
{aa, ab, bb} 1 a b
{ε} {}
{b}
{a, b}
{bb}
{aa, ab, bb} 1 a b
{ε} {}
{b}
{a, b}
{bb}
{aa, ab, bb} 1
Every SDD node represents a set of strings.
The one terminal node represents the singleto of the empty string.
0-terminal node represents the empty set.
And the set of strings of an internal node is recursively defined by this formula.
The set of strings of an SDD is the one represented by its root node.
For convenience, we identify a node with the SDD
%That has the node as its root.
consisting all the nodes reachable from that node.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

10. Comparison to ADFA a b c a b  accept state  reject state 1 a b c 1 a1 a b
{a, b}
{bb}
{aa, ab, bb}
{aa, ab, bb} a b c
{a, b} a b
{b}
Let’s compare SDD with Acyclic Deterministic Finite Automata.
Roughly speaking, one way think of an SDD as a binarized ADFA.
Here is a node of an ADFA with three outgoing edges labeled with a, b and c,
Which correspond to three nodes of an SDD labeled with a , b and c.
In an SDD,
One checks whether the input symbol is a, and if so one takes the edge labeled with a,
And otherwise checks whether it is b or not, and so on.
Such a sequential process is represented by a series of SDD nodes in this way. b c a
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

11. Reduction process a・{} ∪ L(N.0) = L (N.0) a Suppressiona
Suppression
N.1 ≠ 0-terminal node
In ADFA, removing edges pointing dead state
Merging
τ(N) = τ(N’) ⇒ N = N’
In ADFA, share all equivalent nodes
Theorem
Under these rules, SDD is unique and minimal
Like ADFA’s have unique canonical form
N.0
N.0
N.0 x N
N.1 N’
An SDD can be reduced through this reduction process.
The first rule is suppression rule.
If an internal node points 0-terminal node by 1-edge,
then such node is deleted.
By definition, the node represents the same set of strings with its 0-child.
So, such node is redundant.
L(P) = a・{} ∪ L(Q) = L(Q)
Second rule is merging rule.
If two different node have the same triple, label, 1-child, and 0-child.
Such nodes are merged.
In ADFA, this rule corresponds to merge all equivalent nodes.
After this reduction process, an SDD get canonical form and it is minimal.
Just like ADFAs have the canonical form.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

12. Characteristic
Almost isomorphic to Acyclic Deterministic Finite Automata
BDD/ZDD techniques are applicable
Binary form
Simple recursive algorithm
Easy to implement
Rich collections of operations
Use of hash tables
To share equivalent nodes
To share intermediate computations
BDD/ZDD
ADFA
As I have explaned, SDD is almost isomorphic to ADFA.
However, there are some differences between SDD and ADFA.
As an advantage of SDD,
We can apply BDD/ZDD techniques to SDD.
Binary form allows simple recursive algorithms for SDD.
It makes implementation of SDD be easier.
Rich collection of operations inherited from ZDD can apply to SDD.
Same as other members of BDD family, SDD use hash tables.
SDDs have hash tables to keep SDDs always reduced and to make set operations work efficiently.
SDD
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

13. Relationship to Acyclic Automata
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

14. Size a b c An SDD node correspond to an ADFA edge
The description size is proportional to |N|: the number of internal nodes in SDD N |A|: the number of edges in ADFA A b c a a b c
We define the size of SDD and ADFA.
SDD size is defined by the number of internal nodes.
On the other hand, ADFA size is defined by the number of edges.
Since the description sizes are proportionally to them each other.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

15. Theorem: Size compare For equivalent an SDD and an ADFA
From an ADFA A to an SDD N
From an SDD N to an ADFA A
SDD |Σ| times can be smaller than ADFA
We compare the size of an SDD to that of ADFA representing the same set of strings.
We proved two theorems on size.
First, SDD is never larger than ADFA.
First, an SDD is not larger than the ADFA.
This equality can be
An ADFA can be |Sigma| times larger than the SDD.
In other words, an SDD can be |Sigma| times smaller than the ADFA.
There are cases that holds this equality.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

16. 0-child sharing e c a b d d c e a c d e b
This slide illustrates why an SDD can be smaller than an ADFA.
Suppose that an ADFA has two distinct nodes that have an edge labeled by the same letter that goes to the same node,
like the edges labeled by c, d and e.
In ADFA those edges are not merged, but in an SDD, those three edges turn to be nodes and merged in this way.
We have 8 edges on the right figure, while we have only 5 nodes on the left.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

17. Example a c a b b c c b c c b a b c a
{anbicj, n = 0, …, 4, i, j = 0, 1}
ADFA A
SDD S 1 a b c a c a b b c c b c c b a b c
This slide shows a typical case that SDD size is smaller than ADFA.
Assume that we want to represent 3 sets of strings, {a,b,c}, {b,c}, {c}
SDD can represent the sets by just 3 nodes, but ADFA needs 6 edges.
In SDD, nodes are labeled, but edges are labeled in ADFA.
However, both SDD and ADFA merges nodes.
It causes this difference of size.
Since their edges are labeled.
They cannot be merged, but SDD can merge their nodes.
This is the example that SDD size is asimptically |Sigma| times smaller.
The number of SDD nodes is 6, and the number of ADFA edges is 14. a
|S| = 6
|A| = 14
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

18. Experiment Input: Canterbury corpus
BibleAll: bible.txt, BibleBi: all bigrams from bible.txt, Ecoli: E.coli.txt
Fac means store all fanctors of input data
This experiment shows the ratio of the sizes of SDD and ADFA for different sets of strings.
We can see the effect of merging in binary form for real data.
From bible.txt, we make two type data.
BibleAll is just the set of the sentences of the text.
BibleBi is all set of bigrams got from bible.txt.
Ecoli is single genome data.
The (Fac) means that the SDD and ADFA stores all factors of inputs.
For BibleALL, the sizes are almost equivalent.
But for other data, SDDs are about 10 to 20 % smaller than ADFAs.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

19. Binary Set Operation Algorithm
SDD has inherited the algorithm to compute binary set operations from BDD.
We define the algorithm and analyze its time complexity.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

20. Set operation A binary set operation ♢ ∈ {∪, ∩, ＼, …}
Input: two SDDs P, Q
Output: SDD R such that L(R) = L(P) ♢ L(Q) P Q
Binary Set Operation
P ♢ Q
We would like to compute the union or intersection of two sets.
By P daia Q we denote the unique SDD R that represents the set L(R) = L(P) daiamond L(Q)
SDD’s binary set operation create SDD P daiya Q for two inputs SDDs P and Q.
Where L(P diaya Q) = L(P) daiya L(Q)
We create a new SDD via Binary set operations .
Given 2 SDDs P and Q, we compute the SDD R, such that
Set operation is a method to create new SDDs by 2 given input SDDs.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

21. Apply algorithm Originally for BDD [Bryant 1986], applied to SDD
Based on the definition L(N) = N.lab ・ L(N.1) ∪ L(N.0)
In operation, (when P.lab = Q.lab) L(P) ♢ L(Q) = P.lab ・ (L(P.1) ♢ L(Q.1)) ∪ (L(P.0) ♢ L(Q.0)) P Q
P♢Q a Q1 Q0 a
P1♢Q1
P0♢Q1 a P1 P0 ♢
For binary set operation, we can use apply algorithm.
It is proposed for BDD by Bryant. It can be used for SDD too.
It compute P daiyamond Q recursively.
Based on the formula of the definition of a set of strings for an SDD node,
Apply algorithm devide the operation P daiamond Q into two direction.
0-side recursion and 1-side recursion.
To compute set operations P daia Q, an algorithm named applie is used.
It is proposed for BDD by Bryant in 1986, but it can apply SDD too.
This simple recursion is enough to create SDD P daia Q.
L(P daia Q) = LP LQ = L(P1) daia L(Q1) = L(P) daia L(Q)
The computation of P daia Q is simply decomposed into two subproblems computing P0 daia Q0 and P1 diamond Q1.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

22. Hash table technique N0 x N N1 Key-Value hash tables Uniquetable
Key: 〈letter x, SDD node N1, SDD node N0〉
Value: SDD node N with τ(N) = 〈x, N1, N0〉
Opcache
Key: 〈operation id ♢, SDD node P, SDD node Q〉
Value: SDD node R which is R = P ♢ Q P ♢ Q
P ♢ Q
Key (triple)
〈x, N1, N0〉
Value (node) N
Uniquetable
Key (triple)
〈♢, P, Q〉
Value (node) R
Opcache
During computing set operations, we use two key-value type hash tables.
The first one is called uniquetable.
Its key is a triple, <label, 1-c, 0-c>, and its value is the node with the triple.
%Key is composed by //.
This is used to prevent from creating 2 nodes that have the same label, 1-child and 0-child.
The other is called opcache.
Its key is also a tripes, <operation id daia, SDD P, SDD Q>
%Key is composed by operation name, right argument P, and left argument Q.
And value is the node of the result of the binary operation P daia Q.
This is used to avoid repeting of computation that has been already done.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

23. 　Node create process
Any SDD node needed during computation is created via this process
Once an internal node is registered in Uniquetable, equivalent nodes will not created anymore.
Check the Uniquetable for key 〈x, N1, N0〉.
Exist
Not exist
Uniquetable is used in the process to create nodes.
Return it.
Create a new node and return it.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

24. Time complexity When P ♢ Q is executed Naïve method This method
Every operation use Opcache
At most |P| × |Q| different instances of recursive calls invoke
(Assume that the access time to hash tables is constant)
Naïve method
Prepare |P| × |Q| size table
This method
No useless or redundant node
Theorem
Worst case O(|P| |Q|) time
Example needs Ω(|P| |Q|) time exist
Lower and upper bound got
Check the Opcache for key 〈♢, P, Q〉.
Exist
Not exist
P ♢ Q is already done,
return it.
Continue to computation on 0-side and 1-side.
We show the time complexity of alpy algorithm.
It is easy to understand, that there are at most PQ different instances of recursive calls.
So, we can complete the computation in O(PQ) time by storing results of subproblems in opcache.
Naïve method will prepare PQ size table to do this.
Our method don’t create needless node, or multiple equivalent nodes.
Although this doesn’t improve the worst case complexity,
Our experiments demonstrate apply algorithm is much more efficient done than the naïve aproach.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

25. Experiment Operation time
Prepare two SDDs for all factors of random texts of length n
Time to compute operation
We prepare two SDDs that store all factors of given random text of length n.
These lines indicate the computation time of set operations for these SDDs.
We can see that apply algorithm runs in almost linear time practically.
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

26. Conclusion Relationship to Acyclic Automata
An SDD can be |Σ| times smaller than an ADFA
For real data, SDDs are 10~20 % more compact than ADFAs
Computational complexity of binary set operations
Worst case time complexity is quadratic
Tight time bound is analyzed
In our experiment, operation time is almost linear
Future work
Efficient implement of various operations
Propose substring index on SDD
Factor SDD construction algorithm
Conclusion.
We strudied rela
Notes on Sequence Binary Decision Diagrams: Relationship to Acyclic Automata and Complexities of Binary Set Operations by Shuhei Denzumi, Ryo Yoshinaka, Shin-ichi Minato, and Hiroki Arimura, (TUE), Prague Stringology Conference 2011

27. Thank you!