1. Sequential PAttern Mining using A Bitmap Representation
Jay Ayres, Johannes Gehrke, Tomi Yiu and Jason Flannick
Dept. of Computer Science
Cornell University
Presenter
林哲存
林庭宇
徐敏容

2. Outline Introduction The SPAM Algorithm Data Representation
Lexicographic Tree for Sequences
Depth First Tree Traversal
Pruning
Data Representation
Data Structure
Candidate Generation
Experimental Evaluation
Synthetic data generation
Comparison With SPADE and PrefixSpan
Consideration of space requirements
Conclusion

3. Introduction I = {i1,i2,...,in} be a set of items
a subset X ⊆ I an itemset
|X| is the size of X
A sequence s = (s1, s2, , sm) is an ordered list of itemsets, where si ⊆ I, i ∈ {1,...,m}
The length l of a sequence s = (s1, s2, , sm) is defined as

4. Introduction Sa：A Sequence. Example：({a}, {b,c})
sup D(Sa)：The support of Sa in database D. Example：Sa = ({a},{b, c}), we know sup D(Sa) = 2
Given a support threshold minSup, a Sequence Sa is called a frequent sequential pattern on D if sup D(Sa) >=minSup.

5. Introduction Consider the sequence of customer 2
the size of this sequence is 2.
the length of this sequence is 4.
the support of Sa is 2, or 0.67.
If the minSup is < 0.67, then Sa is deemed frequent.
{b, c, d} 7 3
{a, b} 5
{a, b, c} 4 2
{b} 6 1
{a, b, d}
Itemset
TID
CID
Sa：({a}, {b,c})
({a, b}, {b, c, d}) 3
({b}, {a, b, c}) 2
({a, b, d}, {b, c, d}, {b, c, d}) 1
Sequence
CID
Table 1
Table 2

6. Introduction Contributions of SPAM
Novel depth-first search strategy that integrates a depth-first traversal of the search space with effective pruning mechanisms.
Vertical bitmap representation of the database with efficient support counting.
SPAM outperforms previous work by up to an order of magnitude.

7. The SPAM Algorithm
The Lexicographic Sequence Tree
Consider all sequences arranged in a sequence tree with the following structure：
The root of the tree is labeled with null
I={a, b}
Assume that there is a lexicographical ordering i<= of the items I in the database. If item i occurs before item j in the ordering, then we denote this by i <= j. This ordering can be extended to sequences by defining sa I <= sb if sa is a subsequence of sb.
If sa is not a subsequence of sb, then there is no relationship in this ordering.

8. The SPAM Algorithm
The Lexicographic Sequence Tree
Each sequence in the sequence tree can be considered as either a sequence-extended sequence or an itemset-extended sequence.
(a, b) {} a
a, a
a, b
a, a, a
a, a, b
a, (a, b)
= S-Step
= I-Step

9. The SPAM Algorithm For example: Sa= ({a, b, c}, {a, b})
The Lexicographic Sequence Tree
For example: Sa= ({a, b, c}, {a, b})
sequence-extended sequence of Sa
itemset-extended sequence of Sa
= S-Step
= I-Step
({a, b, c}, {a, b})
({a, b, c}, {a, b}, {d})
………
S-Step
({a, b, c}, {a, b})
({a, b, c}, {a, b, d})
I-Step

10. The SPAM Algorithm A standard depth-first manner.
Depth First Tree Traversal
A standard depth-first manner.
sup D (S)  minSup, store S and repeat DFS.
(a, b) {} a
a, a
a, b
a, a, a
a, a, b
a, (a, b)
sup D (S) < minSup, not need to repeat DFS.
A huge search space.

11. The SPAM Algorithm S-step Pruning By Apriori principle
Suppose we are at node ({a}) in the tree and suppose that S({a}) = {a, b, c, d}, I({a}) = {b, c, d}.
Suppose that ({a}, {c}) and ({a}, {d}) are not frequent.
By Apriori principle
({a},{a},{c}) ({a},{b},{c}) ({a},{a,c}) ({a},{b,c})({a},{a},{d}) ({a},{b},{d}) ({a},{a,d}) ({a},{b,d})are not frequent Hence, when we are at node ({a}, {a}) or ({a}, {b}), we do not have to perform I-step or S-step using items c and d,
i.e. S({a},{a}) = S({a},{b}) = {a, b}
I({a},{a}) = {b}, and I({a}, {b}) = null .

12. The SPAM Algorithm I-step Pruning
Let us consider the same node ({a}) described in the previous section. The possible itemset-extended sequences are ({a, b}), ({a, c}), and ({a, d}). If ({a, c}) is not frequent, then ({a, b, c}) must also not be frequent by the Apriori principle. Hence, I({a, b}) = {d}, S({a,b}) = {a, b}, and S({a, d}) = {a, b}.

13. Pseudocode for DFS with pruning

14. Data Representation
Data Structure
Our algorithm uses a vertical bitmap representation of the data.
A vertical bitmap is created for each item in the dataset, and each bitmap has a bit corresponding to each transaction in the dataset.
A bit is set to 1 if the transaction it represents contains the last itemset in the sequence, and previous transactions contain all previous itemsets in the sequence
(i.e. the customer contains the sequence of itemsets)

15. If item i appears in transaction j, then the bit corresponding to transaction j of the bitmap for item i is set to one; otherwise, the bit is set to zero.
If the size of a sequence is between 2k + 1 and 2k+1, we consider it as a 2K+1-bit sequence.
(the size of a sequence =3), k=1, 21+1-bit=4-bit

16. Data Representation Data Structure Dataset sorted by CID and TID
{b, c, d} 7 3
{a, b} 5
{a, b, c} 4 2
{b} 6 1
{a, b, d}
Itemset
TID
CID
Dataset sorted by CID and TID
{a}
{c}
{d}
CID TID

17. Data Representation
(Candidate Generation) Process
S-step requires that we set the first 1 in the current sequence’s bit slice to 0, and all bits afterward to 1, to indicate that the new item can only come in a transaction after the last transaction in the current sequence.
I-step, only the AND is necessary, since the item on the end occurs within the same transaction as the last itemset in the current sequence:

18. S-step Process & S-step result process 1 0 0 0 0 1 0 0 0 1 1 1 0 0 1 1
({a})s
({a})
{b}
({a},{b})
S-step
process
&
result
({a, b}, {b, c, d}) 3
({b}, {a, b, c}) 2
({a, b, d}, {b, c, d}, {b, c, d}) 1
Sequence
CID

19. I-step Process & result 0 1 0 0 0 0 0 0 1 1 1 0 0 1 1 0 {d}
({a},{b, d})
&
result
({a},{b})
({a, b}, {b, c, d}) 3
({b}, {a, b, c}) 2
({a, b, d}, {b, c, d}, {b, c, d}) 1
Sequence
CID

20. Experimental Evaluation
Synthetic data generation
Using the 「IBM AssocGen program」 to generate dataset.
We also compared the performance of the
algorithms as the minimum support was varied for several datasets of different sizes.

21. Experimental Evaluation
Comparison With SPADE and PrefixSpan
We compared the three algorithms on several small, medium, and large datasets for various minimum support values.
This set of tests shows that SPAM outperforms SPADE by about a factor of 2.5 on small datasets and better than an order of magnitude for reasonably large datasets.
PrefixSpan outperforms SPAM slightly on very small datasets, but on large datasets SPAM outperforms PrefixSpan by over an order of magnitude.

22. Sample running time graph
Figure 6: Varying support for small dataset #1
Figure 7: Varying support for small dataset #2

23. Sample running time graph
Figure 8: Varying support for medium-sized
dataset #1
Figure 9: Varying support for medium-sized
dataset #2

24. Sample running time graph
Figure 10: Varying support for large dataset #1
Figure 11: Varying support for large dataset #2

25. Experimental Evaluation
Comparison With SPADE and PrefixSpan
SPAM performs so well for large datasets.
PrefixSpan runs slightly faster for small datasets.
The SPAM excels at finding the frequent sequences for many different types of large datasets.

26. Figure 12: Varying number of customers
Figure 13: Varying number of transactions per customer

27. Figure 14: Varying number of items per transaction
Figure 15: Varying average length of maximal sequences

28. Figure 16: Varying average length of transactions
Figure 16: Varying average length of transactions
within maximal sequences
Figure 17: Varying support with large number of
customers in dataset

29. Experimental Evaluation
Consideration of space requirements
Uses a depth-first traversal of the search space, it is quite space-inefficient in comparison to SPADE.
Using the vertical bitmap representation to store transactional data is inefficient
when an item is not present in a transaction, storing a zero to represent that fact.
The representation of the data that SPADE uses is more efficient.

30. Experimental Evaluation
Consideration of space requirements
SPAM is less space-efficient then SPADE as long as 16T < N(the total number of items across all of the transactions).
Thus the choice between SPADE and SPAM is clearly a space-time trade off.

31. Conclusion
Consideration of space requirements
We presented an algorithm to quickly find all frequent sequences in a list of transactions.
The algorithm utilizes a depth-first traversal of the search space combined with a vertical bitmap representation to store each sequence.
Experimental results demonstrated that our algorithm outperforms SPADE and PrefixSpan on large datasets.

32. Discussion Strongest part of this paper Weak points of this paper
SPAM outperforms other algorithms on large dataset.
Weak points of this paper
Space utility may not good.
Need to load all data into main memory.
Possible improvement
Using linking list to improve the vertical bitmap space utility.
Possible extension & applications
To forecast in the stock market or financial investment.

33. Sequential PAttern Mining using A Bitmap Representation
Q&A
Sequential PAttern Mining using A Bitmap Representation
Presenter
林哲存
林庭宇
徐敏容