1. Mining Time-Changing Data Streams
Advisor: Dr. Hsu
Graduate: Yung-Chu Lin
2002/3/6
IDS Lab Seminar

2. Outline Motivation Objective Hoeffding Bounds The VFDT Algorithm
The CVFDT Algorithm
Window Size
Time and Space Complexity
Empirical Study
Conclusion
Opinion
2002/3/6
IDS Lab Seminar

3. Motivation The volume and time span of accumulated data for future use
Real large database is not random sample drawn from a stationary
2002/3/6
IDS Lab Seminar

4. Objective Solving the classification problem: concept drift 2002/3/6
IDS Lab Seminar

5. Hoeffding Bounds n examples confidence 1-δ variable r, range R
probability: R=1
information gain: R=log C
2002/3/6
IDS Lab Seminar

6. Concept of VFDT Algorithm
Initialize the HT
Repeat
Scan nmin examples (a window size)
Compute Gain(x) of every attribute
Split the leaf node or not
2002/3/6
IDS Lab Seminar

7. The VFDT Algorithm
2002/3/6
IDS Lab Seminar

8. Example for Algorithms
Day
Outlook
Tempera-ture
Humidity
Wind
Play-Tennis D1
Sunny
Hot
High
Weak No D2
Strong D3
Overcast
Yes D4
Rain
Mild D5
Cool
Normal D6 D7 D8 D9
D10
D11
D12
D13
D14
2002/3/6
IDS Lab Seminar

9. Concept of CVFDT Algorithm
An extension to VFDT, which adds the ability to detect and respond to changes in example-generating
Not need to learn a new model from scratch every time a new example arrives
Scan HT and alternate trees periodically  look for internal nodes whose sufficient statistics indicate better attribute
When alternate subtree becomes more accurate, the old subtree is replaced by the new one
2002/3/6
IDS Lab Seminar

10. The CVFDT Algorithm
2002/3/6
IDS Lab Seminar

11. The CVFDTGrow Procedure
2002/3/6
IDS Lab Seminar

12. The ForgetExample and CheckSplitValidity procedure
2002/3/6
IDS Lab Seminar

13. Window Size
One windows size w will not be appropriate for every concept and every type of drift
Shrink w when
many of the nodes in HT become questionable at once
or in response to a rapid change in data rate
Increase w when
Few questionable node  concept is stable
2002/3/6
IDS Lab Seminar

14. Time and Space Complexity
VFDT
O(lvdvc)
CVFDT
O(lcdvc)
O(ndvc)
n: #nodes in CVFDT’s main tree and alternate trees
d: #attributes
v: max number of values per attribute
c: #class
lv,lc: longest path
2002/3/6
IDS Lab Seminar

15. Empirical Study
Synthetic data
Web data
2002/3/6
IDS Lab Seminar

16. Synthetic Data
A hyperplane in d-dimensional space is the set of points x that satisfy
where xi is the ith coordinate of x.
positive: negative:
2002/3/6
IDS Lab Seminar

17. Synthetic Data (cont’d)
Initialize weight to .2 except for w0 which is .25d
Substitute its coordinates into the left hand side of Equation to obtain a sum s
|s|<=.1*w0  positive
|s|<=.2*w0  negative
xi=[0,1]
2002/3/6
IDS Lab Seminar

18. Synthetic Data (cont’d)
5 million training examples
δ=0.0001
f=20,000
Nmin=300;┬=0.05;w=100,000
2002/3/6
IDS Lab Seminar

19. Synthetic Data (cont’d)
2002/3/6
IDS Lab Seminar

20. Synthetic Data (cont’d)
CVFDT took 4.3 times longer than VFDT
VFDT’s average memory allocation over the course of the run was 23MB while CVFDT’s was 16.5MB
The average number of nodes in VFDT’s tree was 2696 and in CVFDT’s tree was 677(132: alternate tree, 545: main tree)
2002/3/6
IDS Lab Seminar

21. Conclusion
Introducing CVFDT, learning accurate models from the most demanding high-speed, concept-drifting data streams
Maintain a decision-tree up-to-date with a windows of examples
2002/3/6
IDS Lab Seminar

22. Opinion Many techniques have the problem, concept drift.
So, maybe we could apply the concept in this paper to our other techniques, like the association rule mining, clustering algorithms, and so on.
2002/3/6
IDS Lab Seminar