date:2004/03/05 Mining Frequent Episodes for relating Financial Events and Stock Trends Anny Ng and Ada Wai-chee Fu PAKDD 2003 報告者： Ming Jing Tsai
Definition Events : financial news,political … e 1,e 2,e 3 ….,e k : event types day record D i :{e i1,e i2,e i3 ….,e ik } Episode:{e 1,e 2,e 3 ….,e k } ， has at least two elements and at least one e j is a stock event type Window = x days
Definition Window frequency ： number of windows that contains an event type DB frequency ： number of occurrences of an event type in DB Frequency of an episode (ex) number of windows the first day of window contains at least one of the event types in episode.
Construct event tree Header in descending db frequencies order Event_set pair sorted in the descending db frequencies node : E :event type,c :counts,b :binary bit
Pruning method window frequencies < min_sup Remove duplicate event type in both firstday part and remaining day part
daysevents 1b 2ac 3b 4d 5b 6ca 7d Window = 3,min_sup =3 An Event database Db frequencies
windows windowDay included Event_set pairs 11,2,3 22,3,4 33,4,5 44,5,6 55,6,7 66,7 77 Ordered frequent event type Window frequencies Window = 3,min_sup =3
{null} {b:1:0} {a:1:1} {c:1:1} {a:1:0} {c:1:0} {b:1:1} {d:1:1} b a c d
{null} {b:2:0} {a:1:1} {c:1:1} {a:1:0} {c:1:0} {b:1:1} b a c d {d:1:0} {b:1:1} {a:1:1} {c:1:1} {d:1:1}
{null} {b:3:0} {a:1:1} {c:1:1} {a:1:0} {c:1:0} {b:1:1} b a c d {d:1:0} {b:1:1} {a:1:1} {c:1:1} {d:1:1}
{null} {b:3:0} {a:2:1} {c:2:1} {a:1:0} {c:1:0} {b:1:1} b a c d {d:1:0} {b:1:1} {a:1:1} {c:1:1} {d:1:1}
{null} {b:3:0} {a:2:1} {c:2:1} {a:2:0} {c:2:0} {b:1:1} b a c d {d:1:0} {b:1:1} {a:1:1} {c:1:1} {d:1:1}
{null} {b:3:0} {a:2:1} {c:2:1} {a:2:0} {c:2:0} {b:1:1} b a c d {d:2:0} {b:1:1} {a:1:1} {c:1:1} {d:1:1}
Mining frequent episode Header table{h 0,h 1, …..,h H } Mining recursively each of the linked list kept at the header table from bottom to top Conditional path can build conditional event tree Object 1:found frequent episodes of form {a} ∪ {h i } first-part frequencies Object 2:found frequent episodes that contain h i and at least two other event types Db frequencies
Traverse conditional path Remove invalid event types Adjust counts of nodes above hi in the path to be equal to that of hi If hi is in the firstdays part, then move all event types in the remainingdays part to the firstdays part Remove hi from the path
Generate frequent episode When a conditional event tree contains only a single path Any subset of firstpart ∪ event base set Any Subsets of firstpart ∪ Any Subsets of remainingpart ∪ event base set
Mining Header d event base set {d} db frequency:{ } First_part frequency:{ } Frequent episode :{bd,ad,cd} min_sup =3 WEvent_set pairs
Recursively Mining Header c event base set {cd} db frequency:{ } First_part frequency:{ } Frequent episode :{bcd,acd}
Recursively Mining Header a event base set {acd} db frequency:{ } First_part frequency:{ } Frequent episode :{bacd}
Mining Header c event base set {c} db frequency:{ } First_part frequency:{ } Frequent episode :{bc} min_sup =3 WEvent_set pairs
Recursively Mining Header a event base set {ac} db frequency:{ } First_part frequency:{ } Frequent episode :{bac} min_sup =3
Mining Header a event base set {a} db frequency:{ } First_part frequency:{ } Frequent episode :{ba} min_sup =3 WEvent_set pairs
Experiment (synthetic data)
Dataset 2 T20,I5,M1000,D3K
Experiment (real data) News event from a internet 121 event types 757 days Stock data Dow Jones,Nasdaq,Hang Seng, 12 top local companies
Experiment (real data)
episodesupport Nasdaq downs, PCCW downs151 Nasdaq ups, SHK properties flats, HSBC flats 178 China Mobile downs, Nasdaq downs, HK Electric flats 178