1. Large Graph Mining: Power Tools and a Practitioner’s guide
Task 5: Graphs over time & tensors
Faloutsos, Miller, Tsourakakis
CMU
KDD '09
Faloutsos, Miller, Tsourakakis

2. Faloutsos, Miller, Tsourakakis
Outline
Introduction – Motivation
Task 1: Node importance
Task 2: Community detection
Task 3: Recommendations
Task 4: Connection sub-graphs
Task 5: Mining graphs over time
Task 6: Virus/influence propagation
Task 7: Spectral graph theory
Task 8: Tera/peta graph mining: hadoop
Observations – patterns of real graphs
Conclusions
KDD '09
Faloutsos, Miller, Tsourakakis

3. Faloutsos, Miller, Tsourakakis
Thanks to
Tamara Kolda (Sandia)
for the foils on tensor definitions, and on TOPHITS
KDD '09
Faloutsos, Miller, Tsourakakis

4. Faloutsos, Miller, Tsourakakis
Detailed outline
Motivation
Definitions: PARAFAC and Tucker
Case study: web mining
KDD '09
Faloutsos, Miller, Tsourakakis

5. Examples of Matrices: Authors and terms
data
mining
classif.
tree
...
John
Peter
Mary
Nick
KDD '09
Faloutsos, Miller, Tsourakakis

6. Motivation: Why tensors?
Q: what is a tensor?
KDD '09
Faloutsos, Miller, Tsourakakis

7. Motivation: Why tensors?
A: N-D generalization of matrix:
KDD’09
data
mining
classif.
tree
...
John
Peter
Mary
Nick
KDD '09
Faloutsos, Miller, Tsourakakis

8. Motivation: Why tensors?
A: N-D generalization of matrix:
KDD’07
KDD’08
KDD’09
data
mining
classif.
tree
...
John
Peter
Mary
Nick
KDD '09
Faloutsos, Miller, Tsourakakis

9. Tensors are useful for 3 or more modes
Terminology: ‘mode’ (or ‘aspect’):
Mode#3
data
mining
classif.
tree
...
Mode#2
Mode (== aspect) #1
KDD '09
Faloutsos, Miller, Tsourakakis

10. Faloutsos, Miller, Tsourakakis
Notice
3rd mode does not need to be time
we can have more than 3 modes
Dest. port
125
... 80
IP source
IP destination
KDD '09
Faloutsos, Miller, Tsourakakis

11. Faloutsos, Miller, Tsourakakis
Notice
3rd mode does not need to be time
we can have more than 3 modes
Eg, fFMRI: x,y,z, time, person-id, task-id
From DENLAB, Temple U.
(Prof. V. Megalooikonomou +)
KDD '09
Faloutsos, Miller, Tsourakakis

12. Motivating Applications
Why tensors are useful?
web mining (TOPHITS)
environmental sensors
Intrusion detection (src, dst, time, dest-port)
Social networks (src, dst, time, type-of-contact)
face recognition
etc …
KDD '09
Faloutsos, Miller, Tsourakakis

13. Faloutsos, Miller, Tsourakakis
Detailed outline
Motivation
Definitions: PARAFAC and Tucker
Case study: web mining
KDD '09
Faloutsos, Miller, Tsourakakis

14. Faloutsos, Miller, Tsourakakis
Tensor basics
Multi-mode extensions of SVD – recall that:
KDD '09
Faloutsos, Miller, Tsourakakis

15. Faloutsos, Miller, Tsourakakis
Reminder: SVD
Best rank-k approximation in L2 n n m A   VT m U
KDD '09
Faloutsos, Miller, Tsourakakis

16. Faloutsos, Miller, Tsourakakis
Reminder: SVD
Best rank-k approximation in L2 n
1u1v1
2u2v2 A  + m
KDD '09
Faloutsos, Miller, Tsourakakis

17. Goal: extension to >=3 modes
I x R
K x R A B
J x R C
R x R x R
I x J x K
+…+ =
KDD '09
Faloutsos, Miller, Tsourakakis

18. Faloutsos, Miller, Tsourakakis
Main points:
2 major types of tensor decompositions: PARAFAC and Tucker
both can be solved with ``alternating least squares’’ (ALS)
KDD '09
Faloutsos, Miller, Tsourakakis

19. Specially Structured Tensors
Tucker Tensor
Kruskal Tensor
Our Notation
Our Notation
+…+ = u1 uR v1 w1 vR wR = U
I x R V
J x R W
K x R
R x R x R
“core”
K x T W
I x J x K
I x J x K
I x R
J x S U V =
R x S x T
KDD '09
Faloutsos, Miller, Tsourakakis

20. Tucker Decomposition - intuition
I x J x K A
I x R B
J x S C
K x T
R x S x T
author x keyword x conference
A: author x author-group
B: keyword x keyword-group
C: conf. x conf-group
G: how groups relate to each other
KDD '09
Faloutsos, Miller, Tsourakakis

21. Intuition behind core tensor
2-d case: co-clustering
[Dhillon et al. Information-Theoretic Co-clustering, KDD’03]
KDD '09
Faloutsos, Miller, Tsourakakis

22. Faloutsos, Miller, Tsourakakisn
eg, terms x documents m k l n l k m
KDD '09
Faloutsos, Miller, Tsourakakis

23. Faloutsos, Miller, Tsourakakis
med. doc
cs doc
med. terms
cs terms
term group x
doc. group
common terms
doc x
doc group
term x
term-group
KDD '09
Faloutsos, Miller, Tsourakakis

24. Faloutsos, Miller, Tsourakakis
Tensor tools - summary
Two main tools
PARAFAC
Tucker
Both find row-, column-, tube-groups
but in PARAFAC the three groups are identical
( To solve: Alternating Least Squares )
KDD '09
Faloutsos, Miller, Tsourakakis

25. Faloutsos, Miller, Tsourakakis
Detailed outline
Motivation
Definitions: PARAFAC and Tucker
Case study: web mining
KDD '09
Faloutsos, Miller, Tsourakakis

26. Faloutsos, Miller, Tsourakakis
Web graph mining
How to order the importance of web pages?
Kleinberg’s algorithm HITS
PageRank
Tensor extension on HITS (TOPHITS)
8 mins
KDD '09
Faloutsos, Miller, Tsourakakis

27. Kleinberg’s Hubs and Authorities (the HITS method)
Sparse adjacency matrix and its SVD:
authority scores
for 2nd topic
authority scores
for 1st topic to
from
hub scores for 1st topic
hub scores for 2nd topic
KDD '09
Faloutsos, Miller, Tsourakakis
Kleinberg, JACM, 1999

28. HITS Authorities on Sample Data
We started our crawl from and crawled 4700 pages,
resulting in cross-linked hosts.
authority scores
for 2nd topic
authority scores
for 1st topic to
from
hub scores for 1st topic
KDD '09
hub scores for 2nd topic
Faloutsos, Miller, Tsourakakis

29. Three-Dimensional View of the Web
Observe that this tensor is very sparse!
KDD '09
Faloutsos, Miller, Tsourakakis
Kolda, Bader, Kenny, ICDM05

30. Three-Dimensional View of the Web
Observe that this tensor is very sparse!
KDD '09
Faloutsos, Miller, Tsourakakis
Kolda, Bader, Kenny, ICDM05

31. Three-Dimensional View of the Web
Observe that this tensor is very sparse!
KDD '09
Faloutsos, Miller, Tsourakakis
Kolda, Bader, Kenny, ICDM05

32. Topical HITS (TOPHITS)
Main Idea: Extend the idea behind the HITS model to incorporate term (i.e., topical) information.
term scores for 1st topic
term scores for 2nd topic
term to
from
authority scores
for 1st topic
authority scores
for 2nd topic
hub scores for 2nd topic
hub scores for 1st topic
KDD '09
Faloutsos, Miller, Tsourakakis

33. Topical HITS (TOPHITS)
Main Idea: Extend the idea behind the HITS model to incorporate term (i.e., topical) information.
term scores for 1st topic
term scores for 2nd topic
term to
from
authority scores
for 1st topic
authority scores
for 2nd topic
hub scores for 2nd topic
hub scores for 1st topic
KDD '09
Faloutsos, Miller, Tsourakakis

34. TOPHITS Terms & Authorities on Sample Data
TOPHITS uses 3D analysis to find the dominant groupings of web pages and terms.
wk = # unique links using term k
Tensor PARAFAC
term scores for 1st topic
term scores for 2nd topic
term to
from
authority scores
for 1st topic
authority scores
for 2nd topic
hub scores for 2nd topic
KDD '09
Faloutsos, Miller, Tsourakakis
hub scores for 1st topic

35. Faloutsos, Miller, Tsourakakis
Conclusions
Real data are often in high dimensions with multiple aspects (modes)
Tensors provide elegant theory and algorithms
PARAFAC and Tucker: discover groups
KDD '09
Faloutsos, Miller, Tsourakakis

36. Faloutsos, Miller, Tsourakakis
References
T. G. Kolda, B. W. Bader and J. P. Kenny. Higher-Order Web Link Analysis Using Multilinear Algebra. In: ICDM 2005, Pages , November 2005.
Jimeng Sun, Spiros Papadimitriou, Philip Yu. Window-based Tensor Analysis on High-dimensional and Multi-aspect Streams, Proc. of the Int. Conf. on Data Mining (ICDM), Hong Kong, China, Dec 2006
KDD '09
Faloutsos, Miller, Tsourakakis

37. Faloutsos, Miller, Tsourakakis
Resources
See tutorial on tensors, KDD’07 (w/ Tamara Kolda and Jimeng Sun):
KDD '09
Faloutsos, Miller, Tsourakakis

38. Tensor tools - resources
Toolbox: from Tamara Kolda:
csmr.ca.sandia.gov/~tgkolda/TensorToolbox
T. G. Kolda and B. W. Bader. Tensor Decompositions and Applications. SIAM Review, Volume 51, Number 3, September 2009
csmr.ca.sandia.gov/~tgkolda/pubs/bibtgkfiles/TensorReview-preprint.pdf
T. Kolda and J. Sun: Scalable Tensor Decomposition for Multi-Aspect Data Mining (ICDM 2008)
KDD '09
ICDE’09
Copyright: Faloutsos, Tong (2009)
Faloutsos, Miller, Tsourakakis
2-38
2-38