1. Computing/Modeling with Big Data
Xindong Wu (吴信东)
中国 · 合肥工业大学计算机与信息学院;
Department of Computer Science
University of Vermont, USA

2. Outline 1 The Era of Big Data 2 Big Data Characteristics 3
A Big Data Processing Framework 4
Streaming Data and Streaming Features 5
Concluding Remarks

3. ICDM ’13 Panel: Data Mining with Big Data
Panel Chair: Xindong Wu
Panelists:
Chris Clifton (NSF & Purdue)
Vipin Kumar (Minnesota, FIEEE,FACM,FAAAS)
Jian Pei (TKDE EiC, Canada, FIEEE)
Bhavani Thuraisingham (UTDallas, Security, FIEEE, FAAAS)
Geoff Webb (DMKD EiC, Australia)
Zhi-Hua Zhou (Nanjing,China,FIEEE)
Big Data: a hot topic, but what useful content?
What new aspects? or is it just data mining?
How does data mining change with Big Data?
What should data miners do to cope with these changes?
Big Data

4. Big Data, from 70s to Now, and 2046
The 1st International Conference on Very Large Data Bases (September 22-24, 1975, Framingham, MA, USA)
Very large = big?
The first ER model paper, QBE, …
XLDB – Extremely Large Databases and Data Management, started on October 25, 2007
?LDB in 2046?
ULDB – Upmost Large Databases 
Cent 01: being big is relative, going big is a deterministic trend
Data mining: keep evolving
J. Pei: Big Data Analytics 101

5. Some comments on big data
David Hand
Imperial College, London
David Hand: Some comments on big data, December 2013

6. The power law theorem of data set size:
The number of data sets of size n is inversely proportional to n
There are vastly more small data sets than very large ones
So small data sets are likely to have a much larger impact on the world than big data sets
David Hand: Some comments on big data, December 2013

7. No-one actually wants data
What people want are answers
Which may be extracted from data
So data are only half the answer
The other half is statistics, data mining, machine learning, and other data analytic disciplines
David Hand: Some comments on big data, December 2013

8. The manure heap theorem of data discoveries
The probability of finding a gold coin in a heap of manure tends towards 1 as the size of the heap tends to infinity. (This theorem is false)
David Hand: Some comments on big data, December 2013

9. Data Science not just for Big Data
Gregory
Analytics, Big Data,
Data Mining, and Data Science Resources
© KDnuggets 2013

10. Finding Useful Patterns in Data
What do we call it?
Same Core Idea:
Finding Useful Patterns in Data
Different
Emphasis
Statistics, 1830-
Data mining, 1980-
Knowledge Discovery in
Data (KDD), 1989-
Business Analytics, 1997-
Predictive Analytics, 2002-
Data Analytics,2011-
Data Science, 2011-
Big Data,
© KDnuggets 2013

11. Big Data > Data Mining > > Predictive Analytics , Data Science
Google Trends search, Jan Sep 2013, Worldwide
© KDnuggets 2013

12. Data Science before “Big Data”
Ancient astronomers
Kepler laws of planetary motion (1609), derived from observations by Tycho Brahe
Genetics – Gregor Mendel found patterns in inheritance of pea plants
Western Medicine …
© KDnuggets 2013

13. Data Science Basic Principles & Ideas
Focus on actionable patterns
Build predictive models - supervised learning (train, test, x-validate)
Avoid overfitting
Calculating similarity of objects - unsupervised learning
Avoid information leakers
Select important variables/features
Model accuracy vs lift: how much more prevalent a pattern is than would be expected by chance
Estimate probability and cost/gain of actions
Help optimize decisions
© KDnuggets 2013

14. What Changes in Data Science with Big Data?
Data munging becomes much more complex
New algorithms, technology needed to deal with Big Data Volume, Velocity, & Variety
New, effective algorithms that require Big Data: e.g.: deep belief networks, recommendations
Predictions become (somewhat ) more accurate
New things become visible: social networks, recommendations, mobility, knowledge ?
However, basic principles remain
© KDnuggets 2013

15. Outline 1 The Era of Big Data 2 Big Data Characteristics 3
A Big Data Processing Framework 4
Streaming Data and Streaming Features 5
Concluding Remarks

16. Motivating Example
On October 4, 2012, the first presidential debate between President Barack Obama and Governor Mitt Romney triggered more than 10 million tweets within two hours.
How to explore the large volumes of data and extract useful information or knowledge for future actions?

17. Big Data Characteristics: HACE Theorem
Xindong Wu, Xinquan Zhu, Gongqing Wu, Wei Ding. Data Mining with Big Data.
IEEE Transactions on Knowledge and Data Engineering (TKDE), 26(2014), 1:
从2014年1月开始，已连续17个月在IEEE XPLORE 数字图书馆里 (所有期刊和会议论文、
所有年份从1884年开始) 每个月的下载量全球第一!
Summarizing the key challenges for Big Data mining
HACE Theorem:
a theorem to model Big Data characteristics
1884年开始，最大的技术学会
Big Data
Google Scholar 已有引用次数：165

18. HACE Theorem Big Data starts with large-volume, Heterogeneous,
Autonomous sources with distributed and decentralized control,
and seeks to explore
Complex and
Evolving relationships among data.

19. Small Example for Big Data (a moving, growing elephant with blind men)
Source: Internet (

20. Outline 1 The Era of Big Data 2 Big Data Characteristics 3
A Big Data Processing Framework 4
Streaming Data and Streaming Features 5
Concluding Remarks

21. A Big Data Processing Framework
Computing

22. A Big Data Processing Framework
Tier I (databases): Big Data computing platform, focusing on distributed, decentralized, low-level data accessing and computing.
Tier II (expert systems): Information sharing and privacy, and application domains, with
high level semantics,
application domain knowledge,
user privacy issues.
Tier III : Data mining: Knowledge discovery.

23. Big Data Mining Challenges (1)
Big Data Computing Platform Challenges
Data accessing
Huge and evolving data volumes
Heterogeneous and autonomous sources
Diverse representations
Unstructured data
Computing processors
High performance computing platforms

24. Big Data Mining Challenges (2)
Big Data Semantics and Application Knowledge
Data sharing and privacy
How data are maintained, accessed, and shared
Domain and application knowledge
What are the underlying applications ?
What are the knowledge or patterns users intend to discover from the data ?

25. Big Data Mining Challenges (3)
Local Learning and Model Fusion for Multiple Information Sources 1
Big Data Mining
Algorithms
Big Data Mining
Algorithms
Mining from Sparse, Uncertain, and Incomplete Data 2
Mining Complex and Dynamic Data 3

26. Outline 1 The Era of Big Data 2 Big Data Characteristics 3
A Big Data Processing Framework 4
Streaming Data and Streaming Features 5
Concluding Remarks

27. Handling Huge and Evolving Big Data
Real-time processing of Big data streams
Real-time processing of Big feature streams

28. PNRS Personalized news recommendation system
Xindong Wu, Fei Xie, Gongqing Wu and Wei Ding, Personalized News
Filtering and Summarization on the Web. IEEE ICTAI 2011,
(Best Paper Award)

29. Personalized Web News Filtering
The Internet
Web news
News Aggregator
News Filter
Filtered News
Personalized Web News Recommendation
Keyword Knowledgebase
User Feedback

30. Personalized News Filtering & Summarization (PNFS)
Personalized Web News Filtering
Web News Summarization
Keyword Knowledgebase

31. Streaming Features
Streaming features: involve a feature stream that flows in one by one over time while the number of training examples remains fixed.
Online feature selection with streaming features：To maintain an optimal feature subset over time from a feature stream, by online identify a redundant feature or an irrelevant feature upon its arrival.

32. A Framework for Streaming Feature SelectionY
Relevancy analysis of X
Removing irrelevant feature X
Redundancy analysis of X
Arriving in a
new feature X
X is redundant? N
Xindong Wu, Kui Yu, Hao Wang, and Wei Ding, Online Streaming Feature
Selection, Proc. the 27th International Conference on Machine Learning
(ICML 2010), 2010,
Xindong Wu, Kui Yu, Wei Ding, Hao Wang, and Xingquan Zhu.
Online Feature Selection with Streaming Features.
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI),
35(2013), 5:
Update the redundancy
of the current model N
Is it over?
One of the stopping criteria：
a predefined prediction accuracy is satisfied, or
the maximum number of iterations is reached, or
no further features are available. Y
Output the current
feature set

33. The OSFS algorithm Online relevancy analysis
Online redundancy analysis

34. Fast-OSFS (a fast version of OSFS）
Online relevancy analysis for X
Redundancy analysis for the current feature set
Online redundancy analysis for X

35. Experimental Results Datasets: 16 high-dimensional datasets
Competing algorithms：Grafting and Alpha-investing
Evaluation metrics: Prediction accuracy and running time
Dataset
#Features
# instances
bankruptcy
147
7063
leukemia
7129 72
sylva
216
14374
prostate
6033
102
madelon
500
2000
lung-cancer
12533
181
arcene
10000
100
breast-cancer
17816
286
dexter
20000
300
ovarian-cancer
2190
dorothea
100000
800
sido0
4932
12678
lymphoma
7399
227
ohsumed
14373
5000
colon 62
apcj-etiology
28228
15779

36. Fast-OSFS and Grafting
Prediction accuracy：the y-axis to the left (top two figures) ; The size of the selected feature subset: the y-axis to the right (bottom two figures).

37. Fast-OSFS and Alpha-investing

38. OSFS and Fast-OSFS

39. Running time-OSFS vs.Fast-OSFS

40. Prediction Accuracy with # Features
Study the change of the prediction accuracy on Knn with respect to the features continuously arriving over time.
In comparison with the prediction accuracy of the baseline Knn classifier trained using all features.
Conclusion: Fast-OSFS achieves better and more stable performance on the models trained from selected streaming features.

41. Prediction Accuracy with # Features
Baseline -> Baseline

42. Prediction Accuracy with # Features

43. Impact of Ordering of Incoming Features
Observations:
Varying the order of the incoming features does impact on the final outcomes.
The results demonstrate that Fast-OSFS is the most stable method and Alpha-investing appears to be highly unstable.

44. A Case Study: Automatic Impact Crater Detection
Impact craters in a 37,500×56,250 m2 test image from Mars

45. Streaming features with impact crater detection
Expensive costs of feature generation
How many features should we should generate?
Problem：Can we interleave feature generation and feature selection?
While rich texture features provide a tremendous source of potential features for use in crater detection tasks, they are expensive to generate and store.

46. A Case Study: Automatic Impact Crater Detection
A framework of streaming feature selection for crater detection

47. Training data and testing data
Training data: consist of 204 true craters and 292 non-crater examples selected randomly from crater candidates located in the northern half of the east region.
Test data：

48. Experimental Results Prediction accuracy on three regions (alpha=0.01)

49. Prediction accuracy with # features arrived

50. Comparison w/ Traditional Feature Selection

51. Outline 1 The Era of Big Data 2 Big Data Characteristics 3
A Big Data Processing Framework 4
Streaming Data and Streaming Features 5
Concluding Remarks

52. Conclusion: HACE Theorem w/ Big Data
Huge Data with Heterogeneous and Diverse Dimensionality
HACE Theorem
Autonomous Sources with Distributed and Decentralized Control
Complex and Evolving Relationships

53. Recent and Ongoing Research Efforts
Integrating and Mining Big Data from Multiple Sources in Biological Networks (NSF)
Cross-Domain, Cross-Platform Integration and Mining of Multiple, Heterogeneous Data (863)
Pattern Matching and Mining with Wildcards and Length Constraints (NSFC)
Group Influence and Interactions in Social Networks (973)
Massive Dynamic Multi-Source Information Processing (PCSIRT)

54. Thanks and Questions
11/29/2018