1. School of Computer Science & Engineering
Artificial Intelligence
Machine Learning,
Pattern Recognition,
Data Mining
Dae-Won Kim
School of Computer Science & Engineering
Chung-Ang University

2. AI Scope 3. Machine Learning/ Pattern Recognition/ Data Mining
1. Search-based optimization techniques for real-life problems
Hill climbing, Branch and bound, A*, Greedy algorithm
Simulated annealing, Tabu search, Genetic algorithm
2. Reasoning: Logic, Inference, and knowledge representation
Logical language: Syntax and Semantics
Inference algorithm: Forward/Backward chaining, Resolution, and Expert System
3. Machine Learning/ Pattern Recognition/ Data Mining
Classification: Bayesian algorithm, Nearest-neighbor algorithm, Neural network
Clustering: Hierarchical algorithm, K-Means algorithm
4. Uncertainty based on Probability theory
5. Planning, Scheduling, Robotics, and Industry Automation

3. Have you ever heard of Big Data?

4. Progress in digital data acquisition and storage technology has resulted in the growth of huge databases.

5. Data mining is the extraction of implicit, previously unknown, and potentially useful information from data.

6. We build algorithms that sift through databases automatically, seeking patterns.

7. Strong patterns, if found, will likely generalize to make accurate predictions on future data.

8. Algorithms need to be robust enough to cope with imperfect data and to extract patterns that are inexact useful.

9. Machine learning provides the technical basis of data mining.

10. We will study simple machine learning methods, looking for patterns in data.

11. People has been seeking patterns in data since human life began.
e.g., Samsung Galaxy: Samsung Pay,
Managers in Samsung want to find consuming patterns of users so that it’d provide personalized services.

12. In data mining, computer algorithm is solving problems by analyzing data in databases.

13. Data mining is defined as the process of (knowledge) discovering patterns in data.

14. Data mining is defined as the process of (knowledge) discovering patterns in data.

15. We start with a simple example.

16. Q: Tell me the name of this fish.

17. Algorithm ??

18. We have 100 fishes, and measured their lengths. (e. g
We have 100 fishes, and measured their lengths. (e.g., fish: x=[length]t)

20. Our algorithm can measure the length of a new fish, and estimate its label.

21. Yes, it is a typical prediction task through classification technique
Yes, it is a typical prediction task through classification technique. But, it is often inexact and unsatisfactory.

22. Next, we measured their lightness. (e.g., fish: x=[lightness])

24. Lightness is better than length.

25. Let us use both lightness and width. (e. g
Let us use both lightness and width. (e.g., fish: x=[lightness, width])

27. Each fish is represented a point (vector) in 2D x-y coordinate space.

28. Everything is represented as N-dimensional vector in coordinate space.

29. The world is represented as matrix

30. We assume that you have learned the basic concepts of linear algebra.

31. The objective is to find a line that effectively separates two groups.

32. How to find the line using a simple Math from high school?

33. We can build a complex nonlinear line to provide exact separation.

35. The formal procedure is given as:

37. This shows a predictive task of data mining, often called as pattern classification/ recognition/ prediction.

38. The act of taking in raw data and making an action based on the category of the pattern.

39. We build a machine that can recognize or predict patterns.

40. Q: How to represent and classify texts?
Opinion mining
Sentiment analysis

41. Another famous task of data mining is a descriptive task
Another famous task of data mining is a descriptive task. Cluster analysis is the well-known group discovery algorithm.

43. We will experience the basic issues in the prediction task (pattern classification) in forthcoming weeks.

44. Some terms should be defined.

45. Given training data set : ‘n x d’ pattern/data matrix:
Fish
Lightness
Length
Weight
Width
Class Label
Fish-1 10
70.3
6.0 36
Salmon
Fish-2
75.5
8.8
128
Fish-3 29
51.1
9.4
164
Sea bass
Fish-4
49.9
8.4
113

46. Given training data set : ‘n x d’ pattern/data matrix:
‘d’ features (attributes, variables, dimensions, fields)
Fish
Lightness
Length
Weight
Width
Class Label
Fish-1 10
70.3
6.0 36
Salmon
Fish-2
75.5
8.8
128
Fish-3 29
51.1
9.4
164
Sea bass
Fish-4
49.9
8.4
113
‘n’ patterns (objects, observations, vectors, records)

47. Classification General description Specific terms
Supervised pattern classification
Labeled training patterns, the groups are known a priori
Constructs rules for classifying new data into the known groups
Specific terms
Pattern=object=observation is represented as a feature vector
Distance measure for numeric and categorical data
Training set (answer database) and test set (new observation)
Prediction performance by accuracy, sensitivity, specificity, …
ex) Bayesian classifier, Nearest-neighbor classifier, SVM, NN, LDA, …

48. Each pattern is represented as a feature vector.

49. The training pattern matrix is stored in a file or database.

50. Given labeled training patterns, the class groups are known a priori.

51. We constructs algorithms to classify new data into the known groups.

52. Training data vs. Test data

53. Training data are used as answers
Training data are used as answers. We are learning algorithms using training data.

54. Test data are a set of new unseen data
Test data are a set of new unseen data. We predict class labels using the learned algorithm.

55. Training data Test data # of data # of features data index feature-1…
feature-N
class label
Feature-1
Test data
# of data
# of features
data index
feature-1
feature-2 …
feature-N
Feature-1

56. For example, we try to classify the tumor type of breast cancer patients

57. Breast-cancer-training.txt
100 30
Patient-1
165 52 …
210
cancer
Patient-2
170 50
230
normal
Patient-100
160 47
250
Breast-cancer-test.txt 10
163 55
215
155
240
Patient-10 45
235

58. To evaluate the performance of prediction algorithms, we need a performance measure (Accuracy).

59. Accuracy = (True Positive + True Negative) /
Gold Standard (Truth)
Positive
Negative
Prediction Result
True Positive
False Positive
False Negative
True Negative
Suspicious Patients with Breast Cancer
Positive (Cancer)
Negative (Normal)
Accuracy = (True Positive + True Negative) /
(True Positive + False Positive + False Negative + True Negative)

60. Accuracy = (30 + 55) / (30 + 5 + 10 + 55) = 0.85 (85%)
Gold Standard (Truth)
Positive
Negative
Prediction Result
True Positive
False Positive
False Negative
True Negative
Suspicious Patients with Breast Cancer
Positive (Cancer)
Negative (Normal) 30 5 10 55
Accuracy = ( ) / ( ) = 0.85 (85%)

61. References # Textbooks # Advanced Topic: Emotional data mining
1) R.O. Duda, et al., “Pattern Classification”
2) S. Theodoridis., “Pattern Recognition”
3) T.M. Mitchell., “Machine Learning”
# Advanced Topic:
Emotional data mining
Personalized music recommendation (Nov.)