1. Features & Decision regions
Classification model,
Features & Decision regions

2. What is classification
Classification is the process of constructing a
model that classifies data based on the training
set and uses it in classifying new data instances.
For example classify
Countries based on climate
Cars based on gas mileage
Customers for credit approval
Card Fraud Detection etc.

4. Example-Steganlysis process
Steganalysis process is shown in figure, that consist of two phase Training phase and testing phase.
(1)In Training phase, features are extracted from training image dataset, then classifier is trained and finally trained classifier will make the decision whether image is clean or stego image.
(2)In testing phase, features are extracted from test image dataset similar to the way as done in training phase. These are fed to classifier and classifier based on the training decides final classification decision.
Steganalysis is like a pattern classification problem having two significant stages feature extraction and classification.

5. Classification in MATLAB
Popular Methods for classification:
Decision trees
Rule learners
Naive Bayes
Decision tables
SVMs
ANN

6. TRAINING SET

7. Feature Extraction

8. ANN Classifier

9. TEST SET – classifying new instances
Training (1)You can explore your data, (2)select features, (3)train models, and (4)assess results. Testing (1)Prepare dataset (3)Give features to trained classifier( classifier-trained from Training Phase)

10. HOW TO MEASURE PERFORMANCE OF CLASSIFIER (1)confusion matrix
Predicted class
Actual class
A(yes)
B(no)
74 (TP)
64 (FN)
30 (FP)
132 (TN)
Correctly classified instances : 206
Incorrectly classified instances : 94
Accuracy – (TP+TN)/(TP+TN+FN+FP)=206/300 = %
Error Rate – (FN+FP)/(TP+TN+FN+FP)=94/ 300 = %

11. Performance evaluation CLASS LABEL-YES
Precision: proportion of the predicted cases that were correct .
P= TP/(TP+FP)=74/104=.71
Recall or TP rate: proportion of positive cases that are correctly identified.
TPR= TP/(TP+FN)=74/138=.536
False Positive Rate (FP) : proportion of negatives cases that were incorrectly classified as positive.
FPR=FP/(FP+TN)=30/162=.185
F-Measure : is a combined measure for precision and recall.
2*Precision*Recall/(Precision+Recall)

12. Performance evaluation CLASS LABEL-NO
Precision: proportion of the predicted cases that were correct
P= TN/(TN+FN)=132/(132+64)=.67
Recall or TP rate: proportion of positive cases that are correctly identified.
TPR= TN/(TN+FP)=132/162 = .81
False Positive Rate (FP) : proportion of negatives cases that were incorrectly classified as positive.
FPR=FN/(TP+FN)=64/138=.46
F-Measure : is a combined measure for precision and recall.
2*Precision*Recall/(Precision+Recall)

13. (2)ROC-Receiver Operating Characteristics
ROC graphs are a way to examine the performance of classifiers . A ROC graph is a plot with the false positive rate on the X axis and the true positive rate on the Y axis.

14. The above figure shows an example of an ROC graph with two ROC curves labeled C1 and C2, and two ROC points labeled P1 and P2.

15. Classification Problem with Overlap

16. Decision Boundaries Decision Boundary Decision Region 1 Decision
Decision boundaries = boundaries between decision regions ( Decsision boundary separate two regions)
Decision
Region 2

17. THANK YOU