Presentation is loading. Please wait.

Presentation is loading. Please wait.

Hybrid Ant Colony Optimization-Support Vector Machine using Weighted Ranking for Feature Selection and Classification.

Published byWilfred Owens Modified over 8 years ago

Similar presentations

Presentation on theme: "Hybrid Ant Colony Optimization-Support Vector Machine using Weighted Ranking for Feature Selection and Classification."— Presentation transcript:

1 Hybrid Ant Colony Optimization-Support Vector Machine using Weighted Ranking for Feature Selection and Classification

2 Dimension of dataset high Decreased Predictive performance More Computation Time F1 F2 F3 F4 F6 F5 F7 DATASET WITH ALL FEATURES DATASET WITH ONLY SELECTED FEATURES NOISY FEATURES Dimension of dataset low Increased Predictive performance Lesser Computation Time

3 Evaluate features using heuristic information Fast but inaccurate E.g. Statistical correlation Feature Selection Algorithms FiltersWrappers Evaluate features using learning algorithm Slow but accurate E.g. Ant Colony Optimization, Genetic Algorithm

4  Uses a hybrid filter-wrapper based approach.  Wrapper method: Ant Colony Optimization  Filter method : Weighted Ranking of features 7/10/20164

5 Ant Colony Optimization (ACO) 2 1 3 Food Source 4 PheromoneOptimal Path  ACO is an iterative process used for solving combinatorial optimization problems  Inspired by the foraging behavior of real ants and their inherent ability to find the shortest path from a food source to their nest.

6  Features equivalent to cities in TSP.  Ants conduct a partial tour in contrast to TSP.  ACO depends on: Heuristic Information on a feature Experience gained by ants in the previous iterations.

7 Filter approach. Each feature was assigned a Weighted Ranking score. Final Rank Score(WR) Information Gain Score (IG) Chi- Square Score (CS) CFS score (CFS) Weight w1 Weight w2 Weight w3 Weighted Rank (WR) of feature f is calculated as: WR f = w1 * IG f + w2 * CS f + w3 * CFS f

8 Proposed Algorithm 1 Rank the features based on the weighted summation of their IG, CS and CFS scores. 2 Initialize Pheromone values on all the links τo Initialize number of ants Initialize other algorithm parameters 3 An ant selects the first feature depending upon a scoring system. Scoring System employed for 90% of the cases. For the rest 10%, feature selected randomly

9 Proposed Algorithm 4 To select the next feature choose between exploitation and exploration 5 Conduct the partial tour in the same manner until maximum number of a priori fixed subset size is reached Depute other ants to conduct their own partial tours. 6 After completion of the tours of all ants, evaluate the quality of the tours employing SVM

10 7 Increase the pheromone concentration of links of the best ant proportional to the quality of the tour (accuracy of the classifier). 8 Decrease the pheromone concentrations on all the links visited by the remaining ants. 9 Repeat steps 3 to 8 for a certain number of iterations. Proposed Algorithm

11  A scoring system has been developed for the selection of the first feature, which depends on:  Total no. of links connecting a feature having a pheromone value better than a threshold.  The sum of all the pheromone values on links starting from that feature.  The fitness value of that particular feature.

12  The subset is decreased progressively from 90% of the original feature set size to 10%.  The subset size which gives the maximum accuracy is selected.

13  In order to test the proposed algorithm, an SVM learning algorithm was used.  After a solution is constructed, each ant passes its feature subset to the classifier and receives its accuracy.  This accuracy is used as a fitness function for selecting the best ant of that iteration. The classifier accuracy was evaluated using 10- fold cross validation.

14  Any good classifier can be employed as a wrapper.  Any combination of good filter-methods can be used with a weighted scoring system of features.

15  Time Consuming.  Solution: Parallel implementation of the algorithm

16 ParametersValues Number of ants100 Number of iterations50 Exploitation Probability Factor (q0) 0.7 Pheromone Update Strength ( φ) 0.25 Pheromone Decay Parameter ( ρ ) 0.98 Pheromone Importance Factor ( β ) 1 Information Gain weight0.3 Chi-Square weight0.3 CFS weight0.3

17  In order to test the proposed algorithm, six datasets were obtained from the UCI (University of California, Irvine) machine learning repository. Dataset NameNo. of featuresNo. of classesClass label type No. of instances Wisconsin Breast Cancer 102REAL683 Hepatitis192REAL155 Lung Cancer563REAL32 Splice602REAL1000 Bupa Liver Disorder 62REAL345 Statlog Heart132REAL270

18  The classifier accuracy was calculated for all the datasets with and without using Feature Selection and the results were compared. Dataset NameNo. of original features No. of Features in the subset % Accuracy without FS % Accuracy with FS Wisconsin Breast Cancer 10565.8996.9957 Hepatitis19357.4267.74 Lung Cancer5627587.5 Splice601685.588.1 Bupa Liver Disorder 6265.104269.401 Statlog Heart 13555.925983.7037

19  The selected subset of features of all the datasets Dataset NameSelected Features Wisconsin Breast Cancer2,3,4,7,9 Hepatitis12, 13, 14 Lung Cancer9, 14 Splice15, 16, 18, 19, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35 Bupa Liver Disorder1,2 Statlog Heart3,7,9,12,13

20  Malignant Tumor that develops from Breast cells  Most Common cause of death among women  To reduce deaths, early diagnosis is important  Early diagnosis is important to distinguish between benign and malignant tumors

21  Biopsy- expensive and invasive.  Several machine learning techniques successfully used to predict Breast Cancer.  Objective- To assign patients to benign or malignant class

22  Dataset obtained from UCI Machine Learning Repository.  Consists of 10 features, 699 instances.  Features describe characteristics of the cell nuclei present in the breast mass image.  An RBF kernel was found to outperform the polynomial and linear kernels for this task, with the value of C=100.  A feature subset of size 5 features obtained an accuracy of 96.9957%.

23 #FEATURESDOMAIN 1.Sample code numberId Number 2.Clump Thickness1-10 3.Uniformity of Cell Size1-10 4.Uniformity of Cell Shape1-10 5.Marginal Adhesion1-10 6.Single Epithelial Cell Size1-10 7.Bare Nuclei1-10 8.Bland Chromatin1-10 9.Normal Nucleoli1-10 10.Mitoses1-10 11.Class2 for benign, 4 for malignant

24 AlgorithmAccuracy (%) NB+CHI96.1373 NB+GAIN96.1373 NB+RelieF96.1373 NB+CFS95.9943 RBF+GA96.7096 KNN+CHIWSS95.4220 NN+CHIWSS74.23 SVM+CHWSS76.29 ACO-SVM+WR96.9957

25  Hybrid filter-wrapper technique.  Reduces complexity of Machine learning algorithms.  Easy implementation.  Increased Predictive Performance  Picks up the correlations between features.  Flexible

26

Download ppt "Hybrid Ant Colony Optimization-Support Vector Machine using Weighted Ranking for Feature Selection and Classification."

Similar presentations

Mustafa Cayci INFS 795 An Evaluation on Feature Selection for Text Clustering.

Mustafa Cayci INFS 795 An Evaluation on Feature Selection for Text Clustering.
CS6800 Advanced Theory of Computation

CS6800 Advanced Theory of Computation
Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
Data Mining Feature Selection. Data reduction: Obtain a reduced representation of the data set that is much smaller in volume but yet produces the same.

Data Mining Feature Selection. Data reduction: Obtain a reduced representation of the data set that is much smaller in volume but yet produces the same.
SVM - Support Vector Machines A new classification method for both linear and nonlinear data It uses a nonlinear mapping to transform the original training.

SVM - Support Vector Machines A new classification method for both linear and nonlinear data It uses a nonlinear mapping to transform the original training.
SVM—Support Vector Machines

SVM—Support Vector Machines
CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.
Particle swarm optimization for parameter determination and feature selection of support vector machines Shih-Wei Lin, Kuo-Ching Ying, Shih-Chieh Chen,

Particle swarm optimization for parameter determination and feature selection of support vector machines Shih-Wei Lin, Kuo-Ching Ying, Shih-Chieh Chen,
Machine Learning Week 1, Lecture 2. Recap Supervised Learning Data Set Learning Algorithm Hypothesis h h(x) ≈ f(x) Unknown Target f Hypothesis Set 5 0.

Machine Learning Week 1, Lecture 2. Recap Supervised Learning Data Set Learning Algorithm Hypothesis h h(x) ≈ f(x) Unknown Target f Hypothesis Set 5 0.
Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.

Content Based Image Clustering and Image Retrieval Using Multiple Instance Learning Using Multiple Instance Learning Xin Chen Advisor: Chengcui Zhang Department.
Hybridization of Search Meta-Heuristics Bob Buehler.

Hybridization of Search Meta-Heuristics Bob Buehler.
Reduced Support Vector Machine

Reduced Support Vector Machine
Ensemble Learning: An Introduction

Ensemble Learning: An Introduction
Ant Colony Optimization Optimisation Methods. Overview.

Ant Colony Optimization Optimisation Methods. Overview.
Bioinformatics Challenge  Learning in very high dimensions with very few samples  Acute leukemia dataset: 7129 # of gene vs. 72 samples  Colon cancer.

Bioinformatics Challenge  Learning in very high dimensions with very few samples  Acute leukemia dataset: 7129 # of gene vs. 72 samples  Colon cancer.
Ant Colony Optimization Algorithms for the Traveling Salesman Problem ACO 3.1-3.5 Kristie Simpson EE536: Advanced Artificial Intelligence Montana State.

Ant Colony Optimization Algorithms for the Traveling Salesman Problem ACO Kristie Simpson EE536: Advanced Artificial Intelligence Montana State.
Ant Colony Optimization: an introduction

Ant Colony Optimization: an introduction
Ant Colony Optimization (ACO): Applications to Scheduling

Ant Colony Optimization (ACO): Applications to Scheduling
A hybrid method for gene selection in microarray datasets Yungho Leu, Chien-Pan Lee and Ai-Chen Chang National Taiwan University of Science and Technology.

A hybrid method for gene selection in microarray datasets Yungho Leu, Chien-Pan Lee and Ai-Chen Chang National Taiwan University of Science and Technology.
Attention Deficit Hyperactivity Disorder (ADHD) Student Classification Using Genetic Algorithm and Artificial Neural Network S. Yenaeng 1, S. Saelee 2.

Attention Deficit Hyperactivity Disorder (ADHD) Student Classification Using Genetic Algorithm and Artificial Neural Network S. Yenaeng 1, S. Saelee 2.

Similar presentations

Ads by Google