Focused Reducts Janusz A. Starzyk and Dale Nelson

What Do We Know? Major Assumption ASSUMPTION: This is ALL we know Real World Model Sampled Data

Problem Size Dilemma …

Rough Set Tutorial Difference between rough sets and fuzzy sets Labeling data Remove duplicates/ambiguities What is a core? What is a reduct?

Rough Sets vs Fuzzy Sets Fuzzy Sets - How gray is the pixel Rough Sets - How big is the pixel

Example Sample HRR Data

Example Label Data Label 1 = Label 2.45 Labeling can be different for different columns/attributes Ranges can be different for different columns/attributes

Remove Ambiguities & Duplicates

Equivalence Classes E 1 ={1, 2, 3} E 2 ={4, 5} E 3 ={6} E 4 ={7} E 5 ={8}

Definitions Reduct - A reduct is a reduction of an information system which results in no loss of information (classification ability) by removing attributes (range bins). There may be one or many for a given information system) Core - A core is the set of attributes (range bins) which are common to all reducts.

Compute Core Signals 6 and 8 are ambiguous upon removal of Range Bin 1. Therefore, Range Bin 1 is part of core. Core - The range bins common to ALL reducts - The most essential range bins without which signals cannot be classified

Compute Core No ambiguous signals therefore, Range Bin 2 is NOT part of core.

Compute Core No ambiguous signals therefore, Range Bin 3 is NOT part of core.

Compute Core No ambiguous signals therefore, Range Bin 4 is NOT part of core.

Compute Reducts Range Bin 1 + Range Bin 2 Range Bin 1 and Range Bin 2 classify therefore, they belong to a reduct

Compute Reducts Range Bin 1 + Range Bin 3 Range Bin 1 and Range Bin 3 do not classify therefore, they do NOT belong to a reduct

Compute Reducts Range Bin 1 + Range Bin 4 Range Bin 1 and Range Bin 4 classify therefore, they belong to a reduct

Reduct Summary Range bins 1 and 2 are a reduct –Sufficient to classify all signals Range bins 1 and 4 are a reduct –Sufficient to classify all signals Range bins 1 and 3 are NOT a reduct –Cannot distinguish target classes 2 and 3 No need to try –Range bins 1, 2, 3 –Range bins 1, 2, 4

Did You Notice? Calculating a reduct is time consuming! n = 29 value = 536,870,911 We are interested in n  50 This is a BIG NUMBER requiring a lot of time to compute reduct which is a f (# signals), too

Why Haven’t Rough Sets Been Used Before?

The Procedure Normalize signal Partition signal –Block –Interleave Wavelet transform Binary multi-class entropy labeling Entropy based range bin selection Determine minimal reducts Fuse marginal reducts for classification

Data Synthetic generated by XPATCH Six targets –1071 Signals per target –128 Range bins/signal –Azimuth -25 o to +25 o –Elevation -20 o to 0 o

Normalize the Data Ensures all data is range normalized Use the 2 Norm Divide each signal bin value by N

Partition the Signal Block Partitioning

Partition the Signal Interleave Partitioning 1st2nd3rd4th5th6th7th8th 1 Piece 2 Pieces 4 Pieces 8 Pieces

Why Use a Wavelet Transform? Original Signal Best- 20/60 signals Classified Best Wavelet 50/60 Signals Classified!! Many features are better than the best from original signal

HRR Signal and Its Haar Transform

Multi-Class Information Entropy Let x i be range bin values across all signals for a target class Define Without assuming any particular distribution we can define the probability as: Using this definition we define two other probabilities where Then multi-class entropy is defined as:

Binary Multi-Class Labeling

Range Bin Selection Total range bins available depends on partition size We chose 50 bins per reduct –Time considerations –Implications Based on maximum relative entropy

Compute Core Computation of core is easy and fast –Eliminate one range bin at a time and see if the training set is ambiguous - only that range bin can discriminate between the ambiguous signals –Accumulate the bins resulting in ambiguous data - that is the core These range bins MUST be in every reduct O(n) process

Compute Minimal Reducts To the core add one range bin at a time and compute the number of ambiguities Select the range bin(s) with the fewest ambiguities-there may be several-save these as we will use them to compute the reduct Add that range bin to the core and repeat previous step until there are no ambiguities - this is a reduct Calculate reducts for all bins with equivalent number of ambiguities-yields multiple reducts O(n 2 ) process

Time Complexity Training Set Size 50 to 400 Attributes (Range Bins) 1602 Signals Test Set Size 4823 Signals Need 50

Fuzzy Rough Set Classification Test signals may have a range bin value very close to labeling division point If this happens we define a distance  where this is considered a “don’t care” region Classification process proceeds without the “don’t care” range bin

Weighting Formula Requirements We desire the following for combining classifications –All Pcc(s) = 0  weight = 0 –All Pcc(s) = 1  weight = 1 –Several low Pcc(s)  weight higher than any of the Pcc(s) –One high Pcc and several low Pcc(s)  weight higher than the highest Pcc

Weighting Formula

Fusing Marginal Reducts Each signal is marked with the classification by each reduct along with the reduct’s performance (Pcc) on the training set A weight is computed for each target class for each signal A signal is assigned the target class with the highest weight

Results - Training

Results Testing

Conjectures Robust in the presence of noise –Due to binary labeling –Due to fuzzification Robust to signal registration –Due to binary labeling –Due to averaging effect of wavelets on interleaved partitions –Due to fuzzification

Rough Set Theoretic HRR ATR - Summary 01TIME METHOD -Normalize Signal -Partition Signal - Block - Interleave -Wavelet Transform -Binary Multi-class Entropy Labeling -Entropy based Range Bin Selection -Determine Minimal Reducts -Fuse marginal reducts for classification BREAKTHROUGHS -Reduct (classifier) generation time from exponential to quadratic ! -Fusion of marginal (poor performing) reducts -Wavelet Transform Aiding -Multi partition to increase number of range bins considered -Use of binary multi-class entropy labeling -Entropy based range bin selection -Performance within 1% of theoretic best -Max problem size increased by 2 orders of magnitude APPLICATIONS -1-D Signals -HRR -LADAR vibration -Sonar -Medical -Stock market -Data Mining Quadratic Exponential

Future Directions Fuzz factor sensitivity study Sensitivity to signal alignment Sensitivity to noise Iterated wavelet transform performance study Effectiveness on air to ground targets Other application areas