(Rare) Category Detection Using Hierarchical Mean Shift Pavan Vatturi Weng-Keen Wong

1. Introduction Applications for surveillance, scientific discovery and data cleaning require anomaly detection Anomalies often identified as statistically unusual data points Many detected anomalies are simply uninteresting or correspond to known sources of noise

1. Introduction Known objects (99.9% of the data) Anomalies (0.1% of the data) Uninteresting (99% of anomalies) Interesting (1% of anomalies) Pictures from: Sloan Digital Sky Survey ( Pelleg, D. (2004). Scalable and Practical Probability Density Estimators for Scientific Anomaly Detection. PhD Thesis, Carnegie Mellon University.

1. Introduction Category Detection [Pelleg and Moore 2004]: human-in-the-loop exploratory data analysis Data Set Build Model Spot Interesting Data Points Ask User to Label Categories of Interesting Data Points Update Model with Labels

1. Introduction Data Set Build Model Spot Interesting Data Points Ask User to Label Categories of Interesting Data Points Update Model with Labels User can: Label a query data point under an existing category Or declare data point to belong to a previous undeclared category

1. Introduction Goal: present to user a single instance from each category in as few queries as possible Difficult to detect rare categories if class imbalance is severe Interested in rare categories for anomaly detection

Outline 1.Introduction 2.Related Work 3.Background 4.Methodology 5.Results 6.Conclusion / Future Work

2. Related Work Interleave [Pelleg and Moore 2004] Nearest-Neighbor-based active learning for rare-category detection for multiple classes [He and Carbonell 2008] Multiple output identification [Fine and Mansour 2006]

3. Background: Mean Shift [Fukunaga and Hostetler 1975] Reference data set Query point Center of Mass Mean shift vector (follows density gradient) Mean shift vector with kernel k

3. Background: Mean Shift [Fukunaga and Hostetler 1975] Reference data set Query point Center of Mass Convergence to cluster center

3. Background: Mean Shift Blurring Reference data set Query point Center of Mass Blurring: When query points are the same as the reference data set Progressively blurs the original data set

3. Background: Mean Shift End result of applying mean shift to a synthetic data set

4. Methodology: Overview 1.Sphere the data 2.Hierarchical Mean Shift 3.Query user

4. Methodology: Hierarchical Mean Shift Repeatedly blur data using Mean Shift with increasing bandwidth: h new = k * h old

4. Methodology: Querying the User The data point closest to the cluster center is the representative data point. Rank representative data points for querying to user according to: 1.Outlierness [Leung et al. 2000] for Cluster C i : Lifetime of C i = Log (bandwidth when cluster C i is merged with other clusters – bandwidth when cluster C i is formed)

4. Methodology: Querying the User Rank representative data points for querying to user according to: 2.Compactness + Isolation [Leung et al. 2000] for Cluster C i :

4. Methodology: Tiebreaker Ties may occur in Outlierness or Compactness/Isolation values. Highest Average Distance heuristic: choose representative data point with highest average distance from user-labeled points.

5. Results NameDimsRecordsClassesSmallest Class Largest Class Abalone %16% Shuttle %64.2% OptDigits %50% OptLetters %24% Statlog %50% Yeast %31.68% Shuttle, OptDigits, OptLetters, and Statlog were subsampled to simulate class imbalance. Data sets used in experiments

5. Results (Yeast) Category detection metric: # queries before user presented with at least one example from all categories

5. Results DatasetHMS-CIHMS- CI+HAD HMS-OutHMS- Out+HAD NNDMInterleave Abalone Shuttle OptDigits OptLetters Statlog Yeast Number of hints to discover all classes

5. Results DatasetHMS-CIHMS- CI+HAD HMS-OutNNDMInterleave Abalone Shuttle OptDigits OptLetters Statlog Yeast Area under the category detection curve

6. Conclusion / Future Work Conclusions –HMS-based methods consistently discover more categories in fewer queries than existing methods –Do not need apriori knowledge of dataset properties

6. Conclusion / Future Work Future Work Better use of user feedback Presentation of an entire cluster to the user instead of a representative data point Improved computational efficiency Theoretical analysis