1. Analyzing Attacks on SLT- based Techniques: Novelty Detection Blaine Nelson, Marco Barreno, Russell Sears, Anthony Joseph {barreno, nelsonb, sears, adj}@cs.berkeley.edu

2. Motivation Learning techniques are becoming more widely used in security-sensitive applications. Relatively little attention has been paid to analyzing the behavior of Statistical Learners when influenced by an attacker. How much of a threat is an attacker to statistical learning techniques?

3. Categories of Attacks  Does it matter which points are misclassified? Yes – “Specific” No – “Numbing”  What sort of errors does the attack cause? Incorrect Acceptance – “Dodging” Incorrect Rejection – “Denial of Service”  Does the attack affect learning directly? Yes – “Indoctrination” No – “Analysis”

4. Novelty Detection  Novelty detection is an important component in many applications where: there is an abundance of normal data while abnormal (e.g. failure) data is scarce. even if abnormal data is available, abnormality is not easily characterized.

5. Types of Novelty Detectors Naïve HypersphereMean-Centered Minimal Minimally Enclosing One-Class SVM

6. Fooling Mean-Centered Approaches Attack: Shift the mean of a hypersphere Assumptions:  Learner: Mean-centered, Fixed Radius  Training Policy: Bootstrapping, no Aging  Attacker: Knows Destination & State of Learner

7. Finding the Optimal Attack M total points T attack iterations D(A*) is the distance the mean is shifted. A* is the optimal attack strategy with sequence of attack points A* = {a t *}

8. Physics Analogy   =3   =4 X0X0 X1X1 X2X2 D({  1,  2 }) R  2 =4 blocks  1 =3 blocks D({  1,  2 })  1 ) X0X0 X1X1 X2X2 Transform X R 2R X A* = {at*} as Stacking Blocks

9. Unconstrained Optimal  D * (M)=1.63 blocks  blocks      Finding the Optimal Solution The physics analogy reveals the unrestricted optimal solution; the block spacing follows the harmonic sequence.

10. Refined Physics Analogy Stacking Variable Weighted Blocks T=4 blocks D T * (M)=1.38 blocks  kg  4 =8.19 kg  3 =3.40 kg  2 =1.41 kg  1 =1 kg   To constrain the duration of the attack, the analogy becomes one of stacking blocks of varying weight and choosing the weights for optimal stacking.

11. Alternative Formulation: Reformulate as Total Cumulative Mass Total Mass (M t ) – the sum of all mass used up to and including iteration t: The optimal solution yielded by the total mass formulation:

12. Ideas for Countering Attacks A Game-Theoretic Approach Identify policies for retraining:  Revise the learner’s retraining strategy.  Bootstrapping Policy – Retrain only on data identified as “normal” by the novelty detector. Introduces bias into the training set and thereby misrepresents the support of the distribution. Censor data based on location (Censoring):  Analysis of the statistical properties of distribution’s biased by the choice of the training set.

13. Conclusion Providing security-analyses for learning applications is essential as such applications are incorporated into security-sensitive environments The simplified model allowed for a rigorous analysis of optimal attack strategies. This sort of analysis can be extended in more realistic ways. We need to perform a rigorous analysis on potential countermeasures and their statistical consequences.