Three New Ideas in SDP-based Manifold Learning Alexander Gray Georgia Institute of Technology College of Computing FASTlab: Fundamental Algorithmic and Statistical Tools

The FASTlab Fundamental Algorithmic and Statistical Tools Laboratory 1.Arkadas Ozakin: Research scientist, PhD Theoretical Physics 2.Dong Ryeol Lee: PhD student, CS + Math 3.Ryan Riegel: PhD student, CS + Math 4.Parikshit Ram: PhD student, CS + Math 5.William March: PhD student, Math + CS 6.James Waters: PhD student, Physics + CS 7.Hua Ouyang: PhD student, CS 8.Sooraj Bhat: PhD student, CS 9.Ravi Sastry: PhD student, CS 10.Long Tran: PhD student, CS 11.Michael Holmes: PhD student, CS + Physics (co-supervised) 12.Nikolaos Vasiloglou: PhD student, EE (co-supervised) 13.Wei Guan: PhD student, CS (co-supervised) 14.Nishant Mehta: PhD student, CS (co-supervised) 15.Wee Chin Wong: PhD student, ChemE (co-supervised) 16.Abhimanyu Aditya: MS student, CS 17.Yatin Kanetkar: MS student, CS 18.Praveen Krishnaiah: MS student, CS 19.Devika Karnik: MS student, CS 20.Prasad Jakka: MS student, CS

10 sample tasks “Find engines like this one” (querying) “Plot the distribution of engine sizes and emissions” (density estimation) “Predict the lifetime maintenance cost” (regression) “Predict existence of fault or not” (classification) “Predict the number of failures next year” (time series analysis) “Show all engines on a 2-d plot” (dimension reduction) “Show or remove the unusual engines” (outlier detection) “Show the different types of engines” (clustering) “Is this group equivalent to this group?” (two-sample testing) “What’s the best action to take based on this behavior?” (reinforcement learning/control) Types of data: Sensor measurements Documents Database records, etc.

Rankmap Can do manifold learning using only ordinal data

Isometric Separation Maps Preserve class proximity

Density-Preserving Maps Preserve densities, not distances

The problem: big datasets D N M Could be large: N (#data), D (#features), M (#models)

Dual-tree All-nearest-neighbors O(N2)  O(N)

Rank-approximate Nearest-neighbor Search Distance approximation  rank approximation

Multi-scale Decompositions e.g. kd-trees [Bentley 1975], [Friedman, Bentley & Finkel 1977],[Moore & Lee 1995] How can we compute these efficiently?

A kd-tree: level 1

A kd-tree: level 2

A kd-tree: level 3

A kd-tree: level 4

A kd-tree: level 5

A kd-tree: level 6

Some application highlights Our software is being put into the pipelines of the world’s massive- scale science projects –Astronomy sky surveys (LSST, Pan- STARRS, DES): 1B objects/month –Large Hadron Collider: 1M events/sec

Some application highlights Others –McAfee spam blacklisting: 300M s/day –Supermarket demand forecasting –Algorithmic trading –Audio fingerprint matching –Legal document browsing and search

Software MLPACK (C++) –First scalable comprehensive ML library MLPACK-db –fast data analytics in relational databases (SQL Server) MLPACK Pro - Very-large-scale data