Data Mining: An Introduction Billy Mutell

“The Library of Babel” Analogy Network of bookshelves with every book ever written All the books one could possibly imagine must exist somewhere in this library Books have titles like ‘Axaxxas mlo’, ‘The Bible’ & ‘Tomorrow's Winning Lottery Numbers’ Roughly 25 1,312,000 or x 10 1,834,097 volumes in library May be viewed as a metaphor for information in today’s society, where there’s growing amounts of data and, but not enough information

Content General Information Approaches to searching for information Project and plans

The nontrivial extraction of implicit, previously unknown, and potentially useful information from data The science of extracting useful information from large data sets or databases What is Data Mining?

With increased data, techniques needed to be created How Did it Evolve to What We Have Today? Information Retrieval Statistics Machine Learning Algorithms Database Management Data Mining

Practical Applications Government Intelligence Insurance Bank Finance Branch Evaluation Pharmaceutical Reactions in Patients

Content General Information Approaches to searching for information Project and plans

There are two models for mining data Predictive: Makes projected conclusions about values based on known results from different data Includes: Regression, Classification, Time Series Analysis Classification: Maps data into predefined groups Example: Identifying potential credit risks Time Series Analysis: Examining the value of an attribute as it varies over time Example: Choosing stocks

There are two models for mining data Descriptive: Identifies patterns or relationships in data Includes: Clustering, Association Rules, Sequence Discovery Clustering: Very similar to Classification, but groups are defined by data and not predefined Association Rules: Identifies specific types of data pairings Example: If someone buys jelly, they’re probably buying peanut butter Sequence Discovery: Highlights patterns on temporal sequences Example: If someone buys a CD player, they’ll probably buy CDs within a week

Statistical Based Algorithms Decision Tree Based Algorithms Rule Based Algorithms Distance Based Algorithms Information Analysis

Linear Regression Examples Regression- Estimation of output value based on input values; takes input data and fits it into a formula according to output

Statistical Based Algorithms By determining the regression coefficients {c 0, c 1, …, c n }, we can estimate the relationship the output parameter, y, and the input parameters, {x 1,…, x n }

Dead or Alive? Alive?Dead? Woman?Man? Non- Mathematician? Mathematician? Modern? Ancient? Pythagoras! Decision Tree Example: 20 Questions

Rule Based Algorithms Works well to perform classification through if-then analysis Trees have an implied order in which there is splitting; rules have no order

Parametric vs Nonparametric Models Parametric Model- Describes the relationship between input and output through algebraic equations where some parameters aren’t specified Nonparametric Model- Data driven and more appropriate for mining applications Creates models based on input while Parametric Methods assume models ahead of time More flexible than Parametric Models and generally easier to work with

Content General Information Approaches to searching for information Project and plans

Quest to improve customer/movie predictability through data mining and linear regression Teams win $1,000,000 prize Must beat Cinematch, Netflix’s current program to predict movie preferences NetFlix: A Case Study

What others have done so far: “If I have seen further, it is by standing on the shoulders of giants.” -Isaac Newton 1676 There are currently 31,443 contestants on 25,713 teams from 167 different countries. Important to remember that everyone is given the same amount of incomplete data, and we have to use that to predict rest of the data (unknown to us, known to Netflix) Current Leaders are from Budapest, Hungry and they’ve accurately predicted the data 8.7% better than Cinematch

K-Nearest Neighbor Algorithm (k-NN) A set of pairs is given, where the x i ’s take values in a metric space X upon which is defined a metric d and the θ i ’s take values in the set {1,2,…M} of possible classes. Each θ i is considered to be an index of the category to which the i th individual belongs, and each x i is the outcome of the set of measurements made upon that individual. A new pair ( x,θ ) is given, where only the measurement of x is observable, and it is desired to estimate θ by using information in the set of correctly classified points. Thus, we will call the nearest neighbor of x if The Nearest-Neighbor classification decision method gives to x the category θ’ n of its nearest neighbor x’ n

K-Nearest Neighbor Algorithm (k-NN) If k=3, we classify the dot as a triangle If k=5, we classify the dot as a rectangle x

NameGenderHeight)Output KristinaF1.6Short JimM2Tall MaggieF1.9Medium MarthaF1.88Medium StephanieF1.7Short BobM1.85Medium KathyF1.6Short DaveM1.7Short WorthM2.2Tall StevenM2.1Tall DebbieF1.8Medium ToddM1.95Medium KimF1.9Medium AmyF1.8Medium WynetteF1.75Medium Suppose we want to know what the entry would be classified as… Set K=5 and find the K nearest neighbors: => SHORT => MEDIUM Thus KNN would classify as SHORT

Take data from Netflix and sift through it Develop a function that maps non-linear data to a linear format so that it may be clustered and regressed Map data to matrices in R n Use Support Vector Machines to map input vectors to a higher dimensional space where a maximal separating hyper-plane is constructed Create a way to interpret this data in the form of movie recommendations Also… Use k-NN Approach along with Latent Semantic Indexing techniques to analyze scripts and key thematic plots and look for correlations/clusters What I plan to do from here:

Questions?