EMIS 8381 – Spring Netflix and Your Next Movie Night Nonlinear Programming Ron Andrews EMIS 8381

EMIS 8381 – Spring Elevator Pitch: Situation: Problem: You have finished your NLP homework, have some downtime but don’t know what movie to watch… Solution: Netflix’s collaborative filter (cf) based movie recommendation engine that utilizes NLP methods. Relevancy: Customer: –The algorithm makes more relevant movie recommendations than nonlinear methods –Better movie choices Company: –Decreased resource utilization (program complexity) –Increased customer retention.

EMIS 8381 – Spring 2012 Agenda: Netflix - Business of movie recommendation Collaborative filtering Linear Aspects Nonlinear Aspects Performance Improvement 3

EMIS 8381 – Spring 2012 Netflix - Business of movie recommendation 4 - Movie recommendation system: System that seeks to predict or anticipate a user’s preference for a film that has not been viewed by utilizing an algorithm that takes into consideration the collaborative nature of the website (user ratings via collaborative based filtering) -Utilizes linear, nonlinear and statistical methods

EMIS 8381 – Spring 2012 Collaborative filtering 5 Collaborative filtering is a process to make automated recommendations based upon crowd sourced information such as preference, taste and patterns. Two advantages: Wisdom of crowds Large Numbers Netflix collects four pieces of information from its users: User Movie Date of grade Grade Group unity

EMIS 8381 – Spring 2012 Linear Approach: 6 Computes a prediction for an item (i) using the weighted sum of items similar to i. Corresponding similarity s i,j Captures how users rate similar items

EMIS 8381 – Spring 2012 Linear Approach Cont.: 7 Traditionally: NxM modeled by NxC N= users M= movies Rows= user feature columns are movie feature vectors Low rank approximations can be found Data Sets are sparse Result: difficult non-convex problem approximation of gradient is difficult to approximate Need: nonlinear aspects.

EMIS 8381 – Spring 2012 Nonlinear approaches to movie recommendation: Nonlinear Principal Component Analysis (h-NLPCA) PCA – Principal Component Analysis –Well established data analysis technique –Transformation of recorded observations to produce independent score variables –Captures linear relationships well –Not sufficient to capture nonlinear patterns Introducing ANN: Artificial Neural Network (models defining a function f: X-> Y) Function approximated. 8 Nonlinear Component, Associative network

EMIS 8381 – Spring 2012 Nonlinear Approach: NLPCA ANN allows for mappings onto a reduced dimensional space. Relies on the SVD: Singular value decomposition (factorization of matrix – first step for CF) Example: 9 Variable of principal components Uniquely determined single value variances Conjugate transpose Hidden layer enables the function to perform nonlinear mapping functions from extraction : X -> Z to generate: Z -> ^X Associative network performs identify mapping, reducing the squared reconstruction error: ½||x^-x||^2

EMIS 8381 – Spring 2012 Nonlinear Approach: NLPCA Cont. Nonlinear principal component analysis provides: –Optimal nonlinear subspace spanned by components (different groups formed) –Constrains nonlinear hierarchical order of linear components –A minimum error between groups using conjugate gradient descent algorithm –N components explain the maximal variance –Tries to search for a k-dimensional subspace of minimal mean square error 10 Application to class: Is used to find a local minimum (not global) Works when function is quadratic (twice differentiable) = step size Update iteration Pi = nonlinear

EMIS 8381 – Spring 2012 Nonlinear Approach: NLPCA Algorithm Step 1: Data representation –Figure out missing values in original user item matrix (we know how to do this) Step 2: Low rank representation –Use conjugate gradient descent algorithm –Hierarchical error minimized Step 3: Neighborhood Formation –Calculate similarity between each user and his closest neighbors. –A = reconstructed matrix, r ij = rating of user u i on item i j. –Summations of l are calculated when both users (ua and ui) have rated a movie 11

EMIS 8381 – Spring 2012 Nonlinear Approach: NLPCA Algorithm Cont. Step 4: Prediction Generation –Matching of neighborhood to user 12 Neighborhood formation User ratings Original item average

EMIS 8381 – Spring 2012 Nonlinear effectiveness 13 Dimensions Accuracy Nonlinear can account for more variance. True accuracy:.7843

EMIS 8381 – Spring 2012 Nonlinear effectiveness: Conclusions Faster convergence Less resources For small data sets, i.e. not many film ratings, nonlinear provides better suggestions faster: More difficult to implement from a programming stand point 14

EMIS 8381 – Spring 2012 Questions? 15