2. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS eCommerce Technology 20-751 Data Mining

3. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Coping with Information Computerization of daily life produces data –Point-of-sale, Internet shopping (& browsing), credit cards, banks... –Info on credit cards, purchase patterns, product preferences, payment history, sites visited... Travel. One trip by one person generates info on destination, airline preferences, seat selection, hotel, rental car, name, address, restaurant choices... Data cannot be processed or even inspected manually

4. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Data Overload Only a small portion of data collected is analyzed (estimate: 5%) Vast quantities of data are collected and stored out of fear that important info will be missed Data volume grows so fast that old data is never analyzed Database systems do not support queries like –“Who is likely to buy product X” –“List all reports of problems similar to this one” –“Flag all fraudulent transactions” But these may be the most important questions!

5. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Data Mining “The key in business is to know something that nobody else knows.” — Aristotle Onassis “To understand is to perceive patterns.” — Sir Isaiah Berlin PHOTO: LUCINDA DOUGLAS-MENZIESLUCINDA DOUGLAS-MENZIES PHOTO: HULTON-DEUTSCH COLLHULTON-DEUTSCH COLL

6. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Data Mining Extracting previously unknown relationships from large datasets –summarize large data sets –discover trends, relationships, dependencies –make predictions Differs from traditional statistics –Huge, multidimensional datasets –High proportion of missing/erroneous data –Sampling unimportant; work with whole population Sometimes called –KDD (Knowledge Discovery in Databases) –OLAP (Online Analytical Processing)

7. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Taxonomy of Data Mining Methods Data Mining Methods Database Segmentation Predictive Modeling Decision Trees Neural Networks Naive Bayesian Branching criteria Deviation Detection Clustering K-Means Link Analysis Rule Associa tion Visualization SOURCE: WELGE & REINCKE, NCSA Text Mining Semantic Maps

8. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Predictive Modeling Objective: use data about the past to predict future behavior Sample problems: –Will this (new) customer pay his bill on time? (classification) –What will the Dow-Jones Industrial Average be on October 15? (prediction) Technique: supervised learning –decision trees –neural networks –naive Bayesian

9. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Predictive Modeling TridasVickieMike Honest Barney WaldoWally Crooked SOURCE: WELGE & REINCKE, NCSA Which characteristics distinguish the two groups?

10. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Learned Rules in Predictive Modeling Tridas Vickie Mike Honest = has round eyes and a smile SOURCE: WELGE & REINCKE, NCSA

11. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Rule Induction Example heighthaireyesclass shortblondblueA tallblondbrownB tallredblueA shortdarkblueB talldarkblueB tallblondblueA talldarkbrownB shortblondbrownB Data: Devise a predictive rule to classify a new person as A or B SOURCE: WELGE & REINCKE, NCSA

12. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Build a Decision Tree hair dark red blond short, blue = B tall, blue = B tall, brown= B {tall, blue = A } short, blue = A tall, brown = B tall, blue = A short, brown = B Completely classifies dark-haired and red-haired people Does not completely classify blonde-haired people. More work is required SOURCE: WELGE & REINCKE, NCSA

13. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Build a Decision Tree hair dark red blond short, blue = B tall, blue = B tall, brown= B {tall, blue = A } short, blue = A tall, brown = B tall, blue = A short, brown = B SOURCE: WELGE & REINCKE, NCSA eye bluebrown short = A tall = A tall = B short = B Decision tree is complete because 1. All 8 cases appear at nodes 2. At each node, all cases are in the same class (A or B)

14. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Learned Predictive Rules hair eyes B B A A dark red blond bluebrown SOURCE: WELGE & REINCKE, NCSA

15. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Decision Trees Good news: a decision tree can always be built from training data Any variable can be used at any level of the tree Bad news: every data point may wind up at a leaf (tree has not compressed the data) B hair B A height short tall eyes A hair blue B B blonde dark brown B blue blonde dark red 8 cases, 7 nodes. This tree has not summarized the data effectively

16. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Database Segmentation (Clustering) “The art of finding groups in data” Kaufman & Rousseeuw Objective: gather items from a database into sets according to (unknown) common characteristics Much more difficult than classification since the classes are not known in advance (no training) Examples: –Demographic patterns –Topic detection (words about the topic often occur together) Technique: unsupervised learning

17. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Clustering Example Are there natural clusters in the data (36,10), (12,8), (38,42), (13,6), (36,38), (16,9), (40,36), (35,19), (37,7), (39,8)?

18. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Clustering K-means algorithm To divide a set into K clusters Pick K points at random. Use them to divide the set into K clusters based on nearest distance Loop: –Find the mean of each cluster. Move the point there. –Redefine the clusters. –If no point changes cluster, done K-means demo Agglomerative clustering: start with N clusters & merge Agglomerative clustering demo

19. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Neural Networks Networks of processing units called neurons. This is the j th neuron: n INPUTS x 1, …, x n n WEIGHTS w 1j, …, w nj Neuron computes a linear function of the inputs 1 OUTPUT y j depends only on the linear function SOURCE: CONSTRUCTING INTELLIGENT AGENTS WITH JAVA Neurons are easy to simulate

20. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Neural Networks INPUTS: 1 PER INPUT LAYER NEURON INPUT LAYER HIDDEN LAYER OUTPUT LAYER OUTPUTS: 1 PER OUTPUT LAYER NEURON DISTINGUISHED OUTPUT (THE “ANSWER”)

21. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Neural Networks Learning through back-propagation 1. Network is trained by giving it many inputs whose output is known 2. Deviation is “fed back” to the neurons to adjust their weights 3. Network is then ready for live data SOURCE: CONSTRUCTING INTELLIGENT AGENTS WITH JAVA DEVIATION

22. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Neural Network Classification “Which factors determine a pet’s favorite food?” Breed = Mixed Owner’s age > 45 Owner’s sex = F food: Chum food: Mr. Dog Species = Dog

23. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Neural Network Demos Demo: Notre Dame football, Automated surveillance, Handwriting analyzerNotre Dame footballsurveillance Handwriting analyzer Financial applications: –Churning: are trades being instituted just to generate commissions? –Fraud detection in credit card transactions –Kiting: isolate float on uncollected funds –Money Laundering: detect suspicious money transactions (US Treasury's Financial Crimes Enforcement Network) Insurance applications: –Auto Insurance: detect a group of people who stage accidents to collect on insurance –Medical Insurance: detect professional patients and ring of doctors and ring of references

24. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Rule Association Try to find rules of the form IF THEN (This is the reverse of a rule-based agent, where the rules are given and the agent must act. Here the actions are given and we have to discover the rules!) Prevalence = probability that LHS and RHS occur together (sometimes called “support factor,” “leverage” or “lift”) Predictability = probability of RHS given LHS (sometimes called “confidence” or “strength”)

25. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Association Rules from Market Basket Analysis  –prevalence = 4.99%, predictability = 22.89%  –prevalence = 0.94%, predictability = 28.14%  –prevalence = 2.11%, predictability = 38.22%  –prevalence = 1.36%, predictability = 41.02%  –prevalence = 1.16%, predictability = 38.01%

26. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Use of Rule Associations Coupons, discounts –Don’t give discounts on 2 items that are frequently bought together. Use the discount on 1 to “pull” the other Product placement –Offer correlated products to the customer at the same time. Increases sales Timing of cross-marketing –Send camcorder offer to VCR purchasers 2-3 months after VCR purchase Discovery of patterns –People who bought X, Y and Z (but not any pair) bought W over half the time

27. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Finding Rule Associations Example: grocery shopping For each item, count # of occurrences (say out of 100,000) apples 1891, caviar 3, ice cream 1088, pet food 2451, … Drop the ones that are below a minimum support level apples 1891, ice cream 1088, pet food 2451, … Make a table of each item against each other item: Discard cells below support threshold. Now make a cube for triples, etc. Add 1 dimension for each product on LHS.

28. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Rule Association Demos Magnum Opus (RuleQuest, free download)RuleQuest See5/C5.0 (RuleQuest, free download)RuleQuest Cubist numerical rule finder (RuleQuest, free download)RuleQuest IBM Interactive MinerInteractive Miner

29. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Text Mining Objective: discover relationships among people & things from their appearance in text Topic detection, term detection –When has a new term been seen that is worth recording? Generation of “knowledge map”, a graph representing terms/topics and their relationships SemioMap demo (Semio Corp.)SemioMap demo –Phrase extraction –Concept clustering (through co-occurrence) not by document –Graphic navigation (link means concepts co-occur) –Processing time: 90 minutes per gigabyte Summary server (inxight.com)Summary server

30. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Catalog Mining SOURCE: TUPAI SYSTEMSTUPAI SYSTEMS

31. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Visualization Objective: produce a graphic view of data so it become understandable to humans Hyperbolic trees SpotFire (free download from www.spotfire.com)www.spotfire.com SeeItIn3D TableLens OpenViz

32. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Major Ideas There’s too much data We don’t understand what it means It can be handled without human intervention Relationships can be discovered automatically

33. 20-751 ECOMMERCE TECHNOLOGY SUMMER 2000 COPYRIGHT © 2000 MICHAEL I. SHAMOS Q A &