DATA MINING LECTURE 1 INTRODUCTION TO DATA MINING

DATA MINING VESIT M.VIJAYALAKSHMI 2 Data Mining Outline –Introduction –Related Concepts –Data Mining Techniques

DATA MINING VESIT M.VIJAYALAKSHMI 3 Introduction Outline Define data mining Data mining vs. databases Basic data mining tasks Data mining development Data mining issues Goal: Provide an overview of data mining.

DATA MINING VESIT M.VIJAYALAKSHMI 4 Introduction Data is growing at a phenomenal rate Users expect more sophisticated information How? UNCOVER HIDDEN INFORMATION DATA MINING

DATA MINING VESIT M.VIJAYALAKSHMI 5 Data Mining Definition Finding hidden information in a huge store of data Fit data to a model Similar terms –Exploratory data analysis –Data driven discovery –Deductive learning

DATA MINING VESIT M.VIJAYALAKSHMI 6 What Is Data Mining? Data mining (knowledge discovery in databases): –Extraction of interesting ( non-trivial, implicit, previously unknown and potentially useful) information or patterns from data in large databases Alternative names and their “inside stories”: –Knowledge discovery(mining) in databases (KDD), knowledge extraction, data/pattern analysis, data archeology, data dredging, information harvesting, business intelligence, etc. What is not data mining? –(Deductive) query processing. – Expert systems or small ML/statistical programs

DATA MINING VESIT M.VIJAYALAKSHMI 7 Potential Applications Market analysis and management –target marketing, CRM, market basket analysis, cross selling, market segmentation Risk analysis and management –Forecasting, customer retention, quality control, competitive analysis Fraud detection and management Text mining (news group, , documents) and Web analysis. –Intelligent query answering

DATA MINING VESIT M.VIJAYALAKSHMI 8 Market Analysis and Management (1) Where are the data sources for analysis? –Credit card transactions, loyalty cards, discount coupons, customer complaint calls, –Target marketing (Find clusters of “model” customers who share the same characteristics: interest, income level, spending habits, etc.) Determine customer purchasing patterns over time Cross-market analysis –Associations/co-relations between product sales –Prediction based on the association information

DATA MINING VESIT M.VIJAYALAKSHMI 9 Market Analysis and Management (2) Customer profiling –data mining can tell you what types of customers buy what products (clustering or classification) Identifying customer requirements –identifying the best products for different customers –use prediction to find what factors will attract new customers Provides summary information –various multidimensional summary reports –statistical summary information (data central tendency and variation)

DATA MINING VESIT M.VIJAYALAKSHMI 10 Fraud Detection and Management Applications –widely used in health care, retail, credit card services, telecommunications (phone card fraud), etc. Approach –use historical data to build models of fraudulent behavior and use data mining to help identify similar instances Examples –auto insurance: detect a group of people who stage accidents to collect on insurance –money laundering: detect suspicious money transactions (US Treasury's Financial Crimes Enforcement Network) –medical insurance: detect professional patients and ring of doctors and ring of references

DATA MINING VESIT M.VIJAYALAKSHMI 11 Other Applications game statistics to gain competitive advantage Astronomy JPL and the Palomar Observatory discovered 22 quasars with the help of data mining IBM Surf-Aid applies data mining algorithms to Web access logs for market-related pages to discover customer preference and behavior pages, analyzing effectiveness of Web marketing, improving Web site organization, etc.

DATA MINING VESIT M.VIJAYALAKSHMI 12 Data Mining Algorithm Objective: Fit Data to a Model –Descriptive –Predictive Preference – Technique to choose the best model Search – Technique to search the data –“Query”

DATA MINING VESIT M.VIJAYALAKSHMI 13 Database Processing vs. Data Mining Processing Query –Well defined –SQL Query –Poorly defined –No precise query language Data Data – – Operational data Output Output – – Precise – Subset of database Data Data – – Not operational data Output Output – – Fuzzy – Not a subset of database

DATA MINING VESIT M.VIJAYALAKSHMI 14 Query Examples Database –Find all credit applicants with last name of Smith. –Identify customers who have purchased more than $10,000 in the last month. –Find all customers who have purchased milk Data Mining –Find all credit applicants who are poor credit risks. (classification) –Identify customers with similar buying habits. (Clustering) –Find all items which are frequently purchased with milk. (association rules)

DATA MINING VESIT M.VIJAYALAKSHMI 15 Data Mining: On What Kind of Data? Relational databases Data warehouses Transactional databases Advanced DB and information repositories –Object-oriented and object-relational databases –Spatial databases –Time-series data and temporal data –Text databases and multimedia databases –Heterogeneous and legacy databases –WWW

DATA MINING VESIT M.VIJAYALAKSHMI 16 Data Mining Models And Tasks

DATA MINING VESIT M.VIJAYALAKSHMI 17 Data Mining Tasks Prediction Methods –Use some variables to predict unknown or future values of other variables. Description Methods –Find human-interpretable patterns that describe the data. Concept description: Characterization and discrimination –Generalize, summarize, and contrast data characteristics, e.g., dry vs. wet regions

DATA MINING VESIT M.VIJAYALAKSHMI 18 Basic Data Mining Tasks Classification & Prediction maps data into predefined groups or classes Finds models (functions) that describe and distinguish classes or concepts for future prediction E.g., classify countries based on climate, or classify cars based on gas mileage Presentation: decision-tree, classification rule, neural network Prediction: Predict some unknown or missing numerical values 3 methods –Supervised learning –Pattern recognition –Prediction

DATA MINING VESIT M.VIJAYALAKSHMI 19 Basic Data Mining Tasks Regression –is used to map a data item to a real valued prediction variable. –Learning a function that best fits the target data Clustering –groups similar data together into clusters. –Class label is unknown: Group data to form new classes, e.g., cluster houses to find distribution patterns –Segmentation –Partitioning

DATA MINING VESIT M.VIJAYALAKSHMI 20 Basic Data Mining Tasks Summarization maps data into subsets with associated simple descriptions. –Characterization –Generalization Link Analysis uncovers relationships among data. –Affinity Analysis –Association Rules –age(X, “20..29”) ^ income(X, “20..29K”)  buys(X, “PC”) [support = 2%, confidence = 60%] –contains(T, “computer”)  contains(x, “software”) [1%, 75%] –Sequential Analysis determines sequential patterns.

DATA MINING VESIT M.VIJAYALAKSHMI 21 Sequence Discovery Given is a set of objects, with each object associated with its own timeline of events, find rules that predict strong sequential dependencies among different events. Rules are formed by first discovering patterns. Event occurrences in the patterns are governed by timing constraints. Patterns similar to association rules but the events are related by time

DATA MINING VESIT M.VIJAYALAKSHMI 22 Are All the “Discovered” Patterns Interesting? A data mining system/query may generate thousands of patterns, not all of them are interesting. Interestingness measures: A pattern is interesting if it is easily understood by humans, valid on new or test data with some degree of certainty, potentially useful, novel, or validates some hypothesis that a user seeks to confirm Objective vs. subjective interestingness measures: –Objective: based on statistics and structures of patterns, e.g., support, confidence, etc. –Subjective: based on user’s belief in the data, e.g., unexpectedness, novelty, etc.

DATA MINING VESIT M.VIJAYALAKSHMI 23 Can We Find All and Only Interesting Patterns? Find all the interesting patterns: Completeness –Association vs. classification vs. clustering Search for only interesting patterns: Optimization –Approaches First general all the patterns and then filter out the uninteresting ones. Generate only the interesting paterns

DATA MINING VESIT M.VIJAYALAKSHMI 24 Data Mining vs. KDD Knowledge Discovery in Databases (KDD): process of finding useful information and patterns in data. Data Mining: Use of algorithms to extract the information and patterns derived by the KDD process.

DATA MINING VESIT M.VIJAYALAKSHMI 25 Data Mining and Business Intelligence Increasing potential to support business decisions End User Business Analyst Data Analyst DBA Making Decisions Data Presentation Visualization Techniques Data Mining Information Discovery Data Exploration OLAP, MDA Statistical Analysis, Querying and Reporting Data Warehouses / Data Marts Data Sources Paper, Files, Information Providers, Database Systems, OLTP

DATA MINING VESIT M.VIJAYALAKSHMI 26 Visualization Techniques Graphical Geometric Icon-based Pixel-based Hierarchical Hybrid

DATA MINING VESIT M.VIJAYALAKSHMI 27 Data Mining: Confluence of Multiple Disciplines Data Mining Database Technology Statistics Other Disciplines Information Science Machine Learning Visualization

DATA MINING VESIT M.VIJAYALAKSHMI 28 Data Mining Development Similarity Measures Hierarchical Clustering IR Systems Imprecise Queries Textual Data Web Search Engines Bayes Theorem Regression Analysis EM Algorithm K-Means Clustering Time Series Analysis Neural Networks Decision Tree Algorithms Algorithm Design Techniques Algorithm Analysis Data Structures Relational Data Model SQL Association Rule Algorithms Data Warehousing Scalability Techniques

DATA MINING VESIT M.VIJAYALAKSHMI 29 Data Mining Issues Human Interaction Overfitting Outliers Interpretation Visualization Large Datasets High Dimensionality Multimedia Data Missing Data Irrelevant Data Noisy Data Changing Data Integration Application

DATA MINING VESIT M.VIJAYALAKSHMI 30 Major Issues in Data Mining (1) Mining methodology and user interaction –Mining different kinds of knowledge in databases –Interactive mining of knowledge at multiple levels of abstraction –Incorporation of background knowledge –Data mining query languages and ad-hoc data mining –Expression and visualization of data mining results –Handling noise and incomplete data –Pattern evaluation: the interestingness problem Performance and scalability –Efficiency and scalability of data mining algorithms –Parallel, distributed and incremental mining methods

DATA MINING VESIT M.VIJAYALAKSHMI 31 Major Issues in Data Mining (2) Issues relating to the diversity of data types –Handling relational and complex types of data –Mining information from heterogeneous databases and global information systems (WWW) Issues related to applications and social impacts –Application of discovered knowledge Domain-specific data mining tools Intelligent query answering Process control and decision making –Integration of the discovered knowledge with existing knowledge: A knowledge fusion problem –Protection of data security, integrity, and privacy

DATA MINING VESIT M.VIJAYALAKSHMI 32 Social Implications of DM Privacy Profiling Unauthorized use

DATA MINING VESIT M.VIJAYALAKSHMI 33 Data Mining Metrics Usefulness Return on Investment (ROI) Accuracy Space/Time

DATA MINING VESIT M.VIJAYALAKSHMI 34 Related Concepts Outline Database/OLTP Systems Fuzzy Sets and Logic Information Retrieval(Web Search Engines) Dimensional Modeling Data Warehousing OLAP/DSS Web Search Engined Statistics Machine Learning Pattern Matching Goal: Examine some areas which are related to data mining.

DATA MINING VESIT M.VIJAYALAKSHMI 35 DB & OLTP Systems Schema –(ID,Name,Address,Salary,JobNo) Data Model –ER –Relational Transaction Query: SELECT Name FROM T WHERE Salary > DM: Only imprecise queries output is a KDD object, say a rule a cluster or a classification

DATA MINING VESIT M.VIJAYALAKSHMI 36 Fuzzy Sets and Logic Fuzzy Set: Set membership function is a real valued function with output in the range [0,1]. f(x): Probability x is in F. 1-f(x): Probability x is not in F. EX: –T = {x | x is a person and x is tall} –Let f(x) be the probability that x is tall –Here f is the membership function DM: Prediction and classification are fuzzy.

DATA MINING VESIT M.VIJAYALAKSHMI 37 Information Retrieval Information Retrieval (IR): retrieving desired information from textual data. Library Science Digital Libraries Web Search Engines Traditionally keyword based Sample query: Find all documents about “data mining”. DM: Similarity measures; Mine text/Web data.

DATA MINING VESIT M.VIJAYALAKSHMI 38 Information Retrieval (cont’d) Similarity: measure of how close a query is to a document. Documents which are “close enough” are retrieved. Metrics: –Precision = |Relevant and Retrieved| |Retrieved| –Recall = |Relevant and Retrieved| |Relevant|

DATA MINING VESIT M.VIJAYALAKSHMI 39 IR Query Result Measures and Classification IR Classification

DATA MINING VESIT M.VIJAYALAKSHMI 40 Dimensional Modeling View data in a hierarchical manner more as business executives might Useful in decision support systems and mining Dimension: collection of logically related attributes; axis for modeling data. Facts: data stored Ex: Dimensions – products, locations, date Facts – quantity, unit price DM: May view data as dimensional.

DATA MINING VESIT M.VIJAYALAKSHMI 41 Relational View of Data

DATA MINING VESIT M.VIJAYALAKSHMI 42 Dimensional Modeling Queries Roll Up: more general dimension Drill Down: more specific dimension Dimension (Aggregation) Hierarchy SQL uses aggregation Decision Support Systems (DSS): Computer systems and tools to assist managers in making decisions and solving problems.

DATA MINING VESIT M.VIJAYALAKSHMI 43 Data Warehousing Operational Data: Data used in day to day needs of company. Informational Data: Supports other functions such as planning and forecasting. Data mining tools often access data warehouses rather than operational data. DM: May access data in warehouse & couls use OLAP queries

DATA MINING VESIT M.VIJAYALAKSHMI 44 Web Search Engines Web Search Engines are similar to IR systems Conventional Search Engines suffer from several problems –Abundance –Limited Coverage –Limited Query –Limited Customization Concept of “Web Mining”

DATA MINING VESIT M.VIJAYALAKSHMI 45 Statistics Simple descriptive models Statistical inference: generalizing a model created from a sample of the data to the entire dataset. Exploratory Data Analysis: –Data can actually drive the creation of the model –Opposite of traditional statistical view. Data mining targeted to business user DM: Many data mining methods come from statistical techniques.

DATA MINING VESIT M.VIJAYALAKSHMI 46 Machine Learning Machine Learning: area of AI that examines how to write programs that can learn. Often used in classification and prediction Supervised Learning: learns by example. Unsupervised Learning: learns without knowledge of correct answers. Machine learning often deals with small static datasets. DM: Uses many machine learning techniques.

DATA MINING VESIT M.VIJAYALAKSHMI 47 Pattern Matching (Recognition) Pattern Matching: finds occurrences of a predefined pattern in the data. Applications include speech recognition, information retrieval, time series analysis. DM: Type of classification.

DATA MINING VESIT M.VIJAYALAKSHMI 48 Data Mining Techniques Outline Statistical –Point Estimation –Models Based on Summarization –Bayes Theorem –Hypothesis Testing –Regression and Correlation Similarity Measures Decision Trees Neural Networks –Activation Functions Genetic Algorithms

DATA MINING VESIT M.VIJAYALAKSHMI 49 Similarity Measures Determine similarity between two objects. Similarity characteristics: Alternatively, distance measure measure how unlike or dissimilar objects are.

DATA MINING VESIT M.VIJAYALAKSHMI 50 Distance Measures Measure dissimilarity between objects

DATA MINING VESIT M.VIJAYALAKSHMI 51 Decision Trees Decision Tree (DT): –Tree where the root and each internal node is labeled with a question. –The arcs represent each possible answer to the associated question. –Each leaf node represents a prediction of a solution to the problem. Popular technique for classification; Leaf node indicates class to which the corresponding tuple belongs.

DATA MINING VESIT M.VIJAYALAKSHMI 52 Decision Tree Example

DATA MINING VESIT M.VIJAYALAKSHMI 53 Neural Networks Based on observed functioning of human brain. (Artificial Neural Networks (ANN) Our view of neural networks is very simplistic. We view a neural network (NN) from a graphical viewpoint. Alternatively, a NN may be viewed from the perspective of matrices. Used in pattern recognition, speech recognition, computer vision, and classification.

DATA MINING VESIT M.VIJAYALAKSHMI 54 Neural Network Example

DATA MINING VESIT M.VIJAYALAKSHMI 55 Genetic Algorithms Optimization search type algorithms. Creates an initial feasible solution and iteratively creates new “better” solutions. Based on human evolution and survival of the fittest. Must represent a solution as an individual. Individual: string I=I 1,I 2,…,I n where I j is in given alphabet A. Each character I j is called a gene. Population: set of individuals.

DATA MINING VESIT M.VIJAYALAKSHMI 56 Genetic Algorithms A Genetic Algorithm (GA) is a computational model consisting of five parts: –A starting set of individuals, P. –Crossover: technique to combine two parents to create offspring. –Mutation: randomly change an individual. –Fitness: determine the best individuals. –Algorithm which applies the crossover and mutation techniques to P iteratively using the fitness function to determine the best individuals in P to keep.