1. Data mining and statistical learning, lecture 1b
Outline
The five pillars of data mining
Supervised and unsupervised learning
Data mining and statistical learning, lecture 1b

2. Data mining and statistical learning, lecture 1b
The process of
Selecting
Exploring
Modifying
Modeling
Assessing
large amounts of data to uncover previously unknown patterns
Data mining and statistical learning, lecture 1b

3. Data mining and statistical learning, lecture 1b
SEMMA
Sample the data by creating one or more data tables
Explore the data by searching for:
(i) anticipated relationships and trends;
(ii) unanticipated relationships and trends;
(iii) anomalies
Modify the data by transforming variables and combining existing variables into new variables
Model the data by searching for a combination of the data that reliably predicts a desired outcome
Assess the data by evaluating the usefulness and reliability of the findings from the data mining process
Data mining and statistical learning, lecture 1b

4. Sample the data and create data tables
Cases and variables
Objects and attributes
Data mining and statistical learning, lecture 1b

5. Data mining and statistical learning, lecture 1b
Examine anticipated relationships: electricity consumption and temperature
Data mining and statistical learning, lecture 1b

6. Data mining and statistical learning, lecture 1b
Examine the presence of outliers: Total nitrogen concentrations in Swedish rivers determined by two different methods
Data mining and statistical learning, lecture 1b

7. Data mining and statistical learning, lecture 1b
Modifying inputs
Transforming inputs or outputs
Combining existing variables into new variables:
Aggregating inputs
Reducing the dimension of the inputs
Data mining and statistical learning, lecture 1b

8. Model selection: credit scoring
Candidate predictors:
Age
Sex
Income
Marital status
Education
Savings
Loans
Payment records
Houseowner .
Subset selection aims to produce a model that is interpretable and has possibly lower prediction error
Data mining and statistical learning, lecture 1b

9. Bias, Variance and Model Complexity
High Bias
Low Variance
Low Bias
High Variance
Test sample
Prediction error
Training sample
Low
High
Model complexity
Data mining and statistical learning, lecture 1b

10. Data mining and statistical learning, lecture 1b
Supervised learning (prediction, classification)
We have a training set of data, in which we observe the outcome and feature measurements for a set of objects
Using this data we build a prediction model, or learner, which will enable us to predict the outcome for new unseen objects
Unsupervised learning (association analysis, clustering)
We observe only the features and have no measurements of the outcome.
Our task is to describe how the data are organized and clustered
Data mining and statistical learning, lecture 1b
Hastie, Tibshirani, and Friedman: The elements of statistical learning

11. Statistical learning problems – some examples
Supervised learning (prediction, classification)
Predict tomorrow’s electricity consumption, from weather forecasts and calendar records (season, weekday, holiday)
Identify the numbers in a handwritten ZIP code, from a digitized image
Unsupervised learning (association analysis)
Identify buying patterns that can be used to design sales promotions
Data mining and statistical learning, lecture 1b

12. Supervised learning: statistical terminology
Prediction of one or more outputs using observations of one or more inputs
Statistical terminology
Inputs = Predictors
Independent variables
Explanatory variables
Outputs = Responses
Dependent variables
Data mining and statistical learning, lecture 1b

13. Data mining and statistical learning, lecture 1b
Naming convention
Regression
Prediction of quantitative outputs using one or more inputs
Classification
Prediction of qualitative outputs using observations of one or more inputs
Data mining and statistical learning, lecture 1b

14. Prediction by learning from data
Assume that we have a data set
which shows the outcome (response) y for a set of investigated objects with features x1, …, xp
Prediction by learning from data implies that we derive a function
that can be used to foresee the outcome for new objects (with known or observed features)
Data mining and statistical learning, lecture 1b

15. Some major types of quantitative prediction models
Linear or nonlinear regression models with i.i.d. error terms
Time series regression models with stochastic noise
Transfer function models
Data mining and statistical learning, lecture 1b