Presentation is loading. Please wait.

Presentation is loading. Please wait.

Advanced Methods and Analysis for the Learning and Social Sciences

Published byStella Byrd Modified over 6 years ago

Similar presentations

Presentation on theme: "Advanced Methods and Analysis for the Learning and Social Sciences"— Presentation transcript:

1 Advanced Methods and Analysis for the Learning and Social Sciences
PSY505 Spring term, 2012 March 12, 2012

2 Today’s Class Clustering

3 Clustering You have a large number of data points
You want to find what structure there is among the data points You don’t know anything a priori about the structure Clustering tries to find data points that “group together”

4 Clustering What types of questions could you study with clustering?

5 Related Topic Factor Analysis Not the same as clustering
Factor analysis finds how data features/variables/items group together Clustering finds how data points/students group together In many cases, one problem can be transformed into the other But conceptually still not the same thing Next class!

6 Trivial Example Let’s say your data has two variables
Pknow Time Clustering works for (and is equally effective in) large feature spaces

7 +3 -3 time pknow

8 k-Means +3 -3 time pknow

9 Not the only clustering algorithm
Just the simplest

10 How did we get these clusters?
First we decided how many clusters we wanted, 5 How did we do that? More on this in a minute We picked starting values for the “centroids” of the clusters… Usually chosen randomly

11 How did we get these clusters?
First we decided how many clusters we wanted, 5 How did we do that? More on this in a minute We picked starting values for the “centroids” of the clusters… For instance…

12 +3 -3 time pknow

13 Then… We classify every point as to which centroid it’s closest to
This defines the clusters This creates a “voronoi diagram”

14 +3 -3 time pknow

15 Then… We re-fit the centroids as the center of the points in the cluster

16 +3 -3 time pknow

17 Then… Repeat until the centroids stop moving

18 +3 -3 time pknow

19 +3 -3 time pknow

20 +3 -3 time pknow

21 +3 -3 time pknow

22 +3 -3 time pknow

23 Questions? Comments?

24 What happens? What happens if your starting points are in strange places? Not trivial to avoid, considering the full span of possible data distributions

25 What happens? What happens if your starting points are in strange places? Not trivial to avoid, considering the full span of possible data distributions There is some work on addressing this problem

26 +3 -3 time pknow

27 +3 -3 time pknow

28 Solution Run several times, involving different starting points
cf. Conati & Amershi (2009)

29 Questions? Comments?

30 How many clusters should you have?
Can you use goodness of fit metrics?

31 Mean Squared Deviation (also called Distortion)
MSD = Take each point P Find the center of P’s cluster C Find the distance D from C to P Square D to get D’ Sum all D’ to get MSD

32 Any problems with MSD?

33 Any problems with MSD? More clusters almost always leads to smaller MSD Distance to nearest cluster center should always be smaller with more clusters

34 Questions? Comments?

35 What about cross-validation?
Will that fix the problem?

36 What about cross-validation?
Not necessarily This is a different problem than classification You’re not trying to predict specific values You’re determining whether any center is close to a given point More clusters cover the space more thoroughly So MSD will often be smaller with more clusters, even if you cross-validate

37 An Example 14 centers, ill-chosen (what you’d get on a cross-validation with too many centers) 2 centers, well-chosen (what you’d get on a cross-validation with not enough centers)

38 +3 -3 time pknow

39 +3 -3 time pknow

40 An Example The ill-chosen 14 centers will achieve a better MSD than the well-chosen 2 centers

41 Solution Penalize models with more clusters, according to how much extra fit would be expected from the additional clusters

42 Solution Penalize models with more clusters, according to how much extra fit would be expected from the additional clusters What comes to mind?

43 Solution Penalize models with more clusters, according to how much extra fit would be expected from the additional clusters Common approach – the Bayesian Information Criterion

44 Bayesian Information Criterion (Raftery, 1995)
Assesses how much fit would be spuriously expected from a random N parameters Assesses how much fit you actually had Finds the difference

45 Bayesian Information Criterion (Raftery, 1995)
The math is… painful See Raftery, 1995; many statistical packages can also compute this for you

46 So how many clusters? Try several values of k
Find “best-fitting” set of clusters for each value of k Choose k with best value of BiC

47 Questions? Comments?

48 Let’s do a set of hands-on exercises
Apply k-means using the following points and centroids Everyone gets to volunteer!

49 +3 -3 time pknow

50 +3 -3 time pknow

51 +3 -3 time pknow

52 +3 -3 time pknow

53 +3 -3 time pknow

54 +3 -3 time pknow

55 +3 -3 time pknow

56 +3 -3 time pknow

57 +3 -3 time pknow

58 +3 -3 time pknow

59 Comments? Questions?

60 Other cluster algorithms
Gaussian Mixture Models Can do fun things like Overlapping clusters Assigning points to no clusters

61 Gaussian Mixture Models
A centroid and a radius Fit with the same approach (some subtleties on process for selecting radius)

62 +3 -3 time pknow

63 Subtlety GMM still assigns every point to a cluster, but has a threshold on what’s really considered “in the cluster” Used during model calculation

64 +3 -3 Mathematically in red cluster, but outside threshold time pknow

65 Can you select appropriate Gaussian Mixture Models?
Everyone gets to volunteer!

66 5 centroids +3 -3 time pknow

67 5 centroids +3 -3 time pknow

68 5 centroids +3 -3 time pknow

69 5 centroids +3 -3 time pknow

70 3 centroids +3 -3 time pknow

71 4 centroids +3 -3 time pknow

72 4 centroids +3 -3 time pknow

73 4 centroids +3 -3 time pknow

74 4 centroids +3 -3 time pknow

75 5 centroids +3 -3 time pknow

76 1 centroid +3 -3 time pknow

77 Questions? Comments?

78 Assessment Can assess with same approaches as before Plus

79 Likelihood (more commonly, log likelihood)
The probability of the data occurring, given the model Assesses each point’s probability, given the set of clusters, adds it all together

80 For instance… +3 -3 Very unlikely point Likely points
-3 Very unlikely point Likely points Less likely points time pknow

81 Disadvantages of GMMs Much slower to create

82 Questions? Comments?

83 Advanced Clustering

84 Spectral Clustering

85 Spectral Clustering +3 -3 time pknow

86 Spectral Clustering Conducts clustering through dimensionality reduction Mathematically equivalent to K-means clustering on a non-linear dimension-reduced space

87 Hierarchical Clustering
Clusters can contain sub-clusters Image from Perera et al. (2009)

88 Hierarchical Agglommerative Clustering (HAC)
Each data point starts as its own cluster Two clusters are combined if the resulting fit is better Continue until no more clusters can be combined

89 Hierarchical Agglommerative Clustering (HAC)
What are the advantages of this method relative to traditional clustering? In what applications might it make more sense?

90 Applications of Clustering

91 Conati & Amershi (2009) Split students by their behaviors in an exploratory learning environment, then looked at learning outcomes Students with lower learning – 2 groups Students with better learning – 1 group

92 Conati & Amershi (2009) Students with lower learning (both groups) tended to move through the environment faster, and did not pause to reflect after moves Failure to Self-Explain

93 Conati & Amershi (2009) The two groups of students with lower learning differed in other aspects Amount of pause after back-tracking Degree to which students adjusted graph But this (apparently) did not affect learning

94 Beal, Qu, & Lee (2006) Clustered students by their behaviors in an intelligent tutor

95 Interestingly… (at minimum to me)
One of the clusters was essentially Gaming the System

96 These Findings These findings establish that gaming the system and failure to self-explain (shallow learning strategies) are Widely occurring Easily distinguishable from other behavior

97 Cluster Analysis In these two cases, the phenomena were already known
But it’s quite possible that this was a faster route to discovering them within a specific learning environment than observational methods And certainly a fast route to having a model of them within the specific context

98 Important points… If you cluster in a well-known domain, you are likely to obtain well-known findings

99 Because of this… Clustering is relatively popular
But relatively prone to uninteresting papers in education research Where usually a lot is already known So use with caution!

100 Asgn. 7 Let’s go over 3 solutions Sweet Zak Mike Wixon

101 Asgn. 8 Questions? Comments?

102 Next Class Friday, March 16 SPECIAL SESSION 3pm-5pm
SSPS Conference Room Clustering Readings Witten, I.H., Frank, E. (2005)Data Mining: Practical Machine Learning Tools and Techniques. Sections 4.8, 6.6. Assignments Due: 7. Clustering

103 The End

Download ppt "Advanced Methods and Analysis for the Learning and Social Sciences"

Similar presentations

Clustering k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein.

Clustering k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein.
Advanced Methods and Analysis for the Learning and Social Sciences PSY505 Spring term, 2012 March 12, 2012.

Advanced Methods and Analysis for the Learning and Social Sciences PSY505 Spring term, 2012 March 12, 2012.
PARTITIONAL CLUSTERING

PARTITIONAL CLUSTERING
ANALYZING MORE GENERAL SITUATIONS UNIT 3. Unit Overview  In the first unit we explored tests of significance, confidence intervals, generalization, and.

ANALYZING MORE GENERAL SITUATIONS UNIT 3. Unit Overview  In the first unit we explored tests of significance, confidence intervals, generalization, and.
Advanced Methods and Analysis for the Learning and Social Sciences PSY505 Spring term, 2012 February 27, 2012.

Advanced Methods and Analysis for the Learning and Social Sciences PSY505 Spring term, 2012 February 27, 2012.
Model Assessment, Selection and Averaging

Model Assessment, Selection and Averaging
Clustering… in General In vector space, clusters are vectors found within  of a cluster vector, with different techniques for determining the cluster.

Clustering… in General In vector space, clusters are vectors found within  of a cluster vector, with different techniques for determining the cluster.
© University of Minnesota Data Mining for the Discovery of Ocean Climate Indices 1 CSci 8980: Data Mining (Fall 2002) Vipin Kumar Army High Performance.

© University of Minnesota Data Mining for the Discovery of Ocean Climate Indices 1 CSci 8980: Data Mining (Fall 2002) Vipin Kumar Army High Performance.
Clustering.

Clustering.
Lecture 4 Unsupervised Learning Clustering & Dimensionality Reduction

Lecture 4 Unsupervised Learning Clustering & Dimensionality Reduction
Adapted by Doug Downey from Machine Learning EECS 349, Bryan Pardo Machine Learning Clustering.

Adapted by Doug Downey from Machine Learning EECS 349, Bryan Pardo Machine Learning Clustering.
KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.

KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.
Clustering Ram Akella Lecture 6 February 23, 2011 1290& 280I University of California Berkeley Silicon Valley Center/SC.

Clustering Ram Akella Lecture 6 February 23, & 280I University of California Berkeley Silicon Valley Center/SC.
Evaluating Performance for Data Mining Techniques

Evaluating Performance for Data Mining Techniques
Methods in Medical Image Analysis Statistics of Pattern Recognition: Classification and Clustering Some content provided by Milos Hauskrecht, University.

Methods in Medical Image Analysis Statistics of Pattern Recognition: Classification and Clustering Some content provided by Milos Hauskrecht, University.
COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.

COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.
CSC321: Neural Networks Lecture 12: Clustering Geoffrey Hinton.

CSC321: Neural Networks Lecture 12: Clustering Geoffrey Hinton.
Chapter 9 – Classification and Regression Trees

Chapter 9 – Classification and Regression Trees
Core Methods in Educational Data Mining HUDK4050 Fall 2014.

Core Methods in Educational Data Mining HUDK4050 Fall 2014.
Data Mining Practical Machine Learning Tools and Techniques Chapter 4: Algorithms: The Basic Methods Section 4.7: Instance-Based Learning Rodney Nielsen.

Data Mining Practical Machine Learning Tools and Techniques Chapter 4: Algorithms: The Basic Methods Section 4.7: Instance-Based Learning Rodney Nielsen.

Similar presentations

Ads by Google