Learning Analytics NEASC 126 th Annual Meeting and Conference December 2011

Malcolm Brown, EDUCAUSE Learning Initiative Dr. Becky Wai-Ling Packard Mt. Holyoke College Johann Ari Larusson Brandeis University

What’s ahead 1.Football 2.What is Learning Analytics? 3.Examples 4.Possible benefits 5.Potential perils and pitfalls 3

4

Run a play Analyze results Run a play Analyze results Etc. 5

Run a play Analyze results Run a play Analyze results Etc. 6

Run a play Analyze results Run a play Analyze results Etc. Conduct a course Analyze evaluations Conduct a course Analyze evaluations Etc. 7

Learning analytics enable real time interventions 8

What is Learning Analytics? 9 Compilation and analysis of student usage data… …to observe and understand learning behaviors… … to enable appropriate interventions.

10 SIS data LMS data Portfolio data Longitudinal data Comparative data Analysis Visualizations Reports Alerts

Audience 11 Instructor facing Student facing

Before & after 12

Before & after 13 Before LA What happened?

Before & after 14 Before LA After LA What happened? What is happening? What will happen?

Examples 15

Social Networks Adapting Pedagogical Practice (SNAPP)

18

19 Bloom’s taxonomy

Your questions 20 Is there a way to see when your students are “in” higher order learning modes? Is there a way to see what activities your students are typically doing when in higher order modes?

Learner dialog types 1.Disputational 2.Cumulative 3.Exploratory 21 Reasoned & equitable co-reasoning challenging evaluative knowledge sharing

Markers 22

23 ModeMarkers ChallengesBut if / have to respond / my view CritiquesHowever / I’m not sure / maybe ResourcesHave you read / more links EvaluationsGood example / good point ExplanationsMeans that / our goals JustificationsI mean/ /we learned / we observed ReasoningNext step / relates to / that’s why Other perspectives Agree / here is another / take your point

24 Bloom’s taxonomy

Learning analytics 25 Uses data (esp. learner-produced data) 1 1 Performs analysis of that data 2 2 Discovers information about the learners 3 3 Enables appropriate intervention 4 4

Possible benefits 1.Evidence-based decisions 2.Measures what students actually do 3.Identify who needs help 4.Identify course-wide patterns 26

Potential pitfalls 1.Privacy 2.Profiling 3.Information sharing 4.Data stewardship 5.Impoverished model of learning 27

Johann Ari Larusson Brandeis University

Point of Originality Example of an R&D Learning Analytics Project

The trend/problem Migration towards larger “gateway” courses Negative impact on the student’s learning process –Less useful for fostering higher order thinking Evaluating/monitoring the quality of students’ work –In terms of the depth of the students’ learning –Extremely time consuming, even in smaller classes –Also, the instructor’s work is self-reflective However –Technology, like blogs, extend the physical boundaries of the classroom, introduce and foster learning communities, even in larger classes –Automatically produce data (electronic form) that can be analyzed –Is in itself a black box but enables us to peek inside the black box

Point of Originality Automated analysis tool: –Via lexical analysis, track students’ language migration from mere paraphrase to mastery –Isolating the moment in time when students demonstrate the ability to explain concepts in their own words, their “point of originality” in time –Recreates the same cognitive activity that educators might ordinarily undergo –Not an automated grading tool Core components: –WordNet: arranges words by their conceptual-semantic and lexical relationships, notes similarity between two words that don’t have literally identical meanings. –Using input query terms (that relate to key course topics), algorithm calculates how far a student’s language has evolved to explain the course content.

Testing the hypothesis Data: –Co-blogging in 25+ interdisciplinary course (heavy reading list) –Correlating originality scores on blogs and papers to blogging activity –Grades assigned two years earlier by person not involved in the project. Some results: –More original co-blogging work leads to higher paper grades –Originality variance: Students with papers above avg. grade have a negative variance. They were at the height of understanding during blogging. –Higher exposure (to other students’ writing/dialogs) in the blogosphere leads to more original papers –Active participation: Making more contributions to the blogosphere yields higher paper originality scores

Dr. Becky Wai-Ling Packard Mt. Holyoke College

Student Retention- Our Coordination Process Identifying the Key People Selecting Appropriate Tools Access Data Sources

Data Sources and Access Grades and Course Schedule: Advisor Learning Disability and Mental Health: Dean Finaid; Transfer Status: Admission Disciplinary; Dorm Change Requests: Res Life Current Activities including Sports: Coach Midsemester Report: Professor, Advisor, Dean Goals, Accomplishments: Career Center

Midsemester Report Is students’ attendance satisfactory? No Written work required, turned in, quality acceptable? No Quality of the student’s work: BORDERLINE Instructor Comments: Jenny, we talked about your absences and missed assignments. As this is a fast-paced course, your absences and missed assignments have added up quickly and have had a negative influence on your overall grade. Please come and see me so we can figure out what is still possible for you. I’d really like to see you succeed.

Visual: Key People Initial Faculty Advisor Coach Pre Prof’l Advisor Coach Student Dept Liaison Student Advisor (dorm) Student Class Dean Career Center Profs Major Advisor

Discussion