Teaching & Learning Research in the Era of Big Data Rebecca Petersen, Sr. Research Manager, HarvardX

Or: Rebecca’s collected thoughts on Research & MOOCs

Hi. We’re HX Research

We make sense of large data sets generated from open online learning environments

We do this through…. Producing rigorous scholarship in the field of learning and analytics Create tools such as Insights, and analytics dashboards for faculty and course teams Provide consultative support to HX faculty and course teams on data interpretation and experimental design Match researchers and course teams to work together

MOOC Big Data Hype

HarvardX & MITx * 54 Courses 2.1MM reg 1.1MM “viewers” ~40K reg/course * Year 2 Benchmark data of all courses from 2012-September 21, 2014

This is a talk about building a research community in the era of big data in teaching and learning

The day-to-day work Maven of Research Administration The Data Guy: Gets the Data clean and useable Amazing Post Docs who analyze the data and run studies

HarvardX Research Community HarvardX Faculty HarvardX Course Teams * Collaborate Evaluate Inspire Serve *

Our early day work is very much about sensemaking and wayfinding with the data through secondary analysis Who are the learners? How do they spend their time in the course? Why are they here?

Our Study Focus Areas Motivation Commitment Background Learning conditions

We also navigate the day-to-day tension of ambitious scope and ideals with course production realities

Research Vignettes Towards Research Enablement Discussion

Evidence from MOOCs supporting instructional #advancedcommonsense

A Cascade of Completion Rates: Years 1 and 2, Across 63 Courses Of survey responders who “do it all” 25% say that they are not interested in certification 50% say that are or have been teachers 20% say that are or have been teachers in this subject 74% have a Bachelor’s degree The median age is 32.

In the MOOC world: Small percentages still represent big numbers Carefully choose your denominator Practice meticulous methodology and know the literature

JusticeX Forum Experiment Careful Methodology

Choose your denominator Registrants v. participants when factoring completion

Toward Research Enablement: Policy, Protocol and Learner Data

Facilitating Collaboration

Legend Research Enablement: Proposal Decision Making Process 24 Proposal submitted Is it possible using Hx platform? Is there available capacity? Will it require substantial Hx resources? Can some version be done using existing data? Provide access to data (or other support) Aligned with priorities? Can the research be aligned with priorities? Execute! Connect to other resources, try to “schedule” Not aligned with priorities Not feasible Yes No #2#3 #1 Pending IRB approval The Research Committee, or a subcommittee, will review unsuccessful proposals to inform development of the edX platform (“not feasible”) as well as research priorities (“not aligned with priorities”)

On Privacy and Policy

How does your individual school treat MOOC learner data? How knowledgeable is your IRB about open online learning data? Will all of the research you do be scholarly (for publication) or is it more closely aligned with Institutional Research (IR)

The Future

Onward!