1. Welcome to SPLICE Workshop 4.0 February 2019

2. What is SPLICE?
NSF support to project: “Collaborative Research: Community-Building and Infrastructure Design for Data-Intensive Research in Computer Science Education”
Ken Koedinger, CMU
Peter Brusilovsky, UPitt
Cliff Shaffer and Steve Edwards, Virginia Tech
Grant period: September 1, 2017 – February 29, 2020

3. Standards, Protocols, and Learning Infrastructure for Computing Education
Mission: support the CS Education community by supplying documentation and infrastructure to help with adopting shared standards, protocols, and tools. We promote:
Development and broader re-use of innovative learning content that is instrumented for rich data collection;
Formats and tools for analysis of learner data; and
Best practices to make large collections of learner data and associated analytics available to researchers in the CSE, data science, and learner science communities.

4. What Have We Accomplished So Far? (1)
Some infrastructure
Website:
Tutorials: LTI, Caliper
GitHub project
Google Group
Workshops
1.0: June 2017 in Pittsburg
2.0: February 2018 in Baltimore
3.0: August 2018 in Espoo, Finland

5. What Have We Accomplished So Far? (2)
Working Groups
Small Code Snapshots
Small Coding Exercise Representation
Packaging Curricular Materials
Collaborations!

6. PART I: Interoperability

7. Interoperability Issues: Vision
Lots of software artifacts available for use by instructors in a rich network
The artifacts generate learner analytics
The learner analytics data are shared to the various interested parties

8. Interoperability Issues: Problems
Tools are hard to integrate. (LTI?)
Incompatible learner analytics data (Caliper? Working groups?)
Hard to get analytics to interested parties (Caliper?)

9. Use Case 1 An LMS and tools Hub-and-spokes
LTI pretty well solves basic connection issues, everyone just needs to support
Still have issues with learner analytics (limited data transfer)

10. Use Case 2 eTextbook with tools talking to LMS
Examples: OpenDSA, MasteryGrid
eTextbook marshals resources, primes the LMS for access
These resources are other smart content, primarily exercises, like Code Workout, OpenDSA exercises, ACOS exercises, on and on
Can support access to 3rd party tools via LTI
But no delivery of data analytics to any interested members of the network
How to get the data from the exercise (invoked by Canvas) back to the eTextbook? For analysis, and for value-added processing like “late policy”

11. PART II: Data Formats and Analysis
All of our working groups are more-or-less working on formats
Only a few instances of shared analysis tools?

12. Part III: Access to Data

13. Access to Data: Vision
Various tool providers are generating rich data sets
Example: Web-CAT has been collecting data for years
Learning scientists can use data collected by others to test hypotheses

14. Access to Data: Problems
Even if we solve all issues of data formats and analysis tools, there are issues related to sharing data, due to privacy issues
Clearing with institutions charged with protecting privacy
Sanitizing data while preserving key relationships
Think about sanitizing program source code with comments!
We have a few collaborations sharing data and basic tools for things like anonymization.