Caliper Event Integration Blackboard Learn & Kaltura Video Platform Dan Rinzel, Senior Product Manager Analytics @ Blackboard Jeff Rubenstein, Vice President Product Development @ Kaltura IMS Global Learning Consortium Learning Impact Leadership Institute May 2016

Agenda Project goals & development process Spec maturity & lessons learned from the leading edge Demonstration Key takeaways

Statements regarding our product development initiatives, including new products and future product upgrades, updates or enhancements represent our current intentions, but may be modified, delayed or abandoned without prior notice and there is no assurance that such offering, upgrades, updates or functionality will become available unless and until they have been made generally available to our customers.

The Challenge Lots of Data Currently accessible only through Kaltura

The Goal Integrated insights for teachers into what students are doing across tools and experiences Usage data first, progress data to come

Project Goals Overall Goal: prove out event interoperability between a launch system and a tool provider. Leverage existing event instrumentation (native or Caliper) in both systems Send instrumented events from the tool provider, including metric profile events not “known” to the launch system Leverage existing integration mechanics between systems, including LTI Extend the LTI launch handshake in the existing B2 securely to include Caliper semantics Leverage existing event capabilities in the launch system Integrate events correctly alongside “native” events in the launch system Use the launch system as (interim or final) Caliper LRS Mostly looking for the smallest practical delta between what we’re already doing in our separate data silos and what we could be doing with Caliper 1.0 spec Blackboard Learn SaaS certfied last July against the pre-release of the spec, and again in Octoober with the 1.0 release, for the metric profiles Assessment Event Assessment Item Event AssignableEvent Outcome Event Session Event Kaltura Application Framework also certified in October with the 1.0 release for the MediaEvent metric profile

Development Process Lightweight internal milestone deliverables (by month or quarter) – best fit with mainline dev schedules on both sides DevOps coordination of Test/Dev systems for integration testing Periodic check-in status meetings & email threads for real work Complexity was largely around fitting this project into other planned work with minimal disruption to schedule

Spec maturity and workarounds Caliper’s third actor – the Learning Record Store is neither a tool consumer nor a tool provider, but could have capabilities of either/both Complicates relays between potential event “receivers” Preserving identifiers from the launch system is not mandated Object life cycles on the launch system and the tool could impact the record store Event payload size and the describe operation Group membership and contextual relevance The relay thing is largely just on Blackboard side, but could also impact Kaltura sending events to a 3rd party event receiver directly We’ve worked around identifier preservation for this effort using object extensions, but that’s because we could negotiate specifics out-of-band during the co-development project If the launch system is responsible for responding to describe requests, how will we manage life cycle operations (removed/updated users or courses, for example) Sometimes the *current* group membership is critical to provide for some events (such as group assignment submission). We used extensions for this as well, but that’s not great for interoperability

xAPI Both Blackboard & Kaltura take the position that we don’t want customers to feel locked in to making tool choices or Learning Record Store choices based on which event syntax is supported, but we’d also strongly prefer not to have to maintain two separate mapping libraries. A convergence here in xAPI and Caliper will make it easier on everyone in the ecosystem

Demonstration

Today and Future - Kaltura Today: Media Metric Profile Next: Assessment Metric Profile Other parameters

Today and Future - Blackboard Today: Caliper event collection & relay Next: Enterprise Learn deployment support, Discussion profile Next next: xAPI support

Key Takeaways Co-development against the spec for systems already instrumented for events is largely enabling the correct handshake and mapping/ transforming event syntax Metric profiles and object extensions provide a great mechanism to support workarounds to enable success between any two systems The context identifiers matter, and are not uniform across system boundaries Some level of interoperability between Caliper & xAPI is highly desirable to avoid competing event ecosystems, but which actor’s responsibility is it?

Thank You! Dan Rinzel Senior Product Manager Jeff Rubenstein for Analytics @ Blackboard dan.rinzel@blackboard.com Jeff Rubenstein VP, Product Development Kaltura, Inc Jeff.Rubenstein@kaltura.com

Configuring Learn SaaS to relay events to an external Caliper record store

Enabling the Caliper “handshake” between Learn and an LTI Tool New bb-manifest.xml permission block <permission type="blackboard.platform.blti.CaliperProfileCallbackPermission" name="*" actions="*"/></permissions>