1. The Avalon Media System
Yale University
November 24, 2014
Jon Dunn
Indiana University
Mark Notess
Indiana University
Julie Rudder
Northwestern University

2. Agenda
Introduction & Overview – Jon Dunn, Indiana Demo – Julie Rudder, Northwestern Technical architecture, development process, release plans – Mark Notess, Indiana NU implementation and services - Julie IU implementations and services – Mark Future directions – Jon Q & A - All

3. Avalon Project Objectives
Goal: Create an open source system to enable libraries and archives to provide online access to video and audio collections
Digital audio/video management and delivery system, focused on needs of libraries and archives
Follow an agile, open source development model
Leverage existing technologies, where feasible
Communicate and market the project broadly to increase awareness and grow the community of users and developers

4. Project Overview
Funded in part by a National Leadership Grant from the Institute of Museum and Library Services
Original name: Variations on Video
Planning grant:
August 2010 – July 2011
Implementation grant:
October 2011 – September 2014 September 2015
Partnership between libraries at Indiana University and Northwestern University
Discuss motivations

5. Project Partners and Advisors

6. Project Partners and Advisors
Jon

7. Avalon is a Hydra Solution Bundle

8. Why Avalon?
Existing repository systems don’t work well for time-based media
DSpace, Fedora, Digital Commons, ContentDM, …
Issues: integration with streaming servers; hard to support time-based navigation; difficulties in integrating access control
Existing media systems don’t work well with repositories
Kaltura, Brightcove, Ensemble, ShareStream, …
Issues: focus on short-term access for teaching and learning; limited metadata and access control capabilities; integration with preservation workflows
Strong desire for community-based solution

9. Demonstration

10. Media Player (mediaelement.js)
Avalon Architecture
Integrations
Users
Avalon Media System
ILS
Desktop, Mobile
Hydrant Rails App
Media Player (mediaelement.js)
LMS, websites
Search
Browse
View
Omni-Auth
All Users
Authentic-ation
Hydra Head
Ruby- horn
Can-Can
Black- light
Authoriz-ation
Browser,
Drop box
Ingest
Describe
Manage
Archival Storage
Collection Managers
Fedora
Solr
Matterhorn
Media Server
 Avalon code – consists of Hydrant Rails App and the Rubyhorn gem. Other pieces are leveraged.

11. Release History 0.1 – July 2012 0.5 – October 2012 1.0 – May 2013
3.2 – December 2014 (expected)

12. Development Team Julie Hardesty Jon Dunn Chris Colvard Mark Notess
metadata specialist
Jon Dunn
project director
Chris Colvard
lead developer
Mark Notess
product owner
Cynthia Ramlo
lead user experience designer
Leah Lee
developer
Brian Keese
developer
Ariadne Rehbein
project assistant
Development Team
Michael Klein
lead developer
Karen Miller
metadata specialist
Stu
Stefan Elnabli
audiovisual specialist
Dan Zellner
testing and production support
Claire Stewart
co-project director
Julie Rudder
product owner

13. Current development process
A single, blended team
Agile Scrum process
Code in public GitHub
Face-to-face meetings at least 2x/year
Daily standups, IRC

14. 0 – 3.1 Release functionality
End-user functionality
Faceted discovery for search and browse
Video and audio playback in browsers or on mobile devices
Stream-level security
Ability to embed media player in other websites
Persistent URLs
RTMP and HTTP streaming, Flash and HTML5-based player
Collection management
Interactive or batch uploading of media files and MODS metadata
Private collection dropboxes
A hierarchical model for permissions that supports a flexible approach to batch collections-based content management
Customized thumbnails by taking a “snapshot” or specifying a timepoint
Direct import of previously transcoded derivatives
Avalon transcoding for multiple quality derivatives
Claire?

15. 0 – 3.1 Release functionality
Integration with other enterprise systems
Red5 and Adobe media servers
Local authentication services (CAS, LDAP and others, using OmniAuth)
Learning management systems, via the Learning Tools Interoperability (LTI) standard
Support for LDAP group access control
Support for persistent URL systems (PURL, Handle)
Master files: leave, delete, or rename/move after transcoding
Installation
Easy installation and configuration via a virtual machine image and other methods
Fully-transparent integration testing using travis-ci.org - every commit and pull request triggers a build
Configuration approach for easier customization where possible
Claire?

16. Upcoming releases (Fall/Winter 2014)
Release 3.2 (December)
Upgrade to latest versions of Hydra, Blacklight, and Bootstrap
Metadata: adding language, original physical description, related item links, terms of use
Let staff select multiple items and publish, unpublish, delete, set access permissions, assign to collection
Release (January/February 2015)
Import descriptive metadata from an Integrated Library System
Import multiple quality levels of pre-transcoded derivatives
Mark
For 3.2 can also mention preparatory work:
Design bulk actions for access control, metadata
Accessibility assessment
For can also mention stories:
This will require mapping of MARC fields to Avalon fields.
When adding individual items and batch ingest items, bibliographic info will be pulled from the OPAC

17. Release 4 (Spring 2015) Structural metadata
Import and store structural metadata
Allow users to navigate using media sections
Prioritize media in the transcoding queue
Bulk update of metadata
Fedora 4 (testing?)
Mark

18. Beyond Release 4 Reporting and metrics Time limits for access
IP restricted access
APIs for ingest, analytics, metadata, etc.
Fedora 4 migration
Accessibility (navigability, closed captioning)
Transcripts/Auxiliary items
Improved metadata
Playlists & clip making
Integration with Wowza and Amazon Cloud Front 
Publish to YouTube or other systems
Integration with Spotlight exhibits tool
Internationalization
Mark

19. Follow our progress For sprint summaries, read our blog
For more details and recordings, visit our wiki: wiki.dlib.indiana.edu/display/VarVideo/Avalon+Media+System
Avalon Current Sprint page
wiki.dlib.indiana.edu/display/VarVideo/Avalon+Current+Sprint
Previous Sprints page
wiki.dlib.indiana.edu/display/VarVideo/Previous+Sprints
Sprint Demos
wiki.dlib.indiana.edu/display/VarVideo/Sprint+Demos
Mark

20. Community and future directions
Multiple implementations currently underway
Stanford, UVa, UToronto, UT Austin, …
Working on new features, including:
Structural metadata
Transcripts and captioning
Starting to deal with code contributions
Working with Hydra community
Leverage common features across “heads”
HydraDAM and Avalon, other preservation-specific integrations
Sustainability/governance/business model
Exploring hosted options

21. Sustainability and Governance
Focus in 2015 with potential funder support
Want to grow the number of implementers
Ongoing costs of software support and development
Potential revenue streams
Membership/sponsorship
Income from hosting
In-kind developer contributions
Governance and management structure
Potential alignment with existing 501(c)(3) organization in higher ed open source community

22. Julie Rudder j-rudder@northwestern
Julie Rudder Digital Initiatives Project Manager
DLF Forum 2014

23. Avalon replaces this system and does more
Avalon replaces this system and does more. We launched a pilot in Summer 2013, support a few course and we went live.

24. Use Cases Course Reserves – restricted access Archival Collections
MMLC – video management by units
outside the library

25. Future Additional units outside the library?
Integration with other campus media solutions?
Better integration with Ares Course Reserves?

26. Usage so far....

27. Production Environment
Existing Production Systems
Load Balancer VM
Streaming - FMS 2
Streaming - FMS 1
Avalon Web App
Shared SOLR
4 TB dedicated Hardware
Dedicated SOLR Core
Transcoding
Shared Production Fedora
Fedora
Production Environment

28. Total Audio-Video Assets Served: 34,146
Background on Streaming at NUL
Integrations
Canvas and Blackboard
Existing handle server
LDAP
Adobe Media Server (secure streaming)
Related -- Ares Course Reserves (request mechanism)
Streaming Media since 2001
Total Audio-Video Assets Served: 34,146

29. Implementation Pilot – Summer 2013 Full production – Fall 2014
A few classes support
Two public collections
Local VM infrastructure
Full production – Fall 2014
All new requests via Avalon (no migration)
MMLC launches Unit
NUIT (central data center) infrastructure

30. Retire current infrastructure by summer 2015
AVALON
All new digitization
Summer 2014
Summer 2015
2004 – 30,000+ assets served
Retire legacy systems

31. Staffing 1 DevOps (install, configure, support)
1 Developer (app customization, support)
1 Project Manager (app oversight, integrations oversight)
1 Service Owner (liaise with units & users, workflow, support)
The DevOps time and my times was a lot of upfront work, the other two continue to have ongoing work.

32. Additional support – 1st tier
Business hours
Library Digital Collection Staff
24 hour -Canvas helpdesk support
24 HOUR
Central Campus IT helpdesk
24 HOUR
Canvas helpdesk
Common problems include, trying to direct link, navigation,

33. Avalon at Indiana University

34. IU Context History of audio/video digitization
Legacy systems for delivery
Eight campuses
New demands
Media Digitization and Preservation Initiative

35. Legacy Systems

36. IU Deployment Environment
University IT Services
Library servers
Redundant Archive Storage
(HPSS)
Master Files
4 core, 1GB RAM VM
1GB Disk (VMware)
Avalon Web App
Solr, Fedora
Shared SAN
Derivatives
12 core, 120GB RAM,8TB dedicated hardware
Transcoding
IU Enterprise Streaming
Adobe Media Server

37. Integrations CAS authentication Microsoft Active Directory (via LDAP)
PURL resolver
Canvas and Sakai (via LTI)
Scholarly Data Archive (HPSS master file storage)
Adobe Media Server (secure streaming)

38. IU Production Instance
Pilots began Fall 2013
Moved to production, October 2014

39. Archival Film

40. Omeka Online Exhibit: World War II Propaganda Frilms

41. Media Within Archival Collections

42. Online Publishing: Ethnomusicology Multimedia

43. Specialized Avalon Instance: IJCCR

44. Specialized Avalon Instance: Kelley School of Business

45. Future Migrations Video e-reserves (in pilot)
LTI integration with Sakai, Canvas
Licensed educational video
LDAP for campus restriction
Variations: music audio
Pedagogical, annotation tools
Media Digitization and Preservation Initiative

46. Production Staffing
10% of a systems administrator (server management; higher initially)
20% of a developer (install, configure, support; higher initially)
20% of project/service manager (app oversight, workflow planning, work with units and users, support)
Does not include collection management or central IT staffing of shared services

47. IU Media Digitization and Preservation Initiative
mdpi.iu.edu

48. Avalon in the IU Media Digitization and Preservation Initiative (MDPI)
8.4 petabytes of audio and video data over 5 years
Avalon serving as key access component
Integrations:
Metadata: MARC/ILS, EAD/XTF
Authentication: CAS
Authorization: Active Directory
Learning Management: Canvas, Sakai
Media streaming: Adobe Media Server
Side-by-side with trusted repository for preservation
Bit storage in existing hierarchical storage management system
Hope to leverage HydraDAM work
Exploring out-of-region options

49. Questions? For more about Avalon
Website:
Avalon list: to subscribe
@avalonmediasys
facebook.com/avalonmediasys

50. Backup Screenshots

51. Faceted browse via Blacklight

52. Desktop/Mobile Player

53. Embedding

54. LTI (in Canvas)

55. LTI (in Canvas)

56. Manage Content

57. Add item: Upload, Dropbox, or Batch

58. Dropbox view

59. Batch Folder

60. Descriptive Metadata: MODS

61. Access Control

62. Permanent URL minted on Publish

63. Bulk actions