1. Jeremy Herr University of Michigan and CERN CHEP 2010 Taipei Thursday, 21 October 21 October 20101Jeremy Herr, CHEP 2010, Taipei

2. U-M / CERN partnership Technology transfer agreement between U-M and CERN started in September 2008, ending in November 2010. The work took place in the CERN IT-UDS-AVC section. Goal: Put web lecture archiving into production at CERN. Step 1: Market survey of lecture archiving technologies (CHEP 2009). Step 2: Choose a system and extend it. Step 3: Integration with CERN infrastructure. Lecture archiving is in production, available to all CERN users. 21 October 20102Jeremy Herr, CHEP 2010, Taipei

3. "Lecture Object" Data object containing audio/video files slide images preview images metadata Contains high-quality media for archiving, not suitable for viewing over the Web 21 October 2010Jeremy Herr, CHEP 2010, Taipei3

4. "Web Lecture" audio/video of speaker next to slides, viewable in any web browser. 21 October 20104Jeremy Herr, CHEP 2010, Taipei

5. Market Survey Outcome CERN has special requirements. High quality masters must be archived for all digital media. Recording system must be independent of presentation PC/laptop. Re-use of as much existing hardware as possible to keep cost down. System must be operated remotely. CERN has lots of existing, home-brewed hardware and software, so extensive integration will be necessary. System of choice: U-M ATLAS Collaboratory Project system already used in production by the U-M CARMA campus recording service. 21 October 20105Jeremy Herr, CHEP 2010, Taipei

6. micala Comprehensive lecture archiving based on U-M's system: recording processing archiving publishing monitoring analytics Code is managed by git, tickets/documentation by trac. Primary languages: Python, Perl Uses: MySQL, jQuery, ffmpeg, AviSynth, Python Image Library, ImageMagick Open source, co-released by U-M and CERN Educational Community License version 2.0 http://micala.sourceforge.net 21 October 20106Jeremy Herr, CHEP 2010, Taipei

7. Existing CERN infrastructure Indico Conference management software User-defined metadata describing most events at CERN CDS - CERN Document Server Metadata and document server Public interface and search for digital records and media Media Archive Transcoder server farm that prepares media for CDS Webcast and videoconferencing hardware Tandberg cameras already installed in many rooms at CERN Webcast hardware/software already installed in many rooms DFS network storage (Windows) Highly robust, backed up storage. 21 October 20107Jeremy Herr, CHEP 2010, Taipei

8. CERN infrastructure integration Indico A new plugin was developed: the "Recording Manager." Event metadata and access control now exported to both micala and CDS. CDS Updated to accept lecture object records from the Media Archive. Updated to accept access control information from Indico. Updated to display web lectures. Media Archive Updated to process lecture objects into viewable web lectures. Webcast and videoconferencing hardware Re-used for recording audio/video. DFS network storage (Windows) Used to store lecture object archive 21 October 2010Jeremy Herr, CHEP 2010, Taipei8

9. CERN infrastructure integration 21 October 2010Jeremy Herr, CHEP 2010, Taipei9

10. CERN infrastructure integration 21 October 2010Jeremy Herr, CHEP 2010, Taipei10

11. CERN infrastructure integration 21 October 2010Jeremy Herr, CHEP 2010, Taipei11

12. CERN infrastructure integration 21 October 2010Jeremy Herr, CHEP 2010, Taipei12

13. CERN infrastructure integration 21 October 2010Jeremy Herr, CHEP 2010, Taipei13

14. Recording setup (room installations) Windows PC (used for both webcast and recording) Audio/video capture card: Osprey 240e Audio system: existing room audio system Video camera: existing Tandberg cameras or AV team camera XGA screen capture: VGA2USB device Sofware Remote operation: Custom Python CGI scripts running on IIS Encoding: Adobe Flash Media Lite Encoder The operator clicks on one button to start/stop both webcast and recording. 21 October 201014Jeremy Herr, CHEP 2010, Taipei

15. Recording setup (mobile unit) Linux laptop Audio/video capture: built-in Firewire port Audio system: portable rack-mount wireless mic receiver, mixer. Camera: Portable pro-sumer Firewire camera. XGA screen capture: VGA2USB device Sofware Local/remote operation: PHP/AJAX scripts running on local web server Encoding: ffmpeg The same software works using an Osprey 100 PCI capture card on a PC, so could also be used for permanent room installations. 21 October 2010Jeremy Herr, CHEP 2010, Taipei15

16. micala: monitoring interface Operators can track progress of every lecture being processed. Problems can be quickly identified and tracked down 21 October 201016Jeremy Herr, CHEP 2010, Taipei

17. micala: monitoring DB MySQL database, basically a fancy log file. Every machine reports status to the database. From start (recording) to finish (web lecture posted), there are 11 tasks to be completed. Detailed information about the system status can be gathered quickly. The database ties all the disparate systems together. 21 October 201017Jeremy Herr, CHEP 2010, Taipei

18. micala: monitoring DB 21 October 201018Jeremy Herr, CHEP 2010, Taipei

19. Tasks 21 October 201019Jeremy Herr, CHEP 2010, Taipei

20. Indico: Recording Manager The Recording Manager is a new plugin for Indico It's a web page, operated by the AV team Operates recording remotely Matches orphan recordings with Indico events Exports metadata to micala system Creates CDS records 21 October 201020Jeremy Herr, CHEP 2010, Taipei

21. Indico: Recording Manager 21 October 2010Jeremy Herr, CHEP 2010, Taipei21

22. micala: analytics 21 October 201022Jeremy Herr, CHEP 2010, Taipei micala analyzes Media Archive web logs.

23. Recordings ATLAS Overview Weeks ATLAS Weekly meetings CMS weekly meetings IT Seminars Software tutorials Shift tutorials 324 lectures archived in 2010, expected to increase substantially. 21 October 201023Jeremy Herr, CHEP 2010, Taipei

24. Scalability One recording PC per room; buy one new PC per room. Currently only one processing server meets our needs, but micala is designed to allow multiple processing servers. The Media Archive is actually a farm of transcoding servers, can be scaled up by adding servers. 21 October 201024Jeremy Herr, CHEP 2010, Taipei

25. Robustness The lecture object archive is stored on DFS and is regularly backed up by IT-FIO. The micala monitoring database is backed up daily and copies are kept for 60 days on DFS. The micala software is maintained in a git repository on sourceforge.net, all changes are tracked and can be undone, and multiple copies exist. If the processing server fails, we will experience a delay until another one is set up. To keep lectures flowing through the pipeline, we should install a backup. 21 October 2010Jeremy Herr, CHEP 2010, Taipei25

26. U-M CERN REU Program U.S. National Science Foundation's Research Experience for Undergraduates A great example where lecture archiving is useful Students can review their own presentation skills Future advisors/employers can evaluate their work NSF funding administrators can review student projects 21 October 201026Jeremy Herr, CHEP 2010, Taipei

27. University of Michigan-CERN REU: 2001-2008 www.um-cern-reu.org 2007 Students: 9 week program for 15 students 109 students: 52 (W), 57 (M), 20 (Under- represented groups) Typical Day : AM: CERN Lecture Series PM: Research Group, Weekends free for travel 2007 Institutions: Benedict, MichSt, MIT, NW, NC St, Norfolk St, Reed, Rose-Hulman, Tulane, UColorado, UDallas, UMich, UMissouri, URochester, UWashington Benedict Student working with Dr. Dan Levin on her CERN project: -the installation and testing of a calibration chamber to monitor gas quality in the ATLAS detector.

28. New podcast format Lecture object can be turned into many different formats. Newest one: video podcast It's ready, currently integrating into MediaArchive/CDS Example: REU talk 21 October 2010Jeremy Herr, CHEP 2010, Taipei28

29. How to request a recording... The following rooms are currently outfitted with the new system: 500-1-001 (Main Auditorium) 40-S2-A01 (Salle Andersson) 40-S2-C01 (Salle Curie) 222-R-001 (Filtration Plant) 31-3-004 (IT Ampitheatre) If you want events recorded elsewhere, let the AVC section know! All rooms that have a videoconferencing system can be upgraded for recording. 21 October 2010Jeremy Herr, CHEP 2010, Taipei29

30. How to request a recording... Book a room and create an event in Indico. Remember to include dates, times, locations, people! Click on the pencil (management area). 21 October 2010Jeremy Herr, CHEP 2010, Taipei30

31. How to request a recording... In management area, click on "Video Services." 21 October 2010Jeremy Herr, CHEP 2010, Taipei31

32. How to request a recording... Click on the "Recording Request" tab. Fill out the form and submit, and we'll contact you. Please give us at least 1 week notice so we can schedule our staff! 21 October 2010Jeremy Herr, CHEP 2010, Taipei32

33. Thanks for your interest! If you'd like to try out micala at your own institution, download it here: http://micala.sourceforge.net Feedback and contributions to the code are welcome! 21 October 2010Jeremy Herr, CHEP 2010, Taipei33