Rashomon Project: A Toolkit for Assembling and Analyzing Videos from Multiple Perspectives Rashomon Project (Under Construction - February 2012) About the Project Online Tools for Assembling Multi-Perspective Chronologies UC Berkeley Data and Democracy Initiative

Rashomon Project 1.Vision, Impact 2.Progress to Date 3.Next Steps 4.Requests to Mozilla Ignite

Who Benefits? Emergency Response (Documentation) Public Safety Citizen Journalism Education Free Speech Rashomon Project (Under Construction - February 2012) About the Project Online Tools for Assembling Multi-Perspective Chronologies

Other Applications documenting natural disasters, wildlife, sports, Training

Rashomon Project 1.Vision, Impact 2.Progress to Date 3.Next Steps 4.Requests to Mozilla Ignite

Beta Version 1.1 Server: Securely Collect, Anonymize, Time-Align Videos/Photos Client: Display color-coded timeline and synchronized videos Allows user to set start frame and loop zone Fully- Functional and Tested Code Base: Server: PHP, mongo, html5 UI, ffmpeg, exiv2 (650 lines) Client: HTML5, CSS3, JavaScript, PopcornJS APIs (1600 lines)

Beta Version 1.1 Client Data Stored in JSON file manifest which points to media (.mp4/.webm) and metadata (.json via exiftool)Synchronizes using metadata Tracks time position of each video relative to master timeline (popcorn baseplayer), auto-resyncs (within 0.75 sec) if drifts Supports media fragment URLs for loop selection

Beta Version 1.1 Server User Authentication using Mozilla’s Persona Drag and drop to upload video and photos Formats:.mov,.mp4,.mkv,.3gp,.webm,.jpgTemporal Metadata analyzed, stored, redacted from media using ffmpeg/exiv2, transcoded with ffmpegDETAILS!: Face Blurring using YouTube XAdmin UI lists uploaded content, tracks status in workflow, produces JSON manifests.

Project Team Dr. Ken Goldberg: Prof of Engineering, Project Lead UC Berkeley Abram “Aphid” Stern: Developer, co-founder, Metavid.org, UC Santa Cruz Dr. Camille Crittenden: Director, Data and Democracy Initiative, UC Berkeley

We Witness: Panel on Digital Video, Social Media, and Political ProtestHuman Rights Day, December 10, 2012, UC Berkeley

Rashomon Project 1.Vision, Impact 2.Progress to Date 3.Next Steps 4.Requests to Mozilla Ignite

Next Steps 1. Use Studies Leila Hilal Co-Director of the Middle East Task Force at the New America Foundation.

Next Steps 2. Fine Time-Alignment using Audio Signals Fine Time-Alignment using Audio Signals

Next Steps 3. Collaborative Annotation and Analysis Popcorn Maker: a powerful and collaborative annotation environment Consolidate 2-9 selected videos into single video (up to 3x3) for export as conductor media into Popcorn Maker Full timeline export into Popcorn Maker using Rashomon plugins, Support for Maker’s ‘sequencer’ (still early in development)

Next Steps 4. Distributed Reconstruction of 3D ScenesReconstruction of 3D Scenes Structure from Video analysis to reconstruct 3D Geometry of Environment, position/orientation of cameras Prof. Avideh Zakhor (UC Berkeley EECS) -- webGL

Next Steps 5. Gigabit Apps: Steaming HD Video

Rashomon Project 1.Vision, Impact 2.Progress to Date 3.Next Steps 4.Requests to Mozilla Ignite

Rashomon Project Requests: 1.$25K for Programming, Travel 2.Outreach for use Cases

Rashomon Project Goals: Support Free Speech, Public Safety, Education Catalyst: video documentation from cellphones Problem 1: video is posted ad-hoc on a variety of sites Problem 2: Faces revealed without permission Problem 3: Videographers may fear prosecution Concept: Provide trusted repositories / Open-Source Toolkit For Citizen Journalists and the public Innovations: Time-Alignment, Face Blurring, HTML5 Adaptive streaming, Distr. Routing, Shared Analysis Tools Future: 3D reconstruction, 3d video, Legal Evidence

Partners