Capture: Preliminary Experiment Instrumentation: – 2 Kinects (1 overhead and 1 side view) – 1 st person audio/video: GoPro glasses on examiner – Boom microphone (fixed) – Consumer video camera – EDA on child and examiner Observations: – Activities with examiner/child take place across the whole room Placement of fixed sensors must be studied carefully or activities more controlled – 1 st person video Go Pro $300 price point offers exciting opportunities for in-home use and wide deployment with examiners and families Challenges with respect to stabilization for viewing/coding Offers the examiner’s view at any time (both a positive and negative)

Kinect vs. HD Video Multiple Kinects – Positives: Inexpensive (~5-10K for system with Kinects, PCs, and 4 Kinects) Gives RGB + depth Body pose estimates (in side views) – Negatives: Overlap between Kinect views yields errors in depth streams Each Kinect intermittently drops video frames at different times Capture is asynchronous, which is challenging for multi-view analysis High-res video capture (using Camera-Link) – Positives: high-definition, high optical quality Synchronized at time of capture – Negatives: Expensive (~$50K for system with capture equip, PC, and 4 cameras) Does not include depth sensing

Workflow Step in workflowSoftwareHardware Capturing parallel streamsOpen source, homebrew, or turnkey capture software? Kinect, consumer video, high quality video, 1 st person camera(s), Affectiva, mic Editing streamsElan? Annotating/coding sessionsElan? Storing of data and Elan filesBackups? Mirroring?Disk array with fault tolerance and mirror SharingWeb server – secure? Where in the workflow does synchronization of streams occur? What method(s) will be used for synchronization? Is synchronization needed?