1 Imaginarium Merging science and practice David Kirsh, UCSD Cognitive Science, Assoc. Director ACCHI Erik Viirre M.D. Ph.D., UCSD School of Medicine Sheldon Brown, UCSD Visual Arts, Director of ACCHI Plus collaborators…
Objectives 1.Observation: Study the nature of distributed creativity in the natural setting of: 1.Brain storming 2.Design 3.Creative Expression 2.Theory: Build a more comprehensive theory of activity and sensory involvement during brainstorming 3.Design: Explore impact of changes in environment design on imagination, creativity and problem solving
Imaginarium Technology Binocular eye tracking among groups Video cameras coordinated with eye tracking Kinects for gesture detection Portable ERP caps Precise sound localization Whiteboard and other surface capture Automated Annotation of design session Remote interaction with other sites
Analytic Techniques Ethnographic coding, activity analysis, Datamining for patterns among different capture records Machine learning using VATIC and crowd sourcing to classify events defined via activity analysis Analyses using theories of distributed, situated and embodied cognition
Collaborators Relevant Research Basis Kirsh: NSF on Distributed Creative Cognition –In choreography –being expanded to music Makeig, Saygin: ERP (Event Related Potentials) during activity Trivedi: Video and Auditory awareness systems. Viirre: Eye Tracking for wayfinding, medical diagnosis and therapy Brown: User analytics of virtual worlds
Observation Set-Up for Distributed Creative Cognition in Choreography - Kirsh Mandeville Auditorium Cal-IT2
Oculomotor Measuring Equipment
Video Engineering Wide Angle Camera to identify head High resolution camera slaved to get face/eye activity Analysis of video image (visible/IR)
Body and Gaze Tracking
Tool for Collaboration Centered around whiteboard, display table, workbench –Annotation –Command interaction (Kinect)
Medical Evaluation Monitoring of Movement control –Fall Risk –Movement control –ADL Evaluation of Disease progression –Vestibular –Parkinsons Diagnosis? –Cerebrovascular vs. Neurodegenerative
Research Questions Are binocular indices sensitive indicators of neural disruption What do “natural” eye movements look like? –Combined Eye-Head –Head Free Scan paths
Embedded Cognition Theory Interaction with physical objects –Mental planning, practice Learning theory of tasks
Principals Erik Viirre, Medicine David Kirsch, Cog Sci Mohan Trivedi, Engineering Sheldon Brown, Visual Arts
Funding Sources Industry Partners Medical Research –DOD/VA –NIH NSF Computing, sensors, creativity, social sciences