1. Scott Poole, UIUC; Noshir Contractor, Northwestern; Mark Hasegawa-Johnson, UIUC; Feniosky Pena-Mora, Columbia; David Forsyth, UIUC; Kenton McHenry, UIUC; Dorothy Espelage, UIUC; Margaret Fleck, UIUC; Alex Yahja, National Center for Supercomputing Apps

2. The Story behind Cultural Artifacts

3. Challenges Socio-cultural consequences of group decisions Inability to collect, analyze, and manage High resolution, High quality, High volume interaction network data Effective computer-aided collaboration among Scholars Scientists Students Volunteers Stakeholders

4. Scientific Challenges We understand small teams co-located (1-6 persons) and we think we understand large aggregations of 1000s We don’t understand large teams: 8-25, 25-70, 50-300, 350-500, 400-1000—the sweet spot of scholarly collaborations and conferences Current studies are surveys and case studies, not direct observation, the gold standard No tech to study these even though we coalesce in natural groups of size 2, 5, 15,… Spatial dispersion and movement make big difference

5. Importance of the Problem Many critical groups are of this size: Design Teams Scholarly Collaborations Cultural Studies Legislative Bodies Disaster Response Teams Archaeology Teams Medical Teams Military Units

6. “Swarming” Disaster Response

7. Supported By Cyber-enabled Discovery and Innovation (CDI) program, National Science Foundation Two Million Dollars Grant National Center for Supercomputing Apps Office of the Vice Chancellor for Research, University of Illinois Year 2 of Five Year Project Project “GroupScope”

8. Approach End-to-end system from data capture to analysis to user and team engagement Video cameras to capture video and audio, of Study subjects such as children on playground Scholars and researchers executing the study—in team and individually Synchronization of video and audio data Annotation of video and audio Coding of video and audio Management of video and audio data Analysis of video and audio; scenario simulation and machine learning Community involvement

9. Data Acquisition (cameras, Kinect, audio recorders, GPS, iPhones, iPads) First-order Data (audios, photos, videos, sensor data) Data Management (Medici content management, ELAN transcription) Second-order Data (visual, audio and text annotations, coding and metadata) Network analysis, Group identification, Interaction categorization What-if Scenario Simulation and Machine Learning Circle of Continuous Improvement

10. 2D Face Tracking (Kalal TLD)

11. Depth-image “Kinect” Skeleton Tracking

12. Human Movement Recognition

13. Social Interaction Recognition

14. Community Engagement Professors and graduate students as primary research participants Students help annotate videos and audios of study objects and artifacts research activities of professors and research assistants Interested folks help transcribe, translate, and annotate videos and annotate Multi-lingual collaboration enabled Scenario “what-if” analyses of interactions and events Annotated videos will “live” across time and place Insights, inspirations, and moments are recorded and not lost to time and place

15. In Closing “GroupScope” tool is designed to provide Computer-assisted collaboration among human teams Natural and native human and professional social- networking—synergistic human machine effort Scholarly collaboration tool with native domain-specific design and interfaces Natural collaboration space By your consent, putting up video cameras to get PNC 2017 networking? Will put up video cameras for NSF Radical Innovation Summit 2013