1. How the Science of Teams Can Inform Team Science Nancy J. Cooke March 13, 2015 Team Science Retreat Wake Forest School of Medicine of Wake Forest Baptist Medical Center

2. Overview Why Team Science? Update on NRC Study My research and experience A Multi-Level Systems Perspective  Micro Level: Challenges and Support  Meso Level: Challenges and Support  Macro Level: Challenges and Support Conclusion

3. Why Team Science? Today’s problems require a team of multidisciplinary individuals Team Science is impactful (highly cited; Wuchty, et al., 2007; Uzzi, et al., 2013) Team Science is innovative (Uzzi, 2013) Team Science is productive (Hall, et al., 2012) Team Science has broad reach/uptake (Stipelman, et al, 2014)

4. Why Team Science? But… Not all science requires a team Team science is difficult

5. Enhancing the Effectiveness of Team Science: Symposium at ICPS March 13, 2015 Board on Behavioral, Cognitive, and Sensory Sciences Division of Behavioral and Social Sciences and Education National Research Council An Update on the NRC Study of Team Science

6. 6 Study Background Rationale: Clear need to provide research-based guidance to improve the processes and outcomes of team science Sponsors: NSF, Computer and Information Systems and Engineering Directorate and Elsevier Goal: Enhance the effectiveness of collaborative research in science teams, research centers, and institutes. Audiences: NSF and other public and private research funders; the scientific community; the SciTS community; universities; research centers and institutes.

7. Committee Charge Conduct a consensus study on the science of team science to recommend opportunities to enhance the effectiveness of collaborative research in science teams, research centers, and institutes… Explore: How individual factors influence team dynamics, effectiveness and productivity Factors at the team, center, or institute level that influence effectiveness Different management approaches and leadership styles that influence effectiveness How tenure and promotion policies acknowledge academic researchers who join teams Organizational factors that influence the effectiveness of science teams (e.g., human resource policies, cyberinfrastructure) Organizational structures, policies and practices to promote effective teams

8. Committee NANCY J. COOKE (Chair), Arizona State University ROGER D. BLANDFORD (NAS), Stanford University JONATHON N. CUMMINGS, Duke University STEPHEN M. FIORE, University of Central Florida KARA L. HALL, National Cancer Institute JAMES S. JACKSON (IOM), University of Michigan JOHN L. KING, University of Michigan STEVEN W. J. KOZLOWSKI, Michigan State University JUDITH S. OLSON, University of California, Irvine JEREMY A. SABLOFF (NAS), Santa Fe Institute DANIEL S. STOKOLS, University of California, Irvine BRIAN UZZI, Northwestern University HANNAH VALANTINE, National Institutes of Health

9. Study Status Report expected in April More information is available at: http://sites.nationalacademies.org/DBASS E/BBCSS/CurrentProjects/DBASSE_080231

10. Research Base for Informing Team Science SciTS – Science of Team Science (itself a multidisciplinary approach) Social Science Complex Systems Communications Management Medicine Physical Sciences

11. The Foresight Initiative National Geospatial-Intelligence Agency (NGA) has awarded Arizona State University a grant of $20 million Five-year partnership known as the Foresight Initiative will examine how climate change affects resources and contributes to political unrest, as well as articulate sustainability and resilience strategies.

12. Foresight: A Science Team Approximately 60 Investigators 15 ASU Faculty from 8 ASU units Post docs, research faculty, graduate students Three National Labs National Geospatial Intelligence Agency Expertise in visualization, modeling climate change, cognitive science, social media, human factors

13. Foresight: Team Science is Challenging Communicating across disciplines Role confusion Meetings Remote participation Goal conflicts Sub-teams Authorship Resource Allocation

14. My Research and Experience Relevant to Team Science Team = Heterogeneous and interdependent group of individuals (human or synthetic) who plan, decide, perceive, design, solve problems, and act as an integrated system (vs. group) Cognitive activity at the team level= Team Cognition Improved team cognition  Improved team/system effectiveness Heterogeneous = differing backgrounds, differing perspectives on situation (surgery, basketball)

15. Teams and Cognitive Tasks I’ve Studied Team Cognition in These Tasks Uninhabited Aerial Vehicle Command and Control Naval Mission Planning Cyber Defense Intelligence Analysis Human-Underwater Robot Interaction Medical Emergency Teams Professional Cooking Human-Robot Search and Rescue

16. Methods: Synthetic Task Environments A compromise between field studies and laboratory experiments 16 Uninhabited Aerial Vehicle – Synthetic Task Environment MacroCog Underwater Robots CyberCog

17. What I’ve Learned Teams Learn Teams Forget Membership Matters Team Training Matters

18. Teams Learn As teams acquire experience, performance improves, interactions improve, but not individual or collective knowledge Individuals are trained to criterion prior to M1 Asymptotic team performance after four 40-min missions (robust finding) Knowledge changes tend to occur in early learning (M1) and stabilize Process improves and communication becomes more standard over time 40-min missions Spring Break

19. Teams Forget Team forgetting is best predicted by interaction based measures, not by individual forgetting (despite shared score components) Regression model made up of individual decrements: F (4, 20) = 2.018, MSe=5880.23, p>.10, R2 =.29 Introduction of coordination and team SA: F (10, 14) = 2.71, MSe = 4011.88, p <.05, R2 =.66 8-10 week retention interval

20. Membership Matters 117 males(92) & females(25) divided into 39 3-person (unfamiliar) Session 2 teams Two between subjects conditions (retention interval and familiarity) randomly assigned with scheduling constraints Participants randomly assigned to one of three roles Session 1: 5 40-min missions Session 2: 3 40-min missions 10 Teams 9 Teams 10 Teams 3-5 weeks10-13 weeks Same Mixed Composition Retention Interval Mixed Condition Session 1Session 2 Retention Interval AVOPLODEMPC AVOPLODEMPC Same Condition Session 1Session 2 Retention Interval AVOPLODEMPCAVOPLODEMPC

21. Team Retention and Composition 3-5 OR 10-13 Weeks All but Short-Intact teams suffer performance loss after the break

22. But a different story for Team Process (quality of team interactions)… Team Process improves for mixed, but not intact teams after the break. (There were no changes in knowledge after the break) 3-5 OR 10-13 Weeks

23. Team Training Matters Cross training (aligned with shared cognition) vs. procedural/rigid training vs. Perturbation training (focused interactions)

24. Shared Mental Models Assumptions Individual is the unit of analysis Measure individuals and aggregate Increasing similarity or convergence over time is associated with better teamwork Focus on knowledge, static cognition (team mental model, shared mental model) A collection of knowledge experts should be an expert team ++ Team Cognition = The collective knowledge of team members

25. Interactive Team Cognition Team interactions often in the form of explicit communications are the foundation of team cognition ASSUMPTIONS 1)Team cognition is an activity; not a property or product 2)Team cognition is inextricably tied to context 3)Team cognition is best measured and studied when the team is the unit of analysis

26. US 2004 Olympic Basketball Team " We still have a couple of days, but I don't know where we are," replied USA head coach Larry Brown … I've got a pretty good understanding of who needs to play. Now the job is to get an understanding of how we have to play." A team of experts does NOT make an expert team Collaborative skill is not additive

27. US 1980 Olympic Ice Hockey Team Herb Brooks and 20 young “no-names” won the 1980 Olympic Gold Medal in Ice Hockey An expert team made up of no- names…

28. A Multi-Level Systems Perspective Micro -individual Meso – team, group Macro - organization, population Borner, Contractor, Falk- Krzesinski, et al., 2010

29. Micro Level: Challenges Who should engage in team science? – Risks of early career tenure-track scientists Who should be on the team? – Team composition – Team assembly Faultlines and subgroups

30. Micro Level: Support Recommender systems Research networking systems Matching task to team assembly

31. Meso Level: Challenges IPO Model (Hackman, 1987) InputProcessOutput DevelopmentConceptualizationImplementationTranslation Four Phase Model of Transdisciplinary Research (Hall, Vogel Stipelman, 2012)

32. Meso Level: Challenges Team Process Behaviors Communication – shared mental models Coordination Conflict Resolution Back-up Behavior Situation Assessment

33. Meso Level: Support Training Leadership Technology Tools for Team Science – NCI Team Science Toolkit

34. Macro Level: Challenges Organizational rewards for team science Disciplinary culture Geographic dispersion Complexity of multi-team systems Mis-aligned goals

35. Macro Level: Support Environment Technology Rewards Collaboration Plans Team Charters

36. Communication plan between teams (modes, media, who to whom) Plan for regular interactions Plan for leadership – shared Identify boundary spanning individuals Asencio, Carter, DeChurch, et al., 2012

37. Conclusion Team science is challenging Team research has implications for making science teams more effective Challenges and support can be found at the micro, meso, and macro levels