Team Thinking Time Variation of Design “Story Telling” in Engineering Design Teams Andy Dong Key Centre of Design Computing and Cognition University of Sydney Shuang Song and Alice M Agogino Department of Mechanical Engineering University of California, Berkeley

Team Thinking Design Teams: Research Questions How can we identify, represent, and visualize “story telling” of engineering design teams? How does the “story telling” evolve over time? What insights into emergent design processes could quantitatively depicting patterns in “story telling” provide?

Team Thinking “Story Telling” in Engineering Design Teams Engineering design is a complex technical pursuit mediated by social processes such as communication, negotiation, and shared agreements. We define “story telling” in design as establishing a coherent “story” about the design process and the designed artifact by bringing coherence to the perspectives and interests of each design team member.

Team Thinking Methods for Studying Design Teams Protocol analysis –Discourse analysis and “Think aloud” Ethnography –Observations, interviews & case studies Computational –Latent Semantic Analysis –Social Network Theory –Speech Act Theory

Team Thinking What are “good” design teams? Balanced teams consisting of members with complementary roles, a plurality of viewpoints, a neutral manager and a “wild card”. (D. Wilde) Establishes a social contract among members that relate to their purpose and guides and obligates how they must work together. (J. Katzenbach) Design strategies, including reflection, that lead to the creation of shared understanding result in effective design outcomes. (R. Valkenburg)

Team Thinking

Measuring Time Variation of Design “Story Telling” Capture design documentation Text processing (keyword extraction, frequency counts) Text analysis (latent semantic analysis) Calculate intra-stage and cross-stage semantic coherence Compare predicted team performance to human assessments

Team Thinking Metrics 1. Intra-Stage Semantic Coherence 2. Cross-Stage Semantic Coherence

Team Thinking Data Set: Product Design Teams Managing the New Product Development Process: Design Theory and Methodology Product definition to alpha prototype Self-managing design teams of students from engineering, business and information science

Team Thinking Assessment Professional Product Designers (IDEO, frog design, Lunar Design and others) Judging Criteria (Scale 1-5) –mission statement– prototype feedback –customer and user needs– financial analysis –concept generation– final prototype –concept selection Correlate computational results to judges’ assessments

Team Thinking Design Coherence Over Time The design process is characterized by an iterative broadening and narrowing of design possibilities, and an iterative reconciliation of design interests and conflicts towards a set of shared agreements. Increasing coherence with cycles of divergence during the design process is desirable. Decreasing coherence in design “story telling” is likely disruptive and increasingly dysfunctional.

Team Thinking Variation of Average Semantic Coherence

Team Thinking Variation of Intra-Stage Semantic Coherence

Team Thinking Cross-Stage Semantic Coherence Variations Faculty Rank Team A8 Team B1 Team C6 Team D5 Team E2 Team F3 Team G4 Team H7

Team Thinking Patterns of Usages

Team Thinking Patterns of Semantic Coherence from Two Sources

Team Thinking Summary This research establishes a formal methodology for providing a real-time window into the design process and coherence of design thinking of the design teams. Evidence to suggest a link between patterns of the semantic coherence of design documentation (accuracy of language in explaining thoughts) and design performance. Latent semantic analysis shows great promise as a tool for modelling and evaluating the cognitive and psychosocial behaviour of design teams.

Team Thinking Future Research How do we distinguish between “creativity” and “lack of vision”? What is the role of individuals (personalities, leadership, gender,etc.) in the “team thinking” process? How information technology (information source, information accessibility, communication, etc.) influences design team cognition?