1. 1 James Herbsleb Carnegie Mellon University jdh@cs.cmu.edu http://conway.isri.cmu.edu/~jdh/ The Architecture of Coordination The author gratefully acknowledge support by the National Science Foundation under Grants IIS-11 0414698, IIS-0534656, and IGERT 9972762

2. 2 Architectures and Organizations  Level of an industry − E.g., modular clusters  Level of an organization − E.g., matching hypothesis  Level of a practicing architect, manager, or engineer − Architectural decisions define coordination problems − Understanding and managing impact of decisions

3. 3 Today...  How can you measure the match of an organization with the structure of the software it is producing?  Coordination theory about the “modularity mismatches” – the failures or limits of modularity

4. 4 Product Structure: Conway’s Law “Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure.” M.E. Conway, “How Do Committees Invent?” Datamation, Vol. 14, No. 4, Apr. 1968, pp. 28–31.

5. 5 Conway’s Law Components Software Teams Organization Homomorphism

6. 6 Designing Coordination Problems Components Software Teams Organization

7. 7 Designing Coordination Problems Components Software Teams Organization ? What kind of coordination is required?

8. 8 Measuring Coordination Requirements (C R ) XX = Task Assignments Task Dependencies (A)(D)(A T ) Coordination Requirements (C R ) a 11 … a 1k a n1 … a nk d 11 … d 1k d k1 … d kk a 11 … a 1n a k1 … a kn cr 11 … cr 1n cr n1 … cr nn Socio-Technical Congruence and Productivity Concept

9. 9 Coordination in a Distributed Project Software firm, 1 product, 114 developers 8 teams, 3 locations, all in US Product separated into components Each component assigned to 1 team Cataldo, M., Wagstrom, P., Herbsleb, J.D., Carley, K. (2006). Identification of coordination requirements: Implications for the design of collaboration and awareness tools. In Proceedings, ACM Conference on Computer-Supported Cooperative Work, Banff Canada, pp. 353-362.

10. 10 A Word About Tools and Data Use archival data from large software development project Modification request (MR) system –Users, testers, developers request changes –Bug fixes, new functionality Version control system –Maintains all changes to all files –Some set of changes correspond to each MR –Has data about who made change when Communication data –IRC chat –Discussions within MR system

11. 11 Measuring Coordination Requirements (C R ) XX = Task Assignments Task Dependencies (A)(D)(A T ) Coordination Requirements (C R ) a 11 … a 1k a n1 … a nk d 11 … d 1k d k1 … d kk a 11 … a 1n a k1 … a kn cr 11 … cr 1n cr n1 … cr nn Socio-Technical Congruence and Productivity Files changed together Developer modified files Transpose of developer modified files Who needs to coordinate with whom Concept Data

12. 12 Volatility in Coordination Requirements Change in coordination group Members of other teams Proportion Week

13. 13 Measuring Congruence 13 Socio-Technical Congruence and Productivity Diff (C R, C A ) = card { diff ij | cr ij > 0 & ca ij > 0 } Congruence (C R, C A ) = Diff (C R, C A ) / |C R | Coordination Requirements (C R ) Actual Coordination (C A ) cr 11 … cr 1n cr n1 … cr nn ca 11 … ca 1n ca n1 … ca nn Team structure Geographic location Use of chat On-line discussion

14. 14 Predicting Resolution Time Table 2: Results from OLS Regression of Effects on Task Performance ( + p < 0.10, * p < 0.05, ** p < 0.01). Model IModel IIModel IIIModel IV (Intercept) 2.987 ** 3.631 ** 1.572 * 1.751 * Dependency 0.897 * 0.653 * 0.784 * 0.712 * Priority -0.741 * -0.681 * -0.702 * -0.712 * Re-assignment 0.423 * 0.487 * 0.304 * 0.324 * Customer MR -0.730 -0.821 -0.932 -0.903 Release -0.154 * -0.137 * -0.109 * -0.098 * Change Size (log) 1.542 * 1.591 * 1.428 * 1.692 * Team Load 0.307 * 0.317 * 0.356 * 0.374 * Programming Experience -0.062 * -0.162 * -0.117 * -0.103 * Tenure -0.269 * -0.265 * -0.239 * -0.248 * Component Experience (log) -0.143 * -0.195 * -0.213 * Structural Congruence -0.526 * -0.483 * Geographical Congruence -0.317 * -0.312* MR Congruence -0.189 * -0.129* IRC Congruence -0.196 * -- Interaction: ReleaseX Structural Congruence 0.007 0.009 Interaction: ReleaseXGeographical Congruence -0.013 -0.017 Interaction: Release X MR Congruence -0.009 + -0.011 + Interaction: Release X IRC Congruence -0.017 * -- N809 1983 Adjusted R 2 0.7870.8720.7560.854 (* p < 0.05, ** p < 0.01)

15. 15 Effects of Congruence Time to complete a work item is reduced by each of the types of congruence –Team structure congruence –Geographic location congruence –Chat congruence –On-line discussion congruence

16. 16 Average Level of Congruence for Top 18 Contributors

17. 17 Average Level of Congruence for the Other 94 Developers

18. 18 The Story So Far... Coordination requirements are highly volatile. Congruent coordination activities are associated with performing the work faster. The top developers behave much more congruently than the rest. Have replicated congruence finding on GNOME projects Have experimented with many different ways of computing dependencies –Logical dependencies the most predictive by far

19. 19 Theory Build on modularity/architecture literature –Product structure and task structure Build on network analysis –Network attributes matter –Relations among people, decisions, components Methodology: build on idea of logical dependencies

20. 20 Coordination: Five Propositions P1: Design progresses by making decisions. P2: Decisions are linked by constraints in a potentially large and complex network. –The “decision network” P3: The need for coordination among individuals arises from –Properties of the decision network –Assignment of decisions to people P4: Effective coordination is the result of coordination actions, moderated by coordination capacity. P5: Coordination breakdowns occur when effective coordination is insufficient for coordination needs

21. 21 Example Domain: Field Robotics

22. 22 Example Network: Field Robotics

23. 23 Coding Method Meeting segments AvionicsOpticsSoftware

24. 24 Coding Method Meeting segments Divide into decision discussions AvionicsOpticsSoftware

25. 25 Coding Method Meeting segments Divide into decision discussions Code according to components involved in the decision discussion –Co-occurrence much like logical dependency AvionicsOpticsSoftware Camera Mobility

26. 26 Example Network: Field Robotics

27. 27 Hardware Discussions Newman Modularity:.39

28. 28 Software Discussions Newman Modularity:.03

29. 29 Network Theory of Coordination Coordination Effectiveness Coordination Effectiveness Outcomes Coordination Requirements Coordination Requirements Coordination Actions Coordination Actions Coordination Capacity Coordination Capacity Task Assignment Decision Network Decision Network

30. 30 Challenges Planning –Predicting coordination requirements –Assessing coordination capacity –Choosing optimal coordination actions Coordinating –Monitoring for unexpected mismatches –Organizational tactics, architectural tactics Theorizing –Principles, patterns for network dependencies –Prior theories as special cases