1. Copyright  2012 Third Millennium Systems, LLC Complexity Measurement Results 2012 Sarah Sheard

2. 2 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC A View of the Spectrum OrderComplexityDisorder StabilityChaosOscillationRandomness Short-Term Order Long-Term Order Short-Term Order Long-Term Disorder Short-Term Disorder Long-Term Disorder Source: http://www.personal.psu.edu/ref7/apparatus/ 2005%20competition/flores.htm Source: http://background-wallpaper.110mb. com/background-wallpaper-fractals2.php

3. Assessing the Impact of Complexity Attributes on System Development Project Outcomes Ph. D. Dissertation of Sarah Sheard Doctoral Advisory Committee Dr. Ali Mostashari, School of Systems and Enterprises, Chairman Dr. Dinesh Verma, Dean, School of Systems and Enterprises Dr. Arthur Pyster, School of Systems and Enterprises Dr. Brian Sauser, School of Systems and Enterprises Dr. Tal Ben-Zvi, Howe School of Technology Management

4. 4 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Goal: Use Complexity Measurement on SE Projects  Determine which measures of complexity might matter to project outcomes How are things complex? What are those things? How might you apply these measures to projects?

5. 5 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC 6 Types of Complexity  Structural Size (SS) Connectivity (SC) Inhomogeneity (SI)  Dynamic Short-term: Operational (DS) Long-term: Evolution (DL)  Socio-Political (SP) Organizational instability Organizational structure Stakeholder Test & Operations Management philosophy 5

6. 6 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC What is Complex? Projects Teams Systems EcoSystem Stakeholders Entities

7. 7 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Methodology 7 Phase 2 Create Survey Phase 1 Collect Measures Phase 3 Survey Phase 4 Analyze

8. Two Outcome Groups 8 Sig: p<0.05; Very sig: p<0.001 Copyright  2012 Third Millennium Systems, LLC

9. 9 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Q32 ‘The project frequently found itself in a fog of conflicting data and cognitive overload.’ Do you agree with this statement? (1)Strongly Agree (2)Agree (3)Neutral (4)Disagree (5)Strongly Disagree Q16d. “Approximately how many system-level requirements did the project have initially? Difficult requirements are considered difficult to implement or engineer, are hard to trace to source, and have a high degree of overlap with other requirements. How many system requirements were there that were Difficult? (1)1-10 (2)10-100 (3)100-1000 (4)1000-10,000 (5)Over 10,000 Q38.“Where did your project fit, on a scale of Traditional, Transitional, or Messy Frontier, in the following eight attributes?” 38f. Stakeholder relationships: (1) Relationships stable; (2) New relationships; (3) Resistance to changing relationships. Wording of Questions 6 9 13 Credits: Q16: Valerdi, COSYSMO, Q32: Ring, Q38F: MITRE

10. 10 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Top Complexity Variables Influence Outcomes Q38f-Changing Stakeholder Relationships & Resistance Changing Relationships Resistance New Stakeholder Personnel + - Schedule Delay Performance Shortfall Imperfect solutions Late decisions Power Struggles Cost overrun Q32- Cognitive Fog More Tests, More Data Usable Inexpensive solutions Q16d-Difficult Requirements Hard to trace to source High Overlap Hard to Implement # Architecture option studies Expensive solutions Political Arguments Requirements changes Instability and Conflicting Data Stakeholder Clarification Wrong decisions Rework 10

11. 11 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Results Black text: Significant (p<0.001). White text: Significant (p<0.05). The three top variables are strong predictors of cost overrun, schedule delay, and performance shortfall 11

12. 12 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Significance, Unsorted S=significant at p<0.05 level; VS= significant at p<0.001 level 12

13. Subsets Size Proj Mgmt Requirements Stakeholders Conflict Uncertainty Changes Skills 13 Copyright  2012 Third Millennium Systems, LLC

14. 14 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Coherence 14

15. 15 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Conclusions  Three measures that can be taken early or in the middle of a project predict cost overrun, schedule delay, and performance shortfall  Twenty other measures predict performance, cost, or schedule, and can presumably be used together for a better prediction  Much more work needs to be done to make this practical

16. 16 Balance ● Growth ● Connections Third Millennium Systems Copyright  2012 Third Millennium Systems, LLC Potential Future Work  Application of results to projects Identify ways to reduce complexity Develop heuristics for “enough reduction” Architecture complexity, requirements complexity, stakeholder complexity, test complexity, etc. measures can be identified and tested Build a “complexity referent” to compare a program to...how much complexity is typically reduced at what points 16