The Effects of Reuse on Legacy DoD Systems By: Meredith Eiband, Dr. Tom Holzer, Dr. Tim Eveleigh & Dr. Shahryar Sarkani September 18, 2018 * This material is abstracted from a dissertation currently in work for The George Washington University in partial fulfillment of the requirements for the Doctor of Philosophy degree.*

Outline Problem Research Objective Hypotheses Initial Findings

Problem DOD program costs are too high, and development times too long Requested budget for Fiscal Year (FY) 2012: $553.1 billion1 Average acquisition lifecycle of DoD programs is 11 years2 DOD emphasizes reuse to help reduce cost and time to delivery But the “solution” is part of the problem: Over a 7 year period, cost overruns aided in a $919 billion increase to the DoD budget3 Reuse was identified as a source of increased costs, not decreased costs

Problem As a result of the reuse of legacy DoD systems, U.S. DoD programs often experience overruns and technical difficulties What allows some reuse programs to be successful when others aren’t?

Definitions What is reuse? What is considered a legacy system? The integration of an already-developed part (e.g. engine), product (e.g. inventory database) or legacy system (e.g. telemetry processing system) into another context or component What is considered a legacy system? “A system or application in which an organization has already invested considerable time and money”4

Research Objective Identify the factors decision makers need to consider when determining whether or not to reuse legacy systems Develop a framework to improve the reuse decision-making process

Literature Research Common themes within literature: Theoretical frameworks for reuse of legacy systems Cost and economic impacts of reusing legacy systems Software tools and applications improving the reuse of legacy systems

Literature Search by Topic Theoretical work in the areas of: Developing a software lifecycle framework and causes of technological uncertanty5 & 6 Implementing design reuse and reuse strategy7 & 8 Creating a better reuse design based on knowledge management techniques9 Defining reuse as a managerial issue10

Literature Search by Topic Cost & economic work in the areas of: Utilizing the COCOMO model as a tool for tying cost to software development11 Updating the COCOMO model for issues including: reengineering, applications composition, etc.12 Evaluating the impacts of the cost of software and its reuse13 & 14

Literature Search by Topic Software tools & applications work in the areas of: Evaluating reuse through a total system approach15, 16 & 17 Exploring reuse types including abstraction and research directions18 & 19 Formalizing reuse processes20

Hypotheses Hypothesis 1: Decision makers overestimate the quantity and quality of legacy documentation available Hypothesis 2: Decision makers underestimate the criticality of legacy system subject matter expertise Hypothesis 3: Decision makers overestimate the feasibility of integrating legacy systems

Approach Utilized an Interpretive Case Study research method to evaluate a group of legacy programs (HW & SW) for factors critical to reuse Controlled data for factors outside the control of the program Derived a functional set of considerations that should be assessed prior to the reuse of legacy DoD systems

Initial Findings Hypothesis 1: Decision makers overestimate the quantity and quality of legacy documentation available Legacy systems rarely have the level of documentation needed Unscoped efforts directly impact program planning, resources & performance Overestimation of legacy documentation has a negative impact on successful reuse

Initial Findings Hypothesis 2: Decision makers underestimate the criticality of legacy system subject matter expertise Subject matter expertise is essential to understanding legacy systems and reusing them Knowledge recovery is frequently unquantified during planning Underestimating the importance of subject matter experts has a negative impact on successful reuse

Initial Findings Hypothesis 3: Decision makers overestimate the feasibility of integrating legacy systems Integrating legacy systems is more difficult, especially as the legacy system ages Integration assumptions impact program risk profiles Overestimating the feasibility of integrating legacy systems has a negative impact on successful reuse

Future Work What’s next for this research: Utilize collected data to build a framework that identifies areas of consideration prior to reusing a legacy DoD system New framework will aid decision makers in determining whether or not to reuse legacy systems Compare how this research relates to industry Refines the framework based on industry best practices

Summary As a result of the reuse of legacy DoD systems, U.S. DoD programs often experience overruns and technical difficulties Preliminary findings support the hypotheses that decision makers: Overestimate the quantity and quality of legacy documentation available Underestimate the criticality of legacy system subject matter expertise Overestimate the feasibility of integrating legacy systems Utilize the findings to develop a framework to help decisions makers better understand the reuse choice

Suggestions for Future Research Define problem areas that arise after programs have decided to reuse legacy systems Extends research to include post-decision problems Develop an acquisition Framework for reuse of legacy DoD systems that can be integrated into the contracts lifecycle Extends research to contract’s problems prior to reuse Correlate the age of legacy DoD systems and effectiveness of reuse Determines a timeframe after which a system's successful reuse potential decreases dramatically

Questions?

References 1 Office of the Under Secretary of Defense, (2011). Program acquisition costs by weapon system. Washington, DC: US Government Printing Office. 2 Tomczykowski, W. (2001). DMSMS Acquisition Guidelines: Implementing Parts Obsolescence Management Contractual Requirements. Washington, DC: US Government Printing Office. 3 Defense Business Board. (2010). Best business practices for fixed-price contracting. Washington, DC: US Government Printing Office. 4 Defense Acquisition University. (2009, April 21). Legacy systems. Message posted to https://dap.dau.mil/aap/pages/qdetails.aspx?cgiSubjectAreaID=1&cgiQuestionID=28188 5 Ahrens, J. D., & Prywes, N. S. (1995). Transition to a legacy- and reuse-based software life cycle. Computer, 28 (10), 27-36. doi: 10.1109/2.467576 6 Fleming, L. (2001). Recombinant uncertainty in technological search. Management Science, 47(1), 117-132. 7 Gil, N. & Beckman, S. (2007). Design reuse and buffers in high-tech infrastructure development: A stakeholder perspective. IEEE Transaction Engineering Management, 54(3), 484–497. 8 Frakes, W., & Terry, C. (1996). Software reuse: Metrics and models. Computing Surveys (CSUR), 28 (2), 415-435. 9 Hicks, B. J. , Culley, S. J. , Allen, R. D. & Mullineux, G. (2002). A framework for the requirements of capturing, storing and reusing information and knowledge in engineering design. International Journal of Information Management, 22(4), 263-280. 10 Isoda, S. (1992). Proceedings from: The 14th International Conference on Software Engineering. New York, NY: ACM.

References 11 Wang, G., Valerdi, R., & Fortune, J. (2010). Reuse in systems engineering. IEEE Systems Journal, 4 (3), 376-384. doi: 10.1109/JSYST.2010.2051748 12 Boehm, B., Abts, C., Brown, A. W. , Chulani, S., Clark, B., Horowitz, E., Madachy, R., Reifer, D. & Steece, B. (2000). Software Cost Estimation With COCOMO II. Upper Saddle River, NJ: Prentice-Hall. 13 Boehm, B. W. (1981). Software Engineering Economics. Upper Saddle. River, NJ: Prentice-Hall. 14 Gaffney, J. E. & Durek, T. A. (1989). Software reuse—Key to enhanced productivity: Some quantitative models. Information and Software Technology, 31(5), 258-267. 15 Kim, Y. & Stohr, E. A. (1998). Software reuse: Survey and research directions. Journal of Management Information Systems, 14(4), 612-623. 16 Mili, A., Mili, F., & Mili, H. (1995). Reusing software: Issues and research directions. IEEE Transactions of Software Engineering, 21(6), 528-562. 17 Isoda, S. (1995). Experiences of a software reuse project. Journal of Systems and Softare, 30(3), 171-186. 18 Freeman, P. (1983). Proceedings from: The ITT Workshop on Reusability in Programming, New York, NY: ITT Programming. 19 Krueger, C. W. (1992). Software reuse. ACM Computer Survey, 24(2), 131–183. 20 Redwine, S. S. & Riddle, W. E. (1989). Proceedings from ISPW ‘88: 4th International Software Process Workshop on Representing and Enacting the Software Process. New York, NY: ACM.