Project Presentation Document Optimization 11 May 2007 Team members: Chris Catalano Chun-Yu Chang Chris Joson David Matthes Sponsors: Huron Consulting Group Aerospace Corporation

EDD Background Electronic Data Discovery (EDD) is the systematic collection, processing and review of electronic files to support the litigation process. EDD is used in: –Alleged stock-back dating –Government reviews of mergers and acquisitions –Other dirty deals e.g. blackmail, fraud, embezzlement The current processing system was designed for component flexibility and variability. The market place is shifting to an environment that holds speed and automation paramount.

Project Objectives Evaluate the current EDD system against two alternatives. Client: Huron Consulting Group Evaluate SysML as an effective modeling language for systems engineering. Client: Aerospace Corporation

Approach Modeled and compared three EDD systems in SysML. Evaluated the EDD systems from a capital budgeting perspective Evaluated quantitatively our experience with SysML.

Agenda Approach - SysML Model Analysis - Trade Study Evaluation - SysML Usability

Approach - SysML Model SysML is a modeling language for: Defining systems Analyzing systems Communicating different system viewpoints EDD Team captured the following viewpoints: Requirements Diagram Use Case Diagram Block Definition Diagram Activity Diagram

Requirements Diagram Captures EDD requirement hierarchy Provides traceability to EDD components

Use Case Diagram Describes EDD contextual relationships Show interactions between entities and the EDD System

Block Definition Diagram Describes the characteristics Describes Behaviors Describes the structured composition of EDD

Activity Model of Current Process Specifies the flow of inputs/outputs and controls, including sequence for coordinating activities.

Partitions Partitions (swimlanes) show responsibility for each activity. Processing Server – 9 programs Engineer – 9 copies/move Unix – 6 Perl scripts Review Team – 6 QCs Mac Client – 1 activity

Alternative (Attenex or Autonomy) process used to compare with the current process Processing Server – 1 program Engineer – 2 copies/move Unix – 2 Perl scripts Review Team – 2 QCs Mac Client – 1 activity

Advantages of Alternative Process Fewer manual steps Reduced probability of error Simpler to maintain Easier to train Less rigid process Shorter time to process documents

Agenda Approach - SysML Model Analysis - Trade Study Evaluation - SysML Usability

Net Present Value Probability Distribution The goal was to model the financial impact of each alternative over three years using Net Present Value (NPV). NPV is a capital budgeting technique used to estimate and compare cash flows for competing systems and projects. For each system the Net Cash Flow was decomposed, modeled, and run in a Monte Carlo simulation to generate NPV estimates. The results are NPV probability distributions for each alternative t – time n – total project time r – discount rate Ct – net cash flow Co – Initial capital expenditures at time zero

Net Present Value Compared to the baseline, the alternative systems increase the processing speed and the ability to accept projects. The trade off is increased costs. Autonomy: –$2,000,000 initial cost –$250,000 annual maintenance cost Attenex: –$500 per gigabyte processed operational cost How does the increased ability to accept new projects and the increased costs impact the profitability of the systems?

NPV – Results Baseline Mean: $12.0 Million Attenex Mean: $12.3 Million Autonomy Mean: $16.2 Million Baseline Attenex Autonomy Probability NPV ($) NPV Probability Density Function The results assume that the alternatives will increase the number of projects allowed into the system by twice the baseline. Autonomy is being pulled into a higher range of profitability!

Conclusions & Recommendations The model shows that by increasing the opportunity to accept new projects the alternative systems can overcome the increased costs! The future system for Huron will be a hybrid of the alternatives. The process used for a particular project will be dependent on the clients’ requirements. The baseline system, while slower, provides a reliable and cost effective solution. For clients who choose higher speeds at higher costs Attenex would be an ideal fit. (Huron already owns licenses for the software!) It is critical to spread the costs of Autonomy across the three EDD groups. In effect distributing the responsibility for recouping the investment!

Agenda Approach - SysML Model Analysis - Trade Study Evaluation - SysML Usability

Purpose of the SysML Evaluation Aerospace asked us to evaluate SysML to determine how effectively SysML and Rational System Developer worked Evaluate SysML as a modeling language for designing systems Evaluate SysML maturity Determine how useful SysML is for systems engineering design and evaluation Evaluate IBM Rational System Developer Determine how well it supports SysML usage

Approach: Survey Created a Multi-Attribute Utility Assessment Evaluation Hierarchy survey Survey contained 41 questions developed to assess the strengths and weaknesses of SysML and Rational System Developer Questions were answered on a 1 to 5 Likert scale with 5 indicating a positive response Surveyed 8 OR680 Students using SysML Electronic Data Discovery (EDD) Tactical Surveillance Satellite (TSS)

Multi-Attribute Utility Assessment Evaluation Hierarchy Overall Utility Effect On Task Performance Usability * Modified from Adelman & Riedel, Handbook for Evaluating Knowledge Based Systems, 1997 Process Quality Product Quality General Ease Of Use Ease of Learning Flexibility Interface Decomposed the evaluation criteria to evaluate different aspects of SysML and Rational System Developer

Tactical Surveillance Satellite (TSS) Electronic Data Discovery (EDD) Values closer to 5 indicate positive responses Utility Results

Survey Analysis SysML Strengths Overall respondents felt SysML was a good language Scored well in usability and flexibility Weaknesses The main weakness in SysML is that it is difficult to learn –Respondents took hours to become a functional user Rational System Developer Strengths Rational System Developer scored highest in usability Survey indicates that people found Rational System Developer fairly easy to use Weaknesses Survey indicted low scores for ease of training The Interface and product quality also scored lower than other areas

Recommendation to Aerospace SysML SysML is difficult to learn and will require investment in training and time –May not be practical for smaller systems or processes with limited complexity –However, if people are already trained, SysML diagrams ensure consistency and provide effective communication across multiple disciplines Rational System Developer Rational supported the creation of models and helped maintain consistency Process descriptions were created and analysis performed using Rational and SysML SysML is well suited for complicated systems with significant hierarchical decomposition, systems common in the National Security Space domain

Summary Huron asked us to evaluate their current EDD system and two alternatives Used SysML and NPV to perform the analysis Determined that the best solution is a mix of the current system for most clients and Autonomy for clients that require faster processing and can afford the increased cost Aerospace asked us to evaluate SysML to determine how effectively it can support system engineering design and analysis Conducted a survey to help answer this question. The survey found that SysML is a useful tool, but the learning curve is steep

Acknowledgements Heather Howard, Shana Lloyd, and Julie Street, Aerospace Corporation Chris Genter, Huron Consulting Group Professor Laskey, George Mason University Sanford Friedenthal, Lockheed Martin Professor Adelman, George Mason University The TSS Team David Alexander, Kevin Sadeghian, Siroos Sekhavat, and Tom Saltysiak

Future Work Optimize Parametric Diagram to make the model executable Run executable model Compare executable model results with results obtained from Microsoft Excel Distribute SysML survey to future students for a larger sample and further analysis

Questions?

Backup

Component Diagram Decompositions

Parametric Diagram Parametric Diagrams were created to express constraints between value properties and allow to perform an executable model. Executable model used to provide analysis for performance, safety, reliability, throughput, weight, cost, etc. High Learning Curve Lack of Time (Estimation of >20+ additional hours to learn SysML limitations) Inexperience with Simulation Toolkit (Estimation of >30+ hours to execute with toolkit) Inexperienced team with Java (Estimation of >70+ hours to learn Java)

Questions focus on either SysML as a language or IBM Rational System Developer as a tool Most questions will be rated on a scale of 1 to 5 Responses will be averaged together to determine a score for each category Sample Questions Overall, SysML improves the system design process. Rational System Developer provides feedback when processing user commands. SysML was easy to learn. I can easily add model elements to the System model. Sample Survey Questions

Survey Example Survey will have participant answer a series of questions

Webpage mason.gmu.edu/~cchang7

General Status

Schedule

NPV Backup

NPV - Formula Where: t – time n – total project time r – discount rate Ct – net cash flow Co – Initial capital expenditures at time zero

NPV - Assumptions Number of Projects Limitations: The number of projects entering into the system can not be greater than the maximum level of availability. Projects Start and Completion Time: All projects started in a month are assumed to be completed within that month. In practice this assumption can be interpreted as larger scale projects are started early in the month while smaller projects are started later in the month. Minimum Revenue: $2500 is the minimum amount of revenue accepted for a job. Autonomy Costs: The Autonomy system has an initial cost of $2 million dollars and an operational cost of $250,000 annually. Attenex Costs: The Attenex system has an operational cost of $500 dollars per GB processed. Prospective Projects: The level at which prospective projects are found is consistent for all systems. Availability Parameter: The availability parameter is being used to model the size and availability of the queue for incoming projects. Pricing Scheme: The pricing scheme is constant for each system over the three year period. No adjustments have been made to the pricing schemes of the higher cost alternatives. Migration Costs: With the exception of initial software costs, all migration costs are ignored in this model.

NPV- Revenue Inputs (1) Annual Revenue: The annual revenue is the sum of twelve monthly revenue estimates. Monthly Revenue: The monthly revenue is the sum of the revenue for each job accepted and completed in a month. Revenue per Project: The revenue per project is the amount of revenue in dollars that a generated by a project. Projects Accepted: This value is the total number of projects entered into the system each month.

NPV- Revenue Inputs (2) Maximum level of System Availability: The maximum level of system availability is the largest number of projects that can enter into the system each month. Number of Prospective Projects: The number of prospective projects describes the number of projects that are available to be entered into the system. Number of Staff: The number of staff plays a critical role in limiting the number of jobs that can be entered into the system each month. Processing Speed: Processing speed describes the rate at which projects can be pulled through the system.

NPV- Cost Inputs Initial Costs: The costs used to procure new software and equipment for the alternative systems at the onset of the migration. The initial costs are incurred once at the beginning of the project. Maintenance Costs: Monthly costs associated with maintaining the software and hardware systems. The maintenance costs include repairing machines, software upkeep and spare parts. Salary Costs: Monthly costs related to employee salaries. Operational: Monthly costs related to procuring additional equipment, software and the overhead costs related to the building and facilities.

NPV – Parametric Diagram