University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE1 CS 577a FCR Feedback, Fall 2009 Winsor Brown, Barry Boehm, Supannika Koolmanojwong October 30,

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE2 ARB Review Success Criteria 2 FCR For at least one architecture, a system built to that architecture will: Support the Ops Concept Satisfy the Requirements Be faithful to the Prototype Be buildable within the budgets and schedules in the Plan Show viable business case Key stakeholders committed to support Foundations Phase (to DCR) DCR For the selected architecture, a system built to the architecture will: Support the Ops Concept Satisfy the Requirements Be faithful to the Prototype Be buildable within the budgets and schedules in the Plan All major risks resolved or covered by risk management plan Key stakeholders committed to support full life cycle

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE3 Overall FCR Feedback Generally done well: presentations, client rapport Reconcile FCR content with ARB Success Criteria Define ALL key terms, acronyms in SID-glossary If youre deferring or skipping a normally-included artifact, explain why (e.g. COTS internals unavailable) and note in SID-document tailoring section. Occassional order changes without telling us in a modified agenda Role of primary DEN/remote student often mis-stated –IS System/Project Engineer (S/PE) –Includes IIV&V (also done by second DEN/remote student) –Includes Shaping (and re-shaping throughout semesters) 3

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE4 Overall FCR Feedback - 2 When asked a question: –Give the answer in brief, this will help you in time management and the Review Board will get the desired information –Do NOT answer back while Review Board attempts to provide guidance Many had poor time management; indicated that presentation(s) had NOT been practiced Occassional pointing at laptop screen, not projected image Very occassionally, slides with NO value added At least one team used "template" from 577a 2008: implies an unethical process 4

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE5 OCD Feedback (1) Generally done well: Organizational goals, Operational concept, System boundary and organizational environment. Some benefits chains need rework: –Added stakeholders: users, clients, developers, IIV&Vers, database administrators, maintainers, interoperators, suppliers –Assumptions are about environment not about outcome –Involvement/use of system before system is built System boundary diagram –If you are using the component in/for your system, remove it from environment, e.g. PHP,.NET framework. 5

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE6 OCD Feedback (2) Organization Goals –Are Benefits Chain End Outcomes, –NOT project Initiative contributions Identify Levels of Service properly –100% availability, 100% reliability - not feasible! –Make sure you can measure LOS goals Prototypes and System are NOT the same (usually) Business Workflow –Use activity-type diagram –Illustrate business activities Not technical/system activity May not even see system explicitly u 6

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE7 Current Business Workflow Example from 2008

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE8 Prototype Feedback Generally done well: GUI Prototypes, Good understanding of clients needs Prototype all high-risk elements, not just GUIs –COTS interoperability, performance, scalability Use user/client-friendly terms –John Doe, 22 Elm St. not abstractions Name1, Addr1 –Use as an opportunity to gather more information and/or examples Identify end users and try to get feedback from end users Focus on important and high priority requirements (initially) 8

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE9 SSRD Feedback Generally done well: Project and Capability requirements, OCD-SSRD traceability Prioritize all the requirements Propagate LOS goals from OCD into SSRD or drop LOS requirements from SSRD (and SSAD) Distinguish between client imposed requirements (SSRD) and developer choice solution (SSAD) Make sure all requirements are testable Qualify 24/7 Availability with noted exceptions Update the new requirements in WikiWinWin tool There is no such thing as an implicit requirement 9

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE10 Result of a good after class question by a team doing NDI/NCS intensive but who found SSRD contents useful. –Why not requested? Because you can't impose "requirements" on COTS or NCS; alternatively, COTS/NCS capability become the requirements –Next year we will have NDI/NCS submit WinWin Report as part of packages –This year: put information in SID; they are needed for Test Case Development. No-SSRD Feedback

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE11 SSAD Feedback Generally done well: Overall views Follow UML conventions (arrows, annotations, etc.) Generalization of actors Uncommon mistakes in use-case diagrams –Two actors-one use case (means BOTH present) –Arrow direction for or Devil is in the detail; simple is the best Only two teams (3 and 14) had an adequate start on Information & Arctifacts Diagram Read the exit criteria for the milestone carefully 11

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE12 Team 3 Information & Artifacts

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE13 Team 14 Information & Artifacts

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE14 LCP Feedback - 1 Generally done well: overall strategy, roles and responsibilities Fill in 577b TBDs Identify required skills for NN new team member(s) (577b; if needed to meet "team size") Show (concentrate on) your future plan; not the past Full Foundations [nee Elaboration] phase plan Dont plan ONLY for documentation –Include Modeling –Include Prototyping; coding; executable architecture 14

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE15 LCP Feedback - 2 COCOMO drivers –Usually differ per the module –PMATs rationale was often wrong: CS577 projects' process maturity is between 2 and 3 NOBODY did the detailed check list in the SwCEwCII book! –Some driver rationales were "ridiculous" Add DEN Student interactions to the Gantt Chart –IIV&V –System/Project Engineer (includes Shaping) Add maintainers responsibilities 15

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE16 FED Feedback Generally done well: Business case framework, risk analysis Specify LOS feasibility plans Include training, operations, maintenance, opportunity costs/effort Few had developers hours as cost Try to quantify benefits, show return on investment Change ROI to reflect on-going costs (possibly savings) Distinguish one-time from annual costs in business case Benefits start in mid 2010 (go to 6 month granularity); Costs start mid 2009 Elaborate process rationale Complete section 6 – COTS Analysis 16

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE17 SID Feedback Requirements traceability –OC WinC SSRD SSAD –Update every time there is a change Update Glossary! Glossary MUST have all system/project specific terms –Non-standard (unusual uses) 17

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE18 QFP Generally done well Some missing traceability injection-removal matrix Some seemed to try to snow us with data, not present just a quick summary Did NOT specify type of "Peer Review" –Pass around or Buddy Check (NOT desirable: no record of concerns) –Agile Artifact Review –Agile Internal/Informal (Role Based) Peer Review Will have lots more to say at DCR! 18

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE19 Remote Students: System/Project Engineer (IIV&V & Shaping) Generally done well Constructive comments-how to improve Some Shapers did not follow instructions: –No WinC summary (mostly numbers) [I am expecting update by for those told] –Other parts incomplete or mis-understood 19

University of Southern California Center for Systems and Software Engineering (C) 2009 USC CSSE20 Things to improve Presentation – communication skill –One word wrong could lead to billion $ loss. Practice in front of others Be concise and precise Consistencies among each artifact Team work vs. integrated individual works Prepare your client: –Tell them what an ARB is (use agenda, success criteria) –Tell them what to expect from ARB Time management –Get in and set-up ASAP –Have documents & client present 20