The Experience Factory May 2004 Leonardo Vaccaro

Leonardo Vaccaro The Experience Factory Introduction Reuse of products, processes and experience originating from the system life cycle is seen today as a feasible solution to the problem of developing higher quality system at lower cost.

Leonardo Vaccaro The Experience Factory Outline Importance of Experience Factory in the Software Business Importance of Experience Factory in the Software Business The Experience Factory The Experience FactoryThe Experience FactoryThe Experience Factory Software Engineering Laboratory Software Engineering Laboratory Q-Labs Experience Management System Q-Labs Experience Management System Experience Factory Implications Experience Factory Implications

Leonardo Vaccaro The Experience Factory Experience Factory (Definition) It is a logical and/or physical organization that supports project developments by analyzing and synthesizing all kinds of experience, acting as a repository for such experience, and supplying that experience to various project on demand

Leonardo Vaccaro The Experience Factory Importance of Experience Factory in the software business The software business requires understanding, continuous improvement, and the packaging of experience for reuse. Reusing experience in the form of processes, products and other forms of knowledge is essential for improvement Experience needs to be packaged Experience must be evaluated Software development must support reuse of experience Packaged experiences need to be integrated

Leonardo Vaccaro The Experience Factory Experiences Factory Schema (Victor Basili)

Leonardo Vaccaro The Experience Factory Project Organization Produce and maintain software Provides the analysis organization with –project and environment characteristics –development data –resource usage information –process information

Leonardo Vaccaro The Experience Factory Quality Improvement Paradigm The basic methodological device for the Experience Factory Developed by Victor Basili Is the result of the application of the scientific method to the problem of software quality improvement Based upon the notion that improving the software process requires the continual accumulation of evaluated experiences (learning) in a form that can be effectively understood and modified (experience models) into a repository of integrated experience models (experience base) that can be accessed and modified to meet the needs of the current project (reuse)

Leonardo Vaccaro The Experience Factory Quality Improvement Paradigm (cont.) It is articulated in six steps:

Leonardo Vaccaro The Experience Factory 1. Characterize Understand the environment based upon –Available models –Data –Intuition Establish baselines with the existing business processes in the organization and characterize their criticality

Leonardo Vaccaro The Experience Factory 2. Set Goals Set quantifiable goals for successful project and organization performance and improvement. –Use of Goal \ Question \ Metric Paradigm Use of Goal \ Question \ Metric ParadigmUse of Goal \ Question \ Metric Paradigm The reasonable expectation are defined upon the baseline provided by the characterization step

Leonardo Vaccaro The Experience Factory 3. Choose Process Choose the appropriate processes for –Improvement –Support methods and tools The processes must be consistent with the goals that have been set

Leonardo Vaccaro The Experience Factory 4. Execute Perform the processes –Constructing the products –Providing project feedback based upon the data on goal achievement that are being collected

Leonardo Vaccaro The Experience Factory 5. Analyze At the end of each specific project, analyze the data and the information gathered to –Evaluate the current practices –Determine the problems –Make recommendation for future project improvements.

Leonardo Vaccaro The Experience Factory Analysis Organization Processes the information received from the development organization Returns direct feedback to each project, together with goals and models tailored from similar project.

Leonardo Vaccaro The Experience Factory 6. Package Consolidate the experience gained in the form of new, or updated models Store it in a experience base so it is available for future projects.

Leonardo Vaccaro The Experience Factory Support Organization Sustains and facilitates the interaction between developers and analysts Saving and maintaining the information Making it efficiently retrievable Controlling and monitoring the access to it.

Leonardo Vaccaro The Experience Factory Experience Base Contains an accessible and integrated set of analyzed, synthesized and packaged experience models that capture past experience

Leonardo Vaccaro The Experience Factory Packaged Experience The experience factory can package all kinds of experience Resource models Change and defect models Process definition and models Product models A variety of quality models

Leonardo Vaccaro The Experience Factory Experience Package Is the main product of the experience factory The content of this product vary upon the kind of experience clustered in the package There is a central element that determines what the package is.

Leonardo Vaccaro The Experience Factory Examples of Experience Package Product Packages Product Packages Processes Package Processes Package Relationship Packages Relationship Packages Tool Packages Tool Packages Management Packages Management Packages Data Packages Data Packages

Leonardo Vaccaro The Experience Factory Product Packages Have as their central element a product, clustered with the information needed to reuse it and the lessons learned in reusing it Examples:ProgramsArchitecturesDesigns

Leonardo Vaccaro The Experience Factory Processes Packages Have as their central element a process, clustered with the information needed to execute it and lessons learned in executing it. Examples: Processes model Methods

Leonardo Vaccaro The Experience Factory Relationship Packages Have as their central element a relationship or a system of relationship among observable characteristics of a software project. These packages are used for analysis and/or forecast of relevant phenomena Examples: Cost and defect models, Resource models

Leonardo Vaccaro The Experience Factory Tool Packages Have as their central element a specific tool either constructive or analytic. Examples: Code generator (constructive) Code generator (constructive) Static analyzer (analytic) Static analyzer (analytic)

Leonardo Vaccaro The Experience Factory Management Packages Have as their central element any container of reference information for project management. Examples: Management handbooks, Management handbooks, Decision support models Decision support models

Leonardo Vaccaro The Experience Factory Data Packages Have as their central element a collection of defined and validated data relevant for a software project or for activities within it. Examples: Project databases Project databases Quality records Quality records

Leonardo Vaccaro The Experience Factory Goal \ Question \ Metric Paradigm The mechanism used by the Quality Improvement Paradigm for defining and evaluating a set of operational goals using measurement. Represents a systematic approach for tailoring and integrating goals with models of the software processes.

Leonardo Vaccaro The Experience Factory Goal \ Question \ Metric Paradigm 1. Development of goals 2. Generation of questions that define the goals 3. Indication of metrics that answer the question Objective: Minimize the number of defects

Leonardo Vaccaro The Experience Factory Software Engineering Laboratory A software organization that has for a long time recognized the value of accumulation and reuse of experience is the NASA Goddard Space Flight Centre which has developed since 1977 the Software Engineering Laboratory

Leonardo Vaccaro The Experience Factory Software Engineering Laboratory Is a very advanced examples of the concept of Experience Factory. The experience packages developed by the Software Engineering Laboratory have mainly focused on project management and control, acquisition and tailoring of new technologies for software development and maintenance

Leonardo Vaccaro The Experience Factory The Software Engineering Laboratory Organization

Leonardo Vaccaro The Experience Factory Software Management Environment The most interesting experience packages developed by Software Engineering Laboratory Is a set of data, tools manuals and analysis techniques supplied to the project management in order to –control the execution of project, –compare it with similar ones –detect and analyze problems, –identify solutions

Leonardo Vaccaro The Experience Factory Q-Labs An internationally distributed software engineering consulting company with office in Europe as well as in the United States Needed an infrastructure to share experience between employees no matter where they are located Q-Labs Experience Management System

Leonardo Vaccaro The Experience Factory Experience Management System A physical implementation of the Experience Factory Composed of –Content Data, Information, Experience –Structure The way the content is organized –Procedures Instructions on how to use, package, delete and update the experience –Tools Visual Query Interface

Leonardo Vaccaro The Experience Factory Q-Labs Experience Management System Stores documents and their description in the Experience Base The search is performed using a tool called Visual Query Interface Visual Query InterfaceVisual Query Interface Experience packages in the context of Q-Labs are single computer document –Texts –Slides –Graphs –Scanned images –Numeric data

Leonardo Vaccaro The Experience Factory Experience Factory Implications Offers an organizational structure that –Separates the product development focus from the learning and reuse focus. –Supports learning and reuse –Generates a tangible corporate asset in the form of packaged experiences. There are costs involved in instituting such a program. –Based upon the SEL experience where a full measurement program has been in progress for over 14 years, project data collection overhead is estimated to be about 5% of the total project cost.

Leonardo Vaccaro The Experience Factory References “ The Experience Factory” – Victor Basili, Gianluigi Caldiera, Dieter Rombach “Implementing the Experience Factory concepts” – Victor Basili, Mikael Lindvall, Patricia Costa Software Engineering Laboratory “Lessons Learned about Structuring and Describing Experience for Three Experience Base” – Mikael Lindvall, Michael Frey, Patricia Costa, Roseanne Tesoriero

Leonardo Vaccaro The Experience Factory Visual Query Interface Visualize the content of the Experience Base The attributes are used to describe the experience packages –Package Id –Name –Submitted X axis represents the “Open Date” Y axis represents “The Person Responsible” The color represents the “Costumer”