IWFST20051 A Research Framework for Empirical Software Engineering Collaboration and Its Application in a Software Development Project Yoshiki Mitani*, Nahomi Kikuchi**, Tomoko Matsumura***, Satoshi Iwamura****, Mike Barker***, Ken-ichi Matsumoto*** *IPA/SEC,NAIST(EASE),**IPA/SEC,***NAIST(EASE),****NTT Software(EASE) Demonstrate and verify empirical software engineering methods A new research framework In response to government policy Field data sharing between Industry & Academia Concept of a “Macro-measurement tools” Target project: A Governmental Multi-Vendor Consortium type Software Project IPA: Information Technology Promotion Agency, Japan SEC: Software Engineering Center EASE: Empirical Approach to Software Engineering project NAIST: Nara Institute of Science and Technology Information-technology Promotion Agency, Japan Software Engineering Center

IWFST20052 In-process Project Measurement and Feedback Structure Bug Tracking System Mailing-list Management System Review Report Sheet Mail Bug Report XML Standard Data Format RDB EPM Basic Analyzer EPM Extended Analyzer Code Clone Analyzer Collaborating Filtering tool Development Environment Program Development Bug Management Review Benchmark Data Report SEC Benchmark Database (over 1,000projects) Configuration Management System Source Code Feedback to in-process Software Development Project 40 Management Items Operation Rule, Input Web Template Operation Rule 20 Management Items Operation Rule, Input Template (CVS) (GNATS) Operation Rile 400 Items Q&A Interview to PM Q&A Check-list Interview Check-list Analyzer 400 Management Items

IWFST20053 Table1 Measurement Data Items Proceeding Project Data Data itemReview record, Bug report(40 items), Specification Change report, Inter company mail with basic document, Basic Design Document, CVS repository with source code ToolCVS, GNATS, Mailman, EPM Data input form & template: Review report form, GNATS web template form Operation ruleCVS, GNATS, Mail, EPM Collected data are kept safely in SEC secret data room. Data collecting Method Individual IT vender development process: Weekly collect data by media from CVS and GNATS. Integration test process: Use single GNATS system and collect data from it. CVS data are same as individual development process. Ended Project Data/ Bench mark Data Data itemSEC defined 400 bench mark data items, IT skill data based on IPA/ITSS Data collecting Method SEC defined bench mark data input form( EXCEL template) Data collection timing; end of basic design and end of development Project Context Monitoring project Observe whole development process, Join to project meeting. EPM: Empirical Project Monitor ITSS: Information Technology Skill Standard

IWFST20054 Table 2 Data Analysis Example Basic Analysis Source line of code transition, Check-in opportunity, Check-out opportunity & frequency, Number & opportunity of mail, Number & opportunity of bur report, Pending bug transition, Residual bug transition, Extended Analysis Review quality analysis from IEEE 982.1(1988):Fault Density, Defect Density, Measurement from PSP (Personal Software Process) ODC (Orthogonal Defect Classification): Review quality File renewal analysis Bench mark Analysis Source Code Clone Analysis Retrieve similar project from 1,000 past project benchmark data. Skill data Analysis Analyze human factor from IT skill data of development persons Code Clone Analysis

IWFST20055 Fig.1 “Macro Measurement Tools” and its Management & Operation for Software Project Measurement ASDP: Advanced Software Development Project CVS, GNATS, Mailman EPM,400itemBenchmarkData,ITSS EASE project, SEC MEXT, METI Prime Minister Leadership Measurement Target Project Software Tools Measurement Platform Industry & Academia Collaboration Framework Ministry Office Policy Government Policy Macro Measurement Tools Trigger Budget and Policy National Policy A&I Collaboration Management & Operation Measurement Platform Management & Operation Software Tool Management & Operation Development Project Management & Operation EPM: Empirical Project Monitor ITSS: Information Technology Skill Standard EASE: Empirical Approach to Software Engineering SEC: Software Engineering Center MEXT: Ministry of Education, Culture, Sports, Science and Technology METI: Ministry of Economy, Trade and Industry

IWFST20056 The major factors to realize measurement and data-sharing The fundamental I&A collaboration research framework Concept and equipment of measurement platform Thoughtful measurement plan Total framework coordination Acknowledgment This work is supported by IPA/SEC, METI and MEXT of Japan as part of the Comprehensive Development of e-Society Foundation Software program. We thank the researchers in SEC and the EASE project who kindly support our project. Information-technology Promotion Agency, Japan Software Engineering Center