Copyright © 2010 Nara Institute of Science and Technology / Osaka University Standardizing the Software Tag in Japan for Transparency of Development Profes 2010 Masateru Tsunoda †, Tomoko Matsumura †, Hajimu Iida †, Kozo Kubo †, Shinji Kusumoto ‡, Katsuro Inoue ‡, and Ken-ichi Matsumoto † †Nara Institute of Science and Technology, Japan ‡Osaka University, Japan

1 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Background Software failure may cause great economic loss for developers and purchasers. Complex systems are widely used in society. They become important infrastructure for society. Ex. Banking, communication systems, … There may be life threatening system failures. Ex. Air-traffic control system, automobile safety system, … There are high risks of software failure. Software systems are still extending but development periods are decreasing. Demand of reducing cost and increasing reliability.

2 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Issues on Non-transparency Purchasers cannot check the quality of software products during software development. The developer collects project data and analysis it. The purchaser does not know project data and the analysis results. Purchaser Order Deliver Developer Project data Software product Data collection Analysis feedback

3 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Goal and Approach Goal Increasing traceability and transparency of software products and development for purchasers. Approach Propose the software tag to share various data between purchasers and developers. It contains various data collected during development. Key technologies Software tag standard 1.0 Data collection and analysis tools Tag application pattern

Copyright © 2010 Nara Institute of Science and Technology / Osaka University Software Tag Usage Model

5 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Software Tag Usage Model Depend on development style, purchasers,... Typical cases In contract-based development, the purchaser uses the software tag for evaluation of development. Software tag data cannot be public (Only purchasers can see it). Legal dispute is occurred between the purchaser and the developer, the software tag is used for evaluation of their liability. When choosing software components, the developer uses the software tag for the component evaluation.

6 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Contract-based Development Project data is selected/abstracted as the software tag, and the tag is associated with software. The purchaser confirms validity of process and product with the software tag. Purchaser Order Deliver Developer Project data Data collection Analysis feedback Software products Associated Software tag Selected/ abstracted

7 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Legal Dispute between the Purchaser and the Developer Software liability arbiter inspects the software tag to resolve the legal issue. Purchaser Order Deliver Developer Project data Data collection Analysis feedback Software products Associated Software tag Selected/abstracted Software liability arbiter Inspect

8 Copyright © 2010 Nara Institute of Science and Technology / Osaka University When developers search for reusable components, they can select reliable components by evaluating the software tag. Evaluation of Software Components Developer Search for reusable components Evaluation Software system integration Select reliable components Reusable component library Software component Software component Software component

Copyright © 2010 Nara Institute of Science and Technology / Osaka University Key technology: Software tag standard 1.0

10 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Software Tag Standard 1.0 We defined Software Tag Standard 1.0 based on discussions with industry and academic collaborators. Project information (12 elements) Basic information for project and system basic info., system info., development info., project organization, other Progress information (29 elements) Development information of the system requirement, design, coding, test, quality, work load, plan and management, other products ClassificationCategoryNo.Tag ElementExplanation Project Information Development Information 7Development ApproachDevelopment process type or techniques 8Organizational StructureStructure of development organization 9Project DurationInformation of development length Progress Information Programming 19Scale Amount of programming products 20Revisions Amount of changed programs 21Complexity Complexity of programs

11 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Establishing Policy of Software Tag Standard We considered Including information that are important to the purchasers. Simple structure for the purchaser to understand. The balance of the tag elements. On the tag standard, Tag elements are selective. Calculation methods of the elements were not described. Thorough the discussion, we saw that appropriate tag elements set and calculation methods are different for organizations.

12 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Arbitrary Matters Tag elements used in the tag. It is not necessary to use all elements. Metrics used for the tag element. Tag Standard 1.0 shows examples of metrics. Ex. Function point or lines of code can be used for tag element “scale” Timing of sending tag data from the vendor to the purchaser. Ex. at the end of the project, at every phase of development (ex. the end of design phase), at every week,...

13 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Empirical Data Supporting data for the software tag. Ex. source code, design documents, bug reports, progress reports,... Encrypted and stored. Decrypted when serious system failure or legal dispute is occurred to inspect the project. Scale Complexity Lines of code Cyclomatic complexity Source code repository Tag elements Mertics Empirical data Software tag

14 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Standardizing Tag Data Format To store, exchange and reuse the software tag, we settled the draft of the standard software tag format. Based on XML format. Software tag support tools treats software tag format data. We are making the tool which converts existing tools’ data into standard software tag format data. Standard software tag format data Visualization tools Convert tool Software configuration management

Copyright © 2010 Nara Institute of Science and Technology / Osaka University Key technology: Data collection and analysis tools

16 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Software Tag Data Collection Tool CollectTag (1/2) Supports collection of empirical data and creation of a software tag. Automatic data collection mechanisms for 11 tag elements using software configuration management or bug tracking system. Ex. LOC (Scale) and CK metrics (Complexity) are automatically collected from SCM tools (CVS/Subversion). Wizard type user interface.

17 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Software Tag Data Collection Tool CollectTag (2/2) 1. A user selects a tag element, and settles a metric for the element. 2. The user selects data input method. 3. CollectTag generates the software tag elements in standard software tag format. [Programming]-[Scale] (#19) is selected The user can select [Lines of code] or [Function point] Input Manually (LOC) Obtain from repositories

18 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Software Tag Visualization Tool TagReplayer (1/2) Provide fundamental features for integrated visualization of the software tag. Employ the metaphor of video player manipulation for its user interface. Users can replay the progress of the project just like watching video on TV. Align progress information from the software tag as a series of events.

19 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Software Tag Visualization Tool TagReplayer (2/2) Transition of LOC during the project Completion rate of each file, computed from cumulative changed LOC at the end of the project Assigned and completed task history of each member Topics list, mail subject list, and a mail body By moving slider, replay goes to a certain point of time

Copyright © 2010 Nara Institute of Science and Technology / Osaka University Key technology: Tag Application Pattern

21 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Tag Application Pattern (1/2) Catalog to explain which metrics should be used and how to analysis them. Software tag standard 1.0 does not include concrete metrics. It is not easy to settle metrics for many tag elements from scratch. Goal achieved by the software tag Metric indicating whether the goal is achieved or not Metrics indicating whether progress is smooth or not. (Tag elements)

22 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Tag Application Pattern (2/2) Explain usages of each KGI/KPI. KGI/KPI Base measure Measurement method Measurement function Decision criteriaCorrective action Defect density at unit testing Number of defects Collect defects at unit testing Number of defects at unit testing / program size Compare the value between modules. Case1: When the value of a module is too low, number of test cases is suspected of insufficiency. Case2: When the value of a module is too high, quality of module is suspected of low. Case1: Confirm number of test cases of the module, and test the module again. Case2: Code review of the module is conducted again. Program size Measure program size by a program size measurement tool How to measure base values How to compute the value How to analysis the metric How to react based on the analysis

23 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Discussions (1/2) There are projects which involve some companies to establishment of software quality [1][2]. The software tag provides a unique approach to involve software purchasers in the quality improvement framework. Tag elements are not very unique. Some developers disclose progress information at the meeting with the purchaser. Still, the software tag is effective to propagate such a good practice involved the purchaser. [1] The Consortium for IT Software Quality: CISQ – The Global Standard for IT Software Quality. [2] The Quamoco Consortium: Quamoco – The Benchmark for Software Quality.

24 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Discussions (2/2) The tag standard is set as a lightweight set. As a standard used for various projects, the set should be minimal and low cost. The risk of tampering software tag data is low. Difficult to tamper several tag elements recorded by time series with keeping consistency. Not difficult to rebuild the software tag from empirical data.

25 Copyright © 2010 Nara Institute of Science and Technology / Osaka University Conclusions and Future Works Conclusions The software tag contains software development data, and it brings purchasers transparency of software development. We identified 41 items for seeing software process and products, and defined them as the standard tag element set. To support software tag scheme, we made tools for collecting, and analyzing tag data. Tag application pattern explains how to use software tag. Future Works Focus on making international/domestic standards of the software tag. Delivery software tag support tools and tag application pattern.

26 Copyright © 2010 Nara Institute of Science and Technology / Osaka University