Metrics for SmartApps 10.12

Contents Software Quality Models ISO 9126 SQuaRE Boehm’s Quality Model Quality Characteristics Regular Apps / Software Program IoT Apps Quality Model Comparison Proposal for SmartApps Motivation for Study References

We need basis for quality metrics….

ISO 9126 Software Quality Characteristics http://www.sqa.net/iso9126.html

Development SQuaRE (Software product Quality Requirements and Evaluation) - a more extensive series of standards to replace ISO/IEC 9126 ISO 25010 has eight product quality characteristics (in contrast to ISO 9126's six), and 31 sub-characteristics Functionality Efficiency Compatibility *new Usability Reliability Security *new Maintainability Portability

Boehm's Quality Model - 1978 Portability, the extent to which the software will work under different computer configurations (i.e. operating systems, databases etc.). Reliability, the extent to which the software performs as required, i.e. the absence of defects. Efficiency, optimum use of system resources during correct execution. Usability, ease of use. Testability, ease of validation, that the software meets the requirements. Understandability, the extent to which the software is easily comprehended with regard to purpose and structure. Flexibility, the ease of changing the software to meet revised requirements. http://www.sqa.net/softwarequalityattributes.html

Characteristics of IoT Applications http://www.ijcee.org/vol8/915-SIM2016-208.pdf

Quality Model Comparison Regular App / Program IoT Applications (measurable using static analysis – looking through the source code only) (quality attributes for evaluating IoT applications - not purely source code only) Reliability Functionality Efficiency Maintainability Portability Reusability https://link.springer.com/chapter/10.1007/978-3-540-73460-4_26

Characteristics of SmartApps Synchronous External HTTP Requests Preferences - Capabilities State Asynchronous External HTTP Requests Events and Subscriptions Notifications Devices Parent-Child SmartApps Modes Routines Scheduling Time Sunset and Sunrise

Metrics for SmartApps Security Reliability Efficiency Maintainability Portability Software quality characteristics (based on SQuaRE - Systems and software Quality Requirements and Evaluation) applicable to SmartApp/ IoT source code - measureable by analyzing the source code only (static analysis)

My Focus SmartApp IoT quality metrics measurable through static analysis – based on the ISO standards Challenge: Software quality characteristics are too abstract – ISO, ISO /IEC SQuaRE, Boehm’s software quality model, etc. How to associate/ relate the SmartApp rules applied to CodeNarc into a quality metrics model

Quality Characteristic Metric 1. Maintainability Cyclomatic number 2. Efficiency Number of unused code 3. Efficiency Number of duplicate code 4. Reliability Number of empty if / switch cases, empty loops, etc. 5. Reliability Number of inconsistent return value 6. Reliability Number of possible ArrayIndexOutOfBounds Exception 7. Functionality Number of incorrect use of Elvis operator 8. Reliability Number of unhandled null values 9. Maintainability Number of undocumented HTTP requests 10. Functionality Number of exposed endpoints 11. Portability Number of recurring short schedules 12. Portability Number of chained runIn calls 13. Functionality Number of unclear subscriptions 14. Functionality Number of unspecific subscriptions 15. Functionality Number of dynamic method execution 16. Functionality Number of hardcoded SMS messages 17. Efficiency Number of busy loops 18. Efficiency Number of synchronized() use … ? -based on SmartThings documentation, manual specifications, code review guidelines and best practices, other applicable standard software metrics, static analysis tool rules, etc.

SQuaRE-based research framework example 13 Quality Characteristics - 47 product metrics - 18 quality in user metrics This research is not related to static analysis. It just proposes a framework… To test the framework – they used user testing not static analysis

SQuaRE-based research framework example Function Suitability indicates whether the software functions satisfy users’ needs X = 1 – A / B A = Number of functional requirements not implemented B = Number of functional requirements

Question… Does coding rule compliance have a correlation with end-user satisfaction? (By checking only the source code quality --static analysis) Direct effect: better code, more flexible to change, maintainable, efficient, etc. (developer perspective) Indirect effect? User satisfaction? (user perspective) Examples: “Better Code for Better Apps: A Study on Source Code Quality and Market Success of Android Applications” “Impact of user satisfaction on Software quality in use “

Motivation for the Study Use metrics to design quality SmartApps Who develops SmartApps? SmartThings users, other developers They become not only users but developers as well Customize it according to their needs People who buy IoT products are more likely interested in technology -> coding..etc. but not every one knows how to code Establishing quality metrics for SmartApps can be helpful for new developers especially if they want to distribute their app to the SmartThings community Encourage growth of IoT developer community

References 1. A Quality Model for Evaluating IoT Applications Mi Kim, Department of Computer Science, Soongsil University, Seoul, Korea http://www.ijcee.org/vol8/915-SIM2016-208.pdf 2. Integrating Quality Models and Static Analysis for Comprehensive Quality Assessment K. Lochmann, et. al, Technische Universität München, Garching b. München, Germany http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.702.4750&rep=rep1&type=pdf

References 3. A SQuaRE-based software quality evaluation framework and its case study H. Nakai, et. al, Waseda University, Tokyo, Japan https://www.researchgate.net/publication/313588217_A_SQuaRE-based_software_quality_evaluation_framework_and_its_case_study 4. A Framework for Measuring and Evaluating Program Source Code Quality H. Wahizaki, et. al, Chiyoda-ku, Tokyo, Japan https://link.springer.com/chapter/10.1007/978-3-540-73460-4_26

End