1. Metrics for SmartApps
10.12

2. Contents Software Quality Models ISO 9126 SQuaRE Boehm’s Quality Model
Quality Characteristics
Regular Apps / Software Program
IoT Apps
Quality Model Comparison
Proposal for SmartApps
Motivation for Study
References

3. We need basis for quality metrics….

4. ISO 9126 Software Quality Characteristics

5. Development
SQuaRE (Software product Quality Requirements and Evaluation) - a more extensive series of standards to replace ISO/IEC 9126
ISO has eight product quality characteristics (in contrast to ISO 9126's six), and 31 sub-characteristics
Functionality
Efficiency
Compatibility *new
Usability
Reliability
Security *new
Maintainability
Portability

6. Boehm's Quality Model
Portability, the extent to which the software will work under different computer configurations (i.e. operating systems, databases etc.).
Reliability, the extent to which the software performs as required, i.e. the absence of defects.
Efficiency, optimum use of system resources during correct execution.
Usability, ease of use.
Testability, ease of validation, that the software meets the requirements.
Understandability, the extent to which the software is easily comprehended with regard to purpose and structure.
Flexibility, the ease of changing the software to meet revised requirements.

7. Characteristics of IoT Applications

8. Quality Model Comparison
Regular App / Program
IoT Applications
(measurable using static analysis – looking through the source code only)
(quality attributes for evaluating IoT applications - not purely source code only)
Reliability
Functionality
Efficiency
Maintainability
Portability
Reusability

9. Characteristics of SmartApps
Synchronous External HTTP Requests
Preferences - Capabilities
State
Asynchronous External HTTP Requests
Events and Subscriptions
Notifications
Devices
Parent-Child SmartApps
Modes
Routines
Scheduling
Time
Sunset and Sunrise

10. Metrics for SmartApps Security Reliability Efficiency Maintainability
Portability
Software quality characteristics (based on SQuaRE - Systems and software Quality Requirements and Evaluation) applicable to SmartApp/ IoT source code
- measureable by analyzing the source code only (static analysis)

11. My Focus
SmartApp IoT quality metrics measurable through static analysis – based on the ISO standards
Challenge:
Software quality characteristics are too abstract – ISO, ISO /IEC SQuaRE, Boehm’s software quality model, etc.
How to associate/ relate the SmartApp rules applied to CodeNarc into a quality metrics model

12. Quality Characteristic
Metric
1. Maintainability
Cyclomatic number
2. Efficiency
Number of unused code
3. Efficiency
Number of duplicate code
4. Reliability
Number of empty if / switch cases, empty loops, etc.
5. Reliability
Number of inconsistent return value
6. Reliability
Number of possible ArrayIndexOutOfBounds Exception
7. Functionality
Number of incorrect use of Elvis operator
8. Reliability
Number of unhandled null values
9. Maintainability
Number of undocumented HTTP requests
10. Functionality
Number of exposed endpoints
11. Portability
Number of recurring short schedules
12. Portability
Number of chained runIn calls
13. Functionality
Number of unclear subscriptions
14. Functionality
Number of unspecific subscriptions
15. Functionality
Number of dynamic method execution
16. Functionality
Number of hardcoded SMS messages
17. Efficiency
Number of busy loops
18. Efficiency
Number of synchronized() use … ?
-based on SmartThings documentation, manual specifications, code review guidelines and best practices, other applicable standard software metrics, static analysis tool rules, etc.

13. SQuaRE-based research framework example
13 Quality Characteristics
- 47 product metrics
- 18 quality in user metrics
This research is not related to
static analysis. It just proposes
a framework…
To test the framework – they used user testing not static analysis

14. SQuaRE-based research framework example
Function Suitability indicates whether the software functions satisfy users’ needs
X = 1 – A / B
A = Number of functional requirements not implemented
B = Number of functional requirements

15. Question…
Does coding rule compliance have a correlation with end-user satisfaction? (By checking only the source code quality --static analysis)
Direct effect: better code, more flexible to change, maintainable, efficient, etc. (developer perspective)
Indirect effect? User satisfaction? (user perspective)
Examples:
“Better Code for Better Apps: A Study on Source Code Quality and Market Success of Android Applications”
“Impact of user satisfaction on Software quality in use “

16. Motivation for the Study
Use metrics to design quality SmartApps
Who develops SmartApps?
SmartThings users, other developers
They become not only users but developers as well
Customize it according to their needs
People who buy IoT products are more likely interested in technology -> coding..etc. but not every one knows how to code
Establishing quality metrics for SmartApps can be helpful for new developers especially if they want to distribute their app to the SmartThings community
Encourage growth of IoT developer community

17. References
1. A Quality Model for Evaluating IoT Applications Mi Kim, Department of Computer Science, Soongsil University, Seoul, Korea 2. Integrating Quality Models and Static Analysis for Comprehensive Quality Assessment K. Lochmann, et. al, Technische Universität München, Garching b. München, Germany

18. References
3. A SQuaRE-based software quality evaluation framework and its case study
H. Nakai, et. al, Waseda University, Tokyo, Japan
4. A Framework for Measuring and Evaluating Program Source Code Quality
H. Wahizaki, et. al, Chiyoda-ku, Tokyo, Japan

19. End