Utilizing CI environment for efficient and effective testing of NFRs  - Liang Yu Date: 2019-05-13

CI NFR Research area Target CI-NFR: NFR testing by using CI environments CI NFR CI: Continuous Integration NFR: Non-Functional Requirement Target

Motivation Quality focused NFR [1] Security Usability Reliability Maintainability Performance [1] D. Mairiza, D. Zowghi, N. Nurmuliani, An investigation into the notion of non-functional requirements, in: Proceedings of the 2010 ACM Symposium on Applied Computing, ACM, 2010, pp. 311–317

Why we use continuous integration Prevent faults Fast feedback Automation

Why we select CI Basic CI pipeline Automation Dev Teams source control server CI server build test verdict trigger feedback commit changes

Goal and research questions What we have done Data analysis Workshop Conference papers Journals articles Goal and research questions Search string Findings Inclusion&exclusion criteria Systematic Literature Review [2] on CI-NFR Findings CI environments are underutilized for NFR testing Very low ratio of industrial studies found Nine testable NFR reported New model CI baseline model for NFR testing [2] Barbara A. Kitchenham, Guidelines for performing Systematic Literature Reviews in Software Engineering Version 2.3, Keele University and University of Durham, EBSE Technical Report, 2007

CI baseline model

Next step Next step plan Case study design Objective: verify and validate the CI baseline model Research method: case study[3] Case study design Technique: Semi-structure interview Obervation Documentary Data: Qualitative Quantitative [3] P. Runeson, M. Höst, Guidelines for conducting and reporting case study research in software engineering, Empirical software engineering 14 (2) (2009) 131.

Augmented CI components/probes The plan Augmented CI environment Baseline model NFR Testing Real-time quality Analysis & prediction Augmented CI components/probes

Perceived benefits of CI-NFR Get feedback and prevent faults fast and earlier Deliver new features fast with NFR secured Reduce cost and time

Summary CI NFR

Interested? Contact: Liang Yu <liang.yu@bth.se> Q&A Interested? Contact: Liang Yu <liang.yu@bth.se>