1. Supporting Continuous Integration in Embedded Software
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Supporting Continuous Integration in Embedded Software
Andrew Graham 09 October 2017
PNSQC ™

2. Agenda Overview Minimum accessibility Abstract communication
SOFTWARE QUALITY CONFERENCE
Agenda
PACIFIC NW
Overview
Our problem
Infrastructure highlight
Our continuous integration process
Minimum accessibility
Abstract communication
Dynamically reconfigurable hardware
PNSQC ™

3. Overview
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Continuous integration is a software development paradigm that encourages the frequent integration of code into a common repository. This simple action is frequently coupled with automated sequences that build, test, and qualify code with the goal of reducing risk (Duvall, Matyas, & Glover, 2007).
Embedded software development is heavily influence by hardware development
The desire to implement continuous integration practices and systems in embedded software is high
PNSQC ™

4. Our Problem Objectives:
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Objectives:
Test software on the hardware it was meant to run on
Accept arbitrary instrument configurations
Change instrument configurations with ease
PNSQC ™

5. Infrastructure Highlight
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
The following applications comprise the continuous integration system used:
Jenkins
Bitbucket
Vsphere
Chef
PNSQC ™

6. Our Continuous Integration Process
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
PNSQC ™

7. Minimum Accessibility
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Minimum accessibility is the amount of additional access, on top of customer-facing interfaces, to the physical product that is tolerable to accommodate continuous integration of the embedded software on hardware.
Use interfaces that will be accessible to end-users
Only explicit exceptions expressly for supporting continuous integration
Understand risk associated with any exception
PNSQC ™

8. Minimum Accessibility
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
PNSQC ™

9. Deciding on Minimum Accessibility
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Questions to guide your decision:
What interfaces are available to a customer? How expressive are they?
User interfaces, programmatic interfaces, physical buttons
What cannot be achieved through an Existing customer interface? Is there a backdoor?
Analyzing diagnostics, error reports, internal hardware states
What things cannot be acquired through any existing interface?
Debug consoles, power operations, physical manipulation of external media/peripherals
PNSQC ™

10. Abstract Communication
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Abstract communication is how a continuous integration system communicates with a device under test while respecting minimum accessibility. This is achieved through intermediary applications, virtual machines, and/or containers that act as representatives to the continuous integration system for the hardware.
Works in concert with minimum accessibility
Use instruments in the test pool
Indirectly leverage benefits of continuous integration system
PNSQC ™

11. Abstract Communication
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
PNSQC ™

12. Abstract Communication Benefits
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Abstracting communication to the instrument with virtual machines or containers provides a few layers of benefits
Exercise customer-facing interfaces
Extend pre-test events into automated tests
VMs (or containers) can be encompassed in configuration management (i.e. Chef or Vagrant)
PNSQC ™

13. Dynamically Reconfigurable Hardware
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Dynamically reconfiguring hardware means taking a device and altering its physical state into another valid configuration. The continuous integration system should tolerate these changes without manual intervention. The testing that the system performs should automatically adjust itself to appropriately test the new structure.
Build off of abstract communication
There may be thousands of possible option permutations
There are likely finite physical resources available to cover as many permutations as possible
PNSQC ™

14. Dynamically Reconfigurable Hardware
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
PNSQC ™

15. Dynamically Reconfigurable Hardware Use Cases
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
If the actual instrument is masked behind an abstract communication interface then its physical state should be easy to update. This is useful in many situations:
Testing edge-case configurations
Facilitating experimental hardware development (especially late changes)
Swapping out failing hardware
PNSQC ™

16. Piecing it Together
SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Assuming some level of risk enabled minimum accessibility
Minimum accessibility enabled abstract communication
Abstract communication enabled dynamically reconfigurable hardware
Combining the three concepts created a durable test environment that allowed embedded software, running on actual hardware, to interface with our continuous integration system
PNSQC ™

17. References SOFTWARE QUALITY CONFERENCE
PACIFIC NW
Atlassian, Inc. (2017, June 17). Bitbucket Server Documentation. Retrieved from Confluence.atlassian.com:
Chef Software, Inc. (2017, June 17). Learn Chef. Retrieved from Chef Docs:
Cloud Bees, Inc. (2017, June 17). Jenkins. Retrieved from Jenkins Documentation:
Duvall, P. M., Matyas, S., & Glover, A. (2007). Continuous Integration: Improving Quality and Reducing Risk. Pearson Education.
HashiCorp. (2017, June 17). Vagrant by HashiCorp. Retrieved from Documentation:
VMware. (2017, June 17). VMware vSphere 6 Documentation. Retrieved from VMware vSphere Documentation:
PNSQC ™