1. DevOps – Test Automation for IOTs
Santosh Vaswani – Technical Manager
Govind Soni – Member of Technical Staff
eInfochips Ltd.

2. Abstract
IoT – Internet of Things is the growing market. With this there is a need of testing IoT Devices in integration with different Software Components, Firmware and Hardware. In this presentation we are going to talk about:
DevOps continuous testing with end-to-end test scenarios in web/mobile and devices
Challenges in continuous testing of IoT Devices as part of DevOps and techniques to overcome these challenges.

3. Connected Devices Time allocated: 2 Mins
80% revenue from billion dollar connected devices companies
10 millions devices enabled so far covering core domains such as Home Automation, Security Surveillance, Medical Devices, Aerospace, Industrial Automation and Consumer Electronics
Our core expertise / Service offering accounting for our major revenue segment is around connected device life cycle management, cloud enablement / integration, and workflow automation (DevOps)
Our experience in workflow automation includes coverage of entire Development and Operations ecosystem taking to account organization’s people, process and technology needs.
For one of our retail IoT Customer (Kroger) we helped them in integrating on Premise smart devices such as Cameras, Bar Code Scanners, Electronic Cart and electronic display to Cloud and manage their life cycle. The end solution allowed Kroger to monitor (live) customer movement on shop floor and provide personalized shopping experience.

4. Connected Devices Testing – Business Challenges
Agile enterprises require a software test environment for continuous testing.
IoT device-makers work with sensors, data storage, electro-mechanical parts and microprocessors.
Enterprise software projects require faster response time to real world problems.
In order to bring the product to market faster, it is required that they are able to test each and every element in real time.
Business teams have three options for addressing
inadequate automation:
Invest heavily in manual tests
Delay the release
Give up
Customer’s major challenges in involving Vendors:
Risk of not getting successful results
Slow automation due to tightly coupled, complex automation framework unable to support devices
Inability to provide end-to-end product automation (software, firmware and devices)
Time allocated: 2 Mins
80% revenue from billion dollar connected devices companies
10 millions devices enabled so far covering core domains such as Home Automation, Security Surveillance, Medical Devices, Aerospace, Industrial Automation and Consumer Electronics
Our core expertise / Service offering accounting for our major revenue segment is around connected device life cycle management, cloud enablement / integration, and workflow automation (DevOps)
Our experience in workflow automation includes coverage of entire Development and Operations ecosystem taking to account organization’s people, process and technology needs.
For one of our retail IoT Customer (Kroger) we helped them in integrating on Premise smart devices such as Cameras, Bar Code Scanners, Electronic Cart and electronic display to Cloud and manage their life cycle. The end solution allowed Kroger to monitor (live) customer movement on shop floor and provide personalized shopping experience.

5. DevOps in Product Engineering
IT DevOps and Product Engineering DevOps differences:
Devices
Multiple Product Development Pipelines
Different Process, Different Skills, Different Teams in each Development
Objective: DevOps in PES is not pure play IT by explaining complexity and challenges in implementation of DevOps in Product Engineering Services. Stage the ground for the DevOps capability slide where we are talking about EI’s expertise on continuous integration, Continuous Testing, Continuous Delivery and Monitoring / Feedback.
In a PES development lifecycle, we have multiple development pipelines on different technologies such as for Firmware, Application and Server Side. Each development pipeline need to come to a stage before these are integrated. There are technical drivers that requires DevOps implementation to enable faster release cycles and better customer experiences.
IT DevOps and PES DevOps differences:
Devices
Multiple Product Development Pipelines
Different Process, Different Skills, Different Teams in each Development
We need to work out a homogenous solution for implementation of DevOps by building a larger DevOps circle that covers all these lines of Development.
Mark, so when you implement DevOps, there will be four DevOps – 3 for each Firmware, Mobile / Devices and Applications.

6. DevOps in Product Engineering Line
Time allocated: 2 Mins
80% revenue from billion dollar connected devices companies
10 millions devices enabled so far covering core domains such as Home Automation, Security Surveillance, Medical Devices, Aerospace, Industrial Automation and Consumer Electronics
Our core expertise / Service offering accounting for our major revenue segment is around connected device life cycle management, cloud enablement / integration, and workflow automation (DevOps)
Our experience in workflow automation includes coverage of entire Development and Operations ecosystem taking to account organization’s people, process and technology needs.
For one of our retail IoT Customer (Kroger) we helped them in integrating on Premise smart devices such as Cameras, Bar Code Scanners, Electronic Cart and electronic display to Cloud and manage their life cycle. The end solution allowed Kroger to monitor (live) customer movement on shop floor and provide personalized shopping experience.

7. DevOps Challenges in Continuous Testing
Complex and fragmented Dev Pipeline: With multiple application delivery end points, including web, mobile, and device(s) the team composition and work flow are more challenging to streamline than pure play IT APPs. 
Environment as a Code: Product engineering space brings additional challenges of virtualizing multiple associated devices along with server infra. The concept of ‘Infrastructure as code’ needs to be extended towards offering complete ‘Environment as code’.
Product Pipeline: More variation on product delivery to multiple customer (customization & enhancement) and market segment (Low, mid, & high). This presents varied production environment difficult to reproduce and maintain at Dev stage.
Legacy Device & Solution Support: Product along with cloud infrastructure brings requirement for managing, updating, and maintaining existing devices on field along with new added, again increasing variations and complexity for Devops.
Release cadence: Different component of solution including firmware, web app, mobile app, & pc app have different release cadence making unified release plan a challenge

8. DevOps - Automation Continuous Integration Continuous Testing
Integration of multiple pipelines (Device, Web and mobile) and prepare main and customized builds
Automate and Configure Environment setup
Trigger automated code review/unit tests/regression tests
Generate and publish Results
Send Notification/ and Monitor
Continuous Testing
Virtualization/Simulation of test environment to enabled continuous testing
Automation execution with simulation and physical devices
Performance Testing with network virtualization
Integration Automation framework with CI tool
Parallel Remote automation execution
Continuous Delivery
Auto Build Deployment on Devices and Servers
Manage Rollback deployment on live environment
Automated Alerts on failure scenarios and performance issues
Generate Business Analytical reports for future reference
Continuous Monitoring
Monitor Production and Test environment
Device Health Monitoring
Functional Alerts
Automated Troubleshooting
Automated / SMS Alerts

9. Continuous Test – Automation Solution

10. Continuous Test – Sensor Virtualization
Telegesis Zigbee USB for Sensor virtualization
Sensors’ Firmware is deployed on Telegesis USB
Python Scripts triggers command to Raspberry Pi to generate specific signals from virtualized Sensors
Virtualized Sensors passes signals to CPE over Zigbee
CPE sends information to Servers based on Signals received from Sensors
Python Scripts verifies actual outcome from Servers (Application logs as well as Database
XMPP Interceptor used for testing of System behaviour such as when there are delays
Load / Stress testing from Touchscreen to Server

11. Continuous Test – Build Verification

12. Virtualization for Test Automation
Raspberry Pi
Web Power Switch
Simulation of Cameras using Nvidia cards
Simulation of Hardware Boards – QEMU
Sensor virtualization with Telegesis USB
Network Virtualization using XMPP Interceptor
Serial Port communication with Device
Local Data centre in Cloud (AWS/Azure)

13. Automated Test Scripts
Virtualization – Video Analytics
Object Tracking and Analysis through Video Surveillance
Automated Test Scripts
Video Files - Test Data
Verification of Alerts by Database Query
Web & Database
NVIDIA Card
Alerts
IOT Board with Application / Algorithm

14. References & Appendix
Computer Weekly, 2014, Time for DevOps to get out of the weeds.

15. Questions & Answers

16. Thank You!!!