DevOps – Test Automation for IOTs Santosh Vaswani – Technical Manager Govind Soni – Member of Technical Staff eInfochips Ltd.

Abstract IoT – Internet of Things is the growing market. With this there is a need of testing IoT Devices in integration with different Software Components, Firmware and Hardware. In this presentation we are going to talk about: DevOps continuous testing with end-to-end test scenarios in web/mobile and devices Challenges in continuous testing of IoT Devices as part of DevOps and techniques to overcome these challenges.

Connected Devices Time allocated: 2 Mins 80% revenue from billion dollar connected devices companies 10 millions devices enabled so far covering core domains such as Home Automation, Security Surveillance, Medical Devices, Aerospace, Industrial Automation and Consumer Electronics Our core expertise / Service offering accounting for our major revenue segment is around connected device life cycle management, cloud enablement / integration, and workflow automation (DevOps) Our experience in workflow automation includes coverage of entire Development and Operations ecosystem taking to account organization’s people, process and technology needs. For one of our retail IoT Customer (Kroger) we helped them in integrating on Premise smart devices such as Cameras, Bar Code Scanners, Electronic Cart and electronic display to Cloud and manage their life cycle. The end solution allowed Kroger to monitor (live) customer movement on shop floor and provide personalized shopping experience.

Connected Devices Testing – Business Challenges Agile enterprises require a software test environment for continuous testing. IoT device-makers work with sensors, data storage, electro-mechanical parts and microprocessors. Enterprise software projects require faster response time to real world problems. In order to bring the product to market faster, it is required that they are able to test each and every element in real time. Business teams have three options for addressing inadequate automation: Invest heavily in manual tests Delay the release Give up Customer’s major challenges in involving Vendors: Risk of not getting successful results Slow automation due to tightly coupled, complex automation framework unable to support devices Inability to provide end-to-end product automation (software, firmware and devices) Time allocated: 2 Mins 80% revenue from billion dollar connected devices companies 10 millions devices enabled so far covering core domains such as Home Automation, Security Surveillance, Medical Devices, Aerospace, Industrial Automation and Consumer Electronics Our core expertise / Service offering accounting for our major revenue segment is around connected device life cycle management, cloud enablement / integration, and workflow automation (DevOps) Our experience in workflow automation includes coverage of entire Development and Operations ecosystem taking to account organization’s people, process and technology needs. For one of our retail IoT Customer (Kroger) we helped them in integrating on Premise smart devices such as Cameras, Bar Code Scanners, Electronic Cart and electronic display to Cloud and manage their life cycle. The end solution allowed Kroger to monitor (live) customer movement on shop floor and provide personalized shopping experience.

DevOps in Product Engineering IT DevOps and Product Engineering DevOps differences: Devices Multiple Product Development Pipelines Different Process, Different Skills, Different Teams in each Development Objective: DevOps in PES is not pure play IT by explaining complexity and challenges in implementation of DevOps in Product Engineering Services. Stage the ground for the DevOps capability slide where we are talking about EI’s expertise on continuous integration, Continuous Testing, Continuous Delivery and Monitoring / Feedback. In a PES development lifecycle, we have multiple development pipelines on different technologies such as for Firmware, Application and Server Side. Each development pipeline need to come to a stage before these are integrated. There are technical drivers that requires DevOps implementation to enable faster release cycles and better customer experiences. IT DevOps and PES DevOps differences: Devices Multiple Product Development Pipelines Different Process, Different Skills, Different Teams in each Development We need to work out a homogenous solution for implementation of DevOps by building a larger DevOps circle that covers all these lines of Development. Mark, so when you implement DevOps, there will be four DevOps – 3 for each Firmware, Mobile / Devices and Applications.

DevOps in Product Engineering Line Time allocated: 2 Mins 80% revenue from billion dollar connected devices companies 10 millions devices enabled so far covering core domains such as Home Automation, Security Surveillance, Medical Devices, Aerospace, Industrial Automation and Consumer Electronics Our core expertise / Service offering accounting for our major revenue segment is around connected device life cycle management, cloud enablement / integration, and workflow automation (DevOps) Our experience in workflow automation includes coverage of entire Development and Operations ecosystem taking to account organization’s people, process and technology needs. For one of our retail IoT Customer (Kroger) we helped them in integrating on Premise smart devices such as Cameras, Bar Code Scanners, Electronic Cart and electronic display to Cloud and manage their life cycle. The end solution allowed Kroger to monitor (live) customer movement on shop floor and provide personalized shopping experience.

DevOps Challenges in Continuous Testing Complex and fragmented Dev Pipeline: With multiple application delivery end points, including web, mobile, and device(s) the team composition and work flow are more challenging to streamline than pure play IT APPs.  Environment as a Code: Product engineering space brings additional challenges of virtualizing multiple associated devices along with server infra. The concept of ‘Infrastructure as code’ needs to be extended towards offering complete ‘Environment as code’. Product Pipeline: More variation on product delivery to multiple customer (customization & enhancement) and market segment (Low, mid, & high). This presents varied production environment difficult to reproduce and maintain at Dev stage. Legacy Device & Solution Support: Product along with cloud infrastructure brings requirement for managing, updating, and maintaining existing devices on field along with new added, again increasing variations and complexity for Devops. Release cadence: Different component of solution including firmware, web app, mobile app, & pc app have different release cadence making unified release plan a challenge

DevOps - Automation Continuous Integration Continuous Testing Integration of multiple pipelines (Device, Web and mobile) and prepare main and customized builds Automate and Configure Environment setup Trigger automated code review/unit tests/regression tests Generate and publish Results Send Notification/Email and Monitor Continuous Testing Virtualization/Simulation of test environment to enabled continuous testing Automation execution with simulation and physical devices Performance Testing with network virtualization Integration Automation framework with CI tool Parallel Remote automation execution Continuous Delivery Auto Build Deployment on Devices and Servers Manage Rollback deployment on live environment Automated Alerts on failure scenarios and performance issues Generate Business Analytical reports for future reference Continuous Monitoring Monitor Production and Test environment Device Health Monitoring Functional Alerts Automated Troubleshooting Automated Email / SMS Alerts

Continuous Test – Automation Solution

Continuous Test – Sensor Virtualization Telegesis Zigbee USB for Sensor virtualization Sensors’ Firmware is deployed on Telegesis USB Python Scripts triggers command to Raspberry Pi to generate specific signals from virtualized Sensors Virtualized Sensors passes signals to CPE over Zigbee CPE sends information to Servers based on Signals received from Sensors Python Scripts verifies actual outcome from Servers (Application logs as well as Database XMPP Interceptor used for testing of System behaviour such as when there are delays Load / Stress testing from Touchscreen to Server

Continuous Test – Build Verification

Virtualization for Test Automation Raspberry Pi Web Power Switch Simulation of Cameras using Nvidia cards Simulation of Hardware Boards – QEMU Sensor virtualization with Telegesis USB Network Virtualization using XMPP Interceptor Serial Port communication with Device Local Data centre in Cloud (AWS/Azure)

Automated Test Scripts Virtualization – Video Analytics Object Tracking and Analysis through Video Surveillance Automated Test Scripts Video Files - Test Data Verification of Alerts by Database Query Web & Database NVIDIA Card Alerts IOT Board with Application / Algorithm

References & Appendix Computer Weekly, 2014, Time for DevOps to get out of the weeds. http://www.computerweekly.com/opinion/Time-for-DevOpsto-get-out-of-the-weeds www.tecadmin.net

Questions & Answers

Thank You!!!