2. DevOps; a Tester’s best friend
Tom Janik – American Family Insurance
DevOps manager – Release management
Ken Robertson – American Family Insurance
SQA Test manager – Specialized testing
Objective of session:
Walk through THE American Family journey to Becoming a company that operates in a CULTURE BASED in DevOps

3. IN the beginning: Release Flow
Deploys done by development teams, could happen at any time
No stability requirements, no triage of deployment / environment issues
4 year development cycle using Amgile SDLC (selective Agile)
Development “Leg” (flow), SIT “Leg”, 2 month effort in Release “Leg”

4. In the BEGINNING: SIT TESTING
Manual testing Testing led by development teams, lacked integration coordination 2 year SIT cycle with multiple code elevations No formalized automation or performance testing No shared test data, test scenarios, or documented financial testing Limited communication between development and testing – throw it over the wall mentality

6. Forming: SIT TESTING Cycles
Introduction of DENT and QA HIST tiers
Point to point integration testing coordinated by development teams
Application integration coordination being introduced
6 month SIT duration – 3 months financial infrastructure testing and 3 months for financials
Documented financial testing in collaboration with controllers

7. ACTIVITY: DEVOPS What’s Important to you?
Using the index card in front of you write down 3 things that are most important to you about DevOps. (1:00 Minute)
Gather in groups of #-# at one of the posters on the wall. (1:00 Minute)
Come up with something common between everyone in the group and write that on top of the poster as your group name. (1:00 Minute)
Come up with one common thing from everyone’s index cards and draw (no words) that one common thing on the rest on the space on the poster. (1:00 Minute)

8. A CULTURE CHANGE TO DEVOPS
STORMING:
A CULTURE CHANGE TO DEVOPS
DevOps is the practice of operations and development engineers participating together in the entire service lifecycle: from design, through the development process, and then into production support
Specifically DevOps is characterized by:
Operations staff making use many of the same techniques as developers for their systems work
Developers incorporating more operational concerns in their planning and coding (designing for operationalization)
DevOps can be a team, a department, an effort, a movement, but most importantly it is a culture!

9. Key Tenets of DEVOPS Plan Architect Dev Test Release Operate
FEEDBACK
Theory of Constraints (“The Goal”)
The 3 Ways (“The Phoenix Project”):
Systems thinking
Amplified feedback loops
Continuous experimentation and learning
Lean principles of manufacturing (“TPS: Toyota Production System”)
People, then process, then tools (Men, Methods, Machines, Measurements)
Collaborative development (Scrum, Agile, Kanban)
Environment provisioning, quicker access to new code for testing
Continuous deployment, integration, testing, delivery, feedback, and improvement

10. The NEW DEVOPS CULTURE Automation Testing Services Team Data Team
Quality Assurance
Automation Testing
Specialty Testing
Data Team
Services Team
DevOps
Release Management
Release Automation
Environment Management
Application Reliability

13. NORMING: WHO DOES WHAT? Release AUTOMATION Environment management
Release management
Quality Assurance
Application Reliability

14. NORMing Release management, Better defined schedules
2 week release activity cycle, Bi-monthly maintenance releases
Overlapping release cycles, Same branching strategy as development legs

15. Norming: Quarterly Releases2

16. norming: FULL RELEASE SCHEDULE2

17. norming: TESTING
Program level test lead for coordination of functional and integration testing
Phased / cycle approach to functional integration and SIT testing
10 week SIT duration with data creation critical path constraint removed
Introduction of forced month ends for financial testing

18. TESTING BENFITS FROM DEVOPS
Continuous integration builds Gated deployments based on testing results Automated unit testing Automated smoke testing Automatic rollback for failed deploys Environment stability Quicker, higher quality code delivery Ability to feature toggle and/or deploy dormant code Automated deployment notification and release notes Better merging strategies ease regression testing requirements Exposure into development process

19. norming: Tools Maven for configuration GIT for code management
CA Harvest for build and deploy
JUnit / GUnit for continuous integration testing
Jenkins for automated deployments
CA Dev Test (formerly Lisa) for automated testing
Word for requirements (moving toward TFS)
Excel for testing plan (moving toward TFS)
Excel release plans
Puppet for IaaS
CA Service Desk for change management and configuration item repository
Artifactory for Enterprise Data Management Solution (EDMS)

20. Questions?
Thank You Tool Appendix follows

21. Appendix A: TOOL Research