YeahMobi CD Practice based on Container -- openstack meetup 叶彦斌 steve.ye@ndpmedia.com November 22, 2018

Agenda Why Automation SCM Points Branch Strategy Our Practice

We are what we repeatedly do. Philosophy We are what we repeatedly do. Excellence then is not an act, but a habit. Aristotle

What’s it now Building repeat Packing repeat Publishing repeat Environment repeat Releasing repeat Deployment repeat Process repeat …

Why not Automation Fast requires automation. Automation requires consistency. Processes Standardized Versioned everything Automated every step

What’s SCM Software Configuration Management (SCM) is a a software-engineering discipline comprising a set of tools and processes that enhances the reliability and quality of software delivery

How？ Dev QA Stage Prod Dedicate & Clean build environment Same build process Same artifacts Same deployment scripts Different Configurations Build Deployment Tools Source Code Risk Management GOAL: Backend servers can be deployed anywhere/anytime Dev QA Stage Prod

Ad Hoc to Process - CMMI

Process = Speed？ Goals The Form vs. Freedom Trade-off Division of labor Reduction in Organizational Noise Speed & Accuracy trumps Freedom Fewer meetings to solve issues in a custom manner The Form vs. Freedom Trade-off

What is a good process？ The best processes are invisible “C.I. Builds” OK processes manual, but leads to a goal “Reviewing the code before check in” Worst processes, scramble drill “Deploy a P0 release” Snowball Emails Read flag meetings Running around for QA and Approvals

Principles of Source Code Management --- Code is locked down and can never be lost (safety). --- Code is base-lined, marking a specific milestone or other point in time. --- Code changed on a branch (variant) can be merged back onto the main trunk (or another variant). --- Source code management provides traceability and tracking of all changes. --- Source code management best practices help improve productivity and quality.

Released periodically

Three Types of Branches Trunk Holder of all things good and wonderful Represents the product going forward Release Branch For the hardening phase Maintains exact replica of production/OTA For Hot Fixes Feature Branch Work in progress for future release Major Release/Feature/Refactor/Project

Good Branching Strategy Engineers always have an appropriate branch to work in Never wait to checkin There is always a branch that matches production Hot Fix builds in under one hour Future work does not pollute release rhythm

Build Engineering ---Builds must be established that are a) Repeatable b) Based on an identifiable baseline c) All dependencies are well understood and controlled.

Build Strategy C.I. Builds – All Branches Every Checkin Unit Tests Find Bugs, etc. Finds issues fast Nightly Builds – All Branches Every ‘Night’ Metrics Deployment Auto BAT Tests Release Builds – Release Branch On Demand Uses workflow Run every test known to man

Environment Configuration ---When build engineering is done correctly, the code is built only once. ---Always point to the correct runtime resources, such as the QA or production database. ---Providing enough test environments so that development can be done efficiently and with the proper utilization of available resources. ---Environment configurations should be documented and understood by all parties. ---Identify and then control.

Principles of Change Control ---Changes should be planned and not just last-minute events. ---Changes should be understandable, including their downstream impacts. ---Authority and approvals for changes should be established and obtained as appropriate. ---Procedures for emergency changes should be established to cover emergency incidents. ---Change control should assess and confirm that all configuration management processes are being followed.

Principles of Release Management ---Releases should be readily identifiable with an immutable version ID. ---Releases should be packaged with all the dependencies included. ---Release packaging should be automated and designed to avoid human error. ---Release management should be fast and reliable to facilitate iterative development.

Goals of Deployment possible problem occurring. ---Promote a release into production without any possible problem occurring. ---Roll back the release if it has a problem so that the business can continue. ---Know exactly what is in production at all times. ---Know immediately whether any unauthorized changes have been made.

Delivery & Deployment Continuous Delivery doesn't mean every change is deployed to production ASAP. It means every change is proven to be deployable at any time(Carl Caum) CD is about how the pizza is made. When it's ready for customers, it's deployed.

CD System Main Points

CD System Framework

Single Stack All backend servers can be deployed anywhere/anytime Dynamically extract subset data for tests Create complete environment for A Clean Project Environments Automated Integration Testing (necessity for automation) A Clean Deployment Environment

Efficiency No more sharing STAGE with QA No more sharing TEST with QA Unlimited number of complete environments Engineers have stable environments Eventually, the company is on a laptop

Q&A？

Thanks!