THE REALITY OF USING CONTAINERS TO BUILD PRODUCTS Gavin Etheridge, Product Owner, 1&1 Internet SE

Topics Our requirements Key design decisions Lessons learned Summary Do you want to get all of the benefits of containers when building products and DevOps tools?  Gavin Etheridge tells you about some the challenges you may face and describes how to implement a product and the associated DevOps tooling using container technologies. He will explain the platform that has been implemented, some of the key challenges faced and how they were overcome. The speech will also cover whether those decisions should be revisited now (or in the near future) as container technologies continue to mature. Objectives: The audience will learn about some of the key challenges faced by 1&1 when they implemented products and DevOps tooling based on container technologies. They will learn how these were addressed and if we may have done things differently as container technologies continue to mature. 18.01.2019 1&1 Internet SE

Build a product that we can sell to 1&1’s hosting customers Usability Our requirements Build a product that we can sell to 1&1’s hosting customers Usability Easy to use Easy to understand pricing Highly performant Include most popular tools and stacks for web developers Maintainability and Security Always up to date Availability 100% uptime Scalability Scalable Multi-tenancy Easy to use – targeted at demanding customers who are not necessarily tech savvy Always up to date - without input from the customer or downtime Scalable - for a global hosting provider 18.01.2019 1&1 Internet SE

The required customer facing product Traditional web hosting workflows Isolated environments Lifecycle management Dedicated resources The customer product will support: traditional web hosting workflows (upload web site) isolated environments – bespoke for each customers code management of components by 1&1 dedicated compute and storage resources 18.01.2019 1&1 Internet SE

Key design decisions taken Containerised approach Platform: OpenShift Usability: Resource allocator Maintainability, Security and Availability: Project updater Template builder Template migrator Scalability: Management stack 18.01.2019 1&1 Internet SE

Lessons learned: Platform OpenShift Why? Multi tenant environment Per customer isolation  security Our experience Easy to use management interface Added features Our lessons learned The key features we needed were on the Kubernetes roadmap and have been developed rapidly. e.g.: Network policy RBAC What would we do differently? Reconsider if OpenShift continues to offer enough additional features to justify an extra architectural layer Why OpenShift was essential to meet the requirement of a multi tenant environment More secure that shared web hosting / db Our experience Easy to use management interface to manage the cluster running the containers Added features were essential when compared to Kubernetes v1.2/v1.3 Our lessons learned RBAC (Beta in v1.6 in March; Stable in v1.8 October) 18.01.2019 1&1 Internet SE

Lessons learned: Usability Resource allocator Why? Paradigm shift We divide the customers resources between their projects Customers can define allocations Our experience Prevents “noisy neighbour” (including within a customers own projects) Our lessons learned Changes result in container restarts Customers following legacy processes expect legacy resource allocations What would we do differently? Consider using quotas, namespaces, request values and limits to provide comparable features and control Different paradigm to shared or dedicated servers that we didn’t want to expose to customers To preserve old workflow, we divide up the resources to protect the environment. Customer can slice. Constraining resources means container restarts for changes to be made – these can cause service interruptions (in our environment) Customers following legacy processes expect legacy resource allocations – that are constrained by hardware and not See https://blog.openshift.com/managing-compute-resources-openshiftkubernetes/ 18.01.2019 1&1 Internet SE

Lessons learned: Maintainability, Security and Availability Project updater, Template builder, Template migrator Why? Maintain and curate an image that upgrades customer environment Never want our customer to be the first to experience a particular combination of images Our experience Keeps customer environments up to date and secure Meets customers expectations of a managed solution Our lessons learned Capacity requirement of rolling update strategy Updating containers with persistent storage results in service interruption What would we do differently? Consider ImageStreams as an alternative Never want our customer to be the first to experience a particular combination of images. We test all possible combinations 18.01.2019 1&1 Internet SE

Lessons learned: Scalability Management stack Why? Experience Agility Our experience Allows detailed control of the customer environment Facilitates agile DevOps processes Our lessons learned Excellent way to ensure that internal teams have detailed and frequent working knowledge of containerised platforms What would we do differently? Nothing! 18.01.2019 1&1 Internet SE

General lessons learned You will find bugs (in all layers) Communities are very good, but they are also evolving Internal process alignment/optimisation is essential In our case, merging legacy processes has been a challenge (both internally and externally) Bonus Storage (not mentioned before), we urge you to follow the latest best practises for persistent volume claims 18.01.2019 1&1 Internet SE

Summary: What does this mean for us as a company? Container tools have matured significantly in the last 18-24 months What decisions would we take today? What are our next steps? Container tools have matured significantly in the last 18-24 months What decisions would we take today? Most of the custom development would be removed or significantly reduced Next steps for 1&1? Refactor Managed Cloud Hosting where possible Continuing to develop products that address the needs of web hosters and professionals as they move towards containerised workloads. 18.01.2019 1&1 Internet SE

So, make decisions based on the roadmap for the tools you need Summary: Advise to you? Expect the tools available to container based platforms to continue to mature rapidly So, make decisions based on the roadmap for the tools you need Actively participate in the community Container tools have matured significantly in the last 18-24 months What decisions would we take today? Next steps for 1&1? Advise to you? Make decisions based on the roadmap for the tools you need and actively participate 18.01.2019 1&1 Internet SE