1. THE REALITY OF USING CONTAINERS TO BUILD PRODUCTS
Gavin Etheridge, Product Owner, 1&1 Internet SE

2. Topics Our requirements Key design decisions Lessons learned Summary
Do you want to get all of the benefits of containers when building products and DevOps tools?  Gavin Etheridge tells you about some the challenges you may face and describes how to implement a product and the associated DevOps tooling using container technologies. He will explain the platform that has been implemented, some of the key challenges faced and how they were overcome. The speech will also cover whether those decisions should be revisited now (or in the near future) as container technologies continue to mature.
Objectives: The audience will learn about some of the key challenges faced by 1&1 when they implemented products and DevOps tooling based on container technologies. They will learn how these were addressed and if we may have done things differently as container technologies continue to mature.
1&1 Internet SE

3. Build a product that we can sell to 1&1’s hosting customers Usability
Our requirements
Build a product that we can sell to 1&1’s hosting customers
Usability
Easy to use
Easy to understand pricing
Highly performant
Include most popular tools and stacks for web developers
Maintainability and Security
Always up to date
Availability
100% uptime
Scalability
Scalable
Multi-tenancy
Easy to use – targeted at demanding customers who are not necessarily tech savvy
Always up to date - without input from the customer or downtime
Scalable - for a global hosting provider
1&1 Internet SE

4. The required customer facing product
Traditional web hosting workflows Isolated environments Lifecycle management Dedicated resources
The customer product will support:
traditional web hosting workflows (upload web site)
isolated environments – bespoke for each customers code
management of components by 1&1
dedicated compute and storage resources
1&1 Internet SE

5. Key design decisions taken
Containerised approach
Platform:
OpenShift
Usability:
Resource allocator
Maintainability, Security and Availability:
Project updater
Template builder
Template migrator
Scalability:
Management stack
1&1 Internet SE

6. Lessons learned: Platform
OpenShift
Why?
Multi tenant environment
Per customer isolation  security
Our experience
Easy to use management interface
Added features
Our lessons learned
The key features we needed were on the Kubernetes roadmap and have been developed rapidly. e.g.:
Network policy
RBAC
What would we do differently?
Reconsider if OpenShift continues to offer enough additional features to justify an extra architectural layer
Why
OpenShift was essential to meet the requirement of a multi tenant environment
More secure that shared web hosting / db
Our experience
Easy to use management interface to manage the cluster running the containers
Added features were essential when compared to Kubernetes v1.2/v1.3
Our lessons learned
RBAC (Beta in v1.6 in March; Stable in v1.8 October)
1&1 Internet SE

7. Lessons learned: Usability
Resource allocator
Why?
Paradigm shift
We divide the customers resources between their projects
Customers can define allocations
Our experience
Prevents “noisy neighbour” (including within a customers own projects)
Our lessons learned
Changes result in container restarts
Customers following legacy processes expect legacy resource allocations
What would we do differently?
Consider using quotas, namespaces, request values and limits to provide comparable features and control
Different paradigm to shared or dedicated servers that we didn’t want to expose to customers
To preserve old workflow, we divide up the resources to protect the environment. Customer can slice.
Constraining resources means container restarts for changes to be made – these can cause service interruptions (in our environment)
Customers following legacy processes expect legacy resource allocations – that are constrained by hardware and not
See
1&1 Internet SE

8. Lessons learned: Maintainability, Security and Availability
Project updater, Template builder, Template migrator
Why?
Maintain and curate an image that upgrades customer environment
Never want our customer to be the first to experience a particular combination of images
Our experience
Keeps customer environments up to date and secure
Meets customers expectations of a managed solution
Our lessons learned
Capacity requirement of rolling update strategy
Updating containers with persistent storage results in service interruption
What would we do differently?
Consider ImageStreams as an alternative
Never want our customer to be the first to experience a particular combination of images. We test all possible combinations
1&1 Internet SE

9. Lessons learned: Scalability
Management stack
Why?
Experience
Agility
Our experience
Allows detailed control of the customer environment
Facilitates agile DevOps processes
Our lessons learned
Excellent way to ensure that internal teams have detailed and frequent working knowledge of containerised platforms
What would we do differently?
Nothing!
1&1 Internet SE

10. General lessons learned
You will find bugs (in all layers)
Communities are very good, but they are also evolving
Internal process alignment/optimisation is essential
In our case, merging legacy processes has been a challenge (both internally and externally)
Bonus Storage (not mentioned before), we urge you to follow the latest best practises for persistent volume claims
1&1 Internet SE

11. Summary: What does this mean for us as a company?
Container tools have matured significantly in the last months
What decisions would we take today?
What are our next steps?
Container tools have matured significantly in the last months
What decisions would we take today?
Most of the custom development would be removed or significantly reduced
Next steps for 1&1?
Refactor Managed Cloud Hosting where possible
Continuing to develop products that address the needs of web hosters and professionals as they move towards containerised workloads.
1&1 Internet SE

12. So, make decisions based on the roadmap for the tools you need
Summary: Advise to you?
Expect the tools available to container based platforms to continue to mature rapidly
So, make decisions based on the roadmap for the tools you need
Actively participate in the community
Container tools have matured significantly in the last months
What decisions would we take today?
Next steps for 1&1?
Advise to you?
Make decisions based on the roadmap for the tools you need and actively participate
1&1 Internet SE