1. Expanding Mesos Ecosystems Enabling Mesos on Windows
Jeff Coffler, Sr. Software Engineer, Microsoft Corporation
Andrew Schwartzmeyer, Software Engineer II, Microsoft Corporation

2. Why Add Windows Support?
Customers with mixed Linux and Windows workloads
Their Linux workloads are already managed by a Mesos cluster, so why not manage Windows too?
Mesos can provide a unified cluster orchestration layer for these mixed environments.
This simplifies system implementation and reduces maintenance costs.
Microsoft is working to enhance Mesos to allow Windows agents to run in existing Mesos clusters. An existing Mesos cluster, running Linux workloads, can run workloads on Windows as well, selectable by O/S type, CPU speeds, and other criteria.
Mesos is not fundamentally changed. Instead, its enhanced to support Window as well. Microsoft is working closely with Mesosphere (and the open source community) to make this happen. (Most of the code is shared.)
Emphasize: one cluster has both operating systems as agents with no changes to the master, etc

3. A Little Bit of History
Porting Mesos to Windows natively was started in August 2015.
Mesos introduced experimental support for Windows.
Our Open Source Technology Center team at Microsoft started on Mesos late 2016.
Initial work as a “straight port” (i.e. Autotools “converted” to CMake without any design changes). In late 2016, multiple engineers in the Open Source Technology Center started working to enhance Mesos. We’re involved not only with Windows, but with architectural changes that are happening within Mesos today.

4. Where We Are Now
The Mesos Containerizer groups tasks into separate job objects to support resource limitations.
The Docker Containerizer can launch arbitrary Windows containers.
The CMake build reliably supports Windows and Linux.
NTFS long paths are natively supported by Mesos.
DC/OS support for Windows deployments on Azure is working.
Mesos Containerizer: Setting CPU and memory limits is in progress.
Docker Containerizer: such as IIS servers.
NTFS long paths: no more registry changes.

5. Launch IIS in Windows Docker Container
This slide demonstrates a Windows host with Docker available. On the left you can see a relatively standard Marathon JSON configuration to launch a Docker container from the “microsoft/iis” image. On the bottom right you can see the IIS server up, and the top right shows the Docker container currently running, as a Mesos task.

6. Hybrid DC/OS Cluster Deployment (1/3)
Here, we have a hybrid DC/OS cluster deployment, with both Linux and Windows agents. For each agent we’ve added an “OS” attribute (“linux” and “windows”), and we use the “LIKE” constraint to launch tasks on the correct agent

7. Hybrid DC/OS Cluster Deployment (2/3)
As you can see from the services screen, the Windows agent tracks the number of CPUs available as well as memory and other factors.

8. Hybrid DC/OS Cluster Deployment (3/3)
Finally, when looking at the nodes, you can see Linux nodes as well as Windows nodes available for tasks.

9. Build System Improvements
CMake system has been rewritten
Dependency graph is now correct
Uses imported third party targets and interface libraries
Debug and Release configurations are supported
Makefile: cmake -DCMAKE_BUILD_SYSTEM=Release
MSBuild: cmake --build . --config Release
Building the Java Mesos bindings is supported
cmake –DENABLE_JAVA=1
Pre-compiled headers on Windows help reduce compilation times.
Link times are on par with Linux.
Link flags are audited and now as correct as Autotools
Java support means Marathon support

10. Why use CMake?
The goal is a consistent and easy-to-use system for every developer.
Autotools cannot build Mesos on Windows.
CMake is a meta build system: it generates build files for platform-native tools.
Linux: generates Makefiles
Windows: generates MSBuild solutions.
That’s not all! It can even generate Ninja builds.
This gives developers freedom of choice, while also providing consistency: the command cmake --build . --target mesos-tests will build mesos-tests for any generated build system.
CMake is built for C and C++
hpp files need not be manually specified
compile and link commands are generated for the targeted compiler

11. Isn’t CMake Complicated?
Refactored and simplified the CMake build
Third party dependencies are now imported targets
Header only libraries are now interface targets
add_library(stout INTERFACE)
Linking to stout doesn’t mean including its headers manually, but simply specifying target_link_libraries(process stout).
1000+ lines of unnecessary code deleted
MasterConfigure.cmake, AgentConfigure.cmake, etc.
Instead of specifying their include and link directories repeatedly, the information is embedded once into the CMake dependency graph.

12. CMake: Before and After
The left is the CMakeLists.txt and AgentConfigure.cmake files necessary to build the agent before refactoring.
The above is the new CMakeLists.txt file. All the necessary link and include information is deduced by CMake through the command to link to libmesos, which exists as a target on the dependency graph, with its own dependencies.

13. Autotools versus CMake
The left is the CMakeLists.txt and AgentConfigure.cmake files necessary to build the agent before refactoring.
The above is the new CMakeLists.txt file. All the necessary link and include information is deduced by CMake through the command to link to libmesos, which exists as a target on the dependency graph, with its own dependencies.

14. ReviewBot for Windows
The Mesos ReviewBot now tests patches on Windows.
Nightly builds of the Mesos binaries for Windows are available.

15. Continuous Integration for Windows
Through our partnership with Mesosphere, we have Jenkins testing each new set of patches to master on Windows, posted to the #windows Slack channel.
This helps identify new problems immediately.
The Linux and Windows code bases are nearly identical; almost all the porting work is hidden in stout. Thus, frequently bugs found on Windows exist on Linux as well.
Not all issues are Windows or Linux only, such as the Docker version mismatch between agents and masters. The regression was found by Windows Docker tests, but the same regression was later found on Linux when they updated Docker too.

16. A Case Study: Ctrip International
They have three clusters in their production environment, and two clusters in their test environment.
They have five masters, one framework, 400+ Windows agent nodes, and 500+ Linux agent nodes, per cluster.
There are 200+ (1C 2G 20G per container) Docker container tasks running per Windows agent node (48C 448G 2.4T per host).
There are 200+ (1C 2G 20G per container) Docker container tasks running per Linux agent node (48C 448G 2.4T per host).
People are really using Mesos on Windows. We’re closely engaged with customers using Mesos on Windows in production already, on rather large clusters. It works!
Ctrip is a very large travel company in China. Ctrip uses a very large hybrid cluster consisting of 200+ docker containers per node with 2.4T per node. They are actively using Windows today, in production, at a scale similar to their use of Linux.

17. Demonstration: Hybrid Cluster

18. What’s Next? Add authentication support for Mesos on Windows.
Add fetcher support to deploy software on Windows.
Increase unit test coverage by porting more tests.
Focus on scalability, reliability, and performance.
Keep engaging with customers to get Mesos on Windows production ready.
Deprecate Autotools in favor of CMake.
Port more services for DC/OS on Windows.
CRAM-MD5 already working, more coming via plugins, just like Linux.
Upcoming DC/OS services: Metrics, Spartan, Navstar

19. Resources github.com/Microsoft/mesos has links to: DC/OS Links:
Jenkins setup, Microsoft progress and plans
JIRA Windows Epic: MESOS-3094
Find us in #windows on Slack
DC/OS Links:
Jenkins: dcos-win.westus.cloudapp.azure.com/mesos-build/master/stable
Windows Repository: github.com/dcos/dcos-windows
github.com/Azure/acs-engine:
Azure Container Service Engine will quickly build and deploy Docker Enabled Clusters.

20. Where are we going from here?
Help us decide!
We are customer focused and customer driven
Join us on the Mesos Slack #windows channel and let us know your thoughts!
Engage with us to ensure that Mesos on Windows supports your needs.

21. Thanks for coming to our session!
Microsoft has a booth – Come visit us!
Any questions?