1. Building highly scalable enterprise systems (on Azure)
Ronald Harmsen
In this practical session I will show you how to design and build a highly scalable and distributed system. Combining the core architectural principles of CQRS, Event Sourcing & Actor Systems to create the design and then building it with .NET core, AKKA.NET & Docker. That’s a lot of technology right? Well, they fit very nicely together and even make sense when combined.
This architecture has been used successfully in some very large scale enterprise systems I’ve been working on. Of course I’ll guide you through all steps from understanding the design to deployment into Azure. Why did I choose these patterns? What are my options for scaling? How did we refactor our old implementation into this solution? And more…
I’ll start with a couple of architectural drawings and the rest is all about the stuff we care about: running code.
@ronaldharmsen
#DevSum19

2. Ronald Harmsen Developer / Consultant / Trainer @ NForza
Development >= 1993
Everything .NET >= 2004
Azure >= 2009
Others: DevOps, Software Quality, Software & Enterprise Architecture
#DevSum19

3. The project where it all started
#DevSum19

5. Our “adventure”
Main dataset: approx. 750TB (billions of tiles:metadata/photographs)
Initial application tests : ~273 days to calculate
Planned run: every quarter 

7. Our “adventure” part 2
Issue with waiting for neighbors to be available
Reloading of data
Storage slow

8. Solution space Can we have ‘caching’ of state ?
Process tiles independently ?
Execute ‘multi-stage’ detections on tiles ?
i.e. run algorithms for water, road, forest, buildings etc. independently?
#DevSum19

9. The Actor Model #DevSum19 Lightweight Never share state
Mailbox = message queue. So message-based or message-passing architecture
One message at a time
Simple threading scenario (code we write is single threaded, abstracted away in actor model)
#DevSum19

10. Goals Resilient Scalable Fast Cloud based (Azure) #DevSum19
Resilient – we don’t want to completely start over when something crasheds
#DevSum19

11. #DevSum19 We had some issues: Programming model
How many people know how to do this?
Slow deployments -> rolling updates of stateful services & hence actors
#DevSum19

12. Azure Kubernetes Services (AKS)
#DevSum19

13. Goals Resilient Scalable Fast Cloud based (Azure) Actor model ?
Resilient – we don’t want to completely start over when something crashes
Fast deployments / updates
Wide spread. More knowledge available.
#DevSum19

14. Which Actor Model ? #DevSum19 Akka. Orleans too much MS again
Akka = open source, pluggable architecture
Scalable -> remoting / clusters / sharding
#DevSum19

15. Demo: akka.net actors #DevSum19
Resilient – we don’t want to completely start over when something crashes
Fast deployments / updates
Wide spread. More knowledge available.
#DevSum19

16. What about resilience? Akka has some useful power-ups: Akka Remoting
Akka Clusters
Persistence
Sharding
Resilient – we don’t want to completely start over when something crashes
Fast deployments / updates
Wide spread. More knowledge available.
#DevSum19

17. Akka Remoting
#DevSum19

18. Akka Clusters
#DevSum19

19. Akka Persistence - CQRS
#DevSum19

20. Akka Persistence - CQRS
#DevSum19

21. Akka Persistence
#DevSum19

22. Akka Sharding
#DevSum19

23. Putting it all together in code
Resilient – we don’t want to completely start over when something crashes
Fast deployments / updates
Wide spread. More knowledge available.
aka DEMO
#DevSum19

24. Summing up Using Docker / Kubernetes for scaling
Relatively simple programming model
Easy to deploy the same architecture into Azure
Resilient
Resilient – we don’t want to completely start over when something crashes
Fast deployments / updates
Wide spread. More knowledge available.
#DevSum19

25. And…. Last but not least – don’t forget to evaluate this session in the DevSum app!