1. Developing 14,000+ messages per second IoT systems
No Rocket Scientist required!
Developing 14,000+ messages per second IoT systems

2. Developers Audience Building High Through-put Systems on CF
Wanting To Monitor Apps on CF
Interested In Significant Load to test Apps
Needing a Recovery Area
Arity Sensor and Insights eXchange project – A_SIX Developers building high throughput systems using Cloud Foundry
Developers wanting to monitor their applications running in Cloud Foundry
Developers interested in generating significant load to test applications
Developers needing a recovery area after this mornings Pancake Stack breakfast

3. Who Am I? Dale Robinson A_Six Anchor Arity Employed by Arity
Arity provides API services in the telematics sensor market for companies such as Allstate, Answer Financial, Esurance.
Anchor for an XP team, working on the Arity Sensor and Insights eXchange project – A_SIX

4. A_SIX application performance
But we wanted…
A_TEN
Used to create the attribution for the Chris Hemsworth image (as per ). Image is found at:

5. JMeter from Java
Left – Spring Boot (v1.5.3) JMeter project files. Highlighted file is the JMeter configuration file
Right top – contents of jmeter.yml
Right centre – JmeterPropertyLoader.java file
Right bottom – build.gradle dependency section

6. JMeter from Java
@RestController class – JmxProcessor.java

7. Monitoring Apps in Cloud Foundry
VisualVM
Enable JMX in (Pivotal) Cloud Foundry
cf set-env <appName> JBP_CONFIG_JMX '{enabled: true}’
Use cf ssh to connect to the container
cf ssh -N -T -L 5000:localhost:5000 <appName>
Create a JMX connection within VisualVM

8. Monitoring - VisualVM

9. Monitoring - VisualVM

10. Monitoring - VisualVM

11. Monitoring VisualVM – What’s missing? Workarounds: Heap dumps MBeans
Spring Actuators

12. Monitoring - MBeans

13. Monitoring - MBeans

14. Monitoring - Spring Spring Actuators
Confirm they are enabled/sensitive
/dump to produce JSON of thread dump
/heapdump to produce GZIP hprof file
/metrics to produce JSON of container data
/env to produce JSON of JVM details
See Spring documentation for more…

15. Monitoring conclusion
Heap dump considerations
Spring Actuator vs. MBeans
Heap dumps are saved to the container
Spring approach will automatically download
MBean approach requires manual SCP
Specify additional disc space accordingly
This is true for both the MBean and Spring options

16. Micro-services are good…
A_SIX
A_SIX
API
API
Persistence
Persistence
Insights
Insights
Subscriptions
Subscriptions
Republisher
Republisher

17. Asynchronous processing and IO
Thread pool is unconstrained
Decoupling reading and writing
Error handling

18. Spring Boot configuration
Tomcat embedded server defaults
200 Threads
10,000 Connections
OS Connection Queue
100 Keep Alive Requests

19. Spring Boot configuration

20. Spring AMQP configuration

21. Spring Boot configuration
So now you can, but should you?
Probably not….
The defaults are very good
Only if you are certain of the issue
Prefer the sledgehammer of more instances

22. Security Driven Development
You load tested using JMeter
You profiled using VisualVM
You employed micro-services
You tweaked Spring Boot / Tomcat
Your application is flying...
…and you just killed Cloud Foundry

23. Bits and bobs… cf curl /v2/apps/$(cf app <app> --guid)/stats
Swapping Tomcat for Jetty or Undertow
AsyncRabbitTemplate and AsyncRestTemplate
PCFDev

24. Links

25. Thanks!