1. Technologies: Server Virtualization, Infrastructure and Application Monitoring November 2, 2010 David Pritchett and John McQuaid

2. 2 What is Virtualization? Method of partitioning one or more physical servers into multiple “virtual machines” A virtual machine: –Runs its own full-fledged operating system –Can be independently rebooted –Is Isolated from other virtual machines running on the same physical hardware –Is Insulated from physical hardware changes

3. 3 Physical Environment Capacity 22 ESXi hosts – 6 Dell R900 & 16 Dell R710 –256 Cores producing 715 GHz –2160 GB RAM –86 TB SAN –Fully vMotion, Dynamic Resource Scheduling, and High Availability –Under 50 network cables

4. 4 Virtual Environment 154 Virtual Machines hosted in the NCC and growing Average approximately 35-50 VMs per physical host P2V: Converted 40 physical servers to virtual servers –Reduced hardware from 40 physical servers to six

5. 5 Virtual Technologies Software manages the deployment and configuration of virtual machines as well as allocation of pooled resources Enhance the reliability and manageability of a server deployment: –VMotion: Capability to move a running virtual machine from one ESX host to another –Storage VMotion: Capability to move a running virtual machine from one storage device to another –Dynamic Resource Scheduler: Balance virtual machine load across an ESX cluster using VMotion –High Availability: In case of hardware failure in a cluster, the virtual servers will automatically restart on another host in the cluster

6. 6 Benefits - More Agile Expedited provisioning for virtual servers…days…not weeks or months Dedicated Windows and Linux servers will move to VMware virtual machines with core and memory charges Many updates to the virtual machine resource configuration (CPU, Memory, etc) can be done with a reboot of the server

7. 7 Process Efficiencies

8. 8 Lower UHHW Cost-Monthly Year One Deployment – Virtual Server Year One - Dell Server UHHW – Server –Assume 2 Core x 4 Gig UCDED – Storage –Same for Both UHHW – Core/Mem –2 Core x 4 Gig UCDED – Storage –Same for Both VM = $56.68 less per month - $680.16 less per year

9. 9 Lower Total UH Cost-Yearly XS/UC – Same UH Charges Recurring –UHHW - $1944 –UHDED – $27062 UH Charges One Time –UHODC Installation - $2000 –UHODC Server Cost - $10000 XS/UC – Same UH Charges Recurring –UHHW - $1264 –UHVM – $13,200 UH Charges One Time –UHODC Installation - $2000 Bottom Line - $24,542 LESS

10. 10 More Power Needed? Update from 2 CPU to 4 CPU –VM – Reboot the VM with new configuration and pay the difference in UHHW costs. –Dell – New hardware purchase including all new installation costs and lead times. Update from 4 gig to 16 gig –VM – Reboot the VM with new configuration and pay the difference in UHHW costs –Dell – Memory stick purchase (if supported), installation downtime, server reboot

11. 11 OTOP's New IT Infrastructure Management Tool: “InfraView” EM7 from ScienceLogic

12. 12 Federal Triangle Cash Cab What is EM7? A.A new music group B.The 2011 Mercedes model C.An ingredient in non-dairy creamer D.IT operations management monitoring applications

13. 13 Monitoring Monitors all types of IT infrastructure devices - - servers, telecom, security, storage Displays real-time information on configurable dashboard Alerts when defined thresholds are exceeded At application level, can check for URL presence/text string, and can submit custom SQL queries, more

14. 14 Displays real-time information on configurable dashboard Monitors all types of IT infrastructure devices -- servers, telecom, security, storage

15. 15 At application level, can check for URL presence/text string, and can submit custom SQL queries, more

16. 16 Data Collection Collects info on system configuration, installed software, active processes, active TCP ports Collects operational status data via SNMP every x minutes -- up/down, memory, CPU, disk, bandwidth

17. 17 Alerts when defined thresholds are exceeded Collects operational status data via SNMP every x minutes -- up/down, memory, CPU, disk, bandwidth

18. 18 Data Reporting & Access Detailed data and graphs on per-device basis Reports in HTML, XLS, other formats Sophisticated "Dynamic Application" programming capability Granular access control Currently internal to OTOP only Customer access? TBD - stay tuned

19. 19 Detailed data and graphs on per-device basis

20. 20 Detailed data and graphs on per-device basis

21. 21 Application Performance Monitoring and Analysis Using OPNET

22. 22 Questions We Attempt to Answer Is the network causing delays? –Congestion –Latency Is the application server slow? Is database server response a problem?

23. 23 ACE Live Real-time monitoring and troubleshooting of application performance –End-user response times –Web transaction analysis –Automatically diagnoses source of application delay –Historical trending –Dynamic thresholds

24. 24 ACE Analyst Deep inspection of individual user transactions Multi-tier breakdown of response-time delay High-definition analysis of network delay What-if prediction WAN-optimization support

25. 25 ICIS Case Study

26. 26 ICIS Production

27. 27 Summary of Monitoring ICIS Production Network considerations (bandwidth, congestion) do not appear to be a major factor in perceived “slow” performance Business Objects XI Server performance (throughput and response) is a contributing factor to perceived “slow” application performance from end-user perspective Database Server performance (throughput and response) is not a factor in perceived “slow” application performance

28. 28 Server Response Time (msec) 23 Jun 0900 – 24 Jun 0900

29. 29 ICIS Server Response Times(msec) 23 Jun 0900 – 24 Jun 0900

30. 30 Response Time Summary – Single User

31. 31 Contacts: David Pritchett pritchett.david@epa.gov 919-541-2798 John McQuaid mcquaid.john@epa.gov 919-541-7679