1. Vital Signs: Performance Monitoring Windows Server
Module 8: Hyper-V
Microsoft Confidential

2. Conditions and Terms of Use
Microsoft Confidential
This training package is proprietary and confidential, and is intended only for uses described in the training materials. Content and software is provided to you under a Non-Disclosure Agreement and cannot be distributed. Copying or disclosing all or any portion of the content and/or software included in such packages is strictly prohibited.
The contents of this package are for informational and training purposes only and are provided "as is" without warranty of any kind, whether express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, and non- infringement.
Training package content, including URLs and other Internet Web site references, is subject to change without notice. Because Microsoft must respond to changing market conditions, the content should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. Unless otherwise noted, the companies, organizations, products, domain names, addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, address, logo, person, place, or event is intended or should be inferred.
Copyright and Trademarks
© 2011 Microsoft Corporation. All rights reserved.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.
For more information, see Use of Microsoft Copyrighted Content at
Microsoft®, Internet Explorer®, and Windows® are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Other Microsoft products mentioned herein may be either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners.
© 2011 Microsoft Corporation Microsoft Confidential

3. Students: How to View this Presentation
Switch to Notes Page view
Click View on the ribbon and select Notes Page
Use page up or page down to navigate
Zoom in or out as needed
Most slides will have supporting text that you can view now or after the delivery
Add notes to your copy of the presentation if you want to.
You take the presentation files home with you.
Microsoft Confidential

4. Microsoft Confidential
Module 8: Hyper-V
Section 1: Introduction to virtualization
Section 2: Processor performance
Section 3: Memory performance
Section 4: Storage performance
Section 5: Network performance
Microsoft Confidential

5. Section 1: Introduction to virtualization
How is virtualization accomplished?
How do child and parent partitions relate?
Impact of virtualization on Performance Monitor
Microsoft Confidential

6. How is virtualization accomplished?
Definition: Running multiple computers (virtual machines, or VMs) inside a single physical machine
How? By using a hypervisor, which is basically a thin layer of software (kernel) running on the hardware
What does it do? Allows VMs to run isolated from each other, and facilitates access to hardware resources
All of the key player hypervisors use the same architecture type. (Hyper-V, Xen, Vmware)
This is a very simplified description of the Type 1 hypervisor architecture. There are many components that are not discussed in detail, because they are out of the scope of this workshop. For more details about Hyper-V, direct your students to our “Implementing Hyper-V R2” premier WorkshopPLUS, available worldwide in PFE.
VM1 (Guest OS)
VM2 (Guest OS)
VM3 (Guest OS)
Hypervisor
Hardware
Microsoft Confidential

7. How do child and parent partitions relate?
Each VM runs in a individual partition—even the parent!
The parent partition creates the child partitions. It is also where the device drivers are loaded, so it can see the physical hardware. And it manages and assigns the child access to hardware devices through Hyper-V.
Child partitions cannot directly access hardware. All they see is emulated virtual devices, which the parent presents.
Parent Partition
Drivers
Child Partition
Server
Devices
Processors
Memory
Virtualization Layers
Hyper-V
Actually, Hyper-V manages CPU and RAM but, again, this is not an in-depth session on Hyper-V architecture.
Microsoft Confidential

8. Impact of virtualization on Performance Monitor
On a physical server the hardware is dedicated
1:1 relationship
With Hyper-V, this is no longer true because multiple operating systems share the same hardware
1:Many
Hyper-V manages CPU scheduling and memory
This can affect the accuracy of several performance counters and can even result in gaps in the graph if there is enough load on the parent and child partitions
Also, because the child partitions access hardware via the parent, you must retrieve some counters from the parent for a “real” view
Example: You need CPU and Disk access on the root to collect PerfMon data. If the VMs are contending for CPU and Disk access, they will interfere in this process.
(continued)
Microsoft Confidential

9. Impact of virtualization on Performance Monitor (continued)
The Hyper-V parent partition is a VM itself, so the counter %Processor Time and Task Manager are incorrect
This happens because the parent own view of the CPU is virtualized, which causes clock skew
Counters that report single values, such as Available Mbytes, and counters that measure latency, such as Avg Disk Sec Read\Write, are usually not impacted
Unless there is heavy hardware utilization, like the child CPU is running at 100%
Counters based on percentages, like Processor, are generally innaccurate because Hyper-V allows all VMs (parent and child) to run equally without preference
MORE INFO AT:
“Neither the Root nor the Guests VM’s control the PM Timer, APIC, … which means their concept of time in virtualized. Something else to be aware of is both the Root and Guest virtual processors share the physical processors in a Round Robin like fashion.”
Microsoft Confidential

10. Microsoft Confidential
Section 1 review
What is the primary function of the hypervisor?
What are the responsibilities of the parent partition?
How does virtualization impact the reliability of Performance Object counters?
Microsoft Confidential

11. Section 1 review (answers)
To allow multiple VMs (operating systems) to run simultaneously and isolated from each other.
To create the child partitions and manage access to hardware resources through the hypervisor.
By virtualizing and adding an extra layer between the operating system and the physical hardware. There is no direct access to the hardware.
Microsoft Confidential

12. Section 2: Processor performance
Overview of processor performance
\HyperV Hypervisor Logical Processor\%Total RunTime counter
\Hyper-V Hypervisor Virtual Processor\%Guest RunTime counter
Processor counter thresholds
Processor ratio
Hyperthreading considerations
Microsoft Confidential

13. Overview of processor performance
Physical processor utilization:
\Hyper-V Hypervisor Logical Processor\%Total RunTime
The sum of guest and Hypervisor runtime, one logical processor (LP) per core.
Virtual processor utilization:
\Hyper-V Hypervisor Virtual Processor\% Guest Run Time
Shows guest utilization of a virtual processor (VP) assigned to VMs but not the parent partition. A single virtual processor can run on different logical processors, but not at the same time.
Microsoft Confidential

14. \Hyper-V Hypervisor Logical Processor\%Total RunTime counter
Notice that the host processor (bottom left) is different than the guest processor (upper right)
See that the 100% CPU utilization of the VM is actually 50% on LP0 and 50% on LP1. Could also be 90% LP0 and 10% LP1.
“This counter can go over 100% just slightly. The problem has to do with how performance counters are computed. If you take the current time then value1 and later the end time and value2 this means value2 has the potential to increase between when end time was read and value2 is read.”
Microsoft Confidential

15. \Hyper-V Hypervisor Virtual Processor\%Guest RunTime counter
Counter used for virtual processor utilization, and which VM the virtual processor is associated with:
Notice the time skew mentioned here: (
Microsoft Confidential

16. Processor counter thresholds
For physical processors:
\Hyper-V Hypervisor Logical Processor(*)\% Total Run Time
For virtual processors:
\Hyper-V Hypervisor Virtual Processor(*)\% Guest Run Time
Thresholds for both logical and virtual:
Healthy: less than 75%
Warning: greater than 75%
Critical: greater than 85%
Avoid % Processor Time
Avoid Task Manager
Microsoft Confidential

17. Processor ratio
Starting in Windows Server 2008 R2, Hyper-V supports up to 64 LPs and 512 VPs per host
Don´t exceed the ratio of 8:1 (512 VPs : 64 LPs) or you can enter Lock Mode (12:1 if all VMs are on Windows 7 SP1)
Some products, such as Exchange Server, support a ratio of only 2:1
Recommended ratio depends on VM role and VP load
Virtualization Feature
Windows Server 2008 Hyper-V
Windows Server 2008 Hyper-V SP2
Windows Server 2008 R2 Hyper-V
Logical-processor support 16 24 64
Total number of VMs powered on
128
192
384
Total virtual processors supported
512
In production we don’t recommend exceeding 5:1 ratio as it could oversubscribe the cores and cause system-wide negative performance impact.
Hyper-V does the thread scheduling between physical cores. So oversubscribing your cores with threads may cause some to wait because they are busy running other threads. See HT considerations slide.
Microsoft Confidential

18. Hyperthreading considerations
It is recommended that you enable hyperthreading (HT) on CPUs where it is available (such as Intel Nehalem and AMD Magny-Cours), because it provides a performance increase of 5–10%
Exceptions
While you are planning capacity, disable HT because it doubles the number of logical cores that are visible to Hyper-V
If a server has more than 32 physical cores, enabling HT creates more logical cores than Hyper-V supports (max is 64)
Do not enable HT if the number of physical cores is not being fully used in virtual processors
Example: if you have 32 LPs and only 10 VMs with 2 VPs each, that means you’re using 20 LPs.
Enabling HT in this scenario will make a CPU take 2 threads VS leaving HT off, where each thread gets sent to a different CPU. This can be more efficient.
Microsoft Confidential

19. Microsoft Confidential
Section 2 review
Which counter should you use on a server that is running Windows Server 2008 with Hyper-V enabled to monitor the utilization of physical CPUs?
What is the maximum supported ratio of virtual to logical CPUs?
When should you disable hyperthreading?
Microsoft Confidential

20. Section 2 review (answers)
The counter \Hyper-V Hypervisor Logical Processor(*)\% Total Run Time
The maximum supported ratio is 8:1, and it is 12:1 if all VMs are running Windows 7 with SP1
When doing capacity planning, and when the number of virtual processors is less than the number of physical cores
Microsoft Confidential

21. Section 3: Memory performance
Overview of Hyper-V memory management
Parent partition memory reserve
Size of memory for child partitions
Dynamic Memory in Windows Server 2008 R2 Hyper-V with SP1
Microsoft Confidential

22. Overview of Hyper-V memory management
Hyper-V virtualizes physical RAM.
Examples of virtual to physical memory mapping:
System Physical Address (SPA): Actual physical memory (RAM) on the machine where the hypervisor runs.
Guest Physical Address (GPA): Physical address space that a child partition (VM) runs on. Think of Working Set as an analogy. All GPA pages in use are mapped to the SPA.
Guest Virtual Address (GVA): The GVA space provides an address space that is much larger than the GPA space, and pages of memory that are not mapped to a GPA location are paged to disk. It is virtual memory, which means all pages are not mapped to the GPA. Not currently in use, like Virtual Bytes.
(continued)
Microsoft Confidential

23. Overview of Hyper-V memory management (continued)
Hyper-V Server 2008 R2 introduces support for Second Level Address Translation (SLAT) processors
These offload the 2 levels of memory translations to the CPU, reducing load and reducing overhead for shadow page tables in earlier versions of Hyper-V
Guest Virtual Address
Guest Physical Address
System Physical Address
Guest OS defines GVA-to-GPA mappings
Hypervisor defines GPA-to-SPA mappings
Guest Virtual Address
System Physical Address
CPU performs SLAT
mapping
AMD-V Rapid Virtualization Indexing (RVI)
Intel VT Extended Page Tables (NPT)
Microsoft Confidential

24. Parent partition memory reserve
It is critical to ensure that the parent partition has enough memory to perform its operations, because all child partitions depend on the parent.
The registry key MemoryReserve does not “lock” a specifed amount of memory, but it does ensure a minimum amount of memory for the parent partition when it starts VMs. Minimum recommended is 512 MB.
Registry key location:
HKLM\SOFTWARE\Microsoft\WindowsNT\CurrentVersion\Virtualization\
Ensure there is sufficient memory installed on the physical computer that hosts the Hyper-V virtual machines – Available physical memory is often the most significant performance factor for BizTalk Server running on a Hyper-V virtual machine. This is because each virtual machine must reside in non-paged-pool memory, or memory that cannot be paged to the disk. Because non-paged-pool memory cannot be paged to disk, the physical computer that hosts the virtual machines should have available physical memory equal to the sum of the memory allocated for each virtual machine plus the following:
300 MB for the Hypervisor
plus 32 MB for the first GB of RAM allocated to each virtual machine
plus another 8 MB for every additional GB of RAM allocated to each virtual machine
plus 512 MB for the host operating system running on the root partition
For example, if a Hyper-V virtual machine is allocated 2 GB of memory in the Hyper-V Manager, the actual physical memory used when running that Hyper-V virtual machine would be approximately 2388MB (300MB for the hypervisor +2GB allocated for the virtual machine + 32MB + 8MB = 2388MB). Because the hypervisor only needs to be loaded once, initialization of subsequent virtual machines does not incur the 300 MB overhead associated with loading the hypervisor. Therefore, if two Hyper-V virtual machines are each allocated 2 GB of memory in the Hyper-V Manager, the actual physical memory used when running these Hyper-V virtual machines would be approximately 4476MB (300MB for the hypervisor +4GB allocated for the virtual machines + 64 MB + 16MB = 4476MB). Note As a general rule of thumb, plan to allocate at least 512 MB memory for the root partition to provide services such as I/O virtualization, snapshot files support, and child partition management.
Microsoft Confidential

25. Size of memory for child partitions
Treat your VMs exactly the same as a physical server
The more physical RAM, the longer that pages can remain in memory without being paged out
Use Memory\Commited Bytes from a VM and ensure there is at least that value plus 10%, in MB, assigned to that VM
Ensure that Memory\Available Mbytes is within thresholds:
Healthy: greater than 10% of RAM
Warning: less than 10% of RAM
Critical: less than 100 MB of RAM
Microsoft Confidential

26. Dynamic Memory in Windows Server 2008 R2 Hyper-V with SP1
Overview of Dynamic Memory
How does Dynamic Memory work?
Startup RAM and Maximum RAM settings
Memory-buffer and memory-priority settings
How much RAM is a VM using?
Microsoft Confidential

27. Overview of Dynamic Memory
Moves away from the static memory model in Hyper-V 2008
Allows for more ways to consolidate VMs, especially in implementations using Virtual Desktop Infrastructure (VDI)
Allows Hyper-V to change RAM assigned to a VM on the fly, without powering down the VM, in order to respond to memory pressure or workload
Looks at all the memory assigned to VMs running on the host, takes the available memory, and hot-adds it to VMs, using the VMBUS and VSC(Child)/VSP(Parent) model
Available from a VM setting, if the VM runs an OS that has Enlightened I/O and the VM has the most recent integration components
Microsoft Confidential

28. How does Dynamic Memory work?
Hot-add
VM Memory
Ballooned Memory
Second VM Memory
Parent Memory
512 MB
1024 MB
Microsoft Confidential

29. Startup RAM and Maximum RAM settings
Startup RAM: The amount of RAM the VM boots with and can never drop below
Maximum RAM: The most RAM the VM can be assigned or allocated dynamically
Microsoft Confidential

30. Memory-buffer and memory-priority settings
Memory buffer: Percentage of memory Hyper-V tries to give the VM based on the current committed memory the VM is using. Default is 20%.
Example: A VM is using 1 GB of RAM. Dynamic Memory tries to assign 200 MB (20%) to the VM, for a total of 1.2 GB.
Memory priority: Priority of a VM, compared to other VMs competing for distribution of the available memory pool.
Microsoft Confidential

31. How much RAM is a VM is using?
Dynamic Memory invalidates the \Memory\Available Mbytes counter and Task Manager inside the VM, because it bounces memory around
To see how much RAM a VM is using:
Check the Hyper-V Dynamic Memory VM\Physical Memory counter on the host, or
Check Current Memory on the Hyper-V Manager console
Microsoft Confidential

32. Microsoft Confidential
Section 3 review
What is the GPA? How does it relate to the SPA?
What is the benefit of using SLAT?
How much memory, at a minimum, should you reserve for the parent partition?
Microsoft Confidential

33. Section 3 review (answers)
GPA is the physical address space where the VM runs, meaning RAM assigned to the VM
It reduces the Hyper-V overhead by offloading memory translations to the CPU
512 MB for the host operating system
Microsoft Confidential

34. Section 4: Storage performance
VM storage types
Fixed-size virtual hard drive (VHD)
Dynamically expading virtual hard drive
Differencing virtual hard drive
Pass-through disks
Counters and thresholds
Microsoft Confidential

35. Fixed-size virtual hard drive (VHD)
Recommended for production environments of VMs with high I/O needs
Basically a flat file on the disk with no block allocation table needed, so less overhead
Max size: 2040 GB
Very close to performance of a pass-through disk
Cannot compact it (but can expand)
Disk space wasted if not used
Longer to create
NOTE: 2 TB is not a limit on the VHD format, just in the Hyper-V UI.
In R2, the performance of dynamically expanding VHDs is almost par with fixed-sized VHDs, and it is supported in production.
Microsoft Confidential

36. Dynamically expanding virtual hard drive
Uses available storage space more efficiently, grows dynamically as needed
Faster to create and smaller size, so easier to copy or move
Can compact it
Read/Write performance during expand is poor
Not supported in some products (Exchange Server)
Misalignment issues possible in block (not partition) I/O because metadata is appended to allocations
Microsoft Confidential

37. Differencing virtual hard drive
Can be associated with a fixed-size or dynamically expanding VHD or with another differencing VHD.
Helps prevent changes to a parent VHD. Also automatically created when taking a snapshot.
Easily identified with .AVHD extension.
Cannot expand manually because the size limitation is inherited from the parent.
Mostly used for testing and pre-production environments.
Parent Disk
Microsoft Confidential

38. Microsoft Confidential
Pass-through disks
Physical disk appears offline to host, presented directly and exclusively to a single VM. (Not CSV-compatible, except for VM data disks.)
Best performance overall, because I/O passes directly to the storage stack, bypassing the NTFS on the host.
(continued)
Microsoft Confidential

39. Pass-through disks (continued)
Fewer CPU cycles required.
Not subject to Hyper-V or VHD format limits. Size is limited only by NTFS.
Cannot use snapshots.
Cannot back up by using Hyper-V VSS writer from the host, so backups need to be taken inside the VM.
For services with high I/O demands, like heavily used file- and-print or SQL Server.
Microsoft Confidential

40. Counters and thresholds
Disk counters inside the VM measure response times same as a physical computer, and the same thresholds and counters apply. If your VM has latency above the following numbers, it’s time to investigate.
\LogicalDisk(*)\Avg. Disk Sec/Read | Write
Healthy: less than 10 ms (0.010)
Warning: greater than 15 ms (0.015)
Critical: greater than 25 ms (0.025)
\LogicalDisk(*)\Disk Transfers/sec
The IOPS from a Windows perspective (see the Disk module).
Microsoft Confidential

41. Microsoft Confidential
Section 4 review
What are the different types of storage available to use in a virtual machine?
What is the recommended scenario for using differencing disks?
What is the benefit of using pass-through disks?
Microsoft Confidential

42. Section 4 review (answers)
Fixed sized, dynamically expanding, and differencing VHDs, and pass-through disks
Testing and pre-production environments
Faster performance, due to bypassing the NTFS stack on the host, consuming fewer CPU cycles, and not being limited to size limits of VHDs
Microsoft Confidential

43. Section 5: Network performance
Overview of Hyper-V networking architecture
Types of Hyper-V networks
Synthetic vs. Legacy adapters
Virtual Machine Queue (VMQ)
Counters and thresholds
Microsoft Confidential

44. Overview of Hyper-V networking architecture
The physical network card in the parent partition is virtualized into a software virtual switch with “virtual ports,” where the virtual network cards created for each VM are connected
The virtual switches route packets where they need to go (Internet, LAN, and so on)
Microsoft Confidential

45. Types of Hyper-V networks
Use Virtual Network Manager to configure all networks that guests use in Hyper-V.
Types of virtual networks:
External : Binds to a physical NIC so virtual machines can access one of the physical networks that the parent partition is connected to. One external network per physical NIC. This is the only type that actually sends packets out on the wire.
Internal : Provides connectivity between VMs and parent partition.
Private: Provides connectivity between VMs only.
Microsoft Confidential

46. Synthetic vs. Legacy adapters
Not optimal
Legacy Network Adapters
Emulated NICs in software. Use the device driver inside the guest. Required for legacy compatibility, such as PXE boot, but add CPU overhead and lower network performance because max speed is 100 MB. Max of 4 per VM. If you need to host VMs with legacy adapters, recommend that you keep them on separate hosts.
Synthetic Network Adapters
Default. Require Integration Services to be installed on the VM. Speeds of up to 10 GB (uses VMBUS), depending on the speed of the physical NIC. Max of 8 per VM.
Optimal
Note: Some legacy apps may need to use 100MB NICs and as such the not optimal statement is entirely subjective.
Microsoft Confidential

47. Virtual Machine Queue (VMQ)
A single host might have 2–4 physical NICs and 20 VMs with 1 virtual NIC each. If all 20 virtual NICs have high need of network bandwith, they can congest the physical NICs.
Windows Server 2008 R2 introduces VMQ support, which uses the physical NIC to route incoming network packets to VMs directly, bypassing the virtual switch. This reduces CPU utilization on the parent partition.
VMQ and Chimney cant be enabled both at the same time because it can corrupt packets. VMQ has better performance than Chimney as a side note.
Must also be enabled for the virtual switch.
VMQ will not work with Windows Server 2003 or earlier Windows Operating System VMs.
(continued)
Microsoft Confidential

48. Virtual Machine Queue (VMQ) (continued)
Must be supported by the physical NIC on the Hyper-V host
Creates a queue for each virtual NIC
Available only on guests running Windows Vista, Windows Server 2008, or later
Microsoft Confidential

49. Counters and thresholds
To monitor VMQ from the Hyper-V host, use \Network Interface(*)\Output Queue Length
Healthy: less than 1 on average
Warning: greater than 1 on average
Critical: greater than 2 on average
To find out which guest is using most of the bandwidth of your physical NIC, use \Hyper-V Virtual Network Adapter(*)\Bytes/sec
Microsoft Confidential

50. Microsoft Confidential
Section 5 review
What is the difference between legacy and synthetic network adapters?
What scenarios require legacy network adapters?
What are the benefits of VMQ?
Microsoft Confidential

51. Section 5 review (answers)
A legacy NIC is an emulated physical device that adds overhead and offers less performance. A synthetic NIC is optimized to run in a virtualized operating system.
PXE Boot scenarios, and scenarios where legacy applications require a speed of 100 Mbps or less.
VMQ helps to reduce the overhead of a physical NIC to deliver packets to a VM.
Microsoft Confidential

52. Questions?

53. References Virtual PC Guy’s Blog
Windows Server 2008 Hyper-V Resource Kit
Clint Huffman's Windows Troubleshooting in the Field Blog
Mark Ghazai’s Blog

54. Appendix: Page file considerations with Dynamic Memory
Dynamic Memory requires a page file on the guest OS to be set for optimal performance and changes the “normal rules.” Use the following for guidance:
Minimum page file: Large enough to cover the memory demands of your largest process or application.
 Maximum page file: (peak commit) – (max physical memory) + (some buffer)
It is crucial that you use the host memory reserve key with Dynamic Memory, because it can cause issues with VMs that use RAM agressively.
Hyper-V allocates RAM to VMs on startup and doesn’t page those pages. So any paging on the host should be minimal.
Microsoft Confidential