VCAP5-DCA – Objective 6.2 – Troubleshoot CPU and Memory Performance

For this objective I used the following documents:

Objective 6.2 – Troubleshoot CPU and Memory Performance

Knowledge

**ITEMS IN BOLD ARE TOPICS PULLED FROM THE BLUEPRINT**

  • Identify resxtop / esxtop metrics related to memory and CPU
    • Here are a list of some of the key CPU and memory related metrics resxtop / esxtop. A list of all metrics can be found in the link above (“Everything ESXTOP)
    • CPU metrics
      • %RDY (higher than 5) – indicates the amount of time the world was ready to run, but the world is stuck in a run queue while it waits on the CPU scheduler to schedule time on a physical CPU
      • %SYS (higher than 15) – indicates that the VM may have a lot of I/O. Represents the percentage of time used by system services on behalf of the world
      • %SWPWT (higher than 3) – indicates the percentage of time the world spends waiting on vmkernel memory swapping
      • %CTSP (higher than 3) – indicates the percentage of time the world spends in a ready, co-deschedule state. Used for SMP virtual machines. The higher this metric the further ahead on vCPU is over another vCPU.
      • %MLMTD (higher than 0) – indicates a VM is being throttled by a CPU limit that has been set. Indicates that the world was ready to run, but wasn’t aloud to due to the limit
    • Memory metrics
      • SWCUR (higher than 0) – indicates the host is swapping memory <- THIS IS BAD!
      • MCTLSZ (higher than 0) – indicates that memory is being ballooned; if memory pressure continues, memory compression may occur
      • ZIP/s (higher than 0) – indicates that memory is being compressed; if memory pressure continues, memory swapping may occur

 

  • Identify vCenter Server Performance Chart metrics related to memory and CPU

 

Skills and Abilities

  • Troubleshoot ESXi host and Virtual Machine CPU performance issues using appropriate metrics
    • Using some of the tools mentioned above you can troubleshoot ESXi host and virtual machine CPU performance, such as esxtop / resxtop and, if you’re running Windows, you can use Windows Perfmon.
    • I won’t go into perfmon here, but here are some key metrics to look at when troubleshooting host and virtual machine CPU performance problems
      • %RDY (higher than 5) – indicates the amount of time the world was ready to run, but the world is stuck in a run queue while it waits on the CPU scheduler to schedule time on a physical CPU
      • %MLMTD (higher than 0) – indicates a world is being throttled by a CPU limit that has been set. Indicates that the world was ready to run, but wasn’t aloud to due to the limit
      • PCPU UTIL(%) (all over 90-95%) – percentage of unhalted CPU cycles per PCPU. Also the average across all PCPUs
      • %SWPWT (higher than 3) – indicates the percentage of time the world spends waiting on vmkernel memory swapping
      • %CTSP (higher than 3) – indicates the percentage of time the world spends in a ready, co-deschedule state. Used for SMP virtual machines. The higher this metric the further ahead on vCPU is over another vCPU.
    • Troubleshoot ESXi host and Virtual Machine memory performance issues using appropriate metrics
      • Again, here are metrics to focus on from esxtop; these are the same metrics listed above in the Knowledgesection.
        • SWCUR (higher than 0) – indicates the host is swapping memory <- THIS IS BAD!
        • MCTLSZ (higher than 0) – indicates that memory is being ballooned; if memory pressure continues, memory compression may occur
        • ZIP/s (higher than 0) – indicates that memory is being compressed; if memory pressure continues, memory swapping may occur

  • Use Hot-Add functionality to resolve identified Virtual Machine CPU and memory performance issues
    • The concept behind hot-add is to be able to add/remove memory and CPU resources on a virtual machine that is powered-on. Not all guest operating systems support this, so make sure you are using one that is supported (I can’t find a list for this, will post if I can find one)
    • One thing to keep in mind is that hot-plug/add functionality must first be enabled before it can be used, and to enable it the virtual machine must be powered off
    • Enable CPU/Memory Hot-Add
      • Log into vCenter or directly to the host using the vSphere client
      • From the Hosts and Clusters view or VMs and Templates view find the virtual machine you want to enable hot-add on
      • If the virtual machine is powered on, power it off
      • Right-click the VM and select Edit Settings
      • Click the Options tab at the top
      • Select Memory/CPU Hotplug
      • Set Memory Hot Add to Enable memory hot add for this virtual machine
      • Set CPU Hot Plug to Enable CPU hot plug for this virtual machine
      • Click OK

image

      • The operation is now complete. You can power-on the virtual machine and hot/remove memory or vCPUs to this VM while it is powered-on

Tools

Comments 6

  1. in Hot-add functionality, when the VM is powered ON, I am only allowed to add not remove. Is that what are you getting too?

    1. Post
      Author

      I’m not sure. I’d have to check it out in my lab. What version of ESXi/vCenter are you running? Which client are you using (C# or web client)?

      1. Hi Josh..I am using vsphere 5.0 in my Lab at home. I also noticed that on our 5.5 version at work when hot-add is enabled on a VM..we can’t remove CPU or Mem only add..Thank you for the reply..

    1. Post
      Author
  2. Hi Joshua,

    I’m using all your DCA guide for my preparation, this is great stuff seriously. There’s a typo at %CTSP it should be %CSTP

    Thank you 🙂

Leave a Reply to Josh Coen Cancel reply

Your email address will not be published. Required fields are marked *

*