Transcript VMware vSphere 5.0 and Cloud Infrastructure Suite (CIS

VMware vSphere 5.0
Performance Benchmark
Reality
October 2011
RECAP
• VMware released (via Principled Technologies) a vSphere 5.0 vs. Windows
Server 2008 R2 SP1 Hyper-V benchmark
• VMware states that when running 30 VM’s, vSphere 5 outperformed Hyper-V
by 18.9%
• With 24 VMs, with only a small amount of memory overcommit, Hyper-V and
vSphere performance was within 2% variance.
• In scaling up 25%, from 24 to 30 VM’s, VMware states they gain an overall
increase in server performance of 11.2%, while Hyper-V decreases 3.3%
indicating a negative impact of Hyper-V Dynamic Memory.
• You can access the summarized report here:
http://www.vmware.com/files/pdf/products/vsphere/VMware-vSphere-vsHyper-V.pdf and the full, detailed report here:
http://www.principledtechnologies.com/clients/reports/VMware/vsphere5densi
ty0811.pdf
• But not is all that is seems…
2
THE FACTS: IN A NON MEMORY OVERCOMMIT SCENARIO
HYPER-V RESULTS ARE SIMILAR TO VSPHERE 5.0
• By VMware's own results, when
provided adequate memory to
support the number of users
the performance variance was
only 2% (using 24VM’s).
• This was based on the Average
Orders per Minute (OPM).
3
THE FACTS: VMWARE SHOW IMPROVED
PERFORMANCE IN A VERY SPECIFIC SCENARIO
• VMware were able to show delta in
performance only when using many
VMs running the exact same
workload with the exact same data
and overcommitting the host, under
specific VM configurations and
settings.
• One example would be the idling of
VMs for 60 minutes prior to the DVD
Store Workload starting, specifically
aiding VMware’s Memory Management
techniques
• vSphere shows an 11.2% gain in Total
OPM, whilst Hyper-V decreases 3.3%
• Read on for the facts…
4
THE FACTS: THIS TEST IS NOT A REALISTIC TEST FOR
CUSTOMERS
• The test relied heavily on Transparent Page Sharing (TPS) –
a form of ‘Memory Deduplication’
• TPS’s default duplicate page detection window is 60 minutes.
• Is it any wonder the VMs were run for 60 minutes before testing started?
• TPS only kicks in when the host is under memory pressure – a 96GB host, with 30 * 4GB
VMs = memory pressure (120GB RAM required)
• Most modern physical servers have Large Page Tables enabled, rendering TPS less
effective, as it can’t deduplicate 2mb pages as easily as it can with regular 4kb pages.
• However, under memory pressure, the system will sacrifice some performance to break down the
2mb pages into 4kb pages, and then deduplicate, freeing up memory.*
• This would have happened in the first 60 minutes of the test, when the VMs were intentionally
left idle, with no statistics, or measurements being taken at that time.
• The ‘freed up’ memory would then have been able to be used to satisfy the demand from the DVD
Store 2.1 SQL workload generator.
http://www.vmware.com/files/pdf/mem_mgmt_perf_vsphere5.pdf (page 9)
5
THE FACTS: IN THE REAL WORLD, MEMORYOVERCOMMIT IS NOT USED WIDELY
Paul Maritz’s PresentationEMC Strategic Forum,
Boston, February 8th 2011
According to Gartner “only a
minority of enterprises are
using it to generate significant
VM density” and the typical
“density of vSphere tends to
average about 10 VMs per
server”
Server Virtualization Choices: VMware or
Microsoft?
Publication Date: 9 August 2011 ID Number: G00213160
http://ir.emc.com/phoenix.zhtml?c=106202&p=irol-EventDetails&EventId=3654941
6
THE FACTS: VMWARE’S OWN PRODUCTS DON’T
WORK WITH MEMORY OVERCOMMIT…
• Perfect Example – The VMware vSphere Storage Appliance 1.0
(turns local ESXi disks into a ‘SAN’)
• “A VSA cluster created with vSphere Storage Appliance 1.0 does not support memory
overcommitment. VMX swapping is enabled by default and swapping to VSA datastores can
make the cluster unstable, and as a result, virtual machines might begin powering off”
• “To prevent virtual machine downtime, do not overcommit memory in the VSA cluster. For
each virtual machine in the VSA cluster, reserve the same amount of memory that is allocated to
that virtual machine and disable the machine from swapping to the VSA datastores”
• “For a VSA cluster that has HA enabled, the cluster reserves additional host memory to support
the restart of virtual machines from a failed peer host. In a 2-member VSA cluster, HA reserves
50% of the reserved memory of each host for the restart of a failed-over virtual machine.
Similarly, in a 3-member VSA cluster, HA reserves 33% of the reserved memory of each host for
the restart of a failed-over virtual machine”
• “If you attempt to power on a virtual machine that exceeds the reserved memory threshold, the
operation fails. Due to the memory threshold of an ESXi host, it is possible that not all virtual
machines can be restarted on a running host in the event of a host failure”
http://communities.vmware.com/docs/DOC-17073 - Page 17
LOOK NO FURTHER: VMWARE STATEMENTS ABOUT
VM DENSITY AND RAM CAPACITY
http://blogs.vmware.com/rethinkit/2011/07/understanding-the-vsphere-5-vram-licensing-model.html
8
To justify vRAM TAX, VMware advocates customers to
not do memory overcommit, as it can increase their
licensing costs (and degrade performance)..
…but to show “better TCO” to Microsoft, they perform a
test with 25% memory overcommitted..
Ask Yourself - are you ready to run your business critical
apps with a 25% memory overcommitment?
9
REALITY: vSPHERE 5.0 PERFORMS ONLY 2% BETTER
THAN HYPER-V UNDER ADEQUATE MEMORY, BUT
COSTS 145% MORE
$14,000
Total: $11,800
$12,000
•
•
Cost
$10,000
2.5X
$8,000
$6,000
$4,000
$2,000
vSphere 5 ,
$6,990
Total: $4,810
Windows Server
(Datacenter),
$4,810
•
•
Windows Server
(Datacenter),
$4,810
$0
•
•
Microsoft
VMware
Costs are based on the Principled
Technology benchmark report
The report assumes 1 host (2 CPU) & 24 or
30 guests running Windows Server 2008
R2 SP1
Costs shown only includes the License cost
for Microsoft and VMware (no support
costs are included)
SQL Server and Management Server
licenses (SMSD, System Center VMM, or
VMware vCenter) are not included
VMware cost includes Windows Server
Datacenter edition for running guests
Cost doesn’t include hardware, storage or
IT costs
Performance should always be related to COST…
2% better performance or 145% higher costs?
10
WHY CAN’T MICROSOFT PUBLISH COMPETING
BENCHMARK RESULTS?
• VMware EULA restricts Microsoft (and other competitors) from publishing
benchmarks which are not approved by VMware
• As in their words “We're Not Against Benchmarking – We’re Only Against
Bad Benchmarking” – or in other words benchmarks which may show our
product not performing as well as our competitors
• The restriction (reproduced below for ease of reference) is located in Section
3.3 of the VMware EULA for ESX 4.0
• “You may use the Software to conduct internal performance testing and
benchmarking studies, the results of which you (and not unauthorized third
parties) may publish or publicly disseminate; provided that VMware has
reviewed and approved of the methodology, assumptions and other
parameters of the study.“
11
SUMMARY
• Customers typically don’t overcommit memory in production
environments
• VMware provides guidance to disable overcommit in
production environments
• Some VMware technologies don’t even support Memory
Overcommit (Fault Tolerance, vSphere Storage Appliance)
• “Perceived” better performance comes with a higher cost. The
tradeoff is between 2% better performance and 145% higher costs
• The benchmark report is a great validation that Hyper-V R2 SP1
works as well as vSphere 5.0 under adequate memory
12