VMware Storage Best Practices
Download
Report
Transcript VMware Storage Best Practices
VSP3868
VMware vStorage
Best Practices
Name, Title, Company
Disclaimer
This session may contain product features that are
currently under development.
This session/overview of the new technology represents
no commitment from VMware to deliver these features in
any generally available product.
Features are subject to change, and must not be included in
contracts, purchase orders, or sales agreements of any kind.
Technical feasibility and market demand will affect final delivery.
Pricing and packaging for any new technologies or features
discussed or presented have not been determined.
2
Agenda
VMware Storage Best Practices
Storage Environment
Performance Improvements and Tuning
Performance Monitoring
Common Storage Issues
3
VMware Storage Best Practices
Overview
Storage Environment
Business Continuity and Disaster Recovery
Vendor Knowledge and Partnership
Hard Disk Performance
4
Business Continuity and Disaster Recovery
Backups
• Number and depth based on application
RPO/RTO
• Test your backups
Support considerations
• Virtual Machine Snapshots are not backups!
• Available backups allow for a wider variety of
options in data loss scenarios
5
Vendor Knowledge and Partnership
Know your SAN environment
• Open communication channel to your internal storage team
• Follow storage vendor best practice guidelines for ESX(i)
Know your storage vendor
• Support relationship – proactive versus reactive
• Software/Firmware updates for storage hardware
Know your storage network or fabric
• HBA and switch software/firmware updates
6
Hard Disk Performance
Areas of Concern
• Disk subsystem bottlenecks
• Spikes and sustained latency
Performance versus Capacity
• Disk performance does not scale with drive size
• Larger drives generally equate lower performance
For example: 1 TB of space is required
for an app
• 2 x 500GB 15K RPM SAS drives = ~300 IOPS
• Capacity needs satisfied, Performance low
• 8 x 146GB 15K RPM SAS drives = ~1,240 IOPS
• Capacity needs satisfied, Performance high
More spindles generally equals greater performance - check with
your SAN array vendor for technology details
7
VMware Storage Best Practices
Overview
Storage Environment
Performance Improvements and Tuning
vStorage APIs for Array Integration (VAAI)
Multipathing
Storage I/O Control (SIOC)
8
vStorage APIs for Array Integration (VAAI)
VAAI
• Enables your host to offload specific virtual machine and storage management
operations to compliant storage hardware.
Performance Benefits
• ESX(i) hosts consume less CPU and memory resources, and less storage
fabric and network bandwidth.
• Improved Migration of virtual machines with Storage vMotion
• Faster deployment and cloning of virtual machines
• Reduces SCSI Reservation conflicts
• Faster zeroing operations (creation of Fault Tolerant VMs)
Benefits derived through new SCSI primitives
Full Copy
Block Zeroing
9
Atomic Test and Set (ATS) Locking
vStorage APIs for Array Integration (VAAI) – Requirements
ESX(i) version 4.1 or later.
VAAI compatible storage solution
VAAI hardware acceleration is enabled by default on ESX(i)
ESX(i) 4.1 – block devices only (NAS not yet supported)
ESXi 5.0 – NFS and block device support available for vendor
implementation
Consult storage vendor for array configuration requirements
10
vStorage APIs for Array Integration (VAAI) - Summary
VAAI
• Is configured by default, can be disabled through host Advanced Settings,
if required by storage vendor
• Storage array must support VAAI SCSI primitives
• Tested VAAI functionality can be reviewed via the vSphere GUI
11
Multipathing - Overview
Pluggable Storage Architecture - PSA
Native Multipathing Plugin – NMP
• Storage Array Type Plugin – SATP
• Path Selection Policy – PSP
Fixed Path
Most Recently Used
12
Round Robin
Multipathing – Path Selection Plugins
Most Recently Used – MRU
• Active/Passive arrays
Fixed
• Active/Active arrays
• Asymmetrical LUN Unit Access – ALUA
Round Robin
• Load balance I/O for improved performance
• Uses Active or Optimized paths, not Standby paths
Vendor/Partner Multipathing Plugin solutions
13
Multipathing – Best Practices
Consistency
• One PSP per LUN across all ESX(i) hosts
Configuration
• Match to storage array type (AP, AA, AA-ALUA)
• All LUNs on AP arrays to all ESX(i) hosts through same storage processor
Multipathing – iSCSI
• One vmkernel port group per physical NIC
• Follow VMware and/or vendor best practice specifics
• VMware iSCSI Configuration Guide (pages 29-51)
• https://www.vmware.com/pdf/vsphere4/r41/vsp_41_iscsi_san_cfg.pdf
14
Storage I/O Control - Overview
Storage I/O Control
• Monitors I/O latency to datastores at each ESX host sharing a physical device.
• When the average normalized latency exceeds a set threshold (30ms by
default), the datastore is considered to be congested.
If congested, SIOC distributes
available storage resources to virtual
machines in proportion to their
configured shares.
Used to determine migration needs
with Storage DRS in ESXi 5.0
15
Storage I/O Control – Usage Scenario
Without SIOC, Disk shares do not provide a fair distributed load on
a shared VMFS.
16
VMware Storage Best Practices
Overview
Storage Environment
Performance Improvements and Tuning
Performance Monitoring
vCenter/vSphere Client
esxtop/resxtop
Latency Values
Partner Vendor Tools
17
Performance Monitoring – vCenter/vSphere Client
Performance Charts
• Performance Advanced Chart Options Datastore/Disk
• Latency and IOPS statistic counters for disks
• Historical availability depends on vCenter Statistics level (vCenter Advanced
Settings)
• Statistics available for ESX(i) host objects
• Datastore
• Read/Write Latency = device average in milliseconds
• Disk
• Physical Device Command/Write/Read Latency = device average in milliseconds
Additional detail on vCenter Performance Metrics
• vSphere Datacenter Administration Guide, Appendix B, page 265:
• http://www.vmware.com/pdf/vsphere4/r41/vsp_41_dc_admin_guide.pdf
18
Performance Monitoring – vCenter/vSphere Client
19
Performance Monitoring – esxtop/resxtop
esxtop/resxtop
• local/remote
• More depth than vSphere Client
Storage statistics
• CMDS/s – I/Ops
• DAVG, KAVG, GAVG – device/kernel/guest latency in milliseconds
• NFS latency, vSphere 4.1 or later
• Statistics for disk device, disk adapter, virtual machine
Using esxtop to identify storage performance issues
• http://kb.vmware.com/kb/1008205
Additional detail on esxtop/resxtop
• vSphere Resource Management Guide, Appendix A, page 89
• http://www.vmware.com/pdf/vsphere4/r41/vsp_41_resource_mgmt.pdf
20
Performance Monitoring – esxtop/resxtop
21
Performance Monitoring – Latency Values
Good latency values are subjective based on application
performance needs and storage environment capabilities
Lower is better!
General rules of thumb:
• 10 milliseconds or less is adequate
• 20 milliseconds or higher, sustained, should be investigated
• Sustained spikes in latency may need to be investigated
Commands not acknowledged by the SAN within 5000 ms will be
aborted
• This may lead to performance issues as the commands are retried
• Multiple aborts should be investigated
22
Performance Monitoring – Partner Vendor Tools
Partner Tools
• vSphere APIs
vCenter Plugins
• Consult with storage vendor
23
VMware Storage Best Practices
Storage
Storage Environment
Performance Improvements and Tuning
Performance Monitoring
Common Storage Issues
Snapshots
Misconfiguration
Improper Device Removal
24
Common Storage Issues – VM Snapshots
Best Practices
• Snapshots are not backups
• Regular monitoring
• Configure vCenter Snapshot alarms – KB 1018029
• Limit to 2-3 snapshots in a chain to prevent performance degradation
• Delete all snapshots before Virtual Machine disk changes
• Confirm via command line if uncertainty of snapshot state exists
• As a general rule, use Snapshots for no more than 24-72 hours
Improvements
• ESX(i) 4.0 U2 – Snapshot deletion takes up less space on disk
• ESXi 5.0 – New functionality to monitor snapshots and provide warning if
snapshots need consolidation.
Snapshot Best Practices – KB 1025279
Understanding Virtual Machine Snapshots in ESX – KB 1015180
25
Common Storage Issues - Misconfiguration
Firmware and Driver issues
• SAN array firmware does not match to ESX(i) version
• HBA, FC switch, physical NIC firmware and/or drivers out of date
• VMware Hardware Compatibility Guide
http://www.vmware.com/resources/guides.html
Inconsistent Path Selection Policy
• (example 1) Host A, LUN A – MRU; Host B, LUN A – Fixed
• (example 2) Host A, LUN A – vendor MMP; Host B, LUN A - Fixed
LUN detected as a snapshot
• Missing datastores resolved via force-mount
• Resignature for long-term solution
• Consistent LUN presentation to all hosts from SAN
26
Common Storage Issues – Improper Device Removal
Improper removal of a physical device containing a VMFS
datastore or RDM
• APD (All Paths Dead) state in ESX(i) 4.x
• Upgrade to 4.0 U2 / 4.1 U1 or more recent
• Follow steps outlined in KB 1015084 when removing a datastore
Rescan after making changes to the storage environment!
VMFS
VMFS
27
RDM
VMware Storage Best Practices
Summary
Storage Environment
Performance Improvements and Tuning
Performance Monitoring
Common Storage Issues
28
Appendix - VMware Storage Best Practices
ESX FC SAN configuration guide
• https://www.vmware.com/pdf/vsphere4/r41/vsp_41_san_cfg.pdf
ESX iSCSI Configuration Guide
• https://www.vmware.com/pdf/vsphere4/r41/vsp_41_iscsi_san_cfg.pdf
MSCS Configuration Guide
• https://www.vmware.com/pdf/vsphere4/r41/vsp_41_mscs.pdf
VMware Performance Best Practices 4.1
• http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.1.pdf
29
COMING DECEMBER 2011
By Mike Laverick
Complete guide to using SRM 5.0 with
both VMware and third-party products.
Available in print and eBook formats and
online through Safari Books Online.
VMWORLD 2011 EXCLUSIVE
Buy the Pre-Publication Edition, Receive the
eBook edition for FREE – a $39.99 Value.
Visit VMworld Bookstore for details.
ISBN: 9780321799920
COMING DECEMBER 2011
By Cody Bunch
Complete with real-world case studies
and code.
Available in print and eBook formats and
online through Safari Books Online.
VMWORLD 2011 EXCLUSIVE
Get a FREE preview chapter from this and
two other VMware Press releases.
ISBN: 9780321799913
PLUS: PRE-ORDER and Save 30%
Visit VMworld Bookstore for details
COMING FEBRUARY 2012
By Mostafa Khalil
Demystifies the "black box" of vSphere
storage.
Available in print and eBook formats and
online through Safari Books Online.
VMWORLD 2011 EXCLUSIVE
Get a FREE preview chapter from this and
two other VMware Press releases.
ISBN: 9780321799937
PLUS: PRE-ORDER and Save 30%
Visit VMworld Bookstore for details