Transcript Document

Qlogic and Vmware ESXi5
Confidential and Proprietary information is contained in
this presentation. DO NOT distribute to customers and
non Qlogic personnel.
Topics and Discussion Points
1. Introduction
2. VMware Network and Storage basics
3. Qlogic drivers and firmware for VMware
4. Qlogic vCenter plugin
5. Using vSphere and vCenter to view Qlogic
information.
6. Troubleshooting Qlogic adapters in a
VMware environment.
VMware vSphere Basic Components and Features
•
•
•
•
•
•
•
•
•
•
•
3
VMware ESXi Host
VMware vCenter Server
VMware vSphere Client
VMware Virtual Machine File System (VMFS)
VMware Virtual SMP
VMware vMotion and Storage vMotion
VMware High Availability (HA)
VMware Distributed Resource Scheduler (DRS)
VMware vSphere SDK
VMware Fault Tolerance
vNetwork Distributed Switch (vDS)
QLogic Confidential
Month DD, YYYY
Vmware ESXi5 Features important to Qlogic
• Configuration of Vmware Network abstraction layers within an
ESXi host has direct impact on how a Qlogic Network adapter
performs.
• vMotion, vSwitch uplinks, portgroup vLANs.
• Configuration of Vmware Storage architecture and plugins within
an ESXi host can have direct impact on how a Qlogic CNA or HBA
performs.
• PSP Path failover, datastore allocation
• The Number of VMs configured on an ESXi host can have
direct performance impact on the Qlogic adapter.
4
QLogic Confidential
Month DD, YYYY
ESXi host – VM
ESXi Host
VM 1
Guest OS
Linux
VM 2
Guest OS
Windows
VMFS Datastore
VM2
Vmkernel
Storage and Network abstraction
Qlogic qlcnic driver
Qlogic qla2xxx driver
Virtual
Physical
Qlogic QLE8152 CNA
Network
5
QLogic Confidential
Month DD, YYYY
Direct Path IO
ESXi Host
Never sees
HBA
VM with Direct
Path IO
Qlogic qla2xxx
Driver installed
Normal VM
vmkernel
Qlogic qla2xxx driver
VMFS
Datastore
Qlogic HBA
6
QLogic Confidential
Month DD, YYYY
Qlogic and Vmware ESXi5
Networking Basics
ESXi HOST Networking Basics
•
•
•
•
•
•
•
•
ESXi Physical Network –NICS, Network Switches, Routers etc
ESXi Host Physical NICs – QLE3xxx, QLE81xx, QLE82xx, others
ESXi Virtual Networking – Vmkernel Network
ESXi Virtual Switch (vSwitch)
vSwitch Uplinks
vSwitch Port Groups
ESXi Virtual Nics (vNIC)
ESXi Host Virtual Distributed Switch (VDS)
VMkernel Networking
ESXi Host
Virtual
Machine
Virtual
Machine
Virtual Nic
Virtual Nic
Vmkernel Networking
vSwitch0 VMK0
Vmkernel
Port iSCSI
Portgroup
Virtual Machines
Uplink
Vmkernel
Port
Vmotion
vSwitch1
Uplink
Vmkernel
Port NFS
Virtual
Physical
Adapter
port 1
vmnic2
QLE8242
Adapter
port 2
vmnic3
Adapter
port 1
vmnic4
QLE8242
Adapter
port 2
vmnic5
Physical Network
9
QLogic Confidential
Month DD, YYYY
10
QLogic Confidential
Port 4973
Virtual Switch
Port 4972
V
Ms
vmnic5
Port 4975
Port 4974
Port 4973
ESX Host
Broadcom
vmnic1
QLE8242
vmnic3
QLE8242
vmnic4
Broadcom
vmnic0
Broadcom
vmnic1
VM
s
vmnic2
vmnic5
VM
s
Port 4972
vmnic4
Broadcom
vmnic0
Port 4975
QLE8242
vmnic3
Port 4974
QLE8242
vmnic2
ESXi Virtual Switch
ESX Host
V
Ms
Virtual Switch
Physical Network
Month DD, YYYY
ESXi Standard Virtual Switch
ESXi Host
VMKernel vmk0
Vmkernel port
Management
VM
VM
Virtual
Nic
Virtual
Nic
PortGroup
VM Network
vSwitch0
vSwitch 1
VMkernel
vmk1
Vmkernel port
Vmotion
vSwitch 2
Virtual
QLogic Confidential
10Gport 2
vmnic7
Physical Network
10G port 1
vmnic6
1G Port3 vmnic3
1G Port2 vmnic2
1G Port1 vmnic1
1G Port0 vmnic0
11
Physical
QLE8242
Broadcom
Month DD, YYYY
ESXi Distributed Virtual Switch
ESX Host 1
VM
s
ESX Host 2
VM
s
VM
s
vCenter
QLE8242
vmnic2
QLE8242
vmnic4
Broadco
m
vmnic0
QLE8242
QLE8242
Port 4962
Port 4965
Port 4964
Port 4972
Port 4975
Port 4974
Distributed Virtual Switch DVS
DVS
Management
Broadco
m
Physical Network
12
QLogic Confidential
Month DD, YYYY
vSphere Configuration Tab - Networking
13
QLogic Confidential
Month DD, YYYY
vSphere Configuration - Network Adapters
14
QLogic Confidential
Month DD, YYYY
Qlogic and Vmware ESXi5
Storage Basics
Vmware Storage Basics - Datastore
• The VMware storage architecture consists of layers of abstraction
that hide the physical storage attributes from the virtual
machines.
• VMWare uses a Proprietary Files system VMFS
• Virtual Machine information is stored as a set of files on a VMFS
volume known as a Datastore.
• Datastores can be located on a local host hard drive, or an
external storage device.
16
QLogic Confidential
Month DD, YYYY
Vmware Storage Types – FC, iSCSI, NAS
ESXi host
ESXi host
ESXi host
VM
Qlogic
FC HBA
SW iSCSI
Qlogic
iSCSI HBA
Qlogic
NIC
LAN
SAN
17
LAN
LAN
VMFS
VMFS
Fibre Channel Array
iSCSI Storage Array
VMFS
QLogic Confidential
NFS
NAS Appliance
Month DD, YYYY
Vmware Storage Types - FCoE
ESXi host
Qlogic QLE8xxx
FCoE
10G Eth Port
(FCoE enabled)
Cisco Nexus 5K
FC port
VMFS
SAN
Fibre Channel Array
18
QLogic Confidential
Month DD, YYYY
Vmware Storage Basics - NMP and SATP
The primary functions of NMP are: (Native MultiPathing)
•
•
•
•
•
Manage physical path claiming and unclaiming.
Register and de-register logical devices.
Associate physical paths with logical devices.
Process I/O requests to logical devices:
• Select an optimal physical path for the request (load balance)
• Perform actions necessary to handle failures and request retries.
Support management tasks such as abort or reset of logical devices.
Storage Array Type Plugin – SATP
An Storage Array Type Plugin (SATP) handles path failover operations.
The primary functions of an SATP are:
• Implements the switching of physical paths to the array when a path has
failed.
• Determines when a hardware component of a physical path has failed.
• Monitors the hardware state of the physical paths to the storage array.
Vmware Storage Basics - PSP
•
Path Selection Plugin – PSP handles load balancing operations and is responsible
for choosing a physical path to issue an I/O request to a logical device
VMware provide three PSP: Fixed, MRU or Round Robin.
# esxcli nmp psp list
Name
Description
VMW_PSP_MRU Most Recently Used Path Selection
VMW_PSP_RR Round Robin Path Selection
VMW_PSP_FIXED Fixed Path Selection
•
•
•
Most Recently Used (MRU) — Selects the first working path discovered at system
boot time. If this path becomes unavailable, the ESX host switches to an alternative
path and continues to use the new path while it is available.
Fixed — Uses the designated preferred path, if it has been configured. Otherwise,
it uses the first working path discovered at system boot time. If the ESX host cannot
use the preferred path, it selects a random alternative available path. The ESX host
automatically reverts back to the preferred path as soon as the path becomes
available.
Round Robin (RR) – Uses an automatic path selection rotating through all available
paths and enabling load balancing across the paths.
Vmware Storage Basics – PSA and MPP
PSA Tasks: Pluggable Storage Architecture
•
Loads and unloads mutipathing plugins MPP. (NMP is Vmware’s native Multipathing scheme).
•
Handles physical path discovery and removal via scanning
•
Handles IO queuing to physical storage HBA and to the logical devices
•
Implements logical device bandwidth sharing between virtual machines
•
Provides LUN and physical path statistics.
MPP Tasks: Multipath Plugin
•
The PSA discovers available storage paths and based on a set of predefined rules, the PSA will determine which MPP
should be given ownership of the path.
•
The MPP then associates a set of physical paths with a specific logical device.
•
The specific details of handling path failover for a given storage array are delegated to a sub-plugin called a Storage
Array Type Plugin (SATP).
•
SATP is associated with paths.
The specific details for determining which physical path is used to issue an I/O request (load balancing) to a
storage device are handled by a sub-plugin called Path Selection Plugin (PSP).
•
21
PSP is associated with logical devices.
QLogic Confidential
Month DD, YYYY
Qlogic and Vmware ESXi5
Qlogic Adapter, Driver and Firmware Basics
VMware Driver Information for Qlogic Adapters
• Qlogic Drivers are downloaded from Vmware’s Website. See Use
Qlogic KB #1849 to obtain links to VMware drivers.
• Go to OEM vendor website to obtain bootcode for OEM branded
adapters. ie Oracle, Dell, HP, IBM, Cisco, EMC, etc….
• Go to Qlogic’s Website to obtain BIOs/Bootcode/Firmware/Flash
for Qlogic Adapters
• Go to Qlogic’s Website to obtain adapter utilities, QCC vCenter
plugin.
• ALWAYS ensure the latest drivers and bootcode are installed on
the adapters.
• VMware has inbox drivers for Qlogic FC/FCoE (qla2xxx driver)
and 1G iSCSI products (qla4xxx driver).
• Network drivers must be installed separately, these are NOT
included in ESX distribution.
23
QLogic Confidential
Month DD, YYYY
Bootcode for Qlogic Adapters
• Bootcode aka Flash image, Multiboot image, firmware, is preinstalled on Qlogic adapters from the factory. This version may
not be current.
• Go to OEM vendor website to obtain bootcode for OEM branded
adapters. ie Oracle, Dell, HP, IBM, Cisco, EMC, etc….
• Go to Qlogic website to obtain bootcode/Flash/Multiboot
download for Qlogic adapters.
• Qlogic Bootcode may be installed via Vmware QCC plugin or
qflasher bootable iso obtained from Qlogic support. Refer to kb
181 to obtain qflasher iso.
• Refer to bootcode readme file for specific installation instructions
for OEM branded adapters.
Vmware Qlogic Driver Matrix (as of 12/2012)
OS Version
Qlogic Model/Function
Driver Version
ESX/ESXi4.x
QLE24xx, QLE25xx, QLE81xx,
QLE82xx FibreChannel/FCoe
FCoE/Fibre Channel Driver qla2xxx
841.k1.48.1-1vmw
ESX/ESXi4.x
QLE81xx NIC
Network Driver qlge 1.0.0.47
ESX/ESXi4.x
QLE82xx QLE32xx NIC
Network Driver qlcnic v4.0.747
ESX/ESXi4.x
QLE40xx,QLE82xx iSCSI
iSCSI driver 524.01.45-2vmw
ESXi5.x
QLE24xx, QLE25xx, QLE81xx,
QLE82xx FibreChannel/Fcoe
FCoE/Fibre Channel Driver qla2xxx
934.5.4.0-1vmw
ESXi5.x
QLE81xx NIC
Network Driver qlge 2.0.0.54
ESXi5.x
QLE82xx QLE32xx NIC
Network Driver qlcnic 5.0.750
ESXi5.x
QLE40xx, QLE82xx iSCSI
iSCSI qla4xxx 624.01.43-2vmw:
NPAR Default Configuration NIC,FCoE,iSCSI
NIC
NIC
NIC
NIC
iSCSI
iSCSI
FCoE
FCoE
PF0
PF1
PF2
PF3
PF4
PF5
PF6
PF7
vPort0
vPort1
vPort2
vPort3
vPort4
vPort5
vPort6
vPort7
Port 0
Port 1
Port 0
Port 1
Port 0
Port 1
Port 0
Port 1
Physical
Port#0
Physical
Port#1
P3+ Adapter
QLE8242 Default Functions
Each QLE82xx function has its own pci address:
PCI address
QLE82xx
Function
Vmkernel ID PCI Vendor/Device ID
Driver Name
00:21:00.0
NIC
vmnic6
x1077/8020
qlcnic
00:21:00.1
NIC
vmnic7
x1077/8020
qlcnic
00:21:00.4
iSCSI
vmhba4
x1077/8022
qla4xxx
00:21:00.5
iSCSI
vmhba5
x1077/8022
qla4xxx
00:21:00.6
FCoE
vmhba6
x1077/8021
qla2xxx
00:21:00.7
FCoE
vmhba7
x1077/8021
qla2xxx
Qle8242
Phy Port 0 (labeled1)
PF0 NIC PF4 iscsi
Vmnic6 Vmhba4
qla4xxx
qlcnic
Phy Port1 (labeled 2)
PF6 FCoE
Vmhba6
qla2xxx
PF1
vmnic7
qlcnic
PF5 iscsi PF7 FCoE
Vmhba5 Vmhba7
qla4xxx qla2xxx
NPAR Configuration Options - Personalities
NIC
NIC
NIC
NIC
NIC or
iSCSI
NIC or
iSCSI
NIC or
FCoE
NIC or
FCoE
PF0
PF1
PF2
PF3
PF4
PF5
PF6
PF7
vPort0
vPort1
vPort2
vPort3
vPort4
vPort5
vPort6
vPort7
Port 0
Port 1
Port 0
Port 1
Port 0
Port 1
Port 0
Port 1
Physical
Port#0
Physical
Port#1
P3+
QLE8242 NPAR Enabled Functions
Each QLE82xx function has its own pci address:
PCI address
QLE82xx
Function
Vmkernel ID
PCI Vendor/Device ID
Driver Name
00:21:00.0
NIC
vmnic6
x1077/8020
qlcnic
00:21:00.1
NIC
vmnic7
x1077/8020
qlcnic
00:21:00.2
NIC
vmnic8
X1077/8020
qlcnic
00:21:00:3
NIC
vmnic9
X1077/8020
qlcnic
00:21:00.4
iSCSI
vmhba4
x1077/8022
qla4xxx
00:21:00.5
iSCSI
vmhba5
x1077/8022
qla4xxx
00:21:00.6
FCoE
vmhba6
x1077/8021
qla2xxx
00:21:00.7
FCoE
vmhba7
x1077/8021
qla2xxx
Qle8242
Phy Port 0 (labeled1)
PF0 NIC PF2 NIC
Vmnic6 Vmnic8
qlcnic
qlcnic
PF4 iscsi
Vmhba4
qla4xxx
PF6 FCoE PF1 NIC
Vmhba6 vmnic7
qla2xxx
qlcnic
Phy Port1 (labeled 2)
PF3 NIC PF5 iscsi
vmnic9 Vmhba5
qlcnic
qla4xxx
PF7 FCoE
Vmhba7
qla2xxx
QLE8242 - NPAR MAC Address Limitation
QLE8242
VM 1
Each NPAR port within a single QLE82xx
physical port can accommodate
communication of up to 8 virtual machines
per NPAR port across the internal QLE8242
eSwitch.. In other words 8 MAC addresses per
NPAR port. Example: VM1 talking to VM9
within the same QLE8242 physical port.
NPAR 0
VM 9
NPAR 2
NPAR 4
VM 17
NPAR 6
VM 25
Port 0
Port 1
VM
NPAR 1
NPAR 3
If virtual machines are associated with NPAR
ports that reside on different QLE8242
physical ports, then the internal adapter
eSwitch is not used and there are no MAC
address limitations imposed.
VM
NPAR 5
NPAR 7
30
VM
QLogic Confidential
Month DD, YYYY
Qlogic vCenter QCC Plugin
Qlogic vCenter QCC Plugin
• Qlogic KB Article 2032 -QLogic KnowHow: Video
Installing CIM Provider and QCC vCenter Plugin
http://www.youtube.com/watch?v=aYn2lOnPVqs
• Qlogic KB Article 64 Plugin Installation instructions.
https://support.qlogic.com/app/answers/detail/a_id/64/kw/Plugin
• Qlogic KB Article 1911 Troubleshooting Qlogic Plugin Issues:
https://support.qlogic.com/app/answers/detail/a_id/1911/kw/Plugin
• New Qlogic Document: User’s Guide QLogic QConvergeConsole
Plug-in for VMware vCenter Server
http://filedownloads.qlogic.com/Files/TempDownlods/84045/UsersGui
de_QCC_PlugIn_for_vCenter_Server_SN0054677-00B.pdf
32
QLogic Confidential
Month DD, YYYY
vCenter QConvergeConsole Tab
33
QLogic Confidential
Month DD, YYYY
vCenter QCC Network Map
34
QLogic Confidential
Month DD, YYYY
vCenter QCC NIC QoS
35
QLogic Confidential
Month DD, YYYY
vCenter QCC QLE8242 NPAR Function 7 Transceiver
36
QLogic Confidential
Month DD, YYYY
vCenter QCC QLE8242 Function 7 FCoE Information
37
QLogic Confidential
Month DD, YYYY
Troubleshooting Qlogic in VMware
ESXi 5.x
Vmware KB Articles
QLogic® KnowHow: Installing QLogic
Adapters in VMware ESXi 5.x
Environment Answer ID: 2031
VMware iSCSI Path Failover with
QLogic iSCSI Adapters
Answer ID: 2046
QLogic® Converged Network Adapter
Featuring Multi-Protocol VMware
vMotion Overview (Video) Answer ID:
1962
Execution Throttle and Queue Depth
with VMware and Qlogic
HBAs Answer ID: 599
QLogic® KnowHow: Installing CIM
Provider and QConvergeConsole®
vCenter Plugin Answer ID: 2032
QLogic® Firmware for Vmware
Answer ID: 1870
Updating the HBA Firmware and BIOS
for VMware ESXi Hosts. Answer
ID: 560
VMware Drivers and Tools for
QLogic® Adapters
Answer ID: 1849
VMware Data Gathering for
Troubleshooting QLogic Products
Answer ID: 1855
Qlogic Driver Commands for VMWare
ESXi 5.0 **Internal**
Answer ID: 1844
QLogic® vCenter Plugin Installation
Instructions Answer ID: 64
Troubleshooting VMware QLogic®
Plugin Issues Answer ID: 1911
39
QLogic Confidential
Month DD, YYYY
Additional Information to Gather for Each ESXi Ticket
Please request a vmsupport dump from the impacted ESXi host – please put the
sftp link within the support ticket so we can retrieve the submitted dumps.
Refer to Qlogic KB 1855 VMware Data Gathering for Troubleshooting QLogic
Products:
https://support.qlogic.com/app/answers/detail/a_id/1855/kw/vmware
If the customer has not stated already in their problem description, please ask
the following questions regarding the issue:
•
•
•
•
•
•
•
40
Is the issue occurring on one host or many?
What is the date/timestamp of the issue?
How long has this issue been going on?
Have there been any recent changes to the environment?
Can this issue be recreated or is it intermittent?
How many VMs are running on the impacted ESX host?
Did the issue occur during times of Heavy IO?
QLogic Confidential
Month DD, YYYY
Decoding the vmsupport ESXi 5. dump
• Basics
•
•
•
•
•
•
\proc\scsi\qla2xxx –Model, FC driver, FW, SN information
\proc\scsi\qla4xxx - Model, iSCSI driver, FW, SN information
\commands\nicinfo.sh.txt - Adapter model NIC driver information
\commands\vmware_-v.txt – ESXi version and build number
\commands\smbiosDump.txt – Physical server model information
\commands\esxcfg\mpath_-b.txt - Provides Storage device type
information per adapter connection
• \var\log\vmkwarning – Warning severity message repository
• \var\log\vmkernel – Vmkernel log messages. This is where Qlogic
driver messages are found.
41
QLogic Confidential
Month DD, YYYY
ESXi5 – Vmsupport Dump - Directories
42
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - ESXi5 \proc\scsi\qla2xxx
43
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - \proc\scsi\qla2xxx
QLogic ISP8214 10GbE Controller for QMD8262:
FC Firmware version 4.10.08 (804), Driver version 934.5.4.0-1vmw
• \proc\scsi\qla2xxx\
Host Device Name vmhba3
Flash FW version 4.10.08
ISP: ISP8021, Serial# RFE1215G02534
Request Queue = 0x81011000, Response Queue = 0x81016000
Request Queue count = 128, Response Queue count = 128
Number of response queues for multi-queue operation: 0
Total number of interrupts = 61
Device queue depth = 0x40
Number of free request entries = 128
Total number of outstanding commands: 0
Number of mailbox timeouts = 0
Number of ISP aborts = 0
Number of loop resyncs = 1
Host adapter:Loop State = <DEAD>, flags = 0x1a268
Link speed = <Unknown>
Dpc flags = 0x100c0
Link down Timeout = 030
Port down retry =
005
Login retry count =
008
ZIO mode = 0x0, ZIO timer = 0
Commands retried with dropped frame(s) = 0
Product ID = 0000 0000 0000 0000
Minidump feature: Enabled, driver_capture_mask = 0x1f
NPIV Supported : Yes
Max Virtual Ports = 63
Ethernet Device Information:
VLAN ID: 0
MAC: 00:00:00:00:00:00
SCSI Device Information:
scsi-qla0-adapter-node=2000d4ae525cd274:000000:0;
scsi-qla0-adapter-port=2001d4ae525cd274:000000:0;
• Adapter Model, Driver and firmware
Version
• Vmkernel ID for adapter port
• Adapter Serial number
• Adapter, Device and Link State
• Driver option settings
• Wwnn, wwpn of adapter
• Attached device information
FC Target-Port List:
44
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - \proc\scsi\qla4xxx
•
•
•
•
•
QLogic iSCSI Adapter
Driver version 624-01-34vmw
Firmware version 4.09.56.44976
SCSI HBA Information:
vmhba5
iSCSI Name = "iqn.200004.com.qlogic:isp8214.000E1E04BE9E.5"
IPv4 Address = 172.31.74.218 (3)
Mask = 255.255.255.0, Gateway = 172.31.1.1
\proc\scsi\qla4xxx\
Driver and firmware Version
Adapter, Device and Link State
Driver option settings
Attached device information
Driver Info:
IPv4 options=0x9000 tcp=0x27
IPv6 options=0x0 add'l=0x0
Number of free request entries = 1024 of 1024
Number of free aen entries
= 512 of 512
Number of Mailbox Timeouts = 0
Total H/W int = 0x4b8
Total RISC int = 0x0
ReqQptr=0x410024060000, ReqIn=0, ReqOut=0
Adapter flags = 0x20543, DPC flags = 0x0
ha=0x41000c01b330
Device state = <READY>
Firmware state = <READY> (0x86811)
Link State = UP
Total number of IOCBs (used/max) = (0/1024)
I/O Port = 0x0
Transciever Plugged In
TxRcvr State = 0x30
Runtime Parameters:
ql4xextended_error_logging=0
recovery_tmo=16
cmd_timeout=20
ql4xdiscoverywait=10
ql4xdontresethba=0
ka_timeout=14
ioctl_timeout=10
ql4xenablemsix=1
SCSI Device Information:
45
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - \commands
46
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - \commands\nicinfo.sh.txt
Network Interface Cards Information.
Name
PCI Device
Driver Link Speed Duplex MAC Address
MTU
Description
---------------------------------------------------------------------------------------vmnic0 0000:001:00.0 bnx2
Up
1000 Full
78:2b:cb:40:be:5f 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic1 0000:001:00.1 bnx2
Up
1000 Full
78:2b:cb:40:be:61 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic2 0000:002:00.0 bnx2
Up
1000 Full
78:2b:cb:40:be:63 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic3 0000:002:00.1 bnx2
Up
1000 Full
78:2b:cb:40:be:65 1500 Broadcom Corporation Broadcom NetXtreme II BCM5709 1000Base-T
vmnic4 0000:005:00.0 igb
Up
1000 Full
00:1b:21:d4:e2:88 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic5 0000:005:00.1 igb
Up
1000 Full
00:1b:21:d4:e2:89 1500 Intel Corporation 82576 Gigabit Network Connection
vmnic6 0000:006:00.0 qlcnic Down
0 Half
00:0e:1e:04:be:98 1500 QLogic Corp QLogic QLE8242 Pci Express to 10GbE, Dual Channel
CNA
vmnic7 0000:006:00.1 qlcnic Up
10000 Full
00:0e:1e:04:be:9c 1500 QLogic Corp QLogic QLE8242 Pci Express to 10GbE, Dual Channel
CNA
vmnic6 0000:006:00.0 qlcnic Down
0 Half
00:0e:1e:04:be:98 1500 QLogic Corp QLogic QLE8242 Pci
Express to 10GbE, Dual Channel CNA
vmnic7 0000:006:00.1 qlcnic Up
10000 Full
00:0e:1e:04:be:9c 1500 QLogic Corp QLogic QLE8242 Pci
Express to 10GbE, Dual Channel CNA
47
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - \commands\nicinfo.sh.txt expanded
NIC:
vmnic7
NICInfo:
Advertised Auto Negotiation: false
Advertised Link Modes: 10000baseT/Full
Auto Negotiation: false
Cable Type: FIBRE
Current Message Level: -1
Driver Info:
NICDriverInfo:
Bus Info: 0000:06:00.1
Driver: qlcnic
Firmware Version: 4.9.56
Version: 5.0.741
Link Detected: true
Link Status: Up
Name: vmnic7
PHY Address: 1
Pause Autonegotiate: false
Pause RX: false
Pause TX: false
Supported Ports: TP, FIBRE
Supports Auto Negotiation: false
Supports Pause: true
Supports Wakeon: true
Transceiver: external
Wakeon: None
48
Ring parameters for vmnic7:
Pre-set maximums:
RX:
8192
RX Mini:
0
RX Jumbo: 1024
TX:
2048
Current hardware settings:
RX:
1024
RX Mini:
0
RX Jumbo: 128
TX:
1024
NIC statistics:
bad_skblen: 0
rcvd_badskb: 0
xmit_called: 78
xmit_finished: 78
rx_dropped: 0
tx_dropped: 0
csummed: 2287108
rx_pkts: 2287554
lro_pkts: 0
lro_bytes: 0
tso_pkts: 0
tso_bytes: 0
rx_bytes: 208672848
tx_bytes: 4680
QLogic Confidential
Month DD, YYYY
\proc\scsi\qla2xxx\7 QLE2562
QLogic PCI to Fibre Channel Host Adapter for QLE2562:
FC Firmware version 5.06.02 (90d5), Driver version
911.k1.1-19vmw
Host Device Name
vmhba2
BIOS version 3.00
FCODE version 3.15
EFI version 2.21
Flash FW version 5.04.01
ISP: ISP2532
Request Queue = 0x83010000, Response Queue = 0x83051000
Request Queue count = 2048, Response Queue count = 512
Number of response queues for multi-queue operation: 2
CPU Affinity mode enabled
Device queue depth = 0x40
Number of free request entries = 546
Total number of outstanding commands: 0
Number of mailbox timeouts = 0
Number of ISP aborts = 0
Number of loop resyncs = 7
Host adapter:Loop State = <READY>, flags = 0x109aa68
Link speed = <4 Gbps>
\commands\esxcfg\mpath_-b.txt
naa.6000d31000310000000000000000000e : COMPELNT
Fibre Channel Disk
(naa.6000d31000310000000000000000000e)
vmhba2:C0:T1:L1 LUN:1 state:active fc Adapter:
WWNN: 20:00:00:24:ff:20:42:92 WWPN:
21:00:00:24:ff:20:42:92 Target: WWNN:
50:00:d3:10:00:31:00:02 WWPN:
50:00:d3:10:00:31:00:2d
vmhba3:C0:T1:L1 LUN:1 state:active fc Adapter:
WWNN: 20:00:00:24:ff:20:42:93 WWPN:
21:00:00:24:ff:20:42:93 Target: WWNN:
50:00:d3:10:00:31:00:02 WWPN:
50:00:d3:10:00:31:00:2d
SCSI Device Information:
scsi-qla0-adapter-node=20000024ff204292:020300:0;
scsi-qla0-adapter-port=21000024ff204292:020300:0;
FC Target-Port List:
scsi-qla0-target-0=5000d3100031000b:010000:81:Online;
scsi-qla0-target-1=5000d3100031002d:010100:82:Online;
49
QLogic Confidential
Month DD, YYYY
\commands\esxcfg\mpath_-b.txt
naa.6000d31000310000000000000000000e : COMPELNT Fibre Channel Disk
(naa.6000d31000310000000000000000000e)
vmhba2:C0:T1:L1 LUN:1 state:active fc Adapter: WWNN: 20:00:00:24:ff:20:42:92 WWPN:
21:00:00:24:ff:20:42:92 Target: WWNN: 50:00:d3:10:00:31:00:02 WWPN: 50:00:d3:10:00:31:00:2d
vmhba3:C0:T1:L1 LUN:1 state:active fc Adapter: WWNN: 20:00:00:24:ff:20:42:93 WWPN:
21:00:00:24:ff:20:42:93 Target: WWNN: 50:00:d3:10:00:31:00:02 WWPN: 50:00:d3:10:00:31:00:2d
naa.6000d31000310000000000000000000f : COMPELNT Fibre Channel Disk
(naa.6000d31000310000000000000000000f)
vmhba2:C0:T1:L2 LUN:2 state:active fc Adapter: WWNN: 20:00:00:24:ff:20:42:92 WWPN:
21:00:00:24:ff:20:42:92 Target: WWNN: 50:00:d3:10:00:31:00:02 WWPN: 50:00:d3:10:00:31:00:2d
vmhba3:C0:T1:L2 LUN:2 state:active fc Adapter: WWNN: 20:00:00:24:ff:20:42:93 WWPN:
21:00:00:24:ff:20:42:93 Target: WWNN: 50:00:d3:10:00:31:00:02 WWPN: 50:00:d3:10:00:31:00:2d
naa.6d4ae520656689001698ee8f1212e97c : Local DELL Disk (naa.6d4ae520656689001698ee8f1212e97c)
vmhba1:C2:T1:L0 LUN:0 state:active Local HBA vmhba1 channel 2 target 1
naa.6d4ae520656689001698ee84116e269b : Local DELL Disk (naa.6d4ae520656689001698ee84116e269b)
vmhba1:C2:T0:L0 LUN:0 state:active Local HBA vmhba1 channel 2 target 0
naa.6000d310003100000000000000000010 : COMPELNT Fibre Channel Disk
(naa.6000d310003100000000000000000010)
vmhba2:C0:T1:L3 LUN:3 state:active fc Adapter: WWNN: 20:00:00:24:ff:20:42:92 WWPN:
21:00:00:24:ff:20:42:92 Target: WWNN: 50:00:d3:10:00:31:00:02 WWPN: 50:00:d3:10:00:31:00:2d
vmhba3:C0:T1:L3 LUN:3 state:active fc Adapter: WWNN: 20:00:00:24:ff:20:42:93 WWPN:
21:00:00:24:ff:20:42:93 Target: WWNN: 50:00:d3:10:00:31:00:02 WWPN: 50:00:d3:10:00:31:00:2d
mpx.vmhba0:C0:T0:L0 : Local PLDS CD-ROM (mpx.vmhba0:C0:T0:L0)
vmhba0:C0:T0:L0 LUN:0 state:active Local HBA vmhba0 channel 0 target 0
t10.DP______BACKPLANE000000 : Local DP Enclosure Svc Dev (t10.DP______BACKPLANE000000)
vmhba1:C0:T32:L0 LUN:0 state:active Local HBA vmhba1 channel 0 target 32
50
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport
\commands\vmware_-v.txt
• Contains the version of ESXi
VMware ESXi 5.0.0 build469512
\commands\smbiosDump.txt
• The smbiosDump.txt file will
provide information
regarding the physical server
model
Dumping live SMBIOS data!
Type 218 Record: #55808
Data 00: da 0b 00 da b2 00 17 00
0e 20 00
BIOS Info: #0
Size: 0x00018
Vendor: "Dell Inc."
Version: "6.1.0"
Date: "10/18/2011“
Manufacturer: "Dell Inc."
Product: "PowerEdge R710"
Serial: "28ZBSS1"
51
QLogic Confidential
Month DD, YYYY
Decoding the vmsupport - \commands\esxcfg-info_-a.txt
52
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - \commands\esxcfg-info
Structure
\tmp\esxcfg-info – Contains entire system configuration.
+Host :
\==+Hardware Info :
\==+Network Info :
\==+Storage Info :
\==+iScsi Interface :
\==+System Info :
\==+Loaded Modules :
\==+Module :
53
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport \var\log\vmkwarning and
\var\log\vmkernel
Search the \var\log\Vmkwarning or
\var\log\vmkernel file for:
• Qlogic driver messages by
name:
qla2xxx,qlcnic,qlge,qla4xxx
• The vmhba or vmnic ID specific
to the Qlogic adapter.
Helpful KBs for vmkernel
messages
• https://support.qlogic.com/app/a
nswers/detail/a_id/628/kw/vmker
nel
• https://support.qlogic.com/app/a
nswers/detail/a_id/2025/kw/vmke
rnel
54
QLogic Confidential
Month DD, YYYY
Decoding the Vmsupport - \var\log\vmkwarning - detail
•
•
•
•
•
•
•
•
•
•
•
•
•
55
2012-07-17T12:50:47.432Z cpu12:2060)WARNING: NMP:
nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.2000000c50c3381c" state in doubt;
requested fast path state update...
2012-07-17T12:50:47.634Z cpu12:2060)WARNING: NMP:
nmp_DeviceRequestFastDeviceProbe:237:NMP device "naa.20000004cfe72909" state in doubt;
requested fast path state update...
2012-07-17T12:55:45.005Z cpu10:2204)WARNING: NMP: nmp_IssueCommandToDevice:2954:I/O could
not be issued to device "naa.2000000c50c33811" due to Not found
2012-07-17T12:55:45.005Z cpu10:2204)WARNING: NMP: nmp_DeviceRetryCommand:133:Device
"naa.2000000c50c33811": awaiting fast path state update for failover with I/O blocked. No prior
reservation exists on the device.
2012-07-17T12:55:45.005Z cpu10:2204)WARNING: NMP: nmp_DeviceStartLoop:721:NMP Device
"naa.2000000c50c33811" is blocked. Not starting I/O from device.
2012-07-17T12:55:45.005Z cpu0:2698)WARNING: NMP: nmpDeviceAttemptFailover:599:Retry world
failover device "naa.2000000c50c33811" - issuing command 0x412440dc0740
2012-07-17T12:55:45.005Z cpu0:2698)WARNING: NMP: nmpDeviceAttemptFailover:658:Retry world
failover device "naa.2000000c50c33811" - failed to issue command due to Not found (APD), try again...
command due to Not found (APD), try again...
2012-07-17T12:56:15.319Z cpu2:2698)WARNING: NMP: nmpDeviceAttemptFailover:708:Logical device
"naa.2000000c50c33811": awaiting fast path state update...
2012-07-17T12:56:25.006Z cpu12:2210)WARNING: NMP: nmp_DeviceStartLoop:721:NMP Device
"naa.2000000c50c33811" is blocked. Not starting I/O from device.
2012-07-17T12:56:25.319Z cpu2:2698)WARNING: NMP: nmpDeviceAttemptFailover:562:Retry world
restore device "naa.2000000c50c33811" - no more commands to retry
2012-07-17T12:56:25.319Z cpu2:2698)WARNING: NMP: nmp_IssueCommandToDevice:2954:I/O could
not be issued to device "naa.2000000c50c33811" due to Not found
2012-07-17T12:56:25.319Z cpu2:2698)WARNING: NMP: nmp_DeviceRetryCommand:133:Device
"naa.2000000c50c33811": awaiting fast path state update for failover with I/O blocked. No prior
reservation exists on the device.
QLogic Confidential
Month DD, YYYY
Vmkernel Message Formats
• Jan 11 15:57:47 pou-esx08 vmkernel: 1:04:57:35.963
cpu12:4197)<6>qla2xxx 0000:0f:00.0: scsi(5:3:4): Abort command
issued -- 0 7ac84ad 2003.
• Message Source - - vmkernel:
• Device - - <6>qla2xxx 0000:0f:00.0 Device module presenting the
error is QLogic FC driver qla2xxx pci bus 0f:00:00
• scsi controller:target:lun) arbitrary number assigned by scsi layer
- scsi(5:3:4):
• Message -- Abort command issued -- 0 7ac84ad 2003. (Driver
qla2xxx issues abort command; reason is not specified in this
message)
56
QLogic Confidential
Month DD, YYYY