Transcript Advanced Network Features - E2EVC Virtualization Conference

Didier Van Hoye
Technical Architect @ FGIA
MVP – Virtual Machine
Microsoft Extended Experts Team Member
[email protected]
@workinghardinit
http://workinghardinit.wordpress.com
• In the host networking stack
• In the NICs
• In the switches
& routers
Receive Side Scaling (RSS)
Receive Segment Coalescing (RSC)
Dynamic Virtual Machine Queuing (DVMQ)
Single Root I/O Virtualization (SR-IOV)
NIC TEAMING
RDMA/Multichannel support for virtual machines on SMB3.0
DHCP Guard/Router Guard/Port Mirroring
• RSS exists for many years. Windows Server 2012
takes RSS to the next generation of servers
• Spreads interrupts across all available CPUs
• Even for those very large scale hosts
• RSS now works across k-groups
• Even RSS is “Numa Aware” to optimize performance
• Now load balances UDP traffic across CPUs
• Coalesces packets in the NIC so the stack
processes fewer headers
• Multiple packets belonging to connection that
arrive within a single interrupt are coalesced to a
larger packet (max of 64 K) by the NIC
• 10 – 30% improvement in I/O overhead
• VMQ is to virtualization what RSS is to
native workloads
• Dynamic VMQ reassigns available
queues based on changing networking
demands of the VMs
Root Partition
CPU
0
CPU
1
CPU
2
Root Partition
Root Partition
CPU
3
CPU
0
CPU
1
CPU
2
Physical NIC
Physical NIC
No VMQ
Static VMQ
CPU
3
CPU
0
CPU
1
CPU
2
CPU
3
Physical NIC
Dynamic VMQ
Adaptive processing = optimal performance across changing workloads
Host
Root Partition
Hyper-V Switch
Host
Virtual
Machine
Virtual NIC
Routing
VLAN Filtering
Data Copy
Physical NIC
Network I/O path without SRIOV
Root Partition
Hyper-V Switch
Virtual
Machine
Virtual Function
Routing
VLAN Filtering
Data Copy
SR-IOV Physical NIC
Network I/O path with SRIOV
Windows Server 2012 supports direct device assignment to virtual machines
without compromising flexibility
DIRECT DEVICE ASSIGNMENT TO VIRTUAL MACHINES WITHOUT COMPROMISING FLEXIBILITY
• Reduces CPU utilization for processing
network traffic
• Reduces latency of network path
• Increases throughput
• Supports Live Migration
• Requires:
– Chipset: Interrupt and DMA
remapping
– BIOS Support
– CPU: Hardware virtualization, EPT or
NPT
Root Partition
Virtual Machine
Hyper-V Switch
Routing
VLAN Filtering
Data Copy
Virtual NIC
VMBUS
Virtual Function
Physical NICSR-IOV Physical NIC
Network
SR-IOV
NetworkI/O
I/Opath
pathwithout
with SR-IOV
Turn On IOV
•
•
•
•
Enable IOV (VM NIC Property)
Virtual Function is “Assigned”
“NIC” automatically created
Traffic flows through VF
•
Live Migration
Post Migration
• Switch back to Software path
• Remove VF from VM
• Migrate as normal
• Reassign Virtual Function
•
Assuming resources are available
Software path is not used
Virtual Machine
Network Stack
“NIC”
Software NIC
“NIC”
VM has connectivity even if
• Switch not in IOV mode
Software Switch
(IOV Mode)
• IOV physical NIC not present
Virtual Function
PhysicalSR-IOV
NIC
Physical NIC
• Different NIC vendor
• Different NIC firmware
Software Switch
(IOV Mode)
Virtual Function
SR-IOV Physical NIC
Windows Server 8 – Developer Preview
Even when hardware fails …
… our customers want continuous availability
TEAMING
Tenant 1: Multiple VM Workloads
Tenant 2: Multiple VM Workloads
Data Center
Windows Server 8 – Developer Preview
• Customers are dealing with
way to many issues.
• NIC vendors would like to get
rid of supporting this.
• Microsoft needs this to be
competitive & complete the
solution stack.
Hyper-V Extensible Switch
LBFO Admin GUI
Frame distribution/aggregation
Failure detection
Control protocol implementation
WMI
LBFO Provider
LBFO Configuration
DLL
IOCTL
Port 1
Port 2
Port 3
•
•
Multiple modes: switch dependent and
switch independent
Hashing modes: port and 4-tuple
Active -Active and Active - Standby
Kernel mode
•
User mode
Virtual miniport 1
IM MUX
Protocol edge
NIC 1
NIC 2
NIC 3
Network switch
VM (Guest Running Any OS)
VM (Guest Running Windows Server 2012)
LBFO Teamed NIC
Hyper-V virtual switch
SR-IOV Not exposed
LBFO Teamed NIC
SR-IOV NIC
SR-IOV NIC
Parent NIC Teaming
Hyper-V virtual
switch
SR-IOV NIC
Hyper-V virtual
switch
SR-IOV NIC
Guest NIC Teaming
• Addresses congestion in network stack by offloading the
stack to the network adapter
• Great for storage traffic: high throughput with low CPU
utilization
• SMB-Direct uses new RDMA capability if the NICs
support this
• Windows Server 2012 now supports RDMA low latency,
high speed application-to-application data transfer
DCTCP/DCB
Consistent Device Naming
Network virtualization
Generic Routing Encapsulation (GRE)
IPSEC Task Offload for Virtual Machines (IPsecTOv2)
Wireless Network Support
1Gbps flow controlled by TCP
• Requires 400 to 600KB of memory
• TCP saw tooth visible
1Gbps flow controlled by DCTCP
• Requires 30KB of memory
• Smooth
• W2K12 deals with network congestion by reacting to
the degree & not merely the presence of congestion.
• DCTCP aims to achieve low latency, high burst tolerance,
and high throughput, with small buffer switches.
• Requires Explicit Congestion Notification (ECN, RFC
3168) capable switches
• Algorithm enabled when it makes sense (low round trip
times, i.e. in the data center)
Running out of buffer in a
switch gets you in to
stop/go hell by getting a
boatload of green, orange
& red lights along your way
Big buffers mitigate this
but are very expensive
You want to be in a green wave
Windows Server 2012 & ECN
provides network traffic control
• Prevents congestion in NIC & network by reserving
bandwidth for particular traffic types
• Windows 2012 provides support & control for DCB,
tags packets by traffic type
• Provides lossless transport for mission critical
workloads
1. Enhanced Transmission Selection (IEEE
802.1Qaz)
2. Priority Flow Control (IEEE 802.1Qbb)
3. (Optional) Datacenter Bridging Exchange
protocol
4. (Not required) Congestion Notification (IEEE
802.1Qau)
• Multi-tenant scenarios: hide the tenant’s multipremise networking from the datacenter’s
networking.
• GRE (RFCs 2784 & 2890) provides the mechanism
to tunnel tenant networks over the datacenter
network
• GRE breaks today’s task offloads if the NIC vendors
don’t support GRE offload
192.168.2.22192.168.5.55
192.168.2.22192.168.5.55
192.168.2.22
10.1.1.11
10.1.1.11
192.168.5.55
10.1.1.12
10.1.1.12
•
•
•
•
IPsec is a CPU intensive workload => Offload to NIC
In demand due to compliance (SOX, HIPPA, etc.)
IPsec is required & needed for secured operations
Only available to host/parent workloads in W2K8R2
– Now extended to VMs
– Managed by the Hyper-V switch
• Manage the Network
Bandwidth with a Maximum
and a Minimum value
• SLAs for hosted Virtual
Machines
• Control per VMs and not per
HOST
Root Partition
VM1
VM NIC
Host NIC
BFE Service
Firewall
Callout
Extensible Switch
Filtering Engine
Extension Protocol
Capture Extensions
Capture
Extensions
WFP Extensions
Extensions
WFP
Filtering Extensions
Filtering
Extensions
Forwarding Extensions
Forwarding
Extensions
Extension Miniport
Physical NIC
VM2
VM NIC
Windowsextensions
Filter
Platform
(WFP)
Extensions
• Capture
Filtering
Forwarding
extensions
candirect
inspect
also
be
traffic,
traffic
implemented
defining
and can
inspect,
drop,
modify,
and
insert
packets
generate
using
the
destination(s)
NDIS
new
filtering
traffic
ofAPIs
for
each
report
packet
purposes
using WFP APIs
Example: VM DoS Prevention by Broadcom
• Capture
Forwarding
extensions
extensions
do not
can modify
captureexisting
and filter
• Extensible
Windows
Antivirus
and
Firewall
software
uses
traffic
Switch
traffic
WFP for traffic filtering
Examples:
Cisco
Nexus
1000V and UCS
Example:
sflow
by
inMon
Example:NEC
Virtual
Firewall by 5NINE Software
OpenFlow