Transcript Jeff Mealiffe Senior Development Lead Microsoft Corporation Session Code: UNC309 Agenda High-level product direction for scale Guidelines and ratios Role specific details Virtualization considerations Toolkit for planning and.

Jeff Mealiffe
Senior Development Lead
Microsoft Corporation
Session Code: UNC309
Agenda
High-level product direction for scale
Guidelines and ratios
Role specific details
Virtualization considerations
Toolkit for planning and sizing
Scale Out vs. Scale Up
Scale out is a strategic choice made by the product
group
Scale out provides the following at low cost:
Large mailboxes
High availability
Rich feature set
Scaling up increases risk that an outage or failure
affects more users
Scaling up usually costs more, and can force feature
decisions due to hardware choices
Consider all factors in the equation, particularly storage
Scale Up Options
Multiple Role Servers (“brick” deployments)
Likely the best option for big hardware (> 2 socket) – best
hardware utilization overall
Be aware of recommendations for max processor & memory
Virtualization
Evaluate whether potential added complexity & monitoring
challenges make this a win
Single role
Product not engineered for single role high scale (> 2 socket)
Extreme caution necessary – validate carefully in a test lab
Supported vs. Recommended
Supported usually means well tested
Support statements define strict boundaries
Recommendations define the “best case” or the
state that we want our customers to achieve
Understand risks of going outside of
recommendations or support boundaries
Processor Core Scalability
Single Role Servers
Recommend a 2-socket platform
4-core processors = 8 total cores
6-core processors = 12 total cores
Expect diminishing returns moving to 16+ cores on >= 4 socket platform
Known issues updating memory across cores
Not Non-Uniform Memory Access (NUMA)-aware or optimized for scale
around data locality
Code can take longer to execute; transaction costs rise
Multiple Role Servers
Recommend 24 cores maximum for high-scale “Enterprise Multiple Role
Server”
Multiple processes from different roles help us scale better
Hyperthreading
Disable on production Exchange servers
Causes monitoring and capacity planning challenges
Role Ratio Guidelines
Processor core ratios
CAS : Mailbox
=3:4
HUB : Mailbox
= 1 : 7 (no A/V on Hub)
= 1 : 5 (with A/V Hub)
GC : Mailbox
= 1 : 4 (32-bit GC)
= 1 : 8 (64-bit GC)
Processor and Memory Config
Recommended Configuration
Role
Maximum Processor Cores
Optimal Memory
Hub & Edge Transport
12 cores
1GB per core
(or 8GB minimum)
Client Access Server
12 cores
2GB per core
(or 8GB minimum)
Mailbox
12 cores
4GB plus 3-30MB per mailbox
Unified Messaging
12 cores
2GB per core
(or 4GB minimum)
Multiple Role Server
24 cores
8GB plus 3-30MB per mailbox
Network Load Balancing
Exchange 2010 requires load balanced CAS for
internal connections
Consider HA needs
Size for connection count spikes
Windows Network Load Balancing (NLB)
Not recommended above 8 nodes
Hardware Load Balancer
Recommended for larger environments
Multiple Role Server High Availability (HA)
scenarios
Mailbox Role
I/O reduced by 70% from Exchange Server 2007
Improved performance for SATA (Tier 2 class) disks
Two socket platform still optimal
Storage performance improvements prioritized over processor scale
improvements – larger TCO advantage
High availability improvements affect sizing
Sizing must account for failure scenarios
Use 4 – 12 total cores for Mailbox
16 core not expected to scale well but ok to deploy – consider TCO
4GB RAM w/3-30MB per mailbox recommended depending on profile
Size and prepare disks correctly
Use Exchange Storage Calculator
Mailbox Role
Mailbox Role Cache Memory Sizing
Design servers with large
quantities of memory
Deep checkpoint depth +
32KB pages allow E2010 to
benefit from larger
memory configurations
than E2K7
More database cache
results in less IOPS/mailbox
Messages Sent+Received
per mailbox per day
(~75KB average message size)
50
100
150
200
250
300
350
400
450
500
Database cache per
mailbox (MB)
3
6
9
12
15
18
21
24
27
30
Mailbox Role
Size for active users on DAG nodes, assuming the possibility of double
failures
Do not overcommit resources
Spread node failure across all available nodes not one or two
Distribute database (DB) copies across nodes in a matrix
Improved DB seed/log shipping performance across WAN
Log Shipping compression/encryption (opt in)
New log shipping architecture (Transport Control Protocol (TCP) socket based as
opposed to Server Message Block (SMB), connection/DB)
Improved high latency capability
Use multiple 1GB networks or 10GB network
Improves LAN re-seed/log replication queue drain performance
Especially with large servers and/or large databases
Client Access Server Role
Connection scalability changes
64K connections / MBX server
Outlook Clients
MBX
Exchange 2007
1 connection :
1 CAS session :
(1
connection
==
1 session,
64K RPC Context handle limit)
1 client session
1 client
session
1 MBX session :
1 client session
100 shared connections
Outlook Clients
Exchange CAS Array
Exchange 2010
(1 connections != 1 session, 250K RPC Context handle limit on MBX)
MBX
Client Access Server Role
Hardware requirements have increased vs. Exchange
2007
“Pay to play” for additional features and services (RPC Client
Access Service, Address Book Service, Remote Powershell,
etc.)
Possible to keep CAS count constant from 2007 to 2010,
with hardware refresh
Use 4 to 12 cores
Recommend larger of 8GB RAM or 2 GB RAM/core
CAS : Mailbox = 3 : 4 Cores
Hub Transport Role
Increased workload in Exchange 2010
Additional CPU required when compared to Exchange 2007
Not significant enough to result in a core ratio change
Use 4-12 cores
4-8 GB of RAM recommended
More than 8GB is not shown to improve TCO or scale
Use battery-backed write cache disk controller
Disk I/O can be a bottleneck on an un-tuned Hub
Log I/O becomes virtually free with a BBWC controller
Hub Transport Role
ESE changes:
ESE page size increased from 8KB to 32KB
ESE database page compression
Intrinsic long value record storage
ESE version store maintenance
DB cache size increased from 128MB to 1GB
Checkpoint depth increased from 20MB to 512MB
Logging buffer size increased from 512KB to 5MB
With transport dumpster changes and ESE improvements, transport IOPS
requirements have been reduced by more than 50%
Unified Messaging Role
Use 8 core for Voice Mail Preview
CPU-intensive workload
4 core recommended for other scenarios
4-8 GB of RAM recommended
More than 8GB is not shown to improve TCO or scale
Not recommended combining with other roles
Audio quality can be affected
Ensure low latency to mailbox servers associated with
UM-enabled accounts
Multiple Role Server
Mailbox, CAS, and Hub Transport roles only
Available solution for high core configurations
Half of cores for Mailbox, half for CAS+Hub
Use 8-24 cores
8GB RAM plus 3-30MB/mailbox recommended (follow
mailbox database cache sizing guidance)
Multiple Role Server
Simple unit of scale (brick) model
Each multi-role server represents a building block
Servers with on-board SATA storage (10-16 disks)
are optimal
Small organization/branch office – server consolidation
Minimize the number of physical servers, operating system
instances, and Exchange server instances to manage
Risk mitigation scenarios
Policies that limit the amount of mailboxes per server
Support Guidelines
TechNet is the single source:
http://technet.microsoft.com/en-us/library/cc794548.aspx
SVVP Support Policy Wizard is a great tool:
http://www.windowsservercatalog.com/svvp.aspx?svvppage=svvpwizard.htm
Always confirm SPW results with our TechNet
article
Check back for updates
Clarifications published frequently
Supportability Quick Reference
Supported
Root: Hyper-V or SVVP
Guest:
Exchange 2010
Windows 2008 SP2 or R2
Mailbox, Client Access, Hub Transport, Edge roles
Meets basic Exchange system requirements
Storage is fixed VHD, SCSI pass through, or iSCSI
Not Supported
Combination of Exchange Mailbox HA and hypervisor-based clustering or
migration technologies
Snapshots, differencing/delta disks
VSS backup of root for passthrough disks or iSCSI disks connected to initiator in
guest
Unified Messaging role
Virtual/logical proc ratio greater than 2:1
Applications running in root partition
Deployment Recommendations
Virtualization isn’t free
Hypervisor adds processor overhead, must account for this
when sizing - ~12% in our Exchange 2010 tests
Workload costs rise as well, though this is more difficult to
characterize
Virtualization doesn’t change Exchange design
requirements from an application perspective
Design for Performance, Reliability and Capacity
(MBX/Hub/Edge)
Design for Usage Profiles (CAS/MBX)
Design for Message Profiles (Hub/Edge)
Exchange 2010 Testing
Goal: Examine Exchange performance on Hyper-V in a typical deployment
scenario
Test configuration:
HP ProLiant BL680 G5, 4 x Quad-Core Intel Xeon E7340
Root: 16 core host, Windows 2008 R2 (build 7100)
Guests: 4 VMs (1 CAS, 1 Hub, 2 Mailbox), Exchange 2010 DF7 (582.10)
Mailbox 1 on Windows 2008 RTM, Mailbox 2 on Windows 2008 R2
4,000 users per mailbox server
Loadgen, 75% Outlook 2007 Cached Heavy + 25% OWA (modified enterprise script) +
10% default EAS workload
Observations:
Logical processor guest runtime higher with 2008 RTM guest vs. 2008 R2 (~13%)
Acceptable performance across all roles
Hub CPU 52.3%, CAS CPU 33.4%
MBX CPU 53.3%, RPC Averaged Latency 6.5ms, RPC Operations/sec 1818
Points To Consider
Accuracy of Perfmon counters in a Guest OS might be a
concern for monitoring
CPU cycles in a VM are relative to the CPU slices provided
from the virtualization layer
May skew results
Investigating the impact on production monitoring
Comprehensive comparison of physical resources and
application consumption is difficult to achieve
Application counters are only available in the Guest OS
Root OS only provides view of resources it owns and HyperV counters
Capacity Planning Tools
Profiling
Exchange Profile Analyzer 2010 (EPA)
Performance Monitor (Perfmon)
Sizing
Exchange Server 2010 Storage Calculator
Validation
Jetstress 2010
Exchange Load Generator 2010 “Loadgen”
Exchange Profile Analyzer 2010
Generates statistical profile of user actions
Messages sent and received/day
Rule counts
Item size and counts
Inputs
Crawls mailboxes with MAPI (previously DAV)
OWA log analysis tool and “summarizer”
Accuracy somewhat dependent on how users manage their
mailbox
Availability planned for Q3CY10
Version that works with Exchange 2003 & 2007 available here:
http://tinyurl.com/233by6
Storage Calculator 2010
Follows Product Group recommendations on:
Storage
Memory
Mailbox sizing
Goal of the calculator is to output:
I/O requirements
Capacity requirements
Logical user number (LUN) design
Available today via the Exchange team blog:
http://msexchangeteam.com/
Jetstress 2010
Exchange I/O simulator
Uses Jet (ESE) database engine
Analyzes server I/O performance for Exchange requirements
What can Jetstress be used for?
Storage performance validation
Storage reliability testing
End-to-end testing of storage components
What can’t Jetstress be used for?
Validation of client experience
Integration testing with third party software solutions
Availability of 2010 version planned for December 2009, will be
announced on Exchange team blog:
http://msexchangeteam.com/
Jetstress 2010
Updated with Exchange 2010 Mailbox I/O Profile
This profile is not yet final and is subject to change between now and
Exchange 2010 release
Database duplication is now multi-cast
Dramatically reduces the time to prepare databases for testing
Now using MSExchange Database I/O counters for I/O
measurement
Allows placing databases and logs on the same volume
Log replication I/O is simulated based on Exchange 2010 HA
architecture
Background Database Maintenance (Checksum) is now
simulated
Exchange Load Generator 2010
The only supported multi-protocol load generator for Exchange
Replaces Loadsim and ESP
Windows UI interface as well as a command-line interface
Both task-based and scripted simulation modes
Consumed both internally at Microsoft and externally
Existing modules include: Outlook 2003/2007 (online and
cached), Post Office Protocol (POP), Internet Message Access
Protocol (IMAP), Simple Mail Transfer Protocol (SMTP), OWA,
ActiveSync… others in development
Availability planned for December 2009, use beta until then:
http://tinyurl.com/yhvpwbf (32-bit)
http://tinyurl.com/yk5vfl7 (64-bit)
Exchange Load Generator 2010
Requires Vista, Windows 7 or Windows 2008 OS
(SP2/R2)
No longer requires Exchange Management Tools
ActiveSync Module
Dynamic mail generator
No need for message files, available in 5 languages,
supports attachments
NSPI connections
Tools Process Flow
User
Profile
User
Profile
Key Takeaways
Exchange continues to reduce I/O requirements,
reducing overall system TCO
New features in Exchange 2010 may require
additional hardware resources, server count
increases can be minimized
Virtualization is a great way to take advantage
of underutilized hardware
Take advantage of the planning & testing toolset
for successful deployments
Related Content
UNC301 – Microsoft Exchange Server 2010 Upgrade and Coexistence with Exchange
Server 2007 and 2003
UNC307 – Microsoft Exchange Server 2010 High Availability
UNC314 – Microsoft Exchange Server 2010 Storage Architecture
UNC315 – Microsoft Exchange Server 2010 Unified Messaging
UNC01-IS – Microsoft Exchange Server 2010 Archiving Q&A
UNC06-IS – Site Resilience in Microsoft Exchange Server 2010
UNC03-IS – Microsoft Exchange Server Virtualisation: Does It Make Sense?
UNC Track Call to Action!
Learn More!
Related Content at TechEd on “Related Content” Slide
Attend in-person or consume post-event at TechEd Online
Check out learning/training resources at Microsoft TechNet
Exchange Server and Office Communications Server
Check out Exchange Server 2010 at
Virtual Launch Experience (VLE) at thenewefficiency.com
Try It Out!
Download the Exchange Server 2010 Trial
Take a simple Web-based test drive of UC solutions through
the 60-Day Virtual Experience
Complete an evaluation
on CommNet and enter to
win an Xbox 360 Elite!
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should
not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.