Transcript System Performance Assessment Tools for Windows Longhorn Richard G. Russell Development Manager Windows Core Operating System Division rgr @ microsoft.com Microsoft Corporation.

System Performance Assessment Tools
for Windows Longhorn
Richard G. Russell
Development Manager
Windows Core Operating System Division
rgr @ microsoft.com
Microsoft Corporation
Introduction
Large difference in platform capability moving forward
High-end dual core 64-bit gaming system vs. low end value
single core system
Media center mobile system vs. thin and light or
ultra-mobile system
There are Microsoft Windows codenamed “Longhorn”
client features that need to scale with platform capability:
The window manager and themes
Video playback
Gaming
TV Recording
How does Longhorn and applications make
scaling decisions?
Situation Today
Scaling decisions are left to applications
Microsoft Windows XP makes no material scaling
decisions
Windows XP provides few tools to help
applications make decisions based on platform
capability
Graphics capability information from D3D is useful
for games
Application can enumerate features via WMI
Only a few applications have their own system
assessment code
Games are the exception
But, many games do a poor job of this – often leaving tunable settings up
to the user
Longhorn Helps Solve These Problems
New Tool: the Windows System Assessment Tool
(WinSAT)
Built into Longhorn
WinSAT assesses system
Features* – something that can be enumerated
or detected
Attributes* – something that must be measured
Capabilities* – a fundamental ability to perform
a task or action
WinSAT provides performance metrics and other
information necessary for Longhorn and
applications to make scaling decisions
* See detailed definitions at the end of the deck
WinSAT Helps Answer Scaling Questions
Can the system efficiently run with desktop
composition enabled?
Can the system efficiently support advanced
window management and themes?
Aero-Express or Aero-Glass
Can the system play high definition video without
dropping frames?
How many channels of TV can a system record
simultaneously?
How many enemy robot AI agents should
a game configure?
Can a game enable full water simulation?
Better Automatic Configuration is the Goal
Thus avoiding:
Making the user experiment or guess at the
best settings
The use of default “lowest common denominator”
settings
Assuming high-end systems – turning on too many
features that will run poorly on some systems
Developers avoid having to develop, test, deploy,
and maintain custom system assessment code
Allow the operating system and applications
to better automatically configure themselves
to provide the best user experience
of which an individual PC is capable
What Does WinSAT Do?
WinSAT assesses these subsystems:
Graphics
Memory
Processor
Storage
WinSAT assess:
Attributes
Features
Capabilities
Only items that are relevant to Longhorn and
application scalability are assessed
WinSAT and Assessment Details
WinSAT Component Diagram
Group
Policies
WinSAT API
Features
Video
Compute
Storage
Memory
D3D
Graphics
WinSAT.EXE
(Command line tool)
XML
Data
Store
WMI
WinSAT.exe
WinSAT.exe is a command line tool that contains the
individual assessments
Graphics, D3D, Memory, Storage, Computation
Video Playback, Feature Enumerator* (TBI)
Easy to run
Lots of parameters for each assessment
Command line help built into Windows* (TBI)
Must be run as administrator from the command line
Canonical output is XML
Human readable output to standard output
Easy to script
* To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution
WinSAT.exe
Full 32-bit and 64-bit parity
Multi-Core/Multi-CPU aware where appropriate
Memory assessment
Computational assessment
Runs on Windows XP (just copy WinSAT.exe)
WinSAT will be updated over time to reflect
changes and improvements in platform
technology
WinSAT API
Provided via a simple COM interface
Provides simple and easy programmatic access
to WinSAT and the data store
Access to the XML data – returns an MSXML DOM
Allows WinSAT.exe to be run
With an arbitrary command line
With a “formal assessment”* (TBI)
From a limited user account* (TBI)
A formal assessment is set of assessments that is executed with
a pre-defined set of parameters
Used to generate metrics used by Longhorn and reported by the
WinSAT API
The API can be disabled through group policies
Off, or administrator only
* To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution
The Data Store
Will only store data from formal assessments
scheduled from the API
Data from assessments run from user command lines are not managed
or accessible through the API
Such data is saved in normal XML files
The XML schemes are very simple
The Schemas will change!
A history of formal assessments is kept* (TBI)
up to 100 assessments (oldest deleted)
Always keeps the first one generated during first user logon
The data store files are read-only to all entities
except the WinSAT API* (TBI)
Data is provided to callers by providing an
MSXML DOM
* To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution
The Graphics Assessment
Designed to assess a system’s ability to
efficiently run a composited desktop and the
Aero themes
Metric is Effective Frames Per Second
Drives the hardware in a very similar way to the
desktop composition engine* (TBI)
Used to make decisions about enabling
desktop composition and theme level
Focused on evaluating shader texture load
performance, system to graphics memory
bandwidth and back-buffer read-back
* To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution
The D3D Assessment
Designed to assess a system’s ability to render
3D gaming graphics
Metric is Effective Frames Per Second
Targeted to Pixel Shader v2.0 or better hardware
Focused on three aspects:
Shader ALU performance
Shader texture load performance
Post-pixel blend performance
Evaluates 8-bit and 16-bit render targets
The System Memory Assessment
Focused on throughput, not latency
Deigned to assess
How well large objects can be moved in system
memory
Metric is Mega bytes per second
Methodology
Uses MMX or SSE registers for copies
Uses temporal reads
Uses non temporal writes
Uses explicit block pre-fetching (16K stride)
All data is aligned
The Storage Assessment
Is a “sweep” test – divides a physical disk into
regions and evaluates the following for each
Random read and write performance
Sequential read and write performance
Uses up to 16 regions
Metric is megabytes (MB) per second
Reports metrics for each region
Reports an aggregated metric for the disk using
the geometric mean of each region
The Processor Assessment
Designed to assess the system’s computational
ability
Metric is Mega bytes per second processed
Is not a synthetic test – uses native
Windows components
Data compression and decompression
Data encryption and decryption
Widows Media Video encoding and decoding* (TBI)
Values from each sub-assessment aggregated
using the geometric mean
* To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution
Windows XP vs. Longhorn Differences
Some assessments may produce moderately
different results on Longhorn vs. Windows XP
Graphics and D3D:
Due to changes in the driver model XPDM vs. LDDM
Due to desktop composition
Processor assessments:
Due to differences in the Windows components used
in those assessments
Demo
Scenarios
For the OEM
Focus on platform performance as it relates to
key Longhorn features
Measuring performance consistency of similarly
configured models
Understand performance differences between models
Diagnostics: misconfigured systems may
perform poorly
For example, the memory test is very good at picking
out misconfigured DRAM
Independent Software Vendor
ISV can use simple (or complex!) rules based on
WinSAT metrics and built into an application to
make scaling decisions
Since WinSAT is built into Longhorn, good
correlation data can be collected during internal
testing and external ALPHA and BETA testing
In short, ISVs can use WinSAT just as Microsoft
uses it for Longhorn and our applications
Media Player Example
User installs a new media player
User then tries to play a high definition video clip
The player checks the WinSAT data via the API
and determines that the system is not capable of
effectively playing HD video
Note that this happens without running any tests,
nor does the player need to guess – it happens
immediately
The player can then inform the user that they
may see frame drops and other video artifacts
– do they wish to continue?
Longhorn Upgrade
User upgrades an existing system to Longhorn
WinSAT runs as part of the first user logon
experience (this used to be called OOBE)
Longhorn uses WinSAT metrics to make
configuration decisions and recommendations
to the user
“Your system is capable of running Aero-Glass, would
you like this option configured?”
Longhorn Graphics Card Upgrade
User installs a new graphics card in their system
– upgrading from UMA graphics
LH detects this after the reboot and recommends
to the user that he or she should reassess the
system (the user says yes)
WinSAT runs and the system determines that it
is now possible to run Aero-Glass
The user is informed, much the same way as
the previous example
Wrapping Up
Where is the Development Process?
The WinSAT tool is in the WinHEC Longhorn build
It is in an early ALPHA state
Large correlation study is underway
Not all assessments and features are implemented
in Longhorn Beta 1
We may add further assessments
The metrics may change
Tuning and enhancements will happen through the
Beta cycles
We may make interim versions of WinSAT available
on the beta site
Key Features and Updates to be Implemented
Consumer suitable visuals for the Graphics and
D3D assessments
Windows Graphics Foundation (WGF) 2.0 support
Video decode assessment
System feature enumeration
Integration with:
The desktop composition system
First user logon process
LUA support
WMI integration
Detailed technical documentation
In Windows Help format
Group Policy Support
Call to Action
Become familiar with WinSAT
Correlate the performance of your product
with WinSAT metrics
Think about how WinSAT metrics can help your
application make scaling and configuration
decisions
Help us make WinSAT better
How does it rank systems and components?
Is it statistically reliable?
How will does it correlate with your product’s
performance?
Report bugs and problems
Community Resources
Windows Hardware & Driver Central (WHDC)
www.microsoft.com/whdc/default.mspx
Technical Communities
www.microsoft.com/communities/products/default.mspx
Non-Microsoft Community Sites
www.microsoft.com/communities/related/default.mspx
Microsoft Public Newsgroups
www.microsoft.com/communities/newsgroups
Technical Chats and Webcasts
www.microsoft.com/communities/chats/default.mspx
www.microsoft.com/webcasts
Microsoft Blogs
www.microsoft.com/communities/blogs
Additional Resources
See the WinSAT command line document on
the WinHEC CD
Please send your questions to Winsatfb @
microsoft.com
We will notify anyone who sends e-mail to this alias of
updates as they become available
© 2005 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
Key Definitions
An Attribute Is:
A system characteristic that is measured
Attributes cannot be predetermined accurately,
they must be measured as the system is built
WinSAT measures the performance attributes of:
System memory
The storage sub-system
The CPU (or CPUs)
The graphics sub-system
A Feature Is:
A prominent or distinctive system part,
component, or characteristic
Features are either present or absent
Features are not measured, they are detected
or enumerated
Examples:
x64, Hyper Threading, memory size, optical drive type
(DVD, CD, writer etc), MMX, SSE, 3DNow, 1394,
number of processors (logical and physical), number of
cores per processor package, cache size, hardware
MPEG-II or WMV decode support, shader 2.0 support
A Capability Is:
A system’s ability to effectively perform a specific
function or task
Capabilities are often present or absent but can
also be present to a matter of degree, or some
measure or metric
To be present, a capability requires:
A specified (or minimum) level of one or more attributes
The presence of one or more specified features
Some capabilities can be assessed directly
For example, the ability to play high definition video
Some must be inferred or determined by
correlation with features and measured attributes
© 2005 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.