Transcript Understanding and Interpreting SQL Server Performance Counters

SQL Server 2005
Performance Measurement
Improvements
Jeffry A. Schwartz
December 5, 2005
Demand Tech Symposium
CMG 2005
Emphasis of Presentation
SQL Server 2005
• Improves performance
• Introduces several long-awaited features
- Overhauled DTS called SSIS
- Overhauled security
- Reporting Services on all platforms
- CLR usage from within SQL
- Horizontal partitioning
Large amount of press about these features
• Dramatic performance measurement enhancements have not been
publicized as widely
Presentation discusses measurement enhancements
2
Overview
SQL Server 2000 provided
• Many PerfMon objects and counters
• Access to some system tables
• A few table-valued functions
SQL Server 2005 provides
• Approximately twice the PerfMon objects and counters
• Numerous views that provide very useful performance
information
• Views replace system table access in many cases
• Some new and useful SQL trace events
3
PerfMon Counter Hierarchy
Three-level hierarchy
Objects at top level
Counters
• Comprise bottom level
• Always pertain to a particular object
Instance level added between object and counter
levels when necessary
Processor object  Processor 0 instance  %
Processor Time counter
Memory object  Page writes/sec counter
4
SQL 2000 Server Objects
One set per SQL Server instance
Each set divided into 17 categories
4 memory-related
2 lock-related
7 measure database backup, replication, and user
settable categories
• Applicable to database backup and replication performance,
as well as specifically defined and maintained user counters
Rest involve database transactions, log handling,
and database access activities
5
SQL 2005 Server Objects
Adds 18 categories
• 29 total for SQL alone excluding SSIS, Agent, .Net
Many more useful metrics
•
•
•
•
•
•
•
6
Tempdb
Wait statistics
Common Language Run-time (CLR) (1)
SSIS (replacement for DTS) (2)
Transactions
SQL Agent (4)
.Net Data Provider for SQL Server
SQL Server Object List With 2005
Changes
SQL Server Objects
7
Category
SQL Server: Access Methods
Database access
SQL Server: Backup Device
Database backup
SQL Server: Buffer Manager
Memory management
SQL Server: Buffer Partition
Memory management
SQL Server: Cache Manager
(deleted on 2005)
Memory management
SQL Server: Databases
Transactions & log handling
SQL Server: General Statistics
User connections (Temp Tables, SOAP added on
2005)
SQL Server: Latches
Locking (SuperLatches added on 2005)
SQL Server: Locks
Locking (AllocUnit, Application,
Metadata, Object added on 2005)
SQL Server: Memory Manager
Memory management
SQL Server: SQL Statistics
SQL command activities
File,
HoBT,
SQL Server 2005 Object List
SQL Server 2005 Objects
Category
SQL Agent: Alerts
Database alerts
SQL Agent: Jobs
Jobs
SQL Agent: JobSteps
Jobs
SQL Agent: Statistics
SQL Server Restarts
SQL Server: Broker Activation
Broker
SQL Server: Broker Statistics
Broker
SQL Server: Broker/DBM Transport
Broker
SQL Server: Buffer Node
Memory management
SQL Server: Catalog Metadata
Memory management (augments 2000
Cache Manager)
SQL Server: CLR
.Net Common Language Runtime Usage
SQL Server: Cursor Manager by Type Cursors
SQL Server: Cursor Manager Totals
8
Cursors
SQL Server 2005 Object List (contd.)
SQL Server 2005 Objects
9
Category
SQL Server: Database Mirroring
Replication
SQL Server: Exec Statistics
Queries (distributed, DTC, extended proc,
OLEDB)
SQL Server: Plan Cache
Queries & Memory management (replaces
2000 Cache Manager)
SQL Server: SQL Errors
Errors
SQL Server: SSIS Service
SSIS (previously DTS)
SQL Server: Transactions
Transactions
SQL Server: Wait Statistics
Wait Information for locks (network I/O, log,
page, worker)
Lockable Resources
Resource
10
Description
AllocUnit (2005)
Allocation unit
Application (2005)
Application-specified resource
Database
Entire database
Extent
Contiguous group of 8 data or index pages
File (2005)
Database file
HoBT (2005)
Heap or B-Tree index
Key
Row lock within an index that protects range of
keys in serializable transactions
Metadata (2005)
Meta data or catalog information
Object (2005)
Job flow synchronization object, table, stored
procedure, or view
Page
8-kilobyte (KB) data or index page
Lockable Resources (contd.)
Resource
11
Description
RID
Row ID. Used to lock a single row within a table
Table (Appears to
be replaced by
Object.)
Entire table, including all data and indices
Locks Object
One of the most important objects
Number of Deadlocks/sec critical
• SQL Profiler/Trace can provide information about how
deadlock was created
- Deadlock graph event (148) can provide additional information
Lock Timeouts/sec also critical
• Monitors each type of lock
• # of lock requests that exceed maximum specified wait time
- Includes internal requests for NOWAIT locks
- Cannot determine number that result in actual delays
12
Lock Counters and Trace Events
New Lock Timeouts/sec > 0 available on SQL
2005
• Does not include NOWAIT timeouts
New trace events available
• Lock:Timeout (timeout > 0)
- # 189
- Use in place of old lock timeout event # 27
• Blocked Process Report also new
- # 137
- Provides information about how and why a process was
blocked
13
Wait Statistics Object
Much of this information was previously
available only via
• SQL trace
• fn_virtualfilestats
Small overlap with Locks object
Useful instances
• Average wait time (ms)
• Cumulative wait time (ms)
• Waits in progress
• Waits started per second
14
Wait Statistics Object
Cumulative wait time (ms) instance extremely
useful
Unfortunately, still only counts values within the
last second
Use sys.dm_os_wait_stats to determine specific
object
15
Wait Statistics Object
Summarizes very useful counters including
• Locks waits
• Log buffer waits
• Log write waits
• Memory grant queue waits
• Network IO waits
• Non-Page latch waits
• Page IO latch waits
• Page latch waits
• Thread-safe memory object waits
• Transaction ownership waits
16
Useful General Views and Functions
Many were available under 2000 with slightly
different names
• 2000 sysobjects -> 2005 sys.objects
sys.configurations
• Provides information regarding OS and SQL Server
configurations
sys.databases
• Lists all databases and their IDs
sys.filegroups
• Lists all file groups and their IDs so proper associations can
be made
17
Database-Specific Views
sys.database_files
• Lists all physical database files and their IDs
• Database-specific, query for each one separately
sys.objects
• Lists all database objects such as tables, views, stored
procedures, etc.
• Database-specific, query for each one separately
Sysindexes
• 2000 name because 2005 layout is very different
• Lists all indices and their associated tables
• Database-specific, query for each one separately
18
Dynamic Management Views and
Functions
Provide significant amount of information regarding
• System
• Databases
• Internal workings of SQL Server 2005
• Performance
Some were possible with complex queries on 2000
• Most were very clumsy to use and required temp tables
• Most required significant understanding of the underlying tables
Greatly simplified on 2005
Some obviate need for using SQL Trace
Many can be reset using command similar to
• DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR)
19
Dynamic Management Views and
Functions Category List
View Category
20
View Category
Common Language Runtime Related
Dynamic Management Views
I/O Related Dynamic Management Views
and Functions
Database Mirroring Related Dynamic
Management Views
Query Notifications Related Dynamic
Management Views
Database Related Dynamic Management
Views
Replication Related Dynamic
Management Views
Execution Related Dynamic Management
Views and Functions
Service Broker Related Dynamic
Management Views
Full-Text Search Related Dynamic
Management Views
SQL Operating System Related Dynamic
Management Views
Index Related Dynamic Management
Views and Functions
Transaction Related Dynamic
Management Views and Functions
I/O Related Dynamic Management Views
and Functions
Most useful are highlighted in red
sys.dm_io_backup_tapes
sys.dm_io_cluster_shared_drives
sys.dm_io_pending_io_requests
• Only retrieves active entries
• Help documentation is wrong!!!! Field ordering is
incorrect
sys.dm_io_virtual_file_stats
21
sys.dm_io_virtual_file_stats
Table-valued function
Returns I/O statistics for data and log files
• ::fn_virtualfilestats(-1,-1) on 2000
• 2005 calling method for all databases and files
- sys.dm_io_virtual_file_stats(NULL, NULL)
ALL field values are since SQL started
• Perfect for periodic sampling because values are
cumulative
Sample command: select * from
sys.dm_io_virtual_file_stats(NULL, NULL)
22
sys.dm_io_virtual_file_stats – Field
Summary
ID of database
ID of file
Number of milliseconds since SQL Server instance started
Number of reads and writes issued against the file
Total number of bytes read from and written to this file
Total time, in milliseconds, users waited for reads and
writes to complete
Total time, in milliseconds, users waited for I/O completions
Number of disk bytes used by this file
23
Database-Related Dynamic Management
Views
Return information regarding space usage
by various entities
• sys.dm_db_file_space_usage
• sys.dm_db_session_space_usage
• sys.dm_db_partition_stats
• sys.dm_db_task_space_usage
24
Execution-Related Dynamic Management
Views and Functions
sys.dm_exec_background_job_queue
sys.dm_exec_background_job_queue_stats
sys.dm_exec_cached_plans
sys.dm_exec_connections
sys.dm_exec_cursors
sys.dm_exec_plan_attributes
sys.dm_exec_query_optimizer_info
sys.dm_exec_query_plan
sys.dm_exec_query_stats
sys.dm_exec_requests
sys.dm_exec_sessions
sys.dm_exec_sql_text
25
sys.dm_exec_query_stats
Returns aggregate performance statistics for cached query
plans
View contains one row per query plan
• Lifetime of a row is tied to the plan itself
Provides summarized query information for all active
queries
• Top <n> can be returned for any category
- Execution count, CPU time, CLR time, and elapsed time
- Physical and logical reads
- Logical writes
Can minimize necessity for SQL statement tracing
No parameters required
26
sys.dm_exec_query_stats Fields – Part 1
“Starting and ending positions of the query that
the row describes within the text of its batch or
persisted object”
Pointer to the plan, which can be passed to the
dm_exec_query_plan dynamic management
function
Times at which plan was compiled and last
executed
# of times plan executed since last compilation
27
sys.dm_exec_query_stats Fields – Part 2
Total, minimum, maximum, and last amounts of
• CPU time, in microseconds, consumed by executions since
compilation
• Physical and logical reads
• Logical writes
• CLR times
• Elapsed times
28
sys.dm_exec_query_stats – Example #1
Returns top 20 query statements that executed
most logical reads
• Select top 20 *, (SELECT SUBSTRING(text,
statement_start_offset/2, (CASE WHEN
statement_end_offset = -1 THEN
LEN(CONVERT(nvarchar(max), text)) * 2 ELSE
statement_end_offset END - statement_start_offset)/2)
FROM sys.dm_exec_sql_text(sql_handle)) AS query_text
FROM sys.dm_exec_query_stats order by
total_logical_reads desc
29
sys.dm_exec_query_stats – Example #2
Returns batches that contain top 20 query statements that
executed most logical reads
• CacheObjects.sql, EQS.creation_time, EQS.execution_count,
EQS.last_execution_time, EQS.total_worker_time,
EQS.total_physical_reads, EQS.last_physical_reads,
EQS.max_physical_reads, EQS.total_logical_reads,
EQS.last_logical_reads, EQS.max_logical_reads, EQS.total_logical_writes,
EQS.last_logical_writes, EQS.max_logical_writes, EQS.total_clr_time,
EQS.last_clr_time, EQS.max_clr_time, EQS.total_elapsed_time,
EQS.last_elapsed_time, EQS.max_elapsed_time from
sys.dm_exec_query_stats EQS, sys.dm_exec_cached_plans
ExecCachedPlans, sys.syscacheobjects CacheObjects where
CacheObjects.bucketid = ExecCachedPlans.bucketid and
ExecCachedPlans.plan_handle = EQS.plan_handle order by
total_logical_reads desc
Example
• Displays actual field names
• Returns much more information
Note how other views are used as well as system tables
30
Other Execution-Related Views
These views enable analysis by session and
connection
• sys.dm_exec_sessions
• sys.dm_exec_connections
sys.dm_exec_background_job_queue_stats
sys.dm_exec_cursors
31
SQL Server Operating System-Related
Dynamic Management Views – Part 1
sys.dm_os_buffer_descriptors
sys.dm_os_memory_pools
sys.dm_os_child_instances
sys.dm_os_performance_counters
sys.dm_os_cluster_nodes
sys.dm_os_schedulers
sys.dm_os_hosts
sys.dm_os_stacks
sys.dm_os_latch_stats
sys.dm_os_sys_info
sys.dm_os_loaded_modules
32
SQL Server Operating System-Related
Dynamic Management Views – Part 2
sys.dm_os_tasks
sys.dm_os_memory_cache_clock_hands
sys.dm_os_threads
sys.dm_os_memory_cache_counters
sys.dm_os_virtual_address_dump
sys.dm_os_memory_cache_entries
sys.dm_os_wait_stats
sys.dm_os_memory_cache_hash_tables
sys.dm_os_waiting_tasks
sys.dm_os_memory_clerks
sys.dm_os_workers
sys.dm_os_memory_objects
33
sys.dm_os_latch_stats
SQL Server uses numerous kinds of
latches
No parameters required
Latch times can be helpful in verifying I/O
subsystem performance
• Large latch wait times often indicate poor I/O
performance
View returns information regarding all
latch types
34
sys.dm_os_sys_info Fields – Part 1
No parameters required
Number of milliseconds since computer was started
Number of logical CPUs on the system
Number of CPU ticks in milliseconds
Ratio of the number of logical and physical processors
Amount of physical memory available
Amount of virtual memory available to the process in user
mode. This can be used to determine whether SQL Server
was started by using a 3-GB switch
Total number of buffers with pages that have associated
memory. This does not include virtual memory
Number of pages committed by the buffer pool to memory
35
sys.dm_os_sys_info Fields – Part 2
Total number of visible buffers in buffer pool
Size of the call stack for each thread created by SQL Server
Quantum for a non-preemptive task, measured in CPU ticks
Quantum (in seconds) = os_quantum / CPU clock speed
Error mode for the SQL Server process
Priority class for the SQL Server process
Maximum number of workers that can be created
Number of user schedulers configured in the SQL Server
process
Total number of schedulers in SQL Server
36
sys.dm_os_wait_stats
No parameters required
Returns information about waits encountered by
threads in execution
• 109 possible wait types
• 3 categories
- Resource
- Queue
- External
Useful for diagnosing performance issues with
• SQL Server
• Specific queries and batches
37
sys.dm_os_waiting_tasks
No parameters required
Reports information about currently
waiting tasks
Important Fields
• Total wait time (ms) for wait type
• Name of wait type
• Task currently holding this resource
• Blocking task session ID
• Description of resource being consumed
38
Index Related Dynamic Management
Views and Functions
sys.dm_db_index_physical_stats
• Execute infrequently because physical files are
interrogated
sys.dm_db_index_usage_stats
sys.dm_db_index_operational_stats
39
sys.dm_db_index_physical_stats
Five parameters required
• { database_id | NULL } (use db_id() for current db)
• { object_id | NULL }
• { index_id | NULL | 0 }
• { partition_number | NULL }
• { mode | NULL | DEFAULT }
LIMITED, SAMPLED, and DETAILED modes
If index or heap has fewer than 10,000 pages,
DETAILED mode is used instead of SAMPLED
Strict locking scheme not used for LIMITED or
SAMPLED
40
sys.dm_db_index_physical_stats
Returned Data
Index_type_desc
• Index type
Index_id, index_level
• 0  heap
Index_depth
• # of index levels
Avg_fragmentation_in_percent
• Logical fragmentation for indexes or extent fragmentation for heaps
• Indices with values > 30 are candidates for rebuild/reorganization
Fragment_count
• Number of fragments in the leaf level
Avg_fragment_size_in_pages
• Average number of pages per fragment in the leaf level
• Larger is better
41
sys.dm_db_index_usage_stats
Helpful for determining which indices actually
used
No parameters
Returns
• Number of seeks, scans, lookups, and updates for user and
system queries
• Time of last seek, scan, lookup, and update for user and
system queries
System queries
• Maintenance, e.g., statistics updates
User queries
• Insert, update, or delete operations
42
sys.dm_db_index_operational_stats
Helpful for determining how indices are used and
identifying contention areas
Four parameters required
• { database_id | NULL } (use db_id() for current db)
• { object_id | NULL }
• { index_id | NULL | 0 }
• { partition_number | NULL }
43
sys.dm_db_index_operational_stats
– Fields Part 1
Cumulative counts
• Leaf-level insert, delete, update, delayed delete
operations
• Above leaf-level insert, delete, update, delayed
delete operations
• Zeroes  heap operations
• Leaf-level page allocations in the index or heap
- For an index, page allocation corresponds to a page split
• Page allocations caused by page splits above leaf
level
44
sys.dm_db_index_operational_stats
– Fields Part 2
Cumulative counts
• Range and table scans started on index or heap
• Single row retrievals from index or heap
• Rows that were fetched through forwarding record
• Row locks requested
• Page locks requested
Cumulative counts and elapsed times database
engine
• Waited on row lock
• Waited on page lock
• Waited because of latch contention
• Waited on I/O page latch
45
sys.dm_db_index_operational_stats
To analyze a common access pattern to the table
or index partition
• leaf_insert_count
• leaf_delete_count
• leaf_update_count
• leaf_ghost_count
• range_scan_count
• singleton_lookup_count
46
sys.dm_db_index_operational_stats
To identify latching and locking contention
• page_latch_wait_count and page_latch_wait_in_ms
- These columns indicate whether there is latch contention on the index
or heap and the significance of the contention
• row_lock_count and page_lock_count
- These columns indicate how many times the Database Engine tried to
acquire row and page locks
• row_lock_wait_in_ms and page_lock_wait_in_ms
- These columns indicate whether there is lock contention on the index or
heap, and the significance of the contention
To analyze statistics of physical I/Os on an index or heap
partition
• page_io_latch_wait_count and page_io_latch_wait_in_ms
- These columns indicate whether physical I/Os were issued to bring the
index or heap pages into memory and how many I/Os were issued
47
Transaction Related Dynamic
Management Views and Functions
sys.dm_tran_active_snapshot_database_transactions
sys.dm_tran_active_transactions
sys.dm_tran_current_snapshot
sys.dm_tran_current_transaction
sys.dm_tran_database_transactions
sys.dm_tran_locks
sys.dm_tran_session_transactions
sys.dm_tran_top_version_generators
sys.dm_tran_transactions_snapshot
sys.dm_tran_version_store
48
sys.dm_tran_locks
Each row represents currently active request to
lock manager for lock that has been granted or is
waiting to be granted
Result set columns are divided into two main
groups — resource and request
• Resource group describes resource on which the lock
request is being made
- Resource type same as lockable resource
• Request group describes the lock request
49
sys.dm_tran_locks – Fields
request_status
• GRANT, CONVERT, or WAIT
request_owner_type
• TRANSACTION, CURSOR, SESSION,
SHARED_TRANSACTION_WORKSPACE, or
EXCLUSIVE_TRANSACTION_WORKSPACE
50
sys.dm_tran_locks – Example
Select * from sys.dm_tran_locks where
request_status <> ‘GRANT‘
Greatly reduces collected data because most
values appear to be GRANT
51
Conclusions
SQL Server 2005 provides a wealth of new
performance information
Additional PerfMon objects and counters provide
much-needed insight into bottleneck
identification
The newly added and numerous Dynamic Views
will be essential for uncovering and resolving
performance bottlenecks
The Dynamic Views are
• Easy to use
• Well documented
• Low overhead with very few exceptions
52
Additional SQL Server Performance Info
Understanding and Interpreting SQL Server
Performance Counters session
• Thursday, December 8 @ 8:00 AM
53
References
Kalen Delaney, Inside Microsoft SQL Server 2000
Robin Schumacher, High Performance SQL
Server DBA
Microsoft SQL Server 2000 Books Online
Microsoft SQL Server 2005 Books Online
Edward Whalen, Marcilina Garcia, Steve DeLuca,
and Dean Thompson, Microsoft SQL Server 2000
Performance Tuning
54