Windows Vista Video Pipeline Architecture And Implementation Glenn Evans Technical Lead DMD/WAVE Microsoft Corporation Session Overview Video pipeline Overview New components for Windows Vista Windows Vista enhancements DirectX Video Acceleration 2.0
Download
Report
Transcript Windows Vista Video Pipeline Architecture And Implementation Glenn Evans Technical Lead DMD/WAVE Microsoft Corporation Session Overview Video pipeline Overview New components for Windows Vista Windows Vista enhancements DirectX Video Acceleration 2.0
Windows Vista Video
Pipeline Architecture
And Implementation
Glenn Evans
Technical Lead
DMD/WAVE
Microsoft Corporation
Session Overview
Video pipeline
Overview
New components for Windows Vista
Windows Vista enhancements
DirectX Video Acceleration 2.0 (DXVA2)
Software components
Display devices
Improvements
Performance and quality
Extensibility and integration
Future directions
Call to action
Video Pipeline Stages
Component
Capture
Processing
Encoding
Storage
Device
Decoding
Processing
Output
Controls
Display 1
Output
Controls
Display N
Desktop
New Windows Vista Components
Application and Pipeline (Media Foundation, DShow, …)
APIs/DDIs
Media
Sources
UVC
Decoders
WDDM
Capture
Transforms
MFT
Media
Sinks
D3D
Manager
EVR
DXVA2
AVStream
Decode
Video
Processing
D3D
DXGI
DDC/CI
OPM
DMM
Plugins
OS Component
App/Pipeline
Example Playback Pipeline
APIs/DDIs
Application
Plugins
Pipeline
GPU assisted
OS Component
SW Decoder
MFT
App/Pipeline
Media
Source
EVR
Decoder
MFT
DXVA2
Decode
Video
Processing
DWM
Xform
MFT
D3D
Manager
Desktop
D3D
DXGI
DDC/CI
OPM
DMM
DXVA2
What is it
DirectX Video Acceleration 2.0 (DXVA2.0)
APIs for decoding and processing
Separate from renderers
Enhanced enumeration, object creation
APIs for D3D device management
Many opportunities for IHVs to differentiate
Power optimizations
Processing quality and features
Parallel processing hardware
Bandwidth optimizations
DXVA2 – Decoding
Simplified enumeration (caps
versus profiles)
Used by decoder directly
Renderers take uncompressed buffers
Support for many codecs
WMV, VC-1, MPEG2, H.264, and more
Extensible
DXVA2 – Video Processing
Single step conversion, scaling
and composition
Vendors can optimize
HQ output to match output device
Deinterlacing
Stream metadata given to renderer
More input formats
RGB inputs
Video Processing
Colorspace conversions
Enhanced input/output colorspace control
High quality output control
Gamma, colorspace, Y’CbCr and fp output,
‘studio RGB’
Output filtering and ProcAmp
Extensive linear processing and progressive
RGB output support
Renderers
Enhanced video render
New render supersedes OvMixer, VMR(7), and VMR9
Video mixing, output timing scheduling
Many enhancements
Composites to output – non-square pixel, colorspace support
Pluggable mixer and presenters
Pull based
Advanced presenter – synchronized with monitor
Automatically handles output mode
Tear free windowed output, DWM support, fullscreen support
Glitch resilience – MMCSS, deep queuing, timeline mapping
Application integration
EVR available in Media Foundation and DShow
Stand-alone Mixer MFT
Not dependent on DXVA decoding
Transforms
Media Foundation Pipeline
Glitch resilience, content protection
Media Foundation Transform (MFT) infrastructure
Simple to use plug-in model
Addresses issues with DMOs
Supports DXVA and D3D processing
MFTs useable outside of pipeline
D3D Manager
Managed pipeline components use of D3D devices
Capture
AVStream
WDDM capture extension
Integration with WDDM and DXVA2 pipeline
Robust queuing, simplifies capture driver
Supports analog, BDA, multi-tuners
A/V sync lock
Multi-stream (A/V) capture devices (e.g., tuners)
Selective suspend on USB
USB Video Class Driver (UVC)
Still image 2 support
Colorspace support
DDC/CI
Monitor configuration control via DDC/CI
Display Data Channel Command Interface
Bi-directional communication and control
of external devices
Remote adjustments
Brightness, contrast, positioning, color temp
Low level/high level APIs
Extensible
Implement DDC/CI for improved
connectivity and usability
PVP-OPM, PVP-UAB
PVP-OPM
Protected Video Path Output
Protection Manager
Analog protection control
Beyond COPP, VideoParameters
Digital protection control
Output device topology discovery
(clone mode)
PVP-UAB
Protected Video Path –
User Accessible Bus
Display Support
Improved TV mode support
Native OS support for TV modes and resolutions
p24, p48, i60, i50, +
1080i, 720p, +
Clone mode
Drivers can populate modes and users can override
Better EDID support
Presentation feedback APIs (DX9L)
History of presented frames
Tighter A/V sync, glitch detection
scRGB framebuffer support
HQ output, including 10 bit Y’CbCr
Video Improvements
Areas of improvements for video
in Windows Vista
Performance
Quality
Extensibility
Integration
Connectivity – input and output
Performance
Capture
High glitch resilience on uncompressed capture
Results available for direct GPU processing (VP, encode,…)
Decoders
Work queued ahead of HW (lock/discard semantics)
Decoder no longer watches HW completions
HW/driver throttled
Video processing
Multiple frames queued for processing
Pipelined into display queue – no stalls
Driver can influence queue
Parallelizable with D3D, decoding – use it!
Video bandwidth reduction
Performance
Display
Frames decoded, processed, queued
in advance of presentation
Renderers schedule/queue ahead
Hybrid planar Y’CbCr formats
NV11, NV12, P208, Y416…
Improved cache usage, fewer DMA engines
needed,…
Fewer conversions/temporaries in pipeline
Simplifies apps, drivers, HW support
Quality
Glitch resilience
Improved queuing and latency control
HW should implement Vsync reporting, deep queues
Multimedia class scheduler
Fidelity
Single step composition to native device characteristics
HQ video processing and scaling
Renderers now monitor aware
End to end pipeline support for color spaces and overscan
Capture, decoders, transforms should implement it
Optimized for output device
Chroma handling reduces conversions
Lossless Y’CbCr <-> RGB conversions and interlace support
High precision fp scRGB support
Integration
Capture
Uncompressed output can pass directly to GPU for DXVA processing
Transforms
MFTs – write once, use everywhere (MF, DShow, custom apps)
Support for HW acceleration (DXVA/D3D)
EVR presenter design
Same presenter plug-in useable in both MF and DShow
Surface formats
Standardized Y’CbCr formats
Fewer conversions and copies
Simplifies capture, transforms, video output formats
Chroma sampling info preserved – avoid resampling
High precision scRGB (fp16) surfaces
Easy mixing of text, video, photos, etc.
Leverage color calibration for video
Future Directions
Expand video processing features
Improve performance, usability and quality
Future Directions
Video analysis
Encode acceleration
Motion estimation, DCT,…
Inverse telecine support (IVT)
Programmable video processing
Advanced de-interlacing and frame rate conversion
Extended output device support
TVs completely integrated into operating system
Cross-process pipelining of video
Efficient streaming of data between CPUs and GPUs
Resource management and load balancing
CPU multi-cores, 3-D
Video decoding and processing
Call To Action
Take advantage of new features in Windows Vista
Graphics IHVs
Pipelining of decoding, processing operations
Parallel hardware to improve concurrency
DXVA2 is power management and quality API/DDI
Higher power density decode, VP, encode, deinterlace
Accurate vsync and deep queuing support
Video processing
Colorspace support, multi-tap, gamma correct scaling
10 bit and scRGB buffer support
WDDM capture drivers
ISVs
Start the transition to MF, MFTs and EVR
Additional Resources
Web Resources
Specifications
http://msdn.microsoft.com/library/default.asp?url=/library/enus/Display_d/hh/Display_d/UserModeDisplayDriver_Guide_9205
cb11-857b-4ea0-b0d0-a4fb5bdb3e83.xml.asp
Related Sessions
Windows Vista Output Content Protection
DXVA 2.0: A New Hardware Video Acceleration
Pipeline For Windows Vista
Questions and comments
askDXVA @ microsoft.com
© 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.