Windows Audio Video Excellence Scott A. Love Program Manager Media Technology Group Microsoft Corporation Richard Russell Development Manager Client Performance Team Microsoft Corporation Andy Glass Program Manager Windows Core OS Microsoft Corporation Alex.
Download ReportTranscript Windows Audio Video Excellence Scott A. Love Program Manager Media Technology Group Microsoft Corporation Richard Russell Development Manager Client Performance Team Microsoft Corporation Andy Glass Program Manager Windows Core OS Microsoft Corporation Alex.
Windows Audio Video Excellence Scott A. Love Program Manager Media Technology Group Microsoft Corporation Richard Russell Development Manager Client Performance Team Microsoft Corporation Andy Glass Program Manager Windows Core OS Microsoft Corporation Alex Goyen Program Manager Media Technology Group Microsoft Corporation Session Outline Overview New Services in Windows Core Audio Fidelity Improvements for Windows codenamed “Longhorn” Video Fidelity Improvements for Longhorn ISR/DPC tool Logo Requirements Windows Audio Video Excellence Overview A/V Playback Improvements Removal of glitches; including resilience to CPU, IO, & Memory stress Fidelity The degree to which Windows accurately and precisely renders, captures, and delivers A/V content Performance Optimized resource consumption and minimized latency in A/V processing Audio/Video Logo Requirements Ensure that AV excellence criteria are met for 3rd party hardware/software New Services in Windows Core to Support Media Andy Glass Program Manager Windows Core OS Microsoft Corporation Issues Addressed by New Services CPU and other resources are underutilized Must have overhead for peak usage Lack of overhead allows other tasks to interfere with robust media CPU contention Disk contention Paging Multimedia Class Scheduler Service (MMCSS) Policy set on a machine basis Ranked set of tasks OEM configurable Applications register with service Identify task (or tasks) Receive index for application thread id MMCSS modifies thread priorities Honors profile and foreground status RT Heap Manager Provides non-blocking heap functions Allows locking of RT code pages in memory Provides set of non-blocking APIs Thread Ordering Functions to allow relational thread execution Simplifies existing functionality Allows more efficient processing Scheduled File I/O Addresses storage conflicts Allows bandwidth reservation Allows prioritization Non-deterministic Longhorn Audio and Video Platform Improvements Alejandro Goyen Program Manager, Media Technology Group Microsoft Corporation What Are We Doing in Longhorn Audio? Significant platform changes for audio Complete rewrite of audio stack in User Mode Low latency, glitch-resilient audio render/capture Resilient to CPU & I/O stress Kernel audio stack is going away Kmixer.sys, Sysaudio.sys, Wdmaud.sys Swmidi.sys, DMusic.sys (Hardware MIDI synth) Splitter.sys, AEC.sys, Redbook.sys Existing apps & drivers “just work” Why Did We Do It? Problems Audio glitching under stress Latency Increases under load 30-80ms at best Robustness/Stability Crash brings down system Expensive to maintain Extensibility Extending stack is complicated, requires kernel developer Competition XP not competitive with CE devices from a quality/perf perspective Solutions Glitch-free audio Leverage new base OS services Synchronous non-blocking streaming Latency Predictable, controllable Invariant under CPU/IO load Robustness/Stability Written in UM, no more bluescreens Extensibility Simple interface (APO/MFT) Result: easy to create user-mode extensions Competition CE devices parity is our goal Longhorn Audio Architecture Win32 “Legacy” Application LH Application PlaySound Media Foundation DirectX WinMM MFT M DShow DMusic MIDI (DRV) Streaming Audio Renderer WaveIn MixerLine A WaveOut DirectSound Hardware MIDI (DRV) Submix Mgr A WASAPI APO (Pre-Mix) MMDEV Global Audio Engine Global Mixer APO (Post-Mix) PortCls Audio Driver User Kernel How Will This Impact Application/Device Compatibility? Goal: No regressions in Audio from Windows XP We’re writing shim layers for most commonly used APIs PlaySound(), PlaySnd() WaveOut(), WaveIn() DirectSound / DirectShow MixerLine MIDIIn() MIDIOut() AEC Aec.sys is going away – per stream AEC versus global AEC (echo) Digital CD playback Redbook.sys is gone MCI-CD will be routed to user-mode audio stack New stack is built on existing WDM driver model Existing drivers will continue to work Quality/Performance Goals CPU / Memory footprint per stream Goal: To match or exceed Windows XP Best Quality 144 dB SNR Low Quality 30 dB SNR 32-bit float audio data in=>out Latency goals Latency 2 X single direction, 4 X in=>out with processing X is processing time-slice (currently 10ms) Pro Audio requirements: 4ms round-trip (capture + render) Requires sub-ms scheduling (500us timer resolution) Value Proposition for Longhorn Audio Why move away from WinMM or DirectX to the new APIs (WASAPI and/or Media Foundation)? Glitch-free audio playback/capture Resilient to CPU & I/O stress Full control of audio graph Data transparency Explicit transforms Multi-channel & per-app volume control Full integration w/Audio Policy Low latency IHV & ISV extensibility MFT and APO MFT/APO comparison MFT APO A/V extensibility model Local / app-specific Configurable by app Synchronous or Asynchronous Specifically designed for audio processing Global system effects Configurable by user (via CPL) or app (if Global Policy allows) Synchronous by design (non-blocking) Examples Examples MFT replace DMO A/V Encoder A/V Decoder SRC Speaker correction Room correction Longhorn Video Platform Improvements Video Platform Improvements in Longhorn DXVA2.0 is now stand alone No longer tied to a DShow filter directly connected to an MS Renderer Decoders etc can implement more advanced video processing and decoding control DXVA2.0 includes h/w acceleration support Video Decoding: MPG2, WMV9, H.264 Video Processing: Filtering, de-interlacing, colorspace conversion, ProcAmp, scaling and inverse telecine Improved video processing support Color information and interlaced images passed to HW for single step compositions Infrastructure for higher quality scaling (more taps) Mitigation of tearing in video in windowed mode Video Platform Improvements in Longhorn Minimizing scaling to improve image sharpness Often only a single step scale Better full-screen support for playback DDC/CI (Display Data Channel Command Interface) support TV out devices are ‘first class devices’ Color info plumbed from capture through decode/process/display Fewer YUV color spaces – hybrid planar formats to reduce conversions TV ouT: 8 bit and 10 bit YUV precision maintained Video dynamic range is maintained (including headroom/toeroom) Formats defined for 10 and 16 bit YUV data A New Long ISR and DPC Detection Tool Richard G. Russell Development Manager, Client Performance Team Windows Core Operating System Division Microsoft Corporation Long ISRs and DPCs are Problematic Overly Long ISRs and DPCs cause many problems Increased interrupt latency for other devices Increased DPC latency Simply “stealing” time from kernel and user mode threads and processes This a primary causes of Audio and Video quality issues Audio clicks, pops and drop outs Video frame drops Audio/Video synchronization problems Can cause other problems as well User responsiveness General performance degradation Why? ISRs and DPCs are Unbounded Interrupt Service Routines (ISRs) are Triggered by hardware Delayed Procedure Calls (DPCs) are scheduled by ISRs Timer DPCs are scheduled by many components ISRs and DPCs always run to completion The OS never preempts them ISRs and DPCs take priority over all threads LH provides ISR and DPC Diagnostics! Badly behaving drivers will have nowhere to hide! Also true for components that schedule timer DPCS LH Will monitor all ISRs and DPCs times! (24/7) This is done very efficiently This feature is client only – not for server Limits may be adjustable Data is captured and logged to the event log. Easily visible to IT pros, support people and sophisticated users Logs are programmatically available We are exploring ways to surface driver problems to users in helpful and actionable way – no firm plans yet Data Used to Improve Components Data will also be sent to Microsoft using the standard error reporting mechanisms: Windows Error Reports (WER) and Service Quality Mechanism (SQM) Subject to privacy policies, user settings, and group policy Data shared with Microsoft partners who own the components; as it is today Problems will be actively triaged and managed Tools to Help You Now We have a new tool to help you find long ISR and DPC problems It is called “RATT” (Richard and Aaron’s Tracing Tool) It “Rats out” long ISR’s and DPC’s It runs on LH and XP It is easy to use Simple to install Easy to turn on and off Reports are easy to read Provided on the WinHEC CD RATT Features Monitors all ISR and DPC activity 24/7 Logs for 3 minutes, then generates reports, repeats Uses the same mechanisms used in Longhorn to monitor ISR and DPC times Yes , this infrastructure is built into the XP kernel (The 24/7 feature is new for Longhorn) Very light weight: Can run in conjunction with other tests and workloads Is designed to be widely deployed Unattended install and operation Can also be run manually Can generate reports for all, some or one driver Future Tools More features added to RATT over time Watch Microsoft Downloads site Is also planned for delivery in the Windows Driver Kit Longhorn’s Driver Verifier will also support this DV will cause a break when a long ISR or DPC is detected Limits are programmable This is planned for Beta-2 WAVE and the LH Logo Program Gold Level: Premium experience Glitch Free High Definition Content Capable Automated Playback tests using HD content VC 1, MPEG 4 part 10, MPEG 2 Up to 30 MB/Sec 1920x1080i at 60Hz Superset of Silver Level requirements Silver Level: It just works Glitch Free Standard Definition Content Capable Automated Playback tests using SD content VC 1, MPEG 4 part 10, MPEG 2 720x480 i 30fps Pass Driver Verifier tool Call to Action! Use the new platform components for your multimedia applications Focus on minimizing ISR and DPC execution times: Use Driver Verifier at development time Incorporate multimedia playback (audio video) tests in your testing regime Capture drivers (kernel): Preserve color info in VIH2 Video Decoders (user mode): Preserve color on output Community Resources Windows Hardware & Driver Central (WHDC) www.microsoft.com/whdc/default.mspx Technical Communities www.microsoft.com/communities/products/default.mspx Non-Microsoft Community Sites www.microsoft.com/communities/related/default.mspx Microsoft Public Newsgroups www.microsoft.com/communities/newsgroups Technical Chats and Webcasts www.microsoft.com/communities/chats/default.mspx www.microsoft.com/webcasts Microsoft Blogs www.microsoft.com/communities/blogs Additional Resources Email: WAVE @ microsoft.com WinHEC 2005 CD © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.