Transcript Advanced 3D Techniques with WPF David Teitlebaum - Program Manager Jordan Parker - Developer Kurt Berglund - Developer Microsoft Corporation.

Advanced 3D Techniques with WPF
David Teitlebaum - Program Manager
Jordan Parker - Developer
Kurt Berglund - Developer
Microsoft Corporation
Introduction to WPF3D
What do you need to do to see something?
Interactive 2D on 3D
Make VisualBrush work like you expected it to
Performance Tips
Learn how to avoid common mistakes
Warmup Time
Pattern: FooBar is a Bar with a Foo
Examples:
ModelVisual3D is a Visual3D with a Model3D
GeometryModel3D is a Model3D with a Geometry3D
MeshGeometry3D is a Geometry3D with a mesh
ScaleTransform3D is a Transform3D with a scale
And so on (generally)
DrawingImage vs. ImageDrawing anyone?
Viewport3D
Visual3D
Camera
Model3DGroup
GeometryModel3D
Material
Light
Geometry3D
Let’s do a simple 3D video example step-by-step
Right-handed
Notice that +Y is up… opposite of 2D!
(Image from http://www.willamette.edu/~gorr/)
Specifies a mesh of triangles
Positions: Triangle vertices
TriangleIndices: Optional indices
Normals: Per-vertex orientation used in
lighting. Generated for you.
TextureCoordinates: Describes the 2D Brush
mapping. NOT generated for you. Not needed
for SolidColorBrushes.
Triangle vertices should be given in CCW order
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
If specified:
positions[indices[0]]
positions[indices[1]]
…
texcoords[indices[0]]
normals[indices[0]]
texcoords[indices[1]]
normals[indices[1]]
…
…
If not specified:
positions[0]
positions[1]
…
texcoords[0]
normals[0]
texcoords[1]
normals[1]
…
…
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
(image from blogs.msdn.com/danlehen)
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
If you don’t specify normals, we generate them by
averaging the “real” normals of the triangles that
share the vertex. This gives a smooth
appearance
If you duplicate a vertex, it will end up with different
normals each time. Triangle specification matters!
If you specify too many, we throw out the extras
If you specify too few, we generate the rest
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Cylinder Cap Generation Example
Viewport3D
All Vertices Shared
Edge Vertices Duplicated
PerspectiveCamera
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Looking down -Z
(-1,1,0)
0
(0,0)
(1,1,0)
3
(1,0)
(-1,-1,0)
1
(0,1)
(1,-1,0)
2
(1,1)
Key:
pos
idx
tc
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
<MeshGeometry3D
Positions=“-1,1,0 -1,-1,0 1,-1,0 1,1,0”
TextureCoordinates=“0,0 0,1 1,1 1,0”
TriangleIndices=“0,1,2 2,3,0”
/>
We’ll let WPF compute the normals
In general, you’ll be exporting a mesh to XAML from a tool
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Reflection/absorption of light
EmissiveMaterial: Surface appears to give off light
SpecularMaterial: Shine
DiffuseMaterial: Soft scattering
Takes a Brush for the base color
Combine them with MaterialGroup
When in doubt, use DiffuseMaterial
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Diffuse
PerspectiveCamera
Viewport3D
Emissive
AmbientLight
Specular
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
<DiffuseMaterial>
<DiffuseMaterial.Brush>
<VisualBrush>
<VisualBrush.Visual>
<MediaElement Source=“foo.wmv” />
</VisualBrush.Visual>
</VisualBrush>
</DiffuseMaterial.Brush>
</DiffuseMaterial>
(In code, you could use a DrawingBrush w/ a VideoDrawing instead)
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Combines the Material and the Geometry3D
Allows you to specify a transform
<GeometryModel3D>
<GeometryModel3D.Geometry>
(the plane mesh)
</GeometryModel3D.Geometry>
<GeometryModel3D.Material>
(the video material)
</GeometryModel3D.Material>
<GeometryModel3D.Transform>
<ScaleTransform3D ScaleX=“1.33333” />
</GeometryModel3D.Transform>
</GeometryModel3D>
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Not unlike reality, without light you can’t see anything*
AmbientLight: Equal contribution everywhere
DirectionalLight: The sun
PointLight: Light bulb
SpotLight: Self explanatory 
Lights are Model3Ds and thus have transforms
We’ll use an AmbientLight in our example for
simplicity
<AmbientLight Color=“white” />
* Except EmissiveMaterials
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Group multiple Model3Ds together
Is a Model3D
<Model3DGroup>
(the ambient light)
(the plane GeometryModel3D)
</Model3DGroup>
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
3D analog of the 2D Visual class
You need at least one just like you need at
least one Visual in 2D (FrameworkElements
are Visuals)
To draw a Model3D, use ModelVisual3D
<ModelVisual3D>
<ModelVisual3D.Content>
(the Model3DGroup)
</ModelVisual3D.Content>
</ModelVisual3D>
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Determine what part of the scene you see and
project 3D onto a 2D plane
OrthographicCamera: parallel lines stay parallel
PerspectiveCamera: perspective foreshortening
MatrixCamera: Specify your own projection
<PerspectiveCamera
Position=“0,0,4” LookDirection=“0,0,-1” />
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
The bridge between the 2D and 3D world
It is a FrameworkElement
Takes a Camera and the root of your Visual3D
tree
<Viewport3D>
<Viewport3D.Camera>
(the PerspectiveCamera)
</Viewport3D.Camera>
(the ModelVisual3D)
</Viewport3D>
PerspectiveCamera
Viewport3D
AmbientLight
DiffuseMaterial
ModelVisual3D Model3DGroup GeometryModel3D MeshGeometry3D
Interactive3DDecorator
Goes around the Viewport3D you want to make
interactive
InteractiveVisual3D
Subclass of ModelVisual3D used to add interactive
2D on 3D
Has DependencyProperties Geometry, Visual, and
Material used to set the appearance of the object
<Viewport3D>
<ModelVisual3D>
<ModelVisual3D.Content>
<GeometryModel3D Geometry=“{StaticResource PlaneMesh}”>
<GeometryModel3D.Material>
<DiffuseMaterial>
<DiffueMaterial.Brush>
<VisualBrush
Visual=“{StaticResource Visual1}” />
</DiffuseMaterial.Brush>
</DiffuseMaterial>
</GeometryModel3D.Material>
</GeometryModel3D>
</ModelVisual3D.Content>
</ModelVisual3D>
</Viewport3D>
<local:InteractiveViewport3D>
<Viewport3D>
<local:InteractiveVisual3D
Geometry=“{StaticResource PlaneMesh}”
Visual=“{StaticResource Visual1}” />
</Viewport3D>
</local:InteractiveViewport3D>
Hidden visual layer placed above Viewport3D
2D on 3D that the mouse is over is placed in
hidden layer
Layer moved so that the point the mouse is
over in 3D is the same as the point the mouse
is over in the hidden layer
public void HitTest(object sender, System.Windows.Input.MouseButtonEventArgs args)
{
Point mouseposition = args.GetPosition(myViewport);
PointHitTestParameters pointparams = new PointHitTestParameters(mouseposition);
VisualTreeHelper.HitTest(myViewport, null, HTResult, pointparams);
}
public HitTestResultBehavior HTResult(System.Windows.Media.HitTestResult rawresult)
{
RayHitTestResult rayResult = rawresult as RayHitTestResult;
if (rayResult != null)
{
RayMeshGeometry3DHitTestResult rayMeshResult =
rayResult as RayMeshGeometry3DHitTestResult;
if (rayMeshResult != null)
{
…
}
}
return HitTestResultBehavior.Continue;
}
RayMeshGeometry3DHitTestResult contains
all the information about the hit test
Point, Mesh, Model, Visual3D hit
Vertex indices of triangle hit
Barycentric coordinates for point of intersection:
can be used to compute texture coordinate of
intersection
Weighted sum of triangle’s vertices
p = VertexWeight1 * v1 + VertexWeight2 * v2 +
VertexWeight3 * v3
Can be extended to texture coordinates
uv = VertexWeight1 * uv1 +
VertexWeight2 * uv2 +
VertexWeight3 * uv3
Use to figure out where on texture you are
Say we want to fill the viewport with a 3D
rectangle
Viewport3D has width, height, and camera is at
origin looking down negative z axis
Horizontal
FieldOfView
-z
Can bind any DependencyProperty
<ScaleTransform3D
ScaleX="{Binding ElementName=sliderX, Path=Value}"
ScaleY="1" ScaleZ="1" />
Do more with less
Performance must be a priority throughout app
construction
Test early, test often
Do not assume you can easily fix performance
problems after your application is functionally complete
Test on the hardware you care about
Old Desktops, New Desktops, Laptops
Most Dev machines are not “Old Desktops”
CPU
Easy to measure with task manager, profilers, etc.
Many potential causes of CPU load
GPU
Hard to measure – there are few public tools to
measure GPU load
Can degrade frame rates without taxing CPU
Few potential causes – typically the result of
insufficient fill rate on the GPU or background
video decode
GPUs are optimized for video games with few state
changes
Know your limits
Coarse indicator of maximum machine capabilities
Queryable via:
(RenderCapability.Tier >> 16)
Tier resides in the high word so don’t forget the shift!
Indicates static capabilities
An upper bound on what can be expected
Does not account for dynamically consumed machine
resources
Value is read only (unfortunately! )
Continued
Rendering Tier
0
1
2
Shader Version
NA
1.0
2.0
Video RAM
NA
30 MB
120 MB
Multitexture Units
NA
2
4
Performance tools
WPFPerf.exe
Profilers
Microsoft F1
AMD Code Analyst
Intel Vtune
Common Pitfalls
#1 Brush Selection
Persistence vs. Nonpersistence of realizations
Intermediate Render Targets
Texture memory swizzling
Texture size
Smaller is better
Watch out for automatic texture sizing
RenderTargetBitmap - best of both worlds?
Common Pitfalls
#1 Continued - RenderTargetBitmap
Rectangle r = new Rectangle();
r.Width = 50;
r.Height = 50;
r.Fill = new LinearGradientBrush(Colors.CornflowerBlue,
Colors.PeachPuff, 13.37);
r.Measure(new Size(50, 50));
r.Arrange(new Rect(new Size(50, 50)));
r.UpdateLayout();
RenderTargetBitmap rtb = new RenderTargetBitmap((int)r.ActualWidth,
(int)r.ActualHeight, 96.0, 96.0, PixelFormats.Pbgra32);
rtb.Render(r);
Common Pitfalls
#1 Continued - Brush Speed on Tier-2
Hardware in 3D ONLY
1
2
3
Brush Speed (Fastest to Slowest)
SolidColorBrush
LinearGradientBrush
ImageBrush
Persistence
Yes
Yes
Yes
IRT
No
No
“No”
4
5
6
DrawingBrush (cached)
VisualBrush (cached)
RadialGradientBrush
Yes
Yes
No
Yes
Yes
Yes
7
8
DrawingBrush (uncached)
VisualBrush (uncached)
No
No
Yes
Yes
Common Pitfalls
#2 Disable Viewport3D.ClipToBounds
WPF uses complex antialised clipping
When Viewport3D.ClipToBounds == False, 3D
content can exceed 2D region of Viewport3D
Implementation of clipping is slow enough that it
should be disabled when it is not needed
Common Pitfalls
#3 Disable Viewport3D.IsHitTestVisible
WPF3D performs mouse-over hit testing on the
CPU
Mouse cursor is projected into object space and
intersected with GeometryModel3D’s axis-aligned
bounding box
If bounding box is intersected, ray of projected
mouse cursor must be intersection-tested against
every triangle in GeometryModel3D, all on the
CPU
If you don’t need hit testing, disable it.
Common Pitfalls
#4 GeometryModel3D Coalescence
Each Model3D requires a change of GPU state
Each requires the projection of its axis-aligned
bounding box to device coordinates
A Few complex GeometryModel3Ds is vastly
preferable to many simple GeometryModel3Ds
Common Pitfalls
#5 Minimize Mesh Animations
They do not scale well due to platform architecture
Entire Collections are remarshalled rather than
collection deltas
Disconnect meshes from 3D scene before
modifying to minimize change notifications
Keep animated meshes small (a few thousand
vertices at the most)
Common Pitfalls
#6 Disable 3D Antialiasing
On Vista, WPF3D will try to render with 4
samples/pixel multisampling
On XP, WPF3D will not try to multisample
To prevent WPF3D from attempting to render with
multisampling on Vista, add the attached property
RenderOptions.EdgeMode=“Aliased” to your
Viewport3D
Common Pitfalls
#7 Text in 3D
Text is expensive
Glyphs are abstractly defined by Beziers
Glyphs are generated and cached according to scale
Small glyphs – Bitmaps / Large glyphs - Polygons
scale changes cause glyph cache invalidation, glyph
regeneration, and glyph cache repopulation
RenderTargetBitmap / ImageBrush is strongly
recommended for static text
Common Pitfalls
#8 Caching of VisualBrush and DrawingBrush
Attached Property - RenderOptions.CachingHint
Use when ImageBrush is not sufficient
Still slower than ImageBrush due to less
aggressive texture memory swizzling on D3D’s
RenderTargetTextures
Frequency of cache regeneration can be controlled
by RenderOptions.CacheInvalidationThresholdMinimum
and RenderOptions.CacheInvalidationThresholdMaximum
Common Pitfalls
#9 BitmapEffects
Should only be used on static 3D content
Implemented on the CPU
Forces affected content to also be rendered on the CPU
Great for static content, otherwise don’t use them
WPF3D
http://blogs.msdn.com/wpf3d
http://blogs.msdn.com/danlehen
http://blogs.msdn.com/wpfsdk/archive/2007/01/15/maximizing-wpf3d-performance-on-tier-2-hardware.aspx
http://www.codeplex.com/3dtools
WPF Perf
http://blogs.msdn.com/timothyc
http://blogs.msdn.com/kiranku
Ask Questions
http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=119
news://microsoft.public.windows.developer.winfx.avalon
Books
Windows Presentation Foundation Unleashed
Petzold’s book coming out Summer 2007
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it
should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.