Transcript Math for Programmers

Rendering and
Rasterization
Lars M. Bishop ([email protected])
Rendering Overview
• Owing to time pressures, we will only
cover the basic concepts of rendering
• Designed to be a high-level overview to
help you understand other references
• Details of each stage may be found in
the references at the end of the section
• No OpenGL/D3D specifics will be given
Essential Math for Games
Rendering: The Last Step
• The final goal is drawing to the screen
• Drawing involves creating a digital image of
the projected scene
• Involves assigning colors to every pixel on
the screen
• This may be done by:
 Dedicated hardware (console, modern PC)
 Hand-optimized software (cell phone, old PC)
Essential Math for Games
Destination: The Framebuffer
• We store the rendered image in the
framebuffer
 2D digital image (grid of pixels)
 Generally RGB(A)
• The graphics hardware reads these
color values out to the screen and
displays them to the user
Essential Math for Games
Top-level Steps
We have a few steps remaining



Tessellation: How do we represent the
surfaces of objects?
Shading: How do we assign colors to
points on these surfaces?
Rasterization: How do we color pixels
based on the shaded geometry?
Essential Math for Games
Representing Surfaces: Points
• The geometry pipeline leaves us with:
 Points in 2D screen (pixel) space (xs, ys)
 Per-point depth value (zndc)
• We could render these points directly by
coloring the pixel containing each point
• If we draw enough of these points, maybe we
could represent objects with them
 But that could take millions of points (or more)
Essential Math for Games
Representing Surfaces: Line
Segments
• Could connect pairs of points with line
segments
 Draw lines in the framebuffer, just like a 2D
drafting program
• This is called wireframe rendering
 Useful in engineering and app debugging
 Not very realistic
• But the idea is good – use sets of vertices to
define higher-level primitives
Essential Math for Games
Representing Surfaces:
Triangles
• We saw earlier that three points define
a triangle
• Triangles are planar surfaces, bounded
by three edges
• In real-time 3D, we use triangles to join
together discrete points into surfaces
Essential Math for Games
A Simple Cube
Cube represented
by 8 points
Cube represented
by triangles
(trust us)
(defined by 8 points)
Essential Math for Games
The Power of Triangles
• Flexible
 We can approximate a wide range of surfaces
using sets of triangles
• Tunable
 A smooth surface can be approximated by more or
fewer triangles
 Allows us to trade off speed and accuracy
• Simple but Effective
 We already know how to transform and project the
three vertices that define a triangle
Essential Math for Games
Vertices – “Heavy” Points
• As we will see, we often need to store
additional data at each distinct point that
define our triangles
• We call these data structures vertices
• Vertices provide a way of storing the
added values we need to compute the
color of the surface while rendering
Essential Math for Games
Common Per-vertex Values
 Position
• The “points” we’ve been transforming
 Color
• The color of the surface at/near the point
 Normal vector
• A vector perpendicular to the surface at/near
the point
 Texture coordinates
• A value used to apply a digital image (akin to a
decal on a plastic model) to the surface
Essential Math for Games
Efficient Rendering of
Triangle Sets
• Indexed geometry allows vertices to be
shared by several triangles
• Uses an array of indices into the array
of vertices
• Each set of three indices determine a
triangle
• Uses much fewer verts than (3 x Tris)
Essential Math for Games
Indexed Geometry Example
2
3
Configuration
18 Individual vertices
(exploded view)
1
0
4
5
7 shared vertices
Index list for shared vertices
(0,1,2),(0,2,3),(0,3,4),(0,4,5),(0,5,6),(0,6,1)
Essential Math for Games
6
Even More Efficient
• Triangle Strips (tristrips)
 Every vertex after the first two defines a triangle:
(i-2, i-1, i)
• Tristrips use much shorter index lists
 (2 + Tris) versus (3 x Tris)
• You may not quite get this stated efficiency
 You may have to include degenerate triangles to fit
some topologies
 For example, try to build our earlier 6-tri fan…
Essential Math for Games
Tristrip Example
0
2
4
6
1
3
5
7
Index list for shared vertices
(0, 1, 2, 3, 4, 5, 6, 7)
Essential Math for Games
Triangle Set Implementation
• Most 3D hardware is optimized for tristrips
• However, transformed vertices are often
“cached” in a limited-size cache
 Indexed geometry is not “perfect” – using a vertex
in two triangles may not gain much performance if
the vertex is kicked out of the cache between them
• Lots of papers available from the hardware
vendors on how to optimize for their caches
Essential Math for Games
Geometry Representation
Summary
• Generally, we draw triangles
• Triangles are defined by three screenspace vertices
• Vertices include additional information
required to store or compute colors
• Indexed primitives allow us to represent
triangles more efficiently
Essential Math for Games
Shading - Assigning Colors
• The next step is known as shading, and
involves assigning colors to any and all
points on the surface of each triangle
• There are several common shading
techniques
• We’ll present them from least complex
(and expensive) to most complex
Essential Math for Games
Triangle Shading Methods
• Per-triangle:
 Called “Flat” shading
 Looks faceted
• Per-vertex:
 Called “Smooth” or “Gouraud” shading
 Looks smooth, but still low-detail
• Per-pixel:
 Image-based texturing is one example
 Programmable shaders are more general
Essential Math for Games
Flat Shading
• Uses colors defined per-triangle directly
as the color for the entire triangle
Essential Math for Games
Gouraud (Smooth) Shading
• Gouraud shading defines a smooth
interpolation of the colors at the three
vertices of a triangle
• The colors at the vertices (C0,C1,C2)
define an affine mapping from
barycentric coordinates (s,t) on a
triangle to a color at that point:
C ( s, t )  C0 s  C1t  C2 (1  s  t )
 C2  (C0  C2 ) s  (C1  C2 )t
Essential Math for Games
Gouraud Shading
Essential Math for Games
Generating Source Colors
• Both Flat and Gouraud shading require
source colors to interpolate
• There are several common sources:
 Artist-supplied colors (modeling package)
 Dynamic lighting
Essential Math for Games
Dynamic Lighting
• Dynamic lighting assigns colors on a
per-frame basis by computing an
approximation of the light incident upon
the point to be colored
 Uses the vertex position, normal, and
some per-object material color information
• Dynamic lighting is detailed in most
basic rendering texts, including ours
Essential Math for Games
Imaged-based Texturing
• Extremely powerful shading method
• Unlike Flat and Gouraud shading,
texturing allows for sub-triangle, subvertex detail
• Per-vertex values specify how to map
an image onto the surface
• Visual result is as if a digital image were
“pasted” onto the surface
Essential Math for Games
Imaged-based Texturing
• Per-vertex texture coordinates (or UVs)
define an affine mapping from
barycentric coords to a point in R2.
• Resulting point (u,v) is a coordinate in a
texture image:
(0,1)
(1,1)
V Axis
(0,0)
Essential Math for Games
U Axis
(1,0)
Texture Applied to Surface
• The vertices of this cylinder include UVs
that wrap the texture around it:
V=1.0
V=0.0
Essential Math for Games
Rasterization:
Coloring the Pixels
The final step in rendering is called
rasterization, and involves
1) Determining which parts of a triangle are
visible (Visible Surface Determination)
2) Determining which screen pixels are
covered by each triangle
3) Computing the color at each pixel
4) Writing the pixel color to the framebuffer
Essential Math for Games
Computing Per-pixel Colors
• Even though rasterization is almost
universally done in hardware today, it
involves some interesting and
instructive mathematical concepts
• As a result, we’ll cover this topic in
greater detail
• Specifically, we’ll look at perspective
projections and texturing
Essential Math for Games
Conceptual Rasterization Order
• Conceptually, we draw triangles to the
framebuffer by rendering adjacent pixels
in screen space one after the other
• Rasterization draws pixel (x,y), then
(x+1,y), then (x+2,y) etc. across each
horizontal span covered by a triangle
• Then, it draws the next horizontal span
Essential Math for Games
Stepping in Screen Space
• If we have a fast way to compute the
color of a triangle at pixel (x+1,y) from
the color of pixel (x,y), we can draw a
triangle quite quickly
• The same is true for computing texture
coordinates
• Affine mappings are perfect for this
Essential Math for Games
Affine Mappings on Triangles
• Gouraud shading defines an affine
mapping from world-space points in a
triangle to colors
• Texturing defines an affine mapping
from world-space points on a triangle to
texture coordinates (UVs)
• Note, however, these are mappings
from world space to colors and UVs
Essential Math for Games
Affine in Screen Space
• For the moment, assume that we have
an affine mapping from screen space
(xs, ys) to color:
C ( xs , ys )  Axs  By s  C0
• Note that the depth value is ignored
Essential Math for Games
Forward Differences
• Note the difference in colors between a
pixel and the pixel directly to the right:
CΔX  C ( xs  1, ys )  C ( xs , ys )
  Axs  1  By s  C0    Axs  By s  C0 
 Axs  1  Axs
 A1  xs  xs 
A
Wow – that’s nice!
Essential Math for Games
Forward Differencing
• This simple difference leads to a useful trick:
• Given the color for a base pixel in a triangle,
the color of the other pixels are:
C ( xs , ys )  C0
C ( xs  i, ys  j )  C0  Ai  Bj
• To step from pixel to adjacent pixel (i=1
and/or j=1) is just an addition!
• This is called forward differencing
Essential Math for Games
Perspective
• Perspective projection is by far the most
popular projection method in games
• It looks the most like reality to us
• Perspective rendering does involve
some interesting math and surprising
visual results
Essential Math for Games
Affine Mapping of Texture UVs
• Over the next few slides, we’ll apply a
texture to a pair of triangles
• We’ll use screen-space affine mappings
to compute the texture coordinates for
each pixel
• First, let’s look at a special case: a pair
of triangles parallel to the view plane
Essential Math for Games
Polygon Parallel to View:
Affine Interpolation
Wire-frame view
Textured view
CORRECT
Essential Math for Games
Looking good so far…
• That worked correctly – maybe we can
get away with using affine mappings
(and thus forward differencing) for all of
our texturing…
• Not so fast – next, we’ll tilt the top of the
square away from the camera…
Essential Math for Games
Polygon Tilted in Depth:
Affine Interpolation
Wire-frame view
Textured view
WRONG!
Essential Math for Games
What Happened?
• We were interpolating using an affine
mapping in (2D) screen space.
• Perspective doesn’t preserve affine maps
• Let’s look at the inverse mapping: namely,
we’ll derive the mapping from world space to
screen space
• If this mapping isn’t affine, then the inverse
(screen to triangle) can’t be affine, either
Essential Math for Games
Example – Projecting a Line
• We’ll project a 2D line segment (y,z) into
1D using perspective:
Line in world space:
Projection of any point (not
necessarily on line) to 1D:
L(t )  P  Dt
yW
yS 
zW
y w  Py  D y t
z w  Pz  Dz t
Essential Math for Games
Projecting a Line Segment
Y-axis
View
Plane
D
P
L(t )Y
yS 
L(t ) Z
Essential Math for Games
Z-axis
Special Case – Parallel to View
Y-axis
View
Plane
D=(DY,0)
L(t )Y
yS 
L(t ) Z
Z-axis
P
PY  DY t

, DZ  0
PZ  DZ t
PY  DY t

 a  bt
PZ
Affine when projected That’s why this case worked
Essential Math for Games
General Case
Y-axis
D
Z-axis
L(t )Y
yS 
L(t ) Z
P
PY  DY t a  bt
yS 

PZ  DZ t c  dt
Not affine when projected –
That’s why this case was wrong
Essential Math for Games
Correct Interpolation
• The perspective projection breaks our
nice, simple affine mapping
 Affine in world space becomes projective in
screen space
 Per-vertex depth matters
• The correct mapping from screen space
to color or UVs is projective:
A  Bx s  Cys
D  Exs  Fys
Essential Math for Games
Perspective-correct Stepping
•
The per-pixel method for stepping our
projective mapping in screen space is:
1) Affine forward diff to step numerator
2) Affine forward diff to step denominator
3) Division to compute final value
•
This requires an expensive per-pixel
division, or even several divisions
Essential Math for Games
Polygon Tilted in Depth:
Correct Perspective
Wire-frame view
Textured view
CORRECT!
Essential Math for Games
Colors versus Texture UVs
• Textures need to be perspective-correct
because the detailed features in most
textures make errors obvious
 This is why the example images that we
have shown involve texturing
• Gouraud-shaded colors are more
gradual and can often get by without
perspective correction
Essential Math for Games
Cheating – Faster Perspective
• SW and HW systems have optimized
perspective correction by approximation
 Subdivide tris or scanlines into smaller bits
and use affine stepping (very popular)
 Use a quadratic function to approximate
the projective interpolation
 Reorder pixel rendering to render pixels of
constant depth together (very rare)
Essential Math for Games
Cheating - Artifacts
• Some tricks are better than others, and
some break down in extreme cases
• Look at some early PS1 games, and
you’ll see plenty of interesting and
different perspective-correction artifacts
• As with most approximations, you tend
to trade speed for accuracy
Essential Math for Games
References
• Eberly, David H., 3D Game Engine
Design, Morgan Kaufmann Publishers,
San Francisco, 2001
• Hecker, Chris, Behind the Screen:
Perspective Texture Mapping (Series),
Game Developer Magazine, Miller
Freeman, 1995-1996
Essential Math for Games
References
• Van Verth, James, Bishop, Lars, Essential
Mathematics for Games and Interactive
Applications: A Programmer’s Guide, Morgan
Kaufmann Publishers, San Francisco, 2004
• Woo, Mason, et al, OpenGL® Programming
Guide: The Official Guide to Learning
OpenGL, Addison-Wesley, 1999
Essential Math for Games