Transcript Buffers and Mappings UBI 516 Advanced Computer Graphics Ar.Gör.Cengiz Güngör

UBI 516
Advanced Computer Graphics
Buffers and Mappings
Ar.Gör.Cengiz Güngör
[email protected]
Historical Background
Vector displays
• Anybody remember Battlezone? Tempest?
Historical Background
1970 : Local illumination models
• Phong shading : Plastic models
1980 : Computer realism,
PCs and home computers
• Global illumination models
– High cost, but very realistic
• Texture mapping
– Low cost, but reasonable results
Historical Background
Until past decade application programmers have
had only indirect access to the frame buffer.
Today : Texture mapping, antialiasing,
compositing and alpha blending are only a few
of the techniques that become possibble when
the API (OpenGL) allows us to work with
discrete buffers.
Buffers and Mappings
Buffers
We have already used two types of standart buffers:
• Color buffers and depth buffers.
A block of memory
• n × m × k bit elements
Bit plane
• Any of the k n × m planes
in a buffer
Pixel
• All k of the elements at a position
pixel 1
bit plane 1
OpenGL Buffers
OpenGL Buffers and the Pixel Pipeline
Color buffers (including frame buffer)
• Double buffering : front / back buffers
• Stereo buffering : right / left buffers
Depth buffer
(= Z-buffer)
For hidden-surface removal
Accumulation buffer
Motion blur, anti-aliasing
Stencil buffer
Used for masking
operation
Read from / Write into Buffers
In a modern graphics system, a user program can both write into
and read from the buffers.
• We occasionally want to read or write a single pixel or bit.
• Rather, we tend to read and write rectangular blocks of pixels (or bits),
known as bit blocks.
– bit-blt (bit-block transfer) = raster-ops, raster operations
• Geometric object operations is different pixel operations.
– OpenGL contains both a geometric pipeline and a pixel pipeline.
write_block( source, n, m, x, y, destination, u, v );
Source is at (x,y),
Destination is at (u,v),
Operation writes m × n bit-block.
Pixel Pipeline
OpenGL support a seperate pixel pipeline and a variety of
buffers.
Data can be moved among these buffers.
And data can be moved between buffers and memory.
There are 16 operations for data transfer.
Writing Modes
Source  Destination writing operation has
• 16 different modes
Using modes :
• mode 3: copy
dest  source
• mode 7: logical OR
dest  source + dest
• mode 6: XOR
dest  source  dest
d’
XOR mode
mode 6 : twice XOR operation returns the original pixel.
• (x  y)  y = x
If we use menus :
• A menu appears when mouse clicked,
• And area of screen that it covered, must be returned orginal state
when menu disappeares.
• S : frame buffer, M : back buffer (backing storage)
S1  S0  M0
(S1 is xored back buffer)
M1  S1  M0 = (S0  M0)  M0 = S0
S2  S1  M1 = (S0  M0)  M1 = (S0  M0)  S0 = M0
Pixels and Images in OpenGL
Read, draw, copy operations start at raster position
void glReadPixels( GLint x, GLint y, GLsizei width,
GLsizei height, GLenum format, GLenum type,
GLvoid* pixels );
void glDrawPixels( GLsizei width, GLsizei height,
GLenum format, GLenum type, const GLvoid* pixels );
void glCopyPixels( GLint x, GLint y, GLsizei width,
GLsizei height, GLenum type );
Mapping Methods
One of the most powerful uses of buffers is for
surface rendering.
• Texture mapping : Surface patterns. In the real world,
we can distinguish among object of similar size and
shape by their textures. It can be 1,2,3 or 4 dimensional.
• Bump mapping : Distort of normal vectors like an
orange object.
• Environment mapping : Reflections.
Texture Mapping
Two-Dimensional Texture Mapping
Texture space (s, t) : Image [0,1] × [0,1]
• Texel : Texture element. pixel in the texture
Object space (xw, yw, zw) is 3 dimensional
Screen space (xs, ys) is 2D
Texture Mapping Methods
texture space
(s, t)
object space
(xw, yw, zw)
screen space
(xs, ys)
forward mapping
backward mapping
pixels
texture space
Example of Texture Mapping
Example of Texture Mapping
glVertex3d (s, s, s)
glTexCoord2d(5, 5);
glVertex3d (s, s, s)
glTexCoord2d(1, 1);
The Art of 3D Computer Animation and
Effects by Isaac Kerlow
Texture Mapping
Mapping from texture space to object space
• Linear mapping for a parametric surface
• Linear mapping function: (s,t)  (u,v)
– u = fu(s,t) = a s + b t + c
– v = fv(s,t) = d s + e t + f
• Invertible mapping when
aebd
s  sm in
u  um in 
(um ax  um in )
sm ax  sm in
t  t m in
v  vm in 
(vm ax  vm in )
t m ax  t m in
Two-part texture mapping
Bier and Sloan, “Two-part texture mapping”, IEEE CG&A, 6(9), 1986.
First step maps the texture to a simple three-dimensional
intermediate surface, such as sphere, cylinder or cube.
• 2D texture
– 2D  simple 3D mapping
Second, 3D intermediate surface mapped to object
surface.
• Simple 3D  complex 3D mapping
• 3D object (maybe complex)
Two-part texture mapping
Two-stage forward mapping
• S mapping
(surface mapping)
– T(u, v)  T’(xi, yi, zi) : intermediate surface
• O mapping
(object mapping)
– T’(xi, yi, zi)  O(xw, yw, zw)
intermediate surface : plane, cylinder, cube, sphere
cylinder의 경우
cube의 경우
Two-part texture mapping
S and O mapping
texture
S mapping
Using intermediate
object normals
O mapping
Using object
normals
Rays from center
of object
Texture Mapping in OpenGL
OpenGL textures
• From one- or two-dimensional texture
• To 1D, 2D, 3D, 4D object
• 2D texture to 3D object
glTexImage2D(…)
void glTexImage2D(
GLenum target,
GLint level,
GLint components,
GLsizei width,
GLsizei height,
GLint border,
GLenum format,
GLenum type,
const GLvoid* pixels
);
GL_TEXTURE_2D: target texture buffer
0, 1, 2, …: level-of-detail for Mip-Map
0, 1, 2, 3 : components
width of texture
height of texture
0 : border width
color format of pixels (R, G, B, A)
type of pixel data (int, short, long, …)
source color array
glTexImage2D – Arg 1
GLenum target
• GL_TEXTURE_2D
• GL_PROXY_TEXTURE_2D
– Provides queries for texture resources
– Proceed with hypothetical texture use (GL won’t
apply the texture)
– After query, call GLGetTexLevelParamter to verify presence
of required system components
– Doesn’t check possibility of multiple texture
interference
glTexImage2D – Arg 2
GLint level
• Used for Level of Detail (LOD)
• LOD stores multiple versions of texture that can be used at
runtime (set of sizes)
• Runtime algorithms select appropriate version of texture
– Pixel size of polygon used to select best texture
– Eliminates need for error-prone filtering algorithms
glTexImage2D – Arg 3
GLint internalFormat
• Describes which of R, G, B, and A are used in internal
representation of texels
• Provides control over things texture can do
– High bit depth alpha blending
– High bit depth intensity mapping
– General purpose RGB
• GL doesn’t guarantee all options are available on given
hardware
glTexImage2D – Args 4-6
GLsizei width, GLsizei height
• Dimensions of texture image
– Must be 2m + 2b (b=0 or 1 depending on border)
– min, 64 x 64
GLint border
• Width of border (1 or 0)
– Border allows linear blending between overlapping
textures
– Useful when manually tiling textures
glTexImage2D – Args 7-9
GLenum format
• Describe how texture data is stored in input array
– GL_RGB, GL_RGBA, GL_BLUE…
GLenum type
• Data size of array components
– GL_SHORT, GL_BYTE, GL_INT…
Const GLvoid *texels
• Pointer to data describing texture map
glGetTexImage(…)
void glGetTexImage(
GLenum target,
GLint level,
GLenum format,
GLenum type,
GLvoid* pixels
);
GL_TEXTURE_2D: target texture buffer
0, 1, 2, …: level-of-detail for Mip-Map
color format of pixels (R, G, B, A)
type of pixel data (int, short, long, …)
texture buffer
Example of Texture Mapping
In display() function :
glEnable(GL_TEXTURE_2D);
glBegin(GL_QUADS);
glTexCoord2f(0.0, 0.0);
glVertex2f(x1, y1, z1);
glTexCoord2f(1.0, 0.0);
glVertex2f(x2, y2, z2);
glTexCoord2f(1.0, 1.0);
glVertex2f(x3, y3, z3);
glTexCoord2f(0.0, 1.0);
glVertex3f(x4, y4, z4);
glEnd( );
Repeat or Clamp
First problem is what happens if we specify a
value of s & t outside the range (0.0, 1.0).
• Values below this interval cause the texture to repeat.
• Values above this interval cause the texture to clamp.
For repeated textures we:
glTexParameter(GL_TEXTURE_WRAP_S, GL_REPEAT);
• For t we use GL_TEXTURE_WRAP_T
• For clamping we use GL_CLAMP
Aliasing (1/3)
● Aliasing is a second problem.
● Sometimes the texel is larger
or smaller than a pixel.
● To magnify the texel:
glTexParameter(GL_TEXTURE_2D,
GL_TEXTURE_MAG_FILTER, GL_NEAREST);
● To minify the texel:
glTexParameter(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,
GL_NEAREST);
● To get a more visually
pleasing image we
use filtering to obtain
an average texel value.
● OpenGL can do this
if we specify GL_LINEAR instead of GL_NEAREST.
Aliasing (2/3)
● If 2nd param = GL_TEXTURE_MIN_FILTER
● 3th param can be
GL_NEAREST,
GL_LINEAR,
point sampling
linear interpolation
or MIPMAPs (texture pyramids) :
GL_NEAREST_MIPMAP_NEAREST
GL_LINEAR_MIPMAP_NEAREST
GL_NEAREST_MIPMAP_LINEAR
GL_LINEAR_MIPMAP_LINEAR
½×½
¼×¼
... Size reduced until 1x1
Aliasing (3/3)
Aliasing with Filtering
OpenGL tries to pick best mipmap level
Question: Which texel corresponds to a particular pixel?
GL_NEAREST (Point Sampling)
• Pick the texel with center nearest pixel
GL_LINEAR (Bilinear Sampling)
• Weighted average of 2x2 closest texels
GL_NEAREST_MIPMAP_LINEAR
• Average nearest texels from two mipmap levels
GL_LINEAR_MIPMAP_LINEAR (Trilinear)
• Average two averaged texels from two mipmaps
gluBuild2DMipMaps
int gluBuild2DMipmaps(
GLenum target,
GLint level,
GLint components,
GLsizei width,
GLsizei height,
GLint border,
GLenum format,
GLenum type,
const GLvoid* pixels
);
GL_TEXTURE_2D: target texture buffer
0, 1, 2, …: level-of-detail for Mip-Map
0, 1, 2, 3 : components
width of texture
height of texture
0 : border width
color format of pixels (R, G, B, A)
type of pixel data (int, short, long, …)
source color array
For a 64 x 64 original array we can set up 32 x 32, 16 x 16, 8 x 8, 4 x 4, 2 x2,
& 1 x 1 arrays through :
gluBuild2DMipMaps ( GL_TEXTURE_2D, 3, 64, 64,
GL_RGB, GL_UNSIGNED_BYTE, my_texels );
MIPMAPS
With versus without MIPMAP
Texture and Shading
● A forth problem is the interaction between texture
and shading.
● Modulation Solution:
– void glTexEnvi(
– GLenum target,
GL_TEXTURE_2D: target texture buffer
– GLenum pname, GL_TEX_ENV_MODE
– GLint param
GL_MODULATE, GL_DECAL
– );
• Example :
– glTexEnv(GL_TEX_ENV, GL_ENV_MODE, GL_MODULATE);
● Decaling Solution:
• If we replace GL_MODULATE by GL_DECAL, the color of the texture
determines the color of the object.
Perspective Projection
Proper texture mapping also depends on what type of
projection is used.
Default is Linear Interpolation.
• It does not work well with perspective projections.
We can use a better interpolation by:
glHint(GL_PERPECTIVE_CORRECTION, GL_NICEST);
Environmental Maps
Environment Mapping
● Blinn, “Models of light reflection for computer synthesized
pictures”, Computer Graphics, 11(2):192~198, 1977.
● Greene, “Environment mapping and other applications of world
projections”, IEEE CG&A, 6(11):21~29, 1986.
● = Reflection mapping
It does not use ray tracing methods.
shiny object
shiny object
object
camera
ray tracing
environment
texture
mapping
camera
object
environment mapping
Environment Mapping
Ray Tracing vs. Environment Map
environment map
ray tracing
Environment Mapping
2 stage algorithm
R : reflection vector
Environment map
(intermediate object)
Environment map
Texture mapping
Environment Mapping
● Used to model a object that reflects surrounding
textures to the eye
• Polished sphere reflects walls and ceiling textures
• Cyborg in Terminator 2 reflects flaming destruction
● Texture is distorted fish-eye view of
environment
● Spherical texture mapping creates texture
coordinates that correctly index into this texture
map
Sphere Mapping
Blinn/Newell Lattitude Mapping
Cube Mapping
Bump Maps
Bump Mapping
Blinn, “Simulation of wrinkled surfaces”, Computer Graphics, 1978.
●Use textures to modify surface geometry
●Use texel values to modify surface normals of
polygon
●Texel values correspond to height field
• Height field models a rough surface
●Partial derivative of bump map specifies change
to surface normal
Bump Mapping
Bump Mapping Process
N(u,v)
P(u,v)
N
P P

u v
P' (u, v)  P(u, v)  B(u, v)  N
P'

P B
N

(P  B  N) 

NB
u u
u u
u
P' 
P B
N

(P  B  N) 

NB
v v
v v
v
B(u,v)
P’(u,v)
P' P'  P B   P B 



N

N
u v  u u   v v 
B
P B
P
N
N

N
u
v v
u
N' 
N’(u,v)
Bump Mapping Process
B
P B
P
N'  N 
N

N
u
v v
u
B
 B

 N
A
B
v
 u

ND
N
N
P
v
P
P
u
P
B  N
u
D
A  N
P
v
Two-pass Bump Mapping
McReynolds and Blythe, “Programming with OpenGL: Advanced
Rendering”, SIGGRAPH’97 Lecture Notes.
● Key Idea :
N’·L = N·L + D·L
● N·L :
Gouraud Shading
● D·L :
image of surface detail
approximation
D
B
P B
P
N

N
u
v v
u
Multirendering and
The Accumulation Buffer
Accumulation Buffer
A buffer for multirendering
• Same as frame buffer resolution
Before operation
glClear(GL_ACCUM_BUFFER_BIT);
• AccumBuffer  0
void glAccum( GLenum op, GLfloat value );
•
•
•
•
GL_LOAD : AccumBuffer  FrameBuffer
GL_ACCUM :
AccumBuffer  AccumBuffer + value * FrameBuffer
GL_MULT : AccumBuffer  AccumBuffer * value
GL_RETURN:
FrameBuffer  AccumBuffer
Accumulation Buffer
●
●
●
●
●
●
●
●
●
clear
make FrameBuffer with camera at (x, y)
AccumBuffer  AccumBuffer + 1.0 * FrameBuffer
make FrameBuffer with camera at (x+1, y)
AccumBuffer  AccumBuffer + 1.0 * FrameBuffer
make FrameBuffer with camera at (x–1, y)
AccumBuffer  AccumBuffer + 1.0 * FrameBuffer
AccumBuffer  AccumBuffer * 0.33
FrameBuffer  AccumBuffer
1/3
1/3
1/3
convoluted image
Accumulation Buffer
glClear( GL_ACCUM_BUFFER_BIT);
for (i=0; i<num_images; i++)
{
glClear( GL_COLOR_BUFFER_BIT |
GL_DEPTH_BUFFER_BIT );
display_image(i);
glAccum( GL_ACCUM, 1.0/ (float) num_images );
}
glAccum( GL_RETURN, 1.0);
Other Multipass Methods: Motion Blur
t = 0.3
t = 0.2
t = 0.1
t = 0.0
motion blurred image
another example
Depth of Field
When camera focused an object, near and far objects are
blured.
focused view
camera #1
Accumulation buffer
can be used for
to generate this effect.
camera #2
camera #3
Accumulation
Sampling and Aliasing
Nyquist sampling theorem
The ideal samples of a continuous function
contain all the information in the original
function if and only if the continuous
function is sampled at a frequency at least
twice the highest frequency in the
function.
We can reconstruct a continuous function
f(x) from its samples {fi} by the formula:
f ( x, y ) 

 fi sinc(x  xi )
i  
sin x
sinc(x) 
x
Aliasing of sinusoid
Images
An image is a 2D function I(x, y) that specifies
intensity for each point (x, y)
Sampling and Image
Our goal is to convert the continuous image to a
discrete set of samples
The graphics system’s display hardware will
attempt to reconvert the samples into a
continuous image: reconstruction
Point Sampling an Image
● Simplest sampling is on a grid
● Sample depends
solely on value
at grid points
Point Sampling
Multiply sample grid by image intensity to obtain a
discrete set of points, or samples.
Sampling Geometry
Sampling Errors
Some objects missed entirely, others poorly
sampled
Fixing Sampling Errors
Supersampling
• Take more than one sample for each pixel and combine
them
150x15 to 100x10
– How many
samples is
enough?
– How do we
know no
features are
lost?
200x20 to 100x10
300x30 to 100x10
400x40 to 100x10
Unweighted Area Sampling
Average supersampled points
All points are weighted equally
Weighted Area Sampling
Points in pixel are weighted differently
• Flickering occurs as object moves
across display
Overlapping regions eliminates flicker
How is this done today?
Full Screen Antialiasing
Nvidia GeForce2
• OpenGL: render image 400% larger and supersample
• Direct3D: render image 400% - 1600% larger
Nvidia GeForce3
• Multisampling but with fancy overlaps
– Don’t render at higher resolution
– Use one image, but combine values of neighboring pixels
– Beware of recognizable combination artifacts
• Human perception of patterns is too good
GeForce3
Multisampling
• After each pixel is rendered, write pixel value to two
different places in frame buffer
GeForce3 - Multisampling
After rendering two copies of entire frame
• Shift pixels of Sample #2 left and up by ½ pixel
• Imagine laying Sample #2 (red) over Sample #1 (black)
GeForce3 - Multisampling
Resolve the two samples into one image by computing
average between each pixel from Sample 1 (black) and
the four pixels from Sample 2 (red) that are
1/ sqrt(2) pixels away
GeForce3 - Multisampling
No AA
Multisampling
GeForce3 - Multisampling
4x Supersample
Multisampling