CS 352: Computer Graphics Chapter 7:

The Rendering Pipeline

Chapter 8 - 2

Overview

Interactive Computer Graphics

   Geometric processing: normalization, clipping, hidden surface removal, lighting, projection (

front end

) Rasterization or scan conversion, including texture mapping (

back end

) Fragment processing and display

Chapter 8 - 3 Interactive Computer Graphics



Geometric Processing

Front-end processing steps (3D floating point; may be done on the CPU)           Evaluators (converting curved surfaces to polygons) Normalization (modeling transform, convert to world coordinates) Projection (convert to screen coordinates) Hidden-surface removal (object space) Computing texture coordinates Computing vertex normals Lighting (assign vertex colors) Clipping Perspective division Backface culling

Chapter 8 - 4 Interactive Computer Graphics

 

Rasterization

Back-end processing works on 2D objects in screen coordinates Processing includes   Scan conversion of primitives including shading Texture mapping       Fog Scissors test Alpha test Stencil test Depth-buffer test Other fragment operations: blending, dithering, logical operations

Chapter 8 - 5 Interactive Computer Graphics

 

Display

Convert frame buffer to video signal Other considerations:  Color correction  Antialiasing

Chapter 8 - 6 Interactive Computer Graphics



Geometric Transformations

Five coordinate systems of interest:  Object coordinates     Eye (world) coordinates [after modeling transform, viewer at the origin] Clip coordinates [after projection] Normalized device coordinates [after ÷ w] Window (screen) coordinates [scale to screensize]

Chapter 8 - 7 Interactive Computer Graphics

 

Line-Segment Clipping

Clipping may happen in multiple places in the pipeline (e.g. early trivial accept/reject) After projection, have lines in plane, with rectangle to clip against

Chapter 8 - 8 Interactive Computer Graphics

    

Clipping a Line Against x

min Given a line segment from (x 1 ,y 1 ) to (x 2 ,y 2 ), Compute m=(y 2 –y 1 )/(x 2 –x 1 ) Line equation: y = mx + h h = y 1 – m x 1 Plug in x min (y intercept) to get y Check if y is between y 1 and y 2 .

 This take a lot of floating-point math. How to minimize?

Chapter 8 - 9 Interactive Computer Graphics

 

Cohen-Sutherland Clipping

For both endpoints compute a 4-bit

outcode

(o 1 , o 2 ) depending on whether coordinate is outside cliprect side Some situations can be handled easily

Chapter 8 - 10 Interactive Computer Graphics



Cohen-Sutherland Conditions

Cases.

    1. If o 1 =o 2 =0, accept 2. If one is zero, one nonzero, compute an intercept. If necessary compute another intercept.

Accept.

3. If o 1 & o 2  0. If both outcodes are nonzero and the bitwise AND is nonzero, two endpoints lie on same outside side. Reject. 3. If o 1 & o 2  0. If both outcodes are nonzero and the bitwise AND is zero, may or may not have to draw the line. Intersect with one of the window sides and check the result.

Chapter 8 - 11 Interactive Computer Graphics

 

Cohen-Sutherland Results

In many cases, a few integer comparisons and Boolean operations suffice.

This algorithm works best when there are many line segments, and most are clipped  But note that the y=mx+h form of equation for a line doesn ’ t work for vertical lines (need a special case in your code)

Chapter 8 - 12 Interactive Computer Graphics

   

Polygon Clipping

Clipping a polygon can result in lots of pieces Replacing one polygon with many may be a problem in the rendering pipeline Could treat result as one polygon: but this kind of polygon can cause other difficulties Some systems allow only convex polygons, which don ’ t have such problems (OpenGL has tessellate function in glu library)

Chapter 8 - 13 Interactive Computer Graphics

Sutherland-Hodgeman Polygon Clipping

   Could clip each edge of polygon individually Pipelined approach: clip polygon against each side of rectangle in turn Treat clipper as “ black box ” pipeline stage

Chapter 8 - 14



Clipping Pipeline

Clip each bound in turn

Interactive Computer Graphics

Chapter 8 - 15 Interactive Computer Graphics



Clipping in Hardware

Build the pipeline stages in hardware so you can perform four clipping stages at once  [A partial answer to the question of what to do with all that chip area, 1 billion + transistors…]

Chapter 8 - 16 Interactive Computer Graphics

 

Clipping complicated objects

Suppose you have many complicated objects, such as models of parts of a person with thousands of polygons each When and how to clip for maximum efficiency?

 How to clip text? Curves?

Chapter 8 - 17 Interactive Computer Graphics

   

Clipping Other Primitives

It may help to clip more complex shape early in the pipeline This may be simpler and less accurate One approach: bounding boxes (sometimes called

trivial accept-reject

) This is so useful that modeling systems often store bounding box

Chapter 8 - 18 Interactive Computer Graphics

    

Clipping Curves, Text

Some shapes are so complex that they are difficult to clip analytically Can approximate with line segments Can allow the clipping to occur in the frame buffer (pixels outside the screen rectangle aren ’ t drawn) Called “

scissoring

” How does performance compare?

Chapter 8 - 19 Interactive Computer Graphics

  

Hidden surface removal

Object space vs. Image space The main image-space algorithm: depth buffer Drawbacks  Aliasing   Rendering invisible objects Doesn’t handle transparency correctly  How would

removal object-space hidden surface

work?

Chapter 8 - 20



Depth sorting

The painter ’ s algorithm: draw from back to front

Interactive Computer Graphics

  Depth-sort hidden surface removal:  sort display list by z-coordinate from back to front  render Drawbacks   it takes some time (especially with bubble sort!) it doesn ’ t work

Chapter 8 - 21 Interactive Computer Graphics

   

Depth-sort difficulties

Polygons with overlapping projections Cyclic overlap Interpenetrating polygons What to do?

Chapter 8 - 22 Interactive Computer Graphics

  

Scan Conversion

At this point in the pipeline, we have only polygons and line segments. Render!

To render, convert to pixels ( “ fragments ” ) with integer screen coordinates (ix, iy), depth, and color Send fragments into fragment-processing pipeline

Chapter 8 - 23 Interactive Computer Graphics

 

Rendering Line Segments

How to render a line segment from (x 1 , y 1 ) to (x 2 , y 2 )?

Use the equation y = mx + h  What about horizontal vs. vertical lines?

Chapter 8 - 24 Interactive Computer Graphics

 

DDA Algorithm

DDA: Digital Differential Analyzer for (x=x 1 ; x<=x 2 ; x++) y += m; draw_pixel(x, y, color) Handle slopes 0 <= m <= 1; handle others symmetrically  Does this need floating point math?

Chapter 8 - 25 Interactive Computer Graphics

 

Bresenham

’

s Algorithm

The DDA algorithm requires a floating point

add

and

round

for each pixel: eliminate?

Note that at each step we will go E or NE. How to decide which?

−

Chapter 8 - 26 Interactive Computer Graphics

 

Bresenham Decision Variable

Note that at each step we will go E or NE. How to decide which?

Hint: consider d=a-b, where a and b are distances to NE and E pixels −

Chapter 8 - 27 Interactive Computer Graphics

   

Bresenham Decision Variable

Bresenham algorithm uses decision variable d=a-b, where a and b are distances to NE and E pixels If d<=0, go NE; if d>0, go E Let d x = x 2 -x 1 , d y =y 2 -y 1 Use decision variable d = d x (a-b) [only sign matters] −

Chapter 8 - 28 Interactive Computer Graphics

  

Bresenham Decision Variable

d =(a-b) d x Let d k be the value of d at x = k + ½ Move E:    d k = d x (a-b) = d x ((j+3/2–y k ) – (y k –(j+1/2))) d k+1 = d x (a-b) = d x ((j+3/2–y k –m) – (y k +m–(j+1/2))) d x+1 – d k = d x (-2m) = - 2 d Y −  Algorithm:   d k+1 d k+1 = d k = d k – 2 d y (if d k > 0) (last move was E) – 2 (d y -d x ) (if d k <= 0) (last move was NE)

Chapter 8 - 29 Interactive Computer Graphics

  

Bresenham’s Algorithm

Set up loop computing

d

for (x=x 1 ; x<=x 2 ; ) x++; d += 2dy; if (d >= 0) { y++; d –= 2dx; } at x 1 , y 1 drawpoint(x,y); Pure integer math, and not much of it So easy that it’s usually implemented in one graphics instruction for several points in parallel

Chapter 8 - 30 Interactive Computer Graphics

 

Rasterizing Polygons

Polygons may be or may not be simple, convex, flat. How to render?

Amounts to inside-outside testing: how to tell if a point is in a polygon?

Chapter 8 - 31 Interactive Computer Graphics



Winding Test

Most common way to tell if a point is in a polygon: the winding test.

 Define “ winding number ” w for a point: signed number of revolutions around the point when traversing boundary of polygon once  When is a point “ inside ” the polygon?

Chapter 8 - 32 Interactive Computer Graphics

  

OpenGL and Complex Polygons

OpenGL guarantees correct rendering only for simple, convex, planar polygons OpenGL tessellates concave polygons Tessellation depends on winding rule you tell OpenGL to use: Odd, Nonzero, Pos, Neg, ABS_GEQ_TWO

Chapter 8 - 33

Winding Rules

Interactive Computer Graphics

Chapter 8 - 34 Interactive Computer Graphics

Scan-Converting a Polygon

  General approach: ideas?

One idea:

flood fill

 Draw polygon edges  Pick a point (x,y) inside and flood fill with DFS flood_fill(x,y) { if (read_pixel(x,y)==white) { write_pixel(x,y,black); flood_fill(x-1,y); flood_fill(x+1,y); flood_fill(x,y-1); flood_fill(x,y+1); } }

Chapter 8 - 35 Interactive Computer Graphics

    

Scan-Line Approach

More efficient: use a scan-line rasterization algorithm For each y value, compute x intersections. Fill according to winding rule How to compute intersection points?

How to handle shading?

Some hardware can handle multiple scanlines in parallel

Chapter 8 - 36 Interactive Computer Graphics

   

Singularities

If a vertex lies on a scanline, does that count as 0, 1, or 2 crossings?

How to handle singularities?

One approach: don ’ t allow.

Perturb

vertex coordinates OpenGL ’ s approach: place pixel centers half way between integers (e.g. 3.5, 7.5), so scanlines never hit vertices

Chapter 8 - 37 Interactive Computer Graphics

 

Aliasing

How to render the line with reduced aliasing?

What to do when polygons share a pixel?

Chapter 8 - 38 Interactive Computer Graphics

   

Anti-Aliasing

Simplest approach: area-based weighting Fastest approach: averaging nearby pixels Most common approach: supersampling (compute four values per pixel and avg, e.g.) Best approach: weighting based on distance of pixel from center of line; Gaussian fall-off

Chapter 8 - 39 Interactive Computer Graphics

 

Temporal Aliasing

Need flicker

motion blur

for motion that doesn ’ t Common approach:

temporal supersampling

 render images at several times within frame time interval  average results

Chapter 8 - 40 Interactive Computer Graphics

   

Display Considerations

Color systems Color quantization Gamma correction Dithering and Halftoning

Chapter 8 - 41 Interactive Computer Graphics

Additive and Subtractive Color

Chapter 8 - 42 Interactive Computer Graphics

Common Color Models

Chapter 8 - 43

    

Color Systems

RGB YIQ CMYK HSV, HLS Chromaticity

Interactive Computer Graphics

 Color gamut

Chapter 8 - 44

   

HLS

Hue: “ direction ” of color: red, green, purple, etc.

Saturation: intensity. E.g. red vs. pink Lightness: how bright To the right: original, H, S, L

Interactive Computer Graphics

Chapter 8 - 45 Interactive Computer Graphics

      

YIQ

Used by NTSC TV Y = luma, same as black and white I = in-phase Q = quadrature The eye is more sensitive to blue-orange than purple-green, so more bandwidth is allotted Y = 4 MHz, I = 1.3 MHz, Q = 0.4 MHz, overall bandwidth 4.2 MHz Linear transformation from RBG:    Y = 0.299 R + 0.587 G + 0.114 B I = 0.596 R – 0.274 G – 0.321 B Q = 0.211 R – 0.523 G + 0.311 B

Chapter 8 - 46

   

Chromaticity

Color researchers often prefer chromaticity coordinates:    t1 = T1 / (T1 + T2 + T3) t2 = T2 / (T1 + T2 + T3) t3 = T3 / (T1 + T2 + T3) Thus, t1+t2+t3 = 1.0.

Use t1 and t2; t3 can be computed as 1-t1-t2 Chromaticity diagram uses XYZ color system based on human perception experiments    Y, luminance X, redness (roughly) Z, blueness (roughly)

Interactive Computer Graphics

Chapter 8 - 47 Interactive Computer Graphics

  

Color temperature

Compute color temperature by comparing chromaticity with that of an ideal black body radiator Color temperature is that were the headed black body radiator matches color of light source Higher temperatures are “ cooler ” colors – more green/blue; warmer colors (yellow-red) have lower temperatures CIE 1931 chromaticity diagram (wikipedia)

Chapter 8 - 48 Interactive Computer Graphics

 

Halftoning

How do you render a colored image when colors can only be on or off (e.g. inks, for print)?

Halftoning: dots of varying sizes  [But what if only fixed-sized pixels are available?]

Chapter 8 - 49 Interactive Computer Graphics

  

Dithering Dithering

(patterns of b/w or colored dots) used for computer screens OpenGL can dither But, patterns can be visible and bothersome. A better approach?

Chapter 8 - 50 Interactive Computer Graphics

Floyd-Steinberg Error Diffusion Dither

  Spread out “ error term ”  7/16 right    3/16 below left 5/16 below 1/16 below right Note that you can also do this for color images (dither a color image onto a fixed 256-color palette)

Chapter 8 - 51 Interactive Computer Graphics

  

Color Quantization

Color quantization: modifying a full-color image to render with a 256-color palette For a fixed palette (e.g. web-safe colors), can use closest available color, possibly with error-diffusion dither Algorithm for selecting an adaptive palette?

 E.g. Heckbert Median-Cut algorithm    Make a 3-D color histogram Recursively cut the color cube in half at a median Use average color from each resulting box

Chapter 8 - 52 Interactive Computer Graphics

     

Hardware Implementations

Pipeline architecture for speed (but what about latency?) Originally, whole pipeline on CPU Later, back-end on graphics card Then, whole pipeline in hardware on graphics card Then, parts of pipeline done on GPUs Now, hardware pipeline is gone—done in software on a large number of GPUs

Chapter 8 - 53 Interactive Computer Graphics

  

Future Architectures?

20 years ago, performance of 1 M polygons per second cost millions    Performance limited by memory bandwidth Main component of price was lots of memory chips Now an iPhone 5 does 115 million triangles (1.8 billion texels) per second Fastest performance today achieved with many cores on each graphics chip (and several chips in parallel) How to use parallel graphics cores/chips?

   All vertices go through vertex shaders – can do in parallel All fragments go through fragment shaders – can do in parallel Resulting fragments combined into a single image – easy with sufficient memory bandwidth

Chapter 8 - 54 Interactive Computer Graphics



NVidia GeForce GTX 590

Facts    “Most powerful graphics card ever built” (of 2011) Dual 512-core GTX 500 GPUs (1024 CUDA unified shaders) 2 x 16 Streaming Multiprocessor (SM) processor groups, each with 32 Streaming Processors (SPs)        PhysX physics engine on the card (used for flowing clothing, smoke, debris, turbulence, etc) Memory bandwidth: 327.7 GB/sec Texture fill rate: 77.7 billion/sec 4X antialiasing 3 billion transistors x 2 Real-time ray tracing Power: 365 w