Graphics Hardware 2004 Grenoble, France

PixelView

A View-Independent Graphics Rendering Architecture

Jason Stewart, Eric P. Bennett, and Leonard McMillan Presented by Anselmo Lastra University of North Carolina at Chapel Hill The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Why View-Independence?

Decouples Rendering from Viewing Eliminates latency Provides uniform framerates Allows increased shading complexity Needed for future applications Shared multi-user virtual environments True three-dimensional (Autostereo) displays The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

True 3-Dimensional Displays

Promising 3-D Display Technologies Lenticular, and fly’s-eye optics Barrier-based methods Reflective optics Holographic optics Technology is Maturing Problem: How to generate the content?

Requires 1000’s of simultaneous views

Benton’s holographic 3D display Matusik’s lenticular 3D display Isaksen’s fly’s-eye display

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Using Today’s Architecture

I guess you could… buy 1024 GPUs 10 years of Moore’s law would yield… 4 doublings in performance (only need 64 GPUs) At least 2 doublings in power (only needs 10 KW) There has to be a better way Traditional graphics architectures are inefficient for view-independent graphics Inefficenc y



more power wasted resources The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Previous Work

Low Latency Rendering 3D Light field Viewing H/W [Regan 99] Frameless Redering [Bishop 94 ] Just-in-time Pixels [Mine & Bishop 93] View-Independent Rendering Multiple viewpoint rendering [Halle 98] 4D Parameterization Light Field [Levoy & Hanrahan 96], Lumigraph [Gortler 96] Micropolygon Rasterization Reyes [Cook 87] Reyes streaming H/W pipeline [Owens 02] The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Rearchitecting the Pipeline

Classic “view-dependent” pipeline View Specification Sampling Geometry Vertex Processing Rasterize Fragment Processing Visibility 2D Framebuffer Scanout The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Rearchitecting the Pipeline

Proposed “view-independent” pipeline Host PC Geometry Sampling Subdivide into points Fragment Processing Point Casting Scatter Hardware Prototype 4D Framebuffer Scanout View Specification The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView Prototype

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView Prototype

100 Mhz XILINX SpartanII-E FPGA 300k Gates 16MB 100MHz SDRAM 5000 lines of Verilog 6000 lines of C# on host PC The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView Demo

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

System Partitioning

The prototype pipeline implementation Host PC Geometry Sampling Subdivide into points Fragment Processing Point Casting Scatter Hardware Prototype 4D Framebuffer Scanout View Specification The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView 4D Framebuffer

16 MBytes of Framebuffer memory Reconfigurable (ex. 8x8x256x256x(rgb + z)) 16 bits (5/6/5) rgb 16 bits z The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

View Selection

Every view is just a 2D planar slice through the 4D framebuffer Which, after some simplifying assumptions, reduces to:

t s v

( (

i i

,

u

(

i

, (

i

, ,

j j j

) )

j

) )

   

A



s i



B s



j



C s



A t



A u



A v



i i i

  

B t



B u



B v



j j j

  

C t



C C u



v



The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Linear Expression Evaluators

Simple datapath replicated for each of s, t, u, and v Pixel rate Trivial H/W cost

ACC row ACC col A i

Easy to parallelize

+

Drop in replacement for traditional scanout

B i

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Memory Access Patterns

For each view the LEE generates: s(i,j), t(i,j), u(i,j), v(i,j)

S (5.7 format) T (5.7 format) U (10.6 format) V (10.6 format)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Scan out Performance

Typical < 10% Memory bandwidth required for scan out 640x480 VGA 100 MHz SDRAM, and order of magnitude behind the state of the art (DDR @ 500MHz) Can easily support multiple simultaneous “views” The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Filling the Framebuffer

Elemental Rendering Primitive is the Outgoing Radiance from a Point The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Point Casting

Instead of the natural “planar” radiance parameterization about each point We align with parameterization planes Simplifies mapping

q

(

s

)



A Pq

(

x

,

y

,

z

)

s



B Pq

(

x

,

y

,

z

)

r

(

t

)



A Pr

(

x

,

y

,

z

)

t



B Pr

(

x

,

y

,

z

) The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

In Other Words…

Parameterize outgoing radiance on fixed planes and resample it.

q

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Unexplored View Coherence

Outgoing radiance from a point is smoother than spatial variations Today’s architectures do not exploit this Still ample spatial coherence We support 2 formats Uniform color Spatially varying The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Unexplored Coherence

Outgoing radiance from a point is smoother than spatial variations Today’s architectures do not exploit this

Occlusion Lambertian View-dependency

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Geometry Subdivision

Subdivide until primitive is “point-sized” Backward compatibility with polygons Reminiscent of Reyes rendering pipeline Every primitive requires a world-space subdivision method However, Reyes subdivision is view-dependent (the stopping criterion is based on pixel grid) Probably better methods The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Prototype Limitations

Points are transferred via USB 1.1

Achieved 80,000 points/second (which means 5,120,000 rays/second) Each pointcast requires at least 64 reads Requires



17% of memory B/W Could easily include 4 or more parallel point-casting units Entire design uses



23% of chip The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

A Practical PixelView System

The prototype demonstrates feasibility, but what would a real system entail?

Improve: Scalability Field of View Subdivision Techniques Output Bandwidth The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Distributing the Frame Buffer

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Expanding Field of View

6 Slabs @ 64x64x1024x1024x(8+8) = 64 GBytes But … 1MB was huge for a 64k PDP-11 The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Addressing Output Bandwidth

Currently we can support only a handful of dynamic views out of the framebuffer An Autostereoscopic Displays would require every pixel on every frame High-speed interconnects are available >5GHz per pin without compression The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Improving Subdivision

Our conservative subdivision methods oversample by a factor of 4 or more after factoring out depth complexity Fast, on-the-fly, hardware friendly, uniform subdivision would be great The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Conclusions

PixelView simultaneously supports low latency and complex shading PixelView supports a wide range of primitives and IBR data structures PixelView is scalable to: A full field of view High resolutions Multi-user environments PixelView can power the next generation of display technologies The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Thank You – Questions?

PixelView

A View-Independent Graphics Rendering Architecture

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL