Transcript Download presentation source

Modeling and Rendering
Architecture from Photographs
A hybrid geometry- and image-based approach
Debevec, Taylor, and Malik
SIGGRAPH 96
Presented by David Luebke
David Luebke
11-17-98
Overview
The Problem and the Idea
 Background
 Model Representation and
Reconstruction
 View-dependent Texture Mapping
 Model-based Stereo
 Conclusion and Discussion

David Luebke
11-17-98
The Problem
Architectural walkthroughs and
flybys are an important application
 Creating detailed models is hard

– Start with blueprints (if they exist…)
– Survey an existing building

Resulting systems don’t look great
– Hard to get all the details
– Hard to get realistic exteriors
David Luebke
11-17-98
The Idea
Wanted: a system to generate
realistic architectural scenes
 Idea: Model and render from photos!

– Take a few widely spaced photographs
– Build simple underlying model of scene
– Use correspondences between photos
to adjust scene parameters
– Paste photos back onto simple
geometry of scene for realistic façade
David Luebke
11-17-98
Background
Computer vision: recover 3D
geometry from 2D images
 Debevec uses some CV concepts:

– Camera calibration: simplify problem
by finding exact pixel  ray mappings
– Structure from motion and stereo
correspondence: triangulating for depth
– Image-based rendering: given image &
depth map, re-render from other views
David Luebke
11-17-98
Photogrammetric Modeling
Extracting 3D surfaces from multiple
images is hard
 Constrain the problem:

– User builds a simple notional model
using blocks: primitive solid shapes

Example: boxes, wedges, prisms, frusta
– User marks correspondences between
images and model
– System fits model to images
David Luebke
11-17-98
Photogrammetric Modeling

Now system need only solve
parameters of blocks!
– Height, width, translation, rotation, etc.
David Luebke
11-17-98
Photogrammetric Modeling

Even better: build in architectural
constraints!
– Roof prism lies flush on building block
– Stacked tower blocks share center axis
David Luebke
11-17-98
Photogrammetric Modeling
Knowns: image  block edge
correspondences
 Unknowns: block parameters,
camera position/orientation
 Constraints reduce # unknowns
 Generally, # correspondences must
equal # unknowns for reconstruction

David Luebke
11-17-98
Photogrammetric Modeling
Represent block parameters as
instances of shared variables
 Lots of math…

– Tweaking model edges to correspond
to recovered edges
– Computing an initial estimate
David Luebke
11-17-98
Photogrammetric Modeling

Results:
David Luebke
11-17-98
Photogrammetric Modeling

Results:
David Luebke
11-17-98
David Luebke
11-17-98
Photogrammetric Modeling
David Luebke
11-17-98
View-Dependent
Texture Mapping
Given the model, treat each camera
position as a “slide projector”
 Some images overlap!

– Idea: pick image taken from viewpoint
closest to desired rendering viewpoint
– Better: use weighted average (Fig 12)
David Luebke
11-17-98
View-Dependent
Texture Mapping
Best: Do view-dependent texture
mapping on per-pixel basis
 Okay: Do it on a per-face basis

– Subdivide large faces
– Use texture hardware!
David Luebke
11-17-98
View-Dependent
Texture Mapping
David Luebke
11-17-98
Model-Based Stereo

Problem: fine architectural details
still not captured
– recessed windows, friezes, cornices

Stereo depth extraction can help!
– Problem: when images are taken from
distant viewpoints, corresponding pixel
neighborhoods can look very different
David Luebke
11-17-98
Model-Based Stereo

Key observation:
– Even though two images of the same
scene may look very different, they
look similar after being projected onto
the approximate model.
– Idea: Warp offset image by projecting
onto the approximate model and rerendering
– Use McMillian warp to render imagewith-depth from novel viewpoints
David Luebke
11-17-98
Conclusion and Discussion
Results speak for themselves
 What problems do you see?

David Luebke
11-17-98