Image-based Water Surface Reconstruction with Refractive Stereo

Nigel Morris University of Toronto

Motivation

 Computational Fluid Dynamics are extremely complex and difficult to simulate  Why not capture fluid effects from reality?

 We present the first step to capturing fluids from reality – reconstructing water surfaces  May eventually be useful for determining fluid flow

Previous Work

 Shape from shading [Schultz94]  Requires large area light source or multiple views  Shape from refractive distortion [Murase90]  Limited wave amplitude, orthographic camera model  Laser range finders [Wu90]  Specialized equipment

Previous work

 Shape from refractive irradiance [J ähne92], [Zhang94] & [Daida95]  Requires underwater lens, orthographic camera model

Goals of our system

 Physically-consistent water surface reconstruction  Reconstruction of rapid sequences of flowing, shallow water  High reconstruction resolution  Use of a minimal number of viewpoints and props

Technical Contributions

 We present a design for a stereo system for capturing sequences of dynamic water  System implementation and results  Refractive stereo matching metrics and analysis  Effective localization of surface points of shallow water

Refraction

 Snell’s Law 

r 1 sin Θ i = r 2 sin Θ r

For air → water: 

sin Θ i = r w sin Θ r

Imaging water

   Image point

f

without water at

q

Image

f

at

q’

water with

qq’

is the refractive disparity

Deriving the surface normal

  Suppose we know the location of the surface point

p

and its depth from the camera

z

We know the angle rays

u

and

v

θ δ

between the refracted  Can compute the incident angle

θ i

, then the normal

n

:

Solution space

 For given refractive disparity, set of solution pairs: 

n

m z m

 For every depth

z

, there is at most one normal

n

Reconstruction with Stereo

 Same setup as with one camera, but with additional calibrated camera  We search through the <

n

m z m

> solution space for a particular refractive disparity  We use the second camera to determine the error for each instance of

n

m z m

 Return best surface point

p

Refractive stereo matching

Camera 2 Camera 1

n 2 n 1 n

Tank Bottom

Matching metric

 Normal collinearity metric   Measure the angle between the two normals

n 1

and

n 2

to give an error. Disparity difference metric  Swap

n 1

and

n 2

and reproject to tank plane, measure disparity from the projection before swapping.

 Seeks to minimize error due to inaccurate normal measurements as water depth approaches localization error range.

Disparity Difference Metric

Camera 2 Camera 1 Tank Bottom

e 1 e 2

Metric Comparison

 Disparity difference metric in red  Normal collinearity metric in blue

Implementation details

 Pattern choice  Checkered pattern used  Tracking pattern and localization  Lucas-Kanade matching  Interpolation of the discrete pattern

System Inputs

 Calibrated stereo camera system  Images of pattern without water from both cameras to give refractive disparities  Distorted pattern image sequences

Corner tracking

 In order to reconstruct a sequence of frames, the corners must be localized at every frame  We employ a Lucas-Kanade matching technique, matching templates of the corners to the next frame

Corner Interpolation

    We cannot assume that our verification ray will land on one of the corners We thus find the four nearest non-collinear corners The surface may be distorted so we cannot assume a grid formation We interpolate between these corners to find the distortion of the verification ray

Results

 Ripple Drop  Waves  Pouring water

Future Work

 Global surface minimization vs local  Planar tank constraint removal  More complex water scenario capturing

References

 [J ähne92] B. Jähne, J. Klinke, P. Geissler, and F. Hering. Image sequence analysis of ocean wind waves. In Proc. International Seminar on Imaging in Transport Processes, 1992.

 [Murase90] H. Murase. Shape reconstruction of an undulating transparent object. In Proc. IEEE Intl. Conf. Computer Vision, pages 313 –317, 1990.

 [Schultz94] H. Schultz. Retrieving shape information from multiple images of a specular surface. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2):195 –201, 1994.

 [Wu90] Z. Wu and G. A. Meadows. 2-D surface reconstruction of water waves. In Engineering in the Ocean Environment. Conference Proceedings, pages 416 –421, 1990.

 [Zhang94] X. Zhang and C. Cox. Measuring the two-dimensional structure of a wavy water surface optically: A surface gradient detector. Experiments in Fluids, Springer Verlag, 17:225 –237, 1994.