Transcript Slide 1

Flexible Voxels for Motion-Aware Videography
Mohit Gupta Amit Agrawal
Motion-Aware Camera
Ashok Veeraraghavan
Srinivasa G. Narasimhan
Why is Building Such a Camera Hard?
Low SR, High TR
High SR, Low TR
High Frame Rate
(500 fps) for Fast
Moving Regions
Sampling of the Space-Time Volume
Conventional Sampling Scheme:
Motion Aware Video
Sensor Plane
High Spatial
Resolution (1 MP)
for Static Regions
y
Frame 1 Frame 2
Frame N
Camera
t
Integration Time
Time
x
Our Sampling Scheme:
Sensor Plane
Fat and short voxels
Thin and long voxels
Post-Capture control of frame rate and
spatial resolution
Frame 1 Frame 2
Frame N
Flexible voxels
Pixel on
Motion-Aware video requires flexible voxels and
a priori knowledge of the scene
Pixel off
Camera
Integration Time
Time
Post Capture Motion-Aware Interpretation of Data
Captured Data
Different Spatio-temporal Interpretations
Diffusion Kernels
High TR
Space
High SR
Time
Re-binning
Captured Frame
TR = 1X
SR = 1X
TR = 2X
SR = ½ X
TR = 4X
SR = ¼X
High SR
High TR
Optical Flow
Red = Fast, Green = Slow,
Blue = Stationary
Reduced Aliasing
Increasing Temporal Resolution
Anisotropic diffusion
Simple re-binning results in aliasing artifacts
Motion-Aware Video Results
Fan Rotating (‘Wagon-wheel effect’)
Camera Integration
Projector Pattern
Coded Motion Blur
Beam
Splitter
Motion-Aware
Reconstruction
Motion-Aware reconstruction using motion information
Hardware Implementation: Fast Per-pixel Shutter
Scene
Diffusion Kernels Aligned
Along Flow Directions
Image Plane
High TR
High SR
Pixel 1
Multi-Use Light
Engine (MULE)
Pixel 2
Projector
Image
Plane
Pixel K
Camera
Co-located projector camera setup
Pico-Projector
(1440Hz.) ($350)
[Mukaigawa et al]
Time
Fast optical modulation
Future work: Passive
implementation using LCoS
Input Frame (Captured at 7.5 FPS)
Reconstructed Frame (60 FPS)
(1 out of 8)