Transcript 3D Reconstruction - University of Tennessee

Imaging, Robotics, and Intelligent Systems
IRIS
Large-Scale 3D Terrain Modeling
David L. Page
Mongi A. Abidi, Andreas F. Koschan
Sophie Voisin, Sreenivas Rangan, Brad Grinstead, Wei Hao, Muharrem Mercimek
Imaging, Robotics, & Intelligent Systems Laboratory
The University of Tennessee
March 23, 2004
March 23, 2006 Slide 2
Imaging, Robotics, and Intelligent Systems
Outline
• 3D Terrain Modeling
–
–
–
–
–
UTK mobile terrain scanning system
Simulation needs and Army benefit
Scanning system pipeline
“Knoxville Proving Grounds”
Research problems
IRIS
March 23, 2006 Slide 3
UTK Mobile Terrain Scanning
System
Imaging, Robotics, and Intelligent Systems
IRIS
Multi-sensor data collection system for road surface.
Video
Camera
GPS
Receiver
GPS Base
Station
3D Range
Sensor
3-Axis IMU and
Computer
March 23, 2006 Slide 4
Imaging, Robotics, and Intelligent Systems
Data Acquisition
3
IRIS
Actual Path
1
4
7
5
2
8
6
1 – Riegl LMS-Z210 Laser
Range Scanner
2 – SICK LMS 220 Laser
Range Scanner
3 – JVC GR-HD1 High
Definition Camcorder
4 – Leica GPS500 D-RTK
Global Positioning
System
5 – XSens MT9 Inertial
Measurement Unit
6 – CPU for acquiring
SICK, GPS, and IMU
data
7 – CPU for acquiring
Riegl data
8 – Power system
Scanned Path
Modular System
Mounted here on a push cart.
Geo-referenced geometric 3D model of an
area near IRIS West in Knoxville.
March 23, 2006 Slide 5
Imaging, Robotics, and Intelligent Systems
IRIS
3D View of Terrain
(Jump to 3D Viewer)
March 23, 2006 Slide 6
Imaging, Robotics, and Intelligent Systems
Outline
• 3D Terrain Modeling
–
–
–
–
–
–
UTK mobile terrain scanning system
Simulation needs and Army benefit
Scanning system pipeline
“Knoxville Proving Grounds”
Research problems
Static scanning
IRIS
March 23, 2006 Slide 7
Simulation Needs for Terrain
Modeling
Imaging, Robotics, and Intelligent Systems
IRIS
Why needed, in general?
• Visualization
– Typical terrains only
available in 30x30 m2 grids
– Probably sufficient with
bump mapping
Bump Mapping
• System analysis
– Requires high-resolution
terrains!
– Multi-body dynamics
– Linear analysis, PSD
• Time series analysis
– Requires high-resolution
terrains!
– Multi-body dynamics
– Motion stands
Discussions with Dr. Al Reid
March 23, 2006 Slide 8
Imaging, Robotics, and Intelligent Systems
Benefit to U.S. Army
• Scanning 3D terrains is a significant
enhancement over traditional towedcart profiling, cart dynamics, 1D
profile, etc.
• Real terrain modeling overcomes
potential limitations of linearity,
stationarity, and normality
assumptions, particularly associated
with PSD (Chaika & Gorsich 2004).
• Research in 3D processing (tools!)
addresses relevant issues in…
– data reduction (Al Reid),
– terrain analysis (3D EMD),
– interpolation, etc.
IRIS
March 23, 2006 Slide 9
Imaging, Robotics, and Intelligent Systems
Profilometers
• Four (4) wheel trailer
• Drawn by a tow vehicle
• Front axle free to rotate about yaw
axis (other constrained)
• Linkage to draw bar of tow vehicle
• Rear axle free to rotate about roll
axis (other constrained)
• No compliant suspension components
between axles and frame
• Inertial gyroscope measures pitch
and roll angle
• Ultrasonic measurement between
axle and terrain (always points
down)
• Shaft encoder every 0.1 in. of travel
• Data acquisitions every 3 inches
Highly correlated sensor data
(GPS, IMU, Range) = Correction
for vehicle dynamics
IRIS
Towed Trailer Profilometer
UTK 3D Terrain Modeling
March 23, 2006 Slide 10
Imaging, Robotics, and Intelligent Systems
Comparison to Profilometer
Path Overlaid on
Aerial View
IRIS
3D vs. 1D
• 120-360 profiles over a
2-8 m swath (3D
surface) vs. 1 profile (1D
signal)
• Correlated data vs.
trailer dynamics
• Agile path vs. linear path
(?)
Zoom View
2 m wide x 8 m length
Path is 300 m length
+/- 0.5 cm resolution
Video Data of Zoom
Notice Cracks in Pavement
March 23, 2006 Slide 11
Imaging, Robotics, and Intelligent Systems
Outline
• 3D Terrain Modeling
–
–
–
–
–
UTK mobile terrain scanning system
Simulation needs and Army benefit
Scanning system pipeline
“Knoxville Proving Grounds”
Research problems
IRIS
March 23, 2006 Slide 12
Imaging, Robotics, and Intelligent Systems
IRIS
System Block Diagram
3D range sensors
RIEGL
SICK
Range
Profiles
Position and orientation sensors
IVP
Leica -GPS
Xsens IMU
3D Position and
Orientation
Inter-profile
Alignment
Visual
Thermal
Sony
Indigo
Video
Sequence
Multi-sensor
Alignment
Multi-modal
Data Integration
Multi-sensor
Visualization
March 23, 2006 Slide 13
Imaging, Robotics, and Intelligent Systems
UTK IRIS Lab 3D Sensors
Genex 3D CAM
IVP RANGER SC-386
Sheet-of-light triangulation-based system
SICK LMS200
Time-of-flight
Structured-light stereo system
Principle of operation
IRIS
X
Laser

Camera
r( s )  B
f tan  - s
f  s tan 
x’
x
c
S1
3D Rendering
c’
S1 and S2 are two sensors.
S2
S1
r  s * t / 2
March 23, 2006 Slide 14
Imaging, Robotics, and Intelligent Systems
Statistical Modeling of Sensors
Roll Measurements
Standard Deviation = 0.0336
Pitch Measurements
Standard Deviation = 0.0338
IRIS
Yaw Measurements
Standard Deviation = 0.0492
Extensive GPS and IMU error characterization and modeling.
March 23, 2006 Slide 15
Imaging, Robotics, and Intelligent Systems
Outline
• 3D Terrain Modeling
–
–
–
–
–
UTK mobile terrain scanning system
Simulation needs and Army benefit
Scanning system pipeline
“Knoxville Proving Grounds”
Research problems
IRIS
March 23, 2006 Slide 16
Imaging, Robotics, and Intelligent Systems
“Knoxville Proving Grounds”
IRIS
Visualization tool built to be able to visualize “z” measurements
Blue Line is the GPS Path for the loops that we
collected.
Cornerstone Drive, off Lovell Road, I-40 Exit #374
Knoxville, Tennessee, Knoxville
Each loop a length of 1.1 mile, Total distance covered on
scanning that day = 2.2 miles ( 2 times) = 4.4 miles of the
same data.
The color of the GPS path encodes the height of the terrain.
Over 4 miles = ~2 GB of data
March 23, 2006 Slide 17
Imaging, Robotics, and Intelligent Systems
Data Collection
IRIS
Automated correction for varying speeds and
dynamics of platform.
March 23, 2006 Slide 18
Imaging, Robotics, and Intelligent Systems
Elevation Change of Terrain
IRIS
17 m
17 m
0m
Pathways – Loop scanning
Full length scanning
0m
March 23, 2006 Slide 19
Imaging, Robotics, and Intelligent Systems
High Accuracy 3D Terrain
IRIS
Full Data
~10 km
Zoom
~1 km
Aerial View
Zoom
~10 m
March 23, 2006 Slide 20
Imaging, Robotics, and Intelligent Systems
Triangulated Terrain Mesh
IRIS
The entire stretch,
1.8 meters
March 23, 2006 Slide 21
Imaging, Robotics, and Intelligent Systems
IRIS
Y
Campus Loop
Latitude and Longitude
Measurements
from the Leica DGPS
Raw Point Cloud
March 23, 2006 Slide 22
Imaging, Robotics, and Intelligent Systems
Outline
• 3D Terrain Modeling
–
–
–
–
–
UTK mobile terrain scanning system
Simulation needs and Army benefit
Scanning system pipeline
“Knoxville Proving Grounds”
Research problems
IRIS
March 23, 2006 Slide 23
Interprofile Registration
Problem
Pt  [ xt g , yt g , z t g ]T
Imaging, Robotics, and Intelligent Systems
IRIS
Vehicle (Scanning) Direction
GPS curve sampled
at 10 Hz.
IMU data @ 100 Hz
(,  ,  )
Video recorded at 30
frames/sec
Dt  [ x t r , y t r , z t r ]T
Range Profiles @
30 Hz 4m wide SICK
2000 Hz and 50cms wide IVP
Rt Dt  Pt  Wt
Raw Data
March 23, 2006 Slide 24
Imaging, Robotics, and Intelligent Systems
Data Interpolation
 γ(d11 )  γ(d1n )
 



γ(d n1 )  γ(d nn )

1
1
 1
IRIS
1 W1  γ(d1 p ) 
1      

1 Wn  γ(d np )
  

0  λ   1 
Correct for non-uniform data collection with terrain modeling.
March 23, 2006 Slide 25
Imaging, Robotics, and Intelligent Systems
IRIS
Pose Localization
Video Sequence
Feature Matching
Pose From Motion
R, T
RANSAC Filtering
N  ceil (log( 1  Γ ) / log( 1  (1  ε) p ))
Oriented Tracks Filtering

x  Xi
1 n
pdf ( x) 
)
 K(
nh i 1
h
GPS drop-outs under certain conditions.
Improve overall localization accuracy.
March 23, 2006 Slide 26
Imaging, Robotics, and Intelligent Systems
Data Reduction
IRIS
Noise Removal
Initial Model
Multiresolution Analysis and Denoising
Adaptive Simplification
Original model
Reduced to 25%
Reduced to 2.5%
363843 triangles
185345 points
90893 triangles
48595 points
9075 triangles
6642 points
March 23, 2006 Slide 27
Imaging, Robotics, and Intelligent Systems
Statistical Modeling of Terrain
IRIS
Dataset from near IRIS West
Reconstructed 3D profile
from the statistical model
The total length of the patch: 20 meters with inter-profile
spcaing around 1 cm.
Mean Longitudinal profile
The 3D terrain was generated using our system
mounted on a van.
The profile is non-linear and non-stationary but all
the IMF’s taken separately are linear and stationary,
which means the PSD of the IMF’s model the data
better than the PSD of the profile alone.
Empirical mode decomposition of the terrain sample shown above.
EMD implementation : Modified Brad’s functions
March 23, 2006 Slide 28
Temporal-Based Stereo
Tire-Soil Terrain Modeling
Camera
Calibration
Imaging, Robotics, and Intelligent Systems
IRIS
Calibration
Image
Rectification
Dense
Matching
Disparity
Estimation
Pipeline of 3D
Reconstruction
Triangulation &
Visualization



1



E[ d ]   c ( x , t , d ) 
e
(
d
x
,
t

d
x
,t )
1 1
2 2




2  x ,t ~ x ,t 
 x ,t 
1 1
2
2
Test Setup
Disparity Map
Input
March 23, 2006 Slide 29
Imaging, Robotics, and Intelligent Systems
3D Model of Military Tire
Tire 150 cm dia., 30
cm width
IRIS
Model Integration
(+/- 0.5 mm)
Registration
(18 Sections, 7 Views)
Final Model
March 23, 2006 Slide 30
Imaging, Robotics, and Intelligent Systems
Questions?
IRIS
17 m
17 m
z (m)
0m
0m
Pathways – Loop scanning