Transcript Heuristic 3D Reconstruction of Irregular LIDAR

Autonomous 3D Reconstruction From
Irregular LiDAR and Aerial Imagery
Nicholas Shorter
BSEE, May 2005; MSEE, Aug. 2006
PhD Committee: Dr. Takis Kasparis (Chair)
Dr. Georgios Anagnostopoulos,
Dr. Michael Georgiopoulos, Dr. Andy Lee,
Dr. Wasfy Mikhael
Email: [email protected]
Website: http://www.nshorter.com
http://www.nshorter.com
1
Presentation Layout
1.
2.
3.
4.
5.
6.
7.
8.
LIDAR Overview
Problem Statement
Present Art
Proposed Solution
Completed Work
Future Work
Potential Implementation Obstacles
Milestones
http://www.nshorter.com
2
LIDAR Overview
• Data Collection
– Plane Equipped with GPS, INS & LIDAR
– LIDAR – Light Detection and Ranging (active sensor)
• Collection of 3D points
• Laser sent out from Emitter, reflects off of Terrain, Returns to
Receiver
– Receiver measures back scattered electromagnetic radiation
(laser intensity)
• Time Difference Determines Range to Target
http://www.toposys.com/
http://www.nshorter.com
3
LIDAR Captured Characteristics
• Range to Target (elevation from INS &
LiDAR)
• Longitude and Latitude (GPS)
• First and Last Return Pulses
– First – shrubbery, vegetation, power lines, birds
and buildings
– Last – buildings (unless vegetation is really
dense, then vegetation too)
• Returned Laser Intensity
http://www.nshorter.com
4
LiDAR Noise
• Geolocation results from LiDAR, GPS and
INS sensor systems
– Accuracy Limitations
– Offset and Drift in both GPS and INS
– Misalignment between INS and LiDAR
• Atmosphere – Intensity and Path Distortion
• Shadowing Effect from Tall buildings
• Artifacts from non uniform sampling from
multiple strips
http://www.nshorter.com
5
Problem Statement
• Input
– LIDAR (Light Detection and Ranging) Data
• Collection of Irregularly Distributed 3-Dimensional
Points
– Aerial Photograph
• Output
– Semi to complete Automatic (minimal user
intervention) development of 3-Dimensional
Virtual Model
http://www.nshorter.com
6
Applications for 3D Reconstruction
• Military Applications
– Automatic Target Recognition
– Reconstructed Models of Opponent Terrain
(UAV?)
• Tourism/Entertainment
– Virtual Walkthrough of Theme Park
• Commercial
– Change Detection (Natural Disasters)
– Network Planning for Mobile Communication
– Noise Nuisance (Universal Studios, 408
Expressway - walls)
– Urban Planning
http://www.nshorter.com
7
Ideal Algorithm Features
• Ideal Attributes
– Handle Complex (non
planar) buildings
– Irregular Points
– Multiple Source as
Available
– Reconstruction exists as
membered simple entities
– Building Isolation
– Complete Automation
– Aerial Image Projection
http://www.nshorter.com
• Dissertation Implementation
– Yes
– Yes
– Only LiDAR and Aerial
Imagery
– Yes
– Yes
– Yes
– Yes
8
Existing 3D Reconstruction Research
• Data Sources
– LIDAR
– Aerial Imagery
– GIS Ground Plans
• Model Based Reconstruction
– Pre-defined models with parameters
– Minimize error between models and data
• Data Driven
– Group Coplanar Pts
– Identify Break Lines
– Derive Model to Minimize Error
http://www.nshorter.com
9
Triangulation Based Methods
• Triangular Irregular Network (TIN)
– Series of Non-Overlapping Triangles Modeling
given Surface
• TIN 3D Reconstruction Methods
– Clustering approach
• Spherical Normal Vectors of Triangles
– TIN region growing approach
• Merge Triangles to Same Region if Normal Vectors
within Threshold
http://www.nshorter.com
10
Existing 3D Reconstruction Methods
• Most still under development
• Most Methods Use ‘Grided’ (Interpolated)
LIDAR Data
– Advantages
• Less Computationally Complex
• DTM & DSM Thresholding to distinguish Building from Non
Building
• Use of additional conventional methods
– Disadvantages
• Decrease in Accuracy
• Uncertainty from Building and Ground Interpolation
http://www.nshorter.com
11
Masters Research
• Greedy Insertion Triangulation
– Implemented Noise Filtering Technique
• Proposed FSART normal vector clustering
• Proposed Planar Regression to combat
Category Proliferation
• Realized simple planar reconstruction
algorithm (MSEE Thesis)
http://www.nshorter.com
12
Post Masters, Pre-Candidacy Research
• Implemented FA, GA, and FSART
Clustering for reconstruction performance
comparison
• Developed Category Proliferation and
Clustering Performance measures
• Ran comparison on 4 buildings
http://www.nshorter.com
13
Proposed Algorithm – System Level
• Anticipated Challenges:
– Automatic Image and LiDAR Registration
– Matlab Rendering Reconstructed City Blocks
– Matlab mapping Images to Models
http://www.nshorter.com
14
Building Detection
• Anticipated Challenges:
– Morphological Filtering needs a priori window
size
http://www.nshorter.com
15
Wall Tri. And Gnd Pt. Identification
• Wall Triangle = (Max diff. in elev. > 2m) &
(pitch > 60 degrees)
• Gnd Pt. = Wall Tri. Lowest elevation pt
http://www.nshorter.com
16
Building Extraction
• Clustering will automatically identify and
separate individual buildings
http://www.nshorter.com
17
Building Features
• Vegetation Points
– Significant first and last return elevation diff.
– Corresponding green aerial image color
– Nearest points have diff. elev. or adjacent triangles
have significantly diff. norm vectors
• Common Building Points
–
–
–
–
Spatially close in terms of long. and lat.
Bounded by aerial img. edges and exterior wall tri.
Bounded building does not contain terrain pts – or Triangulation of all points, building points connected
http://www.nshorter.com
18
Building Reconstruction
http://www.nshorter.com
19
Building Reconstruct Cluster Features
• Surface change - normal vector orientation
difference between adjacent triangles
• (X,Y,Z) - Longitude, Latitude, Elevation
• Edge - Aerial imagery edge detection
• Color - Aerial imagery corresponding color
(building surface differs from clutter)
• Triangle planar coef, pt. height diff., same normal
vector, or planar equation
• Feedback – difference between reconstruction and
raw LiDAR and Aerial Image Edges
http://www.nshorter.com
20
Algorithm Novelty
• Novel Building Detection Method– Irregular
LiDAR
– Uses raw pts and triangulation
• Novel Automated Building Extraction
– Clustering for automated extraction
• Novel Clustering for Reconstruction
– Capable of handling complex building structure
– Proposing Clustering instead of Existing (ART, KMeans, etc.)
• Automated LiDAR and Image Registration
http://www.nshorter.com
21
Implementation Obstacles
• Matlab Obstacles
– Matlab Rendering of Large City Sets
– Matlab displaying Large City Models with
Mapped Images
• Will have to program Open GL or other
rendering solution (learning curve)
• Additional reader development for different
data sources (Fairfield mostly simple
buildings)
http://www.nshorter.com
22
Future Tasks and Milestones – Su 07
• Investigate systematic noise removal
• Continue Debugging and Testing of GIT for
150k+ pts
– Currently works for 10k pts in > 10 seconds
• Development of Building Detection
– Investigate no a-priori max build size for
morphological filtering
http://www.nshorter.com
23
Future Tasks and Milestones – Fa 07
• Develop Automatic LiDAR & Image
Registration
• Development of Building Extraction
– Investigate which clustering approach
• Publish Detection and Extraction Procedure
http://www.nshorter.com
24
Future Tasks and Milestones – Spr 08
• Develop Reconstruction Algorithm
– Propose Clustering Method
• Automatically map images to constructed
models
• Develop OpenGL to render multiple
building subsets (neighborhood block sizes)
http://www.nshorter.com
25
Future Tasks and Milestones – Su 08
• Publish Reconstruction Algorithm
• Write Dissertation (~July 21 deadline)
• Defend Dissertation (~July 10 deadline)
http://www.nshorter.com
26
Acknowledgements
• Data Contributors
–
–
–
–
Dr. Simone Clode, Dr. Franz Rottensteiner, AAMHatch
Mr. John Ellis, AeroMap
Mr. Steffen Firchau, TopoSys
Mr. Paul Mrstik, Terra Point
• Advisor (Committee Chair)
– Dr. Takis Kasparis
• Committee Members
– Dr. Michael Georgiopoulos , Dr. Georgios
Anagnostopoulos, Dr. Andy Lee, and Dr. Wasfy
Mikhael
http://www.nshorter.com
27
Accomplishments
• WSEAS International Conference on Systems
Theory and Computation – August 2006
• WSEAS Transactions on Signal Processing –
August 2006
• MSEE – August 2006
• Invited Harris Talk
• UCF Graduate Research Forum Poster Board
Presentation (Spring 2006)
• UCF Graduate Research Forum Oral Presentation
(Spring 2007)
http://www.nshorter.com
28
Thank You for your Attendance
Additional Questions?
http://www.nshorter.com
29