Transcript mahsam.ir
Kinect 3D reconstruction Mahsa Mohammadkhani Course project - CMPUT 499 Background • KinectFusion • o Microsoft SDK o o [1. Izadi S. et al.] [2. Newcombe, R. A. et al.] Reconstructme [3] << This is me. :) Motivation • KinectFusion and Reconstructme have: Complex methods o No texture o GPU implementation Working with high GPU processors o • • NVIDIA GeForce GTX 560 AMD Radeon HD 6850 Powerful GPUs are not available everywhere What I Wanted to Do ... • 3D reconstruction with Kinect • Make the reconstruction processes more • • simpler Decrease complexity, but processing fast enough Capturing some viewpoints of scene and object like an capturing images o Instead of capturing the whole views Method Steps 1. Input a. b. c. Grabbing the frames with i. OpenNI ii. Freenekt Depth Intensity RGB color image ( 640 x 480 ) Method Steps 2. Camera Calibration o + Depth and RGB calibration [4] Method Steps 3. Extracting RGB features (Detecting objects) o Detectors FAST (Features from Accelerated Segment Test) [5] • SIFT (Scale-invariant feature transform) [6] SURF (Speeded Up Robust Features) [7] • o Corner detector A robust local feature detector Descriptors SIFT • Describes local features SURF BRIEF (Binary Robust Independent Elementary Features)[8] Method Steps 4. Finding the closest view and best matches 5. Two optimization options for finding corresponding points 5.1 RANSAC (RANdom SAmple Consensus) 2D image-based & best matches 5.2 ICP (Iterative Closest Point) Point-based (PCL Library) 6. Computing Normals 7. Rendering 3D Point Clouds Technical Description • Building the framework based on o o o • PCL library [9] Nestk (RGBDemo) [10][11] OpenCV Capturing data with kinect Results • Framework features: 3D Reconstruction (Scene or object) Capturing the whole view Capturing some view points o Object reconstruction Detect object Segment the object in 3D view o • Comparing Feature matching options Optimizers Result of Comparing Detector Descriptor Result FAST BRIEF Slow / Not complete reconstruction / Wrong matching SURF SURF Fast / Good result SIFT SIFT Slow / Not complete reconstruction SIFT/FAST SURF Slow / Not complete reconstruction SURF BRIEF Slow / Wrong matching Input For Scene Reconstruction Comparing Feature Matching • FAST - BRIEF (Not complete reconstruction) Comparing Feature Matching SURF- BRIEF (RANSAC) (Wrong matching) ICP vs. RANSAC • RANSAC (SURF- SURF) (Not good result) ICP vs. RANSAC • ICP (SURF) - Good Result Obj Reconstruction • Video References (1) [1] Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, and Andrew Fitzgibbon. 2011. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA, 559-568. [2] Newcombe, Richard A.; Davison, Andrew J.; Izadi, Shahram; Kohli, Pushmeet; Hilliges, Otmar; Shotton, Jamie; Molyneaux, David; Hodges, Steve; Kim, David; Fitzgibbon, Andrew; "KinectFusion: Real-time dense surface mapping and tracking," Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on , vol., no., pp.127-136, 26-29 Oct. 2011. [3] http://Reconstructme.net [4] Herrera C., Daniel; Kannala, Juho; Heikkilä, Janne, "Joint Depth and Color Camera Calibration with Distortion Correction," Pattern Analysis and Machine Intelligence, IEEE Transactions on , vol.34, no.10, pp.2058,2064, Oct. 2012. [5] E. Rosten and T. Drummond (May 2006). "Machine learning for high-speed corner detection,". European Conference on Computer Vision. References (2) [6] Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision. 2. pp. 1150–1157. [7] Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346--359, 2008. [8] Michael Calonder, Vincent Lepetit, Christoph Strecha, and Pascal Fua. 2010. BRIEF: binary robust independent elementary features. In Proceedings of the 11th European conference on Computer vision: Part IV (ECCV'10), Kostas Daniilidis, Petros Maragos, and Nikos Paragios (Eds.). Springer-Verlag, Berlin, Heidelberg, 778-792. [9] http://pointclouds.org/ [10] http://nicolas.burrus.name/index.php/Research/KinectUseNestk [11] http://labs.manctl.com/rgbdemo/ Thank you :)