Transcript Document
Introduction to Computer Vision CS223B, Winter 2005 Richard Szeliski – Guest Lecturer • Ph. D., Carnegie Mellon, 1988 • Researcher, Cambridge Research Lab at DEC, 1990-1995 • Senior Researcher, Interactive Visual Media Group, Microsoft, 1995• Research interests: • computer vision (stereo, motion), computer graphics (image-based rendering), parallel programming 1/25/2005 Introduction to Computer Vision 2 What is Computer Vision? What is Computer Vision? • • • • Image Understanding (AI, behavior) A sensor modality for robotics Computer emulation of human vision Inverse of Computer Graphics Computer vision World model 1/25/2005 World model Computer graphics Introduction to Computer Vision 4 Intersection of Vision and Graphics rendering surface design animation user-interfaces modeling - shape - light - motion - optics - images IP shape estimation motion estimation recognition 2D modeling Computer Graphics Computer Vision 1/25/2005 Introduction to Computer Vision 5 Computer Vision [Trucco&Verri’98] 1/25/2005 Introduction to Computer Vision 6 Image-Based Modeling image processing graphics Images (2D) Geometry (3D) shape + Photometry appearance vision 3 Image processing 2.1 Geometric image formation 4 Feature extraction 5 Camera calibration 7 Image alignment 6 Structure from motion 2.2 Photometric image formation 8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 12 Photometric recovery 14 Image-based rendering 1/25/2005 Introduction to Computer Vision 7 Syllabus Image Transforms / Representations • filters, pyramids, steerable filters • warping and resampling • blending • image statistics, denoising/coding • edge and feature detection 1/25/2005 Introduction to Computer Vision 11 Image Pyramid Lowpass Images Bandpass Images 1/25/2005 Introduction to Computer Vision 12 Pyramid Blending 1/25/2005 Introduction to Computer Vision 13 Parametric (global) warping Examples of parametric warps: translation affine 1/25/2005 rotation perspective Introduction to Computer Vision aspect cylindrical 14 Syllabus Optical Flow • least squares regression • iterative, coarse-to-fine • parametric • robust flow and mixture models • layers, EM 1/25/2005 Introduction to Computer Vision 15 Image Morphing 1/25/2005 Introduction to Computer Vision 16 Syllabus Projective geometry • points, lines, planes, transforms Camera calibration and pose • point matching and tracking • lens distortion Image registration • mosaics 1/25/2005 Introduction to Computer Vision 17 Panoramic Mosaics + 1/25/2005 + … + Introduction to Computer Vision = 18 Syllabus 3D structure from motion • two frame techniques • factorization of shape and motion • bundle adjustment 1/25/2005 Introduction to Computer Vision 19 3D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996 1/25/2005 Introduction to Computer Vision 20 Face Modeling 1/25/2005 Introduction to Computer Vision 21 Syllabus Stereo • correspondence • local methods • global optimization 1/25/2005 Introduction to Computer Vision 22 View Morphing Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’96] 1/25/2005 Introduction to Computer Vision 23 Z-keying: mix live and synthetic Takeo Kanade, CMU (Stereo Machine) 1/25/2005 Introduction to Computer Vision 24 Virtualized RealityTM Takeo Kanade, CMU • collect video from 50+ stream reconstruct 3D model sequences http://www.cs.cmu.edu/afs/cs/project/VirtualizedR/www/VirtualizedR.html 1/25/2005 Introduction to Computer Vision 25 Virtualized RealityTM Takeo Kanade, CMU • generate new video • steerable version used for SuperBowl XXV “eye vision” system 1/25/2005 Introduction to Computer Vision 26 Syllabus Tracking • eigen-tracking • on-line EM • Kalman filter • particle filtering • appearance models 1/25/2005 Introduction to Computer Vision 27 Syllabus Recognition • subspaces and local invariance features • face recognition • color histograms • textures Image editing • segmentation • curve tracking 1/25/2005 Introduction to Computer Vision 28 Edge detection and editing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain," Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374-381, June, 1998. 1/25/2005 Introduction to Computer Vision 29 Image Enhancement High dynamic range photography [Debevec et al.’97; Mitsunaga & Nayar’99] • combine several different exposures together 1/25/2005 Introduction to Computer Vision 30 Syllabus Image-based rendering • Lightfields and Lumigraphs • concentric mosaics • layered models • video-based rendering 1/25/2005 Introduction to Computer Vision 31 Concentric Mosaics Interpolate between several panoramas to give a 3D depth effect [Shum & He, SIGGRAPH’99] 1/25/2005 Introduction to Computer Vision 32 Applications • Geometric reconstruction: modeling, forensics, special effects (ILM, RealVis,2D3) • Image and video editing (Avid, Adobe) • Webcasting and Indexing Digital Video (Virage) • Scientific / medical applications (GE) 1/25/2005 Introduction to Computer Vision 33 Applications • • • • • • Tracking and surveillance (Sarnoff) Fingerprint recognition (Digital Persona) Biometrics / iris scans (Iridian Technologies) Vehicle safety (MobilEye) Drowning people (VisionIQ Inc) Optical motion capture (Vicon) 1/25/2005 Introduction to Computer Vision 34 Projects Let’s look at what students have done in previous years … Stanford http://www.stanford.edu/class/cs223b/winter01-02/projects.html CMU http://www-2.cs.cmu.edu/~ph/869/www/869.html UW http://www.cs.washington.edu/education/courses/cse590ss/01wi/ GA Tech http://www.cc.gatech.edu/classes/AY2002/cs4480_spring/ 1/25/2005 Introduction to Computer Vision 35