Micro Phase Shifting Mohit Gupta and Shree K. Nayar Computer Science Columbia University Supported by: NSF and ONR.
Download ReportTranscript Micro Phase Shifting Mohit Gupta and Shree K. Nayar Computer Science Columbia University Supported by: NSF and ONR.
Micro Phase Shifting Mohit Gupta and Shree K. Nayar Computer Science Columbia University Supported by: NSF and ONR Structured Light 3D Scanning Gaming Biometrics Wafer defect Archiving Heritage Defect Inspection Shape from Structured Light camera image plane image correspondence scene image plane pattern projector Structured Light Coding Schemes Binary Code Phase radiance radiance radiance Peak Location time Ambiguity Correspondence Correspondence Correspondence time time Light Striping Binary Codes Phase Shifting [Shirai and Suwa, 1971] [Agin and Binford, 1976] [Minou et al., 1981] [Posdamer et al., 1982] [Srinivasan et al., 1985] [Wust and Capson, 1991] Phase Shifting Accurate but Ambiguous amplitude wmax Unambiguous but Noisy wmean Broad Frequency Band frequency (w) wmin Phase Shifting: Issues Interreflections Defocus camera scene projector scene P Q interreflections projector projected image received image Phase Shifting: Issues Interreflections Defocus camera defocus blur scene projector scene P Q interreflections projector projected image received image blurred Phase Shifting and Interreflections camera Direct Radiance Interreflections P Q R interreflections projector radiance scene time Phase Shifting and Interreflections camera Direct Radiance Total Radiance Phase Error P Q R radiance scene time projector Phase Shifting and Interreflections point interreflections projector Concave Bowl Phase Shifting and Interreflections Errors due to interreflections Concave Bowl Reconstructed Shape Phase Shifting and Interreflections camera p interreflection P Q ππ = illumination pattern × R scene projector light transport coefficients Phase Shifting and Interreflections camera p interreflection P Q ππ = illumination pattern × R scene projector light transport coefficients Phase Shifting and Interreflections camera p interreflection P Q R scene projector illumination pattern ππ = light transport coefficients * pixels pixels Phase Shifting and Interreflections illumination pattern light transport coefficients interreflection ππ = * pixels pixels Phase Shifting and Interreflections projected patterns illumination pattern light transport coefficients interreflection πΌπ = bandlimit × frequency frequency Interreflections corrupt phase for low frequency sinusoids Achieving Invariance to Interreflections projected patterns illumination pattern high frequencies interreflection πΌπ light transport coefficients = bandlimit × frequency High Frequency Illumination frequency Invariance to Interreflections Phase Shifting: Issues Interreflections Defocus camera defocus blur scene projector scene P Q interreflections projector projected image received image blurred Phase Shifting and Defocus projected patterns ideal irradiance profile projector defocus kernel = * time actual irradiance profile time time Phase Shifting and Defocus projected patterns ideal irradiance profile projector defocus kernel × frequency actual irradiance profile = frequency frequency Phase Shifting and Defocus projected patterns ideal irradiance profile projector defocus kernel × projected patterns actual irradiance profile = frequency frequency frequency ideal irradiance profile projector defocus kernel actual irradiance profile = × frequency frequency Large Number of Unknowns frequency Achieving Invariance to Defocus projected patterns ideal irradiance profile Narrow Band projected patterns projector defocus kernel × actual irradiance profile = Similar amplitudes frequency frequency frequency ideal irradiance profile projector defocus kernel actual irradiance profile Narrow Band frequency Narrow Frequency Band × = frequency Similar amplitudes frequency Invariance to Defocus Micro Phase Shifting wmean wmin Narrow, High-Frequency Band amplitude wmax frequency (w) Invariance to Interreflections amplitude wmax wmean wmin High Mean Frequency (wmean) light-transport bandlimit frequency (w) Invariance to Defocus wmean wmin Narrow Bandwidth (d) amplitude wmax Similar amplitudes frequency (w) How to Disambiguate Phase? wmax wmean wmin How Can We Disambiguate Phase Without Low Frequency Patterns? How to Disambiguate Phase? 51Hz. 49Hz. + w1 w2 = w1 + 2d 1Hz. Beat Frequency = d Phase Disambiguation: Number Theory period: π pixels phase: π pixels correspondence: πΆ pixels πΆ =ππ+π number of periods (unknown) Phase Unwrapping: Micro Phase Shifting π1 π1 πΆ ππ πΆ πΆ = π1 π1 + π1 unknown ππ known ππΉ πΆ πΆ = ππ ππ + ππ unknown ππΉ known πΆ = ππΉ ππΉ + ππΉ unknown Solve System of Simultaneous Congruences known Chinese Remainder Theorem πΆ = π1 π1 + π1 πΆ = ππ ππ + ππ πΆ = ππΉ ππΉ + ππΉ Theorem: There exists an integer C solving the above system of simultaneous congruences, if p1 ,β¦, pf ,β¦, pF are positive integers which are pairwise coprime. [The Mathematical Classic by Sun Zi, 3rd century AD] Efficient Algorithms Available for Solving How Many Frequencies Are Required? π1 ππΉ πΆ π πΆ periods of projected frequencies Number of columns: π period of emulated low frequency π1 × β― × ππΉ = π Condition for unambiguous phase recovery: π β₯ π Two Frequencies are Necessary How Many Frequencies Are Required? π1 π2 πΆ π πΆ Choose two frequencies so that Number of columns: π π1 × π2 β₯ π Two Frequencies are Sufficient How Many Images Are Required? offset amplitude (interreflections) (defocus) πΌππ phase 2ππ = π + π cos β + πΎ radiance for kth shift of wi π number of shifts F = number of frequencies Number of Unknowns = F+2 How Many Images Are Required? offset amplitude (interreflections) (defocus) πΌππ phase 2ππ = π + π cos β + πΎ π radiance for kth shift of wi number of shifts Four Images are Sufficient Conventional vs. Micro Phase Shifting Conventional Phase Shifting: Three Images Micro Phase Shifting: Four Images Current State-of-the-Art β’ Binary patterns [Xu and Aliaga, 2009] 400-1600 images [Couture et al., 2011] 200 images [Gupta et al., 2011] 42 images β’ Modulated Phase Shifting x = [Gu et al., 2011] [Chen et al., 2008] Low SNR. 7+ images. Ceramic Bowl: Interreflections Input Projected Projected and Input Images Conventional Phase Shifting [7 images, 2 Frequencies] Modulated Phase Shifting [7 images, 1 Frequency] Micro Phase Shifting [Our] [7 images, 5 Frequencies] Shape Comparison (seven input images) Conventional Phase Shifting Modulated Phase Shifting [Gu et al.] Micro Phase Shifting [Our] Lemon: Subsurface Scattering subsurface scacttering point projector Shape Comparison (seven input images) Conventional Phase Shifting Modulated Phase Shifting [Gu et al.] Micro Phase Shifting [Our] Russian Dolls: Defocus Shape Comparison (seven input images) Holes in low albedo regions Conventional Phase Shifting Micro Phase Shifting [Our] Wax Bowl: Interreflections + Scattering Shape Comparison (seven input images) Conventional Phase Shifting Modulated Phase Shifting [Gu et al.] Micro Phase Shifting [Our] Recovered Shape: Micro Phase Shifting Failure Case: Shiny Metal Bowl Specular interreflections Shape Comparison Conventional Phase Shifting Modulated Phase Shifting [Gu et al.] Micro Phase Shifting [Our] Discussion: Frequency Selection light transport bandwidth defocus kernel projected frequency frequency frequency Invariance to interreflections Amplitude attenuation defocus kernel similar amplitudes projector resolution frequency frequency Invariance to defocus Not resolvable by projector amplitude Summary: Micro Phase Shifting Narrow, High-Frequency Band frequency (w) Patterns in Narrow High-Frequency Band Shape Recovery with Interreflections and Defocus