Dermoscopic Interest Point Detector and Descriptor Howard Zhou1, Mei Chen2, James M.
Download ReportTranscript Dermoscopic Interest Point Detector and Descriptor Howard Zhou1, Mei Chen2, James M.
Dermoscopic Interest Point Detector and Descriptor Howard Zhou1, Mei Chen2, James M. Rehg1 1School of Interactive Computing, Georgia Tech 2Intel Research Pittsburgh Skin cancer Skin cancer : most common type of cancer ( > 1 million ) Skin Lung Breast Prostate Bladder 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 [ Top 5 categories of estimated annual cancer incidence for 2009 from National Cancer Institute ] 2009-07-01 2 Skin cancer Skin cancer : most common type of cancer ( > 1 million ) forms in tissues of the skin Skin lesions 2009-07-01 3 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Skin cancer Skin cancer : most common type of cancer ( > 1 million ) forms in tissues of the skin Benign lesions 2009-07-01 Skin cancer 4 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Skin cancer Skin cancer : most common type of cancer ( > 1 million ) forms in tissues of the skin Benign lesions Skin cancer Basal cell carcinoma Squamous cell carcinoma Melanoma 2009-07-01 5 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopy Non-invasive imaging technique Improve diagnostic accuracy by 30% Skin cancer Basal cell carcinoma Squamous cell carcinoma Melanoma 2009-07-01 6 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopy Non-invasive imaging technique Improve diagnostic accuracy by 30% Clinical view 2009-07-01 7 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopy Non-invasive imaging technique Improve diagnostic accuracy by 30% Microscope + light + liquid medium Dermatoscope 2009-07-01 8 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopy Non-invasive imaging technique Improve diagnostic accuracy by 30% Microscope + light + liquid medium Reveal pigmented structures Dermatoscope Dermoscopy view 2009-07-01 9 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Pigmented structures revealed by dermoscopy Dermoscopy view 2009-07-01 10 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Pigmented structures revealed by dermoscopy Dermoscopy view Blue-white veil 2009-07-01 11 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Pigmented structures revealed by dermoscopy Dermoscopy view Blue-white veil Scar-like depigmentation 2009-07-01 12 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Pigmented structures revealed by dermoscopy Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules 2009-07-01 13 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Pigmented structures revealed by dermoscopy Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules Negative network 2009-07-01 14 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Pigmented structures revealed by dermoscopy [Betta et al. 2006], [Grana et al. 2006], [Iyatomi et al. 2007],… Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules Negative network 2009-07-01 15 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Over 100 dermoscopic features Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules Negative network … 2009-07-01 16 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features Over 100 dermoscopic features Multiple binary classifiers for each image Dermoscopy view Blue-white veil BW classifier Scar-like depigmentation SLD classifier Brown globules BG classifier Negative network NN classifier … … 2009-07-01 17 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features General detector? Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules Negative network … 2009-07-01 18 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features General detector? Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules Negative network … Dermoscopic features consist of low level image characteristics (ridges, blobs, streaks, pigmentation,…) 2009-07-01 19 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic features General detector? Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules Negative network … Dermoscopic features consist of low level image characteristics (ridges, blobs, streaks, pigmentation,…) interest points 2009-07-01 20 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic Interest Point (DIP) General detector: concentration/configuration of interest points bag-of-visual-words approach Dermoscopy view Blue-white veil Scar-like depigmentation Brown globules Negative network … Dermoscopic features consist of low level image characteristics (ridges, blobs, streaks, pigmentation,…) interest points 2009-07-01 21 [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ] Dermoscopic Interest Point (DIP) Inspired by general interest point detector and descriptors We propose Dermoscopic Interest Point (DIP) (SIFT & SURF) detector - to extract these low level building blocks descriptor – for constructing a general visual vocabulary for dermoscopic features 2009-07-01 22 Dermoscopic Interest Point (DIP) Compared to the general interest point detector and descriptors (SIFT & SURF) Same key issues Similar to SIFT & SURF Repeatable Distinctive Robust to noise and deformation (geometric and photometric) Corners and blobs Scale and rotation invariant In addition Curvilinear features (fibrillar pattern and radial streaming) Color component 2009-07-01 23 Detector Corners and blobs Fast-Hessian detector [Bay, et al. 2006] Hessian matrix 2009-07-01 24 Detector Corners and blobs Fast-Hessian detector [Bay, et al. 2006] Hessian matrix Box filter approximation to replace Gaussian derivatives Fast using Integral image 2009-07-01 25 Detector Corners and blobs Fast-Hessian detector [Bay, et al. 2006] Hessian matrix Curvilinear structures Curvilinear detector [Steger, 1996] 2009-07-01 26 Detector Corners and blobs Fast-Hessian detector [Bay, et al. 2006] Hessian matrix Curvilinear structures Curvilinear detector [Steger, 1996] 2009-07-01 27 Detector Corners and blobs Fast-Hessian detector [Bay, et al. 2006] Hessian matrix Curvilinear structures Curvilinear detector [Steger, 1996] 2009-07-01 28 Detector Corners and blobs Fast-Hessian detector [Bay, et al. 2006] Hessian matrix Curvilinear structures Curvilinear detector [Steger, 1996] 2009-07-01 29 Descriptor Distinctiveness Spatially localized information Distribution of gradient-related features Dermscopic: color features Invariance (Repeatability) Relative strength to reduce the effect of photometric changes Relative orientation for rotation invariance 2009-07-01 30 Descriptor Distinctiveness Invariance (Repeatability) Spatially localized information Distribution of gradient-related features Dermscopic: color features Relative strength to reduce the effect of photometric changes Relative orientation for rotation invariance To construct Reproducible orientation 2009-07-01 31 Descriptor Distinctiveness Invariance (Repeatability) Spatially localized information Distribution of gradient-related features Dermscopic: color features Relative strength to reduce the effect of photometric changes Relative orientation for rotation invariance To construct Reproducible orientation 2009-07-01 32 Descriptor Distinctiveness Invariance (Repeatability) Spatially localized information Distribution of gradient-related features Dermscopic: color features Relative strength to reduce the effect of photometric changes Relative orientation for rotation invariance To construct Reproducible orientation Feature vector 2009-07-01 33 Descriptor Orientation For rotation invariance Haar-wavelet responses in x and y direction (in a circular neighborhood) 2009-07-01 34 Descriptor Orientation For rotation invariance Haar-wavelet responses in x and y direction (in a circular neighborhood) Reponses represented as 2D vectors dy dx 2009-07-01 35 Descriptor Orientation For rotation invariance Haar-wavelet responses in x and y direction (in a circular neighborhood) Reponses represented as 2D vectors dy Average responses in a sliding window of 60 degree dx 2009-07-01 36 Descriptor Orientation For rotation invariance Haar-wavelet responses in x and y direction (in a circular neighborhood) Reponses represented as 2D vectors dy Average responses in a sliding window of 60 degree dx The longest vector indicates the orientation 2009-07-01 37 Descriptor Descriptor components Context of the descriptor: a square region oriented along the orientation (centered around the interest point) Local statistics 2009-07-01 Uniform 4 x 4 subregions Intensity gradients (I): Sum of Haar-wavelet responses: dx, dy, |dx|, |dy| Color statistics (C): Coarse color histogram of the region (alpha & beta channels in L*a*b space) [ Image courtesy of Bay et al. 2006] 38 Dermoscopy Interest Point 2009-07-01 39 Dermoscopy specific Common interest point descriptor ignores linear features SURF 2009-07-01 DIP 40 Experiment 2009-07-01 41 Conclusion A generalized framework for characterizing dermoscopic features using Dermoscopic Interest Point (DIP) A feature detector and a descriptor specifically designed for this purpose Initial experiments showed that our scheme achieves a comparable level of invariance to lighting, scale, and rotation changes 2009-07-01 42 Future work Build a vocabulary of dermoscopic features using DIP Explore the possibility of using DIP in skin CAD related applications: Dermoscopic feature extraction and classification Dermoscopy image registration Dermoscopy image search and retrieval via dermoscopic features 2009-07-01 43 Acknowledgement Collaborators (in alphabetical order) Dr. Laura K. Ferris M.D. Ph.D. UPMC Richard Gass, Intel Research Pittsburgh Casey Helfrich, Intel Research Pittsburgh Many thanks to our anonymous reviewers for their helpful comments and suggestion 2009-07-01 44 Thank you Thank you ! 2009-07-01 45 Related publications Interest pointer detector and descriptors Distinctive image features from scale-invariant keypoints David G. Lowe Intl. J. of Computer Vision (IJCV), 2004 Surf: Speeded up robust features Herbert Bay, Tinne tuytelaars, and Luc Van Gool, in Eur. Conf. on Computer Vision (ECCV), 2006 An unbiased detector of curvilinear structures Carsten Steger, IEEE Trans. Pattern Anal. Machine Intell.(PAMI) 1996 2009-07-01 46 Outline Introduction Detector Descriptor Corners and blobs Curvilinear structures Orientation Descriptor components Validation Conclusion 2009-07-01 47 Dermoscopic features A Pigmented Skin Lesion (PSL) typically has several dermoscopic features Over 100 of these features 2009-07-01 48 Detecting line points Cross section L’ = 0 L’’ large L(x) Curve n(x) n(x) 2009-07-01 49 [ Steger 1998, ”An Unbiased Detector of Curvilinear Structures” ] Experiment 2009-07-01 50