15862 Final Project Nisarg Vyas (nisarg at cmu dot edu)

Using Geometric Blur for Point Correspondence

Point correspondence

Fix a point in one image, find the corresponding point in the other image.

For example,

Point Correspondence is very crucial component for many computer vision and image analysis tasks.

* 3D reconstruction,

* object detection and recognition,

* image alignment and matching and

* camera calibration techniques

Here I find correspondences using a technique called “Geometric Blur”.

Geometric Blur is a notion of blurring, developed especially to compute measure of similarity between image patches (templates). It targets making the correspondence procedure more by making the templates discriminative and the matching robust.

We assume that under the presence of an affine distortion, that fixes a single point, the distance the piece of signal changes is linearly proportional to the distance that the piece of signal is away from the feature point. This assumption can be encoded in the filter by making use of a spatially varying blurring kernel. Hence, instead of blurring by Gaussian filter of constant standard deviation (σ), we propose using a variable standard deviation, in linear proportion with the distance. (e.g. α|x| ).

(a) (b)

Figure 1: a) Image with the feature point marked with Red Dot b) Subsampled version of Geometric Blur:

Here one can see that the blurring increases as one goes farther and farther from the feature point. Figure taken from [1]

This is different from blurring kernel normally used (uniform Gaussian Blur).

Figure 2: Comparison between geometric blur and uniform Gaussian blur. Geometric blur blurs the signal more farther from the origin (Figure Courtesy [1])

Geometric blur is more effective when applied to sparse signals hence I compute geometric blur on 4 distinct gradient channels: i) positive gradient in x direction, ii) negative gradient in x direction, iii) positive gradient in y direction and iv) negative gradient in y direction.

( a ) ( b )

Figure 3: (a) original image (b) four different sparse gradient channels (Figure Courtesy [2])

Experiments

I have used face subset of the Caltech 101 dataset [3] of object catergories for point correspondence experiments.

Step 1 :- Find Harris Interest points from pairs of images

I conducted Harris corner detection from set of pairs from the dataset we used. Figure (4) displays the outputs of applying Harris corner detector for a few sample pairs of images.

(a) (b)

( c ) ( d )

(e) (f)

Figure 4 (a-f) – Best 50 Harris interest points for image pairs, the interest points are marked with ‘+’

Step 2: - Compute the descriptor over the interest points

The geometric blur descriptor is taken by subsampled points of concentric circles around feature points. Geometric blur descriptor of each feature point consists of total 10 concentric circles, with each circle having 8 points subsampled. Calculation of geometric blur is carried out over four gradient channels. Thus, we get the final geometric descriptor having total of 320 dimensions per each feature point selected by Harris corner detector.

Step 3:- Match the descriptors in both images by SSD measure

( a ) ( b )

( c ) ( d )

( e ) ( f )

Figure 5 – point correspondences derived from geometric blur extractor technique, the correspondence is established between images (a)-(b) , (c)-(d) and (e)-(f), the correct correspondence is marked with same color and same shape at the similar locations in the pair of images. First pair has 17 correct matches, second pair has 18 correct correspondences and the third pair has 16 correct correspondences.

For comparison, I established point correspondences using SSD for uniform Gaussian blur for the same images,sample results are shown in the following figure.

( a ) ( b )

( c ) ( d )

( e ) ( f )

Figure 5 – point correspondences derived from SSD on uniform Gaussian blur, the correspondence is established between images (a)-(b) , (c)-(d) and (e)-(f), the correct correspondence is marked with same color and same shape at the similar locations in the pair of images. First pair has 11 correct matches, second pair has 15 correct correspondences and the third pair has 12 correct correspondences.

We tested on 30 pairs of facial images, once with same person featuring in image pairs, and once with different persons featuring in image pairs. The performance for both geometric blur and uniform blur techniques can be summarized from the following table.

The results can be summarized as follows:

Description

Average number of successful correspondences out of first 25 correspondences

Accuracy Percentage

Geometric blur, Same person featuring in image pairs

68 %

Uniform Gaussian blur, Same person featuring in image pairs

12.2

48. 8 %

Geometric blur, different persons featuring in image pairs

10.4

41.6%

Uniform Gaussian blur, different persons featuring in image pairs

7.8

31.2 %

Results convey that geometric blur descriptors perform much better than the uniform Gaussian blur descriptors for the task of point correspondence. These early experiments provide motivation of using geometric blur for applications which require template matching, such as stereo vision and object detection.

References

[1] A. C. Berg and J. Malik, Geometric Blur for Template Matching, In International Conference on Computer Vision and Pattern Recognition, 2001

[2] Jia Jane Wu, Comparing Visual Features for Morphing-based Recognition, MIT CSAIL-TR-2005-35

[3] L. Fei-Fei, R. Fergus and P. Perona, Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. IEEE CVPR 2004