An Affine Invariant Iterative Image Matching Approach for Matching Images with Different Views and Illumination

.

The pioneering work in the area of image matching was developed by Schmid and Mohr [8] who demonstrated the application of Gaussian derivatives in developing a rotationally invariant descriptor around a Harris corner. Lowe [9] further extended this work by incorporating scale invariance and published his work as Scale Invariant Feature Transform (SIFT) algorithm. He identified distinctive invariant features in an image and used them to identify robust and reliable matching. Reliable image matching and distinctive invariant features are two important attributes of image matching. A number of researchers have come up with a variety of approaches for identifying image descriptors that are invariant under affine transformation [10, 11 and 12]. These approaches Harris corner detector, maximally stable region and stable local phase structures [13]. Literature also presents different methods that have been evolved for evaluating these approaches. The evaluation has typically been carried in regard to interest point repeatability [14] and descriptor performance [15]. This paper presents one such approach for image matching which is not prone to view and illumination variations. The proposed iterative approach first defines the affine invariant data points in each image with the help of ASIFT. Descriptors based on gradient magnitude and orientation is assigned to be followed by the calculation of maximum disparity range. The area around all the assigned key points within the maximum disparity range is then selected iteratively. A disparity category having similar pairs of regions is identified and initial probabilities are assigned using Bayes theory. The probabilities are further updated with the help of certain updating rules. Once all the iterations are over the final classes are identified and displayed. The proposed approach is validated by performing experimental validation and empirical analysis of images with different tilts and illumination. Both absolute and transitional tilts are considered. The transitional and absolute tilts were considered for tilts varying from 10 degree to 80 degrees. In order to verify the illumination invariance the brightness of the images were varied. The results are compared with that of ASIFT based image matching. The results demonstrates the suitability of the proposed approach in delivering image matching that is not prone to view and illumination changes, the results also suggests that the performance is close to ASIFT based matching.
II. RELATED WORK SIFT is one of the pioneering methods used for extracting the points of interest in an image. SIFT operates by extracting points of interests in addition to extracting those features that are present around these points of interest. It is one of the most reliable approaches for matching different viewpoint of a scene or object in an image. SIFT approach is robust in that it is not only invariant to image orientation but also to image scale. It's hence capable of providing matching even in the presence of affine distortions. The performance of SIFT has also been found to be better in regard to 3D viewpoint, variation in illumination and presence of noise. SIFT operates by extracting features from reference images and storing them in a database. Image matching is then performed by comparing feature of the image with those stored in the data base. Euclidean distance is employed for finding candidate matches. Ke and Sukthankar [16] presented an "improved" version of SIFT descriptor by employing Principal Component Analysis (PCA) to identify the local features instead of the SIFT smoothed weighted histograms. The PCA-SIFT enhanced speed of implementation of SIFT matching process by an order of magnitude. Even though it could speed up SIFT it failed in delivering results that are as distinctive as that of the SIFT. Bay. H developed SURF [17],SURF stands for Speeded-Up Robust Features and its objective is to improve the strength of the leading existing feature detectors and descriptors (i.e. SIFT and PCA-SIFT). Unlike PCA-SIFT, SURF speeded up the SIFT's detection process without damaging the quality of the detected points.
Over the decade many significant developments in image matching have been published. Zhang [18] provided a dense pattern of mass points for Digital Surface Model (DSM) generation using Geometrically-Constrained Cross-Correlation (GC3) algorithm. Since SIFT is only capable of detecting blob-like feature points and result in relatively limited matching results [19], many researchers have used it to achieve reliable seed points. Long [20] and Silveira [21] presented a hybrid matching approach that combines a feature-based algorithm and an area-based algorithm. Wu [22] and Zhu [23] proposed a triangulation-based hierarchical image matching method that employs few seed points to generate the initial Delaunay triangulations. The interest points are then matched under the triangle constraint and the epipolar constraint.
Xiong [24,25] developed a robust interest point matching algorithm that first detects the super points with the greatest interest strength. Then, a control network is constructed, and each interest point is assigned a unique position and angle as its descriptor. IAlruzouq and Habib [26] used Modified Iterated Hough Transform (MIHT) to estimate the corresponding linear features between stereo pairs. Colerhodes et al. [27] introduced a registration algorithm that combined a simple yet powerful search strategy based on a stochastic gradient with two similarity measures, correlation and mutual information. Eugenio and Marques [28] developed two contourmatching techniques for satellite imagery, based on a general affine transformation, which modeled directly the corrections in the image domain without an explicit identification of the distortion sources. Semi-Global Matching (SGM) [29,30] successfully combines the concepts of global and local stereo methods for accurate, pixel-wise matching at low runtime; the core algorithm considers pairs of images with known intrinsic and extrinsic orientation, and the method have been implemented for rectified and unrectified images [31]. Humenberger et al. [32] introduced a stereo matching approach consisting of a combination of census-based correlation, SGM disparity optimization, as well as segmentation-based plane fitting for enhancements on textureless and occluded areas.
III. PROPOSED APPROACH The primary objective of the proposed work is to design and develop a novel image matching algorithm which is not prone to view and illumination variations. This section describes in detail various steps involved in the proposed image matching algorithm.
Step 1: Key-points selection in both the images with the help of ASIFT.
Step 2: For every keypoint from both the images a descriptor is computed.
Step 3: An area is selected around every right keypoint node, considering possible maximum disparity range.
Step 4: All the keypoints are found in the area selected in step 3, around a right keypoint node in the right image.
Step 5: The procedure of step 3 and 4 is iteratively performed for all the right keypoints and the area around each right keypoint is selected considering the approximate maximum disparity range.
Step 6: The procedure in steps 3 and 4 is repeated for all left keypoints from left image iteratively. But here area around the left keypoint is a fixed sample area of size 16 x 16.
Step 7: The left keypoint node is paired with every right keypoint node and the pair is called as category pair.
Step 8: For every category pair, Euclidean distance between the descriptors of the keypoints is calculated.
Step 9: Weight is assigned to every right keypoint , in the selected area. The weight is inversely proportional to the Euclidean distance between the corresponding descriptors.
Step 10: For every category the weight is calculated as k is a positive constant. A disparity category which associates highly similar pairs of region will have large weight value. ( ) will be in the interval [0, 1] and weight is inversely proportional to Euclidean distance.
Step 11: For every category set c, ̅ is undefined disparity category. Considering weight ( ̅ ) for ̅ which is undefined i.e. keypoint ( , ) from left image does not correspond to any keypoint in the right selection area of right image. The weights cannot be used as probability estimates as ( ) is undefined and weights will not sum up to 1.
Step 12: Considering the keypoint matching as a classification problem, bi is classified to one of the category . Initial probability for undefined category is given as Step 13: Application of Bayes rule Conditional probability that has category c as matching, given that is matchable

-̊( ̅ ) : prior probability that is matchable
Step 14: Estimating ( | ) as below Step 15: Initial probabilities are assigned to every category from right selection by equation (2), (3) and (4). Initial probabilities which depend only on the similarity of neighborhood of candidate matching points can be improved using consistency property. The probability updating rule should have the property that the new probability ( ) should tend to increase when descriptors with highly probable category consistent with c are found nearby the keypoint region. Categories are considered consistent if they represent nearly the same disparity i.e. Step 16: The category probability is updated using where denominator acts as normalizing factor ( ) = ∑ ( ) Here category probability is updated iteratively. The values in the iteration are used to calculate values in + 1 iteration. are the weights associated with contribution of different neighbors of . N is the number of neighborhood points of bi.
IV. TESTING DATA The data set used for testing comprises of images with different degrees of tilt (both absolute and transitional tilt). These images help to validate the suitability of the proposed approach in performing image matching under different viewing conditions. Similarly, images with different degree of illumination are also considered for analysis and validation. Fig.1 illustrates images having an absolute tilt of specified degrees. It has a total of 9 images including the reference image with zero tilt. Fig. 2  Illumination invariance property of the proposed approach is demonstrated by testing the algorithm for images with different degrees of illumination. Fig.3 illustrates 2 images from that dataset, having different degrees of illumination. V. RESULTS AND DISCUSSION As described in the previous section, different images with different viewpoints characterized by different degrees of absolute tilt and transitional tilt have been considered. Similarly images with different illuminations have also been considered to demonstrate the algorithm being not prone to view and illumination changes. One of the important and the foremost steps in the algorithm is to identify the key points. The following figures illustrate the keypoints / descriptors identified for certain images chosen from the data set. Number of keypoints identified eventually influences the number of matches identified and hence play very key role in the image matching algorithm. Fig. 4 illustrates different keypoints identified in the frontal image with absolute tilt of zero degrees, while Fig.5 depicts the descriptors identified in the image having an absolute tilt of 60 degrees compared to the frontal image.  The descriptor for the image with transitional tilt of 4 is given in the Fig.6 while the descriptors for two different images with different degrees of illumination are given in Fig.7 and      Table (2) and Table (3) depicts the comparison in the performance; image matching as done by ASIFT based approach and the proposed one. It can be clearly observed from the results that the proposed method delivers a better performance across all the three scenarios when compared with ASIFT. It is interesting to note that the performance delivered in terms of number of matches identified by the proposed approach is extremely high in the case of images with transitional tilt. VI. CONCLUSION A Novel frame work for view and illumination invariant image matching approach has been designed and successfully presented in this paper. The performance of the proposed algorithm was evaluated with images having different degrees of tilt and illumination. It can be clearly inferred from the results that the proposed approach has outperformed the ASIFT in terms of identifying the number of image matches. The accuracy of the proposed approach can also be visualized from the results; where in few best matching points have been displayed for better visualization. It can be observed from those depictions that the proposed approach has a high degree of accuracy in matching images under different view and illumination. It was also observed during the experimentation, that the proposed approach is faster than ASIFT making it suitable for real time applications.