Template based Shape Feature Selection for Plant Leaf Classification

— The current work proposes a new way for recognition of plant species from digital leaf images. A new Shape Feature Selection Template (SFST) is developed for the selection of shape features for different types of leaf. The selection of the shape feature depends on the aspect ratio of the leaf images. Rather than extract all shape features for all leaf shapes, different shape features are selected for the categorization of different types of shapes. Six types of aspect ratios are considered in this study. For this reason six combinations of shape features are developed for the classification of six types (Square, Very Wide, Wide, Medium, Narrow and Very Narrow) of leaf images. Efficacy of the proposed method is studied by using neuro-fuzzy controller (NFC) and Neural Network (NN). Experimental results demonstrate that the proposed approach is effective in recognizing leaves with varying shape, size and orientations to an acceptable degree.

. Architecture of the proposed approach A query image (Q) is first subjected to a set of pre-processing (PP) operations before being redirected to a simple shape (SS) module where its visual characteristics are represented as feature vectors, which are then fed to a classifier for identification and classification. PP is used to normalize the scaling and orientation factors of a leaf so that it can be compared with other leaves of different characteristics. The output from the PP module is a normalized query along with a segment number (S) assigned based on the leaf aspect ratio (length by width). The normalized query is fed to the simple shape (SS) for feature extraction. In the second step, feature vectors are generated from the shape module to mathematically represent the visual features of a leaf. This work uses a 20-element vector for simple shapes SSF (1) ... SSF (20). In the third step, the feature vectors are fed to a set of custom classifiers to discriminate between various classes of leaves with different shape properties. The following sections provide details on each of these modules. To test the effectiveness of this approach experimentations are performed on 58 classes of simple green leaves. Results and accuracies obtained are tabulated in the experimentations section.

A. Pre-Processing Module (PP)
In general a leaf image can have varying sizes, orientation angles and translation factors with respect to the image origin due to which they first need to be normalized with respect to these factors before they can be compared with each other. The pre-processing module initializes the recognition system by imposing certain standardization rules for normalizing these features. A leaf image (Q) is typically a color image (in RGB space) oriented at a random angle and having a random size, Fig. 2(a). To identify the angle by which the leaf is inclined to the horizontal, the orientation of the major-axis of the ellipse with the same normalized second central moments as the leaf shape is extracted from the image. The major axis is then rotated to align the leaf along the horizontal direction, as in Fig. 2(b). To standardize the translation factors with respect to the origin, the background is shrunk until the leaf just fits within its bounding rectangle, Fig. 2(c). A leaf image can be of arbitrary size; hence standardization also requires rescaling them to pre-defined dimensions. However since the aspect ratio (ratio of major axis to minor axis) of various leaves are different, scaling them all to a single size can distort the leaf shape and affect the recognition performance, especially where shape based features are used. To minimize this distortion, different leaf types are scaled to different predefined sizes called "segments" depending on their aspect ratio values (R). Based on the actual values observed in representative leaves, six different segment sizes with corresponding names are defined to which a leaf image is scaled to, with little or no distortion. Table 1 provides details of the aspect ratios and corresponding segment numbers, names and dimensions. Here R = 1.0 corresponds to a circle while larger values denote ellipses with smaller heights (rows) and larger widths (columns). Images within a range of x.0 to x.9 are subsequently resized to a middle dimension of x.5, where x ranges from 1 to 6. The segment 1, where the Aspect ratio of the leaf is between 1 and 1.9, is referred as Square; the segment 2, where the Aspect ratio of the leaf is between 2 and 2.9, is referred as Very Wide; the segment 3, where the Aspect ratio of the leaf is between 3 and 3.9, is referred as Wide; the segment 4, where the Aspect ratio of the leaf is between 4 and 4.9, is referred as Medium; the segment 5, where the Aspect ratio of the leaf is between 5 and 5.9, is referred as Narrow and the segment 6, where the Aspect ratio of the leaf is between 6 and 6.9, is referred as Very Narrow. Aspect ratios more than 6.9 are also clubbed in segment 6. For computational convenience the width of all leaves is fixed at 300 pixels. A normalized image (NQ) from the PP module is shown in Fig. 2(c). Very Narrow 6.0 ≤ R ≤ 6.9 300/5.5, 300

B. Simple Shape Module (SS)
The objective of the SS module is to extract shape information from a simple leaf image which can be used for its classification. Shape based modelling of a simple leaf is initialized by analysing how shape features vary with shapes i.e. which features are optimized for which shapes. As part of the pre-processing operation, leaves have been divided into 6 segments based on their aspect ratios (R). Some basic shape extraction methods are selected depending on a new Shape Feature Selection Template (SFST) for each segment as different types of leaf shapes may require different set of shape feature for classification. A set of four basic shape parameters namely major axis length (M), minor axis length (N), leaf area (A), leaf contour perimeter (P) along with six derived parameters namely, E (eccentricity), F (form factor), G (ratio of perimeter to major axis), R (aspect ratio), S (ratio of perimeter to the sum of principal axes), T (rectangularity), are used for shape characterization. These are defined below.
To conduct the optimization analysis, a SFST is developed where leaves are represented by their closest geometrical counterpart i.e. ellipses. The aspect ratio R of the ellipses is varied from 1.0 to 6.9 in steps of 0.1 to generate 60 samples clubbed into 6 segments as per Table I. See Fig. 3. Each row in the figure indicates a segment and each column depicts variations within a segment. As the aspect ratio increases, the appearance of the ellipses within each segment look quite similar as is evident from the last 3 rows in the figure. This justifies the clubbing of higher aspect ratios above 7 into the same segment 6. For each ellipse, the following features are computed: A, E, F, G, P, S, T. To classify the ellipses, each segment is treated as a class and the ellipses within a segment as the samples within each class. It is evident that last few ellipses of each segment are inherently similar to the first few ellipses of the next segment. To take into account the partial overlap between the classes, a fuzzy classifier is used which produces weight factors for each sample indicative of its membership over all classes. Experimentations indicate best classification results are produced if the first and last samples are used for training and the remaining 8 samples for testing. The results of classification are used to generate the following optimized feature set for each class which is henceforth referred to as rules is listed in Table 2. The SS module uses the segment number S, obtained from the PP module, to divert incoming leaf images into 6 sub-modules SSR-1 to SSR-6, as per Table I. These sub-modules compute features to generate a 20-element feature vector SSF (1) to SSF (20) as per Table II. These rules are henceforth referred as SFST rules.

III. CLASSIFICATION
The i-th leaf class is characterized by a collection of n member images, divided into training and testing samples. During the training phase, the 20-element simple shape features are computed from the training samples. The features are stored in a plant database (PDB). This ensures that comparisons during the testing phase will only occur within one of these subsets as determined by an appropriate discriminator. The flow line depicted in Fig. 1 is used for classification of a test sample. Classification for shape modules is done using NFCbased on a scaled conjugate gradient algorithm and NN. The motivation of using a neural classifier arises from the fact that like most pattern recognition problems, there is no fixed mathematical model based on which data samples could be classified, rather it needs to be done solely on the basis of a non-linear mapping between a set of input and output observations. Since a particular leaf sample can have similarities with multiple classes, a fuzzy classifier is used as it accounts for probabilities of a sample for belonging to several classes unlike a neural network which imposes exclusive classification. Based on the test leaf type the searching process always takes place within an appropriate subset of the total database, which helps to optimize the computational load.

A. Dataset
Experimentations are done using 1160 leaf images divided into 58 classes, collected from [11,2]. Out of 20 images per class, 10 are used for training and 10 for testing. Fig. 4 shows samples of each class in the dataset.

B. Preprocessing
The pre-processing step involves the sample images rescaled to standard segment sizes.

C. Recognition of Simple Leaves using Shape features
Before the recognition of simple green leaves the images are redirected to one of the six sub-modules SSR-1 to SSR-6 based on their segment number S. The current dataset contains 21 classes for Square segment (1 to 21), 16 classes for Very Wide segment (22 to 37), 8 classes for Wide segment (38 to 45), 5 classes for Medium segment (46 to 50), 5 classes for Narrow segment (51 to 55) and 3 classes for Very Narrow segment (56 to 58). See Table III.
Square Segment is governed by SFST Rule-1 which includes four features: A, E, F, G, effective over 21 classes (1 to 21). The performance results in % using NFC and NN are in Table IV and Table V. Overall accuracy is 90% using NFC and 73.81% using NN. Fig. 5 shows the variation of these features averaged for first 15 classes.         . 11 shows the NFC output class labels produced for each of the 58 test samples. Fig. 11. NFC class output labels for 580 simple green test samples The overall accuracy for 58 classes of simple green leaves is (90 + 90.6 + 95 + 92 + 96 + 100)/6 = 94%.
To cross-check whether the chosen features for each segment provides optimum performance, each feature set is applied over all segments and accuracy results are compared. Table XVI shows the comparative accuracies. It indicates that for Square Segment, the best results are produced by Rule-1 (92.7%), for Very Wide segment, the best results are produced by Rule-2 (88.8%), for Wide segment, the best results are produced by Rule-3 (100%), for Medium segment, the best results are produced by Rule-4 (94%), for Narrow segment, the best results are produced by Rule-5 (83.8%), for Very Narrow segment, the best results are produced by Rule-6 (100%). Fig. 12 shows a plot of how various rules (along the horizontal axis) produce percent accuracies for each segment.
This provides experimental confirmation of the fact that rules and feature sets when customized based on leaf type tend to provide better results than when a single set of features is applied for every leaf type. This is the basis for the improved performance of the hierarchical architecture. The combination of 20 shape features is applied to the simple green leaf samples. The overall percentage of recognition rate in this case is 79.1%.
Depending upon all the experimentations it can be said that selecting different types of features for different types of leaf images is better rather selecting a single set of features for all the leaf images.

V. ANALYSIS
The current work is compared to other contemporary works, by applying some of the approaches found in extant literature to the current dataset to observe their performances.
In [1] leaf shapes are modelled using Fourier descriptors, morphological features and shape defining features. For the current dataset it was found that the smooth sinusoidal basis functions were unsuitable for modelling the transient signals occurring along sharp leaf contours. Also as the combination of above said three types of features (total six features) are used for classification, there can be a chance to increase the time for the classification. Overall accuracy is 39.6%. Fig.13 compares accuracies of approach [1] (blue) with current approach (red). Fig.13. Accuracies using approach [1] (blue) and current approach (red) In [12] leaf shapes are modeled using Fourier descriptors. For the current dataset it was found that the smooth sinusoidal basis functions were unsuitable for modelling the transient signals occurring along sharp leaf contours. Overall accuracy is 23.02%. Fig.14 compares accuracies of approach [12] (blue) with current approach (red). Fig.14. Accuracies using approach [12] (blue) and current approach (red) Hierarchical clustering used in [5] clusters the leaf images depending on the similarity of the texture. If the image of the leaf does not contain any prominent structure or if the quality of the scanned data is poor, then this method of clustering the images depending the texture does not produce better recognition rates. Overall accuracy is 47.2%. Fig.15 compares accuracies of approach [5] (blue) with current approach (red). Fig.15. Accuracies using approach [5] (blue) and current approach (red) In [14] the Zernike moment and HoG method is used as shape descriptor for the classification of leaves. HoG depends largely on the block size, cell size and number of orientation bins, which tend to be different for different shapes. Overall accuracy is 27.4%. Fig.16 compares accuracies of approach [14] (blue) with current approach (red). Fig.16. Accuracies using approach [14] (blue) and current approach (red) In [15] the Moment Invariant, convexity, perimeter ratio, multi scale distance ratio, average margin distance, and margin statistics is used for the classification of leaves using LDC. As all the features are shape features, the leaf images with similar shapes may not be handled by this approach. Overall accuracy is 48.1%. Fig.17 compares accuracies of approach [15] (blue) with current approach (red). Fig.17. Accuracies using approach [15] (blue) and current approach (red) In [4] the combination of boundary shape (Complex network and multi-scale fractal dimension), texture (Haralick descriptor) and color (Color moments) is used for leaf classification using MLP. As total 19 set of features are used for classification, so the computational time can be high. Overall accuracy is 13.8%. Fig.18 compares class accuracies of approach [4] (blue) with current approach (red). Fig.18. Accuracies using approach [4] (blue) and current approach (red) Table XVII shows the comparison of recognition rates of previous works with the current approach.

Approach
Accuracy (%) Approach of [1] 29.8 Approach of [12] 23.02 Approach of [5] 47.2 Approach of [14] 27.4 Approach of [15] 48.1 Approach of [4] 13.8 Current Approach 94 VI. CONCLUSIONS AND FUTURE SCOPE This article proposes a method of utilizing different leaf types differing in shape. New SFST approach is developed for the selection of shape features for different leaf categories. A approach is followed which consists of a pre-processing step, for normalizing the scale and orientation of different leaves, a shape analysis step involving shape based modelling of the leaf. Classification is done using NFC and NN. NFC based classification is done to take advantage of fuzzy similarities between leaves.
Future directions of research for improving the system would involve handling fragmented leaves where parts of the leaf surface area are missing.