On Tree Detection, Counting & Post-Harvest grading of fruits Based on Image Processing and Machine Learning Approach-A Review

— This paper reports involvement of image processing and machine vision technique to detect and count of fruits on-tree, in field condition, have been reviewed. In addition, this paper also associated with the grading of fruits in post-harvesting. Different types of algorithms are available for counting and to extract the feature of fruit characters by capturing the on-tree fruit image by any conventional RGB camera. With the help of this counting algorithm and feature extraction technique, fruit is detected and counted. This work also surveys grading method applied to the post-harvest fruits. Grading method involves: identification of mature & immature fruits, intact & diseased fruits and also predict the weight of the fruit from its shape. The grading of fruit can be done by using different types of the classifier. The main features, drawback and future prospective of previous work in this area are summarized.

detecting and grading system. Here, the relationship between meta theory, methodology & method was explored which are used for detecting, counting and grading of fruits, so as to retrieve the best suitable techniques for implementation which fulfills all the necessary requirements.
II. RELATED WORK Machine vision and computer vision have been mainly used for the quality analysis and grading of fruits and vegetables. These have the capability to automate manual grading processes and minimize monotonous inspection tasks. Computer vision method is mainly used for defective part identification, classification and finding the ripeness of fruits based on their appearance. Randomized Hough transform is used for elliptical shape detection. Robotic harvesting and sensor technology are used for automation. This work summarize the review of the various work using different image processing and machine learning technique such as K-means clustering, fuzzy logic, artificial neural network (ANN), support vector machine (SVM), histogram technique, RGB color space technique, color mapping technique, pixel spectral process, approach of bag-of-words, wavelet transform, morphological operation and watershed transform.
A. K-means Clustering K-means clustering is a vector quantization method. It is very popular for cluster analysis. This method aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean, shown in Figure 1. Y. Song et al. [1] have presented automatic fruit recognition and counting from multiple fruit images. In this case feature extraction using (MSCR, BOW model) in frequency domain and SVM classifier is the major image processing method. K-means clustering is used for segmentation of the image. In this case, it has achieved a correlation of 74.2% between automatic and manual counts of fruit. Zeeshan Malik et al. [2] have described ontree citrus fruit detection and counting for yield estimation of crop. The database has been created for 83 tree image samples with 4001 citrus fruits from three different fields. K-means segmentation is used for recognition of fruits. The accuracy of the on-tree fruit detection was 91.3%. The correlation between the manual and the automated count of fruits having coefficients of determination R2 up to 0.99. Mohammad Bagher Lak et al [3] have described machine vision technique, for identification of apple fruits at the first stage of robotic harvesting. Image processing algorithm, fruit detection, apple harvesting methods are used. In this paper, k-means clustering is used for unsupervised classification. This algorithm was finally assessed. This method has ability to identify apple fruits with 83.33% and detect its locations with a precision of 85.17%. K. Parvati et al. [4] have proposed image segmentation using gray-scale morphology and marker-controlled watershed transformation. For unsupervised classification k-means clustering was used. K-means is a learning algorithm, which separates the input data set into different clusters based on their inherent distance between each other. It minimizes the sum of the distance between the objects and respective cluster centres. It is an iterative process, which moves objects between the clusters until the sum is no longer minimized. Radnabazar Chinccullue et al. [5] have described citrus yield mapping system by using machine vision technique. This paper presents machine vision technique with two charged coupled device, ultrasonic sensor and a differential global positioning system to estimate fruits. Amruta Pandit et al. [6] described object counting using image processing techniques. In this case, k-means clustering was successfully segmented the image which was taken by a conventional camera (Table 1). In this case, k-means clustering was successfully segmented the image which was taken by a conventional camera. This method will very helpful in any kind of vegetable and fruits segmentation. In this case, the histogram of these pixels shows the brightness distribution found in the object.

B. Histogram Method
We will implement this method in future also for better performance. In this case, some descriptors of texture based on the intensity histogram were obtained such as mean, standard deviation, smoothness, third moment.
In future also we can use this method for calculation of properties. In this case, an 8-gray label image was generated by the pre-processing step of median filter, histogram based system.
In this case connectivity component was a major aspect.
A histogram shows history representation of distribution of numerical data. It is an estimate of the probability distribution of continuous variable. D. S. Jayas et al. [7] have described multi-layer neural networks for image analysis of agricultural products. In this case, the histogram of these pixels shows the brightness distribution found in the object. The mean brightness represents the average brightness of an object. Libin Zhang et al. [8] have proposed recognition of greenhouse cucumber fruit using computer vision. In this case, some descriptors of texture based on the intensity histogram were obtained such as mean, standard deviation, smoothness & third moment. By comparison, the "third moment" was the most effective parameter for upper part determination. Zania S Pothen et al. [9] have proposed texture-based fruit detection using the smooth patterns of fruit images. Candidate fruit locations that has passed to the initial filter are classified using modified histogram of oriented gradients combined with a pairwise intensity comparison texture descriptor and random forest classifier. Here overall F1 accuracy score of 0.82 for grapes and 0.80 for apples. Palaniappan Annamalai, et al. [10] have proposed citrus yield mapping system using machine vision. In this paper images of the citrus fruits were analysed by using histogram & pixel distributions of various classes (citrus, leaf, and background) were developed. The algorithm was tested on 59 validation images and the R2 value between the number of fruits counted by the machine vision algorithm and the average number of fruits by manual counting was 0.76. Annamalai, et al. [11 ] have described papaya size grading using central profile analysis of the digital image. Otsu's method for automatic threshold selection from a histogram of the image was successfully applied to various segmentation cases. This method is based on selecting the lowest point between two classes of the histogram by using class variance. Q. M. Jonathan Wu et al. [12] have proposed a fruit recognition method for automatic harvesting. Here shape-based recognition is efficient and reliable by using depth histogram representation. The first step is to derive depth histograms of objects of interest. The statistical distribution of the depth histogram of targeted fruit can be used to characterize the shape of target objects. Sashi. D.Buluswar et al. [13] have proposed colour models for outdoor machine vision. The points for each of the point represent two linear cluster histogram which together forms a dichromatic plot. Ulzii-Orshikh Dorj et al. [14] have described a comparative analysis on tangerine fruits detection, counting by using yield estimation algorithm. Removal of noise and counting methods are used and executed to perform counting algorithm. For better performance histogram of colour component Cb in YCbCr and thresholding in Cb component has used. Minjun Wang et al. [15] have proposed a novel algorithm for green circuits detection based on reticulate gray ladder feature. In this case, an 8-gray label image was generated by the pre-processing step of median filter, histogram based system ( Table 2). Vision system used for this project is based on a colour camera that supplies the HSI colour components. Hue and Saturation histograms are employed to perform a thresholding to segment the image.

C. HSI Technique
This technique will very useful in the segmentation process. Apple images were performed for the purpose of classification into yellow or green groups using the HSI (hue, saturation, intensity) colour system method. This results in an accuracy of 90%.
This method will use any kind of fruits for better results.
Alireza Khoshroo et al. [16] have proposed detection of red tomato on plants using image processing techniques. Colour space transformation is a powerful tool for colour feature extraction. HSI (hue, saturation, and intensity) space is known as one of the most powerful colour spaces. HSI space is developed based on the concept of visual perception in human eyes; therefore their colour measurements have a better relationship with visual significance of fruit surfaces. Results obtained from testing the developed algorithm showed an encouraging accuracy (82.38%) to develop an expert system for online recognition of red tomatoes. A.R. JimeHnez et al. [17] have developed automatic fruit recognition using pattern recognition methods. Vision system used for this project is based on a colour camera that supplies the HSI colour components. Hue and Saturation histograms are employed to perform a thresholding to segment the image. The three-dimensional information is obtained by a stereo matching of two different images of the same scene. About 90% of the ripe tomatoes are detected and the most frequent errors are due to occlusions. Narendra V G et al. [18] have proposed quality analysis of agricultural food products by computer vision method. Fruits of apple image were taken and performed for the purpose of identification into yellow or green groups using HSI (hue, saturation, intensity) colour model. The result of accuracy was 95%. Tadhg Brosnan et al. [19] have proposed quantify and grading of agricultural products by machine vision systems. The results show that an accuracy of over 90% (Table 3). D. Colour-Mapping Technique J.C. Noordam et al . [20] have proposed high-speed potato grading and quality inspection based on a colour vision system. The colour segmentation technique uses linear discriminate analysis (LDA) in addition with the Mahalanobis distance classifier to classify the pixels. Basically Mahalanobis distance is the distance between any point P and a distribution D. This method was introduced by P. C. Mahalanobis in 1936. Krishna Kumar Patel et al. [21] have proposed machine vision system: a tool for quality inspection of food and agricultural products. At the current stage, the quality has been assessed traditionally by hand inspecting the products individually or sampling large batches which is time consumeing and unreliable in nature. C. S. Nandi et al. [22] have described machine vision based techniques for automatic mango fruit sorting and grading based on maturity level and size. Colour vision systems have been developed for agricultural grading applications which include direct colour mapping system to evaluate the quality of tomatoes. Ms. Rupali et al. [23] have proposed a fruit quality management system based on image processing. This paper presents a fruit size detecting and grading system based on image processing. The early assessment of fruit quality requires new tools for size and colour measurement. The side view images of fruits have captured. After that some fruit characters have extracted by using detecting algorithms. K. Parvati et al. [24] have proposed image segmentation using gray-scale morphology and marker-controlled watershed transformation. Segmentation, a new method, for colour, grayscale MR medical images, and aerial images, has proposed. Edge detection algorithm includes function edge and marker-controlled watershed segmentation. Miss. Anuradha Gawande et al. [25] have described implementation of fruits grading and sorting system by using image processing and data classifier. Texture, PCA, pattern classification etc. methods are used. S.Arivazhagan et al. [26] have proposed fruit recognition using colour & texture features. Machine vision techniques used to detect a fruit rely having four basic features, which characterize the object's intensity, colour, shape and texture properties. This research work approaches an efficient mixture of colour & texture features for fruit recognition. D. Surya Prabha et al. [27] have proposed assessment of banana fruit maturity by image processing technique. This analysis has attempted to use image processing methods to detect the maturity stage of fresh banana fruit by its colour and size value of their images precisely. In the respective work, a total of 120 images comprising 40 images from each stage such as undermature, mature and over-mature were used for developing algorithm and accuracy prediction. Devrim Unay et al. [28] have developed thresholding-based segmentation and apple grading by machine vision. M.Z. Abdullah, et al., [29] proposed discrimination and classification of fresh-cut star fruits (Averrhoa carambola L.) using automated machine vision system (Table 4). The colour segmentation procedure uses linear discriminate analysis (LDA) in combination with a Mahalanobis distance classifier to classify the pixels.
This technique will further use in any type of segmentation process.
Patel et al. have proposed machine vision system: a tool for quality inspection of food and agricultural products. (2012) The colour mapping technique has used for quality assessment of the food.
This will further use in any type of vegetable quality analysis. The study attempted to use image processing technique to detect the maturity stage of fresh banana fruit by its colour and size value of their images precisely.
The various technique will use in various aspects. The color based technique will major aspects. This method will give better accuracy in all the cases.

E. RGB Color Space Method
Narendra V G et al. [30] have proposed quality analysis of agricultural fruits by using computer vision method. In this research work, it has used automated grading system for oil palm fruits using RGB color model. The grading technique was used to distinguish between the three different categories of oil palm fruit bunches. Krishna Kumar Patel et al. [31] have proposed machine vision system: a tool for quality inspection of food and agricultural products. It was developed an automated grading system for oil palm bunches using the RGB color model to distinguish between the three different categories of oil palm fruit bunches. In their study, the result showed that the ripeness of fruit bunch could be differentiated between different categories of fruit bunches based on RGB intensity. Van Huy Pham et al. [32] have proposed an image segmentation approach for fruit defect detection using k-means clustering and graph-based algorithm. It was used different threshold values for color channels in RGB space to distinguish different varieties of objects of interest and the background. Haisheng GAO et al. [33] have proposed automatic grading of the post-harvest fruit. RGB and HSI model were often used in computer vision system to describe the color, which is more similar to the manner of human vision. The HSI model includes three elements: hue, saturation and intensity. Mathew George, et al. [34] have proposed multiple fruit and vegetable sorting system using machine vision. The image transferred into the form of Red, Green and Blue (RGB) color model by using Matlab. This model uses the combination of all the three components for creating a color. The segmentation using this particular model is less efficient as lighting, reflection and other factors disrupt the different and hence consistent segmentation cannot be achieved. Sakya Sarkar et al. [35] have proposed a comprehensive approach for fruit pattern recognition under the framework of digital image processing. The pioneer researchers have presented in their approach a Matlab based RGB model of image recognition technique. Classifies and identifies fruits successfully up to 90% accuracy. This system is also useful tool in a variety of fields such as educational, image retrieval and plantation science. Jagadeesh D.Pujari et al. [36] have proposed recognition and classification of normal and affected agricultural product using reduced color and texture features. A new model of grading system for oil palm fruit has developed using the RGB color model and fuzzy logic. Machine vision techniques has developed for the image processing parts like the segmentation of colors, calculation of mean color intensity based on RGB color technique and the decision making process using fuzzy logic is used to train the data and make the classification for the oil palm fruit. The average accuracies of classification have increased to 88.28% and 83.80% for normal and affected agricultural product respectively. H.N Patel et al. [37] have proposed automatic segmentation and measurement of fruit using shape analysis. Color and shape analysis was used to segment the images of different fruits like apple, pomegranate, oranges, peach and plum obtained by any conventional camera. At first the input section tree image was converted from RGB into L*a*b color space. The results show that the proposed method can accurately segment the fruits having accuracy of 98% and the average measurement error was found as 31.4 % ( Table 4).  The computer program is developed for the image processing part like the segmentation of colours, calculation of mean colour intensity based on RGB colour model and the decision-making process using fuzzy logic to train the data and make the classification for the oil palm fruit.

Analysis of Algorithm Related to RGB COLOR SPACE Method Author & Years
For better segmentation analysis we will use RGB colour space method. At first the input section tree image was converted from RGB into L*a*b colour space. The results indicate that the proposed method can accurately segment the fruits having efficiency of 98% and the average measurement error was found as 31.4 %.
It will helpful in any kind of fruits or vegetable analysis.

III. CLASSIFICATION TECHNIQUE F. Support Vector Machine
Support Vector Machines (SVMs) are a supervised learning technique widely used for many different kinds of classification tasks. They were initially conceived to solve classification problems between only two classes, but they can be employed in multiclass problems by using one-against-all or one-against-one techniques. In machine learning method, support vector machines (SVMs, also support vector networks) are consist of learning algorithms that analyse data and identify patterns, used for classification and analysis (Figure 2).  Pixel classification based on superpixel over-segmentation, clustering of dense SIFTS features into visual words and bag-of-visual-word super-pixel classification using SVMs (support vector machine). This method will have very broad future prospective.
W.S. Qureshi et al. [38] have proposed machine vision for counting fruits on mango tree canopies. Pixel based classification on super-pixel over-segmentation, clustering of dense SIFTS features into visual words and bagof-visual-word super-pixel classification using SVMs (support vector machine) etc methods have used. Shuiguang Deng et al. [39] have proposed a feature-selection algorithm based on support vector machinemulticlass for hyper spectral analysis. This article has proposed a novel feature selection algorithm named support vector machine-multiclass forward feature selection (SVM-MFFS). SVM-MFFS adopts the wrapper and forward feature selection strategy, explores the stability of spectral variables, and uses classical SVM as classification and regression model to select the most relevant wavelengths from hundreds of spectral data. They compare SVM-MFFS with successive projection analysis and uninformative variable elimination in the experiment of identifying different brands of oil. The results show that SVM-MFFS outperforms in accuracy, receiver operating characteristic curve, prediction and cumulative stability, and it will provide a reliable and rapid method in food quality inspection (Table 6). For fruits recognition system, the KNN algorithm performs fruit classification by using the distance measure, which is the Euclidean distance metric to measure the distance between the attributes of the unknown fruit with the stored fruit.

G. K-Nearest Neighbors Classifier (KNN)
KNN will very much use in future prospective.

Miss. Anuradha Gawande et al. have
proposed Implementation of fruits grading and sorting system by using image processing and data classifier. (2015) The defected area from fruit images and grading them as per their level of infection and by using KNN classifier.
KNN will be also used in vegetables grading purpose.
Woo Chaw Seng et al. [40] have proposed a new method for fruits recognition system. This section outlines the methodology and data that are used to develop the fruit recognition system, and presents the pseudo-code for the developed system. For fruits recognition system, the KNN algorithm performs fruit classification by using the distance measure, which is the Euclidean distance metric to measure the distance between the attributes of the unknown fruit with the stored fruit. Proposed fruit recognition system analyses and identifies fruits successfully up to 90% accuracy. This system also serves as a useful tool in a variety fields such as educational, image retrieval and plantation science. Miss. Anuradha Gawande et al. [41] have proposed implementation of fruits grading and sorting system by using image processing and data classifier. Experimental results suggest that the proposed approach have the ability to find out the defected area from fruit images and grading them as per their level of infection using KNN classifier. It can accurately classify the infected images and store in their respective database (Table 7).

H. Artificial Neural Network
Recently neural networks method has become very popular, which is used to characterize biological processes. It has best decision-making capability which can be used in image analysis of biological products, where the size and shape classification is not achieved by any mathematical function. When it is combined with hightechnology handling systems, it gives consistent performance, which is the most important benefit of these artificial classifiers in classification of agricultural products. These networks are based on the concept of the biological nervous system, and have proved to be robust in dealing with the ambiguous data and the kind of problems that require the interpolation of large amounts of data ( Figure 3).

Figure 3 Multi layered artificial network
Peilin Li et al. [42] have proposed study on citrus fruit image data separability by segmentation methods. Artificial neural network (ANN) and decision theoretic classifier were used for the segmentation of apples image for the harvesting robot. Both methods achieved 80% fruits detected. Jagadeesh Devdas Pujari et al. [43] have proposed quality analysis and classification of anthracnose fungal disease of fruits based on statistical texture features. These features are then used for classification purpose using ANN classifier. They conducted experimentation on a dataset of 600 fruits' image samples. The accuracies of classification of normal and affected fruit types are 84.65% and 76.6% respectively. This work finds application in developing machine vision technique in the horticulture field. Gurea Tonguc et al. [44] have proposed fruit grading using image processing technique. All this work is supported by ANN and some custom programming method. Uravashi Solanki et al. [45] have proposed detection of disease and fruit grading. The classifier is used for classifying images based on their features. There are many classifiers are available. Naive bays classifier, K-Nearest neighbours (K-NN), support vector machine (SVM), artificial neural network (ANN) and random forest tree classifier. The respective paper has presented identification of different features of fruits, a different classifier for diseases detection and different fruit segmentation technique for fruit quality analysis (Table ). Artificial neural network (ANN) and decision theoretic classifier were used for the segmentation of apples in the image for the harvesting robot. Artificial neural network (ANN) and random forest tree classifier. This paper presented different features of fruits, a different classifier for diseases detection and different fruit segmentation technique for fruit grading.

I. Fuzzy Logic Technique
Fuzzy logic has been used in a wide range of problem domains. Applications area of fuzzy logic is very wide: process control, management & decision making, operations research, economics, pattern recognition and classification. FL is used to handle uncertainty, ambiguity and vagueness. Once the features are fixed, they are led in input to a classifier which outputs a value associated to the classification of the quality (integer value) or a quality index (real value). The classification can be divided into two approaches: conventional classification and computational intelligence-based classification. ". Fuzzy Logic provides a structure to model uncertainty, the human way of reasoning and the perception process. Fuzzy Logic is based on natural language and through a set of rules an inference system is built which is the basis of the fuzzy computation ( Figure 4).

FUZZY INFERENCE SYSTEM
X Y Figure 4 Fuzzy logic concept Jagadeesh D.Pujari et al. [46] have proposed recognition and classification of normal and affected agriculture produce using reduced colour and texture features. This paper has developed a new model of automated grading system for oil palm fruit is developed using the RGB colour model and artificial fuzzy logic. The average classification accuracies have 88.28% respectively. Queerzzs akeemu amin et al. [47] have proposed Image processing for quality and safety control in horticultural industries. In this paper, an automatic classification method of tobacco leaves based on the digital image processing and the fuzzy sets theory is presented. A grading system based on image processing techniques was developed for automatically inspecting flue-cured tobacco leaves. Jagadeesh Devdas Pujari et al. [48] have proposed grading and classification of anthracnose fungal disease of fruits based on statistical texture features. It has developed a new model of automated grading system for oil palm fruit is developed using the RGB colour model and artificial fuzzy logic. The classification accuracies for intact and affected anthracnose fruit types are 84.65% and 76.6% respectively. D. S. Gaikwad et al. [49] have proposed image processing approach for grading and identification of diseases on pomegranate fruit. The recognition is done by the minimum distance classifier based upon the statistical and co-occurrence features derived from the Wavelet transformed sub-bands.
First, we extract the feature from the segmented portion of the images that are being used for the training and store in a feature database. After feature extraction, images are classified by using different classification techniques. These are artificial neural network, support vector machine, fuzzy logic, and K-nearest neighbour. S.Arivazhagan et al. [50] have proposed fruit recognition using colour and texture features. The recognition has done by the minimum distance classifier based upon the statistical and co-occurrence features derived from the wavelet transformed. Experimental results on a database of about 2635 fruits from 15 different classes confirm the effectiveness of the proposed approach (Table 9).
IV. DISCUSSION AND SUMMERIZATION This survey report outlines all the previous presented paper, with respect to used methodology, accuracy and year of publication is presented in (Graph 1). The subject of fruit detection and counting is undergoing from last few years. However the study used few numbers of methods applied as computer vision, pattern recognition, BPN, segmentation method, texture analysis; yield estimation, Hough transformation etc. Here we study the different methods applied in this field in last 10 year. Graph  • In the year 2011, Peilin Li has proposed color-based segmentation method. In this case he has got fruitful result. He got the accuracy of 84.2% for citrus fruit detection. • In 2012, H.N Patel has proposed the machine vision method. During his research work he has got 84.1% of accuracy for grading of fruits. • D. Surya Prabha, in the year 2013 has proposed texture analysis method and he got the accuracy of 84.5% for fruits detection. • Yield estimation algorithm has used by D.S. Gaikuwad and in his work he got the accuracy of 90.8% for fruits counting in the year 2014. • In 2015 again machine vision method has used by Amruta Pandit in his review work and she has got the accuracy of 84.4% for recognition and quality analysis of fruits. • In the year 2016, Uravashi Solanki has proposed Hough transform method. He has got the accuracy of 90.9% for fruit detection. V. CONCLUSION This paper presents the survey on, On-tree detection, counting and post harvest grading of fruits. This survey of recent works in this field should be very useful for researcher in this interesting area. In case of detection, various methods like k-means clustering, color-space segmentation, RGB color recognition, HSI technique etc are used. In the whole review case we have conclued that the k-means clustering segmentation is a very effective method for both mature and immature fruits detection. For counting of fruits there were very few work has been done. Mainly for counting of fruits morphological operation has been used. The paper reviews the recent developments in computer vision for the agricultural industry. Computer vision systems have been used increasingly in industry for inspection and quality evaluation purposes as they can provide rapid, economic, hygienic, consistent and objective assessment. However, difficulties still exist, evident from the relatively slow commercial uptake of computer vision technology in all sectors. Even though adequately efficient and accurate algorithms have been produced, processing speeds still fail to meet modern manufacturing requirements. For grading, various classification methods are used. From the above studies all classification technique was evaluated accurately. But among all, artificial neural network and support vector machine techniques are give the better accuracy compare to other technique. Some of the future aspects are following which can help the researchers working in the same field of research • The research area is not only limited for detection, identification and counting of on-tree fruits in agricultural field but also overlapping of fruits and shape recognition can be separated by using various algorithms. • Grading of fruits is another wide area of research, that to know the quality of fruits and according to that price can be determine for commercial purpose. • Mobile based application can be designed by using an automate on-tree fruits detection and counting technique that will help for providing quick and accurate solution. The on-tree counting and post-harvest grading area of research can not be limite to above mentioned future scope. There are more other area where researcher have to explore for new innovation. In this review, as there are so many methods are proposed and implimented to detect, count and quality analysis of fruits using image processing and machnie vision technique. The main objective of the paper is to describe an overview of technical concepts used in the method existing in the litrature review.