A Computer Vision Based Fruit Grading System

Introduction...

This project was completed as a part of my third year university course in which we were required to undertake a personal investigation within the realm of physical computing. Within this module both an academic report and supporting artefact was created. Right from the start of hearing about this module I was keen to pursue a topic related to computer vision, as I feel it is a very applicable and fast growing industry, and is also a subject I have been very passionate about for a long time. Initially, I was looking into topics such as facial and emotional rcognition, traffic management and control, as well as various applications within agriculture. After much deliberation it was decided that I would pursue automated produce grading, in particular, the automated defect detection and grading of oranges. The full title for the project I have completed is; "The Effectiveness of a Colour Based Computer Vision Fruit Grading System in Comparison to those which Employ the Use of Neural Networks and Machine Learning Algorithms". The artefact I have created to support this investigation is a desktop application which takes two images of an orange as input, extracts the defects, and then grades the oranges appropriately. After testing, it was found to be just under 60% accurate at grading the fruit.

Technologies Used...

• Python.
• OpenCV.
• PyQt5 (for the GUI).

How the Program Works...

The application was developed using a combination of Python, PyQt, and OpenCV. The user is displayed with an interface, which features six containers for the images and three buttons. The first two buttons allow the user to navigate their computers file system in order to chose an image file of an orange they wish to test. Upon pressing the grade specimen button, the images will be analysed and the features will be extracted. This is where OpenCV comes in. An HSV colour range was defined as a NumPy array that covers the colour orange, anything outside of this range will be converted to black and removed from the image. The majority of the defects that oranges have are discolourations, thus this was deemed an effective way of detecting defects. Once this image segmentation has occured, OpenCV's contouring functions are employed which highlights all the areas in black on the inside of the orange. These are then counted and have their area calculated. The total area of these defects is compared to the total area of the specimen and the percentage that is damaaged is calculated. It is this figure that is used to grade the orange. The oranges themselves were split into three cateogires prior to grading; A, B, and C. A refers to specimens that have little or no defects, B refers to specimens with some defects, whereas C refers to specimens that have a large amount of the skin defected. Examples of this process are shown below.

Effectiveness, Limitations, and Improvements...

As mentioned above, the program is currently just under 60% effective (59.83% to be exact) at grading oranges. As this system just looks at the colour for defect detection, it wasn't expected to be extremely accurate, however, this is a promising figure and goes to show how important determining colour is in defect detection. In terms of variables, lighting and the quality of the image were determined to be most important. The lighting effects the results as both reflections and shadows off of the fruit can be incorrectly identified as defects. The quality of the image is also a factor as lower quality images do not represent colour as well, and thus some defects can go unnoticed. Also, the amount of specimens I have currently tested on the system on is fairly low (30), thus it is difficult to accurately state the effectiveness of this program. For future developments, I would be keen to implement different techniques of fruit grading such as neural networks, classifiers, and other computer vision algorithms that are not available within OpenCV. The effectiveness of each technique could then be compared to one another and directions for future research be suggested.

The Demonstration...

Below is a desktop recording I have created showing how the program works. I test three specimens, one of grade A, one of grade B, and one of grade C throughout this video. I then display the results to show how the automated defect detection works! As you can see, there are a few false positives in the form of defects that are actually reflections or shadows. However, these are mainly negligible in terms of the actual defects that are detected. If you could like to view the source code to this project, it is available on my GitHub here.

The Research Paper...

Along with the artefact, a research paper was also written. This paper consilidates the results and findings of the tests of the above program. I discuss reasons for the outcomes and also talk about the limitations of the study. The paper also features a literature review in which I discuss the current advancements in the field of computer vision and agriculture as well the effectiveness of other systems that have been created. The effectiveness of some of these systems is then compared to the effectiveness of my implementation. If you are interested in reading this at all, you can view and download the PDF here!