By Vassilis G. Kaburlasos
Precision viticulture aims at maximizing grape yield and quality by minimizing input costs. Grape harvesting is the most important viticulture operation since the choice of the harvest time determines the desired quality of the yield. Identifying the maturity levels in vineyards could enhance the efficiency of harvesting operations; especially in wine production, where the optimal harvest time, associated with specific concentrations of certain compounds is strongly related to the desired wine quality. The precise time for grape harvest depends on the location, the environmental conditions, the duration of the growing season, the grape variety, the vine tree load and the intended use of grapes, i.e., eating, or wine production. Therefore, estimation of the exact harvest time is rather challenging; however, grape ripeness estimation is a less complex process and is performed regularly during veraison.
Traditional ripeness estimation is performed by experts who assess the maturity grade based on sensory attributes in combination with exhaustive sampling followed by chemical analyses. The latter is not economically feasible, especially for commercial vineyards. Moreover, the procedure is subjective, depending on the person who performs the sensory evaluation and sampling. Additionally, chemical analyses are destructive, time-consuming and usually involve sophisticated equipment that is costly and difficult to be operated by non-experts. Furthermore, destructive analyses presuppose extensive and frequent sampling that are made only on a finite number of fruit samples; statistical relevance implies precision loss.
In this context, automated solutions for grape ripeness estimation are to be sought. Lately, research focus on developing non-destructive, cost-effective and environmental-friendly techniques. Machine vision is currently used excessively for agricultural related tasks. The technological improvement in hardware provides sensors that combine high performance and reasonable pricing, while innovative software design provides algorithms that can support effective real-time artificial vision systems. Towards this end, machine vision has been introduced to in-field applications for grape ripeness estimation, as a quick, efficient and attractive alternative to chemical analysis, due to its simplicity, flexibility and low cost.
During ripening many physical and biochemical changes occur that affect grape characteristics such as color and morphology. Machine vision approaches can cope with color, shape and texture from the analysis of grape images, offering automated, non-destructive, rapid and cost-effective techniques.
Red-Green-Blue (RGB) color imaging is the most cost-effective way to determine color channel values and characteristics such as texture and shape. However, in RGB color imaging only three visible bands are available resulting in finite identification capability; RGB color channels exhibit high levels of correlation and display a smaller range of colors than human eyes can perceive. No color space can perfectly represent a color; different color spaces are investigated to address issues that others cannot deal with.
Hyperspectral imaging is considered an evolving process analytical tool. It can be used instead RGB for more demanding applications since it can record numerous bands across a wide spectral bandpass. The latter bands are contiguous and are extended beyond the visible part of spectrum. Hyperspectral imaging associates spectroscopy with conventional imaging and, therefore, both spectral and spatial information of an object can be obtained.
Near InfraRed (NIR) spectroscopy has also been proven a powerful analytical tool to define bioactive compounds in grapes, such as soluble solids and pH. In NIR spectroscopy, first NIR radiation is applied to the object and then the transmitted/reflected radiation is measured. The spectral characteristics of radiation are altered as it enters the object due to scattering and absorption that depend on the wavelength. Modifications depend on the chemical composition of the object and the light scattering properties. The main advantage of NIR spectroscopy over the rest of the reported methods, RGB and hyperspectral imaging, is the chemical-free sample preparation and the ability to determine efficiently the optical properties of the fruit that are strongly related to chemical and physical properties, and thus, to maturity.
Machine-vision-based monitoring is challenging when it comes to agricultural products such as grapes, that are of variable sizes, shapes, color and texture. The latter features are not stable, but vary over the growing season. Moreover, green grapes are even more difficult to be located due to the same color as the foliage. Public grape datasets need to include variable cases, such as grapes of all colors at all growing stages, different cultivars, occlusion cases from leaves and branches, and images under varying illuminations.
Harvesting robots usually include a robotic arm and a camera mounted on a robotic vehicle. Irregularities of the terrain would cause vehicle vibrations, resulting in unstable vision measurements. Defoliation practices could expose the grape bunches and facilitate grape bunch detection and removal. Moreover, the robot should ideally harvest under stable lighting conditions. The images to train the machine vision algorithms should be optimally taken at the same time and under the same conditions of lighting, distance, height, etc.
Another challenge is the trade-off between speed, accuracy and robustness. In general, effective systems need higher accuracies in lower processing time. Processing time is of great importance when it comes to real-time applications, to the point of sacrificing a bit of accuracy in order to achieve better processing time toward automating time-consuming processes.
The development of agrobots combines many disciplines and specialists: agronomists, engineers, mechatronics, intelligent modeling, system design, deep learning, machine vision, etc. The requirements of such a robot yell for powerful equipment and talented specialists. High precision robotic arms, dexterous end-effectors, powerful computational devices, precise imaging systems and corresponding robust algorithms. The latter poses an additional challenge, the low development cost of an agrobot. The existing technologies and algorithms may overcome many difficulties in processing time and demonstrate high accuracies with relatively low costs. However, there is still room for improvements. Future agrobots must rely on affordable equipment and be capable of decision making in more complex situations, responding to sudden environmental changes.
According to the above, the integration of machine vision algorithms for ripeness estimation into harvesting robots needs to overcome numerous challenges. It is a long and complex process that is already underway by researchers.
See all at: https://www.mdpi.com/2311-7524/7/9/282
Vassilis G. Kaburlasos has received the Diploma degree from the National Technical University of Athens, Greece, in 1986, and the M.Sc. and Ph.D. degrees from the University of Nevada, Reno, NV, USA, in 1989 and 1992, respectively, all in electrical engineering.
He currently serves as a Tenured Full Professor in the Department of Computer Science at the International Hellenic University (IHU), Greece; since 2019 he also serves as an elected member of IHU’s Research Committee. He is founder and director since 2016 of the HUman-MAchines INteraction (HUMAIN) research Lab (http://humain-lab.cs.ihu.gr/?lang=en ) at IHU having accessed projects of total budget over 5M EUR. He has been participant or (principal) investigator in 32 research projects, funded either publicly or privately, in the USA and in the European Union. He has been a member of the technical/advisory committee or an invited speaker in numerous international conferences and a reviewer of more than 40 indexed (WoS) journals. He has (co)authored more than 200 scientific research articles in indexed journals, refereed conferences, edited volumes and books. His research interests include cyber-physical system modeling applications with breakthrough contributions in the “Lattice Computing (LC) paradigm”.
Dr. Kaburlasos is a member of several professional, scientific, and honor societies around the world including the Sigma Xi, Phi Kappa Phi, Tau Beta Pi, Eta Kappa Nu, and the Technical Chamber of Greece. He is a member in the IEEE CTSoc Technical Stream Committee in HMI. His name is included in the top 2% of 215,114 researchers worldwide in the field “Artificial Intelligence & Image Processing” according to Mendeley Data, V2, doi: 10.17632/btchxktzyw.2.