Rapid estimation of apple phenotypic parameters based on 3D reconstruction

Hao Ma, Xu Zhu, Jiangtao Ji, Hui Wang, Xin Jin, Kaixuan Zhao

Abstract


In order to obtain the phenotypic parameters of apple quickly and accurately, which were commonly used as the basis of fruit sorting, a fast estimation method of apple phenotypic parameters based on three-dimensional (3D) reconstruction was proposed in this study. In this study, a three-dimensional model was constructed to estimate the phenotypic parameters of apple, such as volume, height, diameter, and fruit shape index. Firstly, an image acquisition system was built to capture sequence images of fruit with a binocular stereo vision system, and the images were extracted and matched using the Accelerated-KAZE algorithm to create the point cloud data. Secondly, the point cloud data were matched with the algorithm of Iterative Closest Point to establish a whole model of apple, and the surface reconstruction model of fruit was obtained by constructing irregular triangulation network. Finally, the apple phenotypic parameters were calculated by means of segmentation, surface complement and integral of the fruit model. Total of 200 apples were used as samples in the experiment. By this method, the phenotypic parameters of the apples were estimated based on their 3D reconstruction model, and the linear regression analysis was carried out between the estimated values and the real values. The results showed that R2 of the linear regression fitting of each parameter was higher than 0.90. Among them, the fitting of volume was the best with R2 of 0.97. In addition, the average errors of apple volume, height, fruit shape index, maximum diameter D and minimum diameter d were 8.73 cm3, 1.43 mm, 1.28%, 0.90 mm, and 1.23 mm, respectively. According to the Chinese national standard of “fresh Apple”, the average error of the estimated result is within the range of allowable error. It indicates that the method of apple phenotypic parameter estimation based on 3D reconstruction has a high accuracy and practicability, and it can be used as the support for fruit sorting.
Keywords: apple, 3D reconstruction, phenotypic parameter, stereo vision system, sequence image, sorting
DOI: 10.25165/j.ijabe.20211405.6258

Citation: Ma H, Zhu X, Ji J T, Wang H, Jin X, Zhao K X. Rapid estimation of apple phenotypic parameters based on 3D reconstruction. Int J Agric & Biol Eng, 2021; 14(5): 180–188.

Keywords


apple, 3D reconstruction, phenotypic parameter, stereo vision system, sequence image, sorting

Full Text:

PDF

References


China statistical yearbook, Beijing: China Statistics Press, 2020. (in Chinese)

Meng X N, Zhang Z H, Li Y, Ren L L, Song Y P. Present situation and development of apple grading. Deciduous Fruits, 2019; 51(6): 24–27. (in Chinese)

Jang Y M, Duan X W. Advances in Postharvest Physiology of Fruits and Vegetables in China. Journal of Tropical and Subtropical Botany, 2019; 27(5): 558–564. (in Chinese)

Sun S G, Zhang P, Li J L. Current situation and development of fruit sorting technology. Agricultural Equipment & Technology, 2018; 44(3): 19–20. (in Chinese)

Li M. Research status and development of fruit sorting technology. Journal of Jiangsu Teachers University of Technology, 2018; 24(2): 121–124. (in Chinese)

Liu Y, Wang H H, Liu Y, Yang Y T, Wang D Z. Research progress of forest-fruit mechanized picking and sorting. World Forestry Research, 2020; 33(3): 20–25. (in Chinese)

Wang G Q, Li S, Liu N, Wang Y Q, Xin Z. A brief discussion on the research status and progress of fruit and vegetable size classifier. Xinjiang Agricultural Mechanization, 2014; 5: 10–11. (in Chinese)

Zhang G X, Fu Q Q, Fu Z T, Li X X, Matetic M, Bakaric M B, et al. A comprehensive peach fruit quality evaluation method for grading and consumption. Applied Sciences-Basel, 2020, 10(4): 1348. doi: 10.3390/ app10041348.

Ji Y, Zhao Q, Bi S. Apple grading method based on features of color and defect. In Proceedings of the 2018 37th Chinese Control Conference (CCC), Wuhan, China, 2018; 5364–5368.

Fu L S, Wang B, Cui Y J, Su S, Gejima Y, Kobayashi T. Kiwifruit recognition at nighttime using artificial lighting based on machine vision. Int J Agric & Biol Eng, 2015; 8(4): 52－59.

Yan B, Guo W C. Identifying expanded ‘Hayward’ kiwifruits based on K-means clustering algorithm and calyx shape. Journal of Northwest A&F University (Social Science Edition), 2020; 48(5): 147–154. (in Chinese)

Wang G, Sun L, Li X M, Zhang M, Lv Q, Cai J R. Design of postharvest in-field grading system for navel orange based on machine vision. Journal of Jiangsu University (Natural Science Edition), 2017; 38(6): 672–676. (in Chinese)

Hu F H, Dong Z W. Online navel orange grading detection system based on machine vision. Journal of China Agricultural University, 2016; 21(3): 112–118. (in Chinese)

Anuja B, Atul B. Fruits and vegetables quality evaluation using computer vision: a review. Journal of King Saud University - Computer and Information Sciences, 2018. doi: 10.1016/j.jksuci.2018.06.002.

Huang C, Fei J Y. Online apple grading based on decision fusion of image features. Transactions of the CSAE, 2017; 33(01): 285–291. (in Chinese)

Fu L S, Sun S P, Li R. Classification of Kiwifruit grades based on fruit shape using a single camera. Sensors, 2016; 16(7): 1012. doi: 10.3390/ s16071012.

Javadikia H, Sabzi S, Rabbani H. Machine vision based expert system to estimate orange mass of three varieties. Int J Agric & Biol Eng, 2017; 10(2): 132–139. doi: 10.3965/j.ijabe.20171002.1737.

Wang Q H, Tang Y H, Xiao Z. Grape size detection and online gradation based on machine vision. Int J Agric & Biol Eng, 2017; 10(1): 226–233.

Jin X, Chen K K, Ji J T, Pang J, Du X W, Ma H. Intelligent vibration detection and control system of agricultural machinery engine. Measurement, 2019; 145: 503–510.

Chenglin W, Yunchao T, Xiangjun Z. Recognition and matching of clustered mature Litchi fruits using binocular charge-coupled device (CCD) color cameras. Sensors, 2017; 17(11): 2564. doi: 10.3390/s17112564.

Wang R T, Li M, Gong Z P. Research on terminal device of globular fruit monomer picking based on binocular vision. Food & Machinery, 2018; 34(7): 93–97, 100. (in Chinese)

Méndez P, Rodrigo, Cheein F A, Rosell-Polo J R. Flexible system of multiple RGB-D sensors for measuring and classifying fruits in agri-food Industry. Computers & Electronics in Agriculture, 2017; 139: 231–242. doi: 10.1016/j.compag.2017.05.014.

Wang B, Chen Z, Gao J, Fu L, Su B, Cui Y. The acquisition of kiwifruit feature point coordinates based on the spatial coordinates of image. In Proceedings of the International Conference on Computer & Computing Technologies in Agriculture. Switzerland, 2015; 478: 399–411.

Yang H, Wang X, Sun G. Three-dimensional morphological measurement method for a fruit tree canopy based on kinect sensor self-calibration. Agronomy, 2019; 9(11): 741. doi: 10.3390/agronomy9110741.

Lockman N A, Hashim N, Onwude D I. Laser-Based imaging for Cocoa pods maturity detection. Food and Bioprocess Technology, 2019; 12(11): 1928–1937. doi: 10.1007/s11947-019-02350-7.

Zhang G B, Zhang J W, Zhou J. A method of fruit shape detection based on line laser. Journal of Chinese Agricultural Mechanization, 2020; 1: 161–166. (in Chinese)

Li X. Design of automatic pineapple harvesting machine based on binocular machine vision. Journal of Anhui Agricultural Sciences, 2019; 47(13): 207–210. (in Chinese)

Mai C Y, Zheng L H, Sun Y, Yang W. Research on 3d reconstruction of fruit tree and fruit recognition and location method based on RGB-D camera. Transactions of the CSAM, 2015; 46(S1): 35–40.

Xu S Y, Lu K, Pan L L, Liu T G, Zhou Y X, Wang B. 3D reconstruction of rape branch and pod recognition based on RGB-D camera. Transactions of the CSAM, 2019; 50(2): 21–27. (in Chinese)

Li Q, Qiang H. Tomato ontology feature detection system based on binocular vision and deep learning. Journal of Southern Agriculture, 2020; 051(001): 237–244. (in Chinese)

Qiao F B, Zeng S W, Song H J, Gao Y Y. Fruit and vegetable depth information acquisition based on binocular stereo vision. Journal of Anhui Agricultural Sciences, 2015; 505(36): 374–378. (in Chinese)

Liu H B, Gao Y W, Lu J S, Rao X Q. Laser-speckle based identification of pear defects and fruiting peels/calyx. Transactions of the CSAE, 2015; 31(4): 319–324. (in Chinese)

Liu G, Zhang X, Zong Z, Guo C L. 3d reconstruction of strawberry based on depth information. Transactions of the CSAM, 2017; 48(4): 165–177. (in Chinese)

Yao D Y, Lin C D, Zhai R F, Chen K. Rapeseed 3D reconstruction and morphological parameter measurement based on laser point cloud. Fifth International Conference on Agro-geoinformatics, Tianjing, China, 2016; 1–6.

Zhang Z. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000; 22(11): 1330–1334. (in Chinese)

Alcantarilla P, Nuevo J, Bartoli A. Fast explicit diffusion for accelerated features in nonlinear scale spaces. British Machine Vision Conference, Bristol, UK, 2013; 1–11.

Weickert J, Grewenig S, Schroers C. Cyclic schemes for PDE-based image analysis. International Journal of Computer Vision, 2016; 118(3): 1–25.

Yang X, Cheng K T. Local difference binary for ultrafast and distinctive feature description. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013; 36(1): 188–194.

Fischler M A, Bolles R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Readings in Computer Vision, 1987; 726–740.

Meshoul S, Batouche M. Combining extremal optimization with singular value decomposition for effective point matching. International Journal of Pattern Recognition and Artificial Intelligence, 2003; 17(7): 1111–1126.

Furukawa Y, Ponce J. Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010; 32(8): 1362–1376.

Besl P J, Mckay H D. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1992; 14(2): 239–256.

GB/T10651-2008, Fresh apple. Beijing: Standards Press of China, 2008.

Likas A, Vlassis N, Verbeek J J. The global K-means clustering algorithm. Pattern Recognition, 2002; 36(2): 451–461.

Xie J L, Li L S, Liu G X, Qiu C J. Measurement of complex closed lumen volume and surface area based on industrial computed tomography. Nuclear Electronics & Detection Technology, 2014; 5: 580–586.




Copyright (c) 2021 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office