Measurement of the banana pseudo-stem phenotypic parameters based on ellipse model

: The measurement of banana pseudo-stem phenotypic parameters is a critical way to evaluate the growth status of bananas, and it can provide essential data support for mechanized cultivation operations such as fertilization and pesticide application. Existing studies mainly measure the diameter of banana pseudo-stem as its phenotypic parameter. The banana pseudo-stem cross section was closer to an ellipse other than a standard circle, so the diameter parameter cannot adequately represent the phenotypic characteristics of the banana plant. In this study, an automatic measuring device for banana pseudo-stem phenotypic parameters was developed. The device, which integrates three different types of sensors: a laser ranging sensor, a rotary encoder, and a digital camera, was used to obtain the point cloud and image data of banana pseudo-stem. A K -means point clouds clustering algorithm based on Euclidean distance was proposed. The point cloud of banana pseudo-stem was identified and extracted. A three-dimensional reconstruction algorithm based on the ellipse model was also proposed. The three-dimensional contour of the pseudo-stem was calculated to obtain three types of phenotypic parameters: the long axis length, the short axis length, and the perimeter. Further, a synchronous trigger image acquisition mechanism was used to take pictures of pseudo-stems during measurement. It can be utilized for manual assessment of the growth status of the banana. Field experimental results showed that the three banana phenotypic parameters had a high correlation with the manual measurement results, and R 2 is always more significant than 0.95, the total average measurement error and relative error were only 6.16 mm and 4.38%, respectively, both are within the acceptable agronomy range. In general, this method has good universality for plant stem detection, and the stem phenotypic parameters can be obtained by means of a non-contact test, which is of great significance to the mechanized cultivation of the forest and fruit industry. Citation: Y, Z. Measurement of the banana pseudo-stem phenotypic parameters based on ellipse model. Int J Agric & Biol Eng, 2022; 15(3): 195–202.


Introduction
Bananas are one kind of the four major fruits in the world and are also the main food crops in many developing countries [1,2] . As an important economic crop and food crop, according to the statistics of the Food and Agriculture Organization of the United Nations (FAO), the global banana planting area and annual output continue to grow year after year [3] . It is the world's largest import and export trade fruit, and also one of the important fresh fruits in the world's tropical and subtropical regions [4] . However, during the growth of bananas, mechanized cultivation operations (such as spraying pesticide, and fertilizer application) and field management operations have become indispensable projects [5] .
With the increase of people's awareness of environmental protection, it has been the focus and hot spot of people to establish different mechanized cultivation models (such as spraying pesticide, and fertilizer application) for different fruit trees to achieve energy conservation and environmental protection involving citrus [6,7] , litchi [8] , apple [9] , etc.
It is known that banana was a large perennial rhizome crop that is emitted from the rhizome and forms a pseudo-stem with a height of 3-6 m under the leaf sheath. The leaves are oblong to oval and some are as long as 3.0-3.5 m, 65 cm wide. There are significant differences in the canopy of banana plants with the spindle plants such as citrus, litchi, and apple. It is difficult to detect and calculate banana leaf area information and root information directly. Banana pseudo-stem phenotypic parameters were measured to reflect banana leaf area and root size indirectly, so as to guide mechanized cultivation operations [16,17] .
In recent years, under the assumption that the stem of a plant is an ideal cylinder, researchers have proposed various methods for measuring the plant diameter, and these methods mainly include contact and non-contact measurement methods. Thalheimer [18] developed a device including an optoelectronic reflex sensor, a microcontroller board, and a flexible tape, the flexible tape has white and black bars, to monitor fruit and stem radial growth, and a flexible tape with was used to tight around the stem, then the diameter and circumference were detected through counting the number of bars on the tape. The contact measurement has the advantage of low cost and high precision, but it has the shortage of low efficiency and easy damage to stem growth. With the rapid development of image processing and laser sensor technology, non-contact measurement methods have become widely used in stem diameter measurement. Che et al. [19] chosen Camera VC-4472 and near-infrared light source to build an imaging system, plant stem diameter was measured through image acquisition, soble edge detection, and horizontal distance calculation of edge points. Bao et al. [20] proposed a high-throughput field-based robotic phenotyping system that performed side-view stereo imaging for sorghum plants, using a user-interactive stem diameter extraction method, the stem edge points were chosen by the user from side-view images, then the stem diameter was calculated by these points. Quan et al. [21] designed a stem diameter inspection spherical robot for tomato greenhouse with a binocular camera installed in it. Based on the binocular image, the tomato stem edge was extracted and vertically corrected, the stem diameter can be calculated through corrected tomato stem edge points, and the results showed that the robot measured diameter has a good linear correlation with the manually measured diameter.
In the study of banana pseudo-stem diameter measurement, Song et al. [16] used Kinect 2.0 to extract the color and depth information of the banana pseudo-stem, and converted the obtained depth images into point cloud. The particle cluster optimization algorithm and least-squares fitting were used to obtain the banana pseudo-stem diameter parameter. In general, the machine vision detection method has problems such as high price, poor real-time performance, and serious influence by natural light. Compared with machine vision, lidar can overcome the above problems and obtain the depth information point cloud of crops, and through the processing of these point cloud data, the phenotypic parameter measurement of crop stem can be realized. Xu et al. [17] used the VL53L0X laser sensor to measure banana buds and pseudo-stems in semi-automatic banana bud removal machine, the experimental results showed that under close-range measurement conditions, the VL53L0X sensor has a strong anti-interference ability to the external environment and high accuracy.
Up to recent, most of the plant stem phenotypic parameter measurement methods assume the plant stem was a cylindrical model and get the diameter through the circular fitting. However, there is no standard cylindrical stem in nature, and assuming that the plant was a cylinder will obviously lead to a distortion of its phenotypic characteristics. Ye et al. [22] utilized laser scanner FARO Focus3D X130 to obtain the forest point cloud data, a robust least square ellipse fitting method was proposed to fitting the shape of the stem, numerical results showed that the ellipse model can describe the stem better, and its DBH (Diameter at Breast Height) estimation accuracy was higher than that based on circular fitting methods with an average RMSE of 1.14 cm.
Ellipses are powerful shapes, besides plant stems, the ellipse model can be fitting for almost most fruits and vegetables. Liu et al. [23] used a multi-elliptical boundary model to detect citrus tree trunks and citrus fruit grown in the natural light environment, the experimental results showed that the correct rate of fruit recognition was 90.8%. Aiming at the overlapping problem in agriculture. Julio et al. [24] proposed a new ellipse matching system to identify individual plants in agricultural overlap scenarios, and the ellipse fitting examples of ellipse model to cucumber, tomato, and serrano pepper were discussed, which is of great significance to the robotic harvesting.
Based on the ellipse model, this study proposed a novel method for extracting banana pseudo-stem phenotypic parameters, which can better describe the phenotypic characteristics of banana pseudo-stems. The main research works of this study were as follows: 1) Designing a data acquisition system with a laser range sensor, rotary encoders, and camera; 2) A K-means point cloud clustering algorithm based on Euclidean distance was proposed to extract the pseudo-stem point cloud data; 3) Pseudo-stem reconstruction and parameter extraction through three-dimensional ellipse reconstruction algorithm.

Data acquisition platform
The data acquisition platform was a four-wheel cart that moved easily between rows, the key components of the system are the laser ranging sensor (GY-53 VL53L1X, STMicroelectronics), rotary encoder (E6B2-CWZ1X, OMRON), and digital camera (Camera shield, seed wiki), and all sensors were installed on this platform. The laser ranging sensor was configured with a measurement range of 0.05-4.00 m, and a range resolution of 1 mm, mounted on the platform, aiming at the position 1 m above the ground of the pseudo-stem to ensure the best view, and measured distances between the sensor and target objects based on the time-of-flight principle. The rotary encoder was mounted on the central position of the rear wheel axle of the platform, and the forward distance of the platform was recorded through a pair of gears with a transmission ratio i=1.5. The digital camera has a resolution of 640×480 pixels, and was mounted on the same height and the same plane as the laser ranging sensor, takes photos of each banana pseudo-stem and saves them, which can be used to determine which pseudo-stem was the measurement result, so as to compare with the manual measurement results. The data acquisition system includes a Secure Digital card (SD card), all sensors connected to the Arduino UNO (a type of microcontroller), and the battery pack that powers the Arduino UNO. In order to control the acquisition system, a switch was designed. When the power is turned on and the switch is pressed, the laser ranging sensor, rotary encoder, and digital camera will start acquiring data at the same time, and the system will store data on an SD card. The data acquisition system diagram is shown in Figure 1.
Note: GY-53 is a type of laser ranging sensor; Arduino UNO is a type of microcontroller; SPI: Serial Peripheral Interface; SD card: Secure Digital card.

Figure 1 Data acquisition system diagram
Push the platform at a proper distance (range 20-50 cm) away from pseudo-stem, the data acquisition system set up a threshold before measurement, and then the operator presses the acquisition switch. The red LED flashing and the data acquisition system starts at this time, the laser ranging sensor and the rotary encoder begin to collect data and save them to the SD card, and the acquisition system also compares the measured value laser ranging sensor with the threshold value. When the measured value of the laser ranging sensor was less than the threshold three consecutive times, the acquisition system sent a high-level trigger signal to the camera shield, and the camera shield take a photo and save it to the SD card, marking the value measured by the rotary encoder at the same time, and then the trigger signal was locked to a low-level state until the measured value of the laser ranging sensor was bigger than the threshold three consecutive times, and then unlocked. When the data acquisition was finished, press the acquisition switch again, and the red LED will stop flashing and the measurement was finished. Figure 2 shows the working flow chart of the data acquisition system. Figure 2 Workflow chart of pseudo-stem phenotypic parameter data acquisition system

Field experiment setup
Field experiments were carried out in the banana orchard at Zhongheng Mountain, South China Agricultural University. Guangzhou, Guangdong Province, China in November 2019. The banana planting variety was Baodao banana, and the banana orchard adopts a wide-narrow row planting model with a plant spacing of 1.5 m, a narrow row spacing of 1.5 m, and a wide row spacing of 3 m. The data acquisition platform was carried out manually between rows and moves forward slowly. The wheel direction was adjusted to control the distance between the laser-ranging sensor and the pseudo-stem center. For developing and testing the proposed algorithm, three rows of bananas were selected for the experiment in the banana orchard on November 2, 2019 (a sunny day), and a total of fifteen plants with different growth stages and different pseudo-stem sizes were selected for manual measurement and analysis. During the manual measurement, on the one hand, a soft ruler was used to measure the circumference of the pseudo-stem by winding at the height of 1 m above the ground, on the other hand, a Vernier caliper was used to measure the long axis length, short axis length, and projection width of the banana pseudo-stem at the height of 1 m above the ground. The manual measurement method is illustrated in Figure 3

Phenotypic parameters measurement
The flow of phenotypic parameters measurement is shown in Figure 4, phenotypic parameters were extracted by filtering, noise reduction, clustering, and ellipse fitting.
As shown in Figure 4, the flow of phenotypic parameters measurement mainly consisted of 1) Convert the data collected by multiple optical sensors into normalized point cloud coordinates and synchronizing it with the ground truth measurement; 2) The synchronous trigger image acquisition mechanism was used to filter and denoise point cloud data to eliminate noise caused by interference such as weeds, support rods, and fallen leaves; 3) Clustering the collected point cloud data to extract the point cloud data points corresponding to the banana pseudo-stem; 4) The direct least square method was used to reconstruct the point cloud data of pseudo-stem and extract the phenotypic parameters of pseudo-stem. The entire process was implemented by Matlab2019a software under the Windows10 operating system, and the details of the mentioned stages were illustrated in the upcoming subsections. The forward direction of the platform was defined as x-direction, the direction perpendicular to the forward direction and pointing to the banana pseudo-stem was defined as y-direction, and the transformed coordinate of the normalized point cloud is defined as (x i , y i ), i∈(1, 2, 3, …, n), where the data collected by the rotary encoder was converted into x value after calculation, and the y value is acquired by the laser ranging sensor.
The rotary encoder was installed on the small gear, and the output value of the rotary encoder is the pulse value. For each pulse generated by the rotary encoder, the rotation angle of the small gear (θ) can be expressed as, Accordingly, the rotation angle of the big gear (r) and the wheel for each pulse can be expressed as, 360 r fi°= The circumference of the wheel is C d = π (3) Thus, the distance corresponding to the advance of n pulses can be obtained as, n d x fi where, n means the number of pulses currently collected; d means the wheel diameter; i means the ratio of gear; f means the sampling frequency of the rotary encoder.

Synchronous trigger image acquisition mechanism
Due to the presence of some weeds, support rods, or broken leaves, the data measured with this device have some noise points, so must filter these data caused by the interference to get the correct point cloud data corresponding to the pseudo-stem. In order to quickly and accurately obtain the pseudo-stem phenotypic parameters of each plant, selecting the point cloud data corresponding to each plant's pseudo-stem was the only need for analysis and processing. To this end, a synchronous trigger image acquisition mechanism was proposed. The specific principle of the synchronous trigger image acquisition mechanism was as follows: when the measured value of the laser ranging sensor is continuously less than the threshold value for three consecutive times, the collecting device will send a high-level trigger signal to the camera shield, which will simultaneously take a photo and save to SD card. After that, the trigger signal will be locked at a low level until the measured value was bigger than the threshold for three consecutive times. The workflow has been illustrated in Figure 2.

K-means cluster analysis of point cloud data
After the point cloud data were acquired, determine which points belong to the banana pseudo-stem points, which points belong to the background noise or which irrelevant points were problems that need to be solved. Xu et al. [17] measured the pseudo-stem parameters of the banana by identifying the edge points of the banana pseudo-stem and classifying the point cloud between the two edge points as the banana pseudo-stem point cloud. However, considering that the point cloud data were discrete, the edge points may not correspond to the actual edges of the banana pseudo-stem. At the same time, the method of judging the edge points of the banana pseudo-stem was too simple and does not consider the impact of measurement errors and environmental interference.
Therefore, this study proposed the clustering algorithm to classify banana pseudo-stem point cloud data to segment banana pseudo-stem point cloud, background point cloud, noise point cloud, and abnormal points.
According to the scale of clustering, there are three methods of point cloud clustering: distance-based clustering, density-based clustering, and interconnectivity-based clustering. Considering that during the point cloud acquisition of banana pseudo-stems, when the laser ranging sensor detects the pseudo-stems, the adjacent points are close to each other and the distance between them was not too far away. When it's an interfering point, the mutation between adjacent points is large. When it's the background point, the background points were the closest because they have been preprocessed to a uniform value. Therefore, this study performed the clustering analysis on the collected point cloud data by extracting the distance characteristics between adjacent points.
Euclidean distance is a classical distance representing the distance between samples, which can be expressed as, 2 1 ( , ) ( ) In order to quickly cluster the point cloud and obtain the pseudo-stem point cloud data, a classical K-means clustering algorithm was selected for clustering analysis in this study, and the K value was selected as 3.

Ellipse model
For the point cloud data, it was regarded as ellipses and used direct least squares elliptic fitting to find the best fitting for the points. The mathematical representation of the conics equation of the ellipse can be expressed as, According to the principle of direct least squares, the fitting of an ellipse be approached by minimizing the summation of squared algebraic distances, the fitted objective function can be expressed as In order to avoid the trivial solution a=0 6×1 , the constraint f =−1 was applied. Note that f does not depend on edge point (x, y), so Equation (6) was a least squares problem.

Direct least squares fitting for banana pseudo-stem
After classifying the pseudo-stem point cloud data, these discrete data points can be used to reconstruct the pseudo-stem. Through field measurements, banana pseudo-stem cross sections tend were found to be elliptical in shape, therefore, the pseudo-stem reconstruction uses an ellipse model to approximate the fit. Ellipse fitting based on the direct least square method was one of the common methods, the basic idea of the algorithm was to minimize the distance error by minimizing the constraint condition 4ac−b 2 =1.
The Lagrange multiplier algorithm was first introduced to obtain the system of equations, and then the system of equations was solved to obtain the optimal fitting ellipse. The advantage of this algorithm is that it supports multiple sets of data for fitting, has high robustness to noise, and the more data used for fitting, the more accurate the fitting result [25] . The specific principle is as follows: Considering that the point cloud data consists of n data points (x i , y i ), i∈ (1, 2, 3, …, n), and find an ellipse as, ax 2 +bxy+cy 2 +dx+ey−1=0 (8) so as to minimize the sum of the distances between these points to the edge of the ellipse.
Setting a=[a, b, c, d, e, f ] T and x=[x 2 , xy, y 2 , x, y, 1] T , so the equation can be expressed as ax=1. Then the optimization problem of fitting ellipse with n discrete points (x i , y i ), i∈ (1, 2, 3, …, n) can be expressed as, 2 min Da (9) According to the Lagrange multiplier algorithm, a new equation can be obtained as follows: Solve ( ) 0 a L a ∂ = , introduce the Lagrange factor λ, and get the following two equations: λ = C Sa a (13) (14) where, S means the scatter matrix D T D.
Solve the eigenvalues and eigenvectors of the Equation (13), if (λ i , u i ) solves Equation (13) then so dose (λ i , μu i ) for any μ and from Equation (14), the value of μ i while μ i u i T Cu i =1 is giving as,

Results
The experimental verification of the proposed algorithm was carried out in this study and the results are described in the following sections.

Per pseudo-stem point cloud extraction results
Due to the presence of some weeds, support rods, or broken leaves, the data measured have some noise points. Figure 5 shows a set of measurement data and analysis results selected at random, a set of point cloud data obtained from the field experiment measurement was compared with the photos taken by the synchronous trigger mechanism at the point clusters where the measurement results are abrupt. As shown in the first peak in Figure 5a, after comparing the photos, here is a banana pseudo-stem (Figure 5b), this set of data was retained and the normalized transformation of the point cloud coordinates was performed to finally obtain the point cloud of a single pseudo-stem for further analysis and extraction of pseudo-stem phenotypic parameters. The experimental results showed that the accuracy of point cloud data of a single banana pseudo-stem extracted by this method was 100%, which can avoid misjudgment caused by interfering objects such as interlacing pseudo-stems, weeds, and support rods, broken leaves, and so on.

Cluster analysis results
In order to obtain high precision pseudo-stem point clouds, the selected fifteen banana pseudo-stem point clouds data were analyzed by K-means clustering algorithm based on Euclidean distance. Continue to select the first peak point cloud in Section 3.1 as an example (Figure 6a) for cluster analysis. The clustering result is shown in Figure 6b and Figure 6c. a. Before cluster analysis b. Scatter plot based on 3D distance c. Clustering results Figure 6 Cluster analysis of pseudo-stem point cloud data It can be clearly seen that after cluster analysis using the K-means clustering algorithm based on Euclidean distance, the 3D scatter classification results based on the 3D distance (x-axis: Depth data measured by lidar, y-axis: Euclidean distance between adjacent points on the left, and z-axis: Euclidean distance between adjacent points on the right) are shown in Figure 6b. It can effectively distinguish pseudo-stem point cloud data, noise point cloud data, and background point cloud data. The classification results after cluster analysis in Figure 6c show the classification results more intuitively.
The blue point cloud means the segmented banana pseudo-stem point cloud, and the red point and black point cloud are the noise point cloud and background point cloud. The red point and black point were eliminated, and the blue point was retained to obtain the final pseudo-stem point cloud data, which was of great significance for the accuracy of the 3D reconstruction of the pseudo-stem and the extraction of phenotypic parameters.

Phenotypic parameter extraction results
In this study, the fifteen selected banana pseudo-stems were fitted with the ellipse fitting algorithm of the direct least square method proposed above to obtain the long axis length, the short axis length, and the circumference of the pseudo-stem. A banana pseudo-stem was randomly selected (Sample 6 in Table 1)    The three phenotypic parameters of long axis length, short axis length, and pseudo-stem circumference were obtained by fitting as 106.15 mm, 93.97 mm, and 319.43 mm, compared with manual measurement, the measurement error and relative error are −2.95 mm (−2.70%), −3.23 mm (−3.32%) and 7.43 mm (2.38%), it has high accuracy and meets the requirements of mechanized cultivation operations such as spraying pesticide and fertilizer application.
In order to further study the advantages and disadvantages of the proposed algorithm, the accuracy of the method was studied by comparing it with manual measurement results, and the comparing results are shown in Table 1 and Figure 8.
The measurement results in Table 1 show that among the three phenotypic parameters, the average measurement errors of the long axis length, short axis length, and pseudo-stem circumference are only 4.58 mm, 5.13 mm, 8.75 mm and the average relative errors are only 4.17%, 6.12%, 2.86%. With the change of phenotypic parameters in the growing period, the measurement error always stays within this normal range.
In addition, the system measurement results have a high correlation with the manual measurement results. On the other hand, it could be seen that the manual long axis measurement results have an obvious difference from short-axis measurement results, the average difference between the long axis and short axis was 21.24 mm, so the ellipse model was more suitable for representing banana pseudo-stem structure. Figure 8a shows that the correlation of long axis length between manual and system measurements is y = 0.94932x+2.85978, R 2 =0.97511 (16) Figure 8b shows that the correlation of short-axis length between manual and system measurements is y = 0.89153x+11.37893, R 2 =0.97318 (17) Figure 8c shows that the correlation of pseudo-stem circumference between manual and system measurements is y = 0.97449x+2.99675, R 2 =0.99375 (18) The experimental results showed that the system measurement has a high correlation with the manual measurement results, the method proposed in this study had high accuracy and the measurement error was always within an acceptable range.

Pseudo-stem photo collection results
Using the synchronous trigger image acquisition mechanism to synchronize the photos of banana pseudo-stems during the collection of pseudo-stem point cloud data, can not only be used for comparison and identification to accurately extract point cloud data of a single pseudo-stem, and can be used to manually evaluate the growth status of bananas. Figure 9 shows a part of the pseudo-stem photos obtained through the synchronous trigger image acquisition mechanism. It can be seen from the photos of the pseudo-stem that the synchronous trigger image acquisition mechanism can clearly capture pseudo-stem images. As shown in Figure 9a, the outer leaf sheath gradually separates from the pseudo-stem, dry and falling during the growth of the pseudo-stem, it was obvious that the pseudo-stem's cross section was closer to an ellipse than a circle. On the other hand, the influence of petiole (Figure 9b) and sword sucker (Figure 9c) may cause slight deviations in the point cloud data collection process. The skewed growth status of the pseudo-stem (Figure 9d) may cause slight deviations during the 3D reconstruction and ellipse model fitting. These are also the main reasons for the errors in the fitting results of the banana pseudo-stem ellipse model described above.
The photos collected through the synchronous trigger image acquisition mechanism provide important reference significance for the analysis of the measurement results of the pseudo-stem phenotypic parameters.

Conclusions
Detection methods and extraction of phenotypic parameters of banana pseudo-stems in the natural environment is an important technology for the development of mechanized cultivation operations such as spraying pesticides and fertilizer application. This study proposed a novel measurement method based on the ellipse model for detecting banana pseudo-stem and extracting phenotypic parameters in a natural environment. The method has high detection accuracy, high detection efficiency, and great robustness. At the same time, compared with the method of banana pseudo-stem detection based on a optical camera, the laser ranging sensor has stronger resistance to natural light interference. The ellipse model breaks through the limitation of the cylinder model for measuring phenotypic parameters in the previous methods. In the measurement and extraction of three phenotypic parameters: long axis length, short axis length, and circumference, the total average measurement error was 6.16 mm, and the total average relative error was 4.38%. The experimental results showed the accuracy of extracting the point cloud of a single pseudo-stem was 100%.
After cluster analysis and three-dimensional reconstruction, the average measurement errors of the long axis length, short axis length, and pseudo-stem circumference were 4.58 mm, 5.13 mm, 8.75 mm, and the average relative errors were 4.17%, 6.12%, and 2.86%, which effectively improved the detection efficiency and accuracy. Noteworthily, the experimental results showed that the average value difference between the long axis and the short axis of the banana pseudo-stem reached 21.24 mm during the whole growth period, which highlights the advantages of using ellipse model fitting to extract the phenotypic parameters, the ellipse model can more effectively characterize the phenotypic parameters of banana pseudo-stem. Meanwhile, this method has good universality for plant stem detection, and the pseudo-stem phenotypic parameters measurement results based on ellipse model can provide key data to guide banana orchard spraying robots and fertilizer robot operation.