Review of smart robots for fruit and vegetable picking in agriculture

The fruit and vegetable picking has posed a great challenge on the production and markets during the harvest season. Manual picking cannot fully meet the rapid requirements of each market, mainly due to the high labor-intensive and time-consuming tasks, even the aging and shortage of agricultural labor force in recent years. Alternatively, smart robotics can be an efficient solution to increase the planting areas for the markets in combination with changes in cultivation, preservation, and processing technology. However, some improvements still need to be performed on these picking robots. To document the progress in and current status of this field, this study performed a bibliometric analysis. This analysis evaluated the current performance characteristics of various fruit and vegetable picking robots for better prospects in the future. Five perspectives were proposed covering the robotic arms, end effectors, vision systems, picking environments, and picking performance for the large-scale mechanized production of fruits and vegetables in modern agriculture. The current problems of fruit and vegetable picking robots were summarized. Finally, the outlook of the fruit and vegetable picking robots prospected from four aspects: structured environment for fruit planting, the algorithm of recognition and positioning, picking efficiency, and cost-saving picking robots. This study comprehensively assesses the current research status, thus helping researchers steer their projects or locate potential collaborators.


Introduction
The fruit and vegetable industry has been the largest and fastest-growing part of the world's market of agricultural production. Among them, China is one of the primary fruit and vegetable producer countries, particularly with a huge total planting area. The production of fruit and vegetable in China has ranked the top in the world. Specifically, the total planting area of fruits and vegetables in China in 2018 was 1.1875×10 7 hm 2 and 2.0439×10 7 hm 2[1] , respectively. Taking China as an example, the phenomenon of population aging has gradually intensified in recent years, and the proportion of the population aged 65 and over to the total population has been increasing year by year, from 5.6% in 1990 to 11.9% in 2018. With the development of social diversification, the labor force has been continuously transferred to other industries in society, and the number of people engaged in agricultural production has become less and less. The reduction of labor resources has led to the problems of difficult employment and high labor costs in agricultural production. Therefore, the current harvesting of fruit and vegetable cannot fully meet the large-scale production in recent years, due mainly to high labor requirements [2] . Moreover, the picking activities can pose some danger to the health of operators [3] . Therefore, it is highly urgent to develop efficient automatic fruit and vegetable picking robots for a higher automation level in agricultural production. A smart harvester has been one of the most important artificial intelligence (AI) robots for fruit and vegetable harvesting in modern agriculture.
In recent years, various fruit and vegetable picking robots have been proposed by researchers from all over the world, such as apple picking robots, tomato picking robots, kiwi fruit picking robots, strawberry picking robots, and so on.
In this article, an extensive literature search was conducted. From the perspective of different picking objects, this review evaluated the performance characteristics of various current fruit and vegetable picking robots.
This study comprehensively assesses the current research status and existing problems of fruit and vegetable picking robots and provides help for related research scholars' projects.

Research status 2.1 Apple picking robots
In this section, five types of apple harvesting robots are going to be reviewed, in terms of structural design, simulation, vision system, and performance. 2.1.1 A robot with a spoon-shaped end-effector Zhao et al. [4] developed an automatic apple harvesting robot. This type of apple harvesting robot consists of a 5-degree of freedom (DOF) structure (PRRRP) manipulator, autonomous mobile driving of crawler-type vehicle, spoon-shaped end-effector, Figure 1. Figure 1 An apple-picking robot using a spoon-shaped end effector The spoon-shaped end effector is specifically designed to equip with special pressure sensors on the left and right ends of the inner side. As such, an excessive clamping force can be effectively avoided to damage the apples, due to the real-time feedback of pressure value acting on the surface of the apple during the picking process. An electric cutting device is also equipped on the side of the end-effector to separate the fruit from the stalk. A plastic pipe is positioned below the end effector. Consequently, the picked apples can enter the collection device along the plastic pipe.
Mathematical models [5] were also established using ANSYS software to clarify the influence of two types of fingers in the robot end-effector on the apple during the grasping process. The comparison demonstrates that the arc-shaped finger can cause less damage to the apple during the picking, compared with the rest. The theoretical threshold of apple damage was also achieved for the picking operation.
In a machine vision system for the apple picking robot, the color features are used the Hue and Saturation components in the HSI color space. Furthermore, the shape features are utilized the invariance technique of rotation, scale, and translation (RST). Specifically, some statistical parameters are also used, including the circular variance, variance ellipse, tightness, as well as the ratio between perimeter and square area.
A Support Vector Machine (SVM) training classifier is used to realize the apple recognition in the image, according to the extracted color and shape features. Among them, the radial basis function is set as a kernel function.
In performance, the picking success rate of the apple robot is 86% under the laboratory environment, while the average picking time for an apple is 14.3 s in spite of setting time. By contrast, the apple picking robot can recognize 39 apples within 10 min in an orchard, where 30 apples were successfully picked, while 6 apples were not picked, and 3 apples fell to the ground after being picked, due to their small size. Consequently, the average picking success rate is 77% taking 15.4 s on average. 2.1.2 A grasp-robot using negative pressure adsorption Another apple harvester can be the grasp-type automatic robot using the negative pressure adsorption developed by Baeten et al. [6] The harvester can be posited on a platform mounted behind an agriculture tractor [6] , as shown in Figure 2. A 6-DOF industrial robotic arm is used in the harvester. The end-effector adopts a typical structure similar to the suction nozzle in a vacuum cleaner. The purpose is to avoid any damage to the apple while providing a firm suction force.
A vision camera is also installed inside the harvester. Once the camera detects that the apple is within the working range of the suction nozzle, the negative pressure can be activated to absorb the apple, and thus the suction nozzle can grasp and pick the apple via the rotating and gently tilting operation. As such, the apple picking robot can take 8-10 s on average to pick an apple, and the picking success rate is about 80%. Nevertheless, some apples cannot be picked, particularly difficult to detect and reach by the robotic arm in this experiment. Davidson et al. [7][8][9] have developed an automatic apple harvesting robot, as shown in Figure 3. The robotic arm adopts a 6-DOF tandem structure. A novel design is utilized to mimic a human index finger for the end-effector.
Specifically, an under-actuated clamp with an angle of 30° is installed on the upper part of the robot arm.
During the picking process, the end-effector can grasp and rotate the fruit in real-time, then drive the clamp to hold the fruit stem, and finally to generate a shearing force at the fruit stem, thus causing the fruit stem to break. This picking method can effectively reduce the high demand for the picking power and relieve the probability of damage to the fruit during the picking process.
In the vision system, a circular Hough transform is utilized to identify independent and clear fruits without shading by leaves or stems in an orchard. A Blob Analysis can also be made to identify partially visible fruits iteratively. An exposure fusion technology can be used to capture five images with low to high exposure. Among them, the best area of each image is calculated for a well-exposed image, thereby improving image quality. The total time required to capture all five images is about 0.34 s, where each image takes 68 ms on average. Correspondingly, the vision system has been tested in the clusters and complex canopy structures, where the accuracy of fruit recognition exceeds 90%.
The specific procedure can be set when picking. The harvesting robot first detects and locates each apple in the region of interest (ROI) view using the RGB-D camera system. The picking priority of each apple is then determined by the expert system. Subsequently, the apple coordinates are transferred to the mechanical system, where the 6-DOF manipulator can be used to drive the end-effector to move towards the corresponding position for the fruit grab and picking. More importantly, the grabbing adopts an open-loop control mode. As such, this procedure can be repeated several times, until all the marked apples are picked. Finally, the vehicle can move back to gain the next ROI view for picking. An average time of 1.5 s is required for the harvesting robot to locate each fruit. Furthermore, the 6-DOF harvesting system (six-DOF manipulator + under-actuated end-effector) successfully picked 127 fruits from 150, with an overall success rate of 84%, and an average picking time of 6.0 s for each fruit.
Besides, the designer clearly stated that this type of harvesting robot can be compatible with the modern apple orchard. Therefore, a V-shaped structure planning for the apple orchard can reduce the obstacle avoidance of the picking robot, thereby simplifying the operation environment for the better picking efficiency of the harvesting robot. The American company Abundant Robotics has developed another type of apple harvesting robot using negative pressure adsorption, as shown in Figure 4. Specifically, a feasible non-contact end-effector can be adopted, where the negative pressure is generated to produce the adsorption force for the apple near the picking opening. As such, the force from negative pressure adsorption is used to complete the fruit picking. Particularly, this type of apple harvesting robot has entered commercialization now [10] . Yang et al. [11,12] designed an apple-picking robot using a multi-DOF parallel manipulator, as shown in Figure 5. The end effector of the robot also adopts the negative pressure. Specifically, a three-level buffer is equipped with the end effector for the effective protection of fruits [13] . In addition, the end effector can be used to fully filter some impurities, such as blades residues, thereby preventing their blockage from the generating device of negative pressure in the picking robot [14,15] . Figure 5 Another apple harvesting robot using negative pressure adsorption

Tomato picking robot
Six sorts of tomato picking robots can be evaluated at present in this section.

A robot using adsorption
Wang et al. [16] developed a tomato picking robot, as shown in Figure 6. The picking claw is composed of three units: fruit adsorption, clamping, and screwing.
The specific picking procedure can be summarized as follows. The vacuum generator can be used to adsorb the tomatoes when moving towards the target. At the same time, the telescopic cylinder is utilized to extend the sleeve, until the fruit is fully inserted into the sleeve, thereby opening the airbag for the clamp of tomato, and finally, the sleeve can be used to rotate to separate the tomato fruit from the stalk.
This type of harvesting robot is designed for the picking of tomato fruits grown in the greenhouse. Furthermore, the tomatoes are required to be planted in rows and arranged on both sides of the track. As such, the harvesting robot can travel straight along the center of the track. More importantly, the ripe fruits to be picked can better be located above the platform of the track, particularly where the tomatoes are densely packed in the case. In addition, the operation area of 100-600 mm 2 can be convenient for this type of robot to carry out picking.
The performance of the harvesting robot demonstrated that it took 4.0 s to locate the tomato fruit, 12.0 s to move the robotic arm, 8.0 s to pick the fruit, and 12.0 s to reset the robotic arm in the field experiment. The picking success rate of the robot is achieved about 83.9%. Additionally, Wang et al. [17] designed a new end effector for picking the fresh tomatoes, as shown in Figure 7.
Four components are also composed of the fruit stalk clamping, fruit clamping, and separation mechanism, as well as one control unit. Specifically, the fruit clamping mechanism is equipped with a sleeve and some airbags for the collection. Once the target fruit enters the sleeve, the handle clamping mechanism can hold to cut  Wang et al. [18] developed a tomato harvesting robot used in a greenhouse, as shown in Figure 8. A four-wheel independent steering system is also designed to automatically navigate and avoid some obstacles when harvesting. The 5-DOF manipulators are used, of which 4-DOF comes from the robotic arm, and 1-DOF comes from the end effector.
The end effector adopts a commonly-used shear grasping method at present. Correspondingly, the picking action includes the tomato grasping, cutting, and separation. When working, the robot can move automatically along the path predetermined by the navigation system, thereby detecting whether there are some ripe tomatoes in the operation area, according to the color difference between the ripe tomatoes and the background. Once the ripe tomatoes are detected, the robot can stop moving. Then, a systematic evaluation can be made further to determine the spatial location of ripe tomatoes and some obstacles that may impede the harvesting using the control system. After that, the robotic can move the arm towards the corresponding position and pick the tomato with the end-effector.
Various machine visions are selected for the harvesting robot. Specifically, the normalized color difference method is utilized to collect the grayscale images. The Otsu algorithm is also selected for segmentation. An ellipse template matching is used to identify the overlapping tomatoes by leaves or stalks in the greenhouse.
In field experiment, 300 static images of the tomato field are captured under the different light intensities (different times on the same day) and random backgrounds. 298 images have been successfully recognized, indicating the success rate is 99.3%, with an average recognition time of 1.5 s per image. Dynamic tests were also performed at different positions by 100 times. Consequently, the minimum positioning error was achieved less than 10 mm, when the distance of picking position was less than 600 mm. Anyway, the picking efficiency of the harvesting robot is about 15 s/unit, while, the picking success rate is about 86%.

A dual-arm tomato harvesting robot
Ling et al. [19] developed a dual-arm tomato harvesting robot, as shown in Figure 9. This sort of harvesting robot can be deployed in the Venlo-type greenhouse, where the mobile platform can move on the heating pipes along crop rows. Two three-DOF robotic arms are symmetrically distributed, similar to the Selective Compliance Articulated Robot Arm (SCARA) with one moving pair and two rotating pairs. The end-effectors of the left and right arm are equipped with a cutting gripper and a vacuum cup, respectively. As such, the dual-arm manipulator can work smoothly, by which the vacuum cup can grasp the target fruit, and then the cutting gripper detaches the fruit from the plant.
A sliding window method can be used in the visual system of the harvesting robot, thereby extracting Haar-like features within each sub-window. An AdaBoost classifier can also be utilized to detect the tomatoes. The average pixel value (APV) classifier can be selected on the I-component image for color analysis, thereby merging the above two detections for the final image.
A total of 171 target tomatoes were tested using 60 sample images. The recognition success rate was 96.5%, and the average detection speed was 85 ms/image.
The experiments of robotic harvesting were conducted on the potted tomatoes in the greenhouse. 80 ripe tomatoes were also randomly selected as the harvesting objects for the robot. Totally, 70 target tomatoes were harvested by the robot. The success rate of robotic harvesting is 87.5%. The average harvesting speed is about 29 s/fruit, except the cruise time for the robot moving on the heating pipes in the greenhouse.

A tomato picking robot in Japan
Kondo et al. [20][21][22][23] did a lot of research work on the tomato-picking robots in Japan. Visual feedback has been used to pick the cherry tomatoes in 1996 [24] . The image processing was also made to identify and locate the entire tomato cluster, including threshold, filter, and segment. The center of a single tomato was also calculated for the single fruit picking.
This type of harvesting robot is equipped with a 7-DOF robotic arm and a suction end-effector. Among them, a photoelectric sensor can rapidly and accurately detect the tomato in the proper position, and then cut the fruit stem with pliers. The picking success rate is 70%, and the average speed is about 3-5 s/fruit.
In ordinary tomatoes, Kondo et al. [22] proposed a two-finger picking end-effector, and then improved in 1998. The improved end-effector can adapt to the different stalk lengths, where the picking success rate increases from 85% to 91%, and the picking efficiency is 15 s/fruit.
A new end-effector was also developed in 2010 [23] . This new type of robot can concurrently pick a whole bunch of tomatoes rather than only for a single one, indicating the higher picking efficiency. The picking speed is about 15 s/bunch, but the success rate is relatively low, only about 50%. 35% of the failures can be attributed to the large profile of end effectors that are hindered by the leaves.

Tanigaki's picking robot
Yaguchi et al. [25] developed a rotary claw end-effector. Three claws are first approaching to clamp the tomato after the end effector is positioned on the target fruit. The tomato was driven to rotate relative to the stalk, and then to pull away from the stalk for the harvest.
In the recognition, the hue, saturation, intensity (HIS) color space is first used to extract the color features. The Euclidean distance is then selected to cluster the point cloud. Finally, the tomato is identified by spherical fitting. It takes about 0.2 s in total.
Particularly, the type of robot is equipped with a 6-DOF robotic arm and a stereo camera suitable for direct sunlight. The picking efficiency is about 85 s/fruit for the second tomato robot competition in the greenhouse. After the optimization of arm movement, the picking efficiency can reach 23 s/fruit under natural light. The picking success rate is about 60%. The failures include the failure to grasp, rupture of the calyx, and grasping multiple fruits. A case study was also conducted on a dual-arm robot to imitate the human behavior of picking [26] , indicating much more feasibility.
Another cherry-harvesting robot developed by Tanigaki et al. [27] has a 4-DOF manipulator, including 3-DOF for rotation and 1-DOF for movement. A 3D vision sensor is equipped with infrared laser diodes, where both laser beams can simultaneously scan the target. In image processing, the locations of the fruits and obstacles can be recognized to determine the trajectory of the end effector for the picking and obstacle avoidance.
As shown in Figure 10, the target fruit trees in the orchard can be planted in a row, and trim the branches to grow vertically, in order to facilitate identification and picking. After that, the target fruit can be sucked by vacuum. At the same time, the small end-effector can move the finger to clamp the fruit stem. Figure 10 Tanigaki's picking robot experimental environment Nevertheless, 41% of the tomatoes cannot be identified in the experiment, due to the hiding behind the leaves in the structured planting and the pruning of the branches. As such, the fingers of the end-effector are equipped with soft rubber components ( Figure  11), in order to separate from another fruit or obstacles when entering between the two fingers. Figure 11 End effector of Tanigaki's picking robot 2.2.6 Picking robots in China A picking robot system developed by Li et al. [28] includes a crawler-type navigation walking platform equipped with a binocular stereo vision system, a 4-DOF articulated robotic arm, as well as a grasping and shearing two-finger pneumatic end-effector. The picking success rate is 86%, and the picking efficiency is 28 s/fruit.
Zhao et al. [29] developed a human-robot collaboration dual-arm picking robot. Two 3-DOF robotic arms are equipped with different end-effectors, where one is used to grasp tomatoes, another is to pick tomatoes. The heating tube is also taken as the guide rail in the greenhouse. Moreover, the vision system contains a 3D reconstruction image display, a robot motion simulation, and a command module for a better graphical interface. However, the reconstruction of 3D images is often restricted by the complex surroundings.
Much more running time is also consumed to directly result in the longer cycle time of picking. Liu et al. [30,31] also conducted a series of researches on the tomato picking robots, including the laser cutting device of the end-effector, the vacuum suction cup [32] , the mechanical damage of the fruit [33] , the creep characteristics [34] , and the clamping collision model [35] .
Chiu et al. [36] designed a four-finger end-effector with a nozzle, as shown in Figure 12. The suction nozzle can absorb the fruit for better grip performance. The four fingers can grasp the tomato to realize the picking via rotating and pulling. Furthermore, the material of four fingers is made of foam rubber in contact with the tomato, indicating free of damage. In addition, a five-axis tandem robotic arm is selected in the harvesting robot. Consequently, the picking success rate can reach 92%, and the average picking time is 74.6 s.

Strawberry picking robots
This section reviewed 13 types of strawberry-picking robots in recent years.

A picking robot for high-ridge strawberries
Arima et al. [24] developed a multifunctional robot covering the operation from the spraying, harvesting, and grading for the high-ridge strawberries.
In harvesting, a three-DOF rectangular coordinate robotic arm is selected in the robot. The specific harvesting procedure can be as follows. First, the mature strawberry is recognized by the color CCD camera. Then, the target strawberry is separated from the surrounding strawberry by the end effector. Finally, the target fruit is cut by the national stem. Correspondingly, there is also a relatively smaller impact on the non-target strawberries, compared with the suction end effector.

A picking robot with four-DOF manipulator
Nagata et al. [37] also developed a strawberry picking robot with a four-DOF Cartesian coordinate manipulator. The end effector of the robot adopts a pneumatic clamping, in order to ensure that the strawberry fruit remains intact during the picking using the integrated clamping and cutting. As such, this type of robot can also effectively reduce fruit damage. It is very necessary to ensure that the air supply is appropriate when cutting the fruit stem. Consequently, the cutting success rate can be close to 100%.

A picking robot for suspended cultivation bed
Arima et al. [38] developed a strawberry picking robot with a four-DOF robotic arm for the suspended cultivation bed [39] . The picking robot can be installed for the cultivation bed environment. Furthermore, the distance between the robot and the fruit can be set as an approximate constant for better depth measurement from the visual sensor. In harvesting, a vacuum suction head is used to suck the fruit, after that the strawberry position is determined by the CCD camera. Subsequently, the actuator wrist is turned to cut off the fruit stem, when confirming that the fruit is in the proper position using three pairs of photoelectric sensors. In this way, the single picking time is about 10 s.

A picking robot for elevated strawberries
Shiigi et al. [40] developed a three-DOF arm picking robot that was installed on a guide rail-mobile platform for the elevated strawberries in the greenhouse cultivation. As shown in Figure 13, five LED lighting devices are used in the vision system. Three CCD color cameras are also utilized to locate the strawberries, two of which are used to detect the position of the target fruit, another is used to detect the stem and its inclination angle. The end effector is also the suction clamping type.
In harvesting, the stalk is clamped with two fingers after the fruit is fixed by the suction device. The stalk is then cut to the left and right, according to the inclination angle. Nevertheless, the picking success rate is only 38%, mainly due to the stereo positioning and incorrect recognition of fruit stems. Yamamoto et al. [41] developed a strawberry picking robot combined with an orbital mobile platform for the elevated strawberries, as shown in Figure 14. A gantry structure can be used to arrange the rail-mobile platform under the cultivation rack of strawberry. A three-DOF articulated robotic arm can also be used to pick the strawberry fruits on both sides of the cultivation rack. Figure 14 Strawberry picking robot 2 In a vision system, a pair of stereo cameras are used to determine the position of the fruit. As such, the robot arm can move the end effector to the side closer to the current position on both sides. As such, the end effector can clamp and harvest the fruits, finally put them into the storage bins on both sides, after identifying the position of the fruit stem via the hand-eye camera. The success rate of actual picking is as high as 89%, while the success rate of single picking is 84%, and the average picking cycle is 22.2 s.

A picking robot for ridge-growing strawberries
Chen [42] designed a robot picking system for the ridge-growing strawberries, as shown in Figure 15. This type of robot is mainly composed of a three-DOF Cartesian coordinate system manipulator, a grasping and shearing end effector, and a vision system. The picking process can be as follows: Camera 1 is used to collect the strawberry images, thereby determining the position of the center of gravity for each acquired target for sorting out. Then, the manipulator can move the end effector to the position of the center of gravity of the first target. Subsequently, Camera 2 is used to collect the strawberry images, thereby further determining the position of the center of gravity and the picking. Finally, the manipulator can move the end effector to the picking position. As such, the manipulator can move back to the designated position for the storage of the strawberries after the end effector completes the picking work. In the vision system, a Luminance and Red Color Difference (LRCD) method is used to segment strawberry images. The color difference of each pixel is also obtained using the RGB color model. Then, the geometric features of the strawberry image are extracted to position under the gray image. Figure 15 A strawberry picking robot 3

A picking robot with full automation and mechanization
Li et al. [43] designed an elevated strawberry picking robot system to realize the automation and mechanization harvesting, as shown in Figure 16. This type of robot can be suitable for the rapid promotion of the elevated strawberry planting area in China. Particularly, the traditional manual strawberry picking cannot fully meet the large-scale production of the strawberry in recent years.
1. Three-degree-of-freedom workbench 2. Walking system 3. Shearing Five components are mainly composed of this type of strawberry picking robot, such as a crawler walking, a three-DOF table moving, an end-effector picking, a camera, and a lighting system. The end effector can be used to implement the cutting and clamping of strawberry stalks.
Another modified part can also consist of a double-rod cylinder, two cutting hands, two cameras, and four lighting devices. Specifically, the total length of the picking mechanism is 15 cm, the width is 8cm, and the height is 5 cm. The cutting hand is composed of a blade and a rubber layer. Among them, one cutting hand is fixed on the bottom plate, and another one is connected with a double-rod cylinder that is fixed on the bottom plate.
In the vision system, two cameras are located above the cutting hand at a fixed angle. The lighting equipment is also evenly distributed around the camera. The specific position of the target strawberry stalk is calculated using the binocular vision measurement. An air pump is also used as the driving source of the shearing hand.
When picking, the end picking mechanism is driven by the three-DOF workbench to move towards the position of the target strawberry stalk. Particularly, the strawberry stalk is located between the two cutting hands. The air pump is then controlled to drive the cylinder for the shearing hand 2 moving quickly. The resulting impact force is used to cut the strawberry stems. The rubber layer is also located on the blade to realize the clamping of the strawberry for better picking. The recognition rate of ripe strawberry can reach over 95%, while the positioning accuracy of binocular strawberry within 1120 mm is 1.5 cm±5%, and each strawberry is picked in about 8s on average.

A strawberry picking robot with/without suction
Hayashi et al. [44] developed a strawberry picking robot, as shown in Figure 17. The type of picking robot consists of a cylindrical manipulator with three-DOF, an end effector, a machine vision unit, a strawberry storage device, and a mobile device. Three components are composed of in the end effector, such as a gripper to simultaneously grab and cut the stem, a suction device to fix the fruit, and a photoelectric sensor to detect the picked fruit. The picking robot can recognize the strawberries via the camera positioning system. The recognition is to identify the color of the image. When picking strawberries, the end effector can choose the working mode with/without suction. In the suction mode, the end effector can approach to suck the target fruit via the forward suction device. Then, the gripper can slide forward on either side of the stem, while grasping and cutting the stem. In the non-suction mode, the suction device can remain in the retracted position.
The picking success rate of the picking robot is 41.3% (suction mode) and 34.9% (non-suction mode), respectively.
The relatively lower rate of picking success can attribute that the vision system cannot well identify the strawberry stems and strawberries with a ripeness higher than 80%. Consequently, the success rate of strawberry grabbing is more than 80%, where the execution time is 11.5 s for successfully harvesting a single fruit, including the time required to transfer the harvested fruit to the tray.
2.3.9 A picking robot with a 5-DOF robotic arm A strawberry harvesting robot system developed by Xiong et al. [45] consists of four subsystems: vision, arm, grabber, and mobile platform. An RGB-D camera is installed on the top frame to detect the front view of the strawberry. A 5-DOF robotic arm is also installed on the self-developed mobile platform, as shown in Figure 18. The end effector is composed of three active fingers, three driven fingers, and a cutting mechanism. Specifically, the active finger can spread to drive the follower finger under the target strawberry, thereby separating the distracting objects near the target. After swallowing the target, a cutter made up of two curved blades inside the device can spin quickly to cut the stem. Besides, three infrared sensors are installed to identify the relative position of the grabber and target. There is also a small space inside the grabber to collect strawberries, resulting in the picking process do not need to go back and forth greatly saving the picking time. Once the space is full, the grabber can be retracted. After that, the lower baffle is opened to pour the strawberries into the target container. In addition, a simple color threshold using RGB channels is also used to detect strawberries in the vision system of the picking robot.
The average cycle of the robot is 7.5 s for the continuous picking of strawberries, where all procedures are 10.6 s. The success rate for the isolated strawberries is 96.8%. Nevertheless, the picking success rate is only 53.6%, due to the piles of strawberry clusters under natural conditions in a greenhouse.
Xiong et al. [46] also improved the strawberry harvesting robot in 2017, in terms of the agricultural applications and cost requirements. A multi-arm 3-DOF Cartesian robotic arm was achieved from the 5-DOF arm using the end effector, positioning driver, motion platform, and vision sensor system.

A picking robot with accurate positioning
De Preter et al [47] developed a picking robot, as shown in Figure 19. This type of robot consists of seven parts: the electric vehicle, positioning, camera and detection, custom-designed robotic arm, fixture, logistics processing module, and quality monitoring software.
The position of the vehicle is determined by wheel encoders, gyroscopes, and an ultra-wideband (UWB) indoor positioning system. The strawberries are first detected by three RGB cameras, according to the color of the image. The end effector is also a 3D printed finger with a soft frame structure. As such, the picking process can mimic human picking. There is no cutting or burning involved in the picking process, only a gentle pull while applying the rotation motion.
In performance, the picking time is 4 s. According to different strawberry varieties, 70%-95% of strawberries can be picked in elevated strawberries. Figure 19 Strawberry picking robot 7 2.3.11 A picking robot for desktop cultivation A strawberry picking robot for desktop cultivation has also been developed by Han et al. [48] , as shown in Figure 19. This type of robot is composed of four main parts: an autonomous car, a manipulator with four-DOF, an end effector with two-DOF, and a color vision system.
The robot manipulator is also designed with an additional rotation axis, including the Cartesian linkage structure, as shown in Figure 20. Each joint axis is set with the X-axis parallel to the farming line, the Y-axis corresponding to the vertical height, and the Z-axis align to the direction of the strawberry plant. Besides, a rotating shaft is introduced on both planting sides. The end effector can be operated to cut the stems of strawberries during harvest, without damaging other stems, leaves, or strawberries. The specific structure of the end effector is shown in Figure 21. Furthermore, a color camera, cutting device, and clamp are installed at the end to compensate for the change in the cutting position caused by the bending effect of the end effector. Picking experiments show that the strawberry picking robot can complete a non-destructive picking operation within 7 s. 2.3.12 A picking robot for two sides.
The strawberry picking robot has developed by Feng et al. [49] . Five parts are composed of a four-wheel mobile platform, a 6-DOF manipulator, an end effector, a binocular camera, and a controller system. The total weight of the robot is 105 kg, and the size of the robot is 1500 mm×700 mm×1600 mm.
As shown in Figure 22, a Denso VS-65556G manipulator is selected in the picking robot, according to a systematic evaluation of the size of the target area and working environment in the greenhouse. This type of manipulator can reach up to 650 mm, while bearing a load of 5 kg, and move with a steady-state accuracy of ±0.02 mm.
This type of robot can also be possible to pick the fruits on both sides. The reason can be that the bottom joint together with 6 rotating joints in the manipulator can rotate in an angle range of 0°-345°. In the end effector, two fingers are mounted on a pneumatic clamp, thereby grasping the fruit stem. A suction cup is used to grasp the fruit. Moreover, an electric cutting device is used to separate the fruit from the fruit stem.
The experimental results show that the successful harvest rate of the strawberry picking robot is 86%, and it takes an average of 31.3 s, indicating that the average error position of the fruit is less than 4.6 mm.

A picking robot with pneumatic devices
A strawberry picking robot has also developed by Guo et al. [50] As shown in Figure 23, this type of robot is composed of five parts: an X-Y-Z three-axis motion platform, pneumatic end effector, global camera, local camera, and personal computer (PC). The end effector also consists of pneumatic scissors and two pneumatic fingers. In the harvesting operation, the scissors are used to cut off the stems, and then the pneumatic fingers are used to pick the stems, and then place the fruit on the rotating wire.
Experimental results show that the robot can achieve 93% detection accuracy of strawberry stems, as well as more than 90% accuracy of maturity and shape quality judgment, indicating fully meeting the requirements of agricultural machinery.

Kiwi fruit picking robot
Three types of kiwi fruit picking robots can be introduced in this section.

A picking robot with a route navigation
Flemmer et al. [51] developed a kiwi fruit picking robot, as shown in Figure 24. This type of robot consists of a vehicle, four 3-DOF robotic arms, an end effector, a vision system, as well as the kiwi storage and replacement device. The end effector is composed of the clamping jaws to fit the kiwi fruit shape, a small motor that drives the clamping jaws to rotate, and a connecting rod structure. The robot can navigate to pick the fruits along a specific route under the identifying benchmarks in the orchard using the GPS. A vision system is also composed of eight cameras to recognize the fruits hanging on the canopy, particularly to distinguish the size and serious defects for the appropriate picking operation [51] .
The specific process of picking can be divided into the following three steps: 1) To open the clamp, and then to clamp the kiwi; 2) To keep the clamp and kiwi relatively still, thus rotating to cut off the stalk on one side of the clamp; 3) To reset the clamp, where the picked kiwi fruit can fall automatically to roll into the collection box along a predetermined route.
In the vision system, two additional cameras are used to identify the plane of the collection box aligned to the fruit placement, thereby evenly filling the storage box. Once the system assumes that the storage box is full, the robot can move to the end of the row and put the fruit down. The robot can be utilized to search for an empty storage box using the vision system, thereby real-time adjusting the pose (such as approaching and forking the arm), and then returning to the original position for the subsequent picking operation.
In performance, the picking speed of each robotic arm is 4s/piece, according to experimental statistics, that is, the overall speed of the robot is 1 s/piece [53] .

A picking robot with multiple sub-modules
Williams et al. [54] designed another kiwi fruit picking robot, as shown in Figure 25. Multiple sub-modules can also be divided into the kiwi fruit picking robot. Specifically, four components are contained in each multiple sub-modules, such as a camera, a robotic arm, an end effector, and a controller. Figure 25 Kiwi Fruit Picking Robot 2 The control system of the kiwi harvester can be divided into the following subsystems: the machine vision system for object recognition, stereo depth positioning, dynamic harvesting scheduler, arm path planning with the servo control, and the control system for fruit grasping and separation.
In the vision system, the robot can recognize the position of the kiwi. The deep neural networks and stereo matching are utilized to reliably detect and locate the kiwis under real lighting conditions. During the picking process, the end effector of the device can drive the 3D printed fingers and a silicon buckle mechanism to complete the grasping and rotating actions of the kiwi fruit for the picking with the minimum damage.
In terms of design recognition, the vision system of the robot can detect 89.6% of the kiwis that can be picked by the current harvester, accounting for 70.0% of all kiwis in the canopy in the field of camera view, but a 25% damage rate. Thus, the current 15.3% missed picking rate and 6.7% false picking rate can be reduced for the higher picking rate of the robot. The picking speed is also about 5.5 s/piece.
A later improved robot [55] can detect 90% of the kiwi in the orchard, using Faster-Regions with Convolutional Neural Networks (RCCN). Specifically, 86.0% of kiwis can be picked by the improved device (55.8% of all kiwis). The actual picking speed was 2.78 s/piece, and the damage rate was reduced to 8.47%.

A picking robot with end effectors in upward rotation
Fu et al. [56] designed and tested the end effector of a kiwi fruit picking robot. An upward rotation is used in the end effector to separate the stalk, as shown in Figure 26.
The specific picking process can be as follows: 1) The coordinate position of the fruit can be first obtained using the machine vision, and then the robotic arm can be used to control the end effector, in order to align the origin of the end effector and the fruit coordinate with the same vertical line; 2) The end effector can raise vertically to automatically approach the fruit within the range, where the infrared switch sensor is also used to detect whether the fruit enters the finger grasping range or waiting line; 3) After the fruit enters the finger grasping range, the front finger can rotate and extend into the gap envelope to separate the adjacent fruits. At the same time, the front and back fingers can be close to grabbing the fruit. The pressure sensor at the center of the front finger can be utilized to detect the clamping force; 4) Once the preset clamping force is reached, the rotary picking mechanism can rotate upward to separate the stalks for the complete picking action. After that, the motor can be reset to release the fruit for the next picking.
The picking success rate of the end effector is up to 96.0%, and the average picking speed is 22 s/piece.  Figure 26 End effector of kiwi fruit picking robot 3

Citrus picking robot
Four types of citrus picking robots can be addressed in this section, in terms of structure components, end effectors, control system, vision and performance. 2.5.1 A picking robot with STM32 microcontroller A citrus fruit picking robot has been designed to pick and box citrus fruits by Liu et al. [57] , as shown in Figure 27. Two components consist of mechanical and control parts. Furthermore, the mechanical part can be included in the end-effector, mechanical arm, moving platform, rising, storage, and transportation mechanism. The control part can be included the OpenMV image acquisition and the control system of each module that is equipped with the STM32 microcontroller as the core chip. The specific working process can be as follows. The camera can be used to first scan the citrus. If the ripe citrus is identified, the citrus coordinate can be transmitted to the STM32 microcontroller through serial communication.
Then, the microcontroller can be used to drive the robot arm for the designated position. At the same time, the servo system can be started to drive the ring scissors for the stalk cutting from the citrus fruit. As such, the citrus can be entered the chute inside the box along the pipe for preliminary sorting and boxing. Thus, the integrated identification, picking, sorting, and boxing can greatly reduce the labor intensity in large-scale orchards.
The end execution device consists of two parts, such as the camera and the ring scissors. The camera can be used to capture the color of the ripe citrus as the recognition target for the OpenMV image processing. The color of the citrus skin can also be collected using the camera at the front end of the robot. As such, the location and coordinate of the citrus can be quickly determined by the camera. Another module is also installed at the end of the robotic arm to expand the recognition range.
The structural design of the ring scissors can be considered the hard stalk of citrus fruit, where the shear mechanisms in most picking methods rely on the manual application of force. Therefore, a servo system can be used to provide a large torque as the power source.
Once the camera identifies the ripe citrus, the mechanical arm can drive the ring scissors to reach the designated position, then to activate the servo, where the drive shaft can drive the second arc linkage in reciprocating motion, with a blade fixed at the arc linkage groove, which in turn cuts the fruit stalk.
The citrus picking robot takes an average time of 2.7 s for recognition and 5.4 s for picking, which picks 1 or 2 fruit per time, and the maximum height of the picking is 1.85 m.

A picking robot using modular design
A modular design concept is adopted for the citrus picking robot that was developed by the School of Mechanical Engineering of Chongqing University. This type of robot consists of four systems, such as visual positioning, visual navigation, robotic arm motion, and end-effector system. The four systems work together to realize the automated picking of citrus, as shown in Figure 28 [58] . a. Citrus harvesting robot exterior b. Citrus harvesting robot interior 1. Tracked chassis 2. Binocular camera 3. Robotic arm 4. End effector 5. Shell 6. Fruit basket 7. Industrial computer Figure 28 Citrus picking robot 2 The mechanical structure of this picking robot is composed of three parts, such as the mechanical arm, end-effector, and mobile chassis. The mechanical arm is also equipped with a six-DOF robot manufactured by Oceanic, where an arm span of 1 m, a mounting position of 1 m from the ground, and a picking range of 0-2 m.
The vision system is based on a BB2 binocular camera to capture the images on the citrus fruits and obstacles, where the accuracy of the binocular camera reached 1.58% in the preliminary laboratory study, fully meeting the positioning requirements in the picking operation [59] .
The end-effector is designed in the picking robot using the bionic concept. Three parts are included, the cutting device, the suction cup and retracting mechanism. Among them, the biting end-effector is designed to imitate the structure of the snake mouth [60,61] . The swallowing action of a snake and the structure of the upper jaw are taken as the prototype in the configuration.
The picking robot can take about 14 s to complete one picking action. A picking success rate of 80.51% can be achieved, together with an obstacle avoidance success rate of 75.79% [62] under the natural environment of the orchard.

A picking robot with five-DOF tandem
As shown in Figure 29, a five-DOF tandem citrus picking robot has also been designed by Song et al. [63] . This citrus picking robot mainly includes the base, waist, large arm, middle arm, small arm, each joint axis and motor, reducer, photoelectric code plate, and coupling unit. A motorized circular tooth saw blade is also used to cut the citrus stalks, which requires less power from the cutting motor and less stalk pressing force, indicating a higher shearing efficiency.

Figure 29 Citrus Picking Robot 3
In picking, the suction cup installed at the front of the threaded suction tube is used to attract the fruit in the distance or hidden in the leaves, where the telescopic motor drives the threaded suction tube to pull the fruit toward the cylinder through the gear pair. Once the fruit is close to the cylinder, the drive motor of the fruit driving ring can pull up the fruit using the pull rope. Then, the driving ring can drive the fruit into the cylinder, while pushing the fruit stalk to the saw blade.
The citrus picking robot can be used to smoothly locate, thereby to precisely and efficiently pick the target citrus. The end positioning accuracy is 3 mm. The average single shearing time of the end-effector is 3 s for the citrus stalk of Φ3 mm, whereas, 9 s for the citrus stalk of Φ6 mm, indicating the more ideal picking effect.

A 7-DOF picking robot
A 7-DOF citrus picking robot developed by Mehta et al. [64] , as shown in Figure 30. A closed-loop control system is also used for timely feedback, as well as dual cameras, one mounted in a fixed position, and another mounted in the center of the end-effector moving with the robot.
The random picking trials of citrus were conducted in the laboratory, where the picking success rate reached 95%. However, this type of robot is only suitable for the effective picking of large and medium-sized citrus rather than small ones.

Eggplant picking robot
This section can review two types of eggplant picking robots for the V-frame cultivation and open system.

A picking robot for V-frame cultivation
Hayashi et al. [65,66] developed a picking robot for the eggplant cultivated in V-frame, as shown in Figures 31 and 32. This type of robot consists of a five-DOF articulated robotic arm, a vision processing, and a mobile platform. A humanoid pneumatic structure in the end-effector can gently grasp a size-compliant target by four rubber actuators and two suction cups, where a cutting mechanism can also be used to cut the fruit stalks. The harvesting process is as follows. Firstly, the target position for the robot movement can be determined by the vision system. Then, the end position of the eggplant is determined by the photoelectric sensor installed at the bottom of the end-effector to grasp the target. The robot arm is moved to tilt the eggplant, thereby separating the fruit from the nearby leaves. After that, the shearing device approached the fruit stalk and cut off. Finally, the target is moved to the designated container for the whole harvesting process.
The machine vision is used to detect the eggplant fruits, according to their color and morphological features. Specifically, the low-gray pixels are segmented using the color features, whereas, the interference of leaves and stems is removed using the morphological features. The picking success rate is 62.5%, and the average picking time is 64.1 s, without considering branches and leaves in the laboratory.

An open eggplant picking robot
Song et al. [67,68] independently developed an open eggplant picking robot, as shown in Figure 33. This type of robot includes a PC, DMC2280 multi-axis motion controller, Yaskawa AC servo drive system, and Daheng DH-CG320. A machine vision system is composed of a typical image capture card, a Panasonic WV-CP470 color camera, and a 4-DOF articulated robot. The vision system of the picking robot uses a histogram-based fixed dual-threshold method to segment the G-B grayscale image, thereby extracting the features of eggplant fruits, such as centroid, cutting point, and grasping position.
Experiments were conducted in a simulated growth environment in the laboratory. The picking robot presented a successful picking rate of 89%, and the average picking time is 37.4 s.

Bell pepper picking robot
Three types of picking robots can be introduced for the green house, V-type cultivation, and plan planting.

Picking robot for sweet peppers in a greenhouse
Kitamura and Oka [69] developed a picking robot for sweet peppers grown in a greenhouse. As shown in Figure 34, the picking robot consists of a mobile device, a three-DOF picking execution, and image processing. Once the sweet pepper is located, the picking execution device can drive the end effector to a designated position. The end effector is also equipped with two cameras to further position the stem of the sweet pepper. The rear end effector can be used to cut the stem for complete picking. However, the picking robot has only a cutting system without a storage part.

Figure 34 Bell pepper picking robot
The picking robot can recognize the fruits of sweet peppers through the binarization of the HIS color under the additional LED lighting compensation. Moreover, the binocular vision system on the end effector can further locate the picking position [70,71] .
Subsequently, Kitamura and Oka [72] developed an articulated arm picking robot for the sweet peppers in the V-shaped cultivation, as shown in Figure 35. The picking robot is composed of three parts, such as recognition, picking, and mobile system. The picking system consists of grabbing and thermal cutting. Specifically, two aluminum chain links are used as the grabbers to grab fruits. As shown in Figure 36, the clamping jaws are driven by a specially designed notch plate in the thermal cutting part. Two electrodes are installed at the front end of the notch plate for the electrical input to the electrodes. Once the jaws are closed to grasp the sweet pepper, the thermal cutting electrode can move forward to complete the stem cutting. The CCD cameras and artificial lighting are used to capture images in the recognition system. The binarization of the HSV color designation is also used to identify the fruit. Halcon software is further selected for real-time image processing. After that, the X, Y, and Z coordinates of bell peppers are taken as the output after the image processing system. The fruits can be detected by the light reflection function, even when the fruits are overlapped or covered by leaves. 2.7.2 A picking robot for V-type cultivation As shown in Figure 37, Wouter et al. [73] introduced an automatic harvesting robot suitable for the 'V'-type sweet pepper cultivation in the greenhouse. The manipulator module of the picking robot includes a 9-DOF robotic arm, an end effector, an air compressor for a pneumatic end effector, as well as a cabinet containing computers and electronic equipment.
Two end-effectors were tested in this study. One is the Fin Ray end effector, where four fingers are used to hold the fruit and then cut the stalk. Another is the Lip-type end effector, where a suction cup is used to absorb the fruit, and then close the lips to cut the stalk.
In the vision system, the color images can be captured by the camera, in order to detect the sweet pepper fruit for maturity. The positioning of the fruit can also be realized to cooperate with the depth image from another camera. The multiple sensors are used in the picking robot to achieve a higher harvest success rate, together with the relatively lower picking efficiency [74,75] . As such, this type of picking robot can be used to cope with the complex plant structure, particularly for the improved ability to perceive and avoid obstacles under a suitable grasping posture.
In performance, the gripping success rate of the sweet pepper picking robot is 80% using the Fin Ray end effector. Nevertheless, the gripping success rate is 52% under a complex environment using the Lip-type end effector. Furthermore, the cutting success rates of both Fin Ray and Lip-type are less than 15%, indicating a relatively low picking success rate. Once the picking environment is simplified, the grasping success rate of Fin Ray end effector (93%) is higher than that of Lip-type end effector (61%), while the cutting success rate of Lip-type end effector (76%) is higher than Fin Ray end effector (29%). The picking cycle of the sweet pepper picking robot is 94s, much longer 6 s than before, indicating an economically feasible picking for the sweet pepper. Figure 37 Bell pepper picking robot  Figure 38, Christopher Lehnert et al., have developed a sweet pepper picking robot [76] . This type of robot consists of a differential drive platform, a 6-DOF rotating manipulator (universal robot UR5), a mobile joint (Thomson LM80), computer hardware, a robot controller, and a forward laser scanner. In the end effector, a blade is used to cut the stalk after the fruit is sucked by the suction cup. Furthermore, a decoupling mechanism is also designed to continuously grasp and independently cut fruits at different selected positions. Figure 38 Bell pepper picking robot In the vision system, an RGB-D camera is used to scan the target crop in the sweet pepper picking robot. Kinect fusion is selected to create a 3D model of the scene using a multi-view point cloud. A Naïve Bayesian classifier is also used to separate the sweet peppers from the background, according to the mean and variance in the rotated-HSV color space. A clustering method is then used to obtain the clusters of different sweet peppers. Finally, the appropriate grasping posture is calculated, according to the point cloud of each sweet pepper [77] .
This type of sweet pepper picking robot is suitable for the sweet pepper with a planarized planting environment. The field experiments demonstrated that the picking success rate of the robot is only 46% under a complex picking environment; while the rate can reach 58% after the picking environment is simplified.

Cucumber picking robot
This section can review four types of picking robots.

A picking robot using autonomous navigation
A cucumber picking robot has developed by Li et al. [78] , as shown in Figure 39. A crawler mobile platform is used to real-time obtain the navigation line image for autonomous navigation. A 4-DOF compact articulated robotic arm is also adopted in the machine. Furthermore, the end effector is composed of a flexible finger, a cutter, and a secondary close-range positioning camera. Among them, the secondary close-range positioning camera can be utilized to calculate the precise coordinates of the cutting point, thereby compensating for the initial positioning error. In performances, the picking success rate of the robot is 85%, and the picking time for a single cucumber is 28.6 s.

A two-finger pneumatic picking robot
Yang et al. [79] designed a novel end effector with two-finger pneumatic flexible joints, according to the cucumber gripping characteristics. As such, the effective gripping rate of the end effector for cucumber is 90%, while the success rate of fruit stem cutting is 100%, and the average picking time is 3 s. This type of 1. Navigation line 2. Energy system 3. Monitor 4. Cucumber 5. Finger 6. Cutter 7. Close-range positioning camera 8. Joint 9. Link 10. Fruit box 11. Binocular support frame 12. Auxiliary light source 13. Support frame 14. Navigation camera 15. Vehicle-mounted industrial computer 16. Track 17. Walking chassis Figure 39 Cucumber picking robot of Li's team [75] picking robot is equipped with the novel end effector, indicating a higher picking success rate of 86% in the cucumber picking experiment that cultivated on diagonal pull lines [75] , where the average picking time for a single cucumber is 18 s [80] . Additionally, an incomplete cucumber recognition from the vision system can be attributed to the picking failure, particularly where the stems were beyond the range of the end effector.

An automatic picking robot in a greenhouse
A greenhouse cucumber automatic picking robot has developed by van Henten et al. [81] , as shown in Figure 40. This type of robot consists of an autonomous car, a 7-DOF manipulator, an end effector, two camera vision systems, as well as various electronic and pneumatic hardware. Specifically, a feasible 7-DOF manipulator is used to position the end-effector during the harvest operation. The manipulator also consists of a linear slide on the top, where also mounted a Mitsubishi RV-E2 manipulator with an anthropomorphic arm and a spherical wrist.
As shown in Figure 41, the end-effector contains the following parts: a gripper and suction cup to grasp the fruit, and a thermal cutting device to separate the fruit from the plant. During the harvest, the two fingers of the electric gripper can grip the stem of the fruit to cut. Moreover, the suction cup installed under the fingers of the gripper can secure the falling fruit for transportation. After that, the fruit can be placed in a horizontal direction, and then gently lowered into the storage box. Finally, the grippers can release the fruit for the next round of picking.
In the vision system, two cameras are used in the harvesting robot. One camera is mounted on a rail of the vehicle, in order to extend on both sides of the manipulator and curve at the front end of the vehicle. As such, the camera can be used to inspect the crop on both the left and right sides of the vehicle. More importantly, this camera can also move independently from the manipulator. Another camera is set as a lightweight system mounted on the top of the end effector. Each camera system has also performed different tasks. The camera mounted on the vehicle is used to detect the fruit, thereby determining the ripeness and quality of the fruit. Another camera is used to perform the 3D positioning of the fruit for the motion planning of the robot. Finally, the gripper is used to fix the cucumber on the end effector for the stereo imaging near the cucumber.
In performance, this type of fully automatic robot can pick more than 80% of cucumbers in a greenhouse, and the average picking time is 45 s/cucumber. Figure 41 End effector of cucumber-picking robot 2.8.4 A picking robot for high-wire cultivation van Henten et al. [82] reported another picking experiment of a cucumber picking robot, as shown in Figure 42. The cucumber picking robot can pick the fruit by grasping and cutting off the stem, and then transporting the cucumber to the collection with a suction cup.

Figure 42 Cucumber picking robot
This type of picking robot is suitable for the high-wire cultivation of cucumber. The high-wire cultivation is much more suitable for automatic picking, compared with traditional one. This large-scale cultivation is also conducive to improving the quality and quantity of cucumbers, but a much more labor intensity is required for crop maintenance.
This type of cucumber picking robot presents an average picking success rate of 74.4% in the harvest test (a total of 195 cucumber fruits are ready to be harvested), and the entire harvesting cycle at a fixed position takes an average of 65.2 s.

Picking robots for other fruits and vegetables
This section included various robots for picking cherry and heavy crops. 2.9.1 A cherry-picking robot As shown in Figure 43, a cherry-picking robot was developed by Tanigaki et al. [27] . This type of robot is mainly composed of a 4 DOF manipulator, a vision sensor, an end effector, and a personal computer. This type of manipulator can carry the end effector to pick fruits around the tree trunk, where one axis of the manipulator can move up and down, and another three axes can move left and right. A three-dimensional vision sensor installed on the robot arm can also move with the manipulator, thereby reducing visual blind spots.
Once the visual sensor receives the position coordinates of the fruit, the end effector can move to the location ready for the picking. A vacuum cleaner can provide negative pressure to the end effector, where the cherry can be absorbed into the vacuum nozzle through the pipe. The mechanical finger of the end effector can be used to clamp the fruit stem, and then pick the fruit together with the stem from the tree, finally to send the fruit into the collecting box for one complete cheery harvesting. The end effector is designed about 80 mm high, 30 mm wide, and 145 mm long. Four devices are included in this type of end effector, such as a fruit suction, an opening, and closing, a back-and-forth movement, as well as a pair of fingers.
When harvesting the target fruit can be sucked under vacuum pressure. The fruit position can be then fixed at the tip of the pipe. The finger can also be opened or closed by the rotation of the servo motor that is connected to the end effector. Once the fingers grasp the branches, the end effector can be lifted to pick the cherries from the tree.

A picking robot for heavy crops
Ali et al. [83] developed a feasible manipulator for the harvesting of heavy crops. This type of manipulator has 5 DOF of the rotary joint (4 DOF of the manipulator and 1 DOF of the end actuator), with an effective load of 25 kg. As such, this type of robot installed on the tractor can be used to identify and pick heavy crops, such as pumpkin, watermelon, melon, and cabbage, while the tractor is moving forward.

Differences in the structure of picking robots
The structure of the end effector is different. The shape, size, and surface characteristics of picking objects are different. Therefore, end effectors with different structures should be designed for picking. Even with the same picking object, different grasping methods will lead to different end effector structures.
The structure of the manipulator is different. The fruit and vegetable picking robots retrieved in this study mainly use serial manipulators, Cartesian coordinate manipulators, and parallel manipulators. Among them, the serial manipulator and the Cartesian manipulator have a large range of motion, flexible joint motion, and relatively low picking efficiency; the parallel manipulator has a small range of motion and relatively high picking efficiency.

Situation analysis
This section can review the structural components, such as robotic arms, end effectors, and vision systems, together with the situation analysis for the planting environment and harvesting performance.

Robotic arm for harvesting
The robotic arm is used to move the end effector of the harvesting robot towards the required position for harvesting. Two types of profile can generally be divided, including series robotic arms and parallel robotic arms. The series robotic arm is much cheaper and commonly used for easier control, while, the parallel robotic arm has high failure resistance and fast-moving speed [84,85] . Furthermore, the current harvesting robots are mostly used in the series structure, particularly with the DOF ranging from 3 to 7 DOF [68,85,87] . In addition to a single series robotic arm, there are also multiple series of robotic arms to improve harvesting efficiency [87,89] .

End effector
The end effectors of agricultural harvesting robots usually need to be adaptively designed for different harvesting objects, particularly for the physical characteristics of the target fruits. There are three types under the various picking methods, including the negative pressure adsorption, gripping twist, and shearing. Furthermore, the selection of picking methods is closely related to the mechanical properties of the target fruit.
In negative pressure adsorption, a vacuum generator is used to generate the negative pressure, thereby adsorbing the fruit body of the target fruit. As a result, the fruit stem can be broken under the transverse shear force.
The gripping twist type also can pose the target on the fruit body. The end effector can clamp the fruit body tightly while twisting, where the fruit stem is sheared to be broken using torsion force.
In shearing type, the cutting is the most traditionally used typical blade to directly cut the fruit stems [90][91][92] . Various forms are also widely used, such that high-temperature electrodes can prevent fruit infection [93] , and laser cutting can reduce the complexity of the end effector due to non-contact [25] . Since the shearing type has the widest range of applications, the shearing structure needs to be greatly simplified, particularly for grasping or receiving the target fruit.
Free of damage is also necessary to be considered in the design of the end effector, due to the fruits generally vulnerable to damage. Specifically, the negative pressure adsorption type can produce a much larger suction force to effectively absorb the target fruit, where the resulting fruit can often reach a much higher speed after separation from the stem. Therefore, a buffer structure also needs to be designed to avoid collision damage.
There is also the generally larger gripping force in the gripping twist type. Especially, the end effector can be required to firmly hold the target fruit, thereby avoiding the relative sliding during twisting in the harvesting process. Therefore, a force sensor is normally required to accurately control the gripping force [94] . The clamping structure of the end effector is often designed to fit the target object, but even the same fruit has still different shapes and sizes. As such, the human-like multi-finger structure is the most widely used because of its flexibility and excellent adaptability. In terms of the index [95][96][97] , two, three, and four fingers can be divided for this type of structure [22,98] . Among them, the two fingers are suitable for the small fruits with regular shapes, whereas the three and four fingers are mostly used for the fruits with irregular shapes or large sizes. Additionally, the fruit is free of damage in the shearing type, where there is no directly exerting force on the fruit.

Visual algorithm
The vision system can greatly contribute to the environmental perception of the harvesting robot, including the fruit recognition and positioning during the harvesting process. However, a highly effective method is still lacking in the current vision system to rapidly identify and accurately locate the target fruit in the complex field environment. Furthermore, the high performance of the vision system can directly dominate the harvesting success rate and harvesting efficiency of the robot [2] .
Most visual algorithms have been proposed for the various harvesting robots collected in Section 2. It can be found that the color features of mature fruits are often utilized to distinguish the target fruit from the background environment. The color spaces are commonly used, including the RGB, HSI, Lab, YIQ, YCbCr, HSV, and LCD. Even one or more color spaces are selected for the vision system in most harvesting robots. For instance, Zhao et al. [99] identified the mature tomatoes in the field using the HIS color features. Si et al. [100] proposed an algorithm to segment the apple images by normalizing the difference between red and green. Yang et al. [101] have also calculated the 2R-G-B in an RGB color space for the Otsu threshold segmentation. More importantly, an adaptive Hough circle transform has been used with some local parameters to identify apples in the figure.
Subsequently, the segmentation can be designed to recognize the target fruit, according to the extracting features and the background. Six algorithms have also been commonly used to segment and identify the fruit in recent years, including the threshold segmentation, clustering algorithm, region growing method, support vector machine (SVM), artificial neural network (ANN), and combined multiple algorithms. For instance, Jiang et al. [102] used an adaptive threshold Otsu algorithm to segment the apple image. Ji et al. [103] also proposed an SVM-based apple recognition classification, according to the extracting color and shape features from the apple images. Specifically, some pixels are taken as the initial seed area, corresponding to the color with the highest probability in the RGB histogram of the apple images. As such, the area growth can be achieved to measure the color similarity between each pixel in the seed area and the 8 neighboring pixels. Lei et al. [104] used the kernel fuzzy C-means clustering to segment the pomegranate images. Song et al. [105] achieved the segmentation of eggplant images using artificial neural networks.
Many detection algorithms have also appeared using deep learning in recent years. Fu et al. [106,107] successively used the Faster R-CNN based on ZFnet and VGG16 to realize the kiwi detection. Liang et al. [108] have used the SSD to real-time detect mangoes in trees. Kushtrim et al. [109] used the YOLOv2 to real-time detect and then count apples and pears in the canopy. Lu et al. [110] used an improved YOLOv3-LITE lightweight neural network to identify the citrus in the orchard.
It is also very necessary to repair the fragmented and incomplete images, particularly for the fruit that is easily blocked by some branches and leaves during the harvesting process. For instance, Xun et al. [111] performed the circle fitting on the occluded apple area that was segmented by the three-layer BP neural network to achieve target positioning. Silwal et al. [112] used a Blob analysis to identify occluded apples.
In the occluded part, if an apple target is divided into multiple regions by the occluded object, these regions can also be merged using a clustering algorithm, according to the principle of Euclidean distance. For instance, Zhuang et al. [113] used the convex hull operation to estimate the occluded part, where the foreground area is used to approximate the real conditions. The new clustering algorithm can also be applied to the images of point cloud. That can also be an efficient way to overcome the interference of occlusion [114] .
In overlapping fruits, Wang et al. [115] used the K-means and Ncut spectral clustering algorithm to segment the apple targets. Fu et al. [116] detected the two pits on the edge that is the closest to two adjacent kiwifruit calyxes, in order to determine whether it is the dividing line of adjacent kiwifruit by the intersection and the angle of the line of two pits, or the line of the two calyxes.
Special lighting environments are considered in various vision algorithms, such as night and fog. For instance, Ji et al. [117] proposed a Retinex image enhancement algorithm for the night apple images, particularly with the edge-preserving features using guided filtering. Jia et al. [118] proposed a WT-ICA fusion noise reduction method to remove the noise in the images of apple night vision. Particularly, the wavelet transform and independent component analysis are introduced into the night vision image processing, In the case of severe haze conditions, Zhu et al. [119] proposed a novel method for the parameter adjustment and improvement in the visual positioning of apple-picking robots, particularly on the dark channel rather than the defogging of apple images. Yang et al. [120] used the super-pixel segmentation to improve the robustness of recognition in different lighting

Planting environment
The crop cultivation and planting environment also require completely different structures and technical designs of harvesting robots.
In tomato harvesting, Yasukawa et al. [121] designed a robot for single-frame cultivation. Feng et al. [122] also designed a robot for the hanging-line cultivation. Liu et al. [123] designed an efficient and accurate harvesting robot for tomato cultivation in the greenhouse.
In strawberries harvesting, the high-ridge and high-rack cultivation are widely applied at present, due to the labor-saving, neat, and high yield, compared with the ordinary ground cultivation. Correspondingly, more and more strawberry harvesting robots are designed for the mainstream high-ridge and high-rack cultivation in Japan [124] . There is also an excellent performance of the strawberry harvesting robot for the high-rack cultivation in China [125][126][127] .
In addition, the artificially cultivated and structured plantation can reserve enough space for the harvesting robot to move. Furthermore, the crops with the artificially restricted planting mode can present denser fruit distribution and fewer obstructions, compared with the crops cultivated in the traditional model. Therefore, smaller working space and higher recognition accuracy can be achieved for the harvesting robot that is designed for the structured orchard.

Harvesting performance
The performance of fruit harvesting can be evaluated by the harvesting success rate, harvesting efficiency, fruit damage, and harvesting environment.
The harvesting success rate depends on the visual recognition and positioning algorithm in the vision system, as well as the end effector. Therefore, the higher picking success rate can be attributed to the higher recognition and positioning accuracy of the vision system, while the lower error rate of the picking action from the end effector.
In picking efficiency, the specific influencing factors can be relatively complex, including the recognition and positioning speed of the vision system, the movement efficiency of the robotic arm, as well as the complexity of picking action from the end effector.
The damage of the fruit is totally determined by the end effector. Much more effective picking methods or reasonable fruit protection measures can contribute to effectively reducing the damage rate of the fruit.
The harvesting environment can also pose a great impact on the harvesting performance.
For instance, the harvesting performance of the robot is significantly better in the structured orchard harvesting environment than that in the unstructured one. The higher harvesting performance can also be attributed to the improved recognition and positioning of the vision system, while the fewer requirements for the obstacle avoidance of the robotic arm during the harvesting. The reason can be that there is a more regular spatial distribution of fruits in the structured orchard, where the branches and leaves are less blocked.

Complex and changeable harvesting environment
The growth environment of fruits can generally be divided into indoor and outdoor modes.
In the outdoor environment, there are complex and changeable influencing factors during the harvesting process. Among them, weather can cause some great variation in the light, temperature, and humidity environment, especially in the light for the identification and positioning of the vision system. Furthermore, a harsh requirement can be addressed on the obstacle avoidance of the picking robot, where the fruits may grow in places with dense branches and leaves in the outdoor environment.
In the indoor environment, there are fewer influencing factors than those in the outdoor environment. The indoor crops are more consistent in the location, while more controllable in the light parameters. However, there is a much greater planting density of crops in the indoor environment, leading that there is less space between crops. Therefore, a higher requirement can be addressed on the shape and obstacle avoidance of the picking robot. In addition, the indoor architectural background may also pose a certain impact on the identification of fruits.

Low accuracy of recognition and positioning in vision system
The fruit identification and positioning algorithms in the vision system can be used to accurately and rapidly locate the target fruit, according to the color, texture, and shape feature of fruits.
The accuracy of recognition and positioning in the vision system is very susceptible to the light parameters because the light can cause much greater interference to the color feature. Specifically, the light spots can be formed on the surface of fruits and vegetables under a well-lit environment, leading to the loss of color and texture information in the target area. By contrast, the overall color of fruits and vegetables can be dark, resulting in the color information being distorted under a low-lit environment. In addition, the uneven lighting can also cause to be darker on some surfaces, while to be brighter on the rest surface of the fruits and vegetables, inevitably leading to errors of identification and positioning.
The shading of branches and leaves can be another influencing factor for the accuracy of recognition and positioning in the vision system. The fruits are easily obscured by other branches and leaves in a natural environment. As such, there is also an incomplete area of segmented fruits using color and texture features.
The centroid position can also be shifted after calculation based on the shading features. If the obscured area is too large, the regions of interest can be unrecognizable for the vision system. Once the covered area of fruits exceeds 80% in the best case, the covered fruits cannot be identified, according to the current report [111] .
Besides, overlapping fruits are very easy to occur, particularly on some fruits in clusters. Therefore, the multiple overlapping fruits may be misjudged as one during identification, resulting in the algorithm collecting the wrong number of fruits and the wrong position of the centroid, even for the subsequent picking path planning.

Picking efficiency chasing commercial requirements
Most current harvesting robots can pick only one fruit or vegetable at a time, indicating a limited improvement of harvesting efficiency. Firstly, some harvesting robots with multiple robotic arms can also bring bulkiness in the profile and cost, as the harvesting efficiency increases.
Secondly, the current harvesting robots from the laboratory environment cannot fully meet the complex and changeable actual working conditions. Various situations can often occur in the natural environment, such as the repeated picking actions for the target fruits difficult to pick, and the redundant picking actions that are caused by misidentification. All these situations can greatly reduce the harvesting efficiency of the harvesting robot.

High cost of harvesting robots
The high manufacturing cost can be attributed to the relatively complicated structure of the harvesting robot. More importantly, a mechanical arm and a special end effector are required to complete the picking action, as well as a corresponding vision system, a control system, and a mobile carrier during harvesting.
In addition, the harvesting robots are only used in the harvesting season of the year, but they idle for the rest of the year, due to the obvious seasonality and timeliness of fruit harvesting, thus leasing to the relatively lower economics of the harvesting robots.

Structured environment for fruit planting
A structured fruit planting environment can be a better choice for harvesting robots in the near future. In contrast, the complex environment of fruit planting often requires a strong ability of the harvesting robot to avoid obstacles for picking operations. Much more DOFs and complicated obstacle avoidance algorithms have been proposed to improve the obstacle avoidance ability, thereby undoubtedly increasing the production cost, while reducing the harvesting efficiency of the harvesting robot. Alternatively, the structured fruit planting environment can be used for the tradeoff. For instance, the sweet pepper plantation in Florida adopts the Spanish planting mode or the Dutch 'V' planting mode, as shown in Figure 44. As such, the growing position of the sweet pepper can be more regularly suitable for automated harvesting [128][129][130] . A further study of planting mode is also under the way for the automated large-scale harvesting in apple orchards [131] .

Higher precision algorithm of recognition and positioning
A higher precision algorithm of recognition and positioning in the vision system can be a high demand for the excellent robustness of illumination in the future harvesting robot. Particularly, it is very necessary to correctly identify and locate the occluded or overlapping fruits and vegetables in the complex environment. For instance, Lv et al. [132] used the adaptive gamma correction to improve the illumination of the image. Zhuang et al. [113] used the convex hull operations to repair the image of occluded citrus. Luo et al. [133] proposed an algorithm to segment overlapping grape clusters for the stalk-cut point. Yang et al. [134] proposed a color image segmentation of Hangzhou white chrysanthemum using the LS-SVM. The success rate of the algorithm can be more than 81%.
In addition to the conventional visual algorithm, the spectrum technologies can also be applied for the recognition and localization to improve the robustness of the vision system in the harvesting robot [135][136][137] .

Picking efficiency to improve
Large-scale commercialization can be a critical indicator to evaluate the efficiency of harvesting robots. Most current harvesting robots still cannot fully meet the efficiency requirements of commercialization at present.
Firstly, one way can be to optimize the structural design of the end effector for a higher harvesting efficiency. Particularly, the end effector can be designed to pick the fruits most concisely and efficiently, according to the characteristics of harvesting fruits.
Secondly, the working time can also be extended to improve the picking efficiency. Therefore, the night image recognition and positioning algorithm can be required for the vision system, where the harvesting robot can work during the day, and then continue at night.

Cost-saving picking robots
A relatively lower cost can greatly contribute to commercialization.
The cost-saving harvesting robots are required to simplify the structure components without loss of harvesting efficiency.
Once a structured fruit planting environment can be used to reduce the requirements for the obstacle avoidance function of the picking robot, the DOF of the robotic arm can be reduced for manufacturing costs. Particularly, many current harvesting robots with more DOF mechanical arms can also be attributed to obstacle avoidance in the complicated planting environment.
Secondly, the less multi-function of the harvesting robot can also relatively reduce the cost. Some fruit harvesting robots have similarities in the design of the robotic arm. If the harvesting robot can easily switch the end effector and software, the harvesting of various types of fruits can be achieved with one picking robot, thereby improving the utilization efficiency of the picking robot with cost savings.

Integration of multiple functions
Combined with the structured planting environment, fruit and vegetable picking robots are developing into a comprehensive mobile harvester that integrates automatic harvesting, automatic sorting, and automatic packaging. At the same time, different end effectors can be automatically replaced to realize bagging, pruning, and other work.