Wouter Bac MSc 27 September 2012

Wageningen UR Greenhouse Horticulture

Toward commercialization of robotic

systems for high-value crops: state-of-the-

art review and challenges ahead

Overview

Introduction

Plant Maintenance Operations (PMOs)

Literature review about harvesting robots

7 factors limiting commercialization

7 future challenges

2

Why this review

Despite 30 years of research no commercial robots for PMOs in high-value crops why?

Quantified performance unknown about state-of-the art

Future challenges to commercialize robotic systems for high-value crops

3

Literature review demarcation

Only high-value crops high labour costs (29 % NL)

Only multiple-harvest crops (tomatoes but not lettuce)

Only Plant Maintenance Operations (PMOs)

Time period 1981-2011

4

Operations in high value crops: PMOs candidate 1. Seeding, grafting and cutting

2. Transplanting

3. Transport of plants to a greenhouse

4. Planting

5. Plant Maintenance Operations – PMOs

1. Attaching plants to a supporting wire or stick

2. Sticking support sticks or knotting wires

3. Side shoot removal

4. Fruit/flower thinning

5. Leaf picking

6. Lowering plants

7. Crop protection/spraying

8. Harvesting (single/multiple)

6. Internal transport of plants or harvested products

7. Grading

8. Packing

9. Crop removal and cleaning

5

Review Methodology

6

Only harvesting robots were reviewed, examples:

7

Cucumber (Van Henten et al., 2003)

Melons (Edan, 1994)

Strawberry (Hayashi et al., 2010)

Egg-plant (Hayashi et al., 2001)

Cherry

(Tanigaki et al., 2008)

Methodology

Performance indicators

Autonomous: true/false

Tested in field or lab

Fruit localization success [%]

Fruit detachment success [%]

Cycle time [s]

Damage rate [%]

Number of fruits evaluated [#]

Detachment attempt ratio [total attempts/succesful attempts]

8

Methodology (2)

Design process techniques

Use of systematic design or systems engineering methods: true/false

Use of an economic analysis

Hardware design decision

Manipulator DOFs used

Off-the-shelve or custom-made hardware components

9

Methodology (3)

Algorithms partially or fully reported for:

Fruit localization

Ripeness determination

Obstacle localization

Task planning

Motion planning

Whether any of these algorithms were adaptive: true/false

10

Results

11

In total, 48 projects have been performed

12

Performance indicators

Cycle time was 35 ± 57 s (N=26) 13

Performance indicators (2)

73 % were autonomous

12.5 % reported the attempts made: avg 1.7 att per fruit

67 % were tested in the field

14

Systematic Design/ Economic analysis

12.5 % used a systematic design method

12.5 % performed an economic analysis

15

Manipulators mostly 3 DOF

16

7 factors limiting commercialization of

robotics systems

17

Factors 1-3 (quantified proof)

1. Cycle time too long (35 ± 53 s)

2. Fruit localization, detachment, and harvest (65.7 %) did not reach 100 %.

What about the other 35 % of the ripe fruits?

Lack of learning capabilities

3. Projects poorly reported

57 % of the performance indicators was based on a known number of test samples

Number of samples evaluated varied: 11 to 2506

Units missing in 24 % of the reported perf. indicators

18

Factors 1-3 (quantified proof)

3. Projects poorly reported

Damages to the plant never reported, exception (Pool & Harrell, 1991)

False-positive detections hardly reported

Requirements hardly described

Functionality of the robot hardly described

Algorithms hardly described (5-63 %)

Algorithms partly described in 49 % of the cases

Failed projects never reported

19

Factors 4-7 (indications only)

4. Robot designs probably suboptimal

Mostly travelling device, manipulator and

end-effector

Manipulator DOF

Alternative designs hardly explored

Economics hardly considered

5. Scattered market

6. PMOs differ among crops & limited machine utilization

7. Added value hardly explored

20

Future Challenges

21

Challenges 1-2 1. Modifying the environment

Different cultivation systems, e.g. cucumber

Supportive mechanisms, e.g. blowers, pushing mechanisms, canopy compression

Alternative cultivation methods, e.g. flower pruning to avoid fruit clusters

Cultivar selection and cross-breeding

2. Enhancing the robotic system

Learning capabilities, e.g. adaptive classifiers, on-line learning algorithms

Human-robot collaboration

Dedicated hardware, e.g. compliant end-effectors, optimized manipulators 22

Challenges 3-4 3. Using systematic design

Involve stakeholders: growers, engineers, academia

Define requirements for several aspects

QFD by Toyota; Cradle-to-cradle; etc.

Systems engineering (Edan & Miles, 1994)

4. Performing economic analyses System costs

Payback time

Damages

Maintenance costs

R&D costs

Example:

sweet-pepper harvester for 180 k€ @ 6s/fruit 23

Challenges 5-7

5. Adding value to the robotic system Tracking and Tracing

Ripeness prediction

Disease detection by chlorophyll fluorescence

Sorting and quality assesment directly after harvest

Phenotyping tasks

6. Multi-operational or multi-crop systems Apple and peach (Sites and Delwiche, 1988)

Cucumber leaf picking and harvesting (Van Henten et al. 2002, 2006)

Grape harvesting, spraying, bagging and thinning (Monta, 1995)

However, trade-off between cost and machine flexibility (Gupta & Goyal, 1989)

7. Improve knowledge transfer 24

Conclusion

Operations summarized

Only 2 projects for PMOs other than harvesting and spraying

48 harvesting robot projects identified

7 Limiting factors identified

7 Future challenges established

25

Thank you for your attention

26