Improving Orchard Efficiency with Autonomous Utility Vehicles

Improving Orchard Efficiency with Autonomous Utility Vehicles

Brad Hamner, Sanjiv Singh, and Marcel Bergerman

Carnegie Mellon University

Comprehensive Automation for Specialty Crops2

Orchard Tasks Today

Maintenance tasks require humans to drive vehicles a large distance over a long time

Comprehensive Automation for Specialty Crops3

Reconfigurable Mobility

• One type of vehicle can perform multiple tasks

– Mowing, spraying, etc.

– In use more often

• One automation package can be applied to multiple vehicles

– Only one system to learn

4Comprehensive Automation for Specialty Crops

Autonomous Prime Mover

• Automated Toro eWorkman

• Demonstrate orchard tasksby towing otherequipment

5Comprehensive Automation for Specialty Crops

Autonomous Prime Mover

• Laser range sensors

– Find trees

– Find obstacles

• Steering angle sensor

• Distance traveled sensor

• Controlled by laptop

Comprehensive Automation for Specialty Crops6

Seeing the Trees, Finding the Row

• From laser sensors,robot sees one slice of the world

• Need to find tree rows from this data

– Two parallel lines

Comprehensive Automation for Specialty Crops7

Seeing the Trees, Finding the Row

• From laser sensors,robot sees one slice of the world

• Need to find tree rows from this data

– Two parallel lines

Comprehensive Automation for Specialty Crops8

Line Representation

Lines represented by their perpendicularto the vehicle

distance d, angle

d

Tree point

Comprehensive Automation for Specialty Crops9

Hough Transform Row Detection

• Hough Transform – Popular line detector in image processing

• Create “votes” for lines

• Looking for parallel lines with most votes

Comprehensive Automation for Specialty Crops10

Hough Transform

• Pick a point in the image

How about this one?

Comprehensive Automation for Specialty Crops11

Hough Transform

• Many lines could go through that point.Draw them, and record their d,

Distance

Angle

1

1

1

1

Comprehensive Automation for Specialty Crops12

Hough Transform

• Pick another point. Draw its vote lines,and record their d,

Distance

Angle

1

2

1

1

1

1

1

Comprehensive Automation for Specialty Crops13

Hough Transform

• Pick another point and repeat.

Distance

Angle

1

1

3

1

1

1

1

1

1

1

Comprehensive Automation for Specialty Crops14

Hough Transform

• And repeat.

Distance

Angle

1

1

4

1 1

1 1

1

1

1 1

1

1

Comprehensive Automation for Specialty Crops15

Hough Transform

• Repeat for all points.Two lines dominate the voting.

Distance

Angle

1 1 1

1 2 1 1 1 1

9 9

1 1 1 1 2

1 1 1 1 1

2 1 1

1 1 3 1

1 1 1 1

2 1 2 2

1 1 1 1 1 1

Comprehensive Automation for Specialty Crops16

Hough Transform

• Find the top two peaks in the matrix.Those are the tree rows.

Distance

Angle

600

500

400

300

200

100

Votes

Actual Hough Matrix

Comprehensive Automation for Specialty Crops17

Hough Transform

• Find the top two peaks in the matrix.Those are the tree rows.

Distance

Angle

600

500

400

300

200

100

Votes

Actual Hough Matrix

Comprehensive Automation for Specialty Crops18

Row Following

• Hough matrix peaks are the tree rows

• Drive midway between the two

Comprehensive Automation for Specialty Crops19

Turning

• We know roughly how wide rows are

• Plan a path into the next row

• As the vehicle gets close, run row detection

20Comprehensive Automation for Specialty Crops

Comprehensive Automation for Specialty Crops21

Results

• Over 100 km autonomous travel

– Orchards in PA and WA

• Demonstrations

– Mowing

– Spraying

– Crop scouting

Comprehensive Automation for Specialty Crops22

Results

• 10 km with an autonomous platform

Comprehensive Automation for Specialty Crops23

Ongoing Work

• Uneven terrain brings the ground into view

– Presents data that looks like tree rows

– Confuses Hough detector

Comprehensive Automation for Specialty Crops24

Ongoing Work

• Particle filter

– Tracks possible rows over time

– Rejects bad rows

– More stable results

Houghalone

ParticleFilter

Comprehensive Automation for Specialty Crops25

Ongoing Work

• Not always much room to turn around

• Detect the available space

• Plan smooth trajectories

• Back up if necessary

Comprehensive Automation for Specialty Crops26

Related Collaboration

• Comprehensive Automation for Specialty Crops– http://www.frc.ri.cmu.edu/casc/CASC/Welcome.html

• Inexpensive localization

– Improve tracking, build maps

• Information management

• Autonomous disease detection

• Extension and usability

– User interface

– Commercialization studies

– Cost benefit analysis

Comprehensive Automation for Specialty Crops27

Acknowledgments

USDA Specialty Crop Research Initiative

Owners and managers ofSoergel Orchards

Penn State Fruit Research and Extension Center

Bear Mountain Orchards

Hollabaugh Orchards

Sunrise Orchards

Valley Fruit Orchards

28Comprehensive Automation for Specialty Crops