Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
RHEA: Robot Fleets for Highly Efficient Agriculture and Forestry Management and Forestry Management
NMP2-LA-2010-245986
The RHEA project The RHEA project The RHEA project The RHEA project
Pablo Gonzalez-de-SantosC t f A t ti d R b tiCentre for Automation and Robotics
5ª Conferencia del VII Programa Marco de I+D de la Unión Europea en España
San SebastianJune 21 2011June 21, 2011
EURO P E A NCOMMISSION
Community Research Outline
1. Development of the proposal 2 RHEA bj ti2. RHEA objectives3. Preliminary system breakdown4 C h 4. Comment on the project execution
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
1 Development of the proposal1. Development of the proposal
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal • Call identifier: FP7-NMP-2009-LARGE-3.• Theme 4: “Nanosciences, Nanotechnologies, Materials and
N d h l NM
p p p
New Production Technologies – NMP”• Activity/Area: NMP-2009-3.4-1 Automation and robotics
for sustainable crop and forestry managementfor sustainable crop and forestry management• Funding scheme: Collaborative Projects, Large-scale
integrating• Two stage proposal
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal Angela Ribeiro (CAR)p p p
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal Angela Ribeiro (CAR)p p p
Elena Garcia (CAR)Pablo Gonzalez de Santos (CAR)
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal p p p
ROBOT FLEETS FOR HIGHLY EFFECTIVE AGRICULTURE AND FORESTRY MANAGEMENT (RHEA)
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal p p p
Perception
obot
s
s
f m
obile
ro
chno
logi
es
Flee
t of
ICT
Tec
Actuation
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal • First stage title: ROBOT FLEETS FOR HIGHLY EFFECTIVE CROP
MANAGEMENT IN MEDITERRANEAN AGRICULTURE
• Acronym: RHEA (Demeter’s mother the Greek Goddess of
p p p
• Acronym: RHEA (Demeter s mother, the Greek Goddess of agriculture)
• First stage goal: To diminish the use of herbicide, improve crop quality and reduce production costs by means of crop quality and reduce production costs by means of sustainable weed management using a fleet of resourceful, collaborative, autonomous, heterogeneous and complementary robots equipped with new sensors endcomplementary robots equipped with new sensors, end-effectors and control algorithms
• Consortium: 14 partners, 17 groups
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal • First stage title: ROBOT FLEETS FOR HIGHLY EFFECTIVE CROP
MANAGEMENT IN MEDITERRANEAN AGRICULTURE
• Acronym: RHEA (Demeter’s mother the Greek Goddess of
p p p
• Acronym: RHEA (Demeter s mother, the Greek Goddess of agriculture)
• First stage goal: To diminish the use of herbicide, improve crop quality and reduce production costs by means of crop quality and reduce production costs by means of sustainable weed management using a fleet of resourceful, collaborative, autonomous, heterogeneous and complementary robots equipped with new sensors endcomplementary robots equipped with new sensors, end-effectors and control algorithms
• Consortium: 14 partners, 17 groups
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• Agronomists R b ti i t
p p p
• Roboticists• Designer/Manufacturers of equipment
D /M f f h l• Designer/Manufacturers of vehicles• Experts in communications, electronics, computer
iscience• Experts in new energy system
d• End users
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal p p p
• Agronomists R b ti i t• Roboticists
• Designer/Manufacturers of equipmentD /M f f h l• Designer/Manufacturers of vehicles
• Experts in communications, electronics, computer iscience
• Experts in new energy systemd• End users
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal: Consortiump p p
CSIC-CAR
Web
CSIC-ICA CSIC-IASFTW
Tropical
WebUCM
U. Pisa
U Fl
SAPUPM-EII
UPM EIA
CogVisCyberbotics
p
U. Florence UPM-EIA
AirRobot
Cemagref
CNH-B BlueBotics
CNH FCemagref CNH-F
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• First stage statistics:
– 149 proposals in 6 areas (18 proposals in the area of
p p p
9 p p ( p pAutomation and robotics for sustainable crop and forestry management)33 proposals over the threshold (5 in our area)– 33 proposals over the threshold (5 in our area)• No official information on the score
• Comments from reviewers:– To extend the project to products and processes all over
Europe– To extend the proposal to forestry applicationso e e d e p oposa o o es y app ca o s
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal • Second stage Title: ROBOT FLEETS FOR HIGHLY EFFECTIVE AGRICULTURE AND
FORESTRY MANAGEMENT (RHEA)
p p p
• Second stage goal: The RHEA project is devoted to change the traditional way of proceeding in agriculture and forestry for weed removal and pest management by putting together a fleet of small, safe, reconfigurable, heterogeneous and complementary robots heterogeneous and complementary robots
– to minimize » chemical products» energy gy» operation time
– to maximize » the quality of products » safety
– to guarantee » the application of the procedures to the entire operation field
Centre for Automation and Robotics
• Consortium: 15 partners, 19 groups
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal: Consortiump p p
CSIC-CAR
Web CN
CSIC-ICA CSIC-IASFTW
Tropical
WebUCM
CN
U. Pisa
U Fl
SAPUPM-EII
UPM EIA
CogVisCyberbotics
p
U. Florence UPM-EIA
AirRobot
Cemagref
CNH-B BlueBotics
CNH FCemagref CNH-F
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal: Consortiump p p
Air RobotCyberbotics
Bluebotics CogVis
CNHBeneficiary Number
Beneficiary name
Beneficiary
short name
Country
at Agencia Estatal Consejo Superior de
CSIC‐IAI, ICA
UPM‐EIA, EII
CNH‐F
Bluebotics
FTW1
(coo
rdin
aor
)
Agencia Estatal Consejo Superior de Investigaciones Científicas - CSIC
a) Centro de Automática y Robóticab) Instituto de Ciencias Agrarias c) Instituto de Agricultura Sostenible
(CSIC-IAI) (CSIC-ICA) (CSIC-IAS)
Spain
2 CogVis GmbH CV AustriaF h T l k ik i W A
UCM
CSIC‐IAS
SPA
Tropical
Cemagref
U. Pisa
U. FlorenceCM
3Forschungszentrum TelekommunikationWien Ltd.
FTW Austria
4 Cyberbotics Ltd CY Switzerland5 Università di Pisa UP Italy6 Universidad Complutense de Madrid UCM Spain7 Tropical TRO Greece8 Soluciones Agrícolas de Precisión S.L. SAP Spain
9Universidad Politécnica de Madrid (UPM)
a) ETS Ingenieros Agrónomos b) ETS Ingenieros Industriales
UPM-EIA UPM-EII
Spain
10 AirRobot GmbH & Co. KG AR Germany
8 countries
3 Research Centers4 U i i i
y11 Università degli Studi di Firenze UF Italy
12Centre National du Machinisme Agricole, du Génie Rural, des Eaux et des Forêts -CEMAGREF
CE France
13Case New Holland Belgium N.V.
- Case New Holland France S.A.CNH
CNH-FBelgiumFrance
4 Universities7 SME1 Industry
Centre for Automation and Robotics
Case New Holland France S.A. CNH F France14 Bluebotics BL Switzerland15 CM Srl CM Italy
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectivesB 2 I l t ti
p p p
• B.2. Implementation• B.3. Impact
B 4 E h l • B.4. Ethical issues• B.5. Gender aspects
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectives• Scientific and technical part
p p p
Scientific and technical part
• Addressing the topics of the call
• Progress beyond the state of the artart
• …………….
• B.2. Implementationp• B.3. Impact• B.4. Ethical issuesB.4. Ethical issues• B.5. Gender aspects
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectives• Scientific and technical part
p p p
Scientific and technical part
• Addressing the topics of the call
• Progress beyond the state of the artart
• …………….
• B.2. Implementationp• B.3. Impact• B.4. Ethical issuesB.4. Ethical issues• B.5. Gender aspects
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectivesB 2 I l t ti
p p p
• B.2. Implementation• Management structures and procedures
• Responsible persons
• Management committees
• ………………………………
• B.3. Impact• B.4. Ethical issuesB.4. Ethical issues• B.5. Gender aspects
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectivesB 2 I l t ti
p p p
• B.2. Implementation• Management structures and procedures
• Responsible persons
• Management committees
• ………………………………
• B.3. Impact• B.4. Ethical issuesB.4. Ethical issues• B.5. Gender aspects
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectivesB 2 I l t ti
p p p
• B.2. Implementation• B.3. Impact
• Contributions to the expected• Contributions to the expected impact of the specific topic
• Steps needed
N i f E h• Necessity of an European approach
• Impact on participants
• Dissemination and exploitation
• Risk assessment and risk plan
• B.4. Ethical issues
Centre for Automation and Robotics
• B.5. Gender aspects
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectivesB 2 I l t ti
p p p
• B.2. Implementation• B.3. Impact
• Contributions to the expected• Contributions to the expected impact of the specific topic
• Steps needed
N i f E h• Necessity of an European approach
• Impact on participants
• Dissemination and exploitation
• Risk assessment and risk plan
• B.4. Ethical issues
Centre for Automation and Robotics
• B.5. Gender aspects
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectivesB 2 I l t ti
p p p
• B.2. Implementation• B.3. Impact
B 4 E hi l i• B.4. Ethical issues• Not applicable
• B 5 Gender aspects• B.5. Gender aspects
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal
• B.1. Concepts and objectivesB 2 I l t ti
p p p
• B.2. Implementation• B.3. Impact
B 4 E h l • B.4. Ethical issues• B.5. Gender aspects
W / b l• Women/men balance
• WPs leaded by women
• Reconciling professional, family and private lives
• etc………………………
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 1. Development of the proposal overview
• Second stage statistics:
– 33 proposals in 6 areas (5 in Automation and robotics for
p p p
p p (sustainable crop and forestry management)
– 16 proposals over the threshold (2 in our area)RHEA d (13 5/15)RHEA scored (13.5/15)
• Comments from reviewers/officer:
– To modify the budget regarding management and y g g g gdemonstration (6 negotiation periods)
– To describe the perception group capabilities in the Description of Work Description of Work
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Project overview
• Duration: 4 yearsB d t
j
• Budget: – Total cost: 8.96 M€– Requested EC contribution: 6 59 M€– Requested EC contribution: 6.59 M€
• Person-month: 1081• WPs: 10• WPs: 10• Milestones: 6• D li bl 68• Deliverables: 68
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Proposal process and datesp p
Call 3 November 19, 20081st stage proposal submission February 17, 20091st stage passed April 8, 2009Meeting in Brussels May 11, 2009
d2nd stage proposal submission July 22, 20092nd stage passed October 6, 20091st negotiation meeting in Brussels November 24, 2009
mon
ths
NEF opened November 25, 2009….(6 consecutive negotiation periods)…NEF closed April 30, 2010
20.5
p ,Consortium Agreement and Grant AgreementProject starting date August1st, 2010
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
2. RHEA objectives
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Overall objective• The RHEA project is devoted to change the traditional way of proceeding in
agriculture and forestry for weed removal and pest management by putting together a fleet of small, safe, reconfigurable, heterogeneous and complementary
j
robots to minimize: chemical products, energy and operation time, while maximizing: the quality of products and safety, and guaranteeing the application of the procedures to the entire operation field
Centre for Automation and RoboticsPreliminary idea (proposal)Preliminary idea (proposal)
EURO P E A NCOMMISSION
Community Research Overall objective
• Improving the initial idea (Kickoff meeting)
j
VIDEO RHEA-fleet.wmv
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
mechanical systems5. To develop a fleet of ground and aerial mobile units equipped for acquiring
images of the task field. The ground mobile units must be capable of l i h i l h i l f d d tapplying mechanical or chemical processes for crop and weed management
6. To design robot guidance devices and algorithms based on computer vision (forward-looking view of the crop rows and obstacle avoidance)
7 To build human machine interfaces for monitoring/controlling autonomous 7. To build human-machine interfaces for monitoring/controlling autonomous outdoor vehicles
8. To develop new strategies for planning the mission with teams of heterogeneous robots, and re-planning the mission after the failure of a
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and
weed mapping based on computer vision
2. To design innovative algorithms for decision-making modules, including
p j
The target is to detect 90% of the weed patchesg g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
g % p
mechanical systems5. To develop a fleet of ground and aerial mobile units equipped for acquiring
images of the task field. The ground mobile units must be capable of l i h i l h i l f d d tapplying mechanical or chemical processes for crop and weed management
6. To design robot guidance devices and algorithms based on computer vision (forward-looking view of the crop rows and obstacle avoidance)
7 To build human machine interfaces for monitoring/controlling autonomous 7. To build human-machine interfaces for monitoring/controlling autonomous outdoor vehicles
8. To develop new strategies for planning the mission with teams of heterogeneous robots, and re-planning the mission after the failure of a
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
mechanical systems5. To develop a fleet of ground and aerial mobile units equipped for acquiring
images of the task field. The ground mobile units must be capable of l i h i l h i l f d d tapplying mechanical or chemical processes for crop and weed management
6. To design robot guidance devices and algorithms based on computer vision (forward-looking view of the crop rows and obstacle avoidance)
7 To build human machine interfaces for monitoring/controlling autonomous 7. To build human-machine interfaces for monitoring/controlling autonomous outdoor vehicles
8. To develop new strategies for planning the mission with teams of heterogeneous robots, and re-planning the mission after the failure of a
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying
4. To improved end-effectors to destroy weeds based on both thermal and The target is a reduction of chemicals in about 75% mechanical systems
5. To develop a fleet of ground and aerial mobile units equipped for acquiring images of the task field. The ground mobile units must be capable of
l i h i l h i l f d d t
g
applying mechanical or chemical processes for crop and weed management6. To design robot guidance devices and algorithms based on computer vision
(forward-looking view of the crop rows and obstacle avoidance)7 To build human machine interfaces for monitoring/controlling autonomous 7. To build human-machine interfaces for monitoring/controlling autonomous
outdoor vehicles8. To develop new strategies for planning the mission with teams of
heterogeneous robots, and re-planning the mission after the failure of a
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
mechanical systems
5. To develop a fleet of ground and aerial mobile units equipped for acquiring images of the task field. The ground mobile units must be capable of
l i h i l h i l f d d t
The goal is to destroy up to 90% of the detected weeds
applying mechanical or chemical processes for crop and weed management6. To design robot guidance devices and algorithms based on computer vision
(forward-looking view of the crop rows and obstacle avoidance)7 To build human machine interfaces for monitoring/controlling autonomous 7. To build human-machine interfaces for monitoring/controlling autonomous
outdoor vehicles8. To develop new strategies for planning the mission with teams of
heterogeneous robots, and re-planning the mission after the failure of a
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
mechanical systems5. To develop a fleet of ground and aerial mobile units equipped for
acquiring images of the task field. The ground mobile units must be bl f l i h i l h i l f d d capable of applying mechanical or chemical processes for crop and weed
management
6. To design robot guidance devices and algorithms based on computer vision (forward-looking view of the crop rows and obstacle avoidance)
The ground mobile units must be small, lightweight, and reliable, with real-time communication facilities and featuring motion on slopes up to 15 degrees with
d t ti t t i ith i l iti i f ±20 d (forward looking view of the crop rows and obstacle avoidance)7. To build human-machine interfaces for monitoring/controlling autonomous
outdoor vehicles8. To develop new strategies for planning the mission with teams of
adaptation to uneven terrain with irregularities in a range of ±20 cm and a maximum speed of 6 Km/h
Centre for Automation and Robotics
heterogeneous robots, and re-planning the mission after the failure of a number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
mechanical systems5. To develop a fleet of ground and aerial mobile units equipped for acquiring
images of the task field. The ground mobile units must be capable of l i h i l h i l f d d tapplying mechanical or chemical processes for crop and weed management
6. To design robot guidance devices and algorithms based on computer vision (forward-looking view of the crop rows and obstacle avoidance)
7 To build human machine interfaces for monitoring/controlling autonomous The mobile nits will follow the rows at a speed of abo t 6 7. To build human-machine interfaces for monitoring/controlling autonomous outdoor vehicles
8. To develop new strategies for planning the mission with teams of heterogeneous robots, and re-planning the mission after the failure of a
The mobile units will follow the rows at a speed of about 6 Km/hour with a positioning accuracy of about ±2cm
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
mechanical systems5. To develop a fleet of ground and aerial mobile units equipped for acquiring
images of the task field. The ground mobile units must be capable of l i h i l h i l f d d tapplying mechanical or chemical processes for crop and weed management
6. To design robot guidance devices and algorithms based on computer vision (forward-looking view of the crop rows and obstacle avoidance)
7 To build human-machine interfaces for monitoring/controlling autonomous 7. To build human-machine interfaces for monitoring/controlling autonomous outdoor vehicles
8. To develop new strategies for planning the mission with teams of heterogeneous robots, and re-planning the mission after the failure of a
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Specific objectives1. To develop advanced systems and algorithms for crop assessment and weed
mapping based on computer vision 2. To design innovative algorithms for decision-making modules, including
p j
g g g , gbehaviours such as coordination, cooperation and collaboration
3. To build enhanced actuators for precise, real-time herbicide spraying 4. To improved end-effectors to destroy weeds based on both thermal and
mechanical systems5. To develop a fleet of ground and aerial mobile units equipped for acquiring
images of the task field. The ground mobile units must be capable of l i h i l h i l f d d tapplying mechanical or chemical processes for crop and weed management
6. To design robot guidance devices and algorithms based on computer vision (forward-looking view of the crop rows and obstacle avoidance)
7 To build human machine interfaces for monitoring/controlling autonomous 7. To build human-machine interfaces for monitoring/controlling autonomous outdoor vehicles
8. To develop new strategies for planning the mission with teams of heterogeneous robots, and re-planning the mission after the failure of a
Centre for Automation and Robotics
number of robots
EURO P E A NCOMMISSION
Community Research Project overview• Project scenarios:
– Wide row crops:
j
• Maize• Sunflower• Sugar beetSugar beet• Tomatoes
– Narrow row crops:• Wheat• Barley
– Woody perennials:Woody perennials:• Olive groves• Walnut trees
Al d
Centre for Automation and Robotics
• Almond trees
EURO P E A NCOMMISSION
Community Research Project overview• Project scenarios:
– Wide row crops:
j
• Maize• Sunflower• Sugar beetSugar beet• Tomatoes
– Narrow row crops:• Wheat• Barley
– Woody perennials:Woody perennials:• Olive groves• Walnut trees
Al d
Centre for Automation and Robotics
• Almond trees
EURO P E A NCOMMISSION
Community Research
3. Preliminary system breakdown
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research System breakdown
1. Base Station and GUI: Friendly interfaces for system operation, for information record-keeping and for optimizing operations
y
p g p g p2. Mission Manager: This module allows the operator to define the main mission
parameters and then it reports a plan for motion coordination in multiple mobile robot teams
3 C i ti t I t f d i ti th t it ff ti 3. Communication system: Interfaces and communications that permit effective real time operation, control and decision making
4. Perception system: The PS aims at weed identification and its precise location based on advanced vision sensors, and image processing techniques and algorithms. This system will be installed onboard the mobile units
5. Actuation system: Provides direct actuation in the crop field through end effectors and actuators for precise herbicide application (conventional crops) and/or mechanical weed control (organic crop)p ) / ( g p)
6. Mobile Units: Medium-sized autonomous vehicles with onboard equipment for navigation and application of treatments (terrestrial, aerial, etc)
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
Base Station & GUI
System breakdown
Mission Manager (MM)
Communication/Location -A (CS)
Communication/Location -B (CS)On the field
In the Base Station
High level decision making em
(A
S)Perception System (PS)
Hi h l l
DGPS
(HLDM)
Low level decision making ct
uatio
n Sy
steHigh level
processing
Pre-processing gDose valve controllers
Device systemA
cImage acquisition
Centre for Automation and Robotics
Mobile Units (MU)Laser cameras
Crop
EURO P E A NCOMMISSION
Community Research 1. Base station and GUI
• Friendly interface for system operation, information record-keeping and optimizing operations
Direct viewCommandsSimulation
Mission Manager (MM) • Data streams fromo Mobile Units
Centre for Automation and Robotics
• Data and commands to the Mission Planner
o Mobile Unitso Actuation system
EURO P E A NCOMMISSION
Community Research 1. Base station and GUI
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
Base Station & GUI
2 Mission Manager:
Mission Manager (MM)
2. Mission Manager: This module allows the operator to define the main mission parameters and
Communication/Location -A (CS)
Communication/Location -B (CS)On the field
In the Base Station
pthen it reports a plan for motion coordination in multiple mobile robot
High level decision making em
(A
S)
teams Perception System (PS)
Hi h l l
DGPS
(HLDM)
Low level decision making ct
uatio
n Sy
steHigh level
processing
Pre-processing gDose valve controllers
Device systemA
cImage acquisition
Centre for Automation and Robotics
Mobile Units (MU)Laser cameras
Crop
EURO P E A NCOMMISSION
Community Research • Task definition• Number of robots
Base Station & GUI 2. Mission Manager:
• Number of robots • Robot features• Field features• P i k i f ti
Mission Manager (MM)
• Previous known information on the field (remote sensing, sampling, …)
Communication/Location -A (CS)
Communication/Location -B (CS)On the field
In the Base Station
High level decision making em
(A
S)Perception System (PS)
Hi h l l
• Sequence of robot actions to accomplish the task/path for
DGPS
(HLDM)
Low level decision making ct
uatio
n Sy
steHigh level
processing
Pre-processing
the task/path for navigation
gDose valve controllers
Device systemA
cImage acquisition
Centre for Automation and Robotics
Mobile Units (MU)Laser cameras
Crop
EURO P E A NCOMMISSION
Community Research
Base Station & GUI3. Communication and location system
Mission Manager (MM)
location system
Communication/Location -A (CS)
Communication/Location -B (CS)On the field
In the Base Station
• Communication and
High level decision making em
(A
S)
• Communication and Location
Perception System (PS)
Hi h l l
DGPS
(HLDM)
Low level decision making ct
uatio
n Sy
steHigh level
processing
Pre-processing gDose valve controllers
Device systemA
cImage acquisition
Centre for Automation and Robotics
Mobile Units (MU)Laser cameras
Crop
EURO P E A NCOMMISSION
Community Research 3. Communication and location system• This system will be installed on the following elements:
– Ground and aerial mobile units
y
– The base stations– The personal mobile terminals
• The communication subsystem will provide: – Flexible interfaces for wireless communication– Data collection and dissemination, among the referred elements – Internet connectivity at the base station for the remote management
of the system• The localization subsystem will provide:
– The position of the ground and aerial mobile units – Identification and localization of operators
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 3. Communication and location system• This system will be installed on the following elements:
– Ground and aerial mobile units
y
– The base stations– The personal mobile terminals
• The communication subsystem will provide: – Flexible interfaces for wireless communication– Data collection and dissemination, among the referred elements – Internet connectivity at the base station for the remote management
of the system• The localization subsystem (GPS) will provide:
– The position of the ground and aerial mobile units – Identification and localization of operators
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
Base Station & GUI 4. Perception system• The PS aims at weed identification and its
Mission Manager (MM)
identification and its precise location based on advanced vision sensors, image
Communication/Location -A (CS)
Communication/Location -B (CS) On the field
In the Base Station
, gprocessing techniques and algorithms
High level decision making em
(A
S)Perception System (PS)
Hi h l l
DGPS
(HLDM)
Low level decision making ct
uatio
n Sy
steHigh level
processing
Pre-processing gDose valve controllers
Device systemA
cImage acquisition
Centre for Automation and Robotics
Mobile Units (MU)Laser cameras
Crop
EURO P E A NCOMMISSION
Community Research 4. Perception system
– Remote sensing: information
p y
– Ground sensing: information ll d f d b gathered by means of
aircrafts or UAVs (multispectral, hyperspectral)
collected from ground robotic platforms (vision cameras, multispectral)
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 4. Perception system: Remote sensingp y g
Video Drone-crops.wmn
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 4. Perception system: Remote sensing
Tetracam mini-MCA• 6 channels configuration (G, R, NIR).• 1080x1024 (1.3 Mpixel).• Weight 700 g.
Olympus EP-1• Visible channels (B, G, R).• 12.3 Mpixel.• Weight 335 g• Weight 335 g.
Flight altitude ≈ 50 mArea ≈ 26x32 m (≈ 0 08 Ha)
Centre for Automation and Robotics INSTITUTE FOR SUSTAINABLE AGRICULTURE
Area ≈ 26x32 m (≈ 0.08 Ha) # Images/Ha ≈ 12
EURO P E A NCOMMISSION
Community Research 4. Perception system: Ground sensing• Real crop field in Arganda (Madrid)• Camera: Basler: (HxV = 1390x1038)• Focal length: 8mm• Height (camera) from the ground: 1.5 m
Di t f th t th fi t h i t l li 2• Distance of the camera to the first horizontal line: 2m• High weed densities
Land marks covering an area of 2.25 x 4m Expected row crops based on the camera geometry Vision system
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
Base Station & GUI 5. Actuation system
Mission Manager (MM)• Provides direct actuation
in the crop through end effectors and actuators
Communication/Location -A (CS)
Communication/Location -B (CS)On the field
In the Base Station for precise herbicide application (conventional crops) and/or
High level decision making em
(A
S)
mechanical and thermal weed control (organic crops)
Perception System (PS)
Hi h l l
DGPS
(HLDM)
Low level decision making ct
uatio
n Sy
steHigh level
processing
Pre-processing gDose valve controllers
Device systemA
cImage acquisition
Centre for Automation and Robotics
Mobile Units (MU)Laser cameras
Crop
EURO P E A NCOMMISSION
Community Research 5. Actuation system
• Device system: is in charge of generating the required actions of the actuation equipment
y
q p– End-effectors to kill weeds
• Chemical tools:B ith l– Bars with spray nozzles
• Physical tools– Mechanical end-effectors– Thermal end-effectors
– Manipulators to handle the end-effectors
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation system
• Device system: is in charge of generating the required actions of the actuation equipment
y
q p– End-effectors to kill weeds
• Chemical tools:B ith l– Bars with spray nozzles
• Physical tools– Mechanical end-effectors– Thermal end-effectors
– Manipulators to handle physical tools
Herbicide Boom
Objectives 3 and 4
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation systemy
Herbicide boomRobot engine and controller
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation system
• Device system: is in charge of generating the required actions of the actuation equipment
y
q p– End-effectors to kill weeds
• Chemical tools:B ith l– Bars with spray nozzles
• Physical tools– Mechanical end-effectors– Thermal end-effectors
– Manipulators to handle the end-effectors
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation system
• Device system: is in charge of generating the required actions of the actuation equipment
y
q p– End effectors for weed control
• Chemical tools:B ith l– Bars with spray nozzles
» Commercial nozzles • Physical tools
4 m(c)
– Mechanical end-effectors– Thermal end-effectors
– Manipulators to handle the end-effectors
Tool
p
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation systemy
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation system: first prototypey p yp
21460.75 m
25 cm4.5 m
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation system: first prototypey p yp
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 5. Actuation system: Olive-tree application toolsy pp=0ºor =-15º
US sensor UM30-····X
Actuator
=15ºor
0º=0º
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
Base Station & GUI
G d bil it
6. Mobile units
Mission Manager (MM)
• Ground mobile units: Medium-sized ground vehicles with onboard equipment for
Communication/Location –A (CS)
Communication/Location -B (CS)On the field
In the Base Stationequipment for
– Navigation– Weed detection– Application of treatments
High level decision making em
(A
S)
pp
• Aerial units: Small aerial units with onboard equipment for
Perception System (PS)
Hi h l l
DGPS
(HLDM)
Low level decision making ct
uatio
n Sy
ste q p
– Acquiring images of the task field
High level processing
Pre-processing gDose valve controllers
Device systemA
cImage acquisition
Centre for Automation and Robotics
Mobile Units (MU)Laser cameras
Crop
EURO P E A NCOMMISSION
Community Research 6.1 Ground mobile units
Sensor box
Herbicide bar
4 m(c)
Height 1 mHerbicide tankers 75 litter ≈0.1×1×0.75 m
Adaptable spread from 0.6 m to 0.95 m
• The same ground mobile unit structure for three different tasks– Spray bar
/– Positioning systems for mechanical/Thermal tools in agricultural tasks– Positioning systems for forestry tasks
• All ground mobile units equipped with a engine and a AC generator
• One ground mobile unit equipped with a fuel cell and active suspension
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 6.1 Ground mobile units
• Based on a commercial tractor (Boomer T3050)(+) Earlier availability( ) B l b l (+) Better reliability (+) Fulfilling lots of norms
(-) Heavier soil compaction(-) More safety problems( ) y p
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 6.1 Ground mobile units
Crop row detector and GPSCommunication
Electronic equipment on-board the ground units
Crop row detector and obstacle detector
GPS
Computers
Weed detection system Laser: obstacle detection system and crop stage
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Electronic equipment on-board the ground units
(SAP)
Bluebotics
CNH
CSIC-CAR
Bluebotics
FTW
Bluebotics
FTW
Bluebotics
Centre for Automation and Robotics
SICK (UPM-EIA)
EURO P E A NCOMMISSION
Community Research Application in wheatpp
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Application in maizepp
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Application in olive treespp
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 6.2 Aerial mobile units
• Two aerial mobile units (quadrotors) equipped with vision cameras cameras
I iti l d t ti f d t h– Initial detection of weed patches– Supervision during operation
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 6.2 Aerial mobile units
Video Drone.mp4
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research 6.2 Aerial mobile units
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Final operation of the fleetp
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research Final demos
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
Exploitation of the overall RHEA system Exploitation of the overall RHEA system
Length: 13.60 mgWidth: 2.45 m – 2.48 mHeight: 2.50 m – 2.70 mRoom for 34 euro-pallets Payload: 25 tony
12 m
2 5 2.5 m
Centre for Automation and Robotics
EURO P E A NCOMMISSION
Community Research
Comment on the project execution
• Main problem:19 working groups with about 40 different versions of
Comment on the project execution
– 19 working groups with about 40 different versions of the project
• Solution:So u o :– Calm, patience and perseverance (In month 10th we
only have about 10 different versions of the RHEA project)
Centre for Automation and Robotics
RHEA: Robot Fleets for Highly Efficient Agriculture and Forestry Management and Forestry Management
NMP2-LA-2010-245986
The The RHEA project RHEA project The The RHEA project RHEA project
Pablo Gonzalez-de-Santos
5ª Conferencia del VII Programa Marco de I+D5ª Conferencia del VII Programa Marco de I+D de la Unión Europea en España
San SebastianJune 21, 2011