Intelligent off-road vehiclesMartin Servin, Department of Physics, 2008-04-02

www.umu.se\proj\ifor

Outline

• Background to the field• Overview IFOR• Autonomous navigation• Crane automation• Simulator based design

Feel free to ask questions and make comments and proposals!

A sample of technological gems…

Mars rover – extreme teleoperation

Deep Blue – reasoning computer

DARPA Grand Challenge – competition with autonomous vehicles

QRIO – balancing robot

Parthenon – virtual 3D reconstruction

HCI – retinal display

The off-road challangeDemand for new technology• Increased productivity• Increased safety and work environment• Environmental sustainability

The forestry challange• Complex work processes to automate

– no computer beats the human in running a harvester• Rough environment with big variations

– sensor vision in forest, robust and sustainable system handling vibrations, moist and dirt

Vision from forestry industry

“2025 – Ingen man i maskinen, ingen hand på spakarna “

• An initiavive for R&D for intelligent off-road technology

• Initiated by the industry in 2001

• Collaboration between academia and industry

= a forum for R&D and a collection of projects focused at IFOR technology

What is IFOR ?

Academia:Umeå University

Swedish University of Agricultural Sciences

Skogforsk

Industry:Komatsu Forest

Holmen Skog

Sveaskog

BAE Systems Hägglunds

LKAB

+ network of other research centers and companies

Technology vision

• Improved work environment

• Increased productivity and cut costs

• Increased safety

• Reduced environmental impact

Technology:Technology:Goals:Goals:

• Control algorithms and modeling

• Interaction – man, machine and environment

• Sensor vision

• Localization and map building

20012001 2025202520120100

Automation Automation of routine of routine work work processesprocesses

Crane tip Crane tip controlcontrol

UnmanneUnmanned vehicles d vehicles

Activities and projects

Autonomous navigationDr Thomas Hellström1 PhD studentsComputing Science Department

Smart Crane ControlProf Anton Shiriaev1 FoAss, 1 PostDoc, 3 PhDDepartment of Applied Physics and Electronics

Vehicle simulatorsDr Martin ServinIn collaboration with VRlab at UmU

Miscellanious- Seminars and workshops- Experiments and pre-studies- Student projects

Equipment• Forest machines – valmet forwarder and harvester• Full sized in-door hydraulic crane• Portable prototyping hardware for feedback control • Sensors (dgps, laser radar, hydraulic pressure,

stereo camera,…) • Simulator systems

Funding> 25 MSEK since 2001Kempe Foundation, Sveaskog, Vinnova,Komatsu Forest, Sparbanksstiftelsen Norrland, Umeå University, LKAB, BAE Systems Hägglunds

Other actors

SLU

Skogforsk

Applied Mathematics – Prof Mats G Larsson

Design Institute

UCIT / ProcessIT Innovations

Autonomous navigation

Autonomous navigationDr Thomas Hellström

1 PhD students

- unmanned transportation of logs

- localization, path tracking and path planning

- RTK-DPGS with cm accuracy

- laser scanners, radars,...

- machine learning and sensor fusion

- first prototype demonstrated in Dec 2005

- ”Simulator in the loop”

Autumn 2008 we are running the student DBT-projects:

- Sensor vision and remote operation

- Simulation of terrain vehicle with autonomous abilities

www.cs.umu.se/research/ifor/IFORnav/videos.htmwww.cs.umu.se/research/ifor/IFORnav/videos.htm

Smart Crane Control

Smart Crane ControlProf Anton Shiriaev – Control System Theory1 FoAss, 1 PostDoc, 3 PhD

- motion planning, motion control for mechanical systems- feedback design for hydraulically actuated cranes- crane tip control- optimized motions – speed and stability- semi-automation, e.g. automatich loading- VR-enabled remote operation- portable prototyping hardware for feedback control

Recent results:- motion faster and more stable than human operator – Valmet forwarder- demonstrated automatic loading in lab

Grant from “Stiftelsen för strategisk forskning” for crane control using only hydraulic measurements at Komatsu Forest

1 industrial PhD have been granted (?) - Komatsu Forest and Umeå University splitting the costs 50-50 – Semi-autonomous harvester control system

Fast crane motion.avi

Motion faster and more stable than human operator is possible!

Virtual Environment Teleoperation

Click control.avi Detection of rotating log.avi

Visual Simulation of Machine Concepts for Forest Biomass Harvesting

Martin Servin, A. Backman, K. Bodin - Umeå University, SwedenU. Bergsten, D. Bergström, T. Nordfjell, I. Wästerlund - Swedish University of Agricultural Sciences, Sweden B. Löfgren - Skogforsk (the Forestry Research Institute of Sweden)

VRIC 2008 – 10th International Conference on Virtual Reality (Laval Virtual)

Outline• Simulator-based design• Forest biomass harvesting

– concept machine and work method

• Experiments in simulator environment– system and procedure– purpose: find optimal harvesting technique and machine design

Training simulator technology – also for concieving new machines concepts and work methods

Simulator-based design (SBD)

Simulation tools are converging – R&D process impoves – cross-disciplinary participation

• Extension of virtual prototyping and simulation to include human-in-the-loop• Fast and sheap• Simulators – complex yet controllable environments

Figure from T. Alm ”Simulator-based design” (2007).

End customer Manufacturer Designer ResearcherEngineer

Simulator training

Application of SBD to:

Forest biomass harvesting

• Increasing demand for forest biomass• Early harvesting/thinning is becoming profitable• Large volumes and areas, small income per unit, energy consumption• Crucial to use optimized technology – economically and environmentally sustainable

• Uncertain on what solution to choose for thinning

• Virtual and real prototypes are important!

New harvesting methods in dense forest stands

Early harvesting = thinning + biomass harvesting - single-tree harvesting - multi-tree harvesting - geometric area based felling

strip roads 3 m wide every 15-20 mcorridors 1x10 m10 trees, 6 m, 50 kgcollect in piles of 50 trees

Machine concept for harvesting in dense forest stands

Size: 4x2 m, 2.5 ton, 8m reachMobility: indv 4W on pendulum armsHarvester head: multi-tree vs bladeControl and HMI: boom-tip control, semi-autonomous,teleoperation (direct or VE), laser scanner & stereo camera, dynamic 3D maps from sattelite and AUV

Experiments in simulator environment- system and procedure

System componentssoftware: Colosseum3D (OSG, Vortex – AgX Multiphysics, lua,…)hardware: full simulator environment (screen projection,

authentic chair and joysticks, motion platform) or portable case, convential multicore PC

models: data from real forest stands in 3D terrain, vehicle = 20 rigid bodies coupled by kinemtaic constraints (wheel suspension, crane joints,…)

vehicle automation and HMI module: vehicle control, automation, sensor, 3D-map engine and HMI interface

The application requires advanced real-time physics: terramechanics, stacking, hydraulics,…

Experiments in simulator environment- system and procedure

Experiment procedureTask: do harvest thinning in a given dense forest stand

Variations:- forest stand (distribution, species, topology)- harvester head (single, multi, sword)- vehicle (existing machines, new proposals)- automation and HMI (manual, semi-automatic, fully auto)- operator

Measurements: - time per biomass unit in kg (strip road, corridor, tree, move to pile, positioning, transport)- energy consumption- work environment

Optimize: find optimal mechine design and work method – data from simulator experiments used in logistics computation