www.ict.csiro.au

Elliot Duff

Mining Robotics Project

Autonomous Systems Laboratory

CSIRO ICT Centre QCAT Brisbane

Mine Integration Plan

www.ict.csiro.auProjects, Past Present and Future

Sensor Networksfor Coal Prep plants

ACARP C11009 (03-04)

Open Cut Automation

ACARP C11054

Traffic ManagementTruck path planning

ACARP C13041 (2004-05)

Excavator GuidanceLaser based measurement

ACARP C14043 (2005-06)

Dragline Swing AssistAlgorithm Refinement

ACARP C13040 (04-05)

Digital Terrain MappingCalibration and IntegrationACARP C13034 (2004-05)

Rope Shovel AutomationAuto digging on scale modelACARP C12030 (2003-04)

Trayscan Truck Load Volume

Transcale ((2000-2004)

Automated LHD'sCommercial Prototype

AMIRA/Caterpillar (99-03)

Automated Explosive LoadingProof-of-Concept

ORICA (2000-01)

Shovel Dipper VolumeLaser Scanning Feasibility

BHP (1998)

Automated Haulage GPS and INS guidanceKomatsu & Caterpillar

Dragline Swing AssistFeasibility demonstrationACARP C5003 (1996-98)

Digital Terrain MappingProof-of Concept

ACARP C10039 (2001-02)

Dragline Swing AssistACIRL Scale Model

ACARP ME3 (1994-95)

Dragline Swing AssistProduction field trials

ACARP C9028 (2001-02)

Automated Explosive LoadingCommercial Prototype

ORICA (2005-)

Rope Shovel Automation On production shovel

ACARP proposal

Shovel Force FeedbackBased upon DTM and performance

ACARP proposal

Boom Monitoring Wireless sensor network

ACARP

Automated DraglineIntegration of DSA with DTM

ACARP proposal

Wireless Sensor Network for Mine Environment

ACARP proposal

ELAPPrototype

ORICA (2002)

Automated Explosive LoadingFeasibility Study - Vision and Laser

Scanning ORICA (1998-99)

Rope Shovel AutomationAuto loading / face mappingACARP C14058 (2005-06)

Surface MiningCompleted

Commercialized

Current

Proposed

HMC AutomationAuto TrammingAlcoa (2005-06)

Software developmentRTX & DDX

Hardware developmentIMU & IMU

www.ict.csiro.auDragline Swing Assist (DSA)

• Since 1994 we have been automating a large part (80%) of a dragline’s cycle (swing, dump and return).

• We have installed two systems on production machines

• In 2003 we ran a two-week production trial on the BE1350

• This work has been supported by five ACARP projects

Optimized swing path

www.ict.csiro.auDragline Digital Terrain Maps (DTM)Dragline Digital Terrain Maps (DTM)

We have installed a protype system on a BE1350 dragline at Boundary Hill (Callide)

It assists the operator in excavating to specific bench heights with increased precision and confidence.

It displays a high resolution image of spoil pile and the surrounding terrain.

DTM can also guides the operator in correct movement and positioning of the dragline tub.

This work has been supported by two ACARP projects

www.ict.csiro.au

DTM Masterplan DTM Masterplan OverviewOverview

Images include cross-section, plan view or 3D.

Operator prompts include a tub positioning reference and critical dig lines

The elevation image is shown. Plan package will also utilise colours and effects to highlight variations between the dig and actual plan

www.ict.csiro.auDTM – Direct Operator Benefits…DTM – Direct Operator Benefits…

It visually prompts the operator where to dig and dump overburden as per the dig plan.

Masterplan clearly and immediately highlights variations between the actual dig and the planned excavation.

Greatly aids in reducing excessive rehandle. It assists the operator in excavating to specific

bench heights with increased precision and confidence.

It displays a high resolution image of spoil pile and the surrounding terrain.

DTM can also guides the operator in correct movement and positioning of the dragline tub.

www.ict.csiro.auRope Shovel Automation (RSA)

Automation of these machines could lead to significant productivity improvements

In 2003/04 we demonstrated autonomous digging on a scale model (1/7 th) machine (this work was supported by ACARP)

We are now running a second ACARP project to further progress the system and autonomously dump into trucks (scale model)

www.ict.csiro.auShovel Traffic Control

A combination of sensors will:• Generate a DTM of the

local environment• Enable the swing loader to

locate each haul truck as it approaches

• Provide communication from the swing loader to each haul truck

Develop a system which schedules and directs haul truck drivers to an optimum loading position for a swing loader.

Develop a system which schedules and directs haul truck drivers to an optimum loading position for a swing loader.

Handover Zone

Work Zone

Incoming Trucks

Outgoing Trucks

Bench

Obstacle

www.ict.csiro.auExcavator Guidance and Measurement

Laser scan of terrain and bucket location

www.ict.csiro.auTruck Volume Measurement (Trayscan) Truck Volume Measurement (Trayscan)

Trayscan is an automated system that measures the volume of material in the tray of haul-trucks using laser-scanning technology. This technology provides the ability to measure the volume in-situ, whilst the truck is moving. Commercialised through Transcale.

A 3D profile is generated, as the haul truck drives under scanning laser.

The in-situ volume is estimated from a comparison of empty and full trucks.

www.ict.csiro.auLoad Haul Dump (LHD) Automation

Demonstrated full speed operation at NorthParkes Mine

System based upon reactive navigation requiring no guidance infrastructure

Navigation technology licensed to DAS (CEPL) with commercial system launched April 2003

Pre-production prototypes developed for a three type of LHD including electric machines

Automated CSIRO loader tested at Northparkes mine

Automated Tamrock electric LHD from Northparkes at QCAT

www.ict.csiro.auAutomation of Explosive Charging (AOC)

We have developed a robotic charging unit to automatically charge blast holes in underground metal mines (ORICA)

A scanning laser is used to create a 3D map in order to locate a ring of blast holes

A vision system is used to guide a hose into the collar of each hole

www.ict.csiro.auAutonomous Hot Metal Carrier

• 30 tonne vehicle for navigation sensor

• moving molten aluminium

• Demonstrate full automation within the next 6 months

• The vehicle is based on a large fork-lift truck

• Will use laser scanners as the primary

www.ict.csiro.auWho are we ?

Team Complement• 7.5 Research staff • 4 Engineering support

– 2 engineers– 2 technicans

• 1 BDM (theme resource)• 1 Executive officer (theme

resource)• 2 Postdocs• 4 Co-supervised PhD

students• Visitors (current)

– 1 IAESTE intern– 5 undergraduate students

Disciplines/Skills

• Electrical engineering

• Mechanical engineering

• Computer Science

• Aerospace engineering

• Physics & Applied maths

www.ict.csiro.auWhat we do?

The intelligent connection of perception to action The intelligent connection of perception to action

Robot navigation

Relative & reactive navigation

Visual homing

Visual servoing

Multiple cameras with sensor fusion

Ranging and measurement

Laser imaging

3D imaging

Omni directional vision Stereo vision

Agent based systems

Low-cost distributed sensor systems Ad-hoc wireless networks

Real time computing

Distributed systems

Real time libraries

Embedded systems

High-performance control systems

Hydraulic

Electro mechanical

www.ict.csiro.auTeam Project Activities

External focus Mining Robots

• Unique issues facing mining sector:

• Dirty, dusty, wet and hot

• Constantly changing environment (the aim of mining!)

Strategic Focus Competent vision-based navigation and control,

• tested on our platforms: ground, air, underwater

Wireless sensor and actuator networks• Robot interaction & Environmental monitoring

Deliver prototypes of complex engineered systems for non-traditional robotics applications (field/outdoor robotics)Deliver prototypes of complex engineered systems for non-traditional robotics applications (field/outdoor robotics)

www.ict.csiro.auAutonomous Underwater Vehicles (AUV)

Starbug: Sea trials in Moreton Bay

www.ict.csiro.auAutonomous Land Vehicle

• Experimental platform for land based navigation

• Based on a ride-on lawnmower

• Equipped with vision, laser scanner and magnetic compass

• All computing on-board

www.ict.csiro.auAutonomous Aerial Vehicles

Mark II – Mantis (2003)

Mark I (1999-2002)

• Experimental platform for testing CSIRO autonomous flight control system

• Equipped with stereo vision, low-cost inertial sensors, magnetic compass and standard GPS

• All computing on-board• We have developed a vision-based velocity

and height estimation system

• Currently investigating asset management

and search support opportunities.

www.ict.csiro.auCable-Array Robot

• A set of spatially distributed motors which actuate cables connected to an end effecter

– Similar to a Stewart platform

• By changing the length of the cables, the end effecter can be moved

• Examples include:– SkyCam – used in football

stadiums for giving an overhead view

– NIST robot crane

– Quay cranes

Schematic view of a cable-array robot

end effecter

actuator

cables

www.ict.csiro.auRobotics & Sensor Networks

Old thinking New thinking

Robot carries/controls small number of expensive sensors

Sensors are everywhere, robot asks for information

Scientists go into field to gather data

Robot goes into field to “harvest” data from sensors

Robot goes into field to gather data itself

Monolithic complex robot

Multiple cooperating ‘physical’ agents

www.ict.csiro.au

ADC/DIO Power Solar

Programmer/Serial port

Connectors/Mounting holes

AdHoc Radio Networks

Fleck 1 Temperature ADC DIO Serial

www.ict.csiro.auAdHoc Radio Networks

Fleck2 temperature motion compass GPS DIO/ADC SD/MMC Video

www.ict.csiro.auEiMU

• Three solid-state gyros

• Two dual-axis accelerometers

• Three Honeywell magnetometers

• HC12 based micro-controller

• RS232 and Canbus outputs

• Small: 50x50x50mm

• Lightweight: 65g

• Complementary filters for fusion

www.ict.csiro.auNavigation Technology

• Laser Sensing– Reactive Navigation– Opportunistic Localization

• Visual Sensing– Optical Flow (speed) – Stereo (range / height)

www.ict.csiro.auNavigation Technology

Odometry: vision vs groundtruth

www.ict.csiro.auDDX: Distributed Data eXchange

Local store

CP

C

C

CP P

Local store

internet

Log file

• Real-time publish and subscribe model• Shared memory within machine• Time stamping• Event handling & synchronization• Multicast between machines

CPU2CPU1