Infrared Thermal Camera-Based Real-Time Identification and Tracking of Large Animals

to Prevent Animal-Vehicle Crashes(AVCs) On Roadways

Debao Zhou

Department of MIE, UMDApril 18, 2023

Objective

To build a system on roadside to automatically detect the presence of large animals, such as deer, and track their motions for the safe driving on roadways

This system can also be installed on cars

Heated objects

Cars at low/high speed

Animals on road

Animals off roadHeated objects

Animals off road

The Problem – Where Are Animals

The Problem – Road Conditions

4

Tree and/or grass, Straight or curved Hill or ditch

Tree

Grass

RoadTree

TreeTree

Road

Hills

Road

or ditches

Case 1 Case 2

Road

Case 3

Case 4

Grass

Thermal Camera Capability Temperature sensitivity: up to 14 mK Optical resolution up to 640 by 480 TV Lines Activity detection of adult human size objects

up to 8 Km with motorized focal length adjustment

Active IR thermal camera and passive IR thermal camera

Example: Thermal-Eye 3620AS camera core:

FOV 17 Range to detect human activity: 1000ft (305m) Real-time 30fps -20°C to 85°C $3500

Technical Approach Through the processing of infrared thermal images to

detect and track big animals Speed method will be used to distinguish big animals from

their environment and high speed cars Pattern matching method will be used to identify big animals

from low speed cars Centroid method will be used to track the motion of big

animals

A mechatronic system with two degrees of freedom will be developed to realize the scanning and tracking by making the camera point to a specific animal 7

Advantages

8

Advantages • Automatic detection and tracking

• All-time – day and night

• Coverage area much bigger than break-the-beam method

Usage:• On-vehicle

• Roadside for infrastructure

Accomplishments To Date Infrared thermal images have been acquired and

analyzed. The results shows that these images can be used to identify the presence of big animals and cars.

Pattern matching method has been applied to identify the presence of cars and deer. The algorithm is capable to tell the object is a deer or a car.

Tracking has been realized using two degrees of freedom system.

A prototype has been built and the scanning and tracking have been realized.

9

System Prototype

10

• Camera and image grabbing

• Stepper motors• Enclosure:

Contain power supply and motion controller

• Computer

Identification and Tracking Realization

11

Image acquisition

Image processing

Motion control

Motion generation

Position measurement

Pixel to step adjustment

+

Image center

Three main phases: Image acquisition Image processing Motion control

Motion Control and Realization

Motion realization

Object Identification and Tracking

Through Centroid and its changing locations:

Pattern Matching: Car and Big-animals

Original Patterns

Original Images

Object Identification

Matched pattern Processed Images

Object Identification and Tracking in Light

16

Three cameras are used: #1: Thermal camera to take the thermal image and put on computer screen #2: Normal digital camera to take video of the whole scenario (video on left) #3: Normal digital camera to take video of the computer screen (video on right)

In light

17

Object Identification and Tracking in Dark

Three cameras are used: #1: Thermal camera to take the thermal image and put on computer screen #2: Normal digital camera to take video of the whole scenario (video on left) #3: Normal digital camera to take video of the computer screen (video on right)

In dark, no lights

Proposed System Setup

Two IR cameras

Heated objects

Cars at low/high speed

Animals on road

Animals off roadHeated objects

Animals off road

Proposed Study Speed measurement:

3D distance measurement and distance change with time Off-road big-animal identification from stand-still objects On-road high-speed vehicles identification

Development of pattern matching algorithms On-road low speed vehicles identification - tell cars from

any big animals when car moves in slow speed Multiple objects tracking Building robust system: electrical, mechanical, suitable

for roadside application, low-high temperature, etc.19

Thanks!

20

Review: On-vehicle / Roadside Natural habitat prevention Roadside vegetation control Electronic mats Population reduction Fencing Deer guards and crosswalks Underpasses and overpasses Roadside reflectors Animal reflectors Flash light sensing IR LASER

24

The Driver Whistles TH-HID Lighting System Forward-collision Sensors Ultrasonic Sensors Radar Camera Auto-identification of humans

in the environment,

Animal Detection System Installed Worldwide: Area-cover systems and Break-the-beam systemsLocation Distance Cost Date(s)7 locations Switzerland 50-200m $11,500 1993Box Uusimaa Finland 220 m $100,000 1996Mikkeli Finland 90 m $70,000 19992 locations in Germany Unknown $20,000 19982 locations Netherlands 200-250 m / $50,000 1999

Rosvik Sweden 100 m / $30,000 1999Colville WA USA 402 m $ 12,000 2000-2002Nugget Canyon WY USA 92 m $200,000 Dec 2000 - May 2001Sequim WA USA 400 m / $73,000 Apr 2000 - present

Marshall MN USA 200 m / $57,000 Jun 2001- Nov 2001Kootenay NP BC CAN 1000 m Unknown Jun 2002- Oct 2003Indiana Toll Road IN USA 1609 m $1,300,000 Apr 2002 Wenatchee WA USA 213 m $40,000 Oct 2002- spring 2004Yellowstone NP MT USA 1609 m $409,000 Oct/Nov 2002

Los Alamos NM USA 30 m $2,500 Nov 2002 - Feb 2003Thompson-town PA USA 91 m / $220,000 May 2004 - Jan 2005Herbertville Quebec CAN 10 m / $8,200 Fall 2004

IR System and Warning Signs in Netherlands

System in Calville, WA, USA – Break-the-beam System

20 June 2000 Hwy 395 north of Spokane,

south of Colville Two lasers, one placed on

each side of the road Two standard deer warning

signs, two smaller rectangular signs that read “When Flashing”, and two solar powered red flashing beacons

Lasers operated on batteries Problems:

Clear line-of-sight in the right-of-way

Heat in the box

FLASH System in Nugget Canyon WY USA, IR Sensor (17m-19m)

Flashing Light Animal Sensing Host (FLASH)

Series of infrared sensors placed at 17-19 m (18.5-20.7 yd) intervals, HW 30; spanned 92m

Body heat of large animals

Signs were placed 300 m (327 yd) before the crossing area

Sing read:

System in Yellowstone National Park, Montana USA (Break-the-beam)

Oct./Nov. 2002; Along a 1,609 m (1 mi) long road section of

US Hwy 191 Transmitter: microwave RF signal (35.5

GHz) to its intended Receiver Once actived, master station then sends the

beacon-on command to the nearest beacons

System in Yellowstone National Park, Montana USA (Break-the-beam)

• Cannot detect the distance/speed of a moving object• Cannot distinguish moving animals from moving cars• Cannot distinguish sun-heated objects from big-

animals • Cannot be able to tell that a fast running animal will

enter the road.

• Intrusion detection rate is not acceptable• Camrix A.L.E.R.T and Xtralis ADPRO

Infrared Cameras for Monitoring Animal Activity Thermacam B1; FLIR (forward-looking infrared) Systems

Inc; Boston, Massachusetts Trailmaster 1500 Active Infrared Trail Monitors; By Goodson

and Associates, Inc.; Lenexa, Kansas FLIR Nightsight Palm IR 250 Digital Camera; Raytheon,

Waltham, MA TM-2500 Trailmonitor; Grypon Engineering, Inc.; Richmond,

Michigan Su320ktx-1.7rt; Goodrich Corporation; Princeton, New

Jersey Hunting Electronics / Military Usage

36

Capability of Infrared Cameras

Distance, from up to 8 Km for the identification of adult human size objects

Temperature: -40F to 100F Resolution: 0.1F

Component Integration

38

Object Identification

Processed image Original image