Semi-Automated Crow Detection System

Ricky Chan

Aaron Gupta

Michael Ma

Donny Sun

Agenda

- Introduction, Background and Project Description

- Hardware Assembly

- GUI (Graphical User Interface)

- Video Transmission

- Image Processing

- Testing and Results

- Future Work

- Difficulties and What We Have Learned

Introduction

The University of Washington, Bothell, Biology

Department is studying the crows and plants of the

North Creek wetlands.

Professor Doug Wacker is interested in studying the

roosting patterns of the crows.

Our project is integrating the quadcopter, which is

made by the mechanical engineering team, to help

Professor Doug Wacker to take images of crows.

Background

From fall to late spring, over 10,000 crows roost in

the North Creek wetlands at dusk.

Professor Wacker’s research includes the following

focuses:

(1) the number of crows that roost in the dusk;

(2) whether spatial patterns exist among individual

crow roosting locations; and

(3) how, if at all, the spatial roosting patterns relate

to the location of nearby plants.

Project Description

Video Transmission

Taking video stream from the quadcopter and Pi camera

Transferring video stream to ground station

Ground Station Computer and GUI

Run a script, download and save the video stream

User can capture images while video is playing

Image Processing

Run a MATLAB script to process the capture image

Hardware Assembly

WiFi Transmitter

Raspberry Pi Monitor ModuleRaspberry Pi ModuleRaspberry Pi NoIR Camera V2

WiFi Receiver

GUI

Language used: Python

Libraries used: PyGame & Tkinter (TeaKay Interface)

Platform: Unix/Linux

GUI

Quits program

Retrieves video from raspberry pi, converts into a usable format and opens video file

Opens video inside gui

Alternates playing and pausing current video

Skips 5 seconds ahead in video

Stops current video playing

Saves current frame as JPEG and processes it

Skips to a specific time frame

Captures and saves the images corresponding to the button clicks

}

Dataflow Diagram

Video Transmission: System

Comprises of :- 2 dual band 2.4Ghz/5Ghz Wifi Adapters- 2 Raspberry Pi’s - Raspberry Pi Camera- USB

Utilizes:- Open source project - wifibroadcast

Inner workings:- Monitor mode & packet injection

R.Pi

Drone

Camera

Wifi adapter

R.Pi

Wifi adapter

USB R.Pi. Screen

live stream

Image Processing: Introduction

Original Image Processed Image

Image Processing: System Characteristics

Characteristic Limitation Reasoning

Crow Detection Detects, but doesn’t recognize, crows

Cannot differentiate between crow and bird-shaped objects

Sports Field is clear of debris; only crows roost on Sports Field

Background LocationUWB Sports Field Doesn’t work in North Creek

woods

Crows roost on Sports Field & S.F. has relatively low noise

Object Distance 5 - 15m Not guaranteed to work outside of this range

Meets contract specifications

Time of Day Dusk Untested at night Tested w/o infrared lights

Camera Perspective Top-down Top-down only Reduces noise, gimbal shape

Image Processing: Active Contour

- General theory

- Main Benefit

- Autonomous & adaptive method

- Drawbacks

- Minute features ignored

- Needs adjustment to increase accuracy

Image Processing: Flowchart

Image Processing: Example Case

1. Load captured screenshot 2. “Grayscale” image

Image Processing: Example Case (con’t.)

3. Adjust image contrast 4. Threshold image

Image Processing: Example Case (con’t.)

5. Trim image 6. Generate mask

Image Processing: Example Case (con’t.)

7. Apply mask to find crows 8. Determine background

Image Processing: Example Case (con’t.)

9. Subtract background 10. Generate final result

Testing/ResultsRange test for video transmission

1. 5 dBi antenna

Distance from soccer field through trees ~183m

Distance from soccer field through short foliage ~203m

2. 9 dBi antenna

Distance from soccer field through trees 237m~

Distance from soccer field through short foliage ~291m

Testing/Results (con’t.)

- Image tested : 13

- Failed Detection(s) : Image #8

- Total crows in 12 images: 57

- Total crows counted : 66

- FP : 66 - 57 = 9, TP: 57, FN = 0, TN = 0

- ≅ 86%

Image 1 2 3 4 5 6 7 8 9 10 11 12 13

# of crows

6 6 4 4 5 6 4 4 6 6 2 4 4

#s counted

6 6 4 6 5 6 6 30 8 5 3 6 5

Future Improvements

- Quadcopter

○ Reduce the operating noise of quadcopter

- Camera

○ Obtain and use high-resolution thermal vision camera

- GUI

○ Add more features (such as playback bar to control the video)

- Image Processing

○ Adapting machine learning algorithm to do the pattern recognition of crows

- Video transmission

○ Change system to 2.4GHz bandwidth to increase range and reliability

What We Have Learned

- The difficulties of working as a team

- Importance of self-motivation → Conducting individual research

- The Value of:

- Good communication skills

- Periodic re-evaluation of the project

Project Difficulties

- Broken quadcopter

- Evolution of project scope

- Hardware limitations

- Crows refused to cooperate

- Coordinating with external help

Thank You!

Questions?