Project14 Final Paper

7/30/2019 Project14 Final Paper

1/23

HEAT SEEKING FIRE EXTINGUISHER

By

Frisco Sembel

Tony Winoto

ECE 445, SENIOR DESIGN PROJECT

FALL 2011

TA: Jane Tu

December 7, 2011

Project No. 14


2/23


3/23

ii

TABLE OF CONTENTS

1. INTRODUCTION ............................................................................................................................................. 1

1.1 Purposes ....................................................................................................................................................... 1

1.2 Features and Benefits .................................................................................................................................. 2

1.3 Specification ................................................................................................................................................. 2

2. DESIGN PROCEDURE .................................................................................................................................... 3

2.1 Pulse Width Modulation .............................................................................................................................. 3

2.2 Vision System ............................................................................................................................................... 3

3. DESIGN DETAILS .......................................................................................................................................... 5

3.1 Beagleboard (Microprocessor) and Operating System ................................................................................ 5

3.2 Panning and Tilting Mechanism .................................................................................................................. 5

3.3 Extinguisher................................................................................................................................................. 6

3.4 Alarm ........................................................................................................................................................... 7

3.5 Voltage Converter Circuit ............................................................................................................................ 7

3.6 IR Camera .................................................................................................................................................... 8

3.7 Power Supply ............................................................................................................................................... 9

3.8 Vision System .............................................................................................................................................. 9

3.9 Main Control .............................................................................................................................................. 12

4. DESIGN VERIFICATION .............................................................................................................................. 13

4.1 Beagleboard (Microprocessor)................................................................................................................... 13

4.2 Panning and Tilting Mechanism ................................................................................................................ 13

4.3 Extinguisher and Alarm ............................................................................................................................. 13

4.4 Vision System ............................................................................................................................................ 14

5. ETHICAL ISSUE ............................................................................................................................................ 15

6. COST ............................................................................................................................................................... 16

7. CONCLUSIONS.............................................................................................................................................. 17

7.1 Accomplishments....................................................................................................................................... 17

7.2 Challenges.................................................................................................................................................. 17

7. 3 Recommendations ..................................................................................................................................... 17

9. REFERENCES .................................................................................................................................................... 18

APPENDIX A ............................................................................................................................................................. A.1

APPENDIX B ............................................................................................................................................................. A.2


4/23

1

1. INTRODUCTIONThe main goal of this project was to build an extinguisher that can detect fire or moving heat source in

this case using webcam. Since we are not allowed to use any form of fire, we will use light bulb to testour project. We got the idea about this project from damages caused by sprinkler in event of false alarm

or just a small source of fire. Common sprinkler will spray water all over the area and will cause great

damage to electronics. Since our system will only apply extinguisher on the source of fire/heat, the

property damages caused by extinguisher are expected to be greatly reduced.

The project is mainly divided into two crucial subprojects which are vision system and pulse width

modulation (PWM) for servo. After these two subprojects are completed, there will another subproject

that will bring these two subprojects together as one. The PWM subproject involves connection between

BeagleBoard and the servos which is linked through voltage converter(shifter). On the other hand, the

vision system involves BeagleBoard and the IR camera. Both the vision system and pulse widthmodulation are software based, so in order to bring them together we will use them as functions that can

be called in the BeagleBoard. The illustration how we connect our device is shown on Fig. 1 below.

Fig. 1. Block diagram

1.1 Purposes

The purpose of our project was to develop an extinguisher that is equipped with a modified webcam andservos. The modified extinguisher will be able to track and pinpoint the source of fire/heat and turn on

the LED to indicate the triggered extinguisher. The main goal of this project is to minimize the damages

caused by common sprinkler system. If our device does not detect the fire/heat in the image from the

webcam, it will continue to scan until the whole area is covered.


5/23

2

1.2 Features and Benefits

Features

Portable fire extinguisher system. Fast fire detection Low power consumption Modular systemBenefits

Extinguishes fire source right on the source Reduce damage cost in case of fire

1.3 SpecificationsThe device is able to locate heat sources and differentiate heat emitting objects and real fire sources. The

device also extinguishes the fire sources based on size of the fire sources from the largest to the smallest

The fire detection algorithm is also able to detect a fire source in less than 50 seconds for each panning

angle. It should also be able to withstand the operating temperature between 0-85oC1.

1Based on the operating temperature of Beagleboard. Refers to Beagleboard-xM Reference Manual on the reference page


6/23

3

2. DESIGN PROCEDURESThis project consists of two major subprojects which are pulse width modulation system and vision

system. Both subprojects are software based projects which are coded in c++ format. We also used

OpenCV library functions to access some features such as detecting contour edges and calculatingcontour areas. The pulse width modulation subproject involves understanding of servos such as duty

cycle at which servos should stop moving and time required to do full rotation. Both subprojects will be

connected using control algorithm which calls both subprojects as functions.

2.1 Pulse Width Modulation

For the pulse width modulation there are two ways to access and generate signal from the PWM pins

from the BeagleBoard: first is by a direct access to the OMAPs processor general-purpose timer

mechanism and second is by using a servo driver for angstrom distribution.

The first choice required interaction with the timer register in Linux kernel module which we do not

have the knowledge of. Thus, we used the other method which helps us a little by providing us the

driver2 to interact with the timer register. This method still took us sometimes to get it up and running

To get the driver work, we need to compile other codes using OpenEmbedded and Bitbake 3 to get

executable file from a Linux host computer. Afterwards, the file is then copied to the boagleboard to get

access to the PWM.

The servos used for our device are HS7954SH which were chosen based on the amount of torque

generated. Each servo can generate 20 kg.cm torques at 4.8 Volt which is strong enough to operate our

device. The torque of the servos is the most critical part of the system, since there will be high backward

momentum from the high pressure extinguisher. The servos also have the capability to turn 360o, this is

necessary for angle turning.

2.2 Vision System

For the vision system, we modified a normal webcam's filter by adding additional negative film on the

filter. We use a normal webcam since commercial thermal cameras are relatively really expensive and

we don't think it is necessary to use such expensive and detail thermal cameras which are used in

manufacturing or monitoring company which require high precision temperature measurement. We alsochose Logitech webcam which is supported by Angstrom distribution on the BeagleBoard since some

other webcams are not working on Angstrom.

2The driver are available as open source package from Scott Ellis. For more information, refer Scott Ellis on the reference

page.3

Cross-compile environment. Facilitate developers to create a complete Linux distribution for embedded system.


7/23

4

Our other consideration beside cost is the fire detection algorithm memory efficiency. Initially, we used

haarcascade technique for our fire detection algorithm. However, this technique has many disadvantages

for our device. It requires a lot of sample images to get accurate detection of an object. The minimum

sample images to get at least 70% detection is around 6000 samples. The samples will be compiled into

xml file that will be used in main control code. Since the file consists of many samples, it requires a

significant amount of memory to run. Thus, it is really slow to run this technique on BeagleBoard which

only has 512 Mb RAM.

Finally, we decided to use moving contour detection technique for our detection algorithm. It basically

works by comparing contour location of current and previous frames. The process is continuous until the

device returns to its original position. This technique is really suitable for our device. It runs faster than

the haarcascade technique on BeagleBoard. Furthermore, this technique is also easier to implement since

it does not require additional file that consists of sample images.


8/23

5

3. DESIGN DETAILS

3.1 Beagleboard (Microprocessor) and Operating System

The main control and process of our project rely heavily on the microprocessor on the system. Since the

project involves real time image streaming and processing, we decided to use Beagleboard-xM as our

microprocessor, considering time it takes to do the image processing algorithm. Beagleboard-xM is alow cost, low powered single board computer produce by Texas Instrument. It runs on OMAP3530

system with ARM-Cortex-A8 CPU. It delivers 1 GHz clock speed with 512 RAM. The board also

comes with various peripheral connections such as USB port, Serial port, DVI-D, expansion ports and

many more. Those various types on peripheral over a large range of interfaces that can be connected to

our modules, infrared camera, servos and I/O signal from and to extinguisher and alarm system.

Beagleboard also has a large and active community that will come in handy when stuck with problems.

Because Beagleboard is a single board computer, it needs an operating system that will run the board

We choose Linux Angstrom Distribution because there are more information available on how to build

custom angstrom distribution and access the ports on the board. This allows us to choose only thenecessary package to be included on the operating system and that increase the performance of the

board. A particular important package needed is OpenCv.

OpenCv is a library of programming function for real time computer vision. This helps us simplify our

image processing algorithm by allowing us to call some necessary functions to process the real time

images from the IR camera. Such functions are capturing images, extract images per frame, save images

and background extraction, etc.

3.2 Panning and Tilting Mechanism

The mechanism used for controlling the panning and tilting of the simple robotic arm is done using the

expansion header on the Beagleboard. The OMAP35304 system offers four PWM pins that can be

accessed trough available headers. For Beagleboard-xM, only three of the pins are available for PWM

connection on expansion port, they are port 4 - GPT9_PWMEVT, port 10 - GPT11_PWMEVT and por

6GPT10_PWMEVT. There are two ways to access and generate signal from the PWM pins: first is by

a direct access to the OMAPs processor general-purpose timer mechanism and second is by using a

servo driver for angstrom distribution.

The first choice required interaction with the timer register in Linux kernel module which we do not

have the knowledge. The other alternative helps us a little by providing us the driver5 to interact with the

timer register. This method still took us sometime to get it up and running. To get the driver work, we

need to compile the codes using OpenEmbedded and Bitbake6 to get the .ko executable file from a

4Texas instrument general purpose f ARM architecture processor.

5The driver are available as open source package from Scott Ellis. For more information, refer Scott Ellis on the reference

page.6

Cross-compile environment. Facilitate developers to create a complete Linux distribution for embedded system.


9/23

6

Linux host computer. The file is then copied to the boagleboard and run to get access to the PWM.

There is another thing to do before the PWM works. By default the kernel 7 version that is used on our

system enable the kernel config option CONFIG_OMAP_RESET_CLOCKS. This is a default kernel

config that help save power consumption on Linux. In order to the get the PWM running, the option

needs to be disabled. It is done by modifying the deconfig file inside the OpenEmbeded recipes folder

and recompiling the kernel.

The servos used for the pan and tilt mechanism are HS7954SH. HS7954SH were chosen based on the

amount of torque generated. Each servo generates 20 kg.cm torques at 4.8 Volt. The torque of the servos

is a critical part of the system, since the there will be high feedback force generated by the high pressure

extinguisher. The servos also have the capability to turn 360o, this is necessary of panning.

Since these are continuous servos, to control the angle produced, we need to keep the duty cycle of the

PWM signal constant, for constant angular speed, and time how long does it take for the servo to do a

full 360o rotation at current duty cycle. The angular speed can be adjusted accordingly. The time is then

used to calculate how long it takes for the servo to do one step rotation (capture angle of the IR camera).

Our measurement showed that our camera is able to view ~37 degrees for panning and ~30 degree for

tilting. For the complete view of the entire area, the camera needs to take 360 degrees view which means

the camera angle will be turned 10 times until all area is covered.

3.3 Extinguisher

The extinguisher is simply a fire extinguisher tube with some modification on the release mechanism.

The release mechanism works just like an automatic restroom faucet. When release signal is high, i

triggers the valve on the tip of the fire extinguisher tubing. One thing to note is that for the purpose of

demo, we are not going to use a real fire extinguisher; instead we are going to use LED to indicate the

trigger signal is high or low.

Compared to PWM, accessing the general-purpose input-output (GPIO) pin on the board is much less

complicated. The GPIO pins are also available through some of the ports on the expansion header

There is no need for driver. The GPIO can be access using sysfs 8 from C code. The signal for the

extinguisher is set to port 17GPIO 132 of the expansion port.

7Kernel is a bridge between applications and the actual date processing done at the hardware level.

8Virtual file system. Exports information about devices and drivers from the kernel device model to user space (Source :

Wikipedia)


10/23

7

3.4 Alarm

Alarm system is what starts all the process of fire detection. When the alarm goes off, it sends signal to

the control system to start look for fire sources. The control then sends a feedback signal to turn off the

alarm signal. All these are done using the GPIO capability of the board. Unlike the extinguisher who

only needs an output GPIO direction, the alarm system needs input and output GPIO direction. For the

purpose of demo, the input signal from the alarm signal is replaced with a push switch and the output of

the alarm signal is replaced with a LED. When being pushed, the switch shorts port 9GPIO136 on the

expansion port to the ground. The output is showed through the LED that is connected to port 19

GPIO 131 on the expansion board.

3.5 Voltage Converter Circuit

Unfortunately, beagleboard only output maximum 1.8 V on its expansion port, this includes PWM and

GPIO signals. So, in order to accommodate the required pulse signal for the servos, which is greater than

4.8V, we need to have a voltage converter circuit that will elevate output voltage from 1.8V to 4.8Vwithout altering the original PWM signal from the beagleboard. We built the circuit using a general

purpose low-power NPN bipolar junction transistor, 2N2222 and some 1000 Ohm resistors. Fig. 2 and

Fig. 3 show the schematic and the simulation of the voltage converter circuit. The simulation shows

three values, the 1.8V output from the beagleboard, the amplified but inverted signal from the left

transistor and the amplified signals from the transistor on the right. It can be seen that by using this

voltage converter circuit will satisfies the input pulse requirement for the servos.

Fig. 2. Voltage Converter Circuit (1.8V-5V)


11/23

8

Fig. 3. Voltage Converter Simulation

3.6 IR Camera

For the IR camera, we modified a regular Logitech webcam. Originally the Logitech webcam capture

RGB( Red, Green, Blue) spectrum and combine them to get color images. However, we want to

maximize the fire detection using the infrared camera. Thus, the filter inside the Logitech webcam is

removed and replaced with a black photographic negative which technically will block all other lights

and only allow infrared radiation to enter the camera sensor. This feature helps us to eliminate other

objects that do not radiate infrared from the images taken and maximize the hit rate or accuracy of

fire/heat source detection. The camera will be powered by BeagleBoard and connected to the USB port

on the BeagleBoard. Fig. 4 and 5 shows how we modified the filter inside the webcam by inserting the

photographic negative strip in between the camera sensor and its optical lens.

Figure 4. Logitech Webcam Figure 5. Logitech webcam with

modified filter (circled in red)


12/23

9

3.7 Power Supply

For the project, we are using [email protected] power supply adaptor that connects to the 120V AC voltage.

This adaptor will be connected to the power pins of both beagle-board and the servos. Other than those

the other components are going to be powered by the beagle-board.

3.8 Vision System

The image processing algorithm is coded in c++. We use some OpenCV functions and libraries in the

code to process the images from the camera. The algorithm mainly works based on the threshold values

of images to identify whether the heat/fire source is spotted in the image. We initially used haarcascade

technique to provide the algorithm with threshold comparison. However, this technique did not work

well for detecting fire since fire does not have certain shape. Finally, we decided to use moving contour

detection technique which is much more suitable for our case.

The haarcascade xml file requires three types of samples: positive, negative, and natural samples. The

positive images are the images that only contain the object we want; in our case, the object is heat

source. Since we are detecting fire/heat source in our project, we are just going to use the threshold

values from the heat source (light bulb) for the positive image (Fig. 6). The negative images are images

that contain background images without the object as illustrated in Fig. 7. The natural images are images

that contain both background images and the object shown in Fig. 8. The positive sample will provide

the threshold for the heat source while the negative will give the threshold for the background which

will be dark for most of the images since we are using IR camera.

Fig. 6. Positive Image Fig. 7. Negative Image Fig. 8. Natural Image

The hit rate is quite high (~70%) for the current algorithm and threshold value around 0.5. However, the

test was conduct using light bulb instead of real fire. Since fire has unpredictable pattern, this method of

detection will not be suitable in real life cases. Thus, we change our method of detection into moving

contour detection.

The moving contour detection also uses opencv library which provide functions that return value of

edges of the contour. First of all, the first frame of the video will be converted into black and white


13/23

10

image by applying threshold to original image in Fig. 9. After the image is already in black and white, it

is clearly shown when there is heat source in the image shown in Fig. 10. Then, we connect the edges of

the contour detected illustrated in Fig. 11 and compare the location of contour with the next frame to

detect movement. Moreover, the areas of each fire sources are also stored. These values will be used to

sort the fire source from the largest to the smallest. Thus, our device will extinguish the fire source

according this sorting. This method is used to distinguish the real fire from static heat source.

Fig. 9. Original Image Fig. 10. Black and White Image

Fig. 11. Detected Contour Image


14/23

11

3.8.1 Vision System Flowchart

Fig. 12. Vision System Flowchart


15/23

12

3.9 Main Control

The main control is the main code that will call functions from the fire detection algorithm and the

servos control to get inputs and give outputs for panning, tilting, extinguisher and alarm system. The

control starts out by loading the PWM driver module to the Angstrom operating system. It then waits for

the signal from the smoke detector alarm to be able to proceed to the next state. If the alarm signal is

triggered by the smoked detector, the control will activate servos and the IR camera and start positioning

the whole system to 10 pan angles and 3 tilt angles until it covers all the area below the ceiling of a

room. While at each particular angle position, it will process each frame of the real time video stream

from the IR camera using the image fire detection algorithm described on the vision system. If fire is

detected, then the angle position and the size of the fire source are saved in arrays which will then be

sorted based on the area of the fire sources detected. It will then extinguishes the fire sources based on

the angle position with the largest area of fire sources first then to rest of the fire sources detected. After

all fire sources have been extinguished, the system will go back to halt and wait the alarm signal.

Fig. 13. Main Control Flowchart


16/23

13

4. DESIGN VERIFICATION4.1 Beagleboard (Microprocessor)

Verification for the Beagleboard was more of qualitative processes. It started out by building the image

of the Angstrom-Distribution OS with the necessary libraries or packages and porting it to micro SDHC

card which will then be inserted into the Beagleboard to boot. The booting process could be seendirectly on the monitor that was connected to the boardboard trough DVI connection. Errors during the

booting process of the Operating System will be listed on the monitor. It took us some trials to get jus

the right packages for the OS to run and the OpenCV to compile9.

4.2 Panning and Tilting Mechanism

Testing the panning and tilting mechanism of the simple robotic arm was done by varying the duty cycle

of the PWM signals through the servo driver to get just the right angular speed needed. The

configuration for 50Hz pulse was 20seconds/360o for panning (running at 7.53% CW and 7.4% CCW

duty cycle) and 3seconds/90o for tilting (running at 7.09% CW and 7.31% CCW duty cycle). The neutral

duty cycle was at 7.21% for tilting and 7.45% for panning. To make sure the output duty cycles matchthe input, we connected the pulse generated by the beagleboard to an oscilloscope. The outputs are

attached on the appendix A.

Servo Duty Cycle(PWM pulse signals run at 50Hz)

Neutral % Clockwise % Counter CW %

Panning 7.45 7.53 7.40 (20s)

Tilting 7.21 7.09 (3s) 7.31 (3s)

Table 1. PWM Signal Configurations

4.3 Extinguisher and Alarm

Verifying signals that goes and comes from the alarm system and the extinguisher was done by watching

the signals generated by the Beagleboard GPIO pins using oscilloscope and see if they match the desired

signals. Fig. 14 below shows the signal generated from the GPIO pin that is used to trigger the fire

extinguisher which in this case is an LED. The wave form runs at frequency of 2 kHz with 50% duty

cycle and maximum voltage of 2V.

9We used references from angstrom-distribution.org and http://elinux.org/Category:ECE497.


17/23

14

4.4 Vision System

We used bright LED to test our fire detection algorithm by moving the LED around to imitate real fire

or just put the LED static to illustrate static heat source. The moving contour detection technique has

high detection rate up to 90%. Our device can clearly distinguish the moving LEDs and static LEDs.

However, since our device is rotated around the room and the algorithm is continuously running until al

angles are covered, there might be false detections when our device is rotated to static LEDs which

reduce the detection rate. The average processing time for each angle can be maintained at 1 second

without including time to rotate the device. Since it takes 20 seconds to cover 360 o, the total time spen

at certain angle is around 3 seconds. Thus, the whole time to cover entire areas is around 3 minutes max.

This time calculation is measured without showing the processed images on the screen since putting the

image will slow down the entire system. The average frame rate per second for the camera is also

measured around 17 fps and it is below Nyquist rate of real fire flicker frequency by 3 fps. However, theoverall detection works pretty high.

Fig. 14 Extingusher's output signal when triggered.


18/23

15

5. ETHICAL ISSUE

The original goal is mainly to minimize the property damages caused by sprinkler system. Despite a lot

of effort put into developing the device so that damage is minimized, there is still a possibility that the

device is considered causing damage by some people. In this case, our device can sprays water toelectronics which is currently very near to the heat source or actually is burning. This case related to

IEEE Code of Ethics 9: to avoid injuring others, their property, reputation, or employment by false or

malicious action. At first, we have the robotic arm that moves our system throughout the entire room.

However, because of the complex mechanical system of the robotic arms, we decided just to mount the

device on the ceiling. Therefore, this will avoid injuring people since the device is static on the ceiling.


19/23

16

6. COST

Name Salary($/hr) Hours Total +Extras(2.5)

Frisco

Sembel

$40.00 180(18hr/week) $7,200 $18,000

Tony Winoto $40.00 180(18hr/week) $7,200 $18,000

Total $36,000.00

Part Cost Quantity Total

Aluminum Plate $20.00 2 $40.00

Webcam Camera $25.00 1 $25.00

Infrared Filter $1.00 1 $1.00

HS-7954SH (RoboticServos)

$100.00 2 $200.00

Transistor 2N2222 $0.25 4 $1.00

LED $1.00 2 $2.00

Wires $10 1 $10.00

Switch $1.00 1 $1.00

Circuit board $3.00 1 $3.00

Resistor 1k $0.20 6 $1.20

Mini extinguisher $30.00 1 $30.00

Beagle-Board $150.00 1 $150.00

Power [email protected]

$25.00 1 $25.00

Total $489.20

Labor Cost Parts Cost Grand Total

$36,000.00 $489.20 $36,489.20


20/23

17

7. CONCLUSIONS7.1 Accomplishments

The standalone vision system which consists of fire detection algorithm has successfully detected light

emitting objects and distinguished whether they are fire sources or just light sources in less than 5

seconds which is one of the specifications. The panning and tilting mechanisms respond accurately tothe input commands with various duty cycles. These mechanisms are able to move the extinguisher to

the right position angles that are specified. The system is able to take input from the alarm system

(switch) and output to the LED when the extinguisher is triggered.

7.2 Challenges

There were some challenges we encountered in integrating the whole project. The main control of the

system processes and stores a lot of information when the system runs. Thus, a considerable problem on

the overall processing time arises. On the specification, the system should be able to perform the fire

detection process in less than 50 seconds for each tilt angle or one full panning rotation. In the end, the

result shows slower detection time at around 1 minute for each tilt angle.

Detecting fire sources in a very bright room also raises a problem. Since the camera uses physical filter

(negative photographic film), the camera will see a wider range of objects when the room is too bright

Therefore, even a plain wall will appear as a light emitting object. This will confuse the fire detection

algorithm and result in a bad decision taken by the algorithm.

In order to create a modular system, we did not connect the subsystems permanently. Instead, we made

connectors for each subsystem. During the demonstration, lose connectors between the beagleboard

PWM ports and the servos caused the servos to behave unpredictably. We figured that the lose

connectors caused unexpected spikes on the pulses for controlling the servos.

7. 3 Recommendations

For future works of the project, we need to optimize the control system and the fire detection algorithm

in order to speed up the whole process. Another way to speed up the fire detection process is to take

advantage of the DSP chip on the beagleboard. Additionally, for precise angular control, we will replace

the continuous servos with regular servos.


21/23

18

9. REFERENCES

[1] S. Ellis. (2010, June 8). Omap3-pwm [Online]. Available:https://github.com/scottellis/omap3-pwm

[2] Beagleboard.org. (2010, April 4).BeagleBoard-xM Rev-B System Reference Manual[Online].

Available: http://beagleboard.org/static/BBxMSRM_latest.pdf

[3] How to Install OpenEmbedded and Bitbake [Online]. Available: http://pixhawk.ethz.ch/wiki

/tutorials/omap/openembedded_bitbake_installation

[4] Linux kernel, PWM and GPIO [Online]. Available: http://elinux.org/Category:ECE497

[5] Robotzone, LLC. (2011).HS 7954sh Servo [Online]. Available: http://www.servocity.com /html/hs-

7954sh_servo.html

[6] K. Koen. (2011, March 21). The Angstrom Distribution [Online]. Available: http://www. angstrom -

distribution.org/

[7] V. Pisarevsky. (2011, August 24). OpenCv [Online]. Available: http://opencv.willowgarage.com/wiki/


22/23

A.1

APPENDIX A


23/23

APPENDIX B

Documents

Project14 Final Paper