IMPLEMENTATION OF A PYROELECTRIC INFRARED TRACKING …

1

IMPLEMENTATION OF A PYROELECTRIC INFRARED TRACKING SYSTEM

USING FPGA

Chng Jin Yang Ray

1, Clement Wong Wai Kit

1, Kevin Sim Yeow Kiat

2, Ng Tiong Huat

2,

1NUS High School of Math and Science, 20 Clementi Avenue 1, Singapore 129957

2DSO National Laboratories, 20 Science Park Drive, Singapore 118230

ABSTRACT

Pyroelectric infrared motion sensors provide cost effective solution that can detect the

movement of a person even in the dark by sensing changes in the infrared emission as

compared to its ambience. It may be used in situations such as to detect suspicious activities

at close proximity of a fenced perimeter. The sensor can be coupled with a surveillance

camera that can rotate and point to the direction where the pyroelectric infrared motion sensor

has detected the motion. This allows the camera to cover a bigger field of view and to be able

to zoom in to the video details in desired direction. A prototype of such a sensor camera

close-loop system is developed and demonstrated in this project. The prototype consists of

integration of the pyroelectric infrared sensors, stepper motor, field programmable gate array

(FPGA) prototyping board and firmware. Testing and evaluation were carried out to

determine the effectiveness of our system. Future improvements to the system are also

proposed.

INTRODUCTION

Surveillance cameras are usually placed at the fenced perimeter of the secured region to

monitor for any intrusions or suspicious activities.

The monitoring camera system will often require a wide field of coverage. The conventional

methods to implement video monitoring system with wide field of view (FOV) and their

drawbacks are:

Fisheye Lens Camera

Fisheye lenses often cause image distortion thus affecting image quality. Under certain light

conditions fisheye lenses will suffer from both lens flare and vignette, which will in turn also

affect the image quality.

Using of Multiple Camera to Cover Wide Field of View (FOV)

Using of multiple camera will help to increase the FOV. However, it may be moore costly to

implement this option. A single camera setup will also allow a more efficient video logging

of the source.

The monitoring of intrusions and suspicious activities often requires the security personnel to

constantly monitor the surveillance cameras. Alarm may also be raised when the fenced

parameter has been physically compromised. In more advanced systems, image processing

may be used to automatically alert the security of possible intrusion. However image

processing brings about a higher risk of false alarms during the day due to the changing

environment. Moreover, under low light conditions, image processing may not perform as

well.

2

This project aims to provide a more robust solution to enhance parameter security. This is

achieved by:

Using Pyroelectric Infrared (PIR) sensors to automatically detect motion near the

fenced parameter and send timely alerts to the security personnel. PIR also works in

the dark to provide all day security coverage.

Integration to the security camera to rotate to the direction of the threat. This provides

timely video information to the security personnel for them to visualize the situation.

Using of a camera mounted on a stepper motor, aims to provide more cost effective

system with better quality image while maintaining wide FOV coverage

PROTOTYPE DESIGN

To demonstrate the concept, a system prototype is designed to detect motion using 4

Pyroelectric Infrared (PIR) sensors. When a motion is detected, the camera will rotate to the

direction of the source of motion.

Figure 1: Block Diagram of Security System

Our system comprises of three main components. The first key part of our system will be the

use of PIR sensors. The PIR sensors will detect motion and determine the location of the

source.

The next key part of our system is the use of a camera mounted on a stepper motor that tracks

human motion. The rotating stepper motor will allow the camera to have a greater field of

view (FOV).

Lastly, in order to integrate and interface the infrared sensors, the camera and stepper motor,

we will use a development board with a Field Programmable Gate Array (FPGA). Using

VHSIC Hardware Description Language (VHDL), we designed a software algorithm that

would be able to effectively integrate the different hardware components to create a complete

system. A chassis has also been made to physically integrate and house the different

components.

The sections will be structured as follows: choices of main parts and materials, specific

methods of implementation of both the software and hardware aspects of the system, an

evaluation of our final prototype and finally, areas of future work.

Sensor 1

Sensor 2

Sensor 3

Sensor 4

FPGA Stepper

Motor

3

CHOICES OF MAIN PARTS

PIR Sensor

We will be using the SEN-01049 PIR sensor. They are able to detect infrared radiation in the

form of heat without requiring any emitters to stimulate it [1]. When an infrared source

passes the detection area of the sensor, the sensor will output a digital high. When the source

leaves the area the sensor will output a digital low. A Fresnel lens is located on the top of the

sensor to widen the FOV and to increase the range [1]. The sensor we are using has a FOV of

approximately 110 degrees and has a maximum working range of about 7 metres. A more

detailed explanation of how the sensor works can be found in the Appendix.

Stepper Motor

We will be using the 28BYJ-48 Stepper Motor and the ULN2003 Driver Board. Stepper

motors takes in current pulses and create a discrete rotation of the motor shaft (Grant, M.

2005). We decided to use stepper motors as they are able to move in discrete step increments

ensuring precision. A more detailed explanation of how the stepper motor works can also be

found in the Appendix.

Camera

We have chosen to use the D-Link DCS-942L Wireless Day and Night Network Camera.

This camera has a night vision mode which utilises the built in IR LEDs that allow the

camera to capture video even in low light conditions. It has a FOV of 45.3/34.5/54.9 degrees

(H/V/D).

FPGA Board

We will be using the Nexys4 FPGA board. FPGAs contains vast amounts of reconfigurable

gate array logic circuitry that can be configured to form a hardware application from software

designs. FPGAs takes advantage of parallel hardware processing paths, therefore speeding up

operations and surpassing the processing power of digital signal processors. FPGAs also offer

unlimited adaptability in terms of configurability (Robbins, R. 2010). Therefore, this board

will be used to integrate the sensors and stepper motor.

METHODS

Hardware

Chassis

A compact acrylic chassis was built to house all the necessary components. Four PIR sensors

were used to detect motion, each 140 degrees apart. The four sensors are attached to a panel

located in the front of the chassis. The main body of the chassis contains a stepper motor

driver, an FPGA board, a power bank and a breadboard, while the stepper motor is secured

under the top piece of acrylic. A circular plate is mounted onto the motor shaft, where the

camera is secured onto. Dimensions of the chassis can be found in the Appendix.

4

PIR Sensor

Various in-depth calculations have been done using Microsoft Excel and Graphmatica to

decide on the arrangement of the four PIR sensors. The sensors are arranged to cover a range

of approximately 180 degrees.

Figure 2: Graphical Drawing of Detection Zones

Figure 3: Graphical Drawing of Detection Zones with Camera Direction

The overlapping of the detection areas of each sensor will give us discrete detection zones.

From Figure 2, we can see that each detection zone (numbered from 1-7) are overlapping

portions of each sensor's detection area. For example, if sensor 1 triggers (first sensor from

the left), it means that movement has been detected in zone 1. While if both sensor 1 and 2

trigger, movement has been detected in zone 2. Therefore, 140 degrees was chosen as it could

give us sufficient detection zones.

After we have derived the detection zone, we came up with the camera angles, which were

the middle of the different detection zones. Table 1 depicts the angles between the detection

zones.

5

Table 1: Difference in Angle between Detection Zones

Zone Difference in angle/degrees

1-2 20

2-3 35

3-4 20

4-5 20

5-6 35

6-7 20

For example, to move from zones 1 to 2, the motor had to move 20 degrees, while from 2-3,

it would be 35 degrees.

Stepper Motor

There are many various techniques to drive a stepper motor, namely half stepping, full

stepping and wave drive. Wave Drive is the simplest to use, Full Step provides the most

torque while Half Step provides the most precision. In our project, we have decided to use

full stepping as it gives us the most torque, while still providing reasonable angular resolution.

Full stepping involves energizing 2 phases at any one time. (Stateham, B. 2015)

Figure 4: Diagram of Full Step Method

From Figure 4, we can see that to rotate the shaft, we have to output: "1001","1100","0110",

"0011" sequentially.

Software

To handle the inputs from the sensor, we stored the input as a 4 bit string, and coded a

process that will map each input to a detection zone. This process will decide which zone the

motor has to move to next. From Table 2, we can tell that if the input is "1000" then

movement has been detected in zone 1.

6

Table 2 - Table of Each Specific 4 Bit String Assigned to Each Detection Zone

Detection Zone 4 Bit String From Sensors

1 “1000”

2 “1100”

3 “1110”

4 “0110”

5 “0111”

6 “0011”

7 “0001”

previous zone others

After knowing which zone to go to next, another process was coded to decide the precise

number of steps and the direction to rotate. This was achieved by finding the difference

between the next and previous zone. If the difference was positive, then the motor had to turn

clockwise, else if it was negative, then the motor had to turn counter clockwise. As explained

above, from the difference in magnitude and sign between the next and previous zone, we

could tell what angle the motor had to turn and in what direction. Table 3 shows the

corresponding angle the motor has to turn from the difference in zones.

Table 3 - Table of the Corresponding Angle from the Difference in Zones

Difference between 2 zones Angle and Direction of the Motor

-6 Counter clockwise, 150 degrees


-4 Counter clockwise, 95 or 110 degrees




0 No movement

1 Clockwise, 20 or 35 degrees


3 Clockwise, 75 degrees

7




After the direction and angle of motion is decided, the same process will convert the angle to

number of steps the motor has to turn. The formula for calculating the number of steps the

motor has to move for an angle is as follows:

This calculated value will be passed to a separate process that handles motor movement. This

process will receive the calculated value and instruct the motor to move the specific angle.

To ensure that only when the motor has finished moving that a new input can be processed,

we implemented a signal that would hold the signal from the sensors and will only pass the

updated values to the subsequent process when the motor was idle.

This also meant that in order to process more inputs, the motor had to move as fast as

possible so that the subsequent input can be computed. The motor could only respond to up to

500 pulses per second. However, the built in clock on the FPGA was a 100 MHz clock. To

overcome this, we implemented a function that would slow the clock down by having a

counter to count up to 200,000 before incrementing. This effectively gave us a 500 Hz clock,

which we could use to drive the motor.

The Finished VHDL Code is found in the Appendix.

TESTING AND RESULTS

During testing, we ported the 4 bit inputs to switches on the FPGA board instead of the

sensors. We were able to make the code work as expected when replacing the sensors inputs

with switches. When we attempted using the inputs from the sensors, the code did not turn

out as expected. This is mainly due to issues with the sensors. More will be elaborated under

the section Limitations.

Construction and implementation of our security system was successful.

Figure 5: Top View of Chassis Figure 6: Side View of Top Panel

8

Figure 7: Interior view of Chassis (Power bank and breadboard not taped)

LIMITATIONS

Despite our best efforts to resolve various issues there are still a few outstanding flaws that

may affect the effectiveness of our system.

A key assumption we made is that the sensors have equal FOV. In practice, this is not the

case as the FOV of each sensor may not be exactly the same. Hence, the representation of the

size of the detection zones will differ in practice as shown in Figure 7.

The PIR sensors are also not very responsive. After being triggered, there will be a short

delay where the sensor will output a digital high causing an undesirable delay. The PIR

sensors are also prone to false triggers and may output a digital high with no external stimuli.

This will cause the motor to rotate unintentionally.

Additionally, our surveillance system will not function properly when there are many targets.

Each sensor’s response time also varies and are not equal, making calibration a lot more

challenging.

AREAS OF FURTHER WORK AND IMPROVEMENTS

Various avenues for future extensions are present with regards to the performance and

functionality of our security system.

In order to improve the functionality of the system, the usage of the PIR sensors could be

changed. One possible way is to limit the FOV of the sensors. This will result in fewer

overlapping regions and will make the algorithm much less complex.

We would also have to alter our code to handle exception cases. To achieve this, a more in

depth analysis needs to be carried out.

There are other minor areas of improvements. Reducing the size of the prototype can be one

of the improvements, such as by using a smaller FPGA board and integrating the motor and

the camera together to make the prototype as compact as possible.

9

ACKNOWLEDGEMENTS

We would like to extend our heartfelt gratitude to our mentors Mr Sim and Dr Ng for their

continuous support throughout the research programme. Their guidance and advice allowed

us to learn much more in the process. We would also like to thank Daniel for helping us with

the code.

10

REFERENCES

[1] Soyer, E.B. (2009). Pyroelectric Infrared (PIR) Sensor Based Event Detection.

Retrieved from

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.296.4053&rep=rep1&type=pdf

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.296.4053&rep=rep1&type=pdf

11

APPENDIX

PIR Sensors The sensors will detect a change in the IR levels emitted by surrounding objects. They are

made of pyroelectric materials such as crystals [1]. On the surface of the sensors, crystalline

material located at the two rectangles detects the infrared radiation and a charge is sensed.

The charge is then converted to a voltage level, which is measured by an amplifier [1].

When the sensor is idle, both rectangular sections will detect the same amount of IR. The

sensor then outputs a low signal on the output pin. When a source passes the field of view of

the sensor, it will intersect the first section and will cause a positive differential change.

When the body intersects the second section, it will then cause a negative differential change

[1]. The sensor detects these varying pulses. For both cases, whenever there is motion

detected the sensor will output a high signal to the output pin.

Figure 8: Illustration of Sensor Motion Figure 9: Top and Side View of Sensor

Detection

To help broaden the field of view and range of the PIR sensor, a Fresnel lens is used. Each

PIR sensor has a lens with multiple sections, each section of which is a Fresnel lens [1].

These Fresnel lenses condenses light, hence providing a larger range [1].

Stepper Motor Inside the stepper motor are two coils wrapped around. A single coil will have two ends. So,

there will be four connections for the ends of the two coils. Since the 28BYJ-48 Stepper

Motor is unipolar, the centre of each coil are attached together, also known as a centre tap.

This leaves us with a total of five wires coming out of the motor. Each wire is associated with

8 teeth, hence there are a total of 32 teeth in the motor, and each separated from each other by

11.25 degrees (Stateham, B. 2015).

12

Figure 10: Internal Diagram of Motor Figure 11: Picture of Motor with Driver

Board

With unipolar motors, we do not require any electronic circuitry to change the direction of

current. Instead, we are instead going to send a voltage through the centre tap and use some

circuitry to pull the appropriate coil ends to ground to change the direction of current. Each In

the centre, the rotor is attached to a permanent magnet. They are used to line up with the

metal teeth based on the magnetic field that is generated by the coils. The stepper motor has a

64:1 gear ratio, so it will take 64 full rotations of the motor rotor for the outer shaft to turn

once (Stateham, B. 2015).

The ULN2003 chip on the driver board holds 7 Darlington Pairs, which are pairs of

transistors where the second transistor amplifies the output current of the first transistor. This

results in higher current gain and requires low voltage to power. The driver board includes

headers to connect the stepper motor to, power headers, 4 I/O pins to control the stepper

motor as well as coil power indicator LEDS (Stateham, B. 2015).

Figure 12: Diagram of ULN2003 Driver Board

13

Bottom Plate of Chassis

Front View of Panel with Sensor

14

Camera Mounting Plate

15

Top Plate of chassis

VHDL Code for FPGA Firmware

library IEEE;

use IEEE.STD_LOGIC_1164.ALL;

use IEEE.NUMERIC_STD.ALL;

library UNISIM;

use UNISIM.VComponents.all;

entity main is

Port ( sensor_in : in STD_LOGIC_VECTOR (3 downto 0);

clk : in STD_LOGIC;

dout : out STD_LOGIC_VECTOR (3 downto 0);

manual : in std_logic;

led : out std_logic_vector (4 downto 0);

led1 : out std_logic_vector(16 downto 10);

but1: in std_logic;

16

but2: in std_logic;

but3: in std_logic;

but4: in std_logic

);

end main;

architecture Behavioral of main is

signal scale : std_logic_vector(17 downto 0);

signal clock_counter, clock_counter_value : integer:= 0;

type state is (state_a, state_b, state_c_1, state_c_2,state_m, state_d_1, state_d_2, state_e_1,

state_e_2);

signal present_state : state := state_a;

signal next_state : state;

signal cw, rest: std_logic;

signal present_zone, next_zone : integer range 1 to 7:= 4;

signal previous_zone : integer range 1 to 7 := 4;

begin

----------------------------------------------------------------------------------------------------------------

inputs : process (sensor_in, previous_zone)

begin

case sensor_in is

when "0000" => next_zone <= previous_zone;

when "0001" => next_zone <= 7;















when others => next_zone <= previous_zone;

end case;

end process;

----------------------------------------------------------------------------------------------------------------

diff_input: process ( present_zone,previous_zone,clock_counter_value)

variable diff : integer;

begin

diff := integer(present_zone) - integer(previous_zone);

17

if diff = 1 then

if present_zone = 3 then

clock_counter_value <= 200; --35degrees

elsif present_zone = 6 then

clock_counter_value <= 200;

else

clock_counter_value <= 112; -- 20degrees

end if;

elsif diff = 2 then



else


end if;

elsif diff = 3 then


elsif diff = 4 then



else


end if;

elsif diff = 5 then


elsif diff = 6 then


elsif diff = -1 then



elsif present_zone = 5 then


else


end if;




else


end if;

18






else


end if;





else clock_counter_value <= 0;

end if;

if diff > 0 then

cw <= '1';

rest <= '0';

elsif diff < 0 then

cw <= '0';

rest <= '0';

else rest <= '1';

cw <= '0';

end if;

end process;

----------------------------------------------------------------------------------------------------------------

zone_check: process (previous_zone,clk)

begin

if rising_edge(clk) then

if previous_zone = 1 then

led1 <= "1000000";

elsif previous_zone = 2 then

led1 <= "1100000";


led1 <= "1110000";


led1 <= "1111000";


19

led1 <= "1111100";


led1 <= "1111110";


led1 <= "1111111";

end if;

end if;

end process;

----------------------------------------------------------------------------------------------------------------

state_reg: process (clk)

begin


present_state <= next_state;

end if;

end process;

----------------------------------------------------------------------------------------------------------------

nxt_state: process(present_state, rest, cw, clock_counter, clock_counter_value, clk)

variable mCw , ccw : std_logic_vector (1 downto 0) := "00";

begin


case present_state is

when state_a =>

led <= "10000";

previous_zone <= present_zone;

next_state <= state_b;

when state_b =>

present_zone <= next_zone;

led <= "01000";

mCw := "00";

ccw := "00";

clock_counter <= 0;

if manual = '1' then

next_state <= state_m;

elsif rest = '1' then

next_state <= state_b;

elsif cw = '1' then

next_state <= state_c_1; --clockwise

else next_state <= state_c_2; --anticlockwise

end if;

when state_c_1 =>

led <= "00100";

present_zone <= present_zone;

previous_zone <= previous_zone;

20

if integer(clock_counter) < integer(clock_counter_value) then

next_state <= state_c_1;

if scale < "110000110101000000" then

scale <= std_logic_vector(unsigned(scale)+1);

else mCw := std_logic_vector(unsigned(mCw) + 1);

scale <= (others => '0');

clock_counter <= clock_counter +1;

end if;

else clock_counter <= 0;

next_state <= state_a;

end if;

case mCw is

when "00" => dout <= "1100";

when "01" => dout <= "0110";

when "10" => dout <= "0011";

when "11" => dout <= "1001";

end case;

when state_c_2 =>

led <= "00010";

present_zone <= present_zone;

previous_zone <= previous_zone;

if integer(clock_counter) < integer(clock_counter_value) then

next_state <= state_c_2;

if scale < "110000110101000000" then


else ccw := std_logic_vector(unsigned(ccw) + 1);



end if;


next_state <= state_a;

end if;

case ccw is

when "00" => dout <= "1001";

when "01" => dout <= "0011";

when "10" => dout <= "0110";

when "11" => dout <= "1100";

end case;

when state_m =>

led <="00001";

if manual = '1' then

if but1 = '1' then

next_state <= state_e_2;

21

elsif but2 = '1' then

next_state <= state_d_2;





else next_state <= state_m;

end if;

else next_state <= state_a;

end if;

when state_d_1 =>

if integer(clock_counter) < integer(112) then


if scale < "110000110101000000" then





end if;



end if;

case mCw is

when "00" => dout <= "1100";

when "01" => dout <= "0110";

when "10" => dout <= "0011";

when "11" => dout <= "1001";

end case;

when state_d_2 =>



if scale < "110000110101000000" then





end if;



end if;

22

case mCw is

when "00" => dout <= "1100";

when "01" => dout <= "0110";

when "10" => dout <= "0011";

when "11" => dout <= "1001";

end case;

when state_e_1 =>



if scale < "110000110101000000" then





end if;



end if;

case ccw is

when "00" => dout <= "1001";

when "01" => dout <= "0011";

when "10" => dout <= "0110";

when "11" => dout <= "1100";

end case;

when state_e_2 =>



if scale < "110000110101000000" then



clock_counter <= clock_counter +1 ;


end if;



end if;

case ccw is

when "00" => dout <= "1001";

when "01" => dout <= "0011";

23

when "10" => dout <= "0110";

when "11" => dout <= "1100";

end case;

end case;

end if;

end process;

end Behavioral;

Documents

IMPLEMENTATION OF A PYROELECTRIC INFRARED TRACKING …