AUTOMATIC PAN-TILT CAMERA POSITIONING SYSTEM

FOR MOTION TRACKING

WONG TECK CHANG

A project report submitted in partial fulfillment

of the requirements for the award of the degree of

Master of Engineering (Electronics and Telecommunications)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

OCTOBER, 2004

iii

Dedicated to my beloved parents,

brothers and friends ...

Thanks for your continuous support and encouragement.

iv

ACKNOWLEDGEMENT

I would like to take this opportunity to extend my deepest gratitude and

appreciation to my supervisor, Prof. Madya Dr. Syed Abdul Rahman Bin Syed Abu

Bakar for his invaluable and insightful guidance, support and encouragement offered

throughout the length of this study.

Special thanks to my friends, course mates, staffs, lecturers in the Faculty of

Electrical Engineering, Universiti Teknologi Malaysia for their help and support. Not

forgetting, to my family for their supports and understanding.

v

ABSTRACT

In this project, a pan-tilt unit (PTU) which is actuated by two DC motors is

used as part of the motion tracking camera system. A camera is mounted on the PTU

and the PTU is responsible to move the camera according to the output from the

motion tracking algorithm. Accuracy and speed are the two concerns when the PTU

is moved because the motion tracking camera is expected to track a single moving

object and keep it at the centre of the field of view. A pan-tilt camera positioning

system has been developed to meet the characteristics of the motion tracking

algorithm and some other hardware constraints. The technique implemented to

realize the pan-tilt camera positioning system is an open-loop control called

coordinate to motion unit translation. The video frames which are at the size of 256

pixels times 256 pixels are indexed into 392 motion units, which are the points that

the camera will centre each time it is being moved. This technique has enabled the

manual directivity feature even when the pan-tilt camera is still moving and in the

meantime preserves the accuracy of the system. The testing results of the system

developed have shown that the motion tracking camera has an accuracy of ±10 pixels

and was able to track a moving object travelling at the speed less than 85 pixels per

second.

vi

ABSTRAK

Dalam projek ini, satu unit “pan-tilt” (PTU) yang digerakkan oleh dua motor

DC digunakan sebagai sebahagian daripada sistem kamera jejak pergerakan. Sebuah

kamera dipasangkan pada PTU tersebut dan PTU itu bertanggungjawab untuk

mengerakkan kamera tersebut mengikut output daripada algoritma jejak pergerakan.

Kejituan dan kelajuan merupakan dua faktor yang penting apabila PTU tersebut

digerakkan kerana sistem kamera jejak pergerakan perlu menjejak satu objek yang

sedang bergerak dan mengekalkannya di tengah-tengah rangka video. Satu sistem

memposisi kamera “pan-tilt” telah direka supaya bersesuaian dengan algoritma jejak

pergerakan dan faktor-faktor perkakasan berkenaan. Teknik yang digunakan untuk

menghasilkan sistem memposisi kamera “pan-tilt” tersebut merupakan sejenis

kawalan gelung-buka yang dinamakan translasi koordinat kepada unit pergerakan.

Rangka video yang bersaiz 256 pixel darab 256 pixel dibahagikan kepada 392 unit

pergerakan, yang merupakan titik-titik yang diketengahkan oleh kamera setiap kali

kamera bergerak. Teknik ini membolehkan arahan pergerakan kamera secara manual

sungguhpun kamera masih dalam pergerakan, tetapi masih mampu mengekalkan

kejituan sistem. Keputusan semasa menguji sistem yang dibangunkan ini

menunjukkan sistem kamera jejak pergerakan ini mempunyai kejituan ±10 pixel dan

dapat menjejak objek yang bergerak dalam kelajuan kurang daripada 85 pixel per

saat.

vii

TABLE OF CONTENTS

CHAPTER TOPICS PAGE

TITLE i

DECLARATION STATEMENT ii

DEDICATION iii

ACKNOWLEDGEMENTS iv

ABSTRACT v

ABSTRAK vi

TABLE OF CONTENTS vii

LIST OF TABLES x

LIST OF FIGURES xi

LIST OF ACRONYMS xiii

LIST OF APPENDICES xiv

CHAPTER I INTRODUCTION 1

1.1 Introduction

1.2 Project Background

1.3 Objective of Work

1.4 Scopes of Work

1.5 Layout of Thesis

1

3

3

4

4

viii

CHAPTER II LITERATURE REVIEW 6

2.1 Introduction

2.2 Main Components of a Motion

Tracking Camera System

2.2.1 Spatial Positioning System

2.3 Motion Tracking Algorithm

2.4 Pan-tilt Camera Positioning System

2.4.1 DC Motor Position Control By

Discrete-time Variable Structure

Controllers

2.4.2 DC Motor Position Control

Using Field Programmable Gate

Arrays

2.4.3 Fast Positioning Using H∞

Control

2.5 Conclusion

6

6

7

8

10

10

11

12

16

CHAPTER III RESEARCH METHODOLOGY 17

3.1 Introduction

3.2 Main Components of the Whole System

3.3 Hardware Constraints

3.4 Calibration of Pan-tilt Camera Unit

3.5 Moving the Pan-tilt Camera

3.6 Positioning the Pan-tilt Camera

3.7 General Concept of the Methodology to

Position the Pan-tilt Camera

3.8 The Implemented Methodology to

Position the Pan-tilt Camera

3.8.1 Coordinate to Motion Unit

Translation

17

17

19

20

23

24

25

27

27

ix

3.8.2 Pan-tilt Camera Positioning

Algorithm

30

CHAPTER IV RESULTS 33

4.1 Introduction

4.2 Performance of the System

4.3 Limitations of the System

33

35

37

CHAPTER V CONCLUSIONS AND

RECOMMENDATIONS

39

5.1 Summary

5.2 Conclusions

5.3 Recommendation for Future Works

39

41

42

REFERENCES 43-45

APPENDIX A-B 46-57

x

LIST OF TABLES

TABLE NO. TITLE PAGE

3.1 The Truth Table for The PTU Movements 12

xii

4.1 The Front View of the PTU 34

4.2 The Side View of the PTU 34

4.3 The Upper View of the PTU 35

4.4 Sequence of Images Extracted from the

Recorded Video during the Testing Process

36

xi

LIST OF FIGURES

FIGURE NO. TITLE PAGE

1.1 The Definition of “Pan” and “Tilt” of A Pan-tilt

Camera

1

2.1 The Main Components of A Motion Tracking

Camera

7

2.2 The Components of A Spatial Positioning

System

8

2.3 Motion Detection Module 9

2.4 Motion Tracking Module 9

2.5 Block Diagram of Positioning System 14

2.6 Generalized Plant for H∞ Mixed Sensitivity

Problem

14

3.1 The Main Components in the Motion Tracking

Camera System

18

3.2 The Panning and Tilting Angle of The PTU 20

3.3 Perspective Projection of Object onto Image

Plane

22

3.4 Open-loop Control System Basic Block Diagram 24

3.5 Overview of The Pan-tilt Positioning Algorithm

Program Flow

26

3.6 Video frame At The Size of 256X256 Pixels

After Indexed Into Motion Units

30

3.7 The Program Flow of The Pan-tilt Positioning

Algorithm

32

xiii

LIST OT ACRONYMS

AC - Alternating Current

CCTV - Closed Circuit Television

CIS - Central Inference System

CVVIP - Computer Vision, Video, and Image Processing

DC - Direct Current

DTVSS - Discrete-time Variable Structure Systems

FPGA - Field Programmable Gate Arrays

LSB - Least Significant Bit

MSB - Most Significant Bit

PC - Personal Computer

PI - Proportional-integral

PTU - Pan-tilt Unit

SPS - Spatial Positioning System

VS - Variable Structure

VSC - Variable Structure Controller

xiv

LIST OF APPENDICES

APPENDIX TITLE PAGE

A Code Listing 46

B Graphic User Interface 57

CHAPTER I

INTRODUCTION

1.1 Introduction

Our world has become a digital world. Many products around us incorporate

digital technology. Cars, cameras, and computers all utilize digital technology.

Surveillance system without any exception has also been evolving in the past few

decades, from analog to digital, from analog CCTV to digital CCTV, and has been

incorporated with a wide range of accessibility. One of the state-of-the-art technologies

of surveillance system is the motion tracking camera system. Motion tracking camera,

as its name suggests, will follow a moving object and will keep the target in the centre of

the field of view. This is especially useful in video conferencing and security

surveillance.

Cameras are traditionally manually controlled. It focuses on one point in space

unless somebody turns it. Often times, however, this is neither useful nor convenient.

For example, suppose the security services is trying to protect a building, in order to

2

cover a fairly small region, either many cameras are installed, or human supervision is

constantly needed, or both. Since emergencies happen rather infrequently, most of the

time, the guards or video tapes are wasted. Such problems can be solved if we have a

motion tracking pan-tilt camera.

A pan-tilt camera is a stationary but rotating camera, where pan means rotating

along the azimuth angle and tilt means rotating along the elevation angle. A motion

tracking pan-tilt camera is able to track a moving object, thus avoiding the object from

getting lost from sight. It is especially useful and convenient for security surveillance

and video conferencing. For example, having a motion tracking pan-tilt camera during a

video conference, we don’t have to sit still in front of the camera but can move around a

room and the camera will track us down. Figure 1.1 shows a pan-tilt camera and the

definition of “pan” and “tilt”.

Figure 1.1 : The Definition of “Pan” and “Tilt” of A Pan-tilt Camera

Elevation angle

Azimuth angle

“Tilt”

“Pan”

3

1.2 Project Background

The project to develop a motion tracking camera system was initiated in year

2001 by a group of researchers in Computer Vision, Video and Image Processing

(CVVIP) research unit at the Faculty of Electrical Engineering of University of

Technology Malaysia. The system however, has poor performance and could not track

accurately. The pan-tilt camera could move when a moving object was detected, but has

low accuracy and was not able to keep it at the centre of the field of view.

It was identified that the problem was with the pan-tilt camera positioning

system. The motion tracking algorithm was actually performing as required, but the

camera positioning system was unable to produce accurate and fast performance.

The project explained by this thesis was to look into this matter, thus developing

an effective positioning system that is able to response accurately according to the

output of the motion tracking algorithm.

1.3 Objective of Work

The objective of this project is to accurately track a single moving object using a

pan-tilt camera. To accurately track a single moving object means to keep a moving

object at the centre region of the camera view.

4

1.4 Scopes of Work

In order to be able to achieve the stated objectives, the scope of the project is

limited to the following:

i. Develop an algorithm using Visual C++, for controlling a dual DC motor pan-tilt

camera mounting.

ii. Incorporate the relay-motor pan-tilt mounting to an existing motion tracking

program which is written in Visual C++.

iii. Control the above mounting through the parallel port of a PC.

iv. Enable the camera view to follow a single moving object.

1.5 Layout of Thesis

In Chapter II, a presentation regarding the main components of a motion tracking

camera system will firstly be done. This is followed by descriptions about the motion

tracking algorithm used in system developed in this project. Most importantly, the

descriptions of previous works and existing methods regarding the scopes of this project

will also be included.

Chapter III explains the techniques and methods that are used to develop the pan-

tilt camera positioning algorithm. The algorithm and the program flow of the pan-tilt

5

camera positioning system will also be explained. Some relevant characteristics and

constraints of the components of the system concerning the development of the

algorithm will first be described.

In Chapter IV, the outcome of the works of this project and the results of the

implementation of the methodology in Chapter III will be presented. The performance

in terms of accuracy and speed of the system developed will be discussed. Besides that,

the limitations of the system will also be listed.

The thesis is concluded by Chapter V, where the conclusions drawn from the

study as well as some recommendations for future works are presented.

43

REFERENCES

1. Boyoon Jung and Gaurav S. Sukhatme (2004). Detecting Moving Objects

using a Single Camera on a Mobile Robot in an Outdoor Environment.

8th Conference on Intelligent Autonomous Systems. March 10-13, 2004.

Amsterdam, The Netherlands: University of Southern California. 980-987.

2. J. Hu, D.M. Dawson and K. Anderson (1995). Position Control of A

Brushless Dc Motor Without Velocity Measurements. IEE Proc.-Electr.

Power Appl. Vol.142, No 2: 113-122.

3. J. Weng, P. Cohen, and M. Herniou (1992). Camera Calibration with

Distortion Models and Accuracy Evaluation. IEEE Transactions on

Pattern Analysis and Machine Intelligence, 14(10): 965-980.

4. Jianyong Niu (2002). Prototyping A DC Motor PI Controller Using Field

Programmable Gate Arrays. University of Sheffield: Master Thesis.

5. Keith Jones (1994). IEC 1131-3 Programming for Motion Control. Savoy

Place, London: IEE.

6. Luigi Di Stefano and Enrico Viarani (1999). Vehicle Detection and Tracking

Using the Block Matching Algorithm. Italy: University of Bologna.

7. Makoto Iwasaki, Kenta Seki and Hiromu Hirai (2000). Fast-Response

Positioning Using H Control in Machine Tools. Nagoya, Japan: Nagoya

Institute of Technology.

44

8. Olivier Faugeras (1993). Three-Dimensional Computer Vision - A Geometric

Viewpoint. MA: MIT Press, Cambridge.

9. P. M. Ngan and R. J. Valkenburg. Calibrating a Pan-tilt Camera Head.

Auckland, New Zealand: Machine Vision Team, Industrial Research Ltd.

10. P.R. Moore and C.M. Chen (1994). The Syncronisation of Servo Drives

Using Fuzzy Logic Control. Savoy Place, London: IEE.

11. Phaik Yong. Y (2003). Accurate Real-Time Object Tracking With Linear

Prediction Method. UTM: Master Thesis.

12. R. Y. Tsai and R. K. Lenz (1989). A New Technique for Fully Autonomous

and Efficient Robotic Hand/Eye Calibration. IEEE Transactions on

Pattern Analysis and Machine Intelligence. 5(3): 345-358.

13. Robert J. Valkenburg (1995). Classification of Camera Calibration

Techniques. Auckland, New Zealand: Machine Vision Team, Industrial

Research Ltd.

14. Zoran Jovanovic and Goran Golo (1997). Dc Motor Position Control By

Discrete-time Variable Structure Controllers. Mechanics, Automatic,

Control and Robotics. 2(7): 291-300.

15. J. Shi, C. Tomasi (1994). Good Features to Track. Proceedings, IEEE

Conference on Computer Vision and Pattern Recognition. 593-600.

16. A. L. Yuille, P. W. Hallinan, D. S. Cohen (1992). Feature extraction from

faces using deformable templates. International Journal of Computer

Vision. 8(2): 99-111.

17. John M. Gauch (1999). Image segmentation and analysis via multiscale

gradient watershed hierarchies. IEEE Transactions on Pattern Analysis

and Machine Intelligence. 8(1): 69-79.

45

18. H. Wechsler, Z. Duric, Fayin Li, and V. Cherkassky (2004). Motion

Estimation Using Statistical Learning Theory. IEEE Transactions on

Pattern Analysis and Machine Intelligence. 26(4): 466- 478.

19. J. Park, B. Jiang, and U. Neumann (1999). Vision-based Pose Computation:

Robust and Accurate Augmented Reality Tracking. IEEE International

Workshop on Augmented Reality. 3-12.