Human Tracking With Wireless Distributed Pyroelectric Sensors

IEEE SENSORS JOURNAL, VOL. 6, NO. 6, DECEMBER 2006 1683

Human Tracking With Wireless DistributedPyroelectric Sensors

Qi Hao, David J. Brady, Senior Member, IEEE, Bob D. Guenther, John B. Burchett,Mohan Shankar, and Steve Feller

Abstract—This paper presents a wireless pyroelectric sensorsystem, composed of sensing modules (slaves), a synchronizationand error rejection module (master), and a data fusion module(host), to perform human tracking. The computation workloaddistribution among slave, master, and host is investigated. Theperformances and costs of different signal-processing and target–tracking algorithms are discussed. A prototype system is describedcontaining pyroelectric sensor modules that are able to detect theangular displacement of a moving thermal target. Fresnel lensarrays are used to modulate the sensor field of view. The sensorsystem has been used to track a single human target.

Index Terms—Fresnel lens, human motion tracking, pyroelec-tric sensor, wireless sensor network.

I. INTRODUCTION

HUMAN MOTION tracking includes capturing body dis-placements and limb movements, such as postures and

gestures, of human targets. It has been proposed for extensiveapplications such as computer vision, robotics, virtual reality,smart space, surveillance, etc. [1]–[5]. Optical and thermalvision-based tracking systems are nonintrusive sensory ap-proaches that can interpret human motions in a natural waywithout interference.

Most vision-based approaches to moving object detectioninvolve intensive real-time computations, such as temporaldifferencing, background subtraction, and thermal flow esti-mation [5]–[7]. In many cases, due to the availability of priorknowledge on target motion dynamics, the intensive and ex-pensive all-purpose imaging detector array appears inefficientand unnecessary. For instance, a video image consisting of100 × 100 pixels with 8-bit gray level contains 80 kbits of data,while the position and velocity can be represented by only afew bits [8].

Recent advances in microprocessor, radio frequency trans-ceiver, and novel sensor technologies have allowed the develop-

Manuscript received September 7, 2005; revised April 15, 2006, June 5,2006, and June 7, 2006. This work was supported by the Army ResearchOffice through Grant DAAD 19-03-1-03552. The associate editor coordinatingthe review of this paper and approving it for publication was Prof. RalphEtienne-Cummings.

The authors are with the Fitzpatric Institute for Photonics and theDepartment of Electrical and Computer Engineering, Duke University,Durham, NC 27708 USA (e-mail: [email protected]; [email protected];[email protected]; [email protected]; [email protected]; [email protected]).

Color versions of Figs. 1, 6, 14, 16, 18, and 20–22 are available online athttp://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/JSEN.2006.884562

ment of a promising alternative to the centralized video sensorsfor tracking, namely a distributed sensor network (DSN) [9].A DSN consists of many small low-cost spatially dispersedsensing nodes which analyze and process the data collectedfrom their surroundings and transfer only the information ofinterest to their decision superiors. For many DSN applications,passive sensors are preferred to active ones because of the lowcost, low power consumption, and low detectability. Despitenumerous applications of DSNs in vehicle tracking [10]–[13]and robot positioning [14], few reports can be found on humanmotion tracking [1].

In our study of human motion tracking, LiTaO3 pyroelectricsensors have been selected because of

1) their lower costs and commercial availability;2) their sensitivity to the radiation emitted by the human

body (8 ∼ 14 µm) [15];3) their sensitivity to angular velocities from 0.1 to 3 rad/s;4) their responsivity only to the motion of targets [16]; and5) the controllability of their field of views (FOVs) using

low-cost Fresnel lens arrays [17].

In this paper, we introduce the concept of a geometric sensorwhich allows a tradeoff between the high-information contentof a distributed sensor system and the bottlenecks produced bynarrow data throughput and limited computation power. A geo-metric sensor is designed with a spatially modulated viewingarea to allow a single detector to extract complex informationabout the target [18], [19]. For example, by using a Fresnellens array, the response signal of one pyroelectric sensor can bemodulated into a windowed sine-like sequence, from which theangular velocity of a target can be easily readout. The spectrumof the sequence can also be used to discriminate between twoindividuals. The spectral signatures are believed to be caused bythe difference in the motion of arms and legs [20]. Fresnel lensalso can shape the sensors’ FOVs to create overlapping detec-tion regions; when a thermal target enters one of these regions,two or more sensors will fire simultaneously, giving the locationof the target [17], [21], [22].

To investigate human motion tracking, we developed a pro-totype distributed pyroelectric sensor system, with four slavemodules, one master module, and one host computer. Eachslave module contains eight pyroelectric sensors with modu-lated FOVs. The sensor platform can collect the sensor responsesignals, convert them into digital event indexes, and send themto the master module via wireless channels. The master modulecan collect event indexes from different slave modules, while

1530-437X/$20.00 © 2006 IEEE

1684 IEEE SENSORS JOURNAL, VOL. 6, NO. 6, DECEMBER 2006

Fig. 1. Main issues for distributed pyroelectric sensor system building.

synchronizing the communications and rejecting the eventerrors. Hidden Markov model (HMM) filtering is performedunder some heuristic constraints before sending data to thehost via a UART serial cable. The host converts event in-dexes into local angular displacements, and then fuses all themeasurements using a simplified Bayesian tracking scheme, toestimate the trajectory of the moving target.

The main design issues in Fig. 1 are outlined below.

1) Signal processing balance between photonics and elec-tronics. We employ Fresnel lenses to modulate the FOVsof eight sensors distributed along a circle allowing themeasurement of angular displacement over 360. To im-prove the resolution, the sensor FOVs overlap.

2) Computation load distributed on host, master, andslave. System performance can be improved by properdistribution of signal filtering, threshold testing, motioninterpretation, error rejection, and tracking synthesisamong the system components.

3) Algorithm tradeoff between performance and cost.We must simplify the algorithms without sacrificing per-formance to be able to perform tracking on our prototypesystem. For example, particle filters are capable of track-ing objects but require intensive computation [23]–[25].Windowed fast Fourier transform (FFT) is another usefultool but consumes computation resources.

4) Communication protocol. As a considerable part of bat-tery power is used for wireless communication, a suitablecommunication protocol must be selected to conservethe limited energy and prolong operation time of thesystem [26].

The remainder of this paper is organized as follows.Section II gives the description of the system setup, the sensormodel, the specifics of the Fresnel lens, and the sensor FOVmodulation, and presents a mathematical description of thetracking problem. Section III discusses the algorithm selectionfor slave and master nodes. Section IV describes the data fusion,tracking synthesis, and summarizes the system implementation.In Section V, we show simulation and experimental results, anddiscuss the strength and weakness of the design. Section VIcontains a discussion of future work.

Fig. 2. Polar plot of visibility pattern of a dual element pyroelectric detectorwithout collection optics.

II. SYSTEM MODEL AND PROBLEM STATEMENT

In this section, we present a pyroelectric sensor model, sensorFOV modulation using Fresnel lens arrays, and a statement ofthe tracking problem.

A. Pyroelectric Sensor Modeling

In general, a distributed sensor system will contain a signalspace, the FOV of the sensor, and an object space containing allconfiguration states of the object. In this paper, those compo-nents are represented by a pyroelectric sensing circuit, a Fresnellens array, and human thermal sources, respectively.

Assuming a linear sensor, the response signal of m sensorss(t) ∈ Rm is given by

s(t) = h(t)∗∫Ω

v(r)ψ(r, t)d r (1)

where “∗” denotes convolution, h(t) is the impulse response ofone sensor, Ω is the object space, v(r) ∈ [0, 1]m is the positivevisibility function between m sensors and the object space, andψ(r, t) is the radiation from the target.

The visibility v(r1, r2) describes the contribution by the fieldat point r2 to the field at point r1. The visibility pattern of adual element pyroelectric detector is shown in Fig. 2. The signalstrength is shown in arbitrary units. The dual lobe visibility pat-tern of Fig. 2 is formed because the two pyroelectric elementsare connected in opposition so that the total signal from theregions where the detectors have identical FOVs is zero.

Pyroelectric detector signals are proportional to the changein temperature on the crystal rather than the temperature itself.Therefore, its transfer function is a high-pass one, but theresponse time of the transconductance amplifier of the detectorlimits the maximum frequency. The resultant transfer functionturns out to be a bandpass one [16].

A measured step response of the system is shown as thedashed line in Fig. 3. The solid line is the step response of afourth-order transfer function given by

H(s)=U(s)Φ(s)

=kg

(s2+2ζtωts

s2 + 2ζtωts+ ω2t

− s2 + 2ζeωes

s2 + 2ζeωes+ ω2e

)

(2)

HAO et al.: HUMAN TRACKING WITH WIRELESS DISTRIBUTED PYROELECTRIC SENSORS 1685

Fig. 3. Step response of the pyroelectric sensor system.

Fig. 4. Visibility pattern of a pyroelectric sensor with an 11-element Fresnellens array as the collection optics. Details of the beam pattern are shown on thesecond beam from the left. (a) Top view. (b) Side view.

where U(s) is the amplified voltage signal; Φ(s) is the thermalflux; the transfer gain kg = 0.26 V/J; the electronic circuitdamping ratio ζe = 0.7; the electronic circuit natural frequencyωe = 2 Hz; the thermal damping ratio ζt = 0.7; and the thermalnatural frequency ωt = 0.6 Hz.

We can see that our detector system is a bandpass systemwith frequency limits between 0.6 and 2 Hz. A moving humantarget, with angular velocities between 0.9 and 3.1 rad/s, wouldproduce signals within this bandwidth, if the sensor has aπ/2 FOV.

B. Fresnel Lens Array and Visibility Modulation

The Fresnel lens array we employ is made of a light-weightlow-cost plastic material with good transmission characteristicsin the 8–10-µm range. A single Fresnel lens array consists of11 Fresnel lenses, located one focal length away from thedetector. Fig. 4 shows the beam pattern of the Fresnel lens(AA0.9GIT1, Fresnel Technologies) used in our experiment.

Fig. 5(a) and (b) illustrate the sensor response signals, whena human target passes through the FOV of a detector with andwithout visibility modulation. We can see that after visibility

Fig. 5. Response signals of a dual element pyroelectric detector to a passingthermal source (a) without visibility modulation and (b) with multiplex visibil-ities produced by a Fresnel lens array.

Fig. 6. Sensor module used in the experiments.

modulation, the sensor response signals have more spatialinformation. The lower measurable limit of angular velocity canbe improved by almost ten times.

The sensor module used in our experiments is shown inFig. 6. This sensor module has eight pyroelectric detectors,with Fresnel lenses arranged in a circle. The visible regions areshown in Fig. 7. Table I lists the association between visibleregions and detection by the eight detectors. A “0” indicatesno signal is present, and a “1” indicates a signal is observed. Itcan be seen that with the simple visibility-coding scheme, theangular resolution is π/8.

C. Tracking Problem Statement

The configuration state x(t) describes the spatial-temporalvarying radiation of the target ψ(r, t) in the FOVs of sensors.


Fig. 7. Visible regions of one sensor module containing eight detectors.

TABLE IVISIBILITY-CODING SCHEME

It consists of the position and velocity of the moving target in atwo-dimensional plane.

We can model the target dynamics as Markov and representit by the conditional density p(xk+1|xk). The problem is thenhow to track a target configuration state with maximum pos-terior probability using sensor response signals s1:k, given thesensor observation likelihood p(sk|xk) and state dynamic priorp(xk+1|xk)

x1:k = arg maxx

p(x1:k|s1:k) (3)

where p(sk|xk) might be derived from a sensor model, visibil-ity function, and prior on noise. It is known as the maximuma posteriori Bayesian tracking problem.

The general sequential Bayesian tracking problem requiresa recursive calculation of a degree of belief in the state xk

with given measurements s1:k. Solutions include prediction andupdate, which are given by

p(xk|s1:k−1) =∫p(xk|xk−1)p(xk−1|s1:k−1)dxk−1

p(xk|s1:k) =p(sk|xk)p(xk|s1:k−1)

p(sk|s1:k−1)(4)

where p(sk|s1:k−1) can be viewed as a normalizing constant.

III. EMBEDDED COMPUTATION

In general, as shown in Fig. 8, the tracking strategy for dis-tributed sensors includes event detection; event digitization, toconvert sampled signals into event indexes; event registration,to reject errors; motion inference, to fuse measurements fromall sensor modules; and trajectory smoothing, to estimate thetrajectory of targets by Bayesian techniques.

In our implementation, we assign the process of eventdetection and event digitization to the slave sensor modules,reducing the required communication bandwidth. The messagebetween slave and master contains 2 B, the slave ID, and theON/OFF status of the eight detectors—the event index. Themaster module synchronizes the event indexes from all the slavemodules. We employ a HMM to allow the master module toregister and smooth the received event indexes, while rejectingerrors. Below, we will examine this process in a little moredetail.

A. Event Detection and Digitization

An event is defined as an occurrence of interest in spatial-temporal space, distinguishable from its environment, and re-peatable. In our case, when the thermal flux measured by apyroelectric sensor is above a threshold value, and the thermalsignal can be associated with motion across one detectionregion, we accept the signal as an event.

We have evaluated three signal processing techniques forevent detection. Our objective was to compare performance andcomputational cost and select the techniques that would provideadequate performance on our prototype platform.1) Kalman Filtering: The Kalman filter can estimate a

process by using a form of feedback control: The filter makesan a priori estimate of the process state, initialized with aguessed value, using a linear process model, and then obtainsan improved posterior estimate with a feedback of noisy mea-surements [27]. By using the sensor model in (2) and assumingthe thermal flux are rectangular pulses, we can obtain a low-pass linear filter.

The whole filter architecture for event detection is shown inFig. 9. When the absolute value of an estimated thermal fluxis larger than a threshold, the corresponding bit of the eventindex will be set to “1”; otherwise, it is set to “0.” A movingaverage filter is used to smooth the event index and generateevent windows. Fig. 10 illustrates how the event windows aregenerated from the sensory data by using the signal processingarchitecture of Fig. 9.


Fig. 8. General tracking scheme for distributed pyroelectric sensors.

Fig. 9. Block diagram of the filter architecture.

Fig. 10. Event detection through a pair of sensors with similar FOVs by using a Kalman filter. (a) Raw data. (b) Filtered signals. (c) Digitized signals. (d) Logicsignals. (e) Event windows.

2) Windowed FFT: An alternative approach for event detec-tion is the windowed discrete Fourier transform. Fig. 11 showsthe windowed power spectrum density of the sensory data andthe derived event windows. We have to increase computationalresources to carry out this approach.

3) Bandpass Sine Filtering: When a human object passesthrough the Fresnel lens modulated FOV, sine-like responsesignals, as shown in Fig. 5(b), are generated. We use a band-pass sine filter as a matched filter, i.e., sin[2π(1 : N)/N ], tocapture and amplify the response signals. Fig. 12 illustrates


Fig. 11. Event detection from windowed power spectrum density of sensory data. (a) Raw data. (b) Data windowed spectra. (c) Digitized signals.(d) Event windows.

Fig. 12. Event detection by using a matched filter. (a) Raw data. (b) Filtered signals. (c) Digitized signals. (d) Logic signals. (e) Event windows.


Fig. 13. Observation sequence generated by one sensor node with false alarms is smoothed by eight state HMM filtering. (a) Event sequences with/without falsealarms. (b) The estimated and true state sequences.

Fig. 14. Data fusion architecture.

how the event time windows are generated from the sensorydata by using a sine filter, when N was chosen as 12.

B. Event Registration

As shown in Table I, there are only 16 unique codes among256 possible 8-bit event indexes that each sensor node can gen-erate. In implementation, this number could be further reduceddue to the deployment of sensor nodes. However, the eventindexes experimentally generated by each slave module arenot limited to what we have designed. We use the term “falsealarm” to describe those event index errors and model the eventsequence produced by one sensor node as a two-layer first-order finite-state discrete-time Markov process [28]. Fig. 13(a)shows an observation sequences with and without false alarms.To simulate the false alarms in the sensor system, each bit ofevent index is set with a flip probability of 0.1. By using HMMsmoothing, the event misregistration rate can be reduced from

58% to 17%. The estimated and true state sequences are shownin Fig. 13(b).

IV. DATA FUSION AND TRACKING SYNTHESIS

Multisensor data fusion is a process through which we obtainreadings from different sensor nodes, remove inconsistencies,and assemble the information into a coherent structure. Vari-ous data fusion schemes and techniques have been proposedfor combining measurements from many sensing nodes [29].Fig. 14 shows the architecture used in our prototype sensorsystem. The slave nodes work at the raw data level. The masterworks at the event level. The host carries out the data fusion atthe feature and tracking level.

A. Motion Inference

Once we have obtained an 8-bit event index of a sensormodule, we can convert it into an angular displacement mea-surement using the visibility function, which is given in Fig. 7and Table I. Fig. 15 illustrates a sequence of events transmittedby one sensor node and its translation in terms of angulardisplacements at the host. As a result, each sensor module turnsout to be a thermal bearing sensor.

For a distributed angular sensor system, a simple way to lin-earize angular measurements is the grid approximation, shownin Fig. 16. When a set of angular displacement measurementsare available, the Cartesian coordinates of the nearest grid pointis chosen to estimate the path of the target.

B. Trajectory Smoothing by Bayesian Tracking

The single target-tracking problem is usually posed as adynamic estimation of a partially observable Markov process,and in most cases, it can be solved by Bayesian recursive


Fig. 15. Eight-bit event sequence from one sensor node and its interpretation in terms of angular displacements. (a) Event sequence. (b) Measured angulardisplacements.

Fig. 16. Grid approximation to linearize the measurements of angular dis-placements with respect to four sensor nodes.

TABLE IICOMPARISON OF DISP ELECTRONIC PLATFORM WITH MOTE

filtering schemes [see (4)]. If the process is linear and Gaussian,a Kalman filter offers the optimal solution. A HMM filter pro-vides the optimal recursion of the posterior density estimation,if the state space of the process consists of only a finite number

Fig. 17. Computation load distribution among slave, master, and host.

Fig. 18. Snapshot of the simulated target tracking. Four sensor nodes detectthe angular displacements of the target, which are illustrated as the shadedbeams. At each iteration, the target position is estimated by Kalman filteringbased on grid approximation.


Fig. 19. Simulation results of tracking a human target with four radial sensor modules shown for the x (upper) and y (down) directions.

of discrete states. When the linear, Gaussian, or finite discretestate assumptions do not hold, Gaussian approximation, grid-based approximation, and particle filters are applied to achievesuboptimal solutions [23].

If the true density is heavily skewed, the Gaussian ap-proximations will have poor performance. Disadvantages ofgrid-based approach include that its state space needs to bepredefined, which consumes data storage space. Also, eachgrid point is usually given the same weight, ignoring the prioron target dynamics. The particle filter technique has heavycomputational needs. For real-time implementation, the compu-tation complexity and practical functionality of a tracking algo-rithm is more valued than its mathematical rigor. Thus, particlefilter tracking is the least favorable for our system, but they canstill serve as a benchmark to evaluate the performance of thewhole system.

In [30], we summarize the three Bayesian tracking strate-gies, namely Kalman, HMM, and Gaussian particle filters.Among the various particle filters, the Gaussian particle filteris chosen for its algorithmic simplicity, and guaranteed conver-gence with a small number of samples. This particle filter hasdemonstrated performance superiority in tracking over variousextended Kalman filters [24]. The comparison of the perfor-mances and computation costs for the different approaches isgiven in Section VI. Based upon those results, we have se-lected a Kalman tracking scheme in real-time implementation,with grid approximations to linearize the angular displacementmeasurements.

V. SYSTEM IMPLEMENTATION

The complete system consists of slave nodes containingpyroelectric sensors and Fresnel lens arrays, a master node, anda host computer. We use a microcontroller MSP430149 andan RF transceiver TRF6901 as the computation and commu-nication platforms. Each MSP430149 contains eight analog-to-digital converter (ADC) channels for sampling detectors’signals. The TRF6901 ISM-band RF transceiver is used to cre-ate multichannel bidirectional wireless communication. Oncethe master/slave communication channel is established, thedirection of control is always from master to slave(s). Table IIshows the feature comparison between our computation andcommunication platform [31] and the well-known MOTEprocessor/radio module [32]. Our platform has higher compu-tation speed, ADC resolution, and data transmission rate but atthe expense of higher power consumption.

The computation load distribution schematic for the entiresystem is shown in Fig. 17. Each slave node samples thesensor response signals and converts them into event indexes, asshown in Fig. 9. After compressing each event index into 1 B,the slave node, broadcasts the data packages to the master.The master node synchronizes the communications withall the slave nodes and registers event indexes as true alarmsusing the visibility-coding table. The master sends the alarmsto the host, which translates those alarms into angular dis-placements and estimates/predicts target motion with maximumlikelihood.


Fig. 20. Histograms of tracking errors by using a Gaussian particle filter in(left) x and (right) y directions.

VI. RESULTS AND DISCUSSION

The tracking scheme was implemented in a 9 × 9-m room.We also ran numerical simulations to pretest its feasibility andto investigate the correlations between sensor resolution, sensordeployment, and tracking precision. Algorithm evaluation wasalso performed via simulations.

A. Simulation Results

Fig. 18 displays a snapshot of one target-tracking scenewith four sensing nodes. The light and dark shaded polygonsrepresent the target and the estimated position, respectively.The sensor node’s resolution is π/8. Despite the measurementerror of sensor node 2, the target still can be tracked using theangular displacement measurements of the other three nodes.In the simulation, we imposed a false alarm rate to statisticallyinclude the system errors. Fig. 19 shows the true trajectories as asolid line and estimated trajectories by HMM (square), Kalman(circle), and particle filter (diamond) tracking techniques, at alow false alarm rate of 0.001. The histograms of particle filtertracking errors in the x and y directions are given in Fig. 20.The standard deviation of the tracking errors are 1.2 m in xand 1.1 m in y.

Fig. 21 shows the performance comparison of three differenttracking algorithms at different false alarm rates. We can seethat given a low false alarm rate, all the tracking schemesperform well, and HMM tracking can even achieve betterperformance than Kalman tracking. When the false alarm rateis increased, the performance of HMM tracking degrades fasterthan the other two. Its poor tracking robustness can be ascribedto the lack of a velocity estimation in its algorithm. Particlefilter tracking outperforms the other two schemes, but its per-formance improvement is less than 10% of that of Kalmantracking, due to the limitation of system resolution.

Table III summarizes the computation complexity and per-formances of the three tracking schemes. Note that, in theory,the computation complexity of the particle filter is higher thanthat of the Kalman filter, roughly by the number of the parti-

Fig. 21. Tracking errors of three tracking schemes given different false alarmrates. In each group of three: the left bar is the particle filter, the middle is theKalman filter, and the right is the HMM filter.

TABLE IIICOMPARISON OF BAYESIAN TRACKING ALGORITHMS

Fig. 22. Computation time per iteration in MATLAB with respect to differenttracking algorithms on the right and signal-processing techniques for eventdetections on the left.

cles; in this implementation, the computation cost is measuredby time for an iteration only seven times for 500 particles,as shown in Fig. 22(a). The Gaussian particle filter trackingperformance is sensitive to parameters, such as the number ofparticles, the observation probability density model, and theassumed input noise level. Choosing a proper observation dis-tribution model is the key factor in obtaining a decent particle


Fig. 23. Response signals at different detectors of the four sensor nodes in our experiment using the prototype design.

filter. Because of the added complexity, we chose the Kalmanfilter with grid approximation over the particle filter for our real-time implementation.

B. Experimental Results

Based on the simulation results, we built a prototype py-roelectric tracking system for one human target tracking. Inthe real-time tracking demo, only the 8-bit messages of eventindexes are transferred between slave sensor nodes, the masternode, and the host. To evaluate the system performance withrespect to different embedded signal processing techniques, wecollected raw response signals via serial cables from all sensornodes. Fig. 23 shows the signals when a human target walkedback and forth along the diagonal of the room. After the eventdigitization and registration, we can convert those temporal-spatial signals into angular displacements with respect to eachsensor node, shown in Fig. 24. By using the Kalman trackingscheme with grid approximation, the tracking results with threeevent detection techniques, namely windowed FFT, Kalman fil-ter, and sine filter, are given in Fig. 25. The standard deviationsof tracking errors for these three event detection schemes are0.75, 0.85, and 0.82, respectively.

Table IV shows the comparison of these three signal-processing algorithms in their performance and costs. The win-dowed FFT has the best performance in event detection but atthe expense of N log2(N) in computation cost. The compu-tation time of these three techniques in MATLAB is shownin Fig. 22. It can be seen that the 12-point sine filter requires

the lowest computation time. Therefore, we chose the bandpasssine filter for real-time embedded computation in a microcon-troller for event detection.

C. Discussion

The explicit advantages of human tracking with a DSNinclude better spatial coverage, robustness, survivability, andmodularity. The concept of distributing the computation to mul-tiple low-complexity nodes reduces requirements upon proces-sor and data storage. The use of the motion detectors helpsmaintain the low requirements on computation and communi-cation bandwidth. The characteristics of the pyroelectric sensorgive the system the ability to operate under all illuminationconditions.

The main motivation of developing geometric sensors intracking and identification is the study of reference structuretomography [19], which suggests that multidimensional fea-tures of a radiation source could be captured at an arbitrarylevel once there exist a set of base functions that structurallypose and numerically condition the reconstruction procedure.A general visibility design procedure has yet to be proposed,but many visibility-coding schemes have already been appliedin different coded-aperture imaging systems, from Hadamardcodes to pseudorandom codes [33], [34].

The ultimate goal of our project is to develop a wireless dis-tributed pyroelectric sensor network, which can track multiplehumans inside a room, while maintaining their identities. Gen-erally speaking, walker tracking and identification are indeed a


Fig. 24. Measured angular displacements from the four nodes. (a) Sensor node 1. (b) Sensor node 2. (c) Sensor node 3. (d) Sensor node 4.

Fig. 25. Tracking results with real sensory data with different event detec-tion schemes. (Circle) sine finite-impulse response, (square) windowed FFT,(diamond) Kalman IIR.

pair of coupled problems, which rely on target identification toreduce the mutual interference of targets. From (1), it can beseen that the source distribution ψ(r) is a hidden variable. Thissuggests that a comprehensive tracking strategy, i.e., a recursiveestimation of ψ(r, t), might be developed in an expectation-maximization scheme [35], [29]:

1) E step: to estimate ψ(r), after updating r(t);2) M step: to estimate r(t), after updating ψ(r).A multiple human tracking sensor system demands high mo-

tion detection resolution and an effective process of data objectassociation. The tracking node we used in experiments has a360 FOV, which is inefficient for tracking humans inside one

TABLE IVCOMPARISON OF EMBEDDED SIGNAL-PROCESSING ALGORITHMS

room. More efficient sensor FOV designs, which can improvemotion detection resolution and facilitate the process of dataobject association, are under study.

VII. CONCLUSION

In this paper, we present an implementation of a wirelessdistributed pyroelectric sensor system for human motion track-ing based on TI’s microcontroller MSP430149 and RF trans-ceiver TRF6901. The system consists of one host, one master,and four slave modules. The processing scheme comprisesevent detection/digitization/registration, motion inference, andtrajectory smoothing. Bandpass sine filtering and Kalman track-ing, based on grid approximation, were chosen in the real-time implementation for their simplicity and effectiveness. Wealso investigated the balance between optics and electronics,computation load distribution among host, master, and slavemodules, and algorithm tradeoff between performance and costfor real-time computation. Our future work includes multiplehuman tracking, multiple walker identification, and an effectiveintegration of multiple human tracking and identification byusing pyroelectric sensors.


REFERENCES

[1] A. S. Sekmen, M. Wilkes, and K. Kawamura, “An application of passivehuman-robot interaction: Human tracking based on attention distraction,”IEEE Trans. Syst., Man, Cybern. A, Syst. Humans, vol. 32, no. 2, pp. 248–259, Mar. 2002.

[2] F. Marzani, E. Calais, and L. Legrand, “A 3-D marker-free system forthe analysis of movement disabilities—An application to the legs,” IEEETrans. Inf. Technol. Biomed., vol. 5, no. 1, pp. 18–26, Mar. 2001.

[3] C.-C. Chang and W.-H. Tsai, “Vision-based tracking and interpretationof human leg movement for virtual reality applications,” IEEE Trans.Circuits Syst. Video Technol., vol. 11, no. 1, pp. 9–24, Jan. 2002.

[4] W. Hu, T. Tan, L. Wang, and S. Maybank, “A survey on visual surveillanceof object motion and behaviors,” IEEE Trans. Syst., Man, Cybern. C, Appl.Rev., vol. 34, no. 3, pp. 334–352, Aug. 2004.

[5] R. T. Collins, A. J. Lipton, H. Fujiyoshi, and T. Kanade, “Algorithmsfor cooperative multisensor surveillance,” Proc. IEEE, vol. 89, no. 10,pp. 1456–1477, Oct. 2001.

[6] C. Stiller and J. Konrad, “Estimating motion in image sequences,” IEEESignal Process. Mag., vol. 16, no. 4, pp. 70–91, Jul. 1999.

[7] F. Xu, X. Liu, and K. Fujimura, “Pedestrian detection and tracking withnight vision,” IEEE Trans. Intell. Transp. Syst., vol. 6, no. 1, pp. 63–71,Mar. 2005.

[8] A. L. Cour-Harbo, “Geometrical modeling of a two-dimensional sensorarray for determining spatial position of a passive object,” IEEE SensorsJ., vol. 4, no. 5, pp. 627–642, Oct. 2004.

[9] C.-Y. Chong and S. P. Kumar, “Sensor networks: Evolution, opportunities,and challenges,” Proc. IEEE, vol. 91, no. 8, pp. 1247–1256, Aug. 2003.

[10] D. Li, K. D. Wong, Y. H. Hu, and A. M. Sayeed, “Detection, classification,and tracking of targets,” IEEE Signal Process. Mag., vol. 19, no. 2, pp. 17–29, Mar. 2002.

[11] F. Zhao, J. Liu, J. Liu, L. Guibas, and J. Reich, “Collaborative signal andinformation processing: An information-directed approach,” Proc. IEEE,vol. 91, no. 8, pp. 1199–1209, Aug. 2003.

[12] J. C. Chen, K. Yao, and R. E. Hudson, “Source localization andbeamforming,” IEEE Signal Process. Mag., vol. 19, no. 2, pp. 30–39,Mar. 2002.

[13] T. M. Hussain, A. M. Baig, T. N. Saddawi, and S. A. Ahmed, “Infraredpyroelectric sensor for detection of vehicular traffic using digital sig-nal processing techniques,” IEEE Trans. Veh. Technol., vol. 44, no. 3,pp. 683–689, Aug. 1995.

[14] E. A. Prigge and J. P. How, “Signal architecture for a distributed magneticlocal positioning system,” IEEE Sensors J., vol. 4, no. 6, pp. 864–873,Dec. 2004.

[15] S. B. Lang, “Pyroelectricity: From ancient curiosity to modern imagingtool,” Phys. Today, vol. 58, no. 8, pp. 31–35, Aug. 2005.

[16] A. Hossain and M. H. Rashid, “Pyroelectric detectors and their ap-plications,” IEEE Trans. Ind. Appl., vol. 27, no. 5, pp. 824–829,Sep. 1991.

[17] M. Shankar, J. Burchett, Q. Hao, B. D. Guenther, and D. J. Brady, “Char-acterization of pyroelectric infrared detectors for human tracking,” Opt.Engr., submitted for publication.

[18] D. J. Brady“Multiplex sensors and the constant radiance theorem,” Opt.Lett., vol. 27, no. 1, pp. 16–18, Jan. 2002.

[19] D. J. Brady, N. P. Pitsianis, and X. Sun, “Reference structure tomogra-phy,” J. Opt. Soc. Amer. A, Opt. Image Sci., vol. 21, no. 7, pp. 16–18,Jul. 2004.

[20] J.-S. Fang, Q. Hao, D. J. Brady, B. D. Guenther, J. Burchett, M. Shankar,N. Pitsianis, and K. Y. Hsu, “Path-dependent human identification usinga pyroelectric infrared sensor and Fresnel lens arrays,” Opt. Exp., vol. 14,no. 2, pp. 609–624, Jan. 2006.

[21] U. Gopinathan, D. J. Brady, and N. P. Pitianis, “Coded aperture for effi-cient pyroelectric motion tracking,” Opt. Exp., vol. 11, no. 18, pp. 2142–2152, Nov. 2003.

[22] Y. Zheng, D. J. Brady, M. E. Sullivan, and B. D. Guenther, “Fiber-optic localization by geometric space coding with a two-dimensional graycode,” Appl. Opt., vol. 44, no. 20, pp. 4306–4314, Jul. 2005.

[23] M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A tutorialon particle filters for online nonlinear/non-Gaussian Bayesian tracking,”IEEE Trans. Signal Process., vol. 50, no. 2, pp. 174–188, Feb. 2002.

[24] J. H. Kotecha and P. M. Djuric, “Gaussian particle filtering,” IEEE Trans.Signal Process., vol. 51, no. 10, pp. 2592–2601, Oct. 2003.

[25] T. S. F. Gustafsson and P.-J. Nordlund, “Marginalized particle filters formixed linear/nonlinear state-space models,” IEEE Trans. Signal Process.,vol. 53, no. 7, pp. 2279–2289, Jul. 2005.

[26] Y. Zou, “Coverage-driven sensor deployment and energy-efficient infor-mation processing in wireless sensor networks,” Ph.D. dissertation, DukeUniv., Durham, NC, Dec. 2004.

[27] G. F. Franklin, J. D. Powell, and M. Workman, Digital Control ofDynamics Systems. Reading, MA: Addison-Wesley, 1998.

[28] L. R. Rabiner, “A tutorial on hidden Markov models and selected appli-cations in speech recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257–286,Feb. 1989.

[29] K. J. Molnar and J. W. Modestino, “Application of the EM algorithmfor the multitarget/multisensor tracking problem,” IEEE Trans. SignalProcess., vol. 46, no. 1, pp. 115–129, Jan. 1998.

[30] Q. Hao, “Mulitple human tracking and identification with pyro-electric sensors,” Ph.D. dissertation, Duke Univ., Durham, NC, May 2006.[Online]. Available: http://www.ee.duke.edu/~qh/

[31] The Texas Instrument Website. (2005). [Online]. Available: http://www.ti.com/

[32] The Crossbow Websitehttp. (2005). [Online]. Available: http://www.xbow.com/

[33] M. Harwit and N. J. A. Soane, Hadamard Transform Optics. Reading,MA: Academic Press, 1979.

[34] A. Busboom, H. D. Schotten, and H. Elders-Boll, “Coded aperture imag-ing with multiple measurements,” J. Opt. Soc. Amer. A, Opt. Image Sci.,vol. 14, no. 5, pp. 1058–1065, May 1997.

[35] L. Frenkel and M. Feder, “Recursive expectation-maximization algo-rithms (EM) for time-varying parameters with applications to multipletarget tracking,” IEEE Trans. Signal Process., vol. 47, no. 2, pp. 306–320,Feb. 1999.

Qi Hao received the B.Eng. and M.Eng. degreesfrom Shanghai Jiao Tong University, Shanghai,China, in 1994 and 1997, respectively, and the Ph.D.degree from Duke University, Durham, NC, in 2006,all in electrical engineering.

His research interests include wireless sensor sys-tems for multiple human tracking and identification.

David J. Brady (M’91–SM’05) received the B.A.degree in physics and mathematics from MacalesterCollege, Saint Paul, MN, and the M.S. and Ph.D.degrees in applied physics, both from CaliforniaInstitute of Technology, Pasadena.

He is the Addy Family Professor of electricaland computer engineering in the Pratt School ofEngineering at Duke University. He joined the Dukefaculty in 2001 and directed the Fitzpatrick Insti-tute for Photonics from 2001 to 2005. He currentlyleads the Duke Imaging and Spectroscopy Program

(www.disp.duke.edu). He was on the faculty of the University of Illinois from1990 to 2001.

Bob D. Guenther received the B.S. degree inphysics/mathematics from Baylor University, Waco,TX, in 1960 and the M.S. and Ph.D. degrees, bothfrom the University of Missouri, Columbia, in 1963and 1968, respectively.

He is currently an Adjunct Professor with thedepartments of Physics and Electrical and ComputerEngineering, Duke University, Durham, NC.

John B. Burchett received the Ph.D. degree in elec-trical and computer engineering from Duke Univer-sity, Durham, NC.

His research interests include developing low-costbiometric sensor networks for simultaneous trackingof multiple humans.


Mohan Shankar received the Bachelor’s degree inelectronics and communications engineering fromNational Institute of Technology, Karnataka, India,in 2002 and the Master’s degree in 2004, from DukeUniversity, Durham, NC, where he is currently work-ing toward the Ph.D. degree.

Steve Feller received the B.S. and M.S. degrees inelectrical systems engineering, both from the Uni-versity of Arkansas, Fayetteville, in 1997 and 1999,respectively.

He is a Research Associate with Fitzpatrick Insti-tute for Photonics, the Department of Electrical andComputer Engineering, Duke University, Durham,NC. His research interests include developing low-cost biometric sensor networks and video trackingsystems.

Documents

Human Tracking With Wireless Distributed Pyroelectric Sensors