SENSOR BASED MOTION TRACKING SYSTEM FOR

HUMAN COMPUTER INTERACTION

-i-

SENSOR BASED MOTION TRACKING SYSTEM FOR

HUMAN COMPUTER INTERACTION

Undergraduate graduation project Report submitted in partial fulfillment of

the requirements for the Degree of Bachelor of Science of Engineering

in

Department of Electronic & Telecommunication Engineering University of Moratuwa

Supervisors Eng. Kithsiri Samarasinghe Dr. Ajith Pasqual

Project Group

S.K.C.L. Fernando 030106 H.A. Fonseka 030112 I.S. Godage 030120 J.G.M. Prabath 030293

31st October, 2007

-ii-

Approval of The Department of Electronic & Telecommunication Engineering

…..………………………………………. Head, Department Of Electronic & Telecommunication Engineering

This is to certify that we have read this project and that in our opinion it is fully adequate, in cope and quality, as an Undergraduate Graduation Project. …….…………………….. …….……………………..

Co-Supervisor Supervisor Members of the examining committee Name Signature 1. …….……………………….. 2. ..…………………………….. 3. ..…………………………….. 4. .…………………………….. 5. …………………………….. Date: …….……………………………..

-iii-

Abstract

SENSOR BASED MOTION TRACKING SYSTEM FOR HUMAN COMPUTER INTERACTION

S.K.C.L. Fernando, H.A. Fonseka, I.S. Godage, J.G.M. Prabath Eng. Kithsiri Samarasighe, Dr. Ajith Pasqual

Keywords: Accelerometers, Magnetic field strength sensors, Bluetooth wireless, digital filtering, mathematical mapping, Human Computer Interaction (HCI)

An accelerometer and magnetic field sensor based motion tracking system that

maps a user’s 6 degrees of freedom hand movements to planer movements on a computer screen and control signals such as “Click”, “Right Click”. The concept of free space motion tracking can be found in many applications such as inertial navigation systems, computer games, toys, 3D graphic manipulations and certain bio medical applications.

At the end of the project, a number of possible applications are introduced.

Unlike the most popular camera tracking and image processing method, here, accelerometers in 3 perpendicular directions and a magnetic field sensor is used. This gives rise to advantages such as elimination of need of using cameras, independence of the lighting condition, not being restricted to a specific area of use and ability of on board processing due to the absence of heavy image processing. Data from the sensors are converted to digital, filtered and processed onboard using relevant mathematical equations. Special algorithms are implemented to give the user the freedom of using the device at any orientation he/ she wishes without getting restricted to a specific hold. Also, the control signals (Click, Right click and Double Click if being used as a mouse) are implemented without using separate buttons; they are also implemented by analyzing the acceleration profiles. The device will communicate with the PC directly via Bluetooth.

The ultimate goal was to make the choice of input signals in the hardware

level by imitating the ordinary mouse movements adhering to its communication protocol, so that no additional driver implementation is necessary. The developed system is capable of detecting the three angles roll, pitch and yaw 1 degrees accuracy and mapping the accelerations into two dimensional displacements and driving the cursor pointer. In the 3-D space it’s possible to draw 3-D lines and visualize it. As a PC mouse it can accurately identify the control signals by hand gesture.

-iv-

To our parents and teachers

-v-

Acknowledgments

Our project was a success with thanks to many persons.

First of all we would like to extend our gratitude to Eng. Kithsiri Samarasinghe and

Dr. Ajith Pasqual, our project supervisors for their continued support, encouragement

and invaluable advice.

We would also like to thank, all other staff members for their time and effort rendered

to make this project a reality.

A special word of thanks to the non-academic staff of the department, who has been a

great support all through out this exercise

Also not forgetting the SLIC, who has agreed to provide us with the funding, and

various electronic component manufacturers and vendors who have provided us with

resources contributing towards the successful completion of our project.

-vi-

Table of Contents

1. Introduction .........................................................................................1

1.1 Human gestures and hand motion...................................................................1

1.1.1 Human gesture tracking ......................................................................2

1.1.2 Importance of gesture tracking ...........................................................2

1.2 Present methods of gesture recognizing .........................................................3

1.2.1 Image processing ................................................................................3

1.2.2 Single camera......................................................................................3

1.2.3 Stereo cameras ....................................................................................3

1.2.4 Acceleration sensors ...........................................................................4

1.3 Difficulties faced in gyro based systems ........................................................4

1.4 Researches on gyro-less systems ....................................................................5

1.5 Our method of gyro-less system .....................................................................5

2. System Implementation.......................................................................6

2.1 System Overview............................................................................................6

2.2 Signal Acquisition on analog section..............................................................7

2.2.1 3-Axis accelerometer signal acquisition .............................................8

2.2.2 Low pass filtering of accelerometer data ..........................................10

2.2.3 3-Axis magnetometer signal acquisition...........................................10

2.2.4 Instrumentation amplifier..................................................................13

2.2.5 Set / Reset Strap design.....................................................................13

2.3 Digital Section - Digitization, Digital filtering and wireless section............17

2.3.1 Microcontroller .................................................................................18

2.3.2 Adaptive filtering of the 3-axis accelerometer signals .....................24

2.3.3 Bluetooth module..............................................................................27

2.3.4 3.3V Internal Charge Pump RS232 Line Driver...............................28

2.3.5 Low Dropout Power Supply smoothing ...........................................28

2.4 Scaling down the design ...............................................................................30

2.5 Mathematical model......................................................................................33

2.5.1 Overview...........................................................................................33

2.5.2 Intake of data.....................................................................................34

-vii-

2.5.3 Signal Conditioning ..........................................................................35

2.5.4 Initializing the device (calibration):..................................................36

2.5.5 Checking stationaryness: ..................................................................37

2.5.6 Calculation of angles: .......................................................................37

2.5.7 Mapping of accelerations (Transformation): ....................................39

2.5.8 Z axis accelerations and Control signal decisions: ...........................40

2.5.9 Displacement: ...................................................................................41

2.7.10 Mathematical Modeling ..................................................................43

2.6 PC subsystem................................................................................................46

2.6.1Problem with Physical RS232 ...........................................................46

2.6.2 Bluetooth Features ............................................................................46

2.6.3 Bluetooth Stack Architecture............................................................47

2.6.4 Virtual COM approach .....................................................................48

2.6.5 RFCOMM approach .........................................................................49

2.6.6 Exploiting the java virtual machine and .NET platform and make a

.NET class with similar features ................................................................51

2.6.8 Microsoft C# developed applications vs. Visual basic .....................56

3. Results and Analysis..........................................................................57

3.1 Development softwares........................................................................57

3.2 Demonstration softwares .....................................................................61

4. Discussion and conclusion.................................................................67

5. Future Work ......................................................................................69

Annex – A: Set/Reset Strap Overview .....................................................i

Annex – B: In-Circuit Programmer .......................................................ii

Annex – C: The Bluetooth Protocol Stack .............................................v

Annex – D: Windows MOUSE API ......................................................vii

-viii-

List of Figures Fig. 1.1 System Overview..............................................................................................1

Fig. 1.2 a MEMS System...............................................................................................4

Fig. 1.3 3-axis along with rotation direction..................................................................4

Fig. 1.4 Piecewise Integration........................................................................................5

Fig. 2.1 Modularization of the system ...........................................................................6

Fig. 2.2 Signal Acquisition of Accelerometer and Magnetometer sensor data .............7

Fig. 2.3 Accelerations with respect to 3-Axis accelerometer frame ..............................8

Fig. 2.4 MMA 7260 Module used in prototyping..........................................................9

Fig. 2.5 MMA 7260 PCB 3-D Model ............................................................................9

Fig. 2.6 MMA7260 Module...........................................................................................9

Fig. 2.7 Magnetic Needle Compass .............................................................................10

Fig. 2.8 Earth’s Magnetic Field distribution................................................................11

Fig. 2.9 Instrumentation Amplifier configuration using OP amp LM 324..................12

Fig. 2.10 A simplistic schematic of a set/reset circuit .................................................15

Fig. 2.11 Totem pole push pull MOSFET driver.........................................................15

Fig. 2.12 HMC1052 Magnetic field strength module schematic.................................16

Fig. 2.13 MMA1052 PCB 3-D model .........................................................................16

Fig. 2.14 MMA1052 model .........................................................................................16

Fig. 2.15 Functional bock diagram of the overall digital section ................................17

Fig. 2.16 Flow chart algorithm of the microcontroller routine ....................................21

Fig. 2.17 Flow chart algorithm of read sensor function...............................................22

Fig. 2.18 Typical A/D cycle timing in PIC18F2523....................................................23

Fig. 2.19Unfiltered acceleration with noise.................................................................24

Fig. 2.20 Zoomed view of unfiltered acceleration.......................................................24

Fig. 2.21 Onboard Adaptive Filter technique ..............................................................25

Fig. 2.22 Filtered Acceleration without noise..............................................................25

Fig. 2.23 Offset corrected accelerations ......................................................................26

Fig. 2.24 Typical data packet structure (25-bytes) ......................................................26

Fig. 2.25 Parani ESD 02 Module .................................................................................27

Fig. 2.26 Power Supply Circuit....................................................................................29

Fig. 2.27 Prototype Board (Complete Hardware)........................................................30

-ix-

Fig. 2.28 overall scaled down schematic .....................................................................31

Fig. 2.29 Final PCB’s (fabricated in china) .................................................................32

Fig. 2.30 Final Project as a complete product..............................................................32

Fig. 2.31 Overall mathematical process.......................................................................33

Fig. 2.32 Procedure flow..............................................................................................33

Fig. 2.33 raw data from the accelerometers .................................................................34

Fig. 2.34 results of “Bringing to zero” method............................................................36

Fig. 2.35 Magnetic output vs heading..........................................................................38

Fig. 2.36 Control Signal Acceleration profiles ............................................................40

Fig. 2.37 Control Signal Algorithm .............................................................................41

Fig. 2.38 Displacement plotting mouse application.....................................................41

Fig. 2.39 3-D Displacement .........................................................................................43

Fig. 2.40 Scope outputs of Simulink Model ................................................................43

Fig. 2.41 Overall Simulink block diagram...................................................................44

Fig. 2.42 Incoming data real-time plotting ..................................................................45

Fig. 2.44 supported layers within the Bluetooth stack.................................................47

Fig. 2.45 device discovery chain process.....................................................................51

Fig. 2.46 service discovery ..........................................................................................52

Fig. 2.47 device discovery and service discovery as a single process.........................53

Fig. 2.48 Stream Reader...............................................................................................54

Fig. 2.49 Buffered Reader............................................................................................54

Fig. 2.50 Overall Bluetooth process ............................................................................55

Fig. 2.51 C# compiler ..................................................................................................56

Fig. 3.1 RS232 data access ..........................................................................................57

Fig. 3.2 HyperTerminal shows the incoming data stream ...........................................58

Fig. 3.3 Sensor data viewer..........................................................................................58

Fig. 3.4 cursor mover ...................................................................................................59

Fig. 3.5 3-D Orientation viewer...................................................................................62

Fig. 3.6 3-D Orientation viewer (Open GL) ................................................................64

Fig. 3.7 2-D Mouse with pitch and roll........................................................................64

Fig. 3.8 Displacement with transformations ................................................................65

Fig. 3.9 Data Logger (Impact) .....................................................................................66

Fig. A.1 How Magnetic domains aligned in a HMC sensor...........................................i

Fig. B.1 JDM Programmer with ICSP support ..............................................................ii

-x-

Fig. B.2 Connection Diagram for ICSP programming .................................................iii

Fig. C.1 Bluetooth Protocol Stack .................................................................................v

Fig. D.1 Most common convention of x-y axis ...........................................................vii

Fig. D.2 Windows mouse emulating software............................................................viii

1

Chapter 1

1. INTRODUCTION

This system is a sensor based motion tracking system which can be used to detect the

position and orientation using accelerometers and magnetic field strength sensors.

Thus it can be named as a 6-Degree of Freedom Motion (6-DOF) tracking System.

Fig. 1.1 System Overview

Traditional Inertial Navigation systems [1] use accelerometers, magnetometers and

gyroscopes to get this 6-DOF accurately. This is a novel way of removing the need of

gyroscope [2] in the system modeling. Inertial navigation system can be used in many

applications such as local navigation for robotics applications, orientation

determination of a body, character animation design aids, 3-D modeling etc. It is a

low cost solution to inertial navigation system.

1.1 Human gestures and hand motion Human gestures and hand motion contributes immensely in our communication. So

important, it is now often used in non-verbal communication systems. For instance,

sign language totally uses hand motion and patterns to make communication possible

among who cannot speak. More over, it is used in signaling extensively as in

runaways for aircrafts where high noise prevents ordinary means of communication.

Gestures and motion form a study field on its own. It is a vast field and overlaps with

human behavioral analysis, psychology, Robotics, animation industry and bio-

mechanics.

-2-

Robotics is a field vastly inspired by the human motion and gestures. Nearly all

robotic arms are derived from human hand and its operating limits. Thanks to the in

depth studies on this, now people can even successfully guess what is going on a

persons mind just by looking at his movements and gestures. This indicates how tight

we are bond to motion.

Hand movements are a key feature in this arena. It is one of the most expressive parts

of our body. Due to this, animators need to capture this subtle yet major movement

without flaws [3]. So researches world wide have tried tirelessly to come up with a

successful solution to address this problem and so far several of methods have been

introduced.

In this report also, we wish to detail our method of tracking human motion using

accelerators and magnetic field sensors.

1.1.1 Human gesture tracking

For some time now, scientists and engineers tried to automate the process of

presenting and interpreting signs and human hand motion to be used in other

applications [4] as it contains a rich set of maneuvers and astounding number of

totally different patterns. So tomorrow, it is not just about pointing your finger but it

will be about controlling something [5] we never imagined, merely using your hand

motion.

1.1.2 Importance of gesture tracking

Gestures and motion are so close to human behavior it is often the parameter

which is examined in surveillance [6]. In animation, one of the hardest things is to

model human characters with ordinary inverse kinematics. Due to the fact that human

body flexibility, organic and natural bents are very unique to humans. Therefore, the

necessity of sophisticated motion tracking system arose. In order to cater that, so

called systems, based on image processing were introduced. Due to their inherent

limitations, the need for a more compact system grew inevitably. And that is where

MEMS based chips came into rescue.

-3-

1.2 Present methods of gesture recognizing

1.2.1 Image processing

For decades motion tracking has been done using image processing. The process is

simple yet very much demanding in resources and computational power. First, easily

identifiable visual clues are attached to the subject placed in a specially created

environment, usually a mono colored enclosure. Then the subject is moved and the

recorded image is then exploited to dig in the visual motion clues. The process itself

was cumbersome and lot of time and resources have to be sacrificed in order to obtain

even a subtle motion.

1.2.2 Single camera

When a single camera is used, apart from the computational burden it inherits the lack

of depth and rotation of the motion. This consists of vital information of the

respective motion and hence user will not be able to make up for it. There fore

systems has been designed to capture the depth of images.

1.2.3 Stereo cameras

Most common methodology for acquiring depth is to use stereo cameras. To increase

the accuracy, in certain systems, not only two but several are employed. This

unavoidably contribute to system complexity and processing time making it very

much challenging to be used in a real time applications. Though with these cons, it

has been the choice of many, especially in animation industry. But still it lacks the

feature of capturing the one-point rotations where only rotation about a stationary

point is present.

Disadvantages

i. Demands lot of computational recourses/ power

ii. Need more external devices

iii. Has to face the cameras

iv. Errors in 3D visualizing

v. Specific environment and clothing needed

vi. Cannot give rotation at a stationary point

-4-

1.2.4 Acceleration sensors

MEMS (Micro electromechanical systems) is the technology of the very small, and

merges at the nano-scale into nano-electromechanical systems (NEMS) and

Nanotechnology. MEMS are also referred to as micro machines, or Micro Systems

Technology (MST). MEMS are separate and distinct from the hypothetical vision of

Molecular nanotechnology or Molecular Electronics. MEMS generally range in size

from a micrometer (a millionth of a meter) to a millimeter (thousandth of a meter). At

these size scales, the standard constructs of classical physics do not always hold true.

Due to MEMS' large surface area to volume ratio, surface effects such as

electrostatics and wetting dominate volume effects such as inertia or thermal mass.

Fig. 1.2 a MEMS System

1.3 Difficulties faced in gyro based systems Motion sensors has been tried out using MEMS backed circuits in past years. In

almost all systems, linear accelerometers were coupled with angular accelerometers

(Gyros). The main function of gyros is to calculate the angular displacement so the

motion even rotation only can be accurately recreated. Without gyros these systems

would fail.

Fig. 1.3 3-axis along with rotation direction

-5-

1.4 Researches on gyro-less systems

Lot of researches have been done on accelerometer based displacement and motion

tracking systems so far. But the usual noise level present in them prevents them to be

used in accurate applications. For calculations, measurements given out by these

sensors have to be integrated twice with respect to time. Since it is done in piece wise

manner, the drift introduced in each time window is getting accumulated and

eventually corrupt the calculations. This contamination of calculation is common to

all A/D converted systems. To minimize this effect, the time slices have to be very

small. In our case, we took every effort to do it in optimum limit that the converter

supports.

Fig. 1.4 Piecewise Integration

So as we can see, when dt increases the drift introduced by the calculation goes up.

More care has been taken to do the sampling at the optimum speed possible to

minimize the drift effect along with filtering on board.

1.5 Our method of gyro-less system Advantages of our system

i. Compact, Reliable

ii. Can mount several devices to increase accuracy

iii. Better in 3D applications

Possible applications of the system

i. Animation

ii. Gaming

iii. Robotic manipulators

iv. Mapping (path recording)

v. Signature verification

-6-

Chapter 2

2. SYSTEM IMPLEMENTATION

2.1 System Overview The system designed as a motion tracking system has several integrated modules.

Each module has a specialized function and all these modules interconnected to get

the final outcome can be shown below as a functional block diagram.

Fig. 2.1 Modularization of the system

These separate modules can be broadly categorized in to 4 main sections

Signal Acquisition section:

This section handles the process of acquiring the analog sensor output and subsequent

amplification to interface with a microcontroller A/D converter. This section further

divides into 2 main streams. i.e. 3-axis accelerometer data acquisition and 3-axis

magnetometer data acquisition.

Digitization, Digital filtering and wireless section:

This section is responsible for sampling of the amplified signals, applying moving

average adaptive filtering and wireless transmission of data through Bluetooth.

-7-

Mathematical modeling section:

Applies the Mathematical and statistical models to pre-conditioned digital data; derive

control signals for the application ‘virtual mouse’ to operate as a windows mouse

PC subsystem:

Includes all the application development for Bluetooth data access of the hardware

system.

2.2 Signal Acquisition on analog section This is the section which will be capturing the sensor data and modifies the captured

signal to a level which is possible to be digitized properly. The major tasks of the

system will be to deal with noise interferences and amplifying the milli-volt signal to

a volt level signal.

Fig. 2.2 Signal Acquisition of Accelerometer and Magnetometer sensor data

Signal acquisition is slightly different for the two main sensors used and hence

therefore subdivided into two sections.

-8-

2.2.1 3-Axis accelerometer signal acquisition

An accelerometer is needed for a motion tracking system mainly due to two reasons.

i. Obtain x, y, z displacements

ii. Calculate Roll and Pitch angles.

Obtaining the smallest possible acceleration from hand movement and mapped into

displacement is the ultimate goal on this section. To capture any small movement a

very good response is needed, after a thorough search an accelerometer which uses

MEMS (Micro Electro-Mechanical Systems) technology was chosen. The MEMS

accelerometers consists of a Mass at the centre of the sensor’s chip, which is

suspended by 4 Beams doped with Piezo-resistive material. When the sensor is

subjected to acceleration in any direction, the movement of the Mass causes the 4

Beams to deform and so change the resistance in the piezo material.

Initially we tested on a 2-axis digital accelerometer (ADXL 202) mainly because of

its nature of noise reduced PWM output. Even though the noise level is sufficient, the

sensitivity of the sensor was only 312mV/g and the results we obtained on capturing

small movements were not acceptable for our applications.

After going through various other accelerometers we decided on MMA 7260 provided

by Freescale Semiconductors [7]. It’s a 3-axis accelerometer and we can obtain the

acceleration with respect to 3 main axes. This analog accelerometer has 4 levels of

sensitivity ±1.5g/2g/4g/6g and we stick to the highest resolution (800mV/g) which

can be obtained by setting the sensitivity g-scale to 1.5g. Below is the top and bottom

view of the accelerometer which defines each axis direction.

Fig. 2.3 Accelerations with respect to 3-Axis accelerometer frame

-9-

Since the sensitivity we used was 800mV/g and when Vdd level is 3.3v, maximum

sensor deflection on ±1.5g scale is 0.45V – 2.85V. Thus no external amplification is

necessary for interfacing with a microcontroller ADC. First prototype was made up

with MMA7260 and designed as a sensor module, such that it can be integrated into

the overall system later.

Gsel11

Gsel22

Vdd3

Vss4

Sleep12

Z_out 13

Y_out 14

X_out 15

U1

MMA7260Q

GND

GND

+3.3v

1kR1

1kR2

1kR3

C1

C2

C3

GND

GND

GND

5kR4

+3 3v

123

P1

Power 123

P2

Output

+3.3v

GND

1 - x_out2 - y_out3 - z_out

Fig. 2.4 MMA 7260 Module used in prototyping

Fig. 2.5 MMA 7260 PCB 3-D Model

Fig. 2.6 MMA7260 Module

Special considerations when designing the circuit and PCB

i. Use of 0.1 µF capacitor on VDD to decouple the power source

ii. Physical coupling distance of the accelerometer to the microcontroller

should be minimal.

iii. Flag underneath package is connected to ground.

iv. Place a ground plane beneath the accelerometer to reduce noise, the ground

plane should be attached to all of the open ended terminals

v. Accelerometer and microcontroller should not be a high current path.

vi. PCB layout of power and ground should not couple power supply noise.

-10-

2.2.2 Low pass filtering of accelerometer data

Accelerometer Internal sampling frequency is 11 kHz and it should not create aliasing

errors with the A/D sampling rate. Also high frequency components of the amplified

signal are not necessary. Hence the amplified signal is passed through a low pass filter

before inputting to the PIC. This filter is placed in the digital section, quite close to

the PIC microcontroller in order to reduce interference that may have been picked up

in the board-to-board connector.

A 5 KHz RC low pass filter has been used, and by using 1K resistor and 0.1uF

capacitor the 3dB cutoff of RC low pass filter is around 4.3 KHz. Thus it blocks the

high frequency components and removes the effect of possible aliasing.

2.2.3 3-Axis magnetometer signal acquisition

A magnetic field sensor is needed for obtaining the yaw angle. To determine the

rotation around z-axis which is aligned with the earth’s gravity vector we cannot use

an accelerometer. Thus we went for the theory of a normal compass.

When we hear the term compass what usually comes to mind is a magnetic needle,

free to rotate in a plastic housing similar to the compass shown below.

Fig. 2.7 Magnetic Needle Compass

The needle aligns itself with the Earth’s north/south magnetic field lines to establish

the north direction. The horizontal direction of the Earth’s field is always pointing

north. This field has a magnetic intensity of 0.5 to 0.6 Gauss. In comparison, a 230 kV

transmission line, under average load, yields a 20 mGauss field at 50 feet.

-11-

Considering the needle is magnetic, objects containing iron, such as belt buckles, nails

in a table, automobiles and mineral deposits in the ground, can affect the needle’s

orientation. When measuring direction with an analog compass, objects like these

must be avoided to maintain accuracy.

The module we built can be described as a digital compass module because starting

from scanning of earth’s magnetic field is driven through electronic sensors. Also it

can be calibrated point to point and thus the magnetic disturbance problems on

different locations can be eliminated.

Honeywell HMC1052 is a high performance magneto resistive sensor design in a

single package [8]. The advantages of this sensor module is magneto resistive chips

positioned for orthogonal two-axis sensing, miniature size and low height profile in a

surface mount package and capable of sensing fields below 0.1 milli gauss having 1.0

mV/V/gauss Sensitivity.

Initially we used the 2-axis magnetic sensor design for yaw determination. 2-axis

design cannot be used in a platform where roll and pitch is changing because earth’s

magnetic field vector is 3-D.

Fig. 2.8 Earth’s Magnetic Field distribution

Thus a 3-axis digital compass design has to be designed which compensates the roll,

pitch tilts to a horizontal plane when calculating the x-heading and y-heading. (more

details on mathematical section)

-12-

Since HMC1052 having 1.0 mV/V/gauss sensitivity, the voltage on the bridge output

difference for earth’s magnetic field (0.5 – 0.6 Gauss) is around 500 – 600 µV range.

Interfacing with a microcontroller A/D it has to be amplified. We initially tested with

LM324 Operational Amplifier [9] for Cascade Inverting configuration, Results we

obtained was sufficient to interface with a PIC ADC. But the resolution was too low

as the output varies only between 1.65V to 1.96V.

Secondly we went for the Instrumentation Amplifier configuration from LM324. The

results we got were successful and the resolution was very good compared to previous

method. It gave the amplified bridge output difference around 2V to 2.8V

Fig. 2.9 Instrumentation Amplifier configuration using OP amp LM 324

For the instrumentation amplifier configuration we need 3 Operational Amplifiers per

one bridge. So for the final 3-axis design we need 9 operational amplifiers, which is

not a good design practice. So we searched for single on-chip instrumentation

amplifiers and found 2 commonly used amplifiers (INA114 and MAX4194) which

supports 10,000 times gain.

After testing with these two Instrumentation Amplifiers we found that INA114 is

giving a distorted signal in one direction on the bridge due to its operating voltage

range. So the final decision was to implement the signal amplification using

MAX4194.

-13-

2.2.4 Instrumentation amplifier

MAX4194 is a variable-gain precision instrumentation amplifier that combines Rail-

to-Rail® single-supply operation, outstanding precision specifications, and a high

gain bandwidth. It can operate on +2.7V Single-Supply [10].

This Instrumentation amplifier is packed with all necessary resistors for the proper

operation and only RG external resistor is used to adjust the gain.

Table 2.1 External Gain resistor selection

Reference voltage is given as 1.5V because the reference voltage of the A/D

conversion is 3.0V. The initial test results a voltage level of 2.0V to 3.0V deflection

for north to south direction; which is well above the required sensitivity.

2.2.5 Set / Reset Strap design

Honeywell’s Anisotropic Magneto-Resistive (AMR) sensors are fabricated with

Permalloy (NiFe) thin films that create changes in resistivity with respect to external

magnetic fields. These film materials are similar to magnetic recording tapes in that

strong magnetic fields can disrupt the magnetic domains of the film particles from a

smooth factory orientation to arbitrary directions. Accuracy and resolution of these

sensors will jam if a higher magnetic field is present. In order to measure magnetic

field strength precisely we need to manually set /reset the sensor element periodically.

-14-

The reasons to perform a set or reset on an AMR sensor are:

i. To recover from a strong external magnetic field that likely has re-

magnetized the sensor

ii. To optimize the magnetic domains for most sensitive performance

iii. To flip domains for extraction of bridge offset under changing temperature

conditions

To perform the application of set and reset magnetic fields on the AMR sensor

elements, Honeywell employs a coil co-located on the silicon die with the sensor.

This coil is a planar winding or “strap” of metal wound through the active area of the

elements so that an electrical current in the strap creates a corresponding magnetic

field into the sensor. The Set/Reset Strap Resistance is around 3.0 to 5.0 ohms in

HMC1052 and Set/Reset Strap Current is 0.4 to 4.0 amps. See Annex A for more

details.

By providing pulses of electrical current creates the needed magnetic fields to realign

the magnetic domains of the sensor resistive elements. The direction of the strap

currents also determines what is considered a “set” or a “reset” pulse.

A set pulse is defined as a positive pulse current entering the S/R+ strap connection.

The successful result would be the magnetic domains aligned in a forward easy-axis

direction so that the sensor bridge’s polarity is a positive slope with positive fields on

the sensitive axis result in positive voltages across the bridge output connections.

A reset pulse is defined as a negative pulse current entering the S/R+ strap

connection. The successful result would be the magnetic domains aligned in a reverse

easy-axis direction so that Sensor Bridge’s polarity is a negative slope with positive

fields on the sensitive axis result in negative voltages across the bridge output

connections.

-15-

Fig. 2.10 A simplistic schematic of a set/reset circuit

In our design we have a totem pole set/reset strap configuration which uses IRF 7509

MOSFET as the high current driver. A reset pulse is sent first, followed by a set pulse

a few milliseconds later. By shoving the magnetic domains in completely opposite

directions, any prior magnetic disturbances are likely to be completely erased by the

duet of pulses.

Fig. 2.11 Totem pole push pull MOSFET driver

Set/ Reset pulse is initiated from the micro-controller and a digital I/O drives the

IRF7509 MOSFET. In normal operation input to the IRF7509 should be high and

when a set pulse to be activated the input is pulled low for 20uS.

-16-

Instrumentation amplifiers, magnetic field sensor and set/reset circuitry is compacted

to fit on a single module and the prototype circuit was designed as below.

1

2

7, 8 Q1A

IRF7509

4

35,

6

Q1B1uF

C1

0.1uF

C2

220R3

GND

GND

+3.3v

GNDGNDGND

GND

+3.3v

-V4

IN+3 IN-2 OUT 6

REF 5

RG1

RG8 +V 7U2

MAX4194

-V4

IN+3 IN-2 OUT 6

REF 5

RG1

RG8 +V 7U3

MAX4194

68R4

68R5

GND

GND

+3.3v

+3.3v5kR1

vref

+1.5v

5kR2

+1.5v

+1.5v

Set/Reset active when logic 0

GND

1234

P1

PowerP

O

GND

+3.3v

set

set

vref

Gnd

_y1

S/R-8

Vdd

5G

nd_x

23

Gnd

_x1

9

x_out- 10

x_out+ 4

y_out- 7

y_out+ 2

S/R+6

U1HMC1052

Fig. 2.12 HMC1052 Magnetic field strength module schematic

The PCB 3-D model and the actual implementation is shown below

Fig. 2.13 MMA1052 PCB 3-D model

Fig. 2.14 MMA1052 model

Special considerations on PCB designing

i. It has been noticed that high current paths will influence the normal

magnetic fields. Thus when designing the PCB we managed to remove all

high current paths outside the sensor area.

ii. Sensor bridge voltage is in µV range, hence those PCB paths should be kept

very close to instrumentation amplifier and should avoid noise induce from

other PCB tracings.

-17-

2.3 Digital Section - Digitization, Digital filtering and wireless section In the analog section, it is illustrated how the sensor data are captured, amplified from

0-500 µV range to 0-3V range and subjected to noise filtering to attain a noise free

signal. Since our system utilizes a Personnel Computer (PC) to visualize, record and

perform further signal processing on the acquired signals it has to be converted to

digital domain and transmitted to a PC. Major components of the digital section can

be categorized as below.

i. Microcontroller

ii. Digital filtering

iii. Bluetooth module

iv. Internal charge pump RS232 line driver

v. Power supply circuit

A functional bock diagram of the overall digital section is illustrated below

Fig. 2.15 Functional bock diagram of the overall digital section

Microcontroller (PIC18F2523), Bluetooth module Parani ESD 02, Internal Charge

pump RS232 line driver is operation at 3.3V while a 3.0 reference voltage is

generated for PIC ADC reference and magnetometer reference voltages.

-18-

2.3.1 Microcontroller

The microcontroller used for digital section is a PIC18F2523 by microchip co-

operation [11]. The specifications of the microcontroller are shown in table below.

The main tasks of the microcontroller are

a. Sample the amplified analog signals at an appropriate sampling rate

b. Adaptive filtering of the 3-axis accelerometer signals

c. Multiplex 6 individual sensor data and create a transmit packet

d. Communicate with the Bluetooth device using RS232 protocol

e. Controlling the Set/Reset strap of magnetic sensor

Table 2.2 PIC18F2523 Special Feature Table

PIC18F2523 is a Nanowatt® microcontroller from microchip and its low power

operation (2.5V to 5.5V) is very useful for battery power applications. It has 10 input

channels on inbuilt Analog to Digital converter module, with a maximum resolution

of 12 bits. The sampling rate is noted as 100ksps which is very much sufficient for

-19-

our application. The analog section of our system consists of six channels, hence only

six A/D conversion channels of the PIC are necessary.

Sample the amplified analog signals

Microchips 16F flash MCU units can only be operate on a maximum of 20 MHz

external clock. i.e. 5 MIPS (Mega Instructions per Second) which is not sufficient for

our mathematical processing inside the microcontroller.

PIC18F2523 can be operated at 40 MHz internal phase locked loop and we have

slightly over clocked for 40.96MHz so that the Instruction cycle speed is 10.24 MIPS.

Also the PIC18F2523 has 8 x 8 Single-Cycle Hardware Multiplier which reduces lot

of computational power.

Since the accelerometer signal bandwidth is 11 KHz, according to the Nyquist

theorem the minimum sampling rate needed to avoid aliasing needs to be only 22

KHz. Despite this theoretical minimum usually a sampling frequency of nearly 10

times the maximum frequency is used in practice. Therefore we use a sampling

frequency of 640 KHz for each channel. Also to minimize the quantization error in

the process we use 12 bit resolution for sampling.

Resolution of the voltage 4

12

1033.712

0.3

−×=−

=v

Initial testing confirmed that it’s sufficient to use 10-bit ADC. Increasing the ADC

resolution increases the packet size. Thus we are down sampling the 12-bit sampled

data into 10-bit sampled data by shifting 2-bits inside the microcontroller.

Calculated down sampled resolution of the voltage 3

10

1093.212

0.3

−×=−

=v

-20-

Extended USART and baud rate calculation

PIC18F2523 has an enhanced USART module which supports automatic detection

and calibration of baud rate. This feature is active only in Asynchronous mode while

the WUE bit is clear. It supports baud rates up to 230400bps.

Since we are capturing human motion data transmission has to be very fast while the

signals are accurate. After some research we found that at least 5ms refresh rate is

needed to reproduce the human motion. Thankfully the new technology PIC

Microcontroller PIC18F2523 having sampling rate of 100ksps are capable. When 10

bit resolution is used it requires 25 bytes to represent a single data sample of the 6

channel ADC data. Therefore

Data bytes needed per second at the PC end = 25

Data bits needed per 2 milliseconds at the PC end = 1002/825 =×

Total data rate of the system = sbits /100000

Thus 115200 bps was selected as the data transmitting speed.

In the filtering section of the microcontroller an adaptive filter is implemented and it

gives a real-time filtering output. Detailed description of the adaptive filtering is

discussed in the later section.

ICSP™ Programming

Size of the device is very important when selecting the microcontroller. PIC18F2523

28-SOIC package was selected for the final design. A prototype board was designed

for programming the microcontroller on a PIC Programmer and later in the final

design we are using the standard ICSP™ High voltage programming mode [12]. For

this programming purpose we have designed a special PCI Programmer based on

standard JDM programmer which supports ICSP™. See Annex B for further details.

5 lines (Vdd, MCLR, Vss, PGD and PGC) have to be placed on the programming

port.

-21-

Microcontroller block diagram

A flow chart which illustrates the algorithm in the microcontroller is shown below

Fig. 2.16 Flow chart algorithm of the microcontroller routine

Natively built USART port of the PIC18F2523 microcontroller is used for serial

communication with the Bluetooth device. This is made possible as the Bluetooth

module supports wired RS-232 communication. This is discussed in detail in the

section for Bluetooth.

-22-

Controlling the set/reset strap includes a set pulse followed by a reset pulse every time

the device switches ON. This pulse is valid only for 20uS time.

Read Sensor data {} and moving average methods is responsible for most of the

important tasks carried out by the microcontroller. Therefore a detailed flow chart of

that function is shown in Fig. xx for better understanding

Fig. 2.17 Flow chart algorithm of read sensor function

Since A/D conversion is done in a recursive loop it is important to calculate the

overall conversion time and decide the dead zone time for switching between

channels. It’s also important to measure this value in order to do the correct timing of

RS232 data transmitting.

-23-

ADC Conversion time calculations

Fig. 2.18 Typical A/D cycle timing in PIC18F2523

Single channel A/D conversion time = TACQT + TAD

(As given in the datasheet) TACQT = 2.96µS

Calculating the TAD

PIC18F2523 Operating frequency = 40.96MHz

TOSC = 0.0244×10-6 s

Conversion clock (64× TOSC) = 64×0.0244×10-6 s

= 1.5616×10-6 s

For 12-bit conversion = 13×1.5616×10-6 s

= 20.3001×10-6 s

Single channel A/D conversion time = TACQT + TAD

= 2.96µS + 20.3001 µS

= 23.26 µS

Since we have to convert 6 channels total time = 6×23.26 µS

= 139.56 µS

By adding 6×20µS inter gap delays it will approximately take 200µS to convert 18

bytes of digital data. As calculated previously sending 25 bytes of data will take about

1.5 – 2.0 ms. So in every 1.7ms - 2.2ms time duration a data packet of 25bytes is

received at the PC end which is acceptable to reproduce the human motion.

-24-

2.3.2 Adaptive filtering of the 3-axis accelerometer signals

In general accelerometers are noisy compared with to gyros. That has become the

main reason for not using accelerometers without sensor fusion for Inertial Navigation

systems. Through out the research carried out it has been evident the noise can be

reduced from a smooth power supply and a wireless communication subsystem.

RS232 will introduce a significant amount of noise due to the PC power supply noise.

However after reducing the applied noise by power supply smoothing still it has a

significant amount of noise. Noise on the unfiltered acceleration data is shown below.

0 500 1000 1500 2000 25000

100

200

300

400

500

600

700

800

900

1000

3 axis Acceleration

xyz

Fig. 2.19Unfiltered acceleration with noise

By zooming over the y acceleration it can be shown that the noise introduced was

disturbing the next step calculations.

500 1000 1500 2000 2500500

550

600

6503 axis Acceleration

xyz

Fig. 2.20 Zoomed view of unfiltered acceleration

-25-

By applying a adaptive moving average filter of window size 8, it’s possible to filter

out the unwanted noise at the hardware level while giving the same output data speed.

Fig. 2.21 Onboard Adaptive Filter technique

At the presence of each data set, most old data set of the moving average window is

removed and new data set is added and averaged along 8 data sets in the window

gives the respective output.

0 500 1000 1500 2000 25000

200

400

600

800

1000Acceleration

xyz

0 500 1000 1500 2000 25000

200

400

600

800

1000Acceleration with moving avarage filtering

xyz

Fig. 2.22 Filtered Acceleration without noise

After applying the adaptive filtering it is shown that the noise is reduced significantly.

Thus the adaptive filter is optimized and implemented in hardware.

-26-

The next step was to remove the offset in different axis voltages and normalize.

0 50 100 150 200 250 300 350 4000

200

400

600

800

1000Acceleration with moving avarage filtering

xyz

0 50 100 150 200 250 300 350 4000

200

400

600

800

1000Acceleration with moving avarage filtering and offset correction

xyz

Fig. 2.23 Offset corrected accelerations

By applying an offset correction factor for each axis the offset was corrected and it is

implemented in the hardware.

Data Packetization

Now the hardware is capable of produce filtered acceleration with offset correction.

The next task is to multiplex the 6 sets of data in to one packet for transmitting it the

PC. Multiplexing is handled by the microcontroller program in a roundly approach. It

selects one A/D channel, reads and stores on a local variable then the second

channel…. up to the 6th channel and do the adaptive filtering and offset correction.

Now it is necessary to create a packet structure to reconstruct the data in the proper

order at the receiving end. Since there are 6 channels, 2 bytes has to be used as stop

bits to differentiate between the channels. The process of packetization is illustrated in

the figure below.

Fig. 2.24 Typical data packet structure (25-bytes)

-27-

T – Tab, Cr – Carriage Return, Lf – Line feed

Each set of acceleration and magnetic data is 3 bytes representing 10-bit ADC value.

2.3.3 Bluetooth module

For Bluetooth communication there are several companies offering Bluetooth

embedded modules. Since it should be fully customizable in hardware level, small

enough to fit & affordable we are using Parani ESD 02 devices by Initium

Corporation [13]. The Bluetooth module and the PIC microcontroller can

communicate with each other serially using RS232 protocol at a baud rate of 115200.

To configure and initialize the Bluetooth module is done with the aid of a custom

made Bluetooth development board. The development board was made by us

according to the specifications given in the Parani-ESD 02 manual.

Special Features:

Class 2 Bluetooth

RF range up to 30m

Power supply 3.3v

USART serial interface

Approved by FCC, CE Fully Bluetooth qualified Fig. 2.25 Parani ESD 02 Module

AT Commands

AT command set is the factory standard language for controlling modems. The AT

command set was developed by Hayes and is recognized by virtually all personal

computer modems. Parani ESD 02 provides the extended AT command set to control

and configure the serial parameters and Bluetooth connection.

AT Response

Parani ESD 02 replies to AT commands with 4 kinds of message, ‘OK’, ‘ERROR’,

‘CONNECT’ and ‘DISCONNECT’.

-28-

When the microcontroller starts, Bluetooth initialize command is sent followed by a

Bluetooth discovery active mode set. This is done by sending the AT command

AT+BTSCAN? Then any Bluetooth device can search and discover the Bluetooth

module. AT responses are captured in order to confirm that the operation occurred

successfully.

2.3.4 3.3V Internal Charge Pump RS232 Line Driver

For debugging and backup reasons we thought of implementing a RS232 driver on the

final circuit. Due to the Electrolytic capacitor usage for the ordinary RS232 line driver

MAX232 was not used. We are using MAX3233 Line Driver IC having an internal

charge pump circuitry [14] and it can operate on 3.3V to 5.5V range. No Electrolytic

capacitors are needed and hence the overall circuitry was compact.

2.3.5 Low Dropout Power Supply smoothing

Unlike in other applications this device operates on 3.3V DC supply. Since the

supply voltage has to be smoothed for proper operation of digital circuitry a switching

mode regulator has to be used. Many regulators has a minimum operating voltage

rating. i.e. for LM7805 to operate on it’s rated specifications should supply 6.5V or

above. So the dropout voltage is 1.5V.

In our device the required voltage is 3.3V and when added a dropout voltage around

1.5V it comes around 4.9V. This gives several disadvantages. We need to find a

power source which is capable of giving 4.9V. When considering Rechargeable

batteries we have to use 5×1.2V batteries in order to supply 4.9V. Thus the power

supply is bulky. Another reason is that dropout voltage (1.5V) is directly proportional

to thermal dissipation and hence the energy loss becomes high.

-29-

IN1

GN

D2

EN3

OUT 5

NR/FB 4

U1

TPS73133

IN1

GN

D2

EN3

OUT 5

NR/FB 4

U2

TPS73130

12

P1

Power

GND

+3.3v

1uF

C1

GND

10nFC2

GND

+3.0v

10nFC3

GNDGND GND

5kR1

+3.0v

GND

+1.5v

5kR2

IN1

GN

D2

EN3

OUT 5

NR/FB 4

U3

TPS73133

+3.3av

10nFC4

GNDGND

100R3

+3.3v

D1Pow

GND

Fig. 2.26 Power Supply Circuit

A solution was identified for all these facts, to use a ultra low dropout regulator which

can regulate up to 150mA. The dropout voltage is only 30mV for the TPS731XX

series regulators [15].

Two 3.3V regulators (TPS73133) were used in order to reduce the power supply noise

occurred on Accelerometer supply. Separate 3.3V power supply was given to the

accelerometer and for rest of the circuits operating on 3.3V there is a separate power

regulator. Using TPS73130 regulator IC 3.0V reference voltage is generated in order

to supply the reference voltages of the Microcontroller ADC and MAX4194’s

reference voltage 1.5V. For visual indication a LED is used on the 3.0V regulator

output.

We use a 3.6V NiMH rechargeable battery as our main power source. Overall circuit

operates below 120mA and thus we can run the system continuously for 2 hours

without recharging. The charge capacity of the NiMH battery is rated as 210mAH.

-30-

2.4 Scaling down the design Considering the main objective of the overall design is a wearable motion tracking

system. So it has to be small enough, compact and no wires coming out of the box.

The prototype board we worked on until the last month was a complete full

functioning system.

Fig. 2.27 Prototype Board (Complete Hardware)

In order to achieve the final goal we had to reduce the size ergonomically while

preserving all the functions of the circuit. In the prototype board there were separate

modules for accelerometer, magnetometer and wireless sections. But in the final

design we thought of putting everything together and further scale down the design.

-31-

Fig. 2.28 overall scaled down schematic

+3.3

v S1 SW-P

B

GN

D

rst

rst

GN

D

+3.0

v

GN

D12

Y1

XTA

LG

ND

GN

D

GN

D

GN

D

Set

/Rese

t act

ive w

hen

logi

c 0

ICSP H

eader

1 -

MCLR

2 -

Vdd

3 -

Vss

4 -

PG

D5 -

PG

C

+3.3

v

GN

D

MCL

R/V

PP/R

E31

RA0/

AN

02

RA1/

AN

13

RA2/

AN

2/V

REF-

/CV

REF

4

RA3/

AN

3/V

REF+

5

RA4/

T0CK

I/C1O

UT

6

RA5/

AN

4/SS

/LV

DIN

/C2O

UT

7

VSS

8

OSC

1/CL

KI/R

A7

9O

SC2/

CLK

O/R

A6

10

RC0/

T1O

SO/T

1CK

I11

RC1/

T1O

SI/C

CP2

12

RC2/

CCP1

/P1A

13

RC3/

SCK

/SCL

14

RC4/

SDI/S

DA

15

RC5/

SDO

16

RC6/

TX/C

K17

RC7/

RX/D

T18

VSS

19V

DD

20

RB0/

AN

12/IN

T021

RB1/

AN

10/IN

T122

RB2/

AN

8/IN

T223

RB3/

AN

9/CC

P224

RB4/

AN

11/K

BI0

25

RB5/

KBI

1/PG

M26

RB6/

KBI

2/PG

C27

RB7/

KBI

3/PG

D28

U8

+3.3

av +3.3

av

Gse

l11

Gse

l22

Vdd

3

Vss

4

Slee

p12

Z_ou

t13

Y_o

ut14

X_o

ut15

U4

MM

A72

60Q

1

2

7, 8

Q1A

IRF7

509

4

3 5, 6

Q1B

GN

D

GN

D

+3.3

v

GN

DG

ND

GN

D

GN

D

+3.3

v

-V4

IN+

3IN

-2

OU

T6

REF

5

RG1

RG8

+V7

U6

MA

X41

94

-V4

IN+

3IN

-2

OU

T6

REF

5

RG1

RG8

+V7

U7

MA

X41

94

GN

D

GN

D

+3.3

v

+3.3

v

+1.5

v

+1.5

v

set

Gnd_y 1

S/R-

8

Vdd5 Gnd_x2 3

Gnd_x1 9

x_ou

t-10

x_ou

t+4

y_ou

t-7

y_ou

t+2

S/R+

6

U5

HM

C105

2

22pF

C10

5kR4

5kR5

1k

R8

1k

R9

1k

R10

1uF

C822

0

R11

68R1

2

0R1

3

68R1

4

0R1

5

0.1u

F

C9

0.1u

FC5 0.

1uF

C6 0.1u

FC7

22pF

C11

18F2

523

Sens

ors a

nd W

irele

ssD

2D

iode

1N

4148

W1

Jum

per

1 2 3 4 5

P2

ICSP

tx rx

8

34

20

6 1

7 19

C1+

13

C2+

15

GN

D18

C1-

14V

CC9

C2-

16

V-

17

V+

12

FORC

EOFF

10

FORC

EON

5

V+

11

INV

ALI

D2

U9

MA

X32

33EE

WP

+3.3

v

GN

D

+3.3

v GN

D

GN

D

GN

D

tx rx

1uF

C12

GND 1

VDD 2

Status 3RST 4

CTS5RTS6 TXD7 RXD8

U10

PRO

MI E

SD-0

2

rst

GN

D

GN

D +3.3

v

txrx

GN

D+3

.3v

+1.5

v

100

R16

D3 Bu

sy

GN

D

set

+3.3

v

usrin

put1

5kR18

usrin

put1

+3.3

v

usrin

put2

5kR19

100k

R17

GN

Dus

rinpu

t2

5kR75kR6

gsel

1gs

el2

gsel

1gs

el2

S2

in-1

S3

in-2

vdd_

icsp

Gnd1Vcc21.5v3

s/r4z_out5

U11

HM

C105

2_z_

boar

d co

nnec

tor

-32-

In the final design we used all SMD (Surface Mounted Devices) and we were able to

reduce the size up to 42mm×48mm. It uses double sided component placement and

routing tracing/spacing is less than 8mil. Thus we had to go for special PCB

fabrication in China.

Fig. 2.29 Final PCB’s (fabricated in china)

High current paths are taken in to special consideration when routing the PCB

especially in magnetometer area. The complete product is shown below.

Fig. 2.30 Final Project as a complete product

Overall system is very compact, light weight and easy to use. When you turn the

system ON/OFF switch to the left side (ON) system is powered up and you can see a

Green LED Lighted. If the device is in operation a RED light can be seen blinking.

Now it’s ready to be connected by any Bluetooth application.

For charging, ON/OFF switch should be placed to OFF position and connect the

charging cable for about 30 minutes.

-33-

2.5 Mathematical model 2.5.1 Overview

The main requirement from this section is to condition the data obtained from the

hardware and process them, mainly to obtain the displacement on the computer screen

and the control signals such as click and right click.

The following figure summaries the whole mathematical process.

Turn on the

device

Signal Conditioning

Initializing the device

(calibration) Stationary?

Calculate yaw, pitch and roll

Mapping filtered values to reference directions

(Transformation)

Significant acceleration

in ref Z direction?

Control signal desicion

Displacement calculation

NO

YES

NO

YES

Fig. 2.31 Overall mathematical process

Following figure shows the alternative procedures that were taken in the attempt to

process data.

Fig. 2.32 Procedure flow

-34-

2.5.2 Intake of data

Raw data from the accelerometers was taken into the PC for processing via serial port

using “serial object” method available in Matlab®. Initially data from the serial port

was written to a text file before another program read that data for plotting. With time,

data from hardware was taken and processed real time using Matlab “Erase-Plot

method” and the “Movie method”.

0 500 1000 1500 2000 25000

100

200

300

400

500

600

700

800

900

1000

3 axis Acceleration

xyz

Fig. 2.33 raw data from the accelerometers

Erase-Plot method draws a new plot for each new sample. Hence it was slow.

The movie method prepares a small movie clip of 10 frames from the incoming data

and plays it. Hence it was not actually real time.

Due to these shortcomings, analysis was shifted to Simulink from Matlab code. There,

the RS232 block set which is available in xPC target was initially tried, but later an

easier method; that is implementing the earlier written serial object code as a Matlab

function using the “Matlab Function” block was done.

Simulink was quite faster since it uses S-functions that run directly on the Windows

kernel, unlike Matlab which runs on a Java Virtual Machine. But since it was not still

satisfactory for real-time movement analysis Mathematical modeling was

simultaneously done in Visual Basic® along with Matlab and Simulink.

-35-

2.5.3 Signal Conditioning

It could be seen from the above raw data figure and the below velocity and

displacement figures that it includes significant noise along with a propagating drift of

velocity and displacement. This meant that the mouse pointer is going to drift on its

own even when the kept stationary. Close observation showed that a shift in

accelerations also contributed in giving rise to the drifts of velocity and displacement

when getting integrated. In other words, the error in accelerations gets integrated

along with real data since the following simple equations are used for converting the

accelerations to displacements.

2

21 tatuS iii +=

tauu iii 21

1 += −

(Here t is the time interval between two samples)

Following are the approaches that were taken in order to condition data.

Averaging:

For each stream, acceleration data was buffered and averaged. But the problem here

was when a larger data set was averaged to suppress noise; the real accelerations also

got absorbed into the average making the program insensitive to the normal hand

movements.

The optimum values that was found was to read data every 50ms (this collects about

16 samples in average) smaller data sets were not enough to suppress noise. Only 1

output data set for 16 input data sets, hence lot of input data had lost variation.

This method also did not provide a solution for the shift and resulting drift problems,

since the average will also accommodate the shift ultimately.

Bringing the acceleration and velocity values to zero:

Each acceleration value’s difference to its previous value is checked. If that does not

exceed the threshold value, zero will be taken as the acceleration. By this the noise

and shifts in accelerations when the device is stationary are eliminated. Same process

is carried out for the velocities calculated using the above equations.

-36-

The effect of noise when real user movement accelerations are present is negligible.

Hence this method was used along with the adaptive filtering done in hardware for

data processing.

Fig. 2.34 results of “Bringing to zero” method

Note: Simulations were done using the Matlab code and VB.

2.5.4 Initializing the device (calibration):

For all applications of this “sensor based motion tracking system”, the magnetic field

sensor data needs to be calibrated since it heavily depends on the geographical

location at macro as well as micro level. At macro level it changes from place to place

on earth and at micro level it depends on the magnetic sources in the vicinity such as

CRTs.

Calibration requires the device to be rotated about its each axis each time the location

of use changes. During this exercise the maximum and the minimum values found for

each axis of the magnetic field sensor is recorded. Then these values are spanned from

0 to 1023. That is, the minimum value is mapped to 0 and the maximum value is

mapped to 1023 and uniform distribution is expected for the angles in between.

0 100 200 300 400 500 600 700-2

0

2

4

6Filtered Acceleration

xy

0 100 200 300 400 500 600 700-1

-0.5

0

0.5

1Filtered Velocity

0 100 200 300 400 500 600 700-1

-0.5

0

0.5

1Filtered Displacement

0 100 200 300 400 500 600 700-5

0

5

10Raw Acceleration

xy

0 100 200 300 400 500 600 700-500

0

500

1000Raw Velocity

0 100 200 300 400 500 600 700-1

0

1

2x 105 Raw Displacement

-37-

The two objectives of doing this are:

i. Eliminating the effect of any irregularity in the magnetic field in one

direction (due to magnetic sources) by normalizing

ii. Eliminating the range difference in the magnetic field between the maximum

and the minimum values from place to place.

After calibration, since all values are uniformly spanned from 0 to 1023 and brought

to a common platform, common equations can be used for analysis thereafter without

depending on the location.

2.5.5 Checking stationaryness:

For each sample stationaryness is checked by the following equation using the

smoothed accelerations:

ax2 + ay

2 + az2 = g2

A margin of ±0.1 g2 was given at implementation after a statistical analysis. Statistical

analysis also confirmed that above equation with the margins served the purpose; that

is the assumption of “hand will not move in constant velocity” is true for more than

98% of the time.

Note: testing was done using the VB program and Matlab Simulink model

2.5.6 Calculation of angles:

Calculation of Pitch and Roll:

As anticipated in the feasibility study, Pitch (rotation about Y-axis), and Roll (rotation

about X-axis) is calculated with the following equations using the smoothed data of

accelerometers.

Pitch = sin-1(X/g)

Roll = sin-1(Z/g)

After calculation it was seen that calculated angle varies in a range of ± 40 even when

stationary. Therefore a moving average method was implemented for the calculated

angle, taking the window size as 30 samples.

Note: testing was done using the Simulink® model and VB.

-38-

Calculation of Yaw:

Earth’s magnetic field: earth has a magnetic field of about 0.6 Gauss. At the equator

the field is almost horizontal and at poles its vertical opponent is very high.

Since Sri Lanka is situated close to the equator, the first approach was to design the

magnetic compass with only two axis laying perpendicular to each other in a

horizontal plane. Once this was implemented it could be seen that it gave correct yaw

angles when it is kept horizontal on a surface, but not with other pitch and roll angles.

This is obviously because the vertical component of the earth’s magnetic field is

significant.

The next approach was to design the 3-axis magnetic compass by adding another

sensor axis in the vertical direction. The below equations are used for calculating the

heading angles in the horizontal plane [16]

X-heading = X × cos(pitch) + Y × sin(roll) × sin(pitch) – Z × cos(roll) × sin(pitch)

Y-headingh = Y × cos(roll) + Z × sin(roll)

where X, Y and Z are magnetic field sensor values in the respective directions

After calculating the X-heading and Y-heading according to their values the quadrant

to which the azimuth angle belongs to can be determined.

Fig. 2.35 Magnetic output vs heading

-39-

Using the following algorithm the azimuth angle can be calculated.

If Heading for (Xh <0);

Azimuth = 180 - arcTan(Yh/Xh)

If (Xh >0, Yh <0);

Azimuth = - arcTan(Yh/Xh)

If (Xh >0, Yh >0);

Azimuth = 360 - arcTan(Yh/Xh)

If (Xh =0, Yh <0);

Azimuth = 90

If (Xh =0, Yh >0);

Azimuth = 270

After calibration, the “Done” button is pressed after keeping the device towards the

desired Yaw = 0 direction. The azimuth value at that point (say azimuth1) is stored in

the memory and subsequent yaw angles rotated from that position can be obtained as,

Yaw = azimuth - azimuth1

2.5.7 Mapping of accelerations (Transformation):

If the user holds the mouse the in a tilted orientation, still the movements should be

mapped on to the original reference axis chosen. Therefore the accelerations got from

the hardware undergo a transformation. The resulting values give the accelerations in

the reference directions.

[ ] [ ] [ ] )31(33)31(''' ××× =× referencetilted ZYXTZYX

Fixed angle rotation matrix transformation is used for this purpose:

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−++−

−=

αβγααβγαβγαγαβαβγαγαγαβγ

ββγβγ

CosCosCosSinCosSinSinCosSinCosSinSinSinCosSinSinSinCosCosCosSinSinSinCos

SinCosSinCosCosT XYZ

..............

..][

Note: testing was done using the block available in the Simulink aerospace block set

and later the matrix was written manually in VB.

-40-

2.5.8 Z axis accelerations and Control signal decisions:

The wearable device is designed and worn such that control signals (click, double

click and right click) correspond to z axis accelerations.

Click: A sudden forward and backward movement along Z axis.

Right Click: A sudden backward and forward movement along Z axis

Double click: Two clicks in close time duration

Fig. 2.36 Control Signal Acceleration profiles

Since a click and a right click cannot happen at the same instance an easier algorithm

was discovered.

Click: A negative peak exceeding a negative threshold is detected as a “click”. A

number of samples following that click detection instance are ignored using a counter.

By this method it’s possible to eliminate the positive peak that follows being detected

as a “right click”

Right Click: Similarly a positive peak exceeding a positive threshold is detected as a

right click. A number of samples are ignored to loose the negative peak that follows.

Double click: Another counter is implemented once a click is detected. If another

click is detected before that counter is expired it will be taken as double click. If not,

just click.

-41-

(Z acc > pos threshold) OR (Z

acc < neg threshold)

NO

YES, neg acc

YES, pos acc

Increment “Click” signal

counter

Send “right click” signal 1.2.3.4………………………………………………50

Counter “1”

1.2.3.4………………………………………………50

Counter “1”

1.2.3.4………………………………………………50

Counter “2”

Another “Click” signal?

Send “Double

click” signal

YES

NO

Send “click” signal

Fig. 2.37 Control Signal Algorithm

The counter values for eliminating the peaks that are following are 50. A time

duration of 100 samples are given for double click detection.

Note: this was tested by implementing in the VB program.

2.5.9 Displacement:

(1) Normal mouse application:

The conditioned and transformed data are used fro this. The following simple

equations are applied on the samples for the X and Y accelerations (Z accelerations

are used for control signal identification) Piecewise integration is done on the

samples.

2

21 tatuS iii +=

tauu iii 21

1 += −

Fig. 2.38 Displacement plotting mouse application

-42-

When the user has stopped moving (accelerations become zero) a counter starts

incrementing and at its value of 20, any velocity value that is present is brought down

to zero. By this, the drifts that are present due to constant end-velocity ( 1−= ii uu ) is

eliminated.

(2) Tilt angle mouse application:

Similar to the method used in a joystick, here the roll and the pitch angles are used to

move the mouse curser. The device can be moved in a range of roll ± 900 and pitch ±

900. These tilt angles are mapped on to the screen of 1024×768 pixels as follows.

Horizontal displacement: 900 512 pixels

Therefore multiplication factor = 512/90 = 5.69 ~ 6

Horizontal displacement = 512 + (pitch) × (6)

Vertical displacement: 900 384 pixels

Therefore multiplication factor = 384/90 = 4.27 ~ 5

Vertical displacement = 384 + (roll) × (5)

Since the tilts themselves do the job, no transformation is needed here.

Note: testing of this application was done using VB

(3) 3D motion tracking application:

Similarly as was used for “Normal mouse application”, applying the two equations on

all three axes would give the displacement in 3D space. Here too the transformation is

not needed and the angle data can be utilized to give the orientation in 3D space.

-43-

020

4060

80 -2000-1000

0

0

2000

4000

6000

8000

10000

3-D Displacement

Fig. 2.39 3-D Displacement

2.7.10 Mathematical Modeling

(1) Mathematical System in Simulink:

The whole mathematical model was implemented in Simulink as follows. It was easy

is analyze and troubleshoot using this since outputs could be visualized with

oscilloscopes and displays. The only problem regarding this was the slowness in data

intake and processing when analysis is done real-time.

Scope outputs: Below is the possible view from the Scope outputs.

Fig. 2.40 Scope outputs of Simulink Model

-44-

Fig. 2.41 Overall Simulink block diagram

-45-

(2) Mathematical System in Visual Basic:

An alternative was to be thought of since the data processing in Matlab and Simulink

was slow. The choice was Visual Basic due to three reasons.

i. Fast data intake and processing

ii. Easiness of programming

iii. Availability of various visual aids for analysis

Following figure shows a screen shot of the interface of the program that was used for

system analysis through out. This program and interface provides facilities for:

a. Reading and viewing data from serial port or Bluetooth

b. Calibrating

c. Conditioning and processing data according to the mathematical model

d. Viewing both mapped and unmapped accelerations and rotation angles

e. Viewing control signal responses (click, double click and right click) and

stationary ness

f. Viewing mapped displacement

Fig. 2.42 Incoming data real-time plotting

-46-

2.6 PC subsystem 2.6.1Problem with Physical RS232

Since we here, expects total independence from the computer where the data is

intended, a wired connectivity such as RS232 not a candidate. It is wired and makes it

difficult to make the most of our device. Therefore a wireless mean of communication

was finalized and it was Bluetooth.

2.6.2 Bluetooth Features

The major features of Bluetooth are:

Bluetooth is wireless and automatic. You don't have to keep track of cables,

connectors, and connections, and you don't need to do anything special to

initiate communications. Devices find each other automatically and start

conversing without user input, expect where authentication is required; for

example, users must log in to use their email accounts.

Bluetooth is inexpensive. Market analysts peg the cost to incorporate

Bluetooth technology into a PDA, cell phone, or other product at around $20

now, and say that it could fall to as little as $5 per unit.

The ISM band that Bluetooth uses is regulated, but unlicensed. Governments

have converged on a single standard, so it's possible to use the same devices

virtually wherever you travel, and you don't need to obtain legal permission in

advance to begin using the technology.

Bluetooth handles both data and voice. Its ability to handle both kinds of

transmissions simultaneously makes possible such innovations as a mobile

hands-free headset for voice with applications that print to fax, and that

synchronize the address books on your PDA, your laptop, and your cell phone.

Signals are omni-directional and can pass through walls and briefcases.

Communicating devices don't need to be aligned and don't need an

unobstructed line of sight.

Bluetooth uses frequency hopping. Its spread spectrum approach greatly

reduces the risk that communications will be intercepted.

-47-

2.6.3 Bluetooth Stack Architecture

The protocol stack makes up the core portion of the Bluetooth implementation. This

stack enables devices to locate each other and establish a connection. Through this

connection, devices can exchange data and interact with one another through various

applications.

The following image map shows the supported layers within the Bluetooth stack. To

link to topics that provide information about the elements in the image map, move

your pointer over the element, and then choose the element.

Fig. 2.44 supported layers within the Bluetooth stack

-48-

2.6.4 Virtual COM approach

Once the driver is installed to the respective Bluetooth module, a direct virtual COM

connection can be made manually. Then that COM can be used in our illustrative

applications as if it is a serial port connection. Although all data rates are supported by

this mean there are some problems.

Problem with virtual RS232

Device dependent

Devices are shipped with their drivers and little or nearly no functionality is exposed

to developers. So in custom applications like this, developers face real trouble. Even if

any modifications had done for a specific device, it would definitely have to be

altered for other devices for they do not share common features when it comes to

implementation. This fact does not serve promising for even testing applications with

several modules let alone commercial applications.

Driver specific

The virtual COMs are hard to hard code because it is very drives specific. For

instance, some modules may assign COM13 for serial port profile (SPP) but others

may take something totally different like COM22. This is again another tiresome

configuration each time when a new module is used. But once configured we can use

it without any trouble for analysis purposes.

Manual configuration

But all the pairing, service discovery, connection and disconnection have to be done

manually. This is not a good property in a deployable application since we need to

eliminate all these tiresome procedure and the product should address even people

who are not familiar with these settings.

Limited support for automation

Although afore mentioned methods are available in the software application they

provide, the support for developers is zero. No library or Software Development Kit

(SDK) is shipped along.

-49-

So programmers are expected to dig into the windows driver level tapping to invoke

windows Bluetooth stack and operations and still its support is limited for a couple of

dozens of devices and the number of devices supported can be found in Microsoft

website.

Tendency to lose packets

The reading of data is done in a buffer, yet the behavior exposed to the user is similar

to a stream reader where the data is pumped as a stream and if user does not use it at

right time the data is lost. This is not a desired feature of our application because there

are a lot of mathematical calculations going behind the curtain and there is chance that

the main program will fail to synchronize with the reading process. Hence what we

need is kind of a buffered reader wrapped around the stream reader (COM port) and

hence data will be available to be used even in presence of subtle synchronization

errors.

2.6.5 RFCOMM approach

RFCOMM (Serial Cable Emulation Protocol) is Bluetooth's adaptation of the

TS07.10 protocol. It serves as a base for COM port emulation facilities and derived

point-to-point protocols. Multiplexing and flow control between devices and

applications are also implemented here.

Inspiration – java micro-edition

Fortunately, sun java has developed its java micro edition for mobile phones through

out the world to support wireless connectivity. IrDA, Bluetooth support have been

extensively covered exploiting very low level commands and wrapping them nicely in

java methods giving users the required authority. Best part is that they do not limit

their device support by the name but with the stack they use, in a sense if the stack is

widely accepted like Microsoft, Widcomm, Bluesoleil and Toshiba then the written

program will not have any issues for the connectivity. Care has been taken by the sun

corporation to support all the major stacks available in the market. Sorceforge® then

had bundled all these nice features into one library called Bluecove[17] and it is

available under GNU license. It is an implementation of JAVA wireless support for

Bluetooth (JSR 82) [18].

-50-

Advantages

Device independent

Since this library wraps the virtual COM connection into a stream underneath in

collaboration with windows operating system and the device specific driver. All

Bluetooth modules work with windows OS can be accessible with this library

regardless of their connectivity, i.e. USB, RS232 or Firewire. Therefore user does not

have to worry over manual configuration.

Full automation capability

Not only this library supports all devices but it also exposes its generic overall

functionality in high level. Hence device discovery, device service discovery, service

connection, data acquisition and closing down can be initiated from the user program.

The really interesting part of this remarkable implementation is that, without this

marvelous library this task would have been an entire project on its own.

Reliable TCP/IP connection

Wireless connection has the inherent deficiency to lose packets owing to several

reasons. But ordinary virtual COM connection does not implement this immunity to

the stream connection; hence the developer has to make necessary measures to fight

that. But this implementation will not guarantee the reliability but some buffering to

avoid sync problems. This library implement the connection on TCP/IP protocol to

provide lossless data transfer within server to client.

Fast data transfer as in Actual COM port

Though all these wrapping and reliability enhancements, amazingly data rates are as

same as the wired counterpart. So the user can go up to maximum to his wish and still

enjoy a fast and lossless connection. In a application like this, which heavily relies on

the continuity and correctness of data this feature is priceless.

Supports authentication

Wireless connections come with the born with weakness of security since the data is

transmitted into free space and anybody within the range can easily intercept and use

it. So when you need to integrate security into your application you would have to

make use of special classes or make queries to the client. But with this system,

-51-

authentication is yet another nice feature on board. You can set to check the client

authentication and server authentication alike thus preventing other applications

secretly mining data transmitted by the system.

Virtual machine, executable creation problem

An application was developed employing Bluecove library (.JAR package –

developed by Intel) to connect to Bluetooth devices and enumeration services

available by them. Data were successfully acquired to the computer for analysis.

Although it supported all the functionalities we needed, yet it ran on Java Runtime

Environment (JRE). So the end user is supposed to have it onboard on his system.

Again we are not able to provide an executable which can be easily run with a click of

a button. There is commercial software which does this but it is not the solution we

were looking for since we were hoping to deploy a cost effective solution.

2.6.6 Exploiting the java virtual machine and .NET platform and make a .NET class

with similar features

Interestingly, JRE and .NET platform share similarities far beyond any other systems

in implementation. Therefore after thoroughly examining the methods and libraries

used in the Bluecove.jar, a comparable .NET library was built that supports our

functionalities.

Device discovery

In the application, we need to search for the sensor among other Bluetooth devices. In

order to achieve that we do a device search within the Bluetooth modules’ receiving

range. This process is a chain of continuous complex processes and it is shown in the

figure below as a state diagram.

Fig. 2.45 device discovery chain process

-52-

Device service discovery

Once the devices have been found then we need to isolate our device. For that matter,

client name property comes in handy. We disable the authenticity feature and request

the client name and we can obtain it as a string. The client name for our device is

hardcoded to be ‘VMMOUSE’. So with a help of easy string comparison we can

single out our device.

Then comes the service identifying process. Ideally, our device is supposed to have

only Serial port service without authenticity. Yet eliminating the margin of confusion,

we again check for the service and service status and takes proper measures to cater

that. Obviously, we can integrate more security and reliability to the application by

introducing a hand shaking process. For instance, inquire the client with a secret

question and checking the answer for that or introducing a unique header to data

packets etc.

The service discovering process is illustrated below.

Fig. 2.46 service discovery

-53-

Even though we discussed the two processes separately, they are not. In fact, service

discovery immediately follows the device discovery as given in the flow diagram

below.

Fig. 2.47 device discovery and service discovery as a single process

Pairing

Enumerating services only does not allow us to obtain data out of our device. To do

so, we need to pair with or connect to the device. As in manual connection, there is a

separate method to invoke connect method. Here also, the security features does

matter. As we have not enabled the security features in our client, this does not

impose problems in this case. But in this case, unlike a raw virtual port, this gives us a

reliable connection wrapped on top of the serial port connection. After pairing we are

able to read data continuously till the application closes.

-54-

Data acquisition

After the connection has been established we are ready to read data pouring into the

application. There are two methods to do this.

Stream reader

Fig. 2.48 Stream Reader

First approach was to use a stream reader. Client Bluetooth module has an inbuilt

buffer to hold data until it gets transmitted. The data is read from the stream reader is

byte wise. This mean does not cater for the speed at which the data flows in. The

connection is lossless; the data not read at once is tend to be buffered by the device

introducing a growing delay and eventually leads the connection to freeze and data

loss owing to buffer overflow in the client side. Do a method to tolerate reading

imperfections and read lot of data at a time was needed.

Buffered reader

Fig. 2.49 Buffered Reader

Buffered reader offers two nice features over stream reader which makes it best

suitable for our purpose. First, it buffers the data in the application side (not in the

device side) and it allows reading more data at a time contributing to free the buffer in

one read. So the device does not need to buffer its data on its buffer eliminating the

-55-

delay in earlier case. This finally gives the application an unbroken chain of data

without a delay.

Connection close

Once application is done with, the Bluetooth connection can be gracefully closed

using the provided ‘connection close’ method. This serves two purposes. First, it

allows to be connected to the device in same application without the need of closing

down. Second, it allows other applications to make connections via the Bluetooth

module because the module is free. Without closing down the stream if we close the

application, the connection will still be there and force us to manually bring down the

connection as it does not allow other connections to be formed till it is occupied.

2.6.7 Putting all together

Putting all together and make a dynamic link library to be used in higher level demo

applications. In order to address diverse users and developers, we thought of

providing them with a all in one dynamic link library with all methods to address all

the functionalities at the same time hiding the vital implementation coding.

Fig. 2.50 Overall Bluetooth process

-56-

This dll is a .NET dll. So it can be used easily with VB.NET, C#.NET and J#.NET.

These languages provide the easiest of programming and executables on windows

Operating systems.

In our demo applications also, this dll has been extensively used. Its portable and user

friendly

2.6.8 Microsoft C# developed applications vs. Visual basic

We use C# developed applications at the first phase of the project to access data. In

that phase we use the RS232 serial port with a baud rate of 9600 bps. (By using a

serial cable) in that case the applications work accurately.

But when we use the Bluetooth communication the C# developed applications began

to give troubles. In this phase of the development we use virtual com port with a data

rate of 115200 bps. In this case the link was lossy and its speed was higher than the

previous stage. The C# application was unable to handle those issues successfully.

But in the VB applications those issues were more successfully handled.

In the C# language the compiler compile the code into an intermediate language and

it is compiled into Native code by the JIT compiler.

Fig. 2.51 C# compiler

But in the VB the language compiler directly compile the code into the native code.

So that the handling the high baud rate virtual ports were easy in the Visual studio 6.0.

Language compiler 

Native code Code metadata 

JIT compiler

Common language runtime

Sauce code 

-57-

Chapter 3

3. RESULTS AND ANALYSIS

We carried out a number of applications to demonstrate various applications this

motion tracking system can be used.

Software usage

The softwares were used to access and process the data coming from the device.

Mainly these soft wares can be divided in to two types. They are

a. Development softwares

b. Demonstration softwares

3.1 Development softwares

We use deferent softwares throughout the development of the device. These softwares

are used to visualize the hardware behavior of the device. Such as noise in the devise

responsivity of the device, sensivity of the device etc.

RS232 data access

Fig. 3.1 RS232 data access

In the first phase we use a Real time RS-232 data capturing software. In this stage we

use 9600 bps. There are various approaches on reading the data from serial port.

i. Wait for the hardware to do the job, and then return to the application, so the

read operation waits until the reading is finished

ii. We don’t wait in a loop for reading the port; we use interrupts, so when the

port has something in an input buffer, the SerialPort component alarms us

via delegate methods, such as DataRecieved.

Device RS232 PC side app 

-58-

iii. By using a timer to read data from the input buffer. So when timer ticks we

can read data.

iv. In all the softwares we use a timer to read data. The reason for using that

method basically lies with the easiness to handle data in the application. By

using that method we introduce a data loss in to the communication link but

reduce the processing load on the application.

v. And we got data line by line (after every set of values device sends a new

line character so by using this new line character we were able to read data

line by line) . In the program each line was split into three separate strings

by using the tab character (‘/t’). Then those strings (group of characters) are

converted to integers. Those integers are processed to give the output.

Fig. 3.2 HyperTerminal shows the incoming data stream

Sensor data Viewer

Fig. 3.3 Sensor data viewer

-59-

How it operates-

Here “start” button is used to connect the device. After that by using the “read” button

we can see the incoming data.

In the above application we get the acceleration in the three axes (which are in hex

format) and from that data we calculate

Voltage output of the accelerometer circuit (back-calculation)

Reason for getting the minus voltage values is that we use middle of 3.3V as zero in

the back calculation process.

Acceleration in multiple of g (gravitational acceleration g=800mV)

And the acceleration in decimal integer (which is proportional to actual acceleration)

From that deduced values we were able get some ideas about the device output.

Data variations are acceptable. (Reliable output)

When apply gravitational field to every three axis angles all were near 1g.

When held each axis perpendicular to gravitational field all were near 0g.

Able to observe sign change in the acceleration when change upside down.

Find the fluctuation range at the output when stationary situations.

This fluctuation is reasonable. As we encode 0-3.3V using 10 digit binary number. So

that small fluctuation in the voltage will result a 10-15 fluctuation in decimal number.

Moving the Generic mouse cursor

Fig. 3.4 cursor mover

-60-

By using the incoming data from the screen we tried to map the device movements to

the windows mouse cursor.

At this stage we use simple motion equations to get displacement of the device from

the acceleration of the device. In this application we have measure displacement from

the middle of the screen. But we can use any point in the cursor drawing area as the

origin.

Here we only use acceleration in the X,Y directions only. Data from the other axis (Z)

is used to cancel the inclination effect on X,Y direction. (This transformation

information is given in the mathematical modeling)

Problems arose at this stage

Cursor moved with out any movements in the device. This was due to the noise. So to

give the user a more stable region around the zero point, we quantized all outputs

below a certain level to 0, For example, if the cutoff were abs(15), the intersection of

all numbers > -15 and all numbers < 15 defined the cutoff region

According to the device it can move any further with out any restriction. But the

cursor is limited to the screen area by the operating system. So when giving input to

the cursor we have to consider about that. So we stop further displacement

calculations when mouse cursor came to an edge. Otherwise it will calculate

displacement but no move in the cursor so when we need to change cursor position

we have to move the distant which is traveled with out cursor movements.

Bluetooth data access

After that we use a Bluetooth connection and access the data form the device. Reads

in data from accelerometer via Bluetooth dongle. To do that, we use a virtual port. In

this case we use 115200 bps. Here also we use the same applications. In this case

some issues arise. The data lose in the connection was a new problem here. Another

problem was with the high data rate.

-61-

To improve the system we change the buffer reading method in the application. In this

situation we read the whole data existing in the buffer at each reading. The input data

was put into a string (group of characters) then we split that string at each and every

new line character (‘/n’). So that we can get set of lines. Then we check the number

of characters in each line. If they contain all the transmitted data the line is again

splited at each and every tab character (‘/t’). If we found that the some characters are

missing the values in the previous correctly transmitted line are used for that line. So

that at each data intake we get sets of values. Then those values are averaged and the

average values are used in the calculations.

This method helped to compensate the data loss to a certain extend. When we read

data one line at time we tried to reduce the timer interval. But in this method reducing

the timer interval after a certain value gave negative results.

Normally the input buffer stores the data coming from the outside device or port. The

buffer is like a first in first out queue. So that when it over flows first received data is

get lost. As with every read we flush the buffer overflow is reduced. But if we

increase the timer interval that overflow will come in to action. So we have to reduce

that. And increasing timer interval decrease the real time nature in the system. But if

we reduced further the number of lines stored in the buffer will decrease and the

averaging becomes useless. So that we have to use a moderate timer interval in our

application we use 50ms as timer interval.

The observations from these softwares were used in developing demonstration

softwares and development of hardware.

3.2 Demonstration softwares

Here we use some softwares to demonstrate various functionality of the device. We

have developed an application to show the ability of the system.

3-D Orientation viewer

Here we used the roll, pitch and yaw angles to rotate a simple cube. The rotation angle

calculation capabilities are demonstrated by mimicking the device rotations in the 3D

object. (Cube)

-62-

In that application we have used one fixed 3-axis and another 3-axis attached to the

cube these axis systems are used to help the observer to understand about the rotation

around deferent axes.

For 3-D rendering there are two popular softwares.

DirectX SDK: Microsoft provided system development kit. This is OS specific. For

graphic rendering direct3D is used.

OpenGL: Open source software which is OS independent. It has implementations

available across many platforms including Microsoft Windows. OpenGL is hardware

independent:

The ability of OpenGL to work with several operating systems adds a positive effect

on it. We have used OpenGL and direct3D in our applications. But due to some

software bugs we were unable to rotate the object in the horizontal plane.

To develop this application we use DirectX SDK which is provided by Microsoft for

system development. That SDK helps to graphics in our application.

Fig. 3.5 3-D Orientation viewer

To do this we have to going to create a device. In short, a device is a direct link to

your graphical adapter. It is an object that gives you direct access to the piece of

-63-

hardware inside your computer. When defining a device we have to choose a certain

device type. Hardware type devices are very fast and give good quality graphics. But

most hardware don’t allow the access to graphical adapter. So we have to use

Software type here graphic rendering is on software (i.e. processor). In our application

we check whether it support hardware type or not. If not we choose software type. If

hardware type is available we use it as it gives good quality rendering.

Then we have to add object to that created device. In this application we have add

three models a rotatable box fixed perpendicular three axes and rotatable

perpendicular three axes. And all the objects are defined in the world coordinate

system. To view the objects we have added a camera to the device.

When developing the application we try to give the 3D effect to the model (Cube). To

do that we use light to illuminate the model so that shadowing will give that effect

otherwise the rotations are difficult to view. Then we have to give the rotational

angles to the models. When we want to rotate a 3D object mathematically we multiply

the model coordinate by a transformation function. The above matrix is calculated

using the yaw, pitch and roll. In DirectX that matrix is calculated by the development

kit. So that we can give three angles and the DirectX rotate the model.

In the first phase of the application the model rotate due to noise from the device. To

remove this we use an averaging algorithm to reduce those fluctuations in the

readings. Thus we were able to keep the model stationary when device was stationary.

When we use this model we were able to measure the Roll and pitch accurately but

the when displaying the yaw the model became somewhat unstable at 00and 1800. So

we were unable to demonstrate the rotations in the horizontal plane by using this

application. This due to a bug in this software but by using the OpenGL it was

successful.

-64-

3-D Orientation viewer (OpenGL)

Fig. 3.6 3-D Orientation viewer (Open GL)

How to operate: Press calibrate button and rotate the device about each and every

three axes. This is required as we use magnetic sensors to get data. Then press ‘Done’

button. Then it gives a message box. In that box we can see the minimum and

maximum values along each three axes. After that we can rotate the cube by rotating

the device.

2-D Mouse with pitch and roll

Fig. 3.7 2-D Mouse with pitch and roll How to operate: First we have to press read button to get the data in to the application.

-65-

Then like the previous application calibration has to be done. Then it gives a message

box. In that box we can see the minimum and maximum values along each three axes.

Then press Map button to map the device movements to mouse cursor.

Then we can control the mouse cursor in horizontal direction by pitch angle and Roll

angle control the cursor in the horizontal plane. By using this software we can identify

control signals from the device.

Click – forward jerk (click label change its color to red)

Right click – backward jerk (Right label change its color to blue)

Double click- two consecutive forward jerks (Double click label change its color to

yellow)

Press Esc button in the keyboard to release the cursor.

Displacement with transformations

Fig. 3.8 Displacement with transformations

How to operate: We have to calibrate the device using the same method explained in

the above application. Then press Map button. Then we can draw lines in the Drawing

-66-

area by using the device. As we use axes transformation here the orientation of the

device is neutralized.

As we have use some thresholds to cutoff noise. We are restricted to draw lines in a

small area. If we use a larger area to move our device we can draw curves also.

We can map those movements to cursor by ticking the mouse check box.

Data logger (Impact)

Fig. 3.9 Data Logger (Impact)

This application is used to log acceleration in all the three axes which are higher than

a certain value. Those values are given to the software by the user. (Using the right

hand side text boxes)

We have to calibrate the device using the same method explained in the above

application. After we have press ‘Done’ button

The application monitors the acceleration in all the three axes. If it exceeds the

threshold it writes the value the axis name and time to a text file. As we use axis

transformation the orientation of the devise is not required. Values are given in world

coordinates.

-67-

Chapter 4

4. DISCUSSION AND CONCLUSION

The area of alternative input devices that operating almost real time using various

tracking systems has become a popular domain of research these days. Methods such

as image processing after camera tracking and accelerometer, magnetometer and (or)

gyroscope method are more commonly used. Each of the methods has its advantages

as well as disadvantages coupled to it. With the development of “sensor based motion

tracking system” we have been able to eliminate some of the disadvantages of the

camera method such as user being restricted to a certain are, lighting condition

dependency and high processing power requirement. Also on the hardware side this

only uses a pair of 3 axis accelerometers and magnetic field sensors. Gyroscopes

which are more commonly used are not introduced to our product. This has drastically

reduced the complexity of hardware and processing as well as the price and size.

The 6DOF motion detecting device is able to sense the tilt angles (roll, pitch and roll)

up to a resolution of 10. This has made way to easy, very low cost navigation system.

The very small size and light weight also has added to its valuability. By using several

of these units at each joint and interacting with a PC via Bluetooth just like that in

done in the project, high tech character animations can be done at a very low cost.

Shock detectors are used by many courier services. This product can also do that. But

the advantage here is due to the transformation done inside, the parcel and the device

need not be at alignment. Due to the transformation, the tracking system will anyway

give shocks with respect to the initial reference axes. Even at other applications, we

find that the user is not restricted to one orientation because of this transformation.

That’s a very significant advantage compared to other similar products.

The auto detection, auto pairing features had made the product user friendly.

Bluetooth connectivity has also given the user the freedom of operating anywhere

within 30m s. The rechargeable batteries last nearly 2 hours. The product has been

tested numerous times, under different conditions and has proved to be very robust

and hardy.

-68-

As a shortcoming, we see that the shape of the final product should have been made

more ergonomical. Also for some applications such as 2D mouse movement and

mouse clicking, we see that a prior training of the user is required.

As conclusion it should be stated that this versatile unit will have many more

applications that has not been even thought of. So it provides a very good opening for

anyone to build human computer interacting applications on this “sensor based

motion tracking system”.

-69-

Chapter 5

5. FUTURE WORK

3D drawing

With this device we are obtaining all parameters to handle all 3D movements. So,

most of the improvements have to be done in Application level such as improved

noise removal techniques, threasholding and advanced mapping. So, one interesting

application is 3D mapping and drawing. It can be developed using Microsoft

DirectX® technology employing obtained data from the device. This can also be used

to integrate with animation software like 3D Studio MAX® and MAYA® for

developing direct inverse kinematics for characters thus saving lot of pain. Even

though such technologies and solutions exist in the current context, we believe that

our product will offer both elegant and cost effective solution so it can be used in

blossoming countries like ours as well.

There are many advantages using this type of device over inverse kinematics done via

a computer alone. It is easy to use and the movement is directly fitted into the

character in one go. The movements are more organic or realistic than computer

generated ones. Not only humans, but also animals can be used to do the character

animation owing to the small size of the device and wireless transmission of data.

Further miniaturization

Reducing the overall physical size is also in the improvement list although it is very

small even now. Therefore we will be able to mount it on a finger or any other place

conveniently improving the number of such sensors and eventually making most out

of the system.

i

6. ANNEX – A: SET/RESET STRAP OVERVIEW

Strong external magnetic fields that exceed a 10 to 20 gauss “disturbing field” limit

can come from a variety of sources. The most common types of strong field sources

come from permanent magnets such as speaker magnets, nearby high-current

conductors such as welding cables and power feeder cables, and by magnetic coils in

electronic equipment such as CRT monitors and power transformers. Magnets exhibit

pole face strengths in hundreds to thousands of gauss. These high intensity magnetic

field sources do not permanently damage the sensor elements, but the magnetic

domains will be realigned to the exposed fields rather than the required easy axis

directions. The result of this re-magnetization of the sensor elements will be

erroneous measurements and indications of “stuck” sensor outputs.

Using the set and reset pulses will magnetically “restore” the sensor. The figure below

shows the typical magnetic domain alignment in normal operation, when a set pulse is

supplied and when a reset pulse is supplied.

Fig. A.1 How Magnetic domains aligned in a HMC sensor

-ii-

7. ANNEX – B: IN-CIRCUIT PROGRAMMER

The following PIC Programmer was advancement to the JDM Programmer which can

support most of the PIC micro controllers in High voltage ICSP programming mode.

Fig. B.1 JDM Programmer with ICSP support

Vpp

1

22

33

44

55

66

77

Gnd

8

Gnd

9

1010

Vdd

11

Gnd

12

Gnd

13

1414

1515

1616

1717

1818

1919

2020

2121

2222

2323

2424

2525

2626

2727

2828

2929

3030

Gnd

31Vd

d32

Gnd

33Gn

d34

Gnd

35Gn

d36

Gnd

37Gn

d38

Scloc

k39

Sdata

40J2 Zif

40 H

eade

r

12

34

56

78

910

P3

12345 6789

1110

J1 D Co

nnec

tor 9

D4 Diod

e 1N4

148

D3 Diod

e 1N4

148

Q3 BC55

7A

1.5K

R1

10K

R2

1KR3

100

R4

10K

R5

cts dtr

gnd_

rs232 rts txd

Q1 BC54

7A pgc

Q2 BC54

7A

D1 Diod

e 1N4

148

D2 Diod

e 1N4

148

D5 Z5.1v

D6 Z6.2v

GND

GND

100u

F

C1 Cap P

ol1

GND

vdd

pgd

100u

F

C2 Cap P

ol1

D7 LED1

GND

GND

GND

100n

F

C3 Cap

GND

vpp

ctsdtrgnd_

rs232

rtstxd

GNDvd

d

vdd

GND

pgc

vpp

vdd

GND

pgc

pgd

23 1

S1 SW-SP

DT

S2 SW-SP

ST28-40

pin

8-18 p

in

pgd

ON -V

pp Fi

rstD

o no

t con

nect

Vdd

to p

the

targ

et b

oard

Pull

dow

n PG

M to

dia

blw

hen

ICSP

usi

ng 1

00k

To p

rogr

am 8

-18

pin

mco

ntro

llers

con

nect

jum

2325

Use

Dtr

, Rts

, Cts

ena

bled

ser

ial c

able

-iii-

To implement ICSP into an application, the user needs to consider three main

components of an ICSP system: Application Circuit, Programmer and Programming

Environment.

Fig. B.2 Connection Diagram for ICSP programming

During the initial design phase of the application circuit, certain considerations have

to be taken into account. Figure 1 shows and typical circuit that addresses the details

to be considered during design. In order to implement ICSP on your application board

you have to put the following issues into consideration:

1. Isolation of the GP3/MCLR/VPP pin from the rest of the circuit.

2. Isolation of pins GP1 and GP0 from the rest of the circuit.

3. Capacitance on each of the VDD, GP3/MCLR/VPP, GP1, and GP0 pins.

4. Interface to the programmer.

5. Minimum and maximum operating voltage for VDD.

Isolation of the GP3/MCLR/VPP Pin from the Rest of the Circuit PIC12C5XX

devices has two ways of configuring the MCLR pin:

MCLR can be connected either to an external RC circuit or

MCLR is tied internally to VDD

-iv-

When GP3/MCLR/VPP pin is connected to an external RC circuit, the pull-up resistor

is tied to VDD, and a capacitor is tied to ground. This circuit can affect the operation

of ICSP depending on the size of the capacitor.

Another point of consideration with the GP3/MCLR/VPP pin, is that when the

PICmicro MCU is programmed, this pin is driven up to 13V and also to ground.

Therefore, the application circuit must be isolated from the voltage coming from the

programmer.

Interface to the Programmer Most programmers are designed to simply program the

PICmicro MCU itself and don’t have strong enough drivers to power the application

circuit. One solution is to use a driver board between the programmer and the

application circuit. The driver board needs a separate power supply that is capable of

driving the VPP, VDD, GP1, and GP0 pins with the correct ramp rates and also

should provide enough current to power-up the application circuit.

The cable length between the programmer and the circuit is also an important factor

for ICSP. If the cable between the programmer and the circuit is too long, signal

reflections may occur. These reflections can momentarily cause up to twice the

voltage at the end of the cable that was sent from the programmer. This voltage can

cause a latch-up. In this case, a termination resistor has to be used at the end of the

signal line.

-v-

8. ANNEX – C: THE BLUETOOTH PROTOCOL STACK

C.1 Introduction The Bluetooth specification is over 1500 pages long and contains the information

necessary to ensure that diverse devices supporting this technology can communicate

with each other worldwide. The specification is divided into two sections: Core

Specification (Volume I) and Profile Definitions (Volume 2).

Here's a high-level view of the architecture of the Bluetooth protocol stack:

Fig. C.1 Bluetooth Protocol Stack

C.2 The responsibilities of the layers in this stack are as follows:

The radio layer is the physical wireless connection. To avoid interference with other

devices that communicate in the ISM band, the modulation is based on fast frequency

hopping. Bluetooth divides the 2.4 GHz frequency band into 79 channels 1 MHz apart

(from 2.402 to 2.480 GHz), and uses this spread spectrum to hop from one channel to

another, up to 1600 times a second. The standard wavelength range is 10 cm to 10 m,

and can be extended to 100 m by increasing transmission power.

-vi-

The baseband layer is responsible for controlling and sending data packets over the

radio link. It provides transmission channels for both data and voice. The baseband

layer maintains Synchronous Connection-Oriented (SCO) links for voice and

Asynchronous Connectionless (ACL) links for data. SCO packets are never

retransmitted but ACL packets are, to ensure data integrity.

SCO links are point-to-point symmetric connections, where time slots are reserved to

guarantee timely transmission. A slave device is allowed to respond during the time

slot immediately following an SCO transmission from the master. A master can

support up to three SCO links to a single slave or to multiple slaves, and a single slave

can support up to two SCO links to different slaves.

Data transmissions on ACL links, on the other hand, are established on a per-slot

basis (using slots not reserved for SCO links). ACL links support point-to-multipoint

transmissions. After an ACL transmission from the master, only a slave addressed

specifically may respond during the next time slot; if no device is addressed, the

message is treated as a broadcast.

The Link Manager Protocol (LMP) uses the links set up by the baseband to establish

connections and manage piconets. Responsibilities of the LMP also include

authentication and security services, and monitoring of service quality.

The Host Controller Interface (HCI) is the dividing line between software and

hardware. The L2CAP and layers above it are currently implemented in software, and

the LMP and lower layers are in hardware. The HCI is the driver interface for the

physical bus that connects these two components. The HCI may not be required. The

L2CAP may be accessed directly by the application, or through certain support

protocols provided to ease the burden on application programmers.

The Logical Link Control and Adaptation Protocol (L2CAP) receives application

data and adapts it to the Bluetooth format. Quality of Service (QoS) parameters is

exchanged at this layer.

-vii-

9. ANNEX – D: WINDOWS MOUSE API

D.1 Relation between Cursor pointer and resolution The mouse is simply a device that supplies the computer with three bytes of

information (minimum) at a time, they are:

Distance Moved in X direction

Distance Moved in Y direction

Button Status

A mouse event occurs when something changes, i.e it is moved or a button is pressed.

The mouse interrupts the operating system to tell it that an event has occurred and

sends it the new data. The operating system normally updates the position of the

marker on the screen does the control function if given.

Most common convention of x-y axis

monitor area

X axis

Y axis

(0,0)

Fig. D.1 Most common convention of x-y axis

But this is not a big issue in our applications as we are taking the relative position of

the cursor. Then by using the user can change the orientation of the device according

to the movements. I.e. if the origin is at left side bottom corner of the screen Y

coordinate increase as moving upwards. But in the common case it decreases in that

direction. So to compensate that problem we have to change the orientation of the

device. In this case we have to change the direction of Y-axis by 1800.

-viii-

D.2 Windows mouse emulating software

In the following software we have control the cursor position and control by buttons.

Fig. D.2 Windows mouse emulating software

D.3 Windows mouse API usage The following Windows API moves the mouse:

[DllImport("user32")]

public static extern int SetCursorPos(int x, int y);

The inputs to the API are cursor coordinates. (int x, int y)

To do controlling we have used

[DllImport("user32.dll")]

public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons,

int dwExtraInfo);

In controlling we pass the status off each button to API and the cursor position.

The windows OS has assign a special number to each status. In the above API int

dwFlags represent that value. Those are OS specific.

-ix-

We have given the specific number for each button status.

LeftDown = 0x02 (left button pressed)

LeftUp = 0x04 (left button released)

RightDown = 0x08 (Right button pressed)

RightUp = 0x10 (Right button released)

Now, what exactly is a mouse click? it's a click down and then up. only sending

LeftDown will be like a mouse hold, and only sending LeftUp...will just be weird,

that's why in order to emulate a mouse click, you have to send both:

If we want only to move cursor we can use the .Net command to do that which is

simpler to handle. But .Net doesn’t supply a direct method to emulate mouse control

functions.

i

Bibliographical References

[1] Chin-Woo Tan, Sungsu Park: Design of Accelerometer-Based Inertial Navigation Systems: IEEE transaction and measurement vol 54, NO. 6, December 2005

[2] Ying Kun Peng, and M.F. Golnaraghi: A Vector-Based Gyro-Free Inertial Navigation System by Integrating Existing Accelerometer Network in a Passenger Vehicle, Department of Mechanical Engineering, University of Waterloo Waterloo, ON N2L 3G1 Canada

[3] Zhengyou Zhang, Ying Wu, Ying Shan, Steven Shafer: Visual Panel: Virtual Mouse, Keyboard and 3D Controller with an Ordinary Piece of Paper: Microsoft Research, 2001 ACM 1-58113-448-7-11/14/01

[4] Young-Seok Kim, Thenkurussi Kesavadas: Fingertip Digitizer: Applying Haptics and Biomechanics to Tactile Input Technology: Virtual Reality Laboratory, ACM SIGGRAPH 2006 emerging technologies

[5] Michael Rohs, Georg Essl: Sensing-based Interaction for Information Navigation on Handheld Displays, Deutsche Telekom Laboratories, MobileHCI 2007, September 11-14, 2007, Singapore

[6] Andrew Wilson, Steven Shafer: XWand: UI for Intelligent Spaces, Microsoft Research, CHI 2003, April 5–10, 2003, ft. Lauderdale, Florida, USA.

[7] Freescale Semiconductor, “MMA7260 - ±1.5g - 6g Three Axis Low-g Micro machined Accelerometer”, Rev 3, 04/2006

[8] Honeywell International Inc, “1, 2 and 3 Axis Magnetic Sensors HMC1051/HMC1052/HMC1053 – datasheet” Form #900308 Rev B March 2006

[9] Texas Instruments Incorporated, “LMV324 Quad, LMV324S Quad with shutdown, low-voltage rail-to-rail output operational amplifiers”, August 1999–Revised July 2007

[10] Maxim/Dallas, “MAX4194 Micro power, Single-Supply, Rail-to-Rail, Precision Instrumentation Amplifiers”, 19-1468; Rev 1; 6/03

[11] Microchip Technology Inc, “PIC18F2523 Microcontroller – Preliminary Data Sheet”, 2007

[12] Microchip Technology Inc, “Flash Microcontroller Programming Specification- PIC18F2423/2523/4423/4523”, 2005

[13] Initium Cooperation, “Parani ESD 02 – User Manual”, Version 2.0

[14] Maxim/Dallas, “MAX 3233 ±15kV ESD-Protected, 1µA, 250kbps, 3.3V/5V, DualRS-232 Transceivers with Internal Capacitors – datasheet”, 19-1473; Rev 2; 8/04

[15] Texas Instruments Incorporated, “TPS731xx - Cap-Free, NMOS, 150mA Low Dropout Regulator with Reverse Current Protection”, September 2003–Revised October 2006

-ii-

[16] Honeywell, SSEC, “Application Note - Applications of Magnetic Sensors for Low Cost Compass Systems”, 2003

[17] http://code.google.com/p/bluecove/

[18] http://jcp.org/en/jsr/detail?id=82