COMPARING THE VALIDITY OF A GPS MONITOR

AND A SMARTPHONE APPLICATION TO

MEASURE PHYSICAL ACTIVITY

Manolis Adamakis, PhD1

1Department of Physical Education and Sport Science, National and Kapodistrian University of Athens, Greece

Background: A recent approach to increasing physical activity levels, managing weight andimproving health has been via technological advances, such as the Web 2.0 technologies, wearableactivity trackers and smartphones. These approaches might be effective due to reduced cost, user-friendly environment and real-time feedback provided. Many of these monitors and smartphoneapplications are marketed to provide personal information on the level of physical activity, howeverlittle or no information is available regarding their validity.

Aims: The purpose of this study was to compare the criterion validity for distance travelled and totalenergy expenditure (TEE) between a commercially available GPS monitor, Garmin Forerunner310XT, and a freeware GPS application for Android smartphones, Runkeeper, under semi-structured activity settings.

Methods: A single, healthy and physically active participant took part in all trials. The same protocol wasrepeated on 40 occasions (20 walking and 20 running sessions). The participant wore the Garmin GPS onthe left wrist and a smartphone with the Runkeeper application activated on the left arm. Distance wascompared against an objectively measured distance with three different methods and energy expenditureestimates for each monitor was evaluated relative to criterion values concurrently obtained from theportable metabolic system Cosmed K4b2. Differences from criterion measures were expressed as a meanabsolute percent error and were evaluated using repeated measures ANOVA and Bland-Altman plots.

Results: For overall group comparisons, the mean absolute percent error values for distance were0.30% and 0.74% for Garmin (walking and running), while higher values were calculated forRunkeeper (3.28% during running and 4.43% during walking), which significantly overestimateddistance in both conditions. For energy expenditure estimation, significant differences were observedfor both monitors (pB0.001). Garmin significantly underestimated energy expenditure compared tothe criterion method in both conditions by 17%, while Runkeeper significantly overestimated it by6.29% during running and 35.52% during walking.

Conclusions: The present study offers initial evidence for the validity of GPS technology of wearableactivity monitors and smartphone applications for measuring distance travelled. However, estimates ofenergy expenditure were poor, except for Runkeeper during running which provided acceptable error.

Journal MTM 6:2:28�38, 2017 doi:10.7309/jmtm.6.2.4 www.journalmtm.com

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 28

IntroductionEvery year approximately 3.2 million deaths and32.1 million illnesses are associated with physicalinactivity.1 In most western societies people between18 and 35 years belong in the high-risk group ofbecoming either overweight or obese, due to lack ofphysical activity (PA).2 In order to reverse thesestatistics, individuals are recommended to accumu-late at least 30 min/day, 5 days/week of moderateintensity PA or 15 min/day, 5 days/week of vigorousintensity PA (75 min/week), or a combination ofboth.3

A recent approach to increasing PA levels, manag-ing weight and improving health has been viatechnological advances, such as the Web 2.0 tech-nologies, activity trackers and smartphones.4

Griffiths and colleagues5 proposed that these ap-proaches might be effective due to reduced cost,user-friendly environment, real-time informationand feedback provided. Sufficient evidence existsto support the positive effect of these programs withthe use of new technologies, especially when theseare combined with other research approaches, suchas face to face interventions. For example, King andcolleagues6 used three mobile applications tochange the physical (in)activity and sedentarybehavior of 95 underactive adults during an 8week intervention program, and the results provid-ed initial support for promoting PA and reducingsedentary behavior. Also Laurson, Welk and Eisen-mann7 asked 111 children to wear pedometers overa seven day period and record their steps. Childrenwho wore the pedometers longer appeared moreactive, with a significant increase of steps perminute.

Global Positioning System (GPS) technology, eventhough it is quite new in PA and exercise, is a lowcost, objective and discreet way to track individuals’movement.8 Most previous studies have concludedthat it is a valid way to measure distance travelled.For example, Specht and Szot9 tested six GPSreceivers and found that logging receivers, such asGarmin Forerunner 310XT, demonstrated the high-est accuracy in determining positions. In anothersimilar research, four wearable GPS monitors werecompared while four adults walked a distance of1.24 km and the most accurate device was GarminForerunner 205.10 Using a similar methodologicalapproach, Lee et al.11 chose four low cost GPSminotors and found that all devices were valid fordistance estimation, with Garmin 60 to be the mostaccurate.

Even though GPS monitors are accurate for dis-tance estimation, commercially available applica-tions that use the GPS sensor of the smartphonesare sparsely tested for their accuracy. Bauer12 forexample compared ten GPS applications over a 1km distance, with the Adidas miCoach been themost accurate and Runkeeper having a divergenceof 20 m.

Nowadays these monitors and applications have theability to estimate, using anthropometric and GPSdata, the energy expenditure (EE) of individualsduring exercise. However, no published research,only a Master’s dissertation13 and a conferencepaper14 have validated these outcomes. Mallula13

compared a Garmin Forerunner 405CX with Nike�application for iPod and found that the application

was more accurate in EE, while the device had a morevalid estimation of distance and speed. Furthermore,Adamakis and Zounhia14 in a preliminary studycompared a Garmin Forerunner 310XT with theRunkeeper application for Android smartphones,with the Garmin monitor having the best results indistance, speed and EE, while Runkeeper overesti-mated all exercise parameters. In conclusion, Bort-Roig and colleagues15 suggested that well designedstudies are needed that comprehensively assess physi-cal activity measurement accuracy, however till thesedays few researches have validated GPS monitors andsmartphone GPS-enabled applications.

The purpose of this study was to compare the criterionvalidity for distance travelled and total energy expen-diture (TEE) between a commercially available GPSmonitor, Garmin Forerunner 310XT, and a freewareGPS application for Android smartphones, Run-keeper, under semi-structured activity settings.

Method

Research design

An experimental research design was used. Theindependent variable was the type of GPS monitor:Garmin Forerunner 310XT and freeware RunkeeperAndroid application. The primary outcome wasdistance and the secondary outcome was total energyexpenditure (TEE), during walking and running.

Participant

One single, healthy and physically active individual,who could run at least 5 km continuously, took partin all trials. The same protocol was repeated on 40occasions (20 walking and 20 running sessions)

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 29

during 40 consecutive days. Relevant characteristicssuch as height (190 cm), weight (80 kg), gender(male) and age (33 years) were entered into eachmonitor separately. The anthropometric measureswere obtained at the beginning of the data collec-tion session. Standing height was measured to thenearest 0.1 cm with the use of a wall mountedHarpenden Stadiometer (Harpenden, London,UK), using standard procedures. Body mass wasmeasured by participants in light clothes and barefeet on an electronic platform scale (Tanita Corp.,Tokyo, Japan) to the nearest 0.1 kg. Followinganthropometric measurements, the participant wasasked to lay down in bed for 10 minutes and thenfitted with the portable indirect calorimeter(Cosmed K4b2) to measure resting energy expendi-ture (REE). REE was measured for 15 minutes withthe subject quiet, but awake. The first 5 min as wellas the last minute of measurement were eliminatedand the REE was obtained from the average of 9min. REE measurement was performed after a 10-hour fast, following previously published guide-lines.16 The REE was expressed as kilocalories(kcal) per minute by dividing the TEE by 9 andthe estimation was 1.05 kcal � min�1.

Instruments

Smartphone - Samsung Galaxy S4 mini (SamsungElectronics Co., Ltd., Suwon, South Korea): TheSamsung Galaxy S4 mini uses Google’s AndroidJelly Beam 4.2.2 mobile operating system equippedwith a built-in GPS receiver, accelerometer andgyroscope. This phone is small, 125 by 61 by 9mm, and lightweight, 107 grams.

Runkeeper Android application: Runkeeper, soft-ware version 4.4.3 (FitnessKeeper, Inc.; http://Run-keeper.com/) is one of the most commercialapplications for Android and iOS smartphones.17

Runkeeper is a free of charge application, whichcalculates average pace and speed, laps’ speed, routedistance, elevation and estimated calorie burn for avariety of fitness activities, in high accuracy andreal time, using the in-built GPS sensor of theAndroid phone. EE is calculated taking into accountdistance computed, speed, time, elevation, acti-vity type, body weight measurements and REE.Body measurements are entered manually in theapplication settings.

It has been downloaded 10 � 50 million times andhas a total of 4.5 out of 5 rate out of 421,124 usersin the Google Play store (as accessed on 21/9/2016).

The program has full and efficient functionalityafter downloading, with no need for additionalsoftware download being necessary. Furthermoreit can record and upload exercise data to acomputer database after been registered to anonline system. It has the ability to track runningand walking sessions separately and calculateenergy expenditure based on these activities. Addi-tionally, we made sure from its description that theapplication relied solely on GPS for trackingthe user. Finally, Runkeeper has been identified asthe application with the most applied behaviorchanging techniques (eight in number) in themarket.18

Garmin Forerunner 310XT: Garmin, software ver-sion 4.50 (Garmin Ltd., USA; https://buy.garmin.com/en-US/US/into-sports/running/forerunner-310XT/prod27335.html), is a GPS-enabled training andheart rate monitor for multisport athletes. Its physi-cal dimensions are 54�56�19 mm, with displaysize 33�20 mm and lightweight, 72 grams. It iswater resistant up to 50 meters, with a 20-hourbattery life and a memory history of 1000 laps.Forerunner tracks time, distance, average and lapspeed and pace, heart rate with a premium heart ratemonitor, on land and estimated calorie burn.

The main method for EE estimation on Garminfitness monitors uses the Firstbeat algorithm.19 Thecalculation takes into account the user’s inputtedvariables including gender, height, weight andfitness class. It then combines the data with heartrate information from the heart rate strap. Morespecifically, it evaluates the time between heartbeats(beat to beat) to determine estimated MetabolicEquivalent (MET), which in turn is used to deter-mine actual work expenditure. This method isinexpensive and with relatively high accuracy, witha marginal error of 7-10%.19 It is considered areliable method for estimating the rate of oxygenconsumption (VO2) and EE, however it mayunderestimate these parameters by 6% to 13%respectively.20 Furthermore, this prediction methodmay be considered sufficiently accurate to deter-mine the average VO2 in field use, but it does notallow precise estimation of VO2.21

Objective-criterion measurements

Distance: The criterion distance was measured by:a. Leica DISTOTM D810 laser range finder, whichprovides typical measuring accuracy 91 mm andhas a range up to 200 m,22 b. Calibrated measuring

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 30

wheel, and c. Google Maps Distance Calculator

(http://www.daftlogic.com/projects-google-maps-

distance-calculator.htm). The three-way estimated

distance course was 3.58 km.

Energy Expenditure: Indirect calorimetry was uti-

lized in this study as the criterion method for

TEE.23 The accuracy of estimated TEE by the two

monitors was compared to that measured breath by

breath with Cosmed K4b2 (Cosmed S.r.I., Rome,

Italy) portable metabolic analyzer. Cosmed allows

measurement of oxygen consumption under free-

living conditions providing valid and reliable results

in the general population.24 Previous research, in

which Cosmed was compared with the Douglas

airbag method25,26 and the traditional metabolic

analyzer Medgraphics D-Series,27 showed that

Cosmed provided comparable energy expenditure

estimates in stable, submaximal exercise intensities

and the results of oxygen consumed, carbon dioxide

eliminated and respiratory quotient ranged within

acceptable limits of agreement. It comprises an

analyzer unit and a facemask. Volume and gas

calibrations (4% CO2, 16% O2) were performed

before each trial by following manufacturer recom-

mendations.

Experimental procedures

The study was conducted at an urban outdoor

environment, on a wooded paved trail with no grade

(0%) around an Athens central park (Figure 1). This

is an area of 49 acres, which formerly was hosting the

military camp ‘Goudi’ and currently is used as a

recreational and sports park. Before the start of

every session, both GPS monitors were activated and

the GPS signal was ensured to be acceptable for

adequate measurement. The participant’s personal

data were entered manually in the smartphone

application and the GPS monitor prior to the start

of the test. The chest strap for measuring heart rate

for the GPS unit as well as the calorimeter was

placed around the chest of the participant, while

Garmin was place on the left wrist and the smart-

phone on the left arm. The calorimeter was placed

on a harness over the subject’s shoulders and

strapped around the subject’s chest and back. The

face mask was placed covering the subject’s mouth

and nose so as to not allow air to escape.

The 3.58 km test began at the quarter mile mark

(point A, depicted on Figure 1) and was run or

walked on an out and back course ending at

the measured ending line (point B, depicted on

Figure 1: The 3.58 km park route.

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 31

Figure 1). The calorimeter needed a brief calibra-tion before being started. This was conducted on thetrail prior to the start of the 3.58 km course. Whenthe calibration was complete, the 3 devices (Gar-min, Runkeeper and calorimeter) were startedsimultaneously (9 2 sec) and the participant beganto walk or run. One researcher started the calorim-eter and the GPS unit, and the participant startedthe Runkeeper. The participant walked, ran at acomfortable pace, and self-selected speed. At theend of the 3.58 km course all measuring deviceswere stopped simultaneously (9 2 sec) by the sameresearcher who started them. The data from alldevices were collected and recorded on a data sheetand, afterwards, data were transferred to the onlinewebsites of each monitor.

In total, the participant conducted 20 running and20 walking sessions. The average time needed inorder to complete the 3.58 km course running was21 min 5 sec955 sec and walking 38 min 22 sec952sec. The average heart rate, as measured byGarmin’s heart rate strap, was 155.6593.95 bpmfor running and 99.6097.91 bpm for walking.

Each monitor uses different outcomes measures tosummarize EE data. The Runkeeper provides esti-mates of TEE, while the Garmin reports estimatesof activity EE. In order to provide comparableestimates in the EE results, it was necessary to addREE to the activity EE values for the Garmin,adding the measured REE obtained prior to theactivity protocol to the estimated EE. This ensuredthat we had comparable outcome measures of TEEfor both monitors. This procedure was implementedin previous studies28 and ensured that we hadcomparable outcome measures of TEE for bothmonitors.

Statistical analysis

The statistical analysis was conducted with the useof the statistical package SPSS version 21.0 (IBMSPSS Corp., Armonk, NY, USA) and the signifi-cance level was set at pB0.05. Data analysis wasbased on Bruton, Conway and Holgate29 recom-mendations. Before the main procedures, variableswere screened for accuracy of data entry, missingvalues, potential outliers and distribution (skewnessand kurtosis) for running and walking separately.No missing values were observed and the box plots,skewness and kurtosis analysis indicated that noextreme values existed and data were approximatelynormally distributed.

Data were analysed using descriptive (mean, stan-dard deviation) and inferential statistics. A two-wayrepeated measures ANOVA (measurement meth-od�type of activity) was performed to detectdifferences in the dependent variables between meth-ods at two intensity levels. Post-hoc analyses usingpaired t-tests with the Bonferroni correction wereconducted to examine specific differences betweenthe two GPS monitors and the criterion methods.Mean absolute percent errors (MAPE) were alsocalculated to provide an indicator of overall mea-surement error (MAPE�[(monitor measurement -actual measurement) / actual measurement]�100)and was used as an outcome measure. A smallerMAPE represents better accuracy, and less than 3%is considered acceptable for distance30 and 10% forTEE.28 This method is a more conservative estimateof error that reflects the true error in estimation andprovides the most appropriate indicator of overallerror. Further analyses included Bland-Altman plots,which were calculated to examine the level ofagreement between each monitor and the criterionmethods across all dependent variables.31 Limits ofagreement were calculated as 92 SD from theoverall mean bias between the dependent variablesand each GPS monitor.

Results

Distance

Table 1 provides descriptive statistics (means, SD)for all of the different monitors compared with themeasured values for distance in the two conditions,walking and running. Separate 3 (2 GPS mon-itors�criterion)�2 (walking or running) repeatedmeasures ANOVA were conducted to examinedifferences in distance obtained from Garmin andRunkeeper. There was no significant interactioneffect between the 3 measurement methods andthe 2 types of activities [F(1,21)�1.58, p�0.23,h2�0.08] and no significant main effect betweenwalking and running [F(1,19)�2.94, p�0.10,h2�0.13]. However there was a significant maineffect between the 3 measurement methods

Mean

walking

SD

walking

Mean

running

SD

running

Measured 3.58 .00 3.58 .00

Runkeeper 3.74 .08 3.70 .11

Garmin 3.59 .04 3.58 .02

Table 1: Descriptive statistics for Distance (km)

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 32

[F(1,21)�64.44, pB0.001, h2�0.77]. Post-hoc

within-subjects contrasts with Bonferroni correc-

tion revealed that Runkeeper significantly over-

estimated distance [F(1,19)�73.75, pB0.001,

h2�0.80], while Garmin provided almost the

same results as the measured distance

[F(1,19)�0.57, p�0.46, h2�0.03].

The MAPE (computed as the average absolute

value of the errors relative to the measured

distance) observed for Garmin during the running

condition was 0.30% and for the walking condition

was 0.74%. A larger MAPE was observed for

Runkeeper during running (3.28%) and walking

(4.43%) (Figure 2).

Bland-Altman plot analysis showed the distribution

of error and assisted with testing for proportional

systematic bias in the estimates. The plots show the

residuals of the various estimates on the y-axis

(measured distance � monitor) relative to the mean

of two methods (x-axis). The plots (see Figure 3)

revealed the narrowest 95% limits of agreement for

Garmin during running (mean bias�0.00; 95%

CI��0.01 to 0.01 km) and Garmin during walking

(mean bias�-0.01 km; 95% CI��0.02 to 0.01 km),

while higher values for Runkeeper during running(mean bias��0.12 km; 95% CI��0.17 to �0.07km) and walking (mean bias��0.16 km; 95%CI��0.20 to -0.12 km) were observed. The slopesfor the fitted lines were significant for all measure-ments; Runkeeper walking (slope��1.93, pB0.001),Runkeeper running (slope��1.96, pB0.001), Gar-min walking (slope��1.78, pB0.001) and Garminrunning (slope��1.05, pB0.001), which suggestssignificant patterns of proportional systematic bias forthese monitors.

TEE

Table 2 provides descriptive statistics (means, SD)for all of the different monitors compared with the

Figure 2: Mean absolute percent error (9SD) for distance estimation.

Mean

walking

SD

walking

Mean

running

SD

running

Cosmed 198.55 6.46 314.05 4.51

Runkeeper 268.80 23.74 333.70 25.01

Garmin 184.60 38.26 259.30 21.95

Table 2: Descriptive statistics for estimated TEE (kcal),

with added measured REE for Garmin

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 33

measured values from the Cosmed in the two

conditions, walking and running. Separate 3 (two

GPS monitors�criterion)�2 (walking or running)

repeated measures ANOVA were conducted to

examine differences in TEE obtained from Garmin

and Runkeeper application. There was a significant

interaction effect between the 3 measurement meth-

ods and the 2 types of activities [F(2,38)�16.10,

pB0.001, h2�0.46], a significant main effect

between walking and running [F(1,19)�267.47,

pB0.001, h2�0.93] and a significant main effect

between the 3 measurement methods

[F(2,38)�128.74, pB0.001, h2�0.87]. Post-hoc

within-subjects contrasts with Bonferroni correc-

tion revealed that Runkeeper significantly over-

estimated TEE for both conditions

[F(1,19)�44.72, pB0.001, h2�0.70], while Garmin

significantly underestimated TEE for both condi-

tions [F(1,19)�15.14, p�0.001, h2�0.44].

The MAPE (computed as the average absolute

value of the errors relative to the measured distance)

observed for Garmin during the running condition

was 17.39% and for the walking condition was

17.32%. A smaller MAPE was found for Runkeeper

during running (6.29%) and larger for walking

(35.52%) (Figure 4).

Bland-Altman plot analysis showed the distributionof error and assisted with testing for proportionalsystematic bias in the estimates. The plots (seeFigure 5) revealed the narrowest 95% limits ofagreement for Garmin during walking (meanbias�13.95 kcal; 95% CI��4.45 to 32.35 kcal)and for Runkeeper during running (mean bias��19.65 kcal; 95% CI��31.10 to �8.20 kcal), whilehigher values for Garmin during running (meanbias�54.75 kcal; 95% CI�43.78 to 65.72 kcal) andRunkeeper during walking (mean bias��70.25kcal; 95% CI��81.77 to �58.73 kcal) wereobserved. The slopes for all fitted lines weresignificant; Runkeeper walking (slope��1.73,pB0.001), Runkeeper running (slope��1.75,pB0.001), Garmin walking (slope��1.94,pB0.001) and Garmin running (slope��2.03,pB0.001), which suggests significant patterns ofproportional systematic bias with these monitors.

DiscussionThe present study aimed to examine the criterionvalidity of Garmin Forerunner 310XT GPS monitorand freeware Android application Runkeeper fordistance and TEE estimation, during walking andrunning. To our knowledge, this is the first studythat has tried to compare the accuracy of a

Figure 3: Bland-Altman plots for distance estimation.

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 34

commercially available GPS and a freeware smart-

phone application under semi-structured settings.

Even though Garmin Forerunner 310XT and Run-

keeper have been validated for distance estimation,

the algorithms that they use for EE prediction have

never been validated previously. Furthermore, stud-

ies, which compare wearable activity monitors and

smartphone PA applications are scarce in the

international literature and only recently some

similar attempts have been conducted.32,33

A unique feature of this study was the naturalistic

design that sought to replicate free-living over-

ground movement. In contrast with traditional

validation studies,34 the participants were given

the option to select the intensity of walking and

running they preferred during exercise. The results

of the present study support the accuracy of these

methods for distance recording, however TEE

estimates had large errors.

A further advancement, as indicated by Bai et al.,35

was that both individual-level and group-level

accuracy in distance and TEE estimation were

evaluated. MAPE values and 95% limits of agree-

ment provided a useful indicator of individual-level

validity and reflected the error that individuals

could expect if they were tracking their personal

activity estimates. Overall distance error estimates

were similar to results from previous evaluations of

wearable monitors and smartphone applica-

tions,12,14 with overall MAPE values typically

ranging from 0.30% to 4.43% for both Garmin

and Runkeeper. Garmin had the smallest MAPE in

both conditions and was the most precise estimate

based on the 95% limits of agreement. Runkeeper

had higher errors, overestimating distance; however,

these errors of 3% to 4% are acceptable. Taking into

account the small difference between the lower and

upper limit of agreement of 0.10 km, this applica-

tion may also be considered valid for distance

counting.

On the other hand, TEE error estimates were larger,

ranging from 6.29% (Runkeeper during running) to

35.52% (Runkeeper during walking). Garmin un-

derestimated TEE, whereas Runkeeper overesti-

mated it. The slopes for the fitted lines in the

Bland-Altman plots were all significant due to at

least two outliers in each graph, which might have

significantly biased the outcomes of these analyses.

Since no previous study has validated the EE of

Figure 4: Mean absolute percent error (9SD) for TEE estimation.

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 35

GPS devices and applications, no direct comparable

results could be made. Most previous studies, which

examined the validity of accelerometer-based wear-

able activity trackers (FitbBit, Jawbone, Nike�,

etc.) came to similar conclusions.28,35,36 Runkeeper

during running gave a MAPE less than 10%,

making it accurate in TEE under this condition,

while MAPE during walking was significantly high-

er. Garmin was more stable during the two condi-

tions, with an overall medium to large MAPE of

17%.

Mean bias and repeated measure ANOVA provided

alternative indicators of group-level accuracy. The

mean bias and repeated measure ANOVA results

favored only Garmin for distance estimation, which

provided group-level validity. Both the monitor and

the application had low group-level accuracy for

TEE estimation, differing significantly from the

criterion measurement.

A novel finding is that Garmin’s TEE estimation

was less accurate than expected. Garmin, in order

to estimate TEE, uses the Firstbeat algorithm,19

which combines the data obtained from the heart

rate strap. Previous research showed that this might

underestimate EE by 6% to 13%.20 In this study, the

error was 17%. The accuracy of EE predictions was

not improved with the addition of heart ratemeasures to traditional GPS device, in accordancewith previous research findings in accelerometer-based monitors.37 A possible explanation for thelimited accuracy of EE prediction could be that thealgorithm used could not accurately compute EEbased on distance and heart rate, even though theseinitial raw data were very accurate. Technicalassistance was sought from the company to ascer-tain specific information regarding the algorithmused to determine EE, however this informationwas not disclosed.

The present study had some limitations. Only onehealthy participant performed all activities and thuswe did not account for different potential con-founding factors such as BMI, monitor placement,gender and age, which could potentially influenceaccuracy. In addition, we examined the accuracyduring walking and running in outdoor terrain, sothe results cannot be generalized in other settings.Lastly, these estimates were obtained for a specificdistance and may not reflect accuracy for longerdistances, in example for a marathon race.

ConclusionIn conclusion, the present study offers initialevidence for the validity of GPS technology of PA

Figure 5: Bland-Altman plots for TEE estimation (adding REE for Garmin).

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 36

monitors and smartphone applications for measur-

ing distance travelled during walking and running.

However, estimates of EE were low, except for

Runkeeper during running which provided accept-

able error. This limits their use for monitoring

energy balance, and therefore as a weight manage-

ment tool. These results can assist consumers,

researchers and health care providers to make

evidence-based choice for a GPS PA monitor to

measure distance during exercise. People who own

Android smartphones have a valid alternative for

distance estimation during walking and running

with this freeware GPS application. Caution should

be made when mixing the results from different

activities into one estimated measure, due to

probable cancelling of overestimation and underes-

timation from these activities, which may lead to an

illusion of improved accuracy.

No competing interestsAll authors have completed the Unified Competing

Interest form at www.icmje.org/coi_disclosure.pdf

(available on request from the corresponding

author) and declare: no support from any organisa-

tion for the submitted work; no financial relation-

ships with any organisations that might have an

interest in the submitted work in the previous 3

years; no other relationships or activities that could

appear to have influenced the submitted work.

References1. World Health Organization. Prevalence of insufficient

physical activity: Situation and trends. 2014. Availiable

at http://www.who.int/gho/ncd/risk_factors/physical_

activity_text/en/ (accessed 25 June 2016).

2. Reither EN, Hauser RM, Yang Y (2009). Do birthcohorts matter? Age-period-cohort analyses of the

obesity epidemic in the United States. Soc Sci Med

2009;69(10):1439�48. doi: 10.1016/j.socscimed.2009.

08.040

3. Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons

C, Catapano AL, et al. European Guidelines on

cardiovascular disease prevention in clinical prac-

tice: The Sixth Joint Task Force of the EuropeanSociety of Cardiology and Other Societies on

Cardiovascular Disease Prevention in Clinical Prac-

tice. Eur Heart J 2016;37:2315�81. doi: 10.1093/

eurheartj/ehw106

4. Bardus M, Smith JR, Samaha L, Abraham, C. Mobile

phone and web 2.0 technologies for weight manage-

ment: A systematic scoping review. J Med Internet Res

2015;17(11):e259. doi: 10.2196/jmir.5129

5. Griffiths F, Lindenmeyer A, Powell J, Lowe P,

Thorogood M. Why are health care interventions

delivered over the internet? A systematic review of the

published literature. J Med Internet Res 2006;8(2):e10.

doi: 10.2196/jmir.8.2.e10

6. King AC, Hekler EB, Grieco LA, Winter SJ, Sheats

JL Buman MP, et al. Effects of three motivationally

targeted mobile device applications on initial physical

activity and sedentary behavior change in midlife and

older adults: A randomized trial. PLoS ONE

2016;11(6):e0156370. doi: 10.1371/journal.pone.0156370

7. Laurson KR, Welk GJ, Eisenmann JC. Estimating

physical activity in children: Impact of pedometer wear

time and metric. J Phys Act Health 2015;12:124�31.

http://dx.doi.org/10.1123/jpah.2013-0111.

8. Duncan MJ, Badland HM, Mummery WK. Applying

GPS to enhance understanding of transport-related

physical activity. J Sci Med Sport 2009;12:549�56. doi:

10.1016/j.jsams.2008.10.010

9. Specht M, Szot T. Accuracy analysis of GPS sport

receivers in dynamic measurements. An Nav

2012;19:165�76. doi: 10.2478/v10367-012-0013-9

10. Wieters KM, Kim JH, Lee C. Assessment of

wearable global positioning system units for physical

activity research. J Phys Act Health 2012;9:913�23.

11. Lee K, Kim JY, Putti K, Bennett DH, Cassady D,

Hertz-Picciotto I. Use of portable Global Positioning

System (GPS) devices in exposure analysis for time-

location measurement. Kor J Envir Health Sc

2009;35(6):461�67. doi: 10.5668/JEHS.2009.35.6.461

12. Bauer C. On the (in-)accuracy of GPS measures of

smartphones: A study of running tracking applica-

tions. In: Proceedings of MoMM ’13 International

Conference on Advances in Mobile Computing &

Multimedia. New York, NY, USA: ACM 2013:335-

40. doi:10.1145/2536853.2536893.

13. Mallula CL. Comparing Garmin Forerunner 405CX

GPS and Nike� iPod to accurately measure energy

expenditure, distance and speed of overground run-

ning. Master’s Thesis, Department of Health, Physical

Education, Recreation and Dance, Cleveland State

University. 2010. Availiable at http://rave.ohiolink.

edu/etdc/view?acc_num�csu1273708884 (accessed 3

September 2016).

14. Adamakis M, Zounhia K. Comparing the validity

and output of the Garmin Forerunner 310XT and

Runkeeper android application in an urban environ-

ment. In: Dasheva D, Antala B, Djobova S, Kuleva

M, (Eds.), Physical Education and sport � Compe-

tences for life: Book of abstracts of the 9th FIEP

European congress. Sofia, Bulgaria: FIEP 2014:59�60.

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 37

15. Bort-Roig J, Gilson ND, Puig-Ribera A, Contreras

RS, Trost SG. Measuring and influencing physical

activity with smartphone technology: A systematic

review. Sports Med 2014;44(5):671�86. doi: 10.1007/

s40279-014-0142-5

16. Compher C, Frankenfield D, Keim N, Roth-YouseyL. Best practice methods to apply to measurement of

resting metabolic rate in adults: a systematic review.

J Am Diet Assoc 2006;106(6):881�903. doi: 10.1016/

j.jada.2006.02.009

17. Duarte L, Ribeiro P, Guerreiro T, Carrico L.

Defining a design space for persuasive cooperative

interactions in mobile exertion applications. Collab

Tech 2014;8658:105�12.

18. Middelweerd A, Mollee JS, van der Wal CN, Brug J,

te Velde SJ. Apps to promote physical activity among

adults: A review and content analysis. Int J Behav

Nutr Phys Act 2014;11(97):1�9. doi: 10.1186/s12966-

014-0097-9

19. Firstbeat Technologies. An energy expenditure esti-

mation method based on heart rate measurement.Firstbeat Technologies Ltd, 2012. Availiable at

http://www.firstbeat.com/userData/firstbeat/Energy_

Expenditure_Estimation.pdf (accessed 16 August

2016).

20. Montgomery PG, Green DJ, Etxebarria N, Pyne DB,

Saunders PU, Minahan CL. Validation of heart rate

monitor-based predictions of oxygen uptake and

energy expenditure. J Strength Cond 2009;23(5):1489�95. doi: 10.1519/JSC.0b013e3181a392777

21. Smolander J, Ajoviita M, Juuti T, Nummela A, Rusko

H. Estimating oxygen consumption from heart rate

and heart rate variability without individual calibra-

tion. Clin Physiol Funct Imaging 2011;31(4):266�71.

doi: 10.1111/j.1475-097X.2011.01011.x

22. Leica Geosystems (2015). Leica DISTOTM D810 touch:The smartest solution to measuring and documenting.

Available at http://www.leica-geosystems.com/en/

Leica-DISTO-D810-touch_104560.htm (accessed

15 August 2016).

23. Hills AP, Mokhtar N, Byrne NM. Assessment of

physical activity and energy expenditure: an overview

of objective measures. Front Nutr 2014;1(5):1�16.doi: 10.3389/fnut.2014.00005

24. McLaughlin JE, King GA, Howley ET, Bassett

DRJr, Ainsworth BE. Validation of the COSMED

K4b2 Portable Metabolic System. Int J Sports Med

2001;22:280�84.

25. Parr BB, Strath SJ, Bassett DR, Howley ET.

Validation of the Cosmed K4b2 portable metabolic

measurement system. Med Sci Sports Exerc

2001;333(Suppl. 5):S300.

26. Eisenmann JC, Brisko N, Shadrick D, Welsh S.

Comparative analysis of the Cosmed Quark b2 and

K4b2 gas analysis systems during submaximal ex-

ercise. J Sports Med Phys Fitness 2003;43(2):150�55.

27. Schrack JA, Simonsick EM. Ferrucci L. Comparison of

the Cosmed K4b2 portable metabolic system in measur-

ing steady-state walking energy expenditure. PLoS ONE

2010;5(2):e9292. doi: 10.1371/journal.pone.0009292

28. Lee JM, Kim Y, Welk GJ. Validity of consumer-

based physical activity monitors. Med Sci Sports

Exerc 2014;46(9):1840�48. doi: 10.1249/MSS.

0000000000000287

29. Bruton A, Conway JH, Holgate ST. Reliability:

What is it, and how is it measured? Physiotherapy

2000;86(2):94�9.

30. Crouter SE, Schneider PL, Karabulut M, Bassett

DRJr. Validity of 10 electronic pedometers for

measuring steps, distance, and energy cost. Med Sci

Sports Exerc 2003;35(8):1455�60.

31. Bland JM, Altman DG. Statistical methods for

assessing agreement between two methods of clinical

measurement. Lancet 1986;8:307�10.

32. Adamakis M. Preliminary study of consumer-level

activity monitors and mobile applications for step

counting under free living conditions. J Mob Technol

Med 2016;5(3):1�8. doi: 10.7309/jmtm.5.3.x

33. Case MA, Burwick HA, Volpp KG, Patel MS.

Accuracy of smartphone applications and wearable

devices for tracking physical activity data. JAMA

2015;313(6):625�26. doi: 10.1001/jama.2014.17841

34. Boyce G, Padmasekara G, Blum M. Accuracy of

mobile phone pedometer technology. J Mob Technol

Med 2012;1(2): 16�22. doi: 10.7309/jmtm.13

35. Bai Y, Welk GJ, Nam YH, Lee JA, Lee JM, Kim Y, et al.

Comparison of consumer and research monitors under

semistructured settings. Med Sci Sports Exerc 2015;

48(1):151�158. doi: 10.1249/MSS.0000000000000727

36. Ferguson T, Rowlands AV, Olds T, Maher C. The

validity of consumer-level, activity monitors in

healthy adults worn in free-living conditions: A

cross-sectional study. Int J Behav Nutr Phys Act

2015;12:42. doi: 10.1186/s12966-015-0201-9

37. Wallen MP, Gomersall SR, Keating SE, Wisloff U,

Coombes JS. Accuracy of heart rate watches: Im-

plications for weight management. PLoS ONE 2016;

11(5):e0154420. doi: 10.1371/journal.pone.0154420

ORIGINAL ARTICLE

#JOURNAL OF MOBILE TECHNOLOGY IN MEDICINE VOL. 6 | ISSUE 2 | AUGUST 2017 38