J.-S. Pan et al. (Eds.): HAIS 2013, LNAI 8073, pp. 618–628, 2013. © Springer-Verlag Berlin Heidelberg 2013

A HMM-Based Location Prediction Framework with Location Recognizer Combining k-Nearest Neighbor

and Multiple Decision Trees

Yong-Joong Kim and Sung-Bae Cho

Department of Computer Science, Yonsei University 50 Yonsei-ro, Seodaemun-gu, Seoul 120-749, Korea

{yongjoong,sbcho}@yonsei.ac.kr

Abstract. Knowing user’s current or next location is very important task for context-aware services in mobile environment. Many researchers have tried to predict user location using their own methods. However, they focused mainly the performance of method, and only few were considered development of real working system on mobile devices. In this paper, we present a location predic-tion framework, and develop a personalized destination prediction system based on this framework using smartphone. The framework consists of two methods of recognizing user location based on the combined method of k-nearest neigh-bor (kNN) and decision tree, and predicting user destination based on the hid-den Markov model (HMM). The destination prediction system is composed of four parts including mobile sensor log collector, location recognition module, location prediction module, and system management module. Experiments on real datasets of five persons showed that our method achieved average predic-tion accuracy above 87%.

Keywords: Location recognition, Location prediction, Location extraction, Lo-cation-based services.

1 Introduction

With the ubiquity and ever-increasing capabilities of mobile devices, smartphone has become a powerful platform to be exploited for mobile context-aware services. More-over, because a variety of sensors have been equipped in recent mobile devices, we could get much information from the sensors. In this regard mobile context-aware services have attracted more attention, and active investigation about inferring user’s mobile contexts is actively being conducted for the services [5, 12, 16, 17]. Above all one of the most important user contexts is location. It allows information and services in the mobile device to be localized. It means the proper services and information can be delivered according to user’s current location or future location. In spite of a lot of research with respect to location prediction, however, the real working system which recognizes and predicts the location on the mobile device has not been developed is still on the way.

A HMM-Based Location Prediction Framework with Location Recognizer 619

In this paper, we propose a location prediction framework, and develop the perso-nalized location prediction system for mobile context-aware services. The contribu-tions of this paper can be summarized as follows.

• Development of the real working system: we develop a location prediction system by integrating location recognition module, location prediction module, sensor log collector, and system management module.

• Exploitation of G-means algorithm: for user trajectories to be discriminatively modeled we extract the intermediate locations by using G-means clustering method which determines the number of clusters automatically by performing statistical test iteratively.

• Management of models and user data: we develop user interface for learning rec-ognition and prediction model and implement the functions of location manage-ment and path management for the system to appropriately manage user’s locations and paths.

The rest of this paper is organized as follows. Section 2 briefly reviews the related works. Section 3 presents the details of the proposed framework. Section 4 presents the personalized location prediction system. Section 5 conducts some experiments and analyzes the experimental results. Section 6 concludes the work.

2 Related Works

Ashbrook et al. extracted user’s significant locations from GPS data and presented a location predictor based on the Markov models [2]. Krumm et al. designed a method called predestination that predicts driver’s destination as trip progresses [9]. Alvarez-Garcia et al. presented a new approach to predict destinations given only data of a partial trip by using hidden Markov models (HMMs) and local street-map [1]. Sim-mons et al. proposed a HMM-based approach by using a map database and GPS sen-sor to providing real-time predictions on driver destination and route [23]. Mathew et al. designed a hybrid method for predicting human mobility on the basis of HMM [13]. Petzold et al. presented a dynamic Bayesian network to predict an indoor next location and compared with the state predictor and multi-layer perceptron predictor [18]. Yavas et al. presented a data mining algorithm for the prediction of user move-ments in a mobile computing system [24]. Monreale et al. proposed trajectory pattern tree aimed at predicting with a certain level of accuracy the next location of a moving object [14]. Morzy mined the database of moving object locations to discover fre-quent trajectories and movement rules, and matched the trajectory of a moving object with the database of movement rules to build a probabilistic model [15].

Petzold et al. compared various methods for next location prediction [19]. In their work, the comparing experiments were conducted using dynamic Bayesian network, multi-layer perceptron, Elman net, Markov predictor and state predictor. Scellato et al. presented a novel framework for predicting user’s next locations based on nonli-near time series analysis [22].

620 Y.-J. Kim and S.-B.

3 Location Predic

3.1 Problem Definition

Basically, the location predjectory ℓ ℓ ℓthe set of locations, user. Fig. 1 shows the probl

Fig. 1.

In this paper, to solve theto the path classification ℓ ℓ ℓuser’s movement paths d denote start and destinatlocations , , , … ,path ; classify an observesolving this problem, we careturning destination locatiproblem is converted to the

Cho

ction Framework

n

diction problem is as follows. Given an observed user ℓ where ℓ is the ith location user visited , , , … , ; predict ℓ , i.e., the next location of lem as a pictorial representation.

Fundamental location prediction problem

e location prediction problem we transform the problemproblem as follows. Given an observed user trajectoℓ , the set of locations , , , … , , the se, , , … , where ℓ , ℓ (subscript s tion indicators, respectively), and the set of intermed between start location and destination location of eed user trajectory T into one of movement paths of user.an get a classified path and predict user’s next locationion of the path. Fig. 2 shows that the location predict path classification problem.

Fig. 2. Problem transformation

tra-and

f the

m in-ory, t of and

diate each By

n by tion

A HMM-Based Location Prediction Framework with Location Recognizer 621

3.2 The Proposed Framework

The proposed framework for predicting user’s next location is briefly described as a graphical representation in Fig. 3. In this figure, we introduce a plate (the box labeled t) that represents t nodes of which three nodes are shown explicitly and a red rectangle to denote the problem transformation. We also introduce extra nodes such as and

, which mean the time when user visits the nth location and transportation mode when user visits the nth location, respectively. The information of these nodes enables user’s path to be modeled discriminatively.

Fig. 3. Graphical representation of the proposed framework

Location Recognition. Location recognition task is very important for predicting user location because sequence of locations is used to learn the path classification model. The green box in Fig. 1 represents the location recognition task. For recognizing us-er’s current location, we use the decision tree based method integrated with kNN. A decision tree is a tree-like graph that uses a branching method to illustrate every poss-ible outcome of a decision, and it could operate efficiently in mobile devices because it can be trained in fast time [4, 11].

Algorithm 1. Location Recognition

Input : Position (Current GPS coordinate) Output : Identified location name or “Unknown”

1: N ← kNN(Position) /* candidate loca-tions */

2: for k = 1, … , N do3: result ← DecisionTree(k)4: if (result = “Yes”) then5: return GetLocationName(k) 6: end if 7: end for 8: return “Unknown”

622 Y.-J. Kim and S.-B. Cho

The location recognition method firstly performs kNN at user’s current position, and then filters the locations of which current position is out of their boundary. Se-condly, decision trees of the filtered locations are invoked to identify current user location. As the input of the method, we use two types of information, GPS and Re-ceived Signal Strength Indicator (RSSI) of Wi-Fi access point. Algorithm 1 shows the location recognition method as a pseudocode.

Location Extraction. To model user path discriminatively, we extract the intermediate locations between start location and destination location by clustering GPS data. Most of previous works based on clustering method used k-means clustering method [2, 7, 8]. The k-means clustering method, however, is not suitable for real-world system because it needs the pre-knowledge about the number of k. Instead of using k-means clustering algorithm, we use Gaussian-means method, which is simply called G-means clustering method. The method is based on statistical test for the hypothesis that a sub-set of data follows a Gaussian distribution, and automatically chooses the number of clusters k by iteratively performing k-means clustering method until the test accepts the hypothesis that the data assigned to each k-means center are Gaussian [6].

Algorithm 2. Location Prediction Input : Observation sequence

: Output : Predicted destination

1: Path ← GetPaths( ) /* user paths starting from */

2: result := 0 3: index := 0 /* for path index */ 4: for k = 1, … , |Path| do 5: temp ← HMMP ( ) 6: /* finding the index having maximum value */ 7: if (result < temp) then 8: result := temp 9: index := k

10: end if 11: end for 12: ← GetDestinationOfPath(index) 13: return Predicted location

Path Classification. For classifying user path, we use HMMs as a classifier. A HMM is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved (hidden) states. HMMs are known for their applica-tion in temporal pattern recognition such as speech, handwriting, gesture recognition and so on [21]. The use of HMMs for path classification enables us to account for location characteristics as hidden states, and also to account for the effects of each individual’s previous actions.

A HMM-Based Lo

Equation (1) is the probarameter of user path of HMMs. The symbol is gognition, the time when uexploiting accelerometer, m

Equation (2) is the HMM cl

We describe the location pr

4 Personalized Lo

In this section, we presentframework. The system is cmodule, destination predicshows the architecture of th

Fig. 4. Architec

4.1 Log Collector

We implemented mobile setypes of sensors that we coas 0.5 second. Fig. 6(a) sho

ocation Prediction Framework with Location Recognizer

ability of the observation sequence , given the HMM . We use multiple information to make observation sym

generated by using location information from location rser visits location, and transportation mode classified

magnetic and orientation sensors [7]. | ∑ | , |lassifier. argmax |rediction algorithm as a pseudocode in Algorithm 2.

ocation Prediction System

t a personalized location prediction system based on composed of the sensor log collector, location recognitction module, and system management module. Fig

he location prediction system.

cture of the personalized location prediction system

ensor log collector, and integrated it into our system. Tllected are shown in Table 1, and we use collection perws the implemented log collector.

623

pa-mbol rec-

d by

(1)

(2)

our tion

g. 4

The riod

624 Y.-J. Kim and S.-B. Cho

Table 1. Various types of sensors used for data collection

No. Sensor Description

1 GPS Latitude, Longitude

2 Wi-Fi MAC address, RSSI

3 Acceleration 3-axis double type data

4 Magnetic Field 3-axis double type data

5 Orientation Orientation, Pitch, Roll

6 Time Stamp Date, Time

7 Transportation Mode Staying, Walking, Vehicle, Subway

4.2 Location Recognition and Prediction Module

Fig. 5 shows the flow of location prediction. Our framework for predicting user’s next location is implemented in the location recognition and prediction modules.

Fig. 5. Flow of location prediction

4.3 System Management Module

System management module consists of three parts including location management, path management, and model management. In location management, the locations labeled by user are managed in relational database. The user can add new locations, modify the information of registered locations, and delete existing locations. Fig. 6(c) shows a screenshot of location setting. The paths of user are also managed in relation-al database. Model management is responsible for learning location recognition and prediction models. Fig. 6(d) is an example of learning location prediction model in smartphone.

A HMM-Based Lo

Fig. 6. Screenshots of the impscreen; (c) specific location set

5 Experiments

To show the performance evaluating prediction accurperformance analysis in the

5.1 Dataset Description

For the experiments, fivesmartphone sensor data preber in 2012. We use the Sacollected data is shown in T

Tab

User #Locatio

User1 19

User2 26

User3 31

User4 23

User5 18

5.2 Performance Analy

To implement the predictioclustering. The number of e

ocation Prediction Framework with Location Recognizer

plemented system; (a) mobile sensor log collector; (b) initial mtting; (d) model learning for destination prediction

of our prediction method, we conduct an experiment racy. In this section, we give an account of the dataset e experiments.

n

e undergraduate students of Yonsei university collecesented in Table 1 for two months from September to Ocamsung Galaxy S3 as the experiment device. The amounTable 2.

ble 2. The information of collected data

on #Path #Intermediate Loca-tion

Data Size

15 97 514MB

8 127 505MB

22 202 754MB

10 109 701MB

7 74 246MB

ysis

on model, we cluster the dataset in Table 2 using G-meextracted intermediate locations is shown in Table 2.

625

map

for and

cted cto-nt of

eans

626 Y.-J. Kim and S.-B.

(a)

(c)

(e)

Fig. 7. Experimental results; (acies for each user-specific mov

Using implemented m(LOOCV) for each dataset sults. Fig. 7(a) shows averato (f) show the accuracy in

Cho

(b)

(d)

(f)

a) average prediction accuracy of users; (b)-(f) prediction accvement paths

models, we perform the leave-one-out cross-validatof the users in Table 2. Fig. 7 shows the experimental

age accuracy of each user’s dataset, and the graphs fromrespect of each path of each user, respectively.

ura-

tion re-

m (b)

A HMM-Based Location Prediction Framework with Location Recognizer 627

As a result of the experiment, the accuracy of several paths of each user is less than 70%, even if the accuracy of most of paths is higher than 80%. It is because the amount of data to learn the HMMs of these paths is not enough, and the movement patterns of each path are too diverse to model the mobility of each path.

6 Conclusion

We have designed a framework for the location prediction problem, and implemented a real system working on mobile devices. Location recognition module performs the kNN to select candidate locations using GPS, and uses decision trees using RSSI of Wi-Fi access points. For location prediction, we extract the intermediate locations of the paths by using G-means clustering method, and generate HMMs using user’s mul-tiple contexts, and predict user’s location using them. An experiment has been per-formed to evaluate the accuracy of prediction model on mobile devices. We achieved the average prediction accuracy which is higher than 87% through the experiment. As a future work, we will study about the incremental learning algorithm of location prediction model for real system to be adaptively learned through real-time data.

Acknowledgments. This research was supported by Samsung Electronics Co., Ltd.

References

1. Alvarez-Garcia, J.A., Ortega, J.A., Gonzalez-Abril, L., Velasco, F.: Trip Destination Pre-diction Based on Past GPS Log Using a Hidden Markov Model. Expert Systems with Ap-plications 37(12), 8166–8171 (2010)

2. Ashbrook, D., Starner, T.: Using GPS to learn significant locations and predict movement across multiple users. Personal and Ubiquitous Computing 7(5), 275–286 (2003)

3. Calabrese, F., Lorenzo, G.D., Ratti, C.: Human Mobility Prediction based on Individual and Collective Geographical Preferences. In: Proceedings of the 13th IEEE Intelligent Transportation Systems, pp. 312–317 (2010)

4. Caruana, R., Niculescu-Mizil, A.: An Empirical Comparison of Supervised Learning Algo-rithms. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 161–168 (2006)

5. Chen, G., Kotz, D.: A Survey of Context-Aware Computing Research. Technical Report TR2000-381, Dartmouth (November 2000)

6. Hamerly, G., Elkan, C.: Learning the k in k-means. In: Advanced in Neural Information Processing Systems, vol. 16 (2003)

7. Hightower, J., Consolvo, S., LaMarca, A., Smith, I., Hughes, J.: Learning and Recognizing the Places We Go. In: Beigl, M., Intille, S.S., Rekimoto, J., Tokuda, H. (eds.) UbiComp 2005. LNCS, vol. 3660, pp. 159–176. Springer, Heidelberg (2005)

8. Kang, J.H., Welbourne, W., Stewart, B., Borriello, G.: Extracting Places from Traces of Locations. In: Proceedings of the 2nd International Workshop on Wireless Mobile Appli-cations and Services on WLAN Hotspots, pp. 110–118 (2005)

628 Y.-J. Kim and S.-B. Cho

9. Krumm, J., Horvitz, E.: Predestination: Inferring destinations from partial trajectories. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 243–260. Springer, Heidelberg (2006)

10. Lee, Y.S., Cho, S.B.: An Efficient Energy Management System for Android Phone Using Bayesian Networks. In: Proceedings of the 32nd International Conference on Distributed Computing Systems Workshops, pp. 102–107 (2012)

11. Lim, T.S., Loh, W.Y., Shih, Y.S.: A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms. Machine Learn-ing 40(3), 203–228 (2000)

12. Löwe, R., Mandl, P., Weber, M.: Context Directory: A Context-Aware Service for Mobile Context-Aware Computing Applications by the Example of Google Android. In: Proceed-ings of the IEEE International Conference on Pervasive Computing and Communications Workshops 2012, pp. 76–81 (2012)

13. Mathew, W., Raposo, R., Martins, B.: Predicting future locations with hidden Markov models. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp. 911–918 (2012)

14. Monreale, A., Pinelli, F., Trasarti, R.: WhereNext: a Location Predictor on Trajectory Pat-tern Mining. In: Proceedings of the 15th International Conference on Knowledge Discov-ery and Data Mining, pp. 637–646 (2009)

15. Morzy, M.: Mining Frequent Trajectories of Moving Objects for Location Prediction. In: Perner, P. (ed.) MLDM 2007. LNCS (LNAI), vol. 4571, pp. 667–680. Springer, Heidel-berg (2007)

16. Noh, H.Y., Lee, J.H., Oh, S.W., Hwang, K.S., Cho, S.B.: Exploiting Indoor Location and Mobile Information for Context-Awareness Service. Information Processing and Manage-ment 48(1), 1–12 (2012)

17. Park, H.S., Oh, K., Cho, S.B.: Bayesian Network-Based High-Level Context Recognition for Mobile Context Sharing in Cyber-Physical System. International Journal of Distributed Sensor Networks (2011)

18. Petzold, J., Pietzowski, A., Bagci, F., Trumler, W., Ungerer, T.: Prediction of Indoor Movements Using Bayesian Networks. In: Strang, T., Linnhoff-Popien, C. (eds.) LoCA 2005. LNCS, vol. 3479, pp. 211–222. Springer, Heidelberg (2005)

19. Petzold, J., Bagci, F., Trumler, W., Ungerer, T.: Comparison of Different Methods for Next Location Prediction. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds.) Euro-Par 2006. LNCS, vol. 4128, pp. 909–918. Springer, Heidelberg (2006)

20. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, Inc., San Francisco (1993)

21. Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1989)

22. Scellato, S., Musolesi, M., Mascolo, C., Latora, V., Campbell, A.T.: NextPlace: A spatio-temporal prediction framework for pervasive systems. In: Lyons, K., Hightower, J., Huang, E.M. (eds.) Pervasive 2011. LNCS, vol. 6696, pp. 152–169. Springer, Heidelberg (2011)

23. Simmons, R., Browning, B., Zhang, Y., Sadekar, V.: Learning to Predict Driver Route and Destination Intent. In: Proceedings of the IEEE Intelligent Transportation Systems Confe-rence 2006, pp. 127–132 (2006)

24. Yavas, G., Katsaros, D., Ulusoy, Ö., Manolopoulos, Y.: A data mining approach for loca-tion prediction in mobile environments. Data & Knowledge Engineering 54(2), 121–146 (2005)