A literature review of Location-Based Services

Tom Hulmethul006 - 1559316

Department of Electrical and Computer Engineeringthul006@aucklanduni.ac.nz

ABSTRACTThis paper looks into the ways in which Location-Based Ser-vices are able to change the way we interact with the worldaround us. Using Augmented and Virtual Reality, and com-bining them with the camera, Wireless, and GPS systems inmobile devices; or combining them with external devices, weare able to create navigation systems to assist users in findingtheir way around both indoor and outdoor settings.

Author KeywordsLocation-based services; mobile navigation; indoornavigation; outdoor navigation; visual location recognition;visual localization; augmented reality; virtual reality;navigation for the visually impaired.

ACM Classification KeywordsH.5.m. Information Interfaces and Presentation (e.g. HCI):Miscellaneous

INTRODUCTIONDue to the ever increasing popularity of mobile devices,Location-based services can be easily created and used to in-teract with the areas around us. Mobile devices with camerascan be used to navigate through areas using virtual or aug-mented reality; where an image is captured through the cam-era, and then overlaid with information to help a user nav-igate. Being that this field of user interaction is fairly new,this paper reviews three existing research papers in their ex-ploration of the area, and how mobile devices can be used torecognise the world around a user and change how they inter-act it. The papers cover the use of a mobile device’s camera,wireless, and GPS systems to approximate the user’s location,and provide feedback through Augmented and Virtual Realitymethods.

BACKGROUND OF LOCATION-BASED SERVICESLocation-Based Services (LBS) are able to assist the waypeople interact with the world. LBS is used almost exclu-sively for navigation, but there are many aspects of navigationwhich can be assisted with the use of this technology. Seeing

that Location-Based Services are tied to the location of a user,navigation is perhaps the core use for LBS.

Forms of NavigationIndoor NavigationThe idea of navigating through indoor spaces with mobile de-vices is not a new one, but constant advancements in boththe technological capabilities and availability of these mobiledevices has meant the ability to easily set up systems to nav-igate indoors is much easier. In later sections we will coverhow indoor navigation is accomplished.

Outdoor NavigationNavigation with the use of maps has existed for thousands ofyears, but with mobile devices users are now able to find theirway through technological means such as GPS. Location-Based Services are able to build upon this foundation to pro-vide more extensive assistance in the area of outdoor naviga-tion.

APPROACHES TO LOCATION-BASED SERVICESLocation-Based Services have been implemented on mobileplatforms in various ways. These techniques often use a com-bination of a mobile device’s Camera, Wireless, and GPS ca-pabilities. In some cases, the potential to extend a mobileplatform has been implemented through the use of externalhardware. This section will cover existing research into im-plementations of Location-Based Services.

Vision Based LocalisationThe most common form of LBS is achieved through VisionBased means, which make use of the mobile device’s camera.In the paper A mobile indoor navigation system interfaceadapted to vision-based localization, [5] Moller et al. usea technique called visual localisation to recognise their sur-roundings using a mobile device. This estimates a user’s loca-tion based on the information received by the mobile device’scamera, and can also request that a user focus in a certain po-sition to improve the accuracy of a reading. This techniqueuses a method known as feature matching which queries ref-erence images for matching characteristics.

Schroth et al. also compared various location approximationservices in their paper Mobile Visual Location Recognition[6], but with a focus on outdoor Location-Based Services andoutdoor location approximation. One of the most interestingmethods used in this paper was similar to that of Moller etal. in that it used visual localisation, however it was alteredin a way such that it would query Google Street View or Mi-crosoft Side-Street Views to approximate the user’s location

1

Figure 1. Google Street View panorama matched to a low resolutionvideo recording (Union Square, San Francisco) using the visual locationrecognition system proposed in the article written by Schroth et al. (a)video frame, and (b) database panorama image. (Figure used courtesyof Google.) (From Schroth el at.)

[6]

(see Figure 1). Again, this method uses visual localisationand feature extraction to query a database (in this case GoogleStreet View) to get an approximation of the user’s location.The method used in this paper also used the most recently ap-proximated location to assist in locating the next position (asthe user can’t have possibly moved a great distance since lastscanning). Features are matched between the local video andGoogle Maps as shown in Figure 2. Any matched with treesand plants are ignored as they would cause variance acrossseasons.

Figure 2. Feature mapping shown over a Google Maps image. Matcheswith trees are ignored. (From Schroth et al.)

[6]

In Positioning and Orientation in Indoor Environments Us-ing Camera Phones [3] Hile et al. also used visual locali-sation to get a user’s position. Under the synonym featuremapping, the feature is the same as visual localisation in thatit creates a map of the user’s indoor surroundings such as;hallways, doors, and corners, then compares that with floorplans to get an approximate location for the user, as shown inFigure 3. This method required floor plans to work success-fully, so would only be viable in buildings where a floor planis readily available to be queried.

Wireless Local Area Network Polling

Figure 3. System diagram for calculating camera pose and overlayinginformation on a cameraphone image. Three input sources generate theaugmented image in a five-step process. (From Hile et al.)

[3]

Moller et al. [5] also reviewed other methods of locationestimation such as polling existing Wireless Local Area Net-work (WLAN) infrastructure, which uses the signal strengthof nearby wireless access points to estimate the location ofusers. This method would be expensive to implement, as alarge number of access points would be required, and wouldnot scale well as there would need to be a constantly densenetwork of WLAN access points.

Similarly, in their paper Indoor Tracking and Navigation Us-ing Received Signal Strength and Compressive Sensing on aMobile Device[1], Au et al. also looked into the ReceivedSignal Strength (RSS) from existing WLAN access pointsto approximate a user’s location, and found that RSS wouldbe a better technique to use over Time of Arrival and Time-Difference of Arrival, as these methods would require addi-tional hardware to generate accurate measurements. Loca-tion approximation using RSS requires an existing databaseof WLAN signal strengths at known locations. This tech-nique is called location finngerprinting, and data is collectedbeforehand and stored in a database (As shown in Figure 4).The actual location of the WLAN access points is not known,as Au et al. outline, but their exact location is not necessaryfor the system to work.

2

Figure 4. Indoor tracking system proposed by Au et al.[1]

The system proposed by Hile et al.[3] was intended intendedfor use in known environments, and as such, nearby WLANsources are queried to get an approximate location to narrowthe search. This provides a localised area to search within tomap. This was expected to provide an accuracy of approxi-mately 5 to 10 meters.

GPS LocalisationDunser et al. in their paper Exploring the use of handheld ARfor outdoor navigation [2] use a system which combines GPSwith a mobile phone’s built in accelerometer and compass toprovide data on the location and orientation of a user. Thisinformation was used to determine the location of a user andassist them with navigation along a path.Schroth et al. [6] speculated that GPS was only viable in anoutdoor setting with very few obstacles, as a device wouldneed to be visible by 4 GPS satellites. They found that themost inaccurate readings were given in urban environmentswhere tall structures obstructed the view of the satellites.Similarly, Legge et al. [4]In Positioning and Orientation in Indoor Environments UsingCamera Phones Hile et at. also state that with GPS systemsthey could potentially move their solution to map outdoorsareas as well - decreasing the search area by getting a user’sGPS coordinates and then attempting to user feature mappingfrom there. This method is somewhat similar to the methodsproposed by Schroth et al. in Mobile Visual Location Recog-nition.

External HardwareIn Indoor Navigation by People with Visual Impairment Us-ing a Digital Sign System [4], Legge et al. explore the use ofadditional hardware to assist the visually impaired with navi-gation indoors. Currently, outdoor navigation is widely avail-able thanks to GPS, and as such visually impaired individualsare able to navigate outdoors using speech based GPS naviga-tions systems. However, indoor environments are not suitablefor GPS navigation and as such must be dealt with differently.Additionally, Vision and WLAN based systems are too unre-liable for the visually impaired as they have huge difficultyin the area of error detection [4]. Legge et al. propose a”Digital Sign System”. This technique required an additionaltag reader and bluetooth headset to be connected to a mobilephone as shown in Figure 5. The Tag Reader would pick upon tags placed around points of interest inside buildings andrelay to the user navigational information.

Figure 5. Illustration of the requirements of the Digital Sign System.(From Legge et al.)

[4]

User InterfacesThe interfaces which provide a user with access to theLocation-Based Services are vital to their usability. Some ofthe papers reviewed had functioning prototypes with differentkinds of interfaces. Those not mentioned here are simply pro-totypes which were created as proof-of-concpets to demon-strate their ability to work.

Augmented RealityMoller et al. experimented using a mixture of Virtual andAugmented Realities to provide navigation services to users.The Augmented Reality (AR) takes the mobile device’s cam-era input and overlays an image which provides the user withthe information required to navigate, like a directional arrowpointing towards a goal for example. This method requiredthe user to hold their phone in a set position which had theissue of user fatigue. (see Figure 6-a) This issue was partiallysolved in virtual reality.

Dunser et al. produced three implementations of their so-lution, the first being Augmented Reality, the second being

3

Figure 6. A comparison of how the user will need to hold the mobiledevice when using an Augmented Reality interface (a) versus a VirtualReality interface (b) (From Moller et al.)

[5]

a Map interface (using Google Maps API) and the third be-ing a combination of both interface options, with a button toswitch between modes. (see Figure 7). The interface changedthe size of the overlain AR location pointer as the user movedcloser to it, transforming it to a 3D cube when the user iscloser than 50m to the location. The bottom-left corner ofthe screen also showed a radar-view which gave the user ageneral idea of their surroundings. (see Figure 7).

Figure 7. Augmented Reality interface with overlain location pointersand Radar (From Dunser et al.)

[2]

Virtual RealityThe implementation used by Moller et al. for Virtual Real-ity displays used a prerecorded image of the user’s location,which is panoramic and can therefore render every directionaround the user (see Figure 6-b). This also reduced the is-sue of arm fatigue as the users were not required to hold thephone near eye-level.

External HardwareThe implementation used by Legge et al. used a Tag Readeras shown in Figure 5. The user interface for this implementa-tion was geared towards the use of the system by the visually

impaired and as such used a bluetooth headset to feed infor-mation to the user. The system ran on a Windows Phone withthe standard 12 buttons (0-9, *, #). Again, this is allowableand preferable as the intended audience is the visually im-paired.

APPLICATION AND EVALUATION OF LOCATION-BASEDSERVICES

Indoor NavigationIndoor navigation was achieved using vision based recogni-tion [5, 6, 3], WLAN Polling [5, 1], and Tag Recognitionthrough external hardware [4]. These services provided accu-rate representations of the user’s location and were viable foruse within indoor environments.Vision Based systems proved most accurate, with the featurematching implementation of an indoor navigation system byHile et al. able to produce consistent indoor results with anaccuracy of 30cm. This was significantly greater than whatis currently achievable with GPS and WLAN polling. Hile etal. also found that such a small error margin meant that theoverlain image was acceptable when used.Schroth et al. found that due to the ready availability of cam-eras in mobile devices, this system is easy to implement asit requires no additional infrastructure or components for theend users. They were also able to produce a system with a2-3 second response time, which is key in enabling users touse he system - as many users would find a slower responsetime acceptable.Moller et al. argued that WLAN infrastructure would beproblematic to setup and would not scale suitably, but insistedthat Vision Based approaches were more suitable.The WLAN implementation produced by Au et al. found thatthe Received Signal Strength sample time of 1 sample persecond was not fast enough, and as a result were limited tohaving user’s of their system move slowly.Legge et al. found that sighted people were able to fasternavigate with their system than those that were visually im-paired. But this was expected, and was not seen to be an issueas they had everyone complete their navigation tasks success-fully. The only issue being that tags were placed in very spe-cific locations, and users had to have the tag reader facing thetag to recognise it.

Outdoor NavigationOutdoor navigation was achieved using vision based recogni-tion [6] and GPS systems [6, 2]. GPS location approximationwas also proposed by Hile et al. to extend their navigationsystem [3].Dunser et al. used GPS systems to get the location of theirusers. They found that the was a margin of error that was atmost 48m for the entirety of their study. This was outweighedby a minimum of 2m, resulting in an average of 7m for theentire study, which is appropriate considering they are navi-gating in outdoor environments. The error was likely due tointerference with GPS systems, as speculated by Schroth etal.Schroth et al. used Vision Based recognition primarily tomatch the surrounding areas with nearby locations on GoogleStreet View. This implementation also stated that it could

4

be extended to use GPS to better estimate the location of theuser, and where to start the search.

Navigation for the visually impairedLegge et al. looked into the application of Location-BasedServices using Tag Readers. They tested their implemenationwith 4 groups, those with normal vision as a control group,those with normal vision but blind folded, a group with aslight vision impairment, and a group of blind people. Theyfound that with their system, visually impaired groups wereable to correctly navigate through an indoor environment.Au et al. also tested their implementation with visually im-paired users and had an overall success rate of 93.3%. Theauthors suggested that their system was suitable for guidingvisually impaired through indoor environments.

User InterfacesMoller et al. found that Virtual Reality was seen to be a morereliable navigation system, as it was ”less influenced by inac-curacies”. The Virtual Reality solution was also better in thatit allowed users to rest their arms by not having to hold themup at eye level. Augmented Reality was also perceived tobe more inaccurate as an incorrect orientation was seen morenegatively than an incorrect location reading, as the super im-posed arrow pointed completely in the wrong direction.Dunser et al. found that users preferred a mixed implementa-tion which offered more options, as they were provided witha mix between the Augmented Reality and Map implemen-tation. They found this most satisfying to use through a userstudy.

SUMMARYThe paper A mobile indoor navigation system interfaceadapted to vision-based localization, [5] explained howMoller et al. were able to approximate a user’s location thenprovide them with a functional navigation service. The au-thors used the mobile device’s camera exclusively, approxi-mating the location with visual localisation, and applied anAugmented or Virtual Reality overlay to provide informationto the user.Schroth et al. used a similar method, but this time withoutdoor capability using a mobile device’s camera alongsideGoogle Street View to approximate the user’s location. Thesystem queried Google Street View to find a location thatmatched what the user’s camera input provided.Hile et al. in the paper Positioning and Orientation in IndoorEnvironments Using Camera Phones provided better accu-racy for Augmented Reality in an indoor setting. This pa-per was also using location-based services to provide naviga-tional support. The system outlined in this paper was signifi-cantly more accurate not only in location, but in direction ofa user - which was the main pitfall faced by Moller et al.Dunser et al. used an outdoor navigation system with an Aug-mented Reality System and Google Maps implementation.This Provided users with multiple options for wayfinding andwas seen as a more favourable approach than AR alone.Au et al. used a WLAN system which matched the ReceivedSignal Strength with a fingerprinting database to get the user’slocation. In their study they found that both sighted and vi-sually impaired people could use such a system, but were

confident their system was significant in helping the visuallyimpaired navigate through indoor spaces, as they received a93.3% success rate in a user study.Legge et al. used a tag reader system to navigate through in-door spaces. They tailored their solution to assist the visuallyimpaired, and as a result their solution was successful whentested with both blind and low vision users.

CONCLUSIONThere are a number of different applications for Location-Based Services, as well as a number of different methods toprovide them. If we are to use Augmented Reality, we needto be sure that we correctly and reliably get the direction auser is facing, as an incorrect heading has more of a negativeimpact on usability. The camera on a mobile device, coupledwith WiFi and GPS positioning seems to be most accuratewhen approximating a user’s location, and may even be cou-pled with Google Street View, or additional external devices.These methods can be used to provide user’s with a new wayto interact with the world around them, and allow anybody toeasily navigate unfamiliar territory.

FUTURE WORKThe area of Location-Based Services is still relatively newand quite under-developed. An application could still be de-veloped to make use of all the systems available, as the cur-rent implementations - as outlined in this report - are somewhat disjointed in both their approach and opinion of whatwill work. A system which could make use of GPS position-ing when outdoors, and WLAN RSS when indoors to get anapproximate location could then utilise that information whenusing Vision Based Localisation to get a user’s location couldbe used to create a more robust location-based navigation ser-vice.

REFERENCES1. Au, A., Feng, C., Valaee, S., Reyes, S., Sorour, S.,

Markowitz, S., Gold, D., Gordon, K., and Eizenman, M.Indoor tracking and navigation using received signalstrength and compressive sensing on a mobile device.Mobile Computing, IEEE Transactions on 12, 10 (Oct2013), 2050–2062.http://ieeexplore.ieee.org.ezproxy.auckland.ac.nz/xpl/articleDetails.jsp?arnumber=6261513.

2. Dunser, A., Billinghurst, M., Wen, J., Lehtinen, V., andNurminen, A. Exploring the use of handheld ar foroutdoor navigation. Computers and Graphics 36, 8(2012), 1084 – 1095. http://www.sciencedirect.com/science/article/pii/S0097849312001586.

3. Hile, H., and Borriello, G. Positioning and orientation inindoor environments using camera phones. IEEEComputer Graphics and Applications 28, 4 (2008),32–39.http://ieeexplore.ieee.org.ezproxy.auckland.ac.nz/xpl/articleDetails.jsp?arnumber=4557953.

4. Legge, G. E., Beckmann, P. J., Tjan, B. S., Havey, G.,Kramer, K., Rolkosky, D., Gage, R., Chen, M.,Puchakayala, S., and Rangarajan, A. Indoor navigation by

5

people with visual impairment using a digital signsystem. PLoS ONE 8, 10 (10 2013), e76783. http://dx.doi.org/10.1371%2Fjournal.pone.0076783.

5. Moller, A., Kranz, M., Huitl, R., Diewald, S., and Roalter,L. A mobile indoor navigation system interface adaptedto vision-based localization. In Proceedings of the 11thInternational Conference on Mobile and UbiquitousMultimedia, MUM ’12, ACM (2012), 4:1–4:10.

http://doi.acm.org.ezproxy.auckland.ac.nz/10.1145/2406367.2406372.

6. Schroth, G., Huitl, R., Chen, D., Abu-Alqumsan, M.,Al-Nuaimi, A., and Steinbach, E. Mobile visual locationrecognition. Signal Processing Magazine, IEEE 28, 4(2011), 77–89.http://ieeexplore.ieee.org.ezproxy.auckland.ac.nz/xpl/articleDetails.jsp?tp=&arnumber=5888650.

6