Embed Size (px)
Citation preview
Simultaneous Localization and Mapping with PowerNetwork Electromagnetic Field
Chris Xiaoxuan LuUniversity of Oxford
Yang Li∗Shenzhen University
Peijun ZhaoUniversity of Oxford
Changhao ChenUniversity of Oxford
Linhai XieUniversity of [email protected]
Hongkai WenUniversity of [email protected]
Rui TanNanyang Technological
Niki TrigoniUniversity of Oxford
ABSTRACTVarious sensing modalities have been exploited for indoorlocation sensing, each of which has well understood limita-tions, however. This paper presents a first systematic studyon using the electromagnetic field (EMF) induced by a build-ing’s electric power network for simultaneous localizationand mapping (SLAM). A basis of this work is a measurementstudy showing that the power network EMF sensed by eithera customized sensor or smartphone’s microphone as a side-channel sensor is spatially distinct and temporally stable.Based on this, we design a SLAM approach that can reliablydetect loop closures based on EMF sensing results. With theEMF feature map constructed by SLAM, we also design anefficient online localization scheme for resource-constrainedmobiles. Evaluation in three indoor spaces shows that thepower network EMF is a promising modality for locationsensing on mobile devices, which is able to run in real timeand achieve sub-meter accuracy.
CCS CONCEPTS• Human-centered computing → Ubiquitous and mo-bile computing;
∗This work was completed while Yang Li was with Advanced Digital Sci-ences Center, Illinois at Singapore.
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACMmust be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from [email protected] ’18, October 29-November 2, 2018, New Delhi, India© 2018 Association for Computing Machinery.ACM ISBN 978-1-4503-5903-0/18/10. . . $15.00https://doi.org/10.1145/3241539.3241540
KEYWORDSElectromagnetic field sensing; Power line network; Simulta-neous localization and mapping; Indoor localization
ACM Reference Format:Chris Xiaoxuan Lu, Yang Li, Peijun Zhao, Changhao Chen, LinhaiXie, Hongkai Wen, Rui Tan, and Niki Trigoni. 2018. Simultane-ous Localization and Mapping with Power Network Electromag-netic Field. In The 24th Annual International Conference on MobileComputing and Networking (MobiCom ’18), October 29-November2, 2018, New Delhi, India. ACM, New York, NY, USA, 16 pages.https://doi.org/10.1145/3241539.3241540
1 INTRODUCTIONRecent years have seen the increasing need of location aware-ness by mobile applications. As of November 2017, 62% of thetop 100 free Android Apps on Google Play require locationservices. While GPS can provide outdoor locations with sat-isfactory accuracy, indoor location sensing for mobiles hasbeen a hard problem. Research in the last two decades hasexploited various sensing modalities for indoor location sens-ing. Examples include various radio frequency (RF) signals(e.g., WiFi [11, 16, 50, 51], GSM [17], FM radio [6]), visiblelight [49, 52, 55], imaging [10, 13, 14, 54], acoustics [41], andgeomagnetism [12, 18, 22, 35, 36, 44]. However, each sensingmodality bears some limitations. RF signals have change-able propagation paths and received signal strengths (RSSes)due to barriers and transmitters’ dynamic power control.Visual lights are vulnerable to blockage. Imaging is powerdemanding and privacy breaching. Geomagnetic sensing issusceptible to change of altitude.This paper explores a new modality for indoor location
sensing, i.e., the ambient electromagnetic fields (EMFs) gen-erated by the electric power networks in civil infrastructures.These power networks supply alternating current (ac) volt-ages with an effective value of 220V or 110V in a nominalfrequency of 50Hz or 60Hz to the electric appliances dis-tributed in the buildings. As a result, the electromagnetic
1
radiations (EMRs) from the powerlines in a building willform an ambient EMF. The noises induced by the powernetwork EMF at electronic devices, called mains hums, areundesirable in general and often removed by the devices’built-in filters. However, recent studies leveraged the peri-odic and synchronous properties of the mains hums to designclock synchronization approaches for indoor sensors [33]and wearables [48]. Moreover, a study in [23] shows that themains hums sensed by indoor mote-class sensors carry timeinformation with sub-second resolution, due to the subtle butspatially homogeneous imperfections in the mains hum’s pe-riodicity. While the powerline EMR’s time-related propertieshave been the focus of existing studies, the spatial propertiesof power network EMF and whether these properties canbe exploited for indoor location sensing have received nosystematic research.Hypothetically, as the powerline EMR intensity decays
with the physical distance from the emitting powerline, thepower network EMF formed by the superposition of theEMRs from the permanent powerlines running in the build-ing infrastructures will give a spatially distinct but tempo-rally stable intensity field, which is a basis for location sens-ing. To verify our intuition, we use two types of sensors, i.e.,a customized EMR receiver and smartphone microphone, toconduct extensive measurements in various buildings. TheEMR receiver is based on a tank circuit tuned to the powernetwork’s nominal frequency, while the smartphone micro-phone can perform side-channel sensing due to the leakedmains hum in audio recordings. Our measurement resultsconfirm positively our hypothesis in the domain of indoorpedestrian walkways. In particular, compared with geomag-netic fields that often show altitude-varying intensity, powernetwork EMFs exhibit better altitude-invariability, suggest-ing that the power network EMF-based location sensing canbe less susceptible to user height.Based on the measurement study, we design a Simulta-
neous Localization and Mapping (SLAM) approach exploit-ing the unique characteristics of power network EMF. Ourapproach jointly estimates both the user locations and thepower network EMF feature map of the workspace, withoutlabor-intensive surveying or fingerprinting. The constructedpower network EMF feature map is metrically consistent,where locations are associated with corresponding powernetwork EMF signatures. In particular, we bespoke our SLAMapproach for the power network EMF modality, and designnovel loop closure detection and curation algorithms to reli-ably recognize previously visited locations from the observedpower network EMF signals. This is of crucial importance forSLAMusing power network EMF, as standard techniques (e.g.directly comparing the EMF signals at different locations)would produce many false positive loop closure proposals,
0 0.04 0.08 0.12 0.16 0.2
Time(s)
-1
-0.5
0
0.5
1
EM
R a
mp
litu
de
EMR
0 0.04 0.08 0.12 0.16 0.2
-400
-200
0
200
400
ac v
olta
ge
(V)
ac voltage
Figure 1: The normalized EMR amplitude around alive wire and the ac voltage between the live wire andthe neutral wire.
and thus significantly jeopardize the consistency of the fea-ture map and deteriorate localization performance. Given theconstructed power network EMF feature map, we also designan efficient online localization scheme capable of positioningthe user in real time on resource-constrained devices.We evaluate the proposed SLAM approach in realistic
indoor/semi-outdoor environments including a lab space, anoffice building and a shopping center. The evaluation showsthat, (i) with the customized EMR sensor, we can constructthe power network EMF feature maps with sub-meter accu-racy, and achieve meter level localization with the registeredmaps, (ii) on commercially off-the-shelf (COTS) platforms,the composite modality of geomagnetism and power net-work EMF (sensed through microphone) can have compara-ble performance, and (iii) by exploiting unique properties ofpower network EMF including its stability and smoothness,we could achieve real-time localization (0.69 s per measure-ment) on embedded platforms.The remainder of this paper is organized as follows. §2
introduces the background of powerline EMR sensing. §3presents a measurement study. §4 presents the details ofthe power network EMF-based SLAM and localization ap-proaches. §5 presents the evaluation results. §6 reviews re-lated work. §7 concludes.
2 POWERLINE EMR SENSING2.1 BackgroundThe EMR around an ac powerline is a composite effect ofthe time-varying electric field generated by the line’s acvoltage and the magnetic field generated by the ac currentgoing through the line. Fig. 1 shows the readings of an EMRreception device (cf. §2.2) placed close to a 220V 50Hz livewire and the corresponding ac voltage between the live wireand the neutral wire. We can see that the EMR follows the acvoltage well. A modern civil infrastructure often has massivepowerlines running in the walls, above ceilings, below floors,to power various appliances distributed in the indoor space.The intensity of the EMR attenuates with the distance fromthe line, where the attenuation can be modeled by the powerlaw. As a result, those many powerlines in the infrastructure
2
10 20 30
Time(s)
100.210099.8
10 20 30
Time(s)
100.210099.8
Fre
quency (
Hz)
Figure 2: The 2nd harmonic of the powerline EMRssensed by an EMR sensor (upper) and a smartphonemicrophone (lower). The smartphone can capture the2nd harmonic faithfully.
will generate an EMF that has a sophisticated spatial intensitydistribution over the indoor space, which highly depends onthe routing of the powerlines. It is thus of great interest thatwhether the power network EMF has the spatial distinctnessand temporal stability properties desired by location sensing.Electromagnetic emanations from building power net-
works have been exploited in several applications. For in-stance, in a circuit seeker system [2], an injector injectsmodulated signals into a power outlet and then a seeker canidentify the routing of nearby embedded powerlines by sens-ing the corresponding powerline EMR. A building’s powernetwork can also be used as an effective AM radio antenna[1], because the size of a building can be close to AM radiowavelengths (i.e., hundreds meters). Different from these ex-isting applications that use the building power networks toemit certain waves, this paper inquiries whether the powernetwork EMF, an inevitable side effect of electricity delivery,can be exploited for location sensing.
2.2 Powerline EMR ReceptionWe use following two techniques to sense powerline EMR.
Customized Powerline EMR Sensor: We build a power-line EMR sensor consisting of a tank circuit and a signalamplifier. Our design uses a combination of a 470mH coilinductor and a 22 µF capacitor, to achieve a resonance fre-quency of 50Hz (i.e., the power network frequency in ourregion). By using a 15 µF capacitor, the tank circuit can betuned to 60Hz, which is the nominal frequency in Americas.A two-stage amplification circuit is applied to amplify theµV-level output of the tank circuit to a volt-level input toan analog-to-digital converter mounted on a Raspberry Piboard. Our customized powerline EMR sensor can be readilyintegrated with various robot and drone platforms.
Side-channel Sensing by Microphone: The internal of amicrophone is a capacitor consisting of a membrane and ametal conductor. The vibration of the membrane in responseto sound results in time-varying capacitance between the
0 10 20 30 40 50
X (m)
0
10
20
30
40
Y (
m)
A B C D
Figure 3: Test points (A, B, C, D) and walking trajecto-ries (in thistle color) in an office building.
membrane and the metal conductor, which is translated to avoltage signal. The power network EMF can induce a time-varying voltage across the metal conductor, resulting in themains hum in audio recordings. The mains hum can be ex-tracted using a band-pass filtering algorithm. In this paper,we use smartphones’ built-in microphones and extract themains hum and its harmonics. Fig. 2 shows the frequencyof the second harmonic sensed by a customized powerlineEMR sensor and a smartphone microphone. We can see thatthe smartphone can capture the second harmonic faithfully.The results based on the above two types of sensors will
provide insights into the limits of power network EMF-bornespatial information and practicality of power network EMF-based location sensing on COTS mobile platforms.
3 MEASUREMENT STUDYWe conduct experiments to investigate whether the powernetwork EMF has the properties desired for location sensing.
3.1 Power Network EMF’s SpatialDistinctness
This set of experiments investigates whether the power net-work EMF intensities at different locations in an indoor spaceare distinct. We use the customized powerline EMR sensorsto perform measurements in an office building with an areaof about 2, 475m2. Fig. 3 shows the floor plan. We set thesampling rate of our sensors to 8 kHz and applied a 5-stepmoving filter to remove high-frequency noises in the sam-pled signals.
Basic Time/FrequencyCharacteristics:Weplace our pow-erline EMR sensors at four locations in a corridor by everythree meters, which are marked by A, B, C, and D in Fig. 3.Fig. 4(a) shows the power spectral densities of the powernetwork EMF intensities collected at the four locations. We
3
50 100 150 200 250
Frequency (Hz)
Magnitude
A B C D
(a)
0 50 100
Time (ms)
Norm
aliz
ed E
MF
A B C D
(b)
Figure 4: Time/frequency Characteristics. (a) Powerspectral density. (b) 100ms signal snippets. The curvesin the figures are vertically displaced for better visualcomparison among them.
0 50 100 150
Distance (m)
-40
-20
0
EM
F (
dB
FS
) 158cm 173cm 188cm
(a)
0 50 100 150
Distance (m)
30
40
50
60
Ma
gn
itu
de
(u
T) 158cm 173cm 188cm
(b)
Figure 5: Power network EMF’s invariability and geo-magnetic field’s variability with user height. (a) Powernetwork EMF. (b) Geomagnetic field.
can see that they have similar characteristics, i.e., the sensedsignal consists of a 50Hz major component and several har-monics. Fig. 4(b) shows the 100ms snippets of the sensedsignals at the four locations in the same time duration. Thepower network EMF intensities at different locations havedistinct waveforms and amplitudes. This distinctness is con-sistent with our understanding, as power network EMF hasa spatial distribution. As discussed in §2.1, this distributionmainly depends on the routing of the powerlines. As the pow-erline routing usually stays fixed, the spatially distributedpower network EMF is promising for location sensing.
Invariability with User Height: The spatial distinctive-ness of indoor geomagnetic signals has been studied [8, 15].Power network EMF and geomagnetic field are different. Ge-omagnetic field is the magnetic field that extends from theEarth’s interior out into space. Due to the metal infrastruc-ture in the buildings, geomagnetic field is distorted in indoorenvironments and can be utilized as the location signature.Geomagnetic intensity is static over time. In contrast, thepower network EMF emitted from the ac powerlines is atime-varying field. Our following measurements show that,compared with the geomagnetic field, the power networkEMF exhibits better invariability with user height.
We recruit three userswith different heights (158cm, 173cm,and 188cm) who carry our powerline EMR sensor and asmartphone, and walk in a 150-meter pathway in a building.
0 10 20 30 40
Distance
0
0.1
0.2
0.3
PD
F Same Position
Different Position
(a) 1-step window size.
0 100 200 300
Distance
0
0.01
0.02
0.03
PD
F
*
Same Position
Different Position
(b) 9-step window size.
Figure 6: Distributions of DTW distance between twopower network EMF RMS segments at the same or dif-ferent locations.
Fig. 5 shows the root mean square (RMS) value measuredby our powerline EMR sensor and the geomagnetic magni-tude measured by the phone’s 3-axis magnetometer. EachRMS value is computed based on readings in a 20ms win-dow. Though both sensing modalities are affected by thesensor altitude, the geomagnetic sensing is apparently moresusceptible as the envelope of the geomagnetic magnitudevaries much with user height. In contrast, the envelope ofthe power network EMF RMS exhibits better invariabilityagainst user height. This result implies that the power net-work EMF signature map can be constructed based on datacrowd-sourced from users with different heights.
Location Discriminability: This set of experiments inves-tigates whether we can discriminate different locations inthe resolution of footsteps based on power network EMFmeasurement traces. A researcher carrying a powerline EMRsensor and a smartphone walks three loops along the trajec-tory showed in Fig 3. This trajectory covers all the pathwaysin the building. We compute the sensed signal’s RMS traceand divide it into non-overlap segments, where a segmentcorresponds to a footstep sensed by the smartphone’s iner-tial measurement units (IMUs). As the segments may havedifferent lengths due to varying walking speed and pattern,we use the dynamic time warping (DTW) to measure thesimilarity between any two segments. Fig. 6a shows the dis-tributions of the DTW distance between any two segmentscollected at the same location or different locations, respec-tively. The two distributions largely overlap, suggesting thatthe power network EMF RMS with a segmentation windowsize of one footstep is not location-discriminative. Differ-ently, if we segment the power network EMF RMS tracewith a segmentation window of nine footsteps, as shown inFig. 6b, the two distributions become less overlapped. Thismeans that with the 9-step window size, though not ideal,the location discriminability of power network EMF can bepotentially exploited by SLAM. In §4, we will explain howthe proposed SLAM approach can reliably distinguish the
4
same or different locations with the observed power networkEMF signals in more detail.
3.2 Power Network EMF’s TemporalStability
This section investigates the temporal stability of the powernetwork EMF. Although conventional SLAM systems onlyrequire short-term stability of loop-closure landmarks, long-term stability is still desirable, because the constructed fea-ture maps can be reused for user navigation afterwards.
Impact of Individual Electric Appliances:Operating sta-tus changes of electric appliances causes varying ac currentsin powerlines and therefore varying magnetic field. We con-duct a set of controlled experiments to investigate the im-pact of the on/off operating statuses of individual electricappliances on the measurements of a powerline EMR sen-sor. Fig. 10a shows the power network EMF amplitude indecibels relative to full scale (dBFS) measured at differentdistances from an electric appliance when it is on and off.We can see that the operating statuses of the appliances canaffect the power network EMF measurements. However, theimpact is less than 1 dBFS when the sensor is more than onemeter from the appliance. This is because that the magneticintensity attenuates with distance and the measurement ofthe sensor is dominated by the superposition of the EMRsfrom all powerlines in the building compared with the EMRcontributed by the tested appliance. Among the eight testedappliances, the operating status changes of the dryer, themicrowave, and the fridge introduce considerable impacton the sensor measurements in their near fields. However,the near fields of these appliances are generally limited com-pared with the walking areas of built environments. Fromthe measurement results, we can observe that switching offsome appliances may increase the measurements at certaindistances. This is because these appliances make negativecontributions to the EMR superposition at those distances.
Impact of Near-by Human Bodies: A researcher holds apowerline EMR sensor and stands still at an indoor loca-tion. Then, more persons join the experiment by standingstill about 0.5m from the researcher. Fig. 8 shows the errorbars of the sensor measurements when different numbersof persons stand around the researcher. We can see thatthe nearby human bodies have little impact on the sensormeasurements. Unlike the short-wavelength RF signals (e.g.,WiFi) that are susceptible to barriers, the 50Hz EMR signalhas an extremely long wavelength and thus are not affectedby small-size barriers like human bodies.
Temporal Stability: We deploy four customized powerlineEMR sensors at four different fixed locations in two countries
for 14 days. Each sensor records power network EMF inten-sities for one minute every two hours. Fig. 10a shows theerror bars of each sensor’s measurements in the one-minuteduration. We can see that their measurements are stable overtime. For comparison, Fig. 9 shows a smartphone’s WiFi RSSvalues for a number of access points over 24 hours. We canclearly see the temporal variations of the WiFi RSS.
We also measure the power network EMF intensity whentraversing a loop along the trajectory shown in Fig. 3 in awinter and the following summer. Fig. 10b plots the powernetwork EMF RMS traces in the two seasons. As the buildingshown in Fig. 3 is located in the Temperate Zone, the electricloadings of the building in winter and summer are differ-ent. However, as shown in Fig. 10b, the power network EMFtraces in the two seasons are desirably similar. This suggeststhat the ac voltage is the major exciter of the power networkEMF, rather than the loading-dependent current. The aboveresults show the power network EMF’s temporal stabilityin the tested sites, and imply that once a power networkEMF fingerprint database is developed through SLAM, it canbe used for localization in considerably long time periods.In case of building infrastructure changes and/or powerlinererouting, it is readily for existing crowd-sourcing localiza-tion tools to update the database timely.
3.3 Location Discriminability of PowerNetwork EMF Sensed by Smartphone
Phone’s PowerNetworkEMFSensing Performance:Asmicrophone is a side-channel sensor, we assess the impact ofvarious daily life sounds on its power network EMF sensing.A Huawei P9 phone is placed at a fixed location to recordaudio for five minutes. Meanwhile, we intentionally createvarious interfering sounds, i.e., music and human speech thatare most probably faced by SLAM/localization systems forpedestrians. Fig. 11 shows the distributions of the normalizedintensity of the 50Hz mains hum and its harmonics. Fromthe figure, we can see that the music and human speech havelarger impact on the 150Hz harmonic, compared with the50Hz mains hum and the 100Hz harmonic. This is becausethe 150Hz harmonic collides with some human voice com-ponents. Although human voice can be down to 80Hz [32]and collides with the 100Hz harmonic, such cases are rare.As some high-end phones can effectively suppress the 50Hzmains hum, we adopt the 100Hz harmonic only, unless weneed multiple harmonics to improve performance (cf. §4.4).
Mitigating Inefficacy ofGeomagnetism-based Sensing:A researcher carries the Huawei P9 phone and walks threeloops along the trajectory shown in Fig. 3. The phone’smagnetometer and microphone continuously record geo-magnetic and power network EMF data traces. This section
5
40 80 120 160
Distance (cm)
-22
-20
-18
-16
EM
F (
dB
FS
)
on off
(a) Boiler
40 80 120 160
Distance (cm)
-19
-18
-17
-16
-15
EM
F (
dB
FS
)
on off
(b) Computer
40 80 120 160
Distance (cm)
-22
-20
-18
-16
-14
EM
F (
dB
FS
)
on off
(c) Hair dryer
40 80 120 160
Distance (cm)
-30
-25
-20
-15
EM
F (
dB
FS
) on off
(d) Fluorescent light
40 80 120 160
Distance (cm)
-30
-20
-10
EM
F (
dB
FS
) on off
(e) Microwave
40 80 120 160
Distance (cm)
-65
-60
-55
-50
EM
F (
dB
FS
) on off
(f) Computer monitor
40 80 120 160
Distance (cm)
-25
-20
-15
EM
F (
dB
FS
)
on off
(g) Air purifier
40 80 120 160
Distance (cm)
-22
-20
-18
-16
EM
F (
dB
FS
) on off
(h) Fridge
Figure 7: Impact of individual electric appliances on power network EMF sensing.
Scenario-20
-18
-16
-14
EM
F (
dB
FS
)
handheld1 person2 persons3 persons
Figure 8: Nearby humanbody impact.
4 8 12 16 20 24
Time (hour)
12345678
Access P
oin
t
-100
-90
-80
-70
-60
RS
S(d
B)
Figure 9: Temporal varia-tion of WiFi RSS.
0 100 200 300
Time (h)
-30
-20
-10
EM
F (
dB
FS
)
P1
P2
P3
P4
(a)
0 100 200 300
Distance (m)
-30
-20
-10
EM
F (
dB
FS
)
winter summer
(b)
Figure 10: Temporal stability. (a) The power networkEMF amplitude measured at four fixed locations intwo countries over 14 days. Themeasurements labeledby P1 and P2 are obtained in countryA; these labeled byP3 and P4 are obtained in country B. (b) The power net-work EMF amplitude tracewhen a researcher carryingthe sensor walks along the same trajectory shown inFig. 3 in a winter and the following summer.
particularly investigates the feasibility of using microphone-sensed EMF to mitigate the localization inefficient points bygeomagnetic sensing. The insight for the mitigation is thatwhen geomagnetic features are less discriminative (smallgap) and insufficient to differentiate locations, power net-work EMF sensing can help enhance location distinctions,
0 0.5 1
Intensity
0
20P
DF
*
Music
Speech
Silent
(a) 50Hz
0 0.5 1
Intensity
0
20
PD
F
***
Music
Speech
Silent
(b) 100Hz
0 0.5 1
Intensity
0
10
20
PD
F Music
Speech
Silent
(c) 150HzFigure 11: Power network EMF captured by a smart-phone microphone.
0 10 20 30Distance
0
0.05
0.1
0.15
PD
F
Mag
micEMR
Mag+micEMR
(a) Same location
0 10 20 30Distance
0
1
2
3P
DF Mag
micEMR
Mag+micEMR
(b) Different locationsFigure 12: DTW distance distributions of variousmodalities at the same and different locations.
sensed with a COTS microphone. The vice is versa for in-efficient cases of same locations. To account for the vary-ing walking speeds in data collection, we use dynamic timewarping (DTW) to calculate the distance between sequencesfor both geomagnetic and EMF signals. The curves labeled“Mag” in Fig. 12a and Fig. 12b are the distributions of thetop 10% largest or smallest DTW distances between any twogeomagnetic signal segments collected at the same locationor different locations, respectively. We can see that amongthese worst cases, the same-location DTW distances are sig-nificantly larger than the different-location DTW distances,
6
Features location
EMF1 (x1,y1)
EMF2 (x2,y2)
EMF3 (x3,y3)
Odometry
EMFSensing
EMF SLAM
EMF Extraction
Segmentation
Loop ClosureDetection
Loop ClosureCuration
GraphOptimization
EMFLocalization
Feature Extraction
Effiicient Feat. Matching
Figure 13: System workflow.
undermining the recall and precision performance in loca-tion identification. The curves labeled “micEMR” are thecorresponding results obtained using the 100Hz powerlineEMR harmonic sensed by the P9’s microphone. Althoughthe micEMR’s two distributions overlap, the micEMR’s recalland precision performance will be clearly better than thegeomagnetism’s. The “Mag+micEMR” curves are the corre-sponding results using the average of the Mag-based andmicEMR-based DTW distances. The above results show that,on smartphone platforms, power network EMF and geomag-netism can be complementary in discriminating locations.A combination of geomagnetism and power network EMFcan mitigate inefficacy of sole geomagnetism-based locationsensing.
Power Consumption of Power Network EMF Sensing:According to the study [40], the power consumption of sam-pling microphone is low. On the other hand, the plethora ofacoustic-based virtual assists on smartphones and wearables(e.g., Apple Siri [37]) also require the microphones to be al-ways on. Therefore, in practice, our proposed approach canleverage the audio data collected by these virtual assistants,without incurring excessive energy consumption.
4 SLAMWITH POWER NETWORK EMFBased on the understanding obtained in §3, this section devel-ops a novel Simultaneous LocalizationAndMapping (SLAM)approach with the power network EMF. As discuss above,power network EMF tends to exhibit better temporal sta-bility comparing to geomagnetism, while providing certainspatial discriminability. However unlike other modalitiessuch as vision, loop closures candidates obtained by directlycomparing the power network EMF signal similarities of-ten contain many false positives, which could significantlydegrade the SLAM performance. To address this, the pro-posed SLAM approach employs novel loop closure detectionand curation algorithms, which are able to discover as manypotential loop closures as possible based on the sequentialpower network EMF measurements, while robustly rejecting
IC0,3
x0 x1 x2 x3 ...u1
z0 z3
u2 u3
Figure 14: Factor graph representation for EMF-SLAM.x is the hidden state of actual user position; z denotesthe power network EMF measurement; c is the loopclosure of same place.
the false positive loop closure proposals by checking theirspatial consistency.Fig. 13 illustrates the workflow of our SLAM and local-
ization approaches. Specifically, the sensed power networkEMF signals are bandpass-filtered to extract the 50/60Hz sig-nal and its harmonics. By using a dead-reckoning moduleon the motion data of the user, her trajectories/odometry isestimated. The filtered power network EMF signal is thensegmented into sequences based on footsteps estimated inodometry information. The power network EMF sequencesand odometry are sent to the loop closure detection moduleto first detect the revisited positions by the user. The detectedloop closures are then curated to remove false positives inthem. By using the SLAM algorithms with the estimatedodometry and curated loop closures, the accurate trajecto-ries of users can be recovered. By looking back the timestampof the recovered locations and their associated power net-work EMF signatures, a feature map can be constructed. Newusers’ location can be predicted via matching their powernetwork EMF readings with the feature map, even if theirodometry information is not available. Note that this SLAMapproach is fundamentally different from the standard fin-gerprinting. In our case, the feature map is automaticallyconstructed as users move across the space, rather than man-ually collected through the labor-intensive fingerprintingprocess, which requires the users to explicitly establish thecorrespondence between specific locations and their EMFfeatures. In addition, our SLAM approach can continuouslyimprove and enrich the EMF feature map as the users ex-plore the space more, e.g. in different walking directions, andeventually obtain a holistic view of the EMF signatures ofthe environment.
In what follows, we first present the problem formulationusing the Graph SLAM machinery [26] in §4.1, and thendiscuss the proposed loop closure detection and curationalgorithms in §4.2-§4.3. Finally, §4.4 provides an efficientonline localization scheme using the constructed power net-work EMF feature maps.
7
4.1 Graph SLAM FormulationStates, Measurements and Outputs: Fig. 14 shows thefactor graph representation of our SLAM formulation. Let xtdenote the hidden state, i.e., actual position of the user in theworkspace at discrete timestamp t . We assume that at eacht , the user observes a power network EMF measurement zt ,which contains the sequence of power network EMF signalsreceived within a window [t − τ , t]. Note that here we con-sider a signal sequence rather than point as the measurementbecause as shown in §3.1, longer power network EMF signalsequences tend to have more spatial discriminative power.We also assume that the relative displacement between twouser positions xt and xt−1 is captured by an odometry edgeut , e.g., estimated with PDR techniques. In practice, we oftenassume the initial location of the user (starting state I) can beknown through other sources [26], e.g., CCTVs or card swipesystems at the entrance of the venue. After the user finishestraversing the workspace, the SLAM module should outputthe estimated sequence of true locations of the user x0, ..., xT ,along with the power network EMF feature map, which canbe obtained by associating the observed power network EMFmeasurements z0, ..., zT with the state sequence.
Loop Closure using Power Network EMF Signals: In atypical SLAM setting, loop closure refers to the recognitionof when the user or agent has returned to a previously vis-ited location, e.g., via observing similar measurements. It isone of the key components of SLAM framework, since inmost cases, odometry measurements tend to be noisy, result-ing in unbounded errors in state and map estimation. Forinstance, our system uses a PDR module which relies on thelow-cost IMUs on mobile devices to estimate odometry, andinherently suffers from drift. Loop closure on the other hand,can enforce constraints (often on distance) between states,e.g., x0 and x3 in Fig. 14 are connected by an edge c0,3, indi-cating that they should represent the same location in theworkspace. This immediately reduces the uncertainty in stateand map estimation, and therefore, the quality of detectedloop closures has a direct impact on SLAM performance.In this work, we exploit the similarity between features at50Hz of the observed power network EMF measurements todetect loop closures. As shown later, by carefully designingdetection and curation algorithms, the power network EMFsignals can be used to generate effective and robust loopclosures across the workspace.
Graph Optimization: Given the odometry and loop clo-sures, the optimal state sequence X∗ = {x∗0, ..., xT ∗} can beestimated by solving:
X∗=argminX
∑i
| | fu (xi , ui )−xi+1 | |2Σi︸ ︷︷ ︸
odometry constraints
+∑<i, j>
| | fc (xi , ci j )−xj | |2Λi j︸ ︷︷ ︸EMF loop closure constraints
Timestamp Closest-pointtimestamp
Closest-pointdistance
1 42 0.94
24 129 0.21
58 280 0.25
129 231 0.18
231 129 0.18
280 58 0.25
(a) CPP
#129 #231#24
#24
#129
#231
(b) Proposal mergingFigure 15: Illustration example of closest-point profile(CPP) and loop-closing proposals. (a) Color-shadedfields are primary loop closures and the transparentfield is a subsidiary one. (b) Merging primary and sub-sidiary loop-closing proposals. Nodes in the graph areproposal nodes, and edges are proposed loop closures.
where fu (xi , ui ) describes the usermotion reported by odome-tery, with uncertainty characterized by covariance matrixΣi . fc (xi , ci j ) is the loop closure model, and in this paper weuse co-location with zero displacement, i.e., fc (xi , ci j ) = xj ,and the covariance is Λi j .
4.2 Loop Closure DetectionThis section presents the proposed approach of detectingloop closures using power network EMF signals.
Distance between Measurements: One key prerequisitefor any loop closure approach is the distance metric betweenthe measurements. In our context, a power network EMFmeasurement contains a sequence of power network EMFsignals. To account for different walking speed of the users,when comparing two measurements zt and zt ′ , we use dy-namic time warping (DTW) to align them, i.e. we first stretchthe two sequences and then use the smallest sum of the Eu-clidean distances between their corresponding points as thedistance metric. For each newly observed measurement zt ,we compute its distance with respect to all previous measure-ments, and maintain the pairwise distance between them ina matrix Q.
Closest Point Profile (CPP): Given the current Q, we tryto discover loop closures, i.e., measurements that are similarto each other and likely to represent the same locations. In-stead of directly working with Q, we firstly derive a closestpoint profile (CPP) representation, where for each measure-ment we only keep the timestamp of its closest peer and theirdistance. For instance, the entry (129, 213, 0.18) in Fig. 15means that for measurement z129, the closest measurement(i.e., most similar one) observed so far is z231, and their dis-tance is 0.18 under the distance metric discussed above. Note
8
x
y
PASS FAIL
x
y
x
yTi Tj Ti
Tj
x
yInitial pose
Final pose
Initial pose
Final pose TjTj
Ti with inverse
odometry
zv
Ti with inverse
odometry
zv
zv’
zuzu zu’
zu
zv (z )v’ (z )v’zv
zu’
zu’
zv’
z r’ z r
(a) Spatial consistency validation.
Ti
Tj
zv’
zv
zu’zuUPDATE Ti
Tj
zv’
zv
zu’zu
(b) Distance update induced by validation.Figure 16: An example of loop closure curation.
that, in practice, CPP may not necessarily be symmetric un-der the distance metric. For example, as in Fig. 15, the closestpeer of z24 is z129, but for z129, z24 is not the closest.
Mining Loop Closure Proposals from CPP: Intuitively,the entries in CPP are potential loop closures, each of whichrepresents the local best knowledge on closest measurementsand associated distances. Like most of the existing SLAMapproaches, we could just naively threshold the distance,and propose the top-k entries with smallest distances as loopclosures. However, in practice, such a technique is not ro-bust to noise, and could propose many false positives. Inthis paper, we consider a new approach exploiting the graphstructure of CPP. Concretely, we first discover the symmetricentries within CPP, e.g., as shown in Fig. 15, both z129 andz231 treat each other as the closest peer. The rationale is that,regardless of the absolute distances, the fact that both mea-surements confirm each other as the closest indicates thatthey are likely to observe the same landmark. Therefore, weinitialize the set of loop closure proposals using those sym-metric entries. To harvest more potential loop closures, wealso add entries whose 2-nearest neighbours are symmetric,e.g., entry (24, 129, 0.21) in Fig. 15(b) will be included in theloop closure proposal set, since measurement z129 appearsin an symmetric entry.
4.3 Loop Closure CurationGiven the set of loop closure proposals, in theory, we coulduse all of them to directly optimize the graph and solvethe SLAM problem (as in §4.1). However, in practice, theSLAM framework can be very sensitive to false positives,
i.e., even a small amount of erroneous loop closures wouldgreatly deteriorate graph optimization, and have a knock-on effect on its performance. To address this, we proposea loop closure curation approach, and exploit the spatialconsistency to reject false positives and iteratively discovernew valid proposals.
Spatial Consistency Validation: In our context, a trueloop closure contains a pair of measurements which observethe same landmark in the workspace, although they maybe captured at different timestamps or even by differentusers. Consider two loop closure proposals, involving twopair of measurements as (zu , z(u′) and zv , zv ′) as shown inFig. 16a(left). In this case, if both loop closures are true, thenwithin any consistent reference frame, the two trajectorysegments Ti and Tj defined by the four measurements willform a loop, since zu and zu′ , zv and zv ′ should be observedin the same locations. Consider the 2D Euclidean referenceframe where zu′ is positioned at the origin (0, 0). Concep-tually, if we move from zu′ (at the origin) to zv ′ using theodometry information encoded in Tj , hop to zv via the loopclosure link from zv ′ , and then traverse back according tothe reverse odometry in Ti , when reaching measurement zuwe should be able to return to the origin. This is because zuand zu′ should correspond to the same landmark. If this isthe case, we confirm that both loop closure proposals aretrue, and mark them as valid (ready to be used by the latergraph optimization). On the other hand, if we fail to returnto the origin, e.g., as shown in Fig. 16a(right), then at leastone of the two loop closure proposals is false positive. Inthat case we will not push any of them to later optimiza-tion, and temporarily keep them in the proposal set for nextvalidation.
Distance Update Induced by Validation: As discussedabove, if two loop closure proposals pass the the spatialconsistency validation, we believe that the two trajectorysegments (see Ti and Tj in Fig. 16b) would share the samestart and end locations in the workspace (zu and zu′ , zv andzv ′ in Fig. 16a(left)). On the other hand, this also indicatesthat measurements within the neighbourhood of those vali-dated loop closures should be spatially close, and there mightstill be valid loop closures missed by the previous detectionalgorithm. To exploit this constraint, we update the pairwisedistances of those measurements within that neighborhood.Concretely, given the validated loop closure containing
two measurements zu and zu′ (zu ∈ Ti , zu′ ∈ Tj ), we firstconsider the neighborhood of 2N + 1 measurements aroundzu and zu′ respectively, i.e., two sequences of measurements{zu−N , ..., zu+N } and {zu′−N , ..., zu′+N }. The size of N is de-termined by the shape similarity of the two trajectory seg-ments Ti and Tj : N ∝ 1/DTW (Ti ,Tj ), where DTW (Ti ,Tj )is the DTW distance based on odometry information. For
9
Lab Space Office Building
Shopping Mall (Indoor)
Shopping Mall (Outdoor)EMR Sensor
Figure 17: Powerline EMR sensor and test venues.
the 2N + 1 window, we define a Gaussian kernel: w (n) =
e−12 (Q(zu ,zu′ ) · nN )2 , where −N ≤ n ≤ N , and Q(zu , zu′ ) is the
distance between measurements zu and zu′ of the validatedloop closure. Then for each pair of measurements zu+n ∈ Tiand zu′+n′ ∈ Tj (−N ≤ n,n′ ≤ N ) between the two 2N + 1windows, we update their distance as follows:
Qnew (zu+n , zu′+n′ )=Q(zu+n , zu′+n′ )−β ·w (n) · Q(zu , zu′ ).
Here, Q(zu+n , zu′+n′ ) is the current distance, and β is theupdating rate which controls how much distance changeshould be applied.
Iterative Loop Closure Curation: With the updated Q,where measurements around the validated loop closureshave new distance values, we then drive the detection al-gorithm to discover any new loop closure proposals. Thisdetection-validation cycle runs iteratively: in each iterationwe first discover loop closure proposals, validate them, andfinally use the validation results as constraints to update ourprevious belief on distances between measurements, whichis then used by the loop closure detection algorithm in thenext iteration. This loop closure curation process continues,until there is no proposal can satisfy the spatial consistencyvalidation. In this way, our system is able to discover as manygenuine loop closures as possible, and as shown later in theevaluation, it is the key for our SLAM framework to buildaccurate and consistent power network EMF feature maps.
4.4 Efficient Real-time LocalizationIn essence, the proposed SLAM framework leverages theodometry information and power network EMF, to jointlyestimate the location of the user and the power network EMFfeature map of the workspace within a metrically consistentframe of reference. In practice, this process typically hap-pens offline in the cloud, since it requires optimization overall states and measurements (as discussed in §4.1). On theother hand, if the power network EMF feature map of theworkspace has been constructed using the proposed SLAMapproach (e.g., via crowdsourcing), we could position theuser directly with respect to the map. In the following, weshow that such localization can be achieved in real-time onresource-constrained devices without infrastructure support:once the power network EMF feature map is downloaded
to the user’s mobile device, she can be localized using thecarried device only.
Localization with Power Network EMF Feature Map:Assume that the workspace is a 2D discrete Euclidean space,and has been thoroughly surveyed by our SLAM approach. Inthis case, the produced power network EMF feature map canbe viewed as a 2D grid, where each grid represents a locationl and the associated power network EMF signature z̄l . In thispaper, z̄l is defined as the DTW centroid [30, 31] of all powernetwork EMF measurements observed at location l , which isalso a sequence of power network EMF signals. Therefore,localization with respect to the feature map can be cast intoa classification problem, where for a live power networkEMF measurement zt , we try to find the location that has theclosest power network EMF signature. In our context, weconsider a similar distance metric as in §4.2, which employsDTW to evaluate the similarity between power network EMFsignal sequences. However, unlike in SLAM (see §4.2) wherewe only use power network EMF features at 50Hz for ro-bustness, during localization we consider three frequencycomponents 50Hz, 100Hz and 150Hz (the harmonics), toexploit more discriminative power of power network EMFsignals. Therefore, the distance between the live power net-work EMF measurement zt and the power network EMFsignature at a given location is evaluated as the weightedsum of distances between signals at the three frequency com-ponents, where weights are learned from the data. Finally,given the pairwise distances between live power networkEMF measurement zt and power network EMF signaturesat different locations, the estimated location is given by aNearest Neighbour (NN) classifier.Sparse PowerNetworkEMFFeatureAnalysis:The abovelocalization scheme is intuitive and easy to implement, butmay incur substantial computational cost since a newly ob-served power network EMF measurement has to be com-pared against power network EMF signatures of all locationswithin the feature map, where each comparison involvesone run of DTW. This would pose significant negative effecton performance when feature maps become bigger/denser,or for resource constrained devices such as wearables. Toaddress this, we propose a sparse power network EMF fea-ture analysis approach which only works on much sparserpower network EMF signals to speed up processing. Theintuition is that within a short period, the power networkEMF signals tend to smooth with low variance, i.e., most val-ues are entered around the sample mean. Therefore, whencomparing a given measurement zt with the power networkEMF signatures in the feature map, we first use the meanvalues to filter out those locations whose mean power net-work EMF signatures are very far away from that of zt . Inour experiments, we find that this pre-processing step is very
10
0 10 20
X (m)
-15
-10
-5
0
5
10
15
Y (
m)
0 10 20
X (m)
-5
0
5
10
15
Y (
m)
GroundTruth
senEMR
0 10 20
X (m)
-5
0
5
10
15
Y (
m)
GroundTruth
micEMR+Mag
-20 0 20 40
X (m)
-40
-20
0
20
40
Y (
m)
-20 -10 0
X (m)
-20
-10
0
10
20
Y (
m)
GroundTruth
senEMR
-20 -10 0
X (m)
-20
-10
0
10
20
Y (
m)
GroundTruth
micEMR+Mag
(a) Floorplans.
-100 0 100 200
X (m)
0
50
100
Y (
m)
(b) Inertial trajectories.
0 50 100 150
X (m)
0
20
40
60
Y (
m)
GroundTruth
senEMR
(c) senEMR-SLAM.
0 50 100 150
X (m)
0
20
40
60
Y (
m)
GroundTruth
micEMR+Mag
(d) micEMR+Mag-SLAM.
Figure 18: Floorplans (a), inertial trajectories produced by PDR (b), and the maps estimated by the proposedsenEMR-SLAM (c) and micEMR+Mag-SLAM (d) in lab (top), office building (middle) and shopping mall (down).
effective: on average it can rule out about 90% of locationsin the feature map, with less than 2% drop in overall local-ization accuracy. In addition, given the sample mean µ (zt )and variance σ (zt ) of a power network EMF measurementzt , the signal sequence encoded in zt can be sparsified bycollapsing all values within [µ (zt ) − λσ (zt ), µ (zt ) + λσ (zt )]as zeros. λ is the sparsity factor, which controls how muchinformation would be omitted from the original signal. Theresulting consecutive zeros are then clustered together, andwe only keep the lengths of them in the sequence. We applythe same process to the power network EMF signatures, andthen run DTW on the sparse representations, which shouldbe much shorter than the original signals. The rationale isthat we have already used mean values to compare zt withthe power network EMF signatures, and thus values close toits mean won’t be able to contribute further information. Asshown later in the evaluation, this sparse analysis approachcan significantly reduce computation latency of localization.
5 EVALUATION5.1 Experiment SetupSites: We evaluated the performance of proposed power net-work EMF-based SLAM and localization approaches in threevenues across two countries, including (i) an office building
(1, 750m2), which was built over 20 years ago with stones andbricks, and reinforced with metal rebars; (ii) a semi-outdoorshopping center (11, 000m2) built recently close to a high-way; and (iii) a lab space (900m2) in another country, whichhas the same 50Hz powerline frequency but different layout.Fig. 17 shows the snapshots of these test venues.
Sensing Modalities and Platforms: In our experimentswe employed a variety of sensing modalities, and collectedthe following data: (i) power network EMF signals from a cus-tomized powerline EMR sensor as shown in Fig. 17, (ii) audiofrom microphones, (iii) geomagnetic field from the smart-phone magnetometer, and (iv) WiFi RSS on smartphones. Weused 5 devices with different specs, including Raspberry Pi 2and 3, Galaxy S7, Huawei P9, and Nexus 6.
Data Collection Protocol: We recruited 10 participantswith different heights (1.58m to 1.88m), and asked them tohold the devices and walk normally in the experiment sites.We collected about 5 hours of walking data in total from thethree sites, where longer trajectories with multiple loops areused to evaluate the proposed SLAM approach, while therest for localization. We employ Google Tango tablet [24]to obtain ground truth for both user trajectories and con-structed feature maps. The visual-inertial odometry is fusedwith manually inserted loop closures using visual SLAM [19].
11
0 2 4 6 8
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F
senEMR
micEMR
Mag
WiFi
micEMR+Mag
micEMR+WiFi
(a) Sensing modality
0 2 4 6
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F Proposed
Vertigo
Max-Mixture
RRR
NoValidation
(b) Validation methodFigure 19: SLAM performance in the lab space.
5.2 Experiment Results
Effectiveness of SLAMusing PowerNetworkEMF: Thefirst experiment studies the effectiveness of SLAM usingpower network EMF. We refer to the SLAM approach usingpower network EMF sensed via the customized receiver assenEMR-SLAM, while via microphone asmicEMR-SLAM. Thetwo proposed approaches are compared against: (i) WiFi-SLAM and (ii) Mag-SLAM, which use WiFi RSS [11, 50] andearth geomagnetic field [12, 44] for SLAM. Fig. 18 showsthe floorplans (a), the inertial trajectories produced by PDR(b), and the maps (shown as trajectories) estimated by theproposed senEMR-SLAM (c) and micEMR+Mag-SLAM (d)across three experiment sites. We see that in all sites both ap-proaches can producemuchmore consistent trajectories thanPDR, which are very similar to the ground truth. Fig. 19(a),20(a) and 21(a) further show the error distributions of mapsconstructed by different SLAM approaches using a varietyof sensing modalities. As we can see, senEMR-SLAM con-sistently outperforms the other single-modality SLAM ap-proaches, and can reach 0.42m accuracy for 90% of the timein the lab space, while achieving 3.34m mean accuracy in thevery challenging shopping center. On the other hand, Mag-SLAM has about 15% more error than senEMR-SLAM, whileWiFi-SLAM performs even worse. Although both exploit thepower network EMF, the performance of micEMR-SLAM isgenerally inferior to senEMR-SLAM due to the backgroundnoise in audio data. This is expected, however in the lab spacemicEMR-SLAM is still better than WiFi-SLAM (Fig. 19(a)). Inthe challenging shopping center, neither micEMR-SLAM norWiFi-SLAM is able to construct a reasonable map (errors notshown in Fig. 21(a)). On the other hand, when fused withother sensing modalities, e.g. micEMR-SLAM + Mag-SLAM,the SLAM accuracy could improve by 20% on average thanMag-SLAM alone.
Performance of Iterative Loop Closure Curation: Thisexperiment investigates the performance of the proposediterative loop closure curation algorithm. We compare ourapproach with the state-of-the-art loop closure checkingalgorithms: (i)RRR [21] which identifies valid loop closuresby clustering them; (ii) Max-mixture [28] which employs amixture model to capture probabilities of valid loop closures;
0 1 2 3 4
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F
senEMR
micEMR
Mag
WiFi
micEMR+Mag
micEMR+WiFi
(a) Sensing modality
0 2 4 6
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F Proposed
Vertigo
Max-Mixture
RRR
NoValidation
(b) Validation methodFigure 20: SLAM performance in the office building.
0 2 4 6 8 10
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F
senEMR
Mag
micEMR+Mag
(a) Sensing modality
0 20 40 60 80 100
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F Proposed
Vertigo
Max-Mixture
RRR
NoValidation
(b) Validation methodFigure 21: SLAM performance in the shopping center.
and (iii)Vertigo [39], which uses latent variables to model thevalidity of loop closures. We also include the baseline NoValidation, which naively pushes all loop closure proposalsto optimization without any validation. Fig. 19(b), 20(b) and21(b) shows the error CDF of different approaches. Firstly, wesee that loop closure validation is a necessary step, and cansignificantly improve SLAM accuracy: in the shopping center,the proposed approach can reduce the error to 20% (from15.06m to 3.34m), compared to no validation. Comparing toother loop closure validation techniques, our approach canachieve approximately 1.5-fold reduction in error and theaccuracy degrades gracefully. This is because our approachis able to iteratively validate and mine genuine loop closures,while the competing ones just run one-off validation.
Localization Accuracy: Now we evaluate the online lo-calization accuracy of different modalities given their con-structed feature maps. Intuitively, the performance of local-ization depends very much on the quality of maps regis-tered by SLAM. Therefore in this set of experiments we onlyconsider the lab and office settings, because feature mapsestimated by single modalities (senEMR, micEMR, Mag orWiFi -SLAM), even the best ones, have non-negligible errors,and it does not make sense to evaluate localization accuracywith respect to them. Similarly, we only compare the pro-posed power network EMF based localization approaches(senEMR-Loc using the customised receiver, and micEMR-Loc using microphones) with magnetic field based approach(Mag-Loc) since WiFi does not perform well. Fig. 22 showsthe CDF of localization accuracy. As we can see, senEMR-locoutperforms the other approaches, achieving mean accuracyof 1.59m in the lab space and 3.93m in the office building.Note that Mag-Loc performs particularly bad on the lab space(see. Fig. 22(a)). This is because we use data collected from
12
0 5 10 15 20 25
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F
senEMR
micEMR
Mag
micEMR+Mag
(a) Lab space
0 10 20 30 40
Error (m)
0
0.2
0.4
0.6
0.8
1
CD
F
senEMR
micEMR
Mag
micEMR+Mag
(b) Office buildingFigure 22: Localization performance of different sens-ing modalities.
a participant of height 162cm to build the magnetic featuremap, while tried to localise another of 190cm. This confirmsthat magnetism based approaches are not robust to height,while the proposed power network EMF based approaches donot suffer from this problem. On the other hand, using onlypower network EMF sensed via microphones (micEMR-Loc)has generally larger errors than that of using customizedreceivers (senEMR-Loc). However, when fused with mag-netic field (micEMR+Mag-Loc), the localization errors couldbe halved comparing to the single best modality: from 8.2m(micEMR-Loc only) to 3.7m in the lab space and 8.4m (Mag-Loc only) to 3.9m in the office building.Sensitivity to Power Network EMF Sequence Length:This experiment looks into the sensitivity of localization per-formance with respect to the sequence length of power net-work EMF signals included in onemeasurement.We considertwo realistic settings: (i) running senEMR-Loc on RaspberryPi (RP2 and RP3), which can be used in various IoT applica-tions; and (ii) running micEMR+Mag-Loc (fusion of micEMR-Loc and Mag-Loc) on smartphones, which provides an alter-native option for infrastructure-free positioning. As shown inFig. 24 and 25, we vary the length of power network EMF sig-nals in terms of steps: e.g. 2-step means in a power networkEMF measurement we include signals observed within thelast 2 steps. We see that senEMR-Loc and micEMR+Mag-Locachieve comparable localization accuracy: generally longersequences result in better accuracy, while the gaps becomemarginal when the sequence length is bigger than 6-step.For instance, for both senEMR-Loc and micEMR+Mag-Loc,using signals of 10-step would only improve localization ac-curacy by <0.2m comparing to 6-step. On the other hand,longer sequences take more time to process, e.g. using 10-step would significantly increase localization time as shownin Fig. 24 and 25 (up to ∼3s for one measurement), which isnot cost-effective.
Performance Gain by Sparse Feature Analysis: The lastset of experiments verify the impact of the proposed sparsefeature analysis on localization performance. We use thesame two settings (that is, senEMR-Loc on Raspberry Pi, and
25.11X 21.12X 20.56X 20.60X
Huawei Nexus RP2 RP30
10
20
Tim
e (
s)
standard
constrained
mean filter(proposed)
full-suite(proposed)
Figure 23: Running time of different feature analysisapproaches on test devices.
Table 1: Localization accuracy (m) of different featureanalysis approaches.
raw DTW constrained mean filter fullsenEMR 1.76 1.89 1.77 2.01micEMR+ Mag 4.15 5.58 4.20 4.71
micEMR+Mag-Loc on smartphone) as in the previous ex-periment, and compare the proposed sparse feature analysisapproach (in §4.4) with: (i) standard DTW, (ii) constrained-DTW [27] which imposes local constraints on searchingwhen aligning sequences, and (iii) a straw man version ofthe proposed approach, using mean filter only to pre-selectmatching sequences. For the proposed approach, we tem-porarily fix the sparsity level (controlled by λ in §4.4) to25%. This means that during sparse feature analysis, 25% ofvalues in the original signal would be set to zero since theyare close to the mean. Fig. 23 shows the wall-clock runningtime of different approaches, and the resulting localizationaccuracy is shown in Tab. 1. We see that even with meanfilter only, we can already run ∼10× faster than standard-DTW, while the drop in accuracy is negligible. On the otherhand, the full version of proposed sparse feature analysisapproach can achieve up to 25× performance gain in run-ning time compared to standard-DTW, at the cost of ∼15%accuracy loss, which is still comparable with the state-of-the-art (constrained-DTW). When we vary the sparse level(from 0% to 50% as shown in Fig. 24, we observed that sparsersignals take shorter time to process (∼0.1s less per 5% onRP3), while resulting in negligible increase in localizationerror (∼0.06m). This confirms that the proposed sparse fea-ture analysis approach can achieve desire trade-off betweencost and accuracy. In particular, as in Fig. 24, when using25% sparsity and 6-step power network EMF signals, it takes0.69s to process one measurement, which is within the timefor one footstep. Therefore, we could run real-time powernetwork EMF based localization on the resource constrainedRaspberry Pi with reasonable accuracy.
13
00
10
Err
or
(m)
5
20 1
Sparsity Level(%)
30 3
Steps40 5
750 9
(a) Error
00
2
10
Tim
e (
s)
20
4
Sparsity Level(%)
9307
Steps40 550 3
1
(b) Running timeFigure 24: Accuracy and running time on RP3.
00
10
Err
or
(m)
5
20 1
Sparsity Level(%)
30 3
Steps40 5
750 9
(a) Error
00
Tim
e (
s)
10
0.5
20
Sparsity Level(%)
9307
Steps40 550 3
1
(b) Running timeFigure 25: Accuracy and running time on Huawei P9.
6 RELATEDWORKAs discussed in §1, recent studies [23, 33, 48] have exploitedpowerline EMR for time-related services. In [29, 38], mod-ulated high-frequency signals are injected into a building’spower network and then the corresponding power networkemanations are sensed by a mobile device equipped with anEMR sensor for localization. However, the requirements ofone or more signal injectors and intensive in situ trainingintroduce significant system deployment overhead. The Hu-mantenna system presented in [9] uses the human body asan antenna to receive powerline EMR and recognize gestures.Preliminary results on recognizing the user’s location amongpre-spotted locations are presented. However, the physicalconnection with the user’s body and system training bringoverheads. To the best of our knowledge, this paper presentsthe first work on using power network ENF measurementscaptured by a standalone sensor or smartphone for SLAMand localization.Originally designed for robotics, SLAM has been applied
to mobile computing applications [3, 43]. However, due tomobile devices’ low-cost and noisy IMUs and human’s com-plicated motion patterns, SLAM for human-centric applica-tions is often challenging. Early solution [11, 50] use WiFiRSS and IMUs for SLAM. However, WiFi RSS suffers fromthe multi-path effects and the performance of SLAM de-grades significantly in dynamic environments, e.g., retailstores. Geomagnetism-based SLAM is recently introduced[12, 18, 44]. However, geomagnetism often exhibits high
variability with altitude. Recent SLAM systems also leveragevision techniques [10, 14], which raise privacy concerns [52],however. Our SLAM system differs from the above systemsin its new sensing modality – power network EMF – that ex-hibits desirable high repeatability. Our system also featuresa bespoke approach that detects power network EMF loopclosures reliably and produces better SLAM results.
Localization can be seen as a special case of SLAM wherethe map is known or obtained through SLAM. Existing local-ization approaches can be divided into two categories. First,model-based approaches estimate the distance between a userdevice and deployed anchors based on models of RSS [5, 7],angle of arrival [20, 46], or propagation time [25, 42, 45].However, non-line-of-sight conditions often degrade the per-formance of these approaches. Differently, powerline EMRwith extremely long wavelengths can well penetrate bar-riers. Second, fingerprinting approaches develop a map offeatures extracted from images [13, 54], sounds [41], geo-magnetic field [22, 35, 36], RF signals [6, 51], visible lights[49, 52, 55] and a mix of them [4, 34, 47]. However, theseapproaches generally require a laborious blanket processof (re-)fingerprinting all locations. Owing to the temporalstability of power network EMF, our approach requires aone-shot profiling only, coinciding with the concept of lightregistration [53]. Moreover, the profiling can be accomplishedvia crowd-sourced SLAM in an unsupervised manner.
7 CONCLUSIONThis paper systematically investigated the spatial distinct-ness and temporal stability of EMFs induced by buildings’power networks. Based on the result, we designed a SLAMap-proach that can reliably detect loop closures with power net-work EMF signals sensed by either a customized receiver ora smartphone’s microphone as a side-channel sensor. More-over, with the power network EMF feature map constructedby SLAM, we design an efficient localization algorithm thatcan position the user in real time on resource-constraineddevices. Our extensive evaluation shows that the power net-work EMF is a promising modality for indoor location sens-ing since it is ubiquitous, spatially distinct, temporally sta-ble, and noise resilient. By exploiting these advantages ofpower network EMF, the proposed SLAM and localizationapproaches are capable of achieving sub-meter accuracy,while run in real time on both smartphones and embeddedhardware. Advanced approaches for fusing power networkEMF measurements with the measurements in other sens-ing modalities, as well as 3D indoor localization with powernetwork EMF are interesting topics for future work.
14
REFERENCES[1] [n. d.]. Carrier Current Radio Systems. ([n. d.]). http://www.
radiosystems.com/carriercurrent.html.[2] [n. d.]. Greenlee Circuit Seeker. ([n. d.]). http://bit.ly/2tzf6EI.[3] Heba Abdelnasser, RehamMohamed, Ahmed Elgohary, Moustafa Farid
Alzantot, He Wang, Souvik Sen, Romit Roy Choudhury, and MoustafaYoussef. 2016. SemanticSLAM: Using environment landmarks for un-supervised indoor localization. IEEE Transactions on Mobile Computing(2016).
[4] Martin Azizyan, Ionut Constandache, and Romit Roy Choudhury. 2009.SurroundSense: mobile phone localization via ambience fingerprinting.In ACM MobiCom.
[5] Paramvir Bahl and Venkata N Padmanabhan. 2000. RADAR: An in-building RF-based user location and tracking system. In IEEE INFO-COM.
[6] Yin Chen, Dimitrios Lymberopoulos, Jie Liu, and Bodhi Priyantha.2012. Fm-based indoor localization. In ACM MobiSys.
[7] Krishna Chintalapudi, Anand Padmanabha Iyer, and Venkata N Pad-manabhan. 2010. Indoor localization without the pain. In ACM Mobi-Com.
[8] Jaewoo Chung, Matt Donahoe, Chris Schmandt, Ig-Jae Kim, PedramRazavai, and Micaela Wiseman. 2011. Indoor location sensing usinggeo-magnetism. In Proceedings of the 9th international conference onMobile systems, applications, and services. ACM, 141–154.
[9] Gabe Cohn, Daniel Morris, Shwetak Patel, and Desney Tan. 2012.Humantenna: using the body as an antenna for real-time whole-bodyinteraction. In Proceedings of the SIGCHI Conference on Human Factorsin Computing Systems. ACM, 1901–1910.
[10] Jakob Engel, Thomas Schöps, and Daniel Cremers. 2014. LSD-SLAM:Large-scale direct monocular SLAM. In European Conference on Com-puter Vision (ECCV).
[11] Brian D Ferris, Dieter Fox, and Neil Lawrence. 2007. WiFi-SLAM usingGaussian process latent variable models. In AAAI.
[12] Chao Gao and Robert Harle. 2015. Sequence-based magnetic loopclosures for automated signal surveying. In Indoor Positioning andIndoor Navigation (IPIN), 2015 international conference on.
[13] Ruipeng Gao, Yang Tian, Fan Ye, Guojie Luo, Kaigui Bian, YizhouWang, Tao Wang, and Xiaoming Li. 2016. Sextant: Towards ubiquitousindoor localization service by photo-taking of the environment. IEEETransactions on Mobile Computing (2016).
[14] Ruipeng Gao, Bing Zhou, Fan Ye, and YizhouWang. 2017. Knitter: Fast,resilient single-user indoor floor plan construction. In INFOCOM.
[15] Brandon Gozick, Kalyan Pathapati Subbu, Ram Dantu, and TomyoMaeshiro. 2011. Magnetic maps for indoor navigation. IEEE Transac-tions on Instrumentation and Measurement 60, 12 (2011), 3883–3891.
[16] Andreas Haeberlen, Eliot Flannery, Andrew M Ladd, Algis Rudys,Dan SWallach, and Lydia E Kavraki. 2004. Practical robust localizationover large-scale 802.11 wireless networks. In Proceedings of the 10thannual international conference on Mobile computing and networking.ACM, 70–84.
[17] Jeffrey Hightower, Sunny Consolvo, Anthony LaMarca, Ian Smith,and Jeff Hughes. 2005. Learning and recognizing the places we go. InInternational Conference on Ubiquitous Computing. Springer, 159–176.
[18] Jongdae Jung, Taekjun Oh, and Hyun Myung. 2015. Magnetic fieldconstraints and sequence-based matching for indoor pose graph SLAM.Robotics and Autonomous Systems (2015).
[19] Christian Kerl, Jurgen Sturm, and Daniel Cremers. 2013. Dense visualSLAM for RGB-D cameras. In IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS).
[20] Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, and Sachin Katti. 2015.Spotfi: Decimeter level localization using wifi. In ACM SIGCOMM.
[21] Yasir Latif, César Cadena, and José Neira. 2013. Robust loop closingover time for pose graph SLAM. The International Journal of RoboticsResearch (IJRR) (2013).
[22] Liqun Li, Guobin Shen, Chunshui Zhao, Thomas Moscibroda, Jyh-HanLin, and Feng Zhao. 2014. Experiencing and handling the diversityin data density and environmental locality in an indoor positioningservice. In ACM MobiCom.
[23] Yang Li, Rui Tan, and David KY Yau. 2017. Natural timestampingusing powerline electromagnetic radiation. In Proceedings of the 16thACM/IEEE International Conference on Information Processing in SensorNetworks. ACM, 55–66.
[24] Eitan Marder-Eppstein. 2016. Project tango. In ACM SIGGRAPH 2016Real-Time Live!
[25] Alex TMariakakis, Souvik Sen, Jeongkeun Lee, and Kyu-Han Kim. 2014.Sail: Single access point-based indoor localization. In ACM MobiSys.
[26] Michael Montemerlo, Sebastian Thrun, Daphne Koller, Ben Wegbreit,et al. 2002. FastSLAM: A factored solution to the simultaneous local-ization and mapping problem. In AAAI.
[27] Cory Myers, Lawrence Rabiner, and Aaron Rosenberg. [n. d.]. Per-formance tradeoffs in dynamic time warping algorithms for isolatedword recognition. IEEE Transactions on Acoustics, Speech, and SignalProcessing ([n. d.]).
[28] Edwin Olson and Pratik Agarwal. 2013. Inference on networks of mix-tures for robust robot mapping. The International Journal of RoboticsResearch (2013).
[29] Shwetak N Patel, Khai N Truong, and Gregory D Abowd. 2006. Power-Line Positioning: A Practical Sub-Room-Level Indoor Location Systemfor Domestic Use. Lncs 4206 (2006), 441–458. https://doi.org/10.1007/11853565
[30] François Petitjean, Germain Forestier, Geoffrey I Webb, Ann E Nichol-son, Yanping Chen, and Eamonn Keogh. 2014. Dynamic time warpingaveraging of time series allows faster and more accurate classification.In IEEE International Conference on Data Mining (ICDM).
[31] François Petitjean, Alain Ketterlin, and Pierre Gançarski. 2011. Aglobal averaging method for dynamic time warping, with applicationsto clustering. Pattern Recognition (2011).
[32] David Andrew Puts. 2005. Mating context and menstrual phase affectwomen’s preferences for male voice pitch. Evolution and HumanBehavior (2005).
[33] Anthony Rowe, Vikram Gupta, and Ragunathan Raj Rajkumar. 2009.Low-power clock synchronization using electromagnetic energy radi-ating from ac power lines. In ACM SenSys.
[34] Guobin Shen, Zhuo Chen, Peichao Zhang, Thomas Moscibroda, andYongguang Zhang. 2013. Walkie-Markie: Indoor pathway mappingmade easy. In USENIX NSDI.
[35] Yuanchao Shu, Cheng Bo, Guobin Shen, Chunshui Zhao, Liqun Li,and Feng Zhao. 2015. Magicol: Indoor localization using pervasivemagnetic field and opportunisticWiFi sensing. IEEE Journal on SelectedAreas in Communications (2015).
[36] Yuanchao Shu, Kang G Shin, Tian He, and Jiming Chen. 2015. Last-milenavigation using smartphones. In ACM MobiCom.
[37] Siri Team. 2017. Hey Siri: An On-device DNN-powered Voice Triggerfor AppleâĂŹs Personal Assistant. (2017). https://machinelearning.apple.com/2017/10/01/hey-siri.html.
[38] Erich P Stuntebeck, Shwetak N Patel, Thomas Robertson, Matthew SReynolds, and Gregory D Abowd. 2008. Wideband powerline position-ing for indoor localization. In ACM Ubicomp.
[39] Niko Sünderhauf and Peter Protzel. 2012. Switchable constraintsfor robust pose graph SLAM. In IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS).
15
[40] Sasu Tarkoma, Matti Siekkinen, Eemil Lagerspetz, and Yu Xiao. 2014.Smartphone energy consumption: modeling and optimization. Cam-bridge University Press.
[41] Stephen P Tarzia, Peter A Dinda, Robert P Dick, and Gokhan Memik.2011. Indoor localization without infrastructure using the acousticbackground spectrum. In ACM MobiSys.
[42] Deepak Vasisht, Swarun Kumar, and Dina Katabi. 2016. Decimeter-Level Localization with a Single WiFi Access Point.. In USENIX NSDI.
[43] He Wang, Souvik Sen, Ahmed Elgohary, Moustafa Farid, MoustafaYoussef, and Romit Roy Choudhury. 2012. No need to war-drive:Unsupervised indoor localization. In ACM MobiSys.
[44] Sen Wang, Hongkai Wen, Ronald Clark, and Niki Trigoni. 2016.Keyframe based large-scale indoor localisation using geomagneticfield and motion pattern. In Intelligent Robots and Systems (IROS), 2016IEEE/RSJ International Conference on.
[45] Yaxiong Xie, Zhenjiang Li, and Mo Li. 2015. Precise power delayprofiling with commodity wifi. In ACM MobiCom.
[46] Jie Xiong and Kyle Jamieson. 2013. Arraytrack: a fine-grained indoorlocation system. In Usenix NSDI.
[47] Han Xu, Zheng Yang, Zimu Zhou, Longfei Shangguan, Ke Yi, andYunhao Liu. 2016. Indoor localization via multi-modal sensing onsmartphones. In ACM Ubicomp.
[48] Zhenyu Yan, Yang Li, Rui Tan, and Jun Huang. 2017. Application-LayerClock Synchronization for Wearables Using Skin Electric PotentialsInduced by Powerline Radiation. In SenSys.
[49] Zhice Yang, Zeyu Wang, Jiansong Zhang, Chenyu Huang, and QianZhang. 2015. Wearables can afford: Light-weight indoor positioningwith visible light. In ACM MobiSys.
[50] Zheng Yang, Chenshu Wu, and Yunhao Liu. 2012. Locating in finger-print space: wireless indoor localization with little human intervention.In ACM MobiCom.
[51] Moustafa Youssef and Ashok Agrawala. 2005. The Horus WLANlocation determination system. In ACM MobiSys.
[52] Chi Zhang and Xinyu Zhang. 2016. LiTell: robust indoor localizationusing unmodified light fixtures. In MobiCom. ACM.
[53] Chi Zhang and Xinyu Zhang. 2017. Pulsar: Towards Ubiquitous VisibleLight Localization. In ACM MobiCOm.
[54] Yuanqing Zheng, Guobin Shen, Liqun Li, Chunshui Zhao, Mo Li, FengZhao, Yuanqing Zheng, Guobin Shen, Liqun Li, Chunshui Zhao, et al.2017. Travi-navi: Self-deployable indoor navigation system. IEEE/ACMTransactions on Networking (TON) (2017).
[55] Shilin Zhu and Xinyu Zhang. 2017. Enabling High-Precision VisibleLight Localization in Today’s Buildings. In ACM MobiSys.
16
