Graph-based Data Fusion of Pedometer and WiFi Measurements for ...
Graph-based Data Fusion of Pedometer and WiFi Measurements for Mobile Indoor Positioning Sebastian Hilsenbeck, Dmytro Bobkov, Georg Schroth, Robert Huitl, and Eckehard Steinbach Institute for Media Technology, Technische Universit¨at M¨unchen, Munich, Germany {s.hilsenbeck, dmytro.bobkov, schroth, huitl, eckehard.steinbach}@tum.de is yet to be found for environments without GPS signal reception. A second challenge is to provide this service for the large range of different devices in use. The often very limited computational and power resources therefore impose strong constraints on the tolerable algorithmic complexity of solutions. This is all the more important since a positioning technology in fact serves as enabler for higher-level locationaware applications which require resources on their own [13].
ABSTRACT
We propose a graph-based, low-complexity sensor fusion approach for ubiquitous pedestrian indoor positioning using mobile devices. We employ our fusion technique to combine relative motion information based on step detection with WiFi signal strength measurements. The method is based on the well-known particle filter methodology. In contrast to previous work, we provide a probabilistic model for location estimation that is formulated directly on a fully discretized, graph-based representation of the indoor environment. We generate this graph by adaptive quantization of the indoor space, removing irrelevant degrees of freedom from the estimation problem. We evaluate the proposed method in two realistic indoor environments using real data collected from smartphones. In total, our dataset spans about 20 kilometers in distance walked and includes 13 users and four different mobile device types. Our results demonstrate that the filter requires an order of magnitude less particles than state-of-theart approaches while maintaining an accuracy of a few meters. The proposed low-complexity solution not only enables indoor positioning on less powerful mobile devices, but also saves much-needed resources for location-based end-user applications which run on top of a localization service.
Recent developments in the field of mobile indoor localization generated a variety of techniques building on various types of signals. These include approaches based on step detection [8], WiFi signal strength [6], magnetic field anomalies [12], and visual appearance [17]. While no single system achieves sufficient reliability, in combination, they can complement each other and compensate their respective weaknesses [11]. In order to leverage this potential, effective sensor fusion techniques yield the key to a ubiquitous solution for indoor positioning [14]. Over the past decade, robust sensor fusion for localization and tracking has seen a lot of success in the robotics community. Especially Bayesian techniques, most prominently the particle filter, provide a powerful probabilistic methodology for fusing data from multiple sensors. Substantial research effort led to well-developed solutions which achieve high accuracy and robustness. These are critical requirements for autonomous robots to operate in uncontrolled environments. Inspired by these results, similar implementations of particle filters have been employed for pedestrian positioning [15].
Author Keywords
Indoor Navigation; Indoor Positioning; Graph-based Sensor Fusion; Ubiquitous Localization; Particle Filter; Location-based Services; Mobile Computing ACM Classification Keywords
On mobile devices, however, low computational complexity and low power consumption are more important qualities than sub-meter accuracy [11]. While robotic agents require high precision for avoiding obstacles or for manipulating objects of interest, people, on the other hand, are usually able to easily perform both of these tasks without the assistance of their mobile phone. Hence, sub-meter location information may be valuable, for instance, in augmented reality applications, but is less important for indoor navigation. Moreover, algorithms that are optimized for highest accuracy consume more resources and hence create limitations for users with less powerful devices.
G.3.8. Mathematics of Computing: Probability and statistics – Probabilistic algorithms (including Monte Carlo) INTRODUCTION
To enable ubiquitous location-based services, a reliable positioning and tracking technology for mobile devices is required that works outdoors as well as indoors [20]. While in outdoor areas navigation satellite systems such as GPS already provide reliable positioning, a corresponding solution Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. UbiComp’14, September 13–17 2014, Seattle, WA, USA. Copyright 2014 ACM 978-1-4503-2968-2/14/09...$15.00. http://dx.doi.org/10.1145/2632048.2636079
Based on these observations, our key contribution is a graphbased sensor fusion approach for processing measurements from multiple sources of sensor information as well as map knowledge. Our method is based on the well-known particle filter methodology, but we propose to formulate the en-
1
or on generalizing training data into signal strength models. While the above RADAR-system [1] employs simple nearest neighbor matching, Haeberlen et al. [10] propose a probabilistic scheme using Markov Localization for which they cluster reference fingerprints for individual rooms and fit a Gaussian model for each room. Youssef et al. [33] adapt this model to account for sample-correlation when aggregating subsequent WiFi measurements. Ferris et al. [6, 5] propose to model signal strength across an entire building using Gaussian processes which allows to extrapolate to areas with no reference data. In contrast to that, Letchner et al. [21] and Biswas et al. [2] argue that a continuous, building-wide model is too complex and unnecessary given the uncertainty of WiFi-based localization. Hence, they opt for an even sparser representation by mapping fingerprints to a graphbased reference database.
tire probabilistic model directly on a completely discretized, graph-based representation of an indoor environment. As a result, the algorithm achieves significantly lower complexity than existing implementations (one to two orders of magnitude compared to recent works in [22, 32]), while maintaining an accuracy of a few meters. To achieve this reduction in complexity, we employ a coarse quantization of the indoor space. However, instead of simply performing uniform quantization, we pursue an adaptive approach. This is possible since the inner layout of buildings defines a certain topology regarding how people tend to move through these spaces. Narrow corridors, for instance, are usually traversed along their length, whereas lateral motion is often of little relevance for the localization task. The purpose of our graph structure is therefore to remove irrelevant degrees of freedom from the map. This way, narrow spaces become one-dimensional, whereas open areas remain two-dimensional. Furthermore, by formulating our probabilistic model of the particle filter directly on this graph, the algorithm can fully benefit from the decreased complexity and focus on the actual tracking task.
A complementary approach to WiFi localization is pedestrian dead reckoning (PDR) using the inertial sensors available on modern smartphones, in particular, accelerometer, gyroscope, and magnetometer (compass). Due to the low accuracy of built-in smartphone sensors, traditional inertial navigation approaches, which perform integration of measurements to produce position estimates, cannot be used as they result in cubic error growth [3, 11, 30]. Hence, state-of-the-art solutions rely on more distinct features in the signal and employ step detection. Displacement (step length) and direction of motion are then estimated for individual steps. To this end, recent research relies on machine learning techniques and activity recognition has been extended from distinguishing not only between the user moving and standing still, but to further include estimating the walking speed, climbing on stairs, taking an elevator, and more [24, 27]. Gusenbauer et al. [9], for instance, use detected activities to constrain the position estimate to known locations of stairs and elevators.
To demonstrate and evaluate our graph-based sensor fusion engine, we combine it with a typical choice of sensors to form a complete indoor positioning and tracking system. We use the compass, an acceleration-based pedometer, and WiFi, although, other sensor inputs are equally possible. We evaluate our proposed approach in two experimental setups to demonstrate its robustness and accuracy in realistic scenarios. All in all, we collected a dataset that spans about 20 kilometers in distance walked with a total duration of four hours. The evaluation includes 13 users and four different mobile device types. In addition, we provide empirical justifications for our theoretical derivations and parameter selection.
While the near-ubiquity of WiFi access points enables absolute positioning in many public buildings, pedestrian dead reckoning adds relative motion information, which together allows seamless pedestrian tracking. As a result, most modern indoor localization approaches for smartphones rely on this combination of sensors [11]. Yet beyond WiFi and PDR, various other approaches have been explored: Haverinen and Kemppainen [12] introduce indoor positioning based on anomalies in the earth magnetic field caused by local building structures. For domestic applications, Patel et al. [25] propose to induce tones into the powerline. Both approaches use fingerprinting to determine the location of the user. Following recent advances in efficient content-based image retrieval, Schroth et al. [28] describe a camera-based indoor localization system that retrieves images (and their locations) that visually match the user’s current environment.
RELATED WORK
In the following, we provide an overview of the state of the art in pedestrian indoor positioning and tracking. As our work is mainly concerned with sensor fusion and, in particular, lowcomplexity solutions for mobile devices, we first briefly review different sensing modalities that are commonly used for indoor localization, before we discuss in more detail existing sensor fusion techniques and current challenges regarding computational complexity and power consumption. Sensor Modalities for Indoor Positioning
The arguably most prominent technique for smartphonebased indoor localization today is WiFi-fingerprinting introduced by Bahl and Padmanabhan [1]. A location is represented by a WiFi-fingerprint which lists visible access points and their respective received signal strength identifiers (RSSI). Positioning is performed by matching the WiFifingerprint that is measured on the mobile device to a database of reference fingerprints which need to be collected beforehand during a calibration phase. The location associated with the closest match is returned as position estimate.
Sensor Fusion for Localization and Tracking
So far, numerous techniques for indoor positioning on smartphones have been developed, yet there is not a single solution that alone can guarantee reliable service for long-term operation throughout a large number of buildings. Moreover, most techniques exhibit their strengths and weaknesses under different conditions so that, in combination, they can com-
Over the past decade, most research effort focused either on improving the matching of measurements to reference data 2
Quigley et al. [26] who rely on WiFi and visual information. Li et al. [22] set the state of the art for pedestrian tracking on hand-held smartphones using only built-in inertial sensors and map knowledge, whereas combinations of WiFi and pedestrian dead reckoning are described in [18, 27, 34].
plement each other and improve not only accuracy but also reliability of service. Consequently, effective fusion of multiple sensing modalities is considered the key to a reliable and ubiquitous indoor positioning service [11, 14]. As a result of their success in fields like robotics [31], Bayesian techniques are widely used for positioning tasks [7]. Particle filters belong to the class of nonparametric Bayes filters and are not restricted to either linear systems or unimodal distributions. In addition, they provide a direct way to take map knowledge into account during tracking. For this reason, they are considered state of the art in sensor fusion for localization and tracking [15].
While the used sensors vary, all these methods have in common that the particle filter operates on a two-dimensional and continuous state-space (i.e., the map). As a consequence, regardless whether the map is represented as a conventional occupancy grid map as used by Quigley et al. [26], a CADfloorplan as used by Li et al. [22], or a polygon-based map as used by Woodman and Harle [32], the degrees of freedom for the estimation problem are equivalent in all cases. The actual difficulty of the tracking task varies of course with the accuracy of the employed sensors, but for smartphone-based solutions, typically, a few thousand particles are required to represent a multimodal belief distribution [15, 21, 22].
The idea behind the particle filter is to represent the state posterior by a finite set of samples (particles). We briefly restate the basic algorithm in order to clarify notation and explain its complexity. The following three steps are performed in each iteration [31]:
Although modern high-end smartphones might just be able to cope with hundreds or thousands of particles from a computational point of view, this certainly limits battery life time and, in consequence, end-user applications which run on top of the positioning service. Examples are navigation, medical monitoring, or smart-home management [13]. Hence, the problem of complexity reduction remains an important open challenge today [11].
1. Predict each particle’s new state by sampling from the state transition probability distribution given its current state: xt ∼ p (xt |xt−1 , ut , M)
(1)
where xt and xt−1 are the respective particle state at times t and t − 1. A possible control input is denoted by ut . M stands for the known map of the environment. 2. For any measurements, update each particle’s importance weight according to the measurement likelihood function given its new state: wt = wt−1 · L(zt |xt , M)
The most effective way to reduce the number of particles is to take degrees of freedom out of the probabilistic estimation problem [19]. To this end, Shin et al. [29] propose to constrain the motion model for a pedestrian to be more likely to produce straight walking paths and ignore small heading deviations. Liao et al. [23], on the other hand, describe a sensor network for tracking people through a building (i.e., an external tracking infrastructure). They were among the first to propose to constrain the particles to move only along the building’s skeleton, defining a one-dimensional—yet still continuous—state space. The idea has also been adopted to WiFi-based pedestrian tracking for smartphones [4, 6, 21].
(2)
where wt and wt−1 are the respective importance weights at the current and previous timestep, and zt denotes the current measurement. 3. Resample the particle population (with replacement) according to the distribution of their importance weights: draw xt ∝ wt
(3)
Contributions
Note that these equations do not specify the actual mathematical models of the required probability distributions. In fact, they have to be specifically defined for any particular application. Since Equations 1 and 2 have to be evaluated for each particle, the complexity of the algorithm is O(N ) where N is the number of particles [31]. Hence, the critical factor regarding computational complexity is the number of particles necessary for reliable operation of the filter. Due to the limited computational and battery resources on a mobile device, it is our goal to formulate the sensor fusion problem so as to require as few particles (and hence resources) as possible.
We go beyond previous work and replace the continuous state space by a completely discretized, graph-based model. Moreover, while the skeleton in [23] gives a good approximation of narrow corridors, we found that it is not sufficient to represent large, open spaces. Our intuitive approach to adaptively construct the graph with respect to the available space in the local environment is hence more general. Furthermore, when switching from a continuous state-space to a discrete one, existing sampling-models cannot be applied directly. Therefore, we provide a completely discretized probabilistic model for pedestrian motion that is directly formulated on our graph-structure (instead of using a conventional formulation and projecting the result onto a graph). Only then, the filter is able to fully exploit the sparse formulation of the localization problem and thus minimize its required resources.
Given the large body of research in this field, there exist of course standard implementations of particle filters for localization: Hightower and Borriello [15] are among the first to demonstrate the strong benefits of particle-filter-based sensor fusion for pedestrian localization by combining ultrasound and infrared beacon signals. Remarkable accuracy is achieved by Woodman and Harle [32] using a combination of foot-mounted inertial sensors and WiFi positioning, and
Our graph-based sensor fusion is intended as a replacement for current particle filter formulations. While we demonstrate 3
its performance using pedometer and WiFi measurements, our sampling-model can, in fact, incorporate measurements from any existing state-of-the-art pedometer algorithm such as [22]. Likewise, additional sensors can be included using existing probabilistic measurement models, e.g., as described in [21] for WiFi or in [26] for visual localization. SYSTEM OVERVIEW
Before we go into more detail on the sensor fusion, we provide a short overview of the components used in this work to generate the necessary measurements. Figure 1. Graph extraction from a floorplan. Left: A two-dimensional floorplan and its skeleton; Right: Final graph after sampling the skeleton (one-dimensionally) and enclosed free space (two-dimensionally).
Our pedometer implementation follows the one by Gusenbauer et al. [9] and consists of two modules, one for activity recognition and one for step detection. Our activity recognition, however, distinguishes only walking from standing still. If the user is believed to be walking, the second module detects steps based on peak detection. In order to translate detected steps into forward motion, a linear step-frequencybased model is used: dt = a · ft + b,
one end to the other, whereas motion in the perpendicular direction is less likely, and also less important for localization. Further, in contrast to robotic applications, sub-meter accuracy is not required for pedestrian navigation systems. Hence, the purpose of our graph structure is to remove degrees of freedom from the map wherever they are irrelevant to the localization task. This way, narrow spaces become onedimensional, while open areas remain two-dimensional.
(4)
where ft denotes the current step frequency and dt the estimated displacement per step. As the parameters, a and b, of this model are heavily user-dependent, we adapt the model to the user’s individual walking patterns over time employing the method described by Li et al. [22]. For estimating the step direction, we rely on orientation measurements provided by the vendor of each smartphone model.
Generating the graph structure for our approach requires only a floorplan, and can be done in a few simple steps as shown in Figure 1: 1. We extract the one-dimensional skeleton from the floorplan. This is equivalent to its Voronoi Diagram.
The pedometer is complemented by measurements of WiFi signal strength, employing a standard fingerprinting approach as described by Biswas et al. [2]. In order to employ our graph-based data fusion approach, we further require a twodimensional floorplan for each floor-level of the indoor environment. While different methods can be used to collect floorplan and registered WiFi data, we rely for our experiments on the mapping platform presented in [16]. This way, we create occupancy grid maps of several campus buildings of our university and collect the necessary reference WiFi signal strength measurements. As explained in the next section, we subsequently employ an adaptive quantization approach in order to extract our graph structure from the floorplan.
2. We find larger open spaces (e.g., a lobby) in the map and add these two-dimensional areas to the Voronoi-Diagram. The result is a combination of one-dimensional traces along narrow parts of the building and two-dimensional regions within larger rooms. 3. To create the final graph, we sample nodes in regular intervals from this structure. The edge length (i.e., the sampling interval) is ideally chosen to be roughly equal to the average pedestrian step length. This way, the lowest number of particles can be used to accurately represent pedestrian motion, as explained in more detail in the section on parameter selection.
GRAPH-BASED SENSOR FUSION
In this section, we describe in detail our graph-based sensor fusion technique. As mentioned above, our positioning system is provided with the following sources of information:
State Space
• Relative displacement estimates from the pedometer
In contrast to standard particle filter models which typically operate on a continuous two-dimensional map, our state space is defined by the fully discrete graph structure which replaces the conventional map. Hence, in our approach, the commonly used two position coordinates are reduced to a single node identifier, and each particle maintains the following state vector:
• Absolute location estimates from WiFi-fingerprinting
• The current location nt in form of a node-ID on the graph
Graph Extraction
• The current orientation θt with respect to magnetic north
• Building layout (a floorplan) • Absolute orientation estimates from the compass
The inner structure of a building typically not only imposes hard constraints on where a pedestrian can walk (walls). It also outlines important priors where people are likely to walk. Narrow corridors, for instance, are usually traversed from
Pedometer Model
In the following, we define the state transition model which is used to predict the new particle state based on the current 4
the measured absolute orientation zθ,t (Equation 12).
state and incoming motion estimates from the pedometer (i.e., orientation and displacement). In order to propagate the filter only when necessary, we condition state updates on the arrival of pedometer measurements. This makes the filter run at step frequency, which lies one to two orders of magnitude below typical raw sensor update rates. This further reduces unnecessary resource consumption.
{nt , θt } ∼ p (nt , θt |nt−1 , θt−1 , zθ,t , zd,t , G) = p (nt |nt−1 , θt , θt−1 , zθ,t , zd,t , G) · p (θt |nt−1 , θt−1 , zθ,t , zd,t , G) = p (nt |nt−1 , θt , zd,t , G) · p (θt |zθ,t )
θt ∼ p (θt |zθ,t )
(5)
This sampling process can be implemented efficiently by using the following decomposition: 1. Sample the new orientation given the measured orientation:
p (nt |nt−1 , θt , zd,t , G) = X = p (nt , dt |nt−1 , θt , zd,t , G)
(6)
=
X
(7)
p (nt |nt−1 , θt , dt , zd,t , G) · p (dt |nt−1 , θt , zd,t , G) (16)
=
X
p (nt |nt−1 , θt , dt , G) · p (dt |zd,t )
(17)
dt
Hence, also the displacement can be sampled separately given its measurement (cf. Equation 7):
(8)
dt ∼ p (dt |zd,t ) 4. Sample the new node given the sampled edge and the particle’s displacement: nt ∼ p (nt |et , dt , G)
(15)
dt
3. Sample one of the current node’s outgoing edges given their respective directions relative to the particle’s orientation: et ∼ p (et |nt−1 , θt , G)
(14)
dt
2. Sample the actual displacement given the estimated displacement: dt ∼ p (dt |zd,t )
(13)
Since the displacement per step is an estimated quantity, we now introduce a random variable for the true displacement dt (Equation 15). As above, the resulting joint distribution of nt and dt is rewritten as the product of a conditional and a marginal probability. This enables us to model also the distribution of dt conditionally independent from all variables except the estimated displacement zd,t . For the distribution of nt , in turn, the estimate zd,t becomes irrelevant given the true displacement (Equation 17).
where the graph G now replaces the map M.
θt ∼ p (θt |zθ,t )
(11) (12)
This allows us to sample the particle orientation from its measurement in a separate step (cf. Equation 6):
Whenever a step is detected, the particles sample a new state {nt , θt } from the state transition probability distribution given the old state {nt−1 , θt−1 }, the measured displacement zd,t (step length), and the current orientation estimate zθ,t . Correspondingly, we rewrite Equation 1 for our approach as {nt , θt } ∼ p (nt , θt |nt−1 , θt−1 , zθ,t , zd,t , G)
(10)
(18)
Since the map is represented by a graph, motion takes the form of transitions along edges in the graph. Therefore, we now introduce a random variable et over edges (Equation 20). As before, the joint probability of nt and et becomes the product of a conditional and a marginal distribution (Equation 21):
(9)
Equation 9 essentially decides whether the current particle makes a transition along the sampled edge et , or stays at its current node, i.e., nt = nt−1 . If a transition is sampled, the particle moves to the node at the other end of edge et . The ratio between those particles that make a transition and particles that stand still should correspond to the ratio between the measured displacement and the edge length. This way, the expected value of the user’s location corresponds exactly to the sampled displacement. It is important to note that the above decomposition results in distributions which are simple to model and can be empirically learned from training data.
p (nt |nt−1 , θt , dt , G) = X = p (nt , et |nt−1 , θt , dt , G)
(19) (20)
et ∈G
=
X
p (nt |et , nt−1 , θt , dt , G) · p (ei |nt−1 , θt , dt , G)
et ∈G
(21) =
X
p (nt |et , dt , G) · p (et |nt−1 , θt , G)
(22)
et ∈G
Since for pedestrians forward motion is usually more likely than walking sideways or even backwards, the edge sampling distribution depends on the current node nt−1 (and thus its outgoing edges and their directions) and the particle’s orientation θt . Edges that are not connected to the current node nt−1 have zero probability to be sampled (cf. Equation 8).
Mathematical Derivation
In this section, we show how the above decomposition follows from Equation 5. The first step is to rewrite the joint probability of nt and θt as the product of a conditional and a marginal one (Equation 11). Given the true orientation θt , the resulting distribution of nt becomes conditionally independent from θt−1 and zθ,t . Likewise, the distribution of θt becomes conditionally independent from all variables except
Once an edge is sampled (thus fixing the direction of motion), the probability of a transition only depends on the edge length 5
Relative frequency
ket k and the sampled displacement dt (Equation 22). Hence, the transition sampling distribution decides whether a particle stays at node nt−1 or makes a transition to the node at the other end of edge et (cf. Equation 9). Pedometer-based Weight Update
As noted above, when sampling an edge, the model reflects typical motion preferences of pedestrians. An edge in a direction that corresponds well with the particle’s orientation is more likely to be sampled than an edge leaving, for example, in the perpendicular direction. The sampling process, however, can only take into account relative likelihoods of different edges. For instance, if only a single edge leaves a given node, all particles would sample this edge, no matter how well (or poorly) the direction of this edge is aligned with the particles’ individual orientation.
−5 0 5 RSSI difference from means [dBm]
10
To apply fingerprinting in our proposed graph-based approach, the reference fingerprints need to be mapped to graph-nodes. In contrast to [2, 26], we specifically want to avoid interpolation of measurements that correspond to different rooms (and hence are subject to different attenuation patterns). Hence, we use ray-casting on the floorplan to determine for each node in the graph all visible locations (i.e., in the same room) where reference fingerprints have been collected. With this subset of measurements, we perform distance-based interpolation to generate the fingerprint for the current node. This way, for each node, a reference fingerprint is interpolated using only measurements that have been recorded in line of sight.
(23)
The next step is to define the probabilistic measurement model: We assume conditional independence between signals of different WiFi access points. Although this is a rather strict assumption, it allows us to obtain a useful likelihood function of low computational complexity [2]. Similar to [26], we derive a likelihood function from Equation 2 as follows:
In our case, the likelihood function derives from the joint measurement probability which is conditioned on all sampled variables: (24)
Given edge, orientation, and displacement, the two measurements become conditionally independent (Equation 26). Both marginal distributions are further independent of the current node nt since they reflect relative motion information, regardless of location. Also, the measurement probability over displacements does not depend on orientation and, vice versa, the measurement probability over orientations does not depend on displacement (Equation 27): p(zd,t , zθ,t |nt , θt , et , dt , G) = = p(zd,t |nt , θt , et , dt , G) · p(zθ,t |nt , θt , et , dt , G) = p(zd,t |et , dt , G) · p(zθ,t |et , θt , G)
0 −10
drift. Whenever a WiFi scan is matched to the reference database, the filter processes the resulting information using a measurement model which updates the particles’ importance weights according to the observation likelihood. This model needs to reflect that, due to dynamic disturbances and gradual removal or addition of access points over time, WiFi-based positioning generally provides location information with substantial uncertainty, i.e., at room-accuracy on average.
The pedometer-based weight update can be derived as follows, starting with Equation 2:
L(zt |xt , M) ∼ p(zd,t , zθ,t |nt , θt , et , dt , G)
0.1 0.05
Figure 2. Distribution of stationary WiFi power measurements over time
The information that is lost due to proportional sampling can be included again by augmenting the state transition model with an importance weight update. This partitioning of information into a sampling and a weighting part comes with the valuable advantage that particles that do not correspond well with the map (i.e., the graph) are increasingly penalized over time. This, in turn, allows the filter to aggregate this information and to use resampling to replace hypotheses that have become unlikely by better ones.
wt = wt−1 · L(zt |xt , M)
0.15
wt = wt−1 · L(zt |xt , M)
(29)
For modelling the noise distribution in the WiFi signal, we choose a mixture of a Gaussian and a uniform distribution. We verified this assumption by performing a stationary observation of WiFi RSSI values over time. During the recording period of about six minutes, people were walking through the room, as can be expected in a realistic indoor environment. The resulting likelihood function has the following form: ! 2 X (zi − hi (xt )) L(zw,t |xt , G) ∼ exp − (30) σi2
(25) (26) (27)
i∈I
We model the likelihood according to the absolute sampling probability over edges given the particle’s orientation. The displacement likelihood, on the other hand, has no effect on the weighting as it contributes no information that is not already included in the sampling step:
where zi is the measured power of transmitter i, and hi (xt ) is the predicted power of transmitter i at location xt . I denotes the set of transmitters present in the current WiFi measurement, and σi is the standard deviation of the signal from transmitter i obtained from the training data (see Figure 2).
L(zt |xt , M) ⇒ L(zθ,t |et , θt , G) ∼ p (et |nt−1 , θt , G) (28)
PARAMETER SELECTION WiFi-based Weight Update
At this point, we have derived the complete mathematical model for our graph-based sensor fusion approach. The following part includes evaluations that are concerned with determining the pedometer error characteristics. From this, we
While the pedometer provides relative motion information, we use WiFi signal strength measurements to gain absolute position information in order to correct for the immanent 6
1.1
0.02 0 −60
−40 −20 0 20 40 Deviation of the heading from the true value [deg]
Average step length [m]
Relative frequency
0.04
60
Figure 3. Error distribution of orientation estimation including building parts that cause local compass biases.
Model Name Samsung Galaxy S3 LG Nexus 4 HTC One Galaxy Nexus
IMU Vendor STMicroelectronics 1 Qualcomm 2 STMicroelectronics 1 Invensense 3
σθ [deg] 9.2 4.9 10.0 9.4
Male 1 Male 2 Male 3 Male 4 Male 5 Female 1 Female 2
1 0.9 0.8 0.7 0.6 0.5 1
1.2
1.4 1.6 1.8 Average step frequency [Hz]
2
2.2
Figure 4. Relation between step frequency and step length for seven different users.
Table 1. Comparison of orientation algorithms provided by manufacturers of built-in inertial measurement unit (IMU) chips.
Height
Female 1.75 1.65
1.84
1.81
Male 1.72 1.87
1.80
Table 2. Body height of seven test users in meters.
derive the actual sampling distributions and train their parameters to implement the filter’s state transition model. Orientation Estimation
User Dependence of Step-length Estimation
We asked seven users to walk through a corridor of 106 m length, roughly following a straight path while holding the device in front of the body. Each run was repeated several times, accumulating close to ten kilometers of recorded data with a total duration of two hours. Figure 3 shows the distribution of deviations from true orientation, using the exact heading of the corridor as reference. It is important to note that this histogram includes not only the noise characteristics of the device’s compass, but also fluctuations due to the swinging motion of walking, as well as local biases due to earth magnetic field distortions throughout the building. While this results in rather large standard deviations σθ , this corresponds, in fact, to the complete error characteristic that the particle filter requires to be able to correctly process orientation measurements. Table 1 shows standard deviations for different device models. For orientation sampling (Equation 6), we approximate these results using a Gaussian distribution around the measured absolute orientation zθ,t : p (θt |zθ,t ) ∼ N zθ,t , σθ2 (31)
Having derived a general model for step length estimation (fitted to all users), we now compare how well it performs for individual users (again, using the above dataset). We distinguish two cases: One, where the user’s true height is known a priori (see Table 2), and one where the average value of 1.78 m is used instead. Figure 5 shows for each user the estimated displacement per step relative to the true displacement. Knowing the user’s true height only marginally reduces the overall standard deviation σd from 0.14 to 0.11. However, significant deviations remain regardless whether the height is known or not. Hence, we adapt our step length model during operation of the filter, following the approach described by Li et al. [22]. For displacement sampling (Equation 7), we approximate these results using a Gaussian distribution around the measured displacement zd,t :
where the standard deviation σd is learned from the deviations in Figure 5.
where the standard deviation σθ is learned from the histogram in Figure 3.
Step Detection
p (dt |zd,t ) ∼ N zd,t , σd2
(32)
Finally, we evaluate the performance of step detection alone, i.e., using ground truth labels for activity recognition. For this experiment, two users with two different devices recorded a dataset while exactly counting the number of true steps. Comparing this to the algorithm’s estimate, the resulting error rate of step detection turns out to be below 0.5%. As this is clearly overshadowed by errors in activity recognition and other modules, we can safely neglect this error source.
Step-length Estimation
In order to parametrize the linear model for step length estimation in Equation 4, we use the same dataset as above. For increased accuracy, the path is further segmented into small parts of known length. For each segment, we compute the average displacement per step, as well as the average step frequency from the duration of the respective run. Figure 4 shows the resulting diagram (step-length vs. step-frequency) for all seven users. We fit a linear model to the entire dataset using least squares to set the parameters for Equation 4.
Edge Sampling
When sampling an edge at a given node, i.e., deciding the direction of motion (Equation 8), we want the model to reflect expected motion patterns of a pedestrian. Hence, we employ for edge sampling a truncated Gaussian distribution over all
1
http://www.st.com/ 2 http://www.qualcomm.com/ 3 http://www.invensense.com/
7
Deviation from true displacement [%]
20
Unknown height Known height
100cm
100cm
130cm
10
80cm
0
Figure 6. State transition sampling (using ten particles for illustration). Left: An 80 cm step along a 1 m edge leads to transitions for eight of the ten particles, resulting in a weighted mean (expected value) at exactly 80 cm distance from the bottom node. Right: Iterative sampling for steps that are longer than a single edge.
−10 −20 Male 1
Male 2
Male 3
Male 4
Male 5 Female 1Female 2
Figure 5. Deviations of the estimated step length from the true displacement for different users. Bright bars: Average height is used (1.78 m); Dark bars: True body height is known.
can be achieved with our low-complexity approach, before assessing the system’s reliability. All in all, the evaluations include 13 users, who did not participate in the previous experiments for parameter selection, and four device types (running Android OS). While we implemented the full system as Android app to guarantee its functionality on a mobile operating system, for better analysis, we also record all sensor measurements during operation. For correct statistical analysis, each of the recorded runs is later evaluated over 100 Monte-Carlo-Simulations (i.e., different seeds for the random number generator).
edges that leave the current node nt−1 : N ∆θt , σe2 if |∆θ| ≤ 90◦ p (et |nt−1 , θt , G) ∼ (33) 0 otherwise where ∆θt = θ(et ) − θt is the angular difference between current particle orientation θt and the direction in which edge et leaves the current node nt−1 . The distribution is truncated beyond 90◦ to account for the extremely low likelihood that people walk backwards (in particular in a navigation scenario). Moreover, the standard deviation σe must also include quantization effects due to the graph-based nature of the filter. Hence, we set σe to 40 degrees.
Experiment 1: Accuracy vs. Complexity
Our first experiment has the goal to show that our graph-based sensor fusion technique is able to achieve comparable accuracy to state-of-the-art approaches in a realistic, complex indoor environment, but using no more than 50 particles.
Transition Sampling
The experiment is conducted on two floors of an office building covering together 5000 m2 and including numerous corridors, office rooms, and staircases. Six users were asked to walk through the building, holding their smartphone in hand. Their path is shown in Figure 7 and leads through corridors, various office rooms, and two staircases where they change the floor level (once up, once down). A single walk is over 300 m long and takes about five minutes per user.
After an edge is sampled which fixes the direction of motion, the decision whether the edge is traversed or not is modelled using the ratio between particle displacement and edge length (Equation 9): if dt ≤ 0 0 p (nt |et , dt , G) ∼ kedtt k if 0 < dt < ket k (34) 1 if dt ≥ ket k
To obtain ground truth, we follow the methods described in [21, 22, 34] and placed markers on the floor every two meters for which we carefully hand-labelled their true coordinates. When a user passes a marker, the current timestamp is recorded by the experiment supervisor (using an auxiliary app) so that the position error can be determined by comparing the particle filter belief at that time with the known coordinates of the marker. For both, initial localization and tracking, the particle filter is set to use exactly 50 particles.
If a displacement dt happens to be larger than the current edge length ket k, the particle makes a certain transition along the current edge, and the pedometer model is run again but using only the remainder of the displacement d0t = dt − ket k. As a result, for any given edge, the expected value of the user’s location will correspond exactly to the sampled displacement, as illustrated in Figure 6. This observation also allows us to find a good choice for the edge length in our graph: Since we want to employ as few particles as possible, we have to set the edge length roughly close to the typical displacement per step (we set it to 1 m). Otherwise, more particles will be required to correctly represent the expected value of the user’s location.
Since accuracies are in general highly dependent on the experimental setup, we run for comparison all our collected data also through a state-of-the-art particle filter implementation, which operates on a continuous state-space as used, e.g., by Li et al. [22] or Woodman and Harle [32]. Figure 8 shows the cumulative error distribution for both versions. The results indicate that our solution is able to track the user with an accuracy of 1.52 m 50% of the time, and 4.53 m 90% of the time. We found that, in order to achieve comparable accuracy, in our experimental setup, a continuous particle filter requires at least 500 particles (which gives an accuracy of 1.38 m 50%
EVALUATION
Based on the filter parameterization derived in the previous section, we provide in the following an analysis of the localization system’s overall performance. Using two rather different experimental setups, we first evaluate the accuracy that 8
Figure 7. Walking path for experiment 1, leading the user about 300 m through several corridors, office rooms, and staircases on two floor levels of a multi-story office building. Our graph for this building is depicted in blue. The building is also shown in Figure 9 where it is located in the upper center. Empirical CDF
1
corridors with a total accessible area of 6000 m2 . Figure 9 shows the corresponding map and path. We asked seven users, who did not participate in the previous experiment, to walk through the entire series of buildings. They were free to turn around anytime, look at office signs or posters on the wall, etc. The recorded data cover more than six kilometers in distance walked. Ground truth is again realized using waypoints as described above, but in this case, leaving more space between markers (on average 30 m) as we are mainly interested in reliability. As above, the particle filter is set to use 50 particles throughout the entire experiment.
0.8
F(x)
0.6 0.4 0.2 0 0
graph-based, 50 particles continuous, 500 particles continuous, 50 particles 2
4
6 8 Positioning error [m ]
10
12
Regarding robustness, we find that 98% of the 700 runs (100 runs per user) are successfully tracked through the entire experiment, while for 2% of the runs, the filter looses track of the user at some point. However, when using WiFi fingerprints not only for likelihood updates but also for sampling a small fraction (we take 10%) of the particles from each incoming measurement, all 700 trials are tracked successfully. Figure 10 shows the cumulative error distribution for this case, where our approach is able to track the user with an accuracy of 2.19 m 50% of the time, and 7.16 m 90% of the time. To give an indication of the availability of WiFi in this experiment, Figure 10 also includes the cumulative error distribution when tracking whithout pedometer, relying on WiFi alone. This gives a median accuracy of 5.90 m.
14
Figure 8. Cumulative error distributions for experiment 1.
of the time, and 6.00 m 90% of the time). Using only 50 particles, however, the continuous particle filter is often unable to follow into the correct room, or fails to track the user entirely, hence deteriorating its accuracy. We further compared the time that is on average required to complete a single update step for a single particle in our graph-based filter as well as in the reference implementation, and found both to be roughly equivalent. Of course, the continuous filter version needs to perform this step ten times more often. Moreover, we observe that the continuous filter needs to apply resampling five times more often than our solution, resulting in additional overhead.
Finally, we observe that taking turns helps the particle filter to constrain its estimate. While this experiment contains many long stretches of straight corridors, experiment one benefits from a more complex path through corridors as well as office rooms, resulting in higher accuracy on average.
Experiment 2: Robustness DISCUSSION
Our second experiment aims at evaluating the robustness of our technique with only 50 particles when used for longer periods of time. The setup for this experiment spans eight connected buildings at our university campus including mostly
The results show that our proposed graph-based sensor fusion enables reliable tracking using no more than 50 particles. This represents a significant reduction of complexity 9
Empirical CDF
1 0.8
F(x)
0.6 0.4 0.2
Graph-based WiFi only
0 0
5 10 Positioning error [m]
15
Figure 10. Cumulative error distributions for experiment 2.
devices, but also saves well-needed resources for higher-level applications, which create actual value for the end-user. Figure 9. The 800 m walking path for experiment 2 leads through a series of eight connected buildings with a total accessible area of 6000 m2 .
CONCLUSION
We present a graph-based particle filter formulation for multisensor-fusion that enables pedestrian indoor positioning on mobile devices due to its low complexity. Our results demonstrate a reduction of at least one order of magnitude in computational complexity compared to state-of-the-art solutions while ensuring robust localization.
compared to the state of the art while maintaining comparable accuracy: Li et al. [22], for instance, rely on two orders of magnitude more particles (3000) as do Letchner et al. [21] who employ 1000 particles. Woodman and Harle [32] benefit from more accurate foot-mounted sensors but, nonetheless, require ten times more particles than our solution.
This is achieved by a coarse, adaptive quantization of the indoor space, which takes into account typical motion patterns of pedestrians in buildings. Hence, instead of employing a continuous state space in the data fusion, the indoor environment is represented by a sparse graph structure. By formulating our probabilistic model of the particle filter directly on this graph, the algorithm can fully benefit from the decreased complexity and focus on the actual tracking task.
On the other hand, there are limitations as well: The algorithm’s discrete nature introduces additional quantization errors compared to conventional map representations. In this paper, we parametrize our probabilistic sensor models such that they account for both measurement and quantization errors. This is possible since, for hand-held smartphones, the pedometer measurement error is by far larger than any quantization effects. However, this may not hold for significantly more accurate sensors such as foot-mounted IMUs or robotic odometry, where our solution might limit the achievable accuracy. Hence, our approach to sensor fusion clearly is most beneficial for pedestrian localization using hand-held devices.
Overall, the proposed approach is able to reliably track a pedestrian through an indoor environment using only 50 particles to process pedometer measurements in combination with WiFi signal strength measurements. In addition, our system is able to adapt to characteristic gait patterns for individual users. Our evaluation in a realistic indoor environment demonstrates the robustness of our approach for various devices and users.
Another limitation is the inherent assumption that the indoor environment features sufficient internal structure. Large empty halls (an extreme example would be an aircraft hangar) do not impose constraints on the filter and hence no notable complexity reduction can be expected. We observe that the approach works best for typical public buildings such as airports, museums, and universities, or likewise for office buildings (with numerous corridors and rooms).
We are currently in the process of extending our graph-based probabilistic model to also include the vision-based localization technique described in [28]. As this will incorporate an additional and very precise (meter-accurate) source of absolute position information, we expect a further boost in localization accuracy. ACKNOWLEDGMENTS
We propose our technique as a drop-in solution for multisensor positioning systems that already use some form of data fusion, especially, in the case of particle filters. Besides the sensor inputs used in this paper, the approach can be combined with any other source of information, and we encourage to extend the system presented here. Due to its low consumption of computational and power resources, the technique not only enables indoor positioning for a large range of mobile
This research project has been supported by the space agency of the German Aerospace Center with funds from the Federal Ministry of Economics and Technology on the basis of a resolution of the German Bundestag under the reference 50NA1307. The authors would also like to thank the reviewers for their constructive comments, and Maryam Banitalebi for her work and assistance in the field of activity recognition. 10
14. Hightower, J., and Borriello, G. A survey and taxonomy of location systems for ubiquitous computing. IEEE computer 34, 8 (2001), 57–66.
REFERENCES
1. Bahl, P., and Padmanabhan, V. RADAR: an in-building RF-based user location and tracking system. In INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, vol. 2 (March 2000), 775–784 vol.2.
15. Hightower, J., and Borriello, G. Particle filters for location estimation in ubiquitous computing: A case study. In UbiComp 2004: Ubiquitous Computing, N. Davies, E. Mynatt, and I. Siio, Eds., vol. 3205 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2004, 88–106.
2. Biswas, J., and Veloso, M. Wifi localization and navigation for autonomous indoor mobile robots. In IEEE International Conference on Robotics and Automation (ICRA) (May 2010), 4379–4384.
16. Huitl, R., Schroth, G., Hilsenbeck, S., Schweiger, F., and Steinbach, E. TUMindoor: an extensive image and point cloud dataset for visual indoor localization and mapping. In IEEE International Conference on Image Processing (ICIP 2012) (Orlando, FL, USA, Sep 2012).
3. Brajdic, A., and Harle, R. Walk detection and step counting on unconstrained smartphones. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’13, ACM (New York, NY, USA, 2013), 225–234.
17. Huitl, R., Schroth, G., Hilsenbeck, S., Schweiger, F., and Steinbach, E. Virtual reference view generation for CBIR-based visual pose estimation. In ACM Multimedia 2012 (Nara, Japan, Nov 2012).
4. Evennou, F., Marx, F., and Novakov, E. Map-aided indoor mobile positioning system using particle filter. In Wireless Communications and Networking Conference, 2005 IEEE, vol. 4 (March 2005), 2490–2494 Vol. 4.
18. Kothari, N., Kannan, B., Glasgow, E. D., and Dias, M. B. Robust indoor localization on a commercial smart phone. In Procedia Computer Science, M. Y. Elhadi Shakshukia, Ed., vol. 10, ELSEVIER (August 2012), 1114–1120.
5. Ferris, B., Fox, D., and Lawrence, N. Wifi-slam using gaussian process latent variable models. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) 2007 (2007), 2480–2485. 6. Ferris, B., H¨ahnel, D., and Fox, D. Gaussian processes for signal strength-based location estimation. In Proc. of Robotics Science and Systems (RSS) (2006).
19. Krumm, J., Williams, L., and Smith, G. Smartmovex on a graph - an inexpensive active badge tracker. In UbiComp 2002: Ubiquitous Computing, G. Borriello and L. Holmquist, Eds., vol. 2498 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2002, 299–307.
7. Fox, D., Hightower, J., Liao, L., Schulz, D., and Borriello, G. Bayesian filtering for location estimation. IEEE Pervasive Computing 2, 3 (2003), 24–33. 8. Foxlin, E. Pedestrian tracking with shoe-mounted inertial sensors. Computer Graphics and Applications, IEEE 25, 6 (Nov. 2005), 38–46.
20. LaMarca, A., Chawathe, Y., Consolvo, S., Hightower, J., Smith, I., Scott, J., Sohn, T., Howard, J., Hughes, J., Potter, F., Tabert, J., Powledge, P., Borriello, G., and Schilit, B. Place lab: Device positioning using radio beacons in the wild. In Pervasive Computing, H.-W. Gellersen, R. Want, and A. Schmidt, Eds., vol. 3468 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2005, 116–133.
9. Gusenbauer, D., Isert, C., and Kr¨osche, J. Self-contained indoor positioning on off-the-shelf mobile devices. In International Conference on Indoor Positioning and Indoor Navigation (IPIN) 2010 (Sept 2010), 1–9. 10. Haeberlen, A., Flannery, E., Ladd, A. M., Rudys, A., Wallach, D. S., and Kavraki, L. E. Practical robust localization over large-scale 802.11 wireless networks. In Proceedings of the 10th Annual International Conference on Mobile Computing and Networking, MobiCom ’04, ACM (New York, NY, USA, 2004), 70–84.
21. Letchner, J., Fox, D., and LaMarca, A. Large-scale localization from wireless signal strength. In Proceedings of the national conference on artificial intelligence, vol. 20, Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999 (2005), 15. 22. Li, F., Zhao, C., Ding, G., Gong, J., Liu, C., and Zhao, F. A reliable and accurate indoor localization method using phone inertial sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, UbiComp ’12, ACM (New York, NY, USA, 2012), 421–430.
11. Harle, R. A survey of indoor inertial positioning systems for pedestrians. Communications Surveys Tutorials, IEEE 15, 3 (Third 2013), 1281–1293. 12. Haverinen, J., and Kemppainen, A. Global indoor self-localization based on the ambient magnetic field. Robotics and Autonomous Systems 57, 10 (Oct. 2009), 1028–1035.
23. Liao, L., Fox, D., Hightower, J., Kautz, H., and Schulz, D. Voronoi tracking: location estimation using sparse and noisy sensor data. In Proceedings 2003 IEEE/RSJ of International Conference on Intelligent Robots and Systems (IROS 2003)., vol. 1 (Oct 2003), 723–728.
13. Hazas, M., Scott, J., and Krumm, J. Location-aware computing comes of age. Computer 37, 2 (Feb 2004), 95–97.
11
24. Park, J.-G., Patel, A., Curtis, D., Teller, S., and Ledlie, J. Online pose classification and walking speed estimation using handheld devices. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, UbiComp ’12, ACM (New York, NY, USA, 2012), 113–122.
29. Shin, H., Chon, Y., and Cha, H. Unsupervised construction of an indoor floor plan using a smartphone. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 42, 6 (Nov 2012), 889–898.
25. Patel, S. N., Truong, K. N., and Abowd, G. D. Powerline positioning: A practical sub-room-level indoor location system for domestic use. In UbiComp 2006: Ubiquitous Computing, P. Dourish and A. Friday, Eds., vol. 4206 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2006, 441–458.
30. Steinhoff, U., and Schiele, B. Dead reckoning from the pocket - an experimental study. In Pervasive Computing and Communications (PerCom), 2010 IEEE International Conference on (March 2010), 162–170. 31. Thrun, S., Burgard, W., and Fox, D. Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, 2005.
26. Quigley, M., Stavens, D., Coates, A., and Thrun, S. Sub-meter indoor localization in unmodified environments with inexpensive sensors. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). (Oct 2010), 2039–2046.
32. Woodman, O., and Harle, R. Pedestrian localisation for indoor environments. In Proceedings of the 10th International Conference on Ubiquitous Computing, UbiComp ’08, ACM (New York, NY, USA, 2008), 114–123.
27. Radu, V., and Marina, M. K. Himloc: Indoor smartphone localization via activity aware pedestrian dead reckoning with selective crowdsourced wifi fingerprinting. In Proceedings of the 4th International Conference on Indoor Positioning and Indoor Navigation (IPIN)., IEEE (2013).
33. Youssef, M., and Agrawala, A. The horus wlan location determination system. In Proceedings of the 3rd International Conference on Mobile Systems, Applications, and Services, MobiSys ’05, ACM (New York, NY, USA, 2005), 205–218.
28. Schroth, G., Huitl, R., Chen, D., Abu-Alqumsan, M., Al-Nuaimi, A., and Steinbach, E. Mobile visual location recognition. IEEE Signal Processing Magazine, Special Issue on Mobile Media Search Vol. 28, No. 4, 77-89 (2011).
34. Zampella, F., Bahillo, A., Prieto, J., Jimnez, A. R., and Seco, F. Pedestrian navigation fusing inertial and rss/tof measurements with adaptive movement/measurement models: Experimental evaluation and theoretical limits. Sensors and Actuators A: Physical 203, 0 (2013), 249 – 260.
12
ABSTRACT
We propose a graph-based, low-complexity sensor fusion approach for ubiquitous pedestrian indoor positioning using mobile devices. We employ our fusion technique to combine relative motion information based on step detection with WiFi signal strength measurements. The method is based on the well-known particle filter methodology. In contrast to previous work, we provide a probabilistic model for location estimation that is formulated directly on a fully discretized, graph-based representation of the indoor environment. We generate this graph by adaptive quantization of the indoor space, removing irrelevant degrees of freedom from the estimation problem. We evaluate the proposed method in two realistic indoor environments using real data collected from smartphones. In total, our dataset spans about 20 kilometers in distance walked and includes 13 users and four different mobile device types. Our results demonstrate that the filter requires an order of magnitude less particles than state-of-theart approaches while maintaining an accuracy of a few meters. The proposed low-complexity solution not only enables indoor positioning on less powerful mobile devices, but also saves much-needed resources for location-based end-user applications which run on top of a localization service.
Recent developments in the field of mobile indoor localization generated a variety of techniques building on various types of signals. These include approaches based on step detection [8], WiFi signal strength [6], magnetic field anomalies [12], and visual appearance [17]. While no single system achieves sufficient reliability, in combination, they can complement each other and compensate their respective weaknesses [11]. In order to leverage this potential, effective sensor fusion techniques yield the key to a ubiquitous solution for indoor positioning [14]. Over the past decade, robust sensor fusion for localization and tracking has seen a lot of success in the robotics community. Especially Bayesian techniques, most prominently the particle filter, provide a powerful probabilistic methodology for fusing data from multiple sensors. Substantial research effort led to well-developed solutions which achieve high accuracy and robustness. These are critical requirements for autonomous robots to operate in uncontrolled environments. Inspired by these results, similar implementations of particle filters have been employed for pedestrian positioning [15].
Author Keywords
Indoor Navigation; Indoor Positioning; Graph-based Sensor Fusion; Ubiquitous Localization; Particle Filter; Location-based Services; Mobile Computing ACM Classification Keywords
On mobile devices, however, low computational complexity and low power consumption are more important qualities than sub-meter accuracy [11]. While robotic agents require high precision for avoiding obstacles or for manipulating objects of interest, people, on the other hand, are usually able to easily perform both of these tasks without the assistance of their mobile phone. Hence, sub-meter location information may be valuable, for instance, in augmented reality applications, but is less important for indoor navigation. Moreover, algorithms that are optimized for highest accuracy consume more resources and hence create limitations for users with less powerful devices.
G.3.8. Mathematics of Computing: Probability and statistics – Probabilistic algorithms (including Monte Carlo) INTRODUCTION
To enable ubiquitous location-based services, a reliable positioning and tracking technology for mobile devices is required that works outdoors as well as indoors [20]. While in outdoor areas navigation satellite systems such as GPS already provide reliable positioning, a corresponding solution Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. UbiComp’14, September 13–17 2014, Seattle, WA, USA. Copyright 2014 ACM 978-1-4503-2968-2/14/09...$15.00. http://dx.doi.org/10.1145/2632048.2636079
Based on these observations, our key contribution is a graphbased sensor fusion approach for processing measurements from multiple sources of sensor information as well as map knowledge. Our method is based on the well-known particle filter methodology, but we propose to formulate the en-
1
or on generalizing training data into signal strength models. While the above RADAR-system [1] employs simple nearest neighbor matching, Haeberlen et al. [10] propose a probabilistic scheme using Markov Localization for which they cluster reference fingerprints for individual rooms and fit a Gaussian model for each room. Youssef et al. [33] adapt this model to account for sample-correlation when aggregating subsequent WiFi measurements. Ferris et al. [6, 5] propose to model signal strength across an entire building using Gaussian processes which allows to extrapolate to areas with no reference data. In contrast to that, Letchner et al. [21] and Biswas et al. [2] argue that a continuous, building-wide model is too complex and unnecessary given the uncertainty of WiFi-based localization. Hence, they opt for an even sparser representation by mapping fingerprints to a graphbased reference database.
tire probabilistic model directly on a completely discretized, graph-based representation of an indoor environment. As a result, the algorithm achieves significantly lower complexity than existing implementations (one to two orders of magnitude compared to recent works in [22, 32]), while maintaining an accuracy of a few meters. To achieve this reduction in complexity, we employ a coarse quantization of the indoor space. However, instead of simply performing uniform quantization, we pursue an adaptive approach. This is possible since the inner layout of buildings defines a certain topology regarding how people tend to move through these spaces. Narrow corridors, for instance, are usually traversed along their length, whereas lateral motion is often of little relevance for the localization task. The purpose of our graph structure is therefore to remove irrelevant degrees of freedom from the map. This way, narrow spaces become one-dimensional, whereas open areas remain two-dimensional. Furthermore, by formulating our probabilistic model of the particle filter directly on this graph, the algorithm can fully benefit from the decreased complexity and focus on the actual tracking task.
A complementary approach to WiFi localization is pedestrian dead reckoning (PDR) using the inertial sensors available on modern smartphones, in particular, accelerometer, gyroscope, and magnetometer (compass). Due to the low accuracy of built-in smartphone sensors, traditional inertial navigation approaches, which perform integration of measurements to produce position estimates, cannot be used as they result in cubic error growth [3, 11, 30]. Hence, state-of-the-art solutions rely on more distinct features in the signal and employ step detection. Displacement (step length) and direction of motion are then estimated for individual steps. To this end, recent research relies on machine learning techniques and activity recognition has been extended from distinguishing not only between the user moving and standing still, but to further include estimating the walking speed, climbing on stairs, taking an elevator, and more [24, 27]. Gusenbauer et al. [9], for instance, use detected activities to constrain the position estimate to known locations of stairs and elevators.
To demonstrate and evaluate our graph-based sensor fusion engine, we combine it with a typical choice of sensors to form a complete indoor positioning and tracking system. We use the compass, an acceleration-based pedometer, and WiFi, although, other sensor inputs are equally possible. We evaluate our proposed approach in two experimental setups to demonstrate its robustness and accuracy in realistic scenarios. All in all, we collected a dataset that spans about 20 kilometers in distance walked with a total duration of four hours. The evaluation includes 13 users and four different mobile device types. In addition, we provide empirical justifications for our theoretical derivations and parameter selection.
While the near-ubiquity of WiFi access points enables absolute positioning in many public buildings, pedestrian dead reckoning adds relative motion information, which together allows seamless pedestrian tracking. As a result, most modern indoor localization approaches for smartphones rely on this combination of sensors [11]. Yet beyond WiFi and PDR, various other approaches have been explored: Haverinen and Kemppainen [12] introduce indoor positioning based on anomalies in the earth magnetic field caused by local building structures. For domestic applications, Patel et al. [25] propose to induce tones into the powerline. Both approaches use fingerprinting to determine the location of the user. Following recent advances in efficient content-based image retrieval, Schroth et al. [28] describe a camera-based indoor localization system that retrieves images (and their locations) that visually match the user’s current environment.
RELATED WORK
In the following, we provide an overview of the state of the art in pedestrian indoor positioning and tracking. As our work is mainly concerned with sensor fusion and, in particular, lowcomplexity solutions for mobile devices, we first briefly review different sensing modalities that are commonly used for indoor localization, before we discuss in more detail existing sensor fusion techniques and current challenges regarding computational complexity and power consumption. Sensor Modalities for Indoor Positioning
The arguably most prominent technique for smartphonebased indoor localization today is WiFi-fingerprinting introduced by Bahl and Padmanabhan [1]. A location is represented by a WiFi-fingerprint which lists visible access points and their respective received signal strength identifiers (RSSI). Positioning is performed by matching the WiFifingerprint that is measured on the mobile device to a database of reference fingerprints which need to be collected beforehand during a calibration phase. The location associated with the closest match is returned as position estimate.
Sensor Fusion for Localization and Tracking
So far, numerous techniques for indoor positioning on smartphones have been developed, yet there is not a single solution that alone can guarantee reliable service for long-term operation throughout a large number of buildings. Moreover, most techniques exhibit their strengths and weaknesses under different conditions so that, in combination, they can com-
Over the past decade, most research effort focused either on improving the matching of measurements to reference data 2
Quigley et al. [26] who rely on WiFi and visual information. Li et al. [22] set the state of the art for pedestrian tracking on hand-held smartphones using only built-in inertial sensors and map knowledge, whereas combinations of WiFi and pedestrian dead reckoning are described in [18, 27, 34].
plement each other and improve not only accuracy but also reliability of service. Consequently, effective fusion of multiple sensing modalities is considered the key to a reliable and ubiquitous indoor positioning service [11, 14]. As a result of their success in fields like robotics [31], Bayesian techniques are widely used for positioning tasks [7]. Particle filters belong to the class of nonparametric Bayes filters and are not restricted to either linear systems or unimodal distributions. In addition, they provide a direct way to take map knowledge into account during tracking. For this reason, they are considered state of the art in sensor fusion for localization and tracking [15].
While the used sensors vary, all these methods have in common that the particle filter operates on a two-dimensional and continuous state-space (i.e., the map). As a consequence, regardless whether the map is represented as a conventional occupancy grid map as used by Quigley et al. [26], a CADfloorplan as used by Li et al. [22], or a polygon-based map as used by Woodman and Harle [32], the degrees of freedom for the estimation problem are equivalent in all cases. The actual difficulty of the tracking task varies of course with the accuracy of the employed sensors, but for smartphone-based solutions, typically, a few thousand particles are required to represent a multimodal belief distribution [15, 21, 22].
The idea behind the particle filter is to represent the state posterior by a finite set of samples (particles). We briefly restate the basic algorithm in order to clarify notation and explain its complexity. The following three steps are performed in each iteration [31]:
Although modern high-end smartphones might just be able to cope with hundreds or thousands of particles from a computational point of view, this certainly limits battery life time and, in consequence, end-user applications which run on top of the positioning service. Examples are navigation, medical monitoring, or smart-home management [13]. Hence, the problem of complexity reduction remains an important open challenge today [11].
1. Predict each particle’s new state by sampling from the state transition probability distribution given its current state: xt ∼ p (xt |xt−1 , ut , M)
(1)
where xt and xt−1 are the respective particle state at times t and t − 1. A possible control input is denoted by ut . M stands for the known map of the environment. 2. For any measurements, update each particle’s importance weight according to the measurement likelihood function given its new state: wt = wt−1 · L(zt |xt , M)
The most effective way to reduce the number of particles is to take degrees of freedom out of the probabilistic estimation problem [19]. To this end, Shin et al. [29] propose to constrain the motion model for a pedestrian to be more likely to produce straight walking paths and ignore small heading deviations. Liao et al. [23], on the other hand, describe a sensor network for tracking people through a building (i.e., an external tracking infrastructure). They were among the first to propose to constrain the particles to move only along the building’s skeleton, defining a one-dimensional—yet still continuous—state space. The idea has also been adopted to WiFi-based pedestrian tracking for smartphones [4, 6, 21].
(2)
where wt and wt−1 are the respective importance weights at the current and previous timestep, and zt denotes the current measurement. 3. Resample the particle population (with replacement) according to the distribution of their importance weights: draw xt ∝ wt
(3)
Contributions
Note that these equations do not specify the actual mathematical models of the required probability distributions. In fact, they have to be specifically defined for any particular application. Since Equations 1 and 2 have to be evaluated for each particle, the complexity of the algorithm is O(N ) where N is the number of particles [31]. Hence, the critical factor regarding computational complexity is the number of particles necessary for reliable operation of the filter. Due to the limited computational and battery resources on a mobile device, it is our goal to formulate the sensor fusion problem so as to require as few particles (and hence resources) as possible.
We go beyond previous work and replace the continuous state space by a completely discretized, graph-based model. Moreover, while the skeleton in [23] gives a good approximation of narrow corridors, we found that it is not sufficient to represent large, open spaces. Our intuitive approach to adaptively construct the graph with respect to the available space in the local environment is hence more general. Furthermore, when switching from a continuous state-space to a discrete one, existing sampling-models cannot be applied directly. Therefore, we provide a completely discretized probabilistic model for pedestrian motion that is directly formulated on our graph-structure (instead of using a conventional formulation and projecting the result onto a graph). Only then, the filter is able to fully exploit the sparse formulation of the localization problem and thus minimize its required resources.
Given the large body of research in this field, there exist of course standard implementations of particle filters for localization: Hightower and Borriello [15] are among the first to demonstrate the strong benefits of particle-filter-based sensor fusion for pedestrian localization by combining ultrasound and infrared beacon signals. Remarkable accuracy is achieved by Woodman and Harle [32] using a combination of foot-mounted inertial sensors and WiFi positioning, and
Our graph-based sensor fusion is intended as a replacement for current particle filter formulations. While we demonstrate 3
its performance using pedometer and WiFi measurements, our sampling-model can, in fact, incorporate measurements from any existing state-of-the-art pedometer algorithm such as [22]. Likewise, additional sensors can be included using existing probabilistic measurement models, e.g., as described in [21] for WiFi or in [26] for visual localization. SYSTEM OVERVIEW
Before we go into more detail on the sensor fusion, we provide a short overview of the components used in this work to generate the necessary measurements. Figure 1. Graph extraction from a floorplan. Left: A two-dimensional floorplan and its skeleton; Right: Final graph after sampling the skeleton (one-dimensionally) and enclosed free space (two-dimensionally).
Our pedometer implementation follows the one by Gusenbauer et al. [9] and consists of two modules, one for activity recognition and one for step detection. Our activity recognition, however, distinguishes only walking from standing still. If the user is believed to be walking, the second module detects steps based on peak detection. In order to translate detected steps into forward motion, a linear step-frequencybased model is used: dt = a · ft + b,
one end to the other, whereas motion in the perpendicular direction is less likely, and also less important for localization. Further, in contrast to robotic applications, sub-meter accuracy is not required for pedestrian navigation systems. Hence, the purpose of our graph structure is to remove degrees of freedom from the map wherever they are irrelevant to the localization task. This way, narrow spaces become onedimensional, while open areas remain two-dimensional.
(4)
where ft denotes the current step frequency and dt the estimated displacement per step. As the parameters, a and b, of this model are heavily user-dependent, we adapt the model to the user’s individual walking patterns over time employing the method described by Li et al. [22]. For estimating the step direction, we rely on orientation measurements provided by the vendor of each smartphone model.
Generating the graph structure for our approach requires only a floorplan, and can be done in a few simple steps as shown in Figure 1: 1. We extract the one-dimensional skeleton from the floorplan. This is equivalent to its Voronoi Diagram.
The pedometer is complemented by measurements of WiFi signal strength, employing a standard fingerprinting approach as described by Biswas et al. [2]. In order to employ our graph-based data fusion approach, we further require a twodimensional floorplan for each floor-level of the indoor environment. While different methods can be used to collect floorplan and registered WiFi data, we rely for our experiments on the mapping platform presented in [16]. This way, we create occupancy grid maps of several campus buildings of our university and collect the necessary reference WiFi signal strength measurements. As explained in the next section, we subsequently employ an adaptive quantization approach in order to extract our graph structure from the floorplan.
2. We find larger open spaces (e.g., a lobby) in the map and add these two-dimensional areas to the Voronoi-Diagram. The result is a combination of one-dimensional traces along narrow parts of the building and two-dimensional regions within larger rooms. 3. To create the final graph, we sample nodes in regular intervals from this structure. The edge length (i.e., the sampling interval) is ideally chosen to be roughly equal to the average pedestrian step length. This way, the lowest number of particles can be used to accurately represent pedestrian motion, as explained in more detail in the section on parameter selection.
GRAPH-BASED SENSOR FUSION
In this section, we describe in detail our graph-based sensor fusion technique. As mentioned above, our positioning system is provided with the following sources of information:
State Space
• Relative displacement estimates from the pedometer
In contrast to standard particle filter models which typically operate on a continuous two-dimensional map, our state space is defined by the fully discrete graph structure which replaces the conventional map. Hence, in our approach, the commonly used two position coordinates are reduced to a single node identifier, and each particle maintains the following state vector:
• Absolute location estimates from WiFi-fingerprinting
• The current location nt in form of a node-ID on the graph
Graph Extraction
• The current orientation θt with respect to magnetic north
• Building layout (a floorplan) • Absolute orientation estimates from the compass
The inner structure of a building typically not only imposes hard constraints on where a pedestrian can walk (walls). It also outlines important priors where people are likely to walk. Narrow corridors, for instance, are usually traversed from
Pedometer Model
In the following, we define the state transition model which is used to predict the new particle state based on the current 4
the measured absolute orientation zθ,t (Equation 12).
state and incoming motion estimates from the pedometer (i.e., orientation and displacement). In order to propagate the filter only when necessary, we condition state updates on the arrival of pedometer measurements. This makes the filter run at step frequency, which lies one to two orders of magnitude below typical raw sensor update rates. This further reduces unnecessary resource consumption.
{nt , θt } ∼ p (nt , θt |nt−1 , θt−1 , zθ,t , zd,t , G) = p (nt |nt−1 , θt , θt−1 , zθ,t , zd,t , G) · p (θt |nt−1 , θt−1 , zθ,t , zd,t , G) = p (nt |nt−1 , θt , zd,t , G) · p (θt |zθ,t )
θt ∼ p (θt |zθ,t )
(5)
This sampling process can be implemented efficiently by using the following decomposition: 1. Sample the new orientation given the measured orientation:
p (nt |nt−1 , θt , zd,t , G) = X = p (nt , dt |nt−1 , θt , zd,t , G)
(6)
=
X
(7)
p (nt |nt−1 , θt , dt , zd,t , G) · p (dt |nt−1 , θt , zd,t , G) (16)
=
X
p (nt |nt−1 , θt , dt , G) · p (dt |zd,t )
(17)
dt
Hence, also the displacement can be sampled separately given its measurement (cf. Equation 7):
(8)
dt ∼ p (dt |zd,t ) 4. Sample the new node given the sampled edge and the particle’s displacement: nt ∼ p (nt |et , dt , G)
(15)
dt
3. Sample one of the current node’s outgoing edges given their respective directions relative to the particle’s orientation: et ∼ p (et |nt−1 , θt , G)
(14)
dt
2. Sample the actual displacement given the estimated displacement: dt ∼ p (dt |zd,t )
(13)
Since the displacement per step is an estimated quantity, we now introduce a random variable for the true displacement dt (Equation 15). As above, the resulting joint distribution of nt and dt is rewritten as the product of a conditional and a marginal probability. This enables us to model also the distribution of dt conditionally independent from all variables except the estimated displacement zd,t . For the distribution of nt , in turn, the estimate zd,t becomes irrelevant given the true displacement (Equation 17).
where the graph G now replaces the map M.
θt ∼ p (θt |zθ,t )
(11) (12)
This allows us to sample the particle orientation from its measurement in a separate step (cf. Equation 6):
Whenever a step is detected, the particles sample a new state {nt , θt } from the state transition probability distribution given the old state {nt−1 , θt−1 }, the measured displacement zd,t (step length), and the current orientation estimate zθ,t . Correspondingly, we rewrite Equation 1 for our approach as {nt , θt } ∼ p (nt , θt |nt−1 , θt−1 , zθ,t , zd,t , G)
(10)
(18)
Since the map is represented by a graph, motion takes the form of transitions along edges in the graph. Therefore, we now introduce a random variable et over edges (Equation 20). As before, the joint probability of nt and et becomes the product of a conditional and a marginal distribution (Equation 21):
(9)
Equation 9 essentially decides whether the current particle makes a transition along the sampled edge et , or stays at its current node, i.e., nt = nt−1 . If a transition is sampled, the particle moves to the node at the other end of edge et . The ratio between those particles that make a transition and particles that stand still should correspond to the ratio between the measured displacement and the edge length. This way, the expected value of the user’s location corresponds exactly to the sampled displacement. It is important to note that the above decomposition results in distributions which are simple to model and can be empirically learned from training data.
p (nt |nt−1 , θt , dt , G) = X = p (nt , et |nt−1 , θt , dt , G)
(19) (20)
et ∈G
=
X
p (nt |et , nt−1 , θt , dt , G) · p (ei |nt−1 , θt , dt , G)
et ∈G
(21) =
X
p (nt |et , dt , G) · p (et |nt−1 , θt , G)
(22)
et ∈G
Since for pedestrians forward motion is usually more likely than walking sideways or even backwards, the edge sampling distribution depends on the current node nt−1 (and thus its outgoing edges and their directions) and the particle’s orientation θt . Edges that are not connected to the current node nt−1 have zero probability to be sampled (cf. Equation 8).
Mathematical Derivation
In this section, we show how the above decomposition follows from Equation 5. The first step is to rewrite the joint probability of nt and θt as the product of a conditional and a marginal one (Equation 11). Given the true orientation θt , the resulting distribution of nt becomes conditionally independent from θt−1 and zθ,t . Likewise, the distribution of θt becomes conditionally independent from all variables except
Once an edge is sampled (thus fixing the direction of motion), the probability of a transition only depends on the edge length 5
Relative frequency
ket k and the sampled displacement dt (Equation 22). Hence, the transition sampling distribution decides whether a particle stays at node nt−1 or makes a transition to the node at the other end of edge et (cf. Equation 9). Pedometer-based Weight Update
As noted above, when sampling an edge, the model reflects typical motion preferences of pedestrians. An edge in a direction that corresponds well with the particle’s orientation is more likely to be sampled than an edge leaving, for example, in the perpendicular direction. The sampling process, however, can only take into account relative likelihoods of different edges. For instance, if only a single edge leaves a given node, all particles would sample this edge, no matter how well (or poorly) the direction of this edge is aligned with the particles’ individual orientation.
−5 0 5 RSSI difference from means [dBm]
10
To apply fingerprinting in our proposed graph-based approach, the reference fingerprints need to be mapped to graph-nodes. In contrast to [2, 26], we specifically want to avoid interpolation of measurements that correspond to different rooms (and hence are subject to different attenuation patterns). Hence, we use ray-casting on the floorplan to determine for each node in the graph all visible locations (i.e., in the same room) where reference fingerprints have been collected. With this subset of measurements, we perform distance-based interpolation to generate the fingerprint for the current node. This way, for each node, a reference fingerprint is interpolated using only measurements that have been recorded in line of sight.
(23)
The next step is to define the probabilistic measurement model: We assume conditional independence between signals of different WiFi access points. Although this is a rather strict assumption, it allows us to obtain a useful likelihood function of low computational complexity [2]. Similar to [26], we derive a likelihood function from Equation 2 as follows:
In our case, the likelihood function derives from the joint measurement probability which is conditioned on all sampled variables: (24)
Given edge, orientation, and displacement, the two measurements become conditionally independent (Equation 26). Both marginal distributions are further independent of the current node nt since they reflect relative motion information, regardless of location. Also, the measurement probability over displacements does not depend on orientation and, vice versa, the measurement probability over orientations does not depend on displacement (Equation 27): p(zd,t , zθ,t |nt , θt , et , dt , G) = = p(zd,t |nt , θt , et , dt , G) · p(zθ,t |nt , θt , et , dt , G) = p(zd,t |et , dt , G) · p(zθ,t |et , θt , G)
0 −10
drift. Whenever a WiFi scan is matched to the reference database, the filter processes the resulting information using a measurement model which updates the particles’ importance weights according to the observation likelihood. This model needs to reflect that, due to dynamic disturbances and gradual removal or addition of access points over time, WiFi-based positioning generally provides location information with substantial uncertainty, i.e., at room-accuracy on average.
The pedometer-based weight update can be derived as follows, starting with Equation 2:
L(zt |xt , M) ∼ p(zd,t , zθ,t |nt , θt , et , dt , G)
0.1 0.05
Figure 2. Distribution of stationary WiFi power measurements over time
The information that is lost due to proportional sampling can be included again by augmenting the state transition model with an importance weight update. This partitioning of information into a sampling and a weighting part comes with the valuable advantage that particles that do not correspond well with the map (i.e., the graph) are increasingly penalized over time. This, in turn, allows the filter to aggregate this information and to use resampling to replace hypotheses that have become unlikely by better ones.
wt = wt−1 · L(zt |xt , M)
0.15
wt = wt−1 · L(zt |xt , M)
(29)
For modelling the noise distribution in the WiFi signal, we choose a mixture of a Gaussian and a uniform distribution. We verified this assumption by performing a stationary observation of WiFi RSSI values over time. During the recording period of about six minutes, people were walking through the room, as can be expected in a realistic indoor environment. The resulting likelihood function has the following form: ! 2 X (zi − hi (xt )) L(zw,t |xt , G) ∼ exp − (30) σi2
(25) (26) (27)
i∈I
We model the likelihood according to the absolute sampling probability over edges given the particle’s orientation. The displacement likelihood, on the other hand, has no effect on the weighting as it contributes no information that is not already included in the sampling step:
where zi is the measured power of transmitter i, and hi (xt ) is the predicted power of transmitter i at location xt . I denotes the set of transmitters present in the current WiFi measurement, and σi is the standard deviation of the signal from transmitter i obtained from the training data (see Figure 2).
L(zt |xt , M) ⇒ L(zθ,t |et , θt , G) ∼ p (et |nt−1 , θt , G) (28)
PARAMETER SELECTION WiFi-based Weight Update
At this point, we have derived the complete mathematical model for our graph-based sensor fusion approach. The following part includes evaluations that are concerned with determining the pedometer error characteristics. From this, we
While the pedometer provides relative motion information, we use WiFi signal strength measurements to gain absolute position information in order to correct for the immanent 6
1.1
0.02 0 −60
−40 −20 0 20 40 Deviation of the heading from the true value [deg]
Average step length [m]
Relative frequency
0.04
60
Figure 3. Error distribution of orientation estimation including building parts that cause local compass biases.
Model Name Samsung Galaxy S3 LG Nexus 4 HTC One Galaxy Nexus
IMU Vendor STMicroelectronics 1 Qualcomm 2 STMicroelectronics 1 Invensense 3
σθ [deg] 9.2 4.9 10.0 9.4
Male 1 Male 2 Male 3 Male 4 Male 5 Female 1 Female 2
1 0.9 0.8 0.7 0.6 0.5 1
1.2
1.4 1.6 1.8 Average step frequency [Hz]
2
2.2
Figure 4. Relation between step frequency and step length for seven different users.
Table 1. Comparison of orientation algorithms provided by manufacturers of built-in inertial measurement unit (IMU) chips.
Height
Female 1.75 1.65
1.84
1.81
Male 1.72 1.87
1.80
Table 2. Body height of seven test users in meters.
derive the actual sampling distributions and train their parameters to implement the filter’s state transition model. Orientation Estimation
User Dependence of Step-length Estimation
We asked seven users to walk through a corridor of 106 m length, roughly following a straight path while holding the device in front of the body. Each run was repeated several times, accumulating close to ten kilometers of recorded data with a total duration of two hours. Figure 3 shows the distribution of deviations from true orientation, using the exact heading of the corridor as reference. It is important to note that this histogram includes not only the noise characteristics of the device’s compass, but also fluctuations due to the swinging motion of walking, as well as local biases due to earth magnetic field distortions throughout the building. While this results in rather large standard deviations σθ , this corresponds, in fact, to the complete error characteristic that the particle filter requires to be able to correctly process orientation measurements. Table 1 shows standard deviations for different device models. For orientation sampling (Equation 6), we approximate these results using a Gaussian distribution around the measured absolute orientation zθ,t : p (θt |zθ,t ) ∼ N zθ,t , σθ2 (31)
Having derived a general model for step length estimation (fitted to all users), we now compare how well it performs for individual users (again, using the above dataset). We distinguish two cases: One, where the user’s true height is known a priori (see Table 2), and one where the average value of 1.78 m is used instead. Figure 5 shows for each user the estimated displacement per step relative to the true displacement. Knowing the user’s true height only marginally reduces the overall standard deviation σd from 0.14 to 0.11. However, significant deviations remain regardless whether the height is known or not. Hence, we adapt our step length model during operation of the filter, following the approach described by Li et al. [22]. For displacement sampling (Equation 7), we approximate these results using a Gaussian distribution around the measured displacement zd,t :
where the standard deviation σd is learned from the deviations in Figure 5.
where the standard deviation σθ is learned from the histogram in Figure 3.
Step Detection
p (dt |zd,t ) ∼ N zd,t , σd2
(32)
Finally, we evaluate the performance of step detection alone, i.e., using ground truth labels for activity recognition. For this experiment, two users with two different devices recorded a dataset while exactly counting the number of true steps. Comparing this to the algorithm’s estimate, the resulting error rate of step detection turns out to be below 0.5%. As this is clearly overshadowed by errors in activity recognition and other modules, we can safely neglect this error source.
Step-length Estimation
In order to parametrize the linear model for step length estimation in Equation 4, we use the same dataset as above. For increased accuracy, the path is further segmented into small parts of known length. For each segment, we compute the average displacement per step, as well as the average step frequency from the duration of the respective run. Figure 4 shows the resulting diagram (step-length vs. step-frequency) for all seven users. We fit a linear model to the entire dataset using least squares to set the parameters for Equation 4.
Edge Sampling
When sampling an edge at a given node, i.e., deciding the direction of motion (Equation 8), we want the model to reflect expected motion patterns of a pedestrian. Hence, we employ for edge sampling a truncated Gaussian distribution over all
1
http://www.st.com/ 2 http://www.qualcomm.com/ 3 http://www.invensense.com/
7
Deviation from true displacement [%]
20
Unknown height Known height
100cm
100cm
130cm
10
80cm
0
Figure 6. State transition sampling (using ten particles for illustration). Left: An 80 cm step along a 1 m edge leads to transitions for eight of the ten particles, resulting in a weighted mean (expected value) at exactly 80 cm distance from the bottom node. Right: Iterative sampling for steps that are longer than a single edge.
−10 −20 Male 1
Male 2
Male 3
Male 4
Male 5 Female 1Female 2
Figure 5. Deviations of the estimated step length from the true displacement for different users. Bright bars: Average height is used (1.78 m); Dark bars: True body height is known.
can be achieved with our low-complexity approach, before assessing the system’s reliability. All in all, the evaluations include 13 users, who did not participate in the previous experiments for parameter selection, and four device types (running Android OS). While we implemented the full system as Android app to guarantee its functionality on a mobile operating system, for better analysis, we also record all sensor measurements during operation. For correct statistical analysis, each of the recorded runs is later evaluated over 100 Monte-Carlo-Simulations (i.e., different seeds for the random number generator).
edges that leave the current node nt−1 : N ∆θt , σe2 if |∆θ| ≤ 90◦ p (et |nt−1 , θt , G) ∼ (33) 0 otherwise where ∆θt = θ(et ) − θt is the angular difference between current particle orientation θt and the direction in which edge et leaves the current node nt−1 . The distribution is truncated beyond 90◦ to account for the extremely low likelihood that people walk backwards (in particular in a navigation scenario). Moreover, the standard deviation σe must also include quantization effects due to the graph-based nature of the filter. Hence, we set σe to 40 degrees.
Experiment 1: Accuracy vs. Complexity
Our first experiment has the goal to show that our graph-based sensor fusion technique is able to achieve comparable accuracy to state-of-the-art approaches in a realistic, complex indoor environment, but using no more than 50 particles.
Transition Sampling
The experiment is conducted on two floors of an office building covering together 5000 m2 and including numerous corridors, office rooms, and staircases. Six users were asked to walk through the building, holding their smartphone in hand. Their path is shown in Figure 7 and leads through corridors, various office rooms, and two staircases where they change the floor level (once up, once down). A single walk is over 300 m long and takes about five minutes per user.
After an edge is sampled which fixes the direction of motion, the decision whether the edge is traversed or not is modelled using the ratio between particle displacement and edge length (Equation 9): if dt ≤ 0 0 p (nt |et , dt , G) ∼ kedtt k if 0 < dt < ket k (34) 1 if dt ≥ ket k
To obtain ground truth, we follow the methods described in [21, 22, 34] and placed markers on the floor every two meters for which we carefully hand-labelled their true coordinates. When a user passes a marker, the current timestamp is recorded by the experiment supervisor (using an auxiliary app) so that the position error can be determined by comparing the particle filter belief at that time with the known coordinates of the marker. For both, initial localization and tracking, the particle filter is set to use exactly 50 particles.
If a displacement dt happens to be larger than the current edge length ket k, the particle makes a certain transition along the current edge, and the pedometer model is run again but using only the remainder of the displacement d0t = dt − ket k. As a result, for any given edge, the expected value of the user’s location will correspond exactly to the sampled displacement, as illustrated in Figure 6. This observation also allows us to find a good choice for the edge length in our graph: Since we want to employ as few particles as possible, we have to set the edge length roughly close to the typical displacement per step (we set it to 1 m). Otherwise, more particles will be required to correctly represent the expected value of the user’s location.
Since accuracies are in general highly dependent on the experimental setup, we run for comparison all our collected data also through a state-of-the-art particle filter implementation, which operates on a continuous state-space as used, e.g., by Li et al. [22] or Woodman and Harle [32]. Figure 8 shows the cumulative error distribution for both versions. The results indicate that our solution is able to track the user with an accuracy of 1.52 m 50% of the time, and 4.53 m 90% of the time. We found that, in order to achieve comparable accuracy, in our experimental setup, a continuous particle filter requires at least 500 particles (which gives an accuracy of 1.38 m 50%
EVALUATION
Based on the filter parameterization derived in the previous section, we provide in the following an analysis of the localization system’s overall performance. Using two rather different experimental setups, we first evaluate the accuracy that 8
Figure 7. Walking path for experiment 1, leading the user about 300 m through several corridors, office rooms, and staircases on two floor levels of a multi-story office building. Our graph for this building is depicted in blue. The building is also shown in Figure 9 where it is located in the upper center. Empirical CDF
1
corridors with a total accessible area of 6000 m2 . Figure 9 shows the corresponding map and path. We asked seven users, who did not participate in the previous experiment, to walk through the entire series of buildings. They were free to turn around anytime, look at office signs or posters on the wall, etc. The recorded data cover more than six kilometers in distance walked. Ground truth is again realized using waypoints as described above, but in this case, leaving more space between markers (on average 30 m) as we are mainly interested in reliability. As above, the particle filter is set to use 50 particles throughout the entire experiment.
0.8
F(x)
0.6 0.4 0.2 0 0
graph-based, 50 particles continuous, 500 particles continuous, 50 particles 2
4
6 8 Positioning error [m ]
10
12
Regarding robustness, we find that 98% of the 700 runs (100 runs per user) are successfully tracked through the entire experiment, while for 2% of the runs, the filter looses track of the user at some point. However, when using WiFi fingerprints not only for likelihood updates but also for sampling a small fraction (we take 10%) of the particles from each incoming measurement, all 700 trials are tracked successfully. Figure 10 shows the cumulative error distribution for this case, where our approach is able to track the user with an accuracy of 2.19 m 50% of the time, and 7.16 m 90% of the time. To give an indication of the availability of WiFi in this experiment, Figure 10 also includes the cumulative error distribution when tracking whithout pedometer, relying on WiFi alone. This gives a median accuracy of 5.90 m.
14
Figure 8. Cumulative error distributions for experiment 1.
of the time, and 6.00 m 90% of the time). Using only 50 particles, however, the continuous particle filter is often unable to follow into the correct room, or fails to track the user entirely, hence deteriorating its accuracy. We further compared the time that is on average required to complete a single update step for a single particle in our graph-based filter as well as in the reference implementation, and found both to be roughly equivalent. Of course, the continuous filter version needs to perform this step ten times more often. Moreover, we observe that the continuous filter needs to apply resampling five times more often than our solution, resulting in additional overhead.
Finally, we observe that taking turns helps the particle filter to constrain its estimate. While this experiment contains many long stretches of straight corridors, experiment one benefits from a more complex path through corridors as well as office rooms, resulting in higher accuracy on average.
Experiment 2: Robustness DISCUSSION
Our second experiment aims at evaluating the robustness of our technique with only 50 particles when used for longer periods of time. The setup for this experiment spans eight connected buildings at our university campus including mostly
The results show that our proposed graph-based sensor fusion enables reliable tracking using no more than 50 particles. This represents a significant reduction of complexity 9
Empirical CDF
1 0.8
F(x)
0.6 0.4 0.2
Graph-based WiFi only
0 0
5 10 Positioning error [m]
15
Figure 10. Cumulative error distributions for experiment 2.
devices, but also saves well-needed resources for higher-level applications, which create actual value for the end-user. Figure 9. The 800 m walking path for experiment 2 leads through a series of eight connected buildings with a total accessible area of 6000 m2 .
CONCLUSION
We present a graph-based particle filter formulation for multisensor-fusion that enables pedestrian indoor positioning on mobile devices due to its low complexity. Our results demonstrate a reduction of at least one order of magnitude in computational complexity compared to state-of-the-art solutions while ensuring robust localization.
compared to the state of the art while maintaining comparable accuracy: Li et al. [22], for instance, rely on two orders of magnitude more particles (3000) as do Letchner et al. [21] who employ 1000 particles. Woodman and Harle [32] benefit from more accurate foot-mounted sensors but, nonetheless, require ten times more particles than our solution.
This is achieved by a coarse, adaptive quantization of the indoor space, which takes into account typical motion patterns of pedestrians in buildings. Hence, instead of employing a continuous state space in the data fusion, the indoor environment is represented by a sparse graph structure. By formulating our probabilistic model of the particle filter directly on this graph, the algorithm can fully benefit from the decreased complexity and focus on the actual tracking task.
On the other hand, there are limitations as well: The algorithm’s discrete nature introduces additional quantization errors compared to conventional map representations. In this paper, we parametrize our probabilistic sensor models such that they account for both measurement and quantization errors. This is possible since, for hand-held smartphones, the pedometer measurement error is by far larger than any quantization effects. However, this may not hold for significantly more accurate sensors such as foot-mounted IMUs or robotic odometry, where our solution might limit the achievable accuracy. Hence, our approach to sensor fusion clearly is most beneficial for pedestrian localization using hand-held devices.
Overall, the proposed approach is able to reliably track a pedestrian through an indoor environment using only 50 particles to process pedometer measurements in combination with WiFi signal strength measurements. In addition, our system is able to adapt to characteristic gait patterns for individual users. Our evaluation in a realistic indoor environment demonstrates the robustness of our approach for various devices and users.
Another limitation is the inherent assumption that the indoor environment features sufficient internal structure. Large empty halls (an extreme example would be an aircraft hangar) do not impose constraints on the filter and hence no notable complexity reduction can be expected. We observe that the approach works best for typical public buildings such as airports, museums, and universities, or likewise for office buildings (with numerous corridors and rooms).
We are currently in the process of extending our graph-based probabilistic model to also include the vision-based localization technique described in [28]. As this will incorporate an additional and very precise (meter-accurate) source of absolute position information, we expect a further boost in localization accuracy. ACKNOWLEDGMENTS
We propose our technique as a drop-in solution for multisensor positioning systems that already use some form of data fusion, especially, in the case of particle filters. Besides the sensor inputs used in this paper, the approach can be combined with any other source of information, and we encourage to extend the system presented here. Due to its low consumption of computational and power resources, the technique not only enables indoor positioning for a large range of mobile
This research project has been supported by the space agency of the German Aerospace Center with funds from the Federal Ministry of Economics and Technology on the basis of a resolution of the German Bundestag under the reference 50NA1307. The authors would also like to thank the reviewers for their constructive comments, and Maryam Banitalebi for her work and assistance in the field of activity recognition. 10
14. Hightower, J., and Borriello, G. A survey and taxonomy of location systems for ubiquitous computing. IEEE computer 34, 8 (2001), 57–66.
REFERENCES
1. Bahl, P., and Padmanabhan, V. RADAR: an in-building RF-based user location and tracking system. In INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, vol. 2 (March 2000), 775–784 vol.2.
15. Hightower, J., and Borriello, G. Particle filters for location estimation in ubiquitous computing: A case study. In UbiComp 2004: Ubiquitous Computing, N. Davies, E. Mynatt, and I. Siio, Eds., vol. 3205 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2004, 88–106.
2. Biswas, J., and Veloso, M. Wifi localization and navigation for autonomous indoor mobile robots. In IEEE International Conference on Robotics and Automation (ICRA) (May 2010), 4379–4384.
16. Huitl, R., Schroth, G., Hilsenbeck, S., Schweiger, F., and Steinbach, E. TUMindoor: an extensive image and point cloud dataset for visual indoor localization and mapping. In IEEE International Conference on Image Processing (ICIP 2012) (Orlando, FL, USA, Sep 2012).
3. Brajdic, A., and Harle, R. Walk detection and step counting on unconstrained smartphones. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’13, ACM (New York, NY, USA, 2013), 225–234.
17. Huitl, R., Schroth, G., Hilsenbeck, S., Schweiger, F., and Steinbach, E. Virtual reference view generation for CBIR-based visual pose estimation. In ACM Multimedia 2012 (Nara, Japan, Nov 2012).
4. Evennou, F., Marx, F., and Novakov, E. Map-aided indoor mobile positioning system using particle filter. In Wireless Communications and Networking Conference, 2005 IEEE, vol. 4 (March 2005), 2490–2494 Vol. 4.
18. Kothari, N., Kannan, B., Glasgow, E. D., and Dias, M. B. Robust indoor localization on a commercial smart phone. In Procedia Computer Science, M. Y. Elhadi Shakshukia, Ed., vol. 10, ELSEVIER (August 2012), 1114–1120.
5. Ferris, B., Fox, D., and Lawrence, N. Wifi-slam using gaussian process latent variable models. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) 2007 (2007), 2480–2485. 6. Ferris, B., H¨ahnel, D., and Fox, D. Gaussian processes for signal strength-based location estimation. In Proc. of Robotics Science and Systems (RSS) (2006).
19. Krumm, J., Williams, L., and Smith, G. Smartmovex on a graph - an inexpensive active badge tracker. In UbiComp 2002: Ubiquitous Computing, G. Borriello and L. Holmquist, Eds., vol. 2498 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2002, 299–307.
7. Fox, D., Hightower, J., Liao, L., Schulz, D., and Borriello, G. Bayesian filtering for location estimation. IEEE Pervasive Computing 2, 3 (2003), 24–33. 8. Foxlin, E. Pedestrian tracking with shoe-mounted inertial sensors. Computer Graphics and Applications, IEEE 25, 6 (Nov. 2005), 38–46.
20. LaMarca, A., Chawathe, Y., Consolvo, S., Hightower, J., Smith, I., Scott, J., Sohn, T., Howard, J., Hughes, J., Potter, F., Tabert, J., Powledge, P., Borriello, G., and Schilit, B. Place lab: Device positioning using radio beacons in the wild. In Pervasive Computing, H.-W. Gellersen, R. Want, and A. Schmidt, Eds., vol. 3468 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2005, 116–133.
9. Gusenbauer, D., Isert, C., and Kr¨osche, J. Self-contained indoor positioning on off-the-shelf mobile devices. In International Conference on Indoor Positioning and Indoor Navigation (IPIN) 2010 (Sept 2010), 1–9. 10. Haeberlen, A., Flannery, E., Ladd, A. M., Rudys, A., Wallach, D. S., and Kavraki, L. E. Practical robust localization over large-scale 802.11 wireless networks. In Proceedings of the 10th Annual International Conference on Mobile Computing and Networking, MobiCom ’04, ACM (New York, NY, USA, 2004), 70–84.
21. Letchner, J., Fox, D., and LaMarca, A. Large-scale localization from wireless signal strength. In Proceedings of the national conference on artificial intelligence, vol. 20, Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999 (2005), 15. 22. Li, F., Zhao, C., Ding, G., Gong, J., Liu, C., and Zhao, F. A reliable and accurate indoor localization method using phone inertial sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, UbiComp ’12, ACM (New York, NY, USA, 2012), 421–430.
11. Harle, R. A survey of indoor inertial positioning systems for pedestrians. Communications Surveys Tutorials, IEEE 15, 3 (Third 2013), 1281–1293. 12. Haverinen, J., and Kemppainen, A. Global indoor self-localization based on the ambient magnetic field. Robotics and Autonomous Systems 57, 10 (Oct. 2009), 1028–1035.
23. Liao, L., Fox, D., Hightower, J., Kautz, H., and Schulz, D. Voronoi tracking: location estimation using sparse and noisy sensor data. In Proceedings 2003 IEEE/RSJ of International Conference on Intelligent Robots and Systems (IROS 2003)., vol. 1 (Oct 2003), 723–728.
13. Hazas, M., Scott, J., and Krumm, J. Location-aware computing comes of age. Computer 37, 2 (Feb 2004), 95–97.
11
24. Park, J.-G., Patel, A., Curtis, D., Teller, S., and Ledlie, J. Online pose classification and walking speed estimation using handheld devices. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, UbiComp ’12, ACM (New York, NY, USA, 2012), 113–122.
29. Shin, H., Chon, Y., and Cha, H. Unsupervised construction of an indoor floor plan using a smartphone. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 42, 6 (Nov 2012), 889–898.
25. Patel, S. N., Truong, K. N., and Abowd, G. D. Powerline positioning: A practical sub-room-level indoor location system for domestic use. In UbiComp 2006: Ubiquitous Computing, P. Dourish and A. Friday, Eds., vol. 4206 of Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2006, 441–458.
30. Steinhoff, U., and Schiele, B. Dead reckoning from the pocket - an experimental study. In Pervasive Computing and Communications (PerCom), 2010 IEEE International Conference on (March 2010), 162–170. 31. Thrun, S., Burgard, W., and Fox, D. Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, 2005.
26. Quigley, M., Stavens, D., Coates, A., and Thrun, S. Sub-meter indoor localization in unmodified environments with inexpensive sensors. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). (Oct 2010), 2039–2046.
32. Woodman, O., and Harle, R. Pedestrian localisation for indoor environments. In Proceedings of the 10th International Conference on Ubiquitous Computing, UbiComp ’08, ACM (New York, NY, USA, 2008), 114–123.
27. Radu, V., and Marina, M. K. Himloc: Indoor smartphone localization via activity aware pedestrian dead reckoning with selective crowdsourced wifi fingerprinting. In Proceedings of the 4th International Conference on Indoor Positioning and Indoor Navigation (IPIN)., IEEE (2013).
33. Youssef, M., and Agrawala, A. The horus wlan location determination system. In Proceedings of the 3rd International Conference on Mobile Systems, Applications, and Services, MobiSys ’05, ACM (New York, NY, USA, 2005), 205–218.
28. Schroth, G., Huitl, R., Chen, D., Abu-Alqumsan, M., Al-Nuaimi, A., and Steinbach, E. Mobile visual location recognition. IEEE Signal Processing Magazine, Special Issue on Mobile Media Search Vol. 28, No. 4, 77-89 (2011).
34. Zampella, F., Bahillo, A., Prieto, J., Jimnez, A. R., and Seco, F. Pedestrian navigation fusing inertial and rss/tof measurements with adaptive movement/measurement models: Experimental evaluation and theoretical limits. Sensors and Actuators A: Physical 203, 0 (2013), 249 – 260.
12
