Transcript SurroundSense: Mobile Phone Localization via Ambience

SurroundSense: Mobile Phone
Localization via Ambience
Fingerprinting
MARTIN AZIZYAN, IONUT CONSTANDACHE, ROMIT
ROY CHOUDHURY
Presented by Lingfei Wu
Outline






Introduction
SurroundSense Architecture
System Design
Implementation
Evaluation
Conclusion
Introduction
Notion of location

Application of logical
localization



Physical coordinates(latitude/longitude)
Logical labels(like Starbucks, Mcdonalds)
Many applications based on logical location
Introduction



Physical coordinate can be reversed to logical location.
However, it often causes error !
Why not compute logical location directly?
Relative work

Active RF



Passive RF


Install special hardware
Ultrasound, Bluetooth
GPS, GSM or WIFI based
Behavior Sensing

Imaging matching
1. Lack accuracy
2. Need pre-installed infrastructure
Motivation

Combine effect of ambient sound, light, color, user
motion

Sound (microphone)


Light / Color (camera)


Wal-Mart VS McDonald
Place may not be unique based on any one attribute


Different thematic light, colors and floors.
Human movement (accelerometer)


Starbucks VS Bookstore
The combination can be unique enough for localization
In this paper, we propose SurroundSense for logical
Starbucks
Bookstore
localization.
Wal-Mart
McDonald’s
SurroundSense Architecture
Candidate list
1.Xxx
2.Yyy
3.zzz
1.Xxx
2.Yyy
1.Xxx
2.Yyy
1.Xxx
2.Yyy
1.Xxx
System Design

Fingerprint generation





Fingerprinting sound
Fingerprinting motion using accelerometers
Fingerprinting color/light using cameras
Fingerprinting Wi-Fi
Fingerprint matching




Wi-Fi filter
Sound filter
Motion filter
Color/light Match
Fingerprinting sound
Normalized
occurrence count
Convert signals to time domain
amplitude value

50
0
-50
time


time
Normalized
amplitude value
100 normalized values as feature of sound
Similarity of fingerprints

Compute 100 pair-wise distance between test fingerprint
and all candidate fingerprint
Dim
1
2
3
…
…
100
A
0.1
0.2
0.1
…
…
0.05
B
0.6
0.3
0.2
…
…
0.1
Fingerprinting Sound

Unreliable to be a matching
scheme



Sound from the same place
can vary over time.
Only use as a filter
If distance > threshold τ
then discard from the
candidate list
Fingerprinting Motion

Use support vector machine(SVM) as classifier
Raw data
SVM


1
moving
-1
stationary
Sequence of states as user’s moving pattern
Movement is prone to fluctuation


In a clothing store, Some users browse for a long time
while others purchase clothes in haste.
Only use as a filter
Fingerprinting Motion

Compute motion fingerprint: Ratio = tmoving / tstatic



Bucket 1: 0.0 <= Ratio <= 0.2 Sitting (cafe)
Bucket 2: 0.2 <= Ratio <= 2.0 Browsing (clothing)
Bucket 3: 2.0 <= Ratio <= ∞ Walking (grocery)
Sitting
Browsing
Walking
Fingerprinting Color / Light


Thematic color and lighting in different stores
Where to capture the picture?



random picture of surrounding
floor
Advantages of taking floor pictures




Privacy concern
Less noisy
Rich diversity in floor color
Easy to obtain
too much noise
Fingerprinting Color / Light

How to extract colors and light intensity?



RGB
too much noise
HSL(Hue-Saturation-Lightness)
Find color cluster and its size using K-means
clustering algorithm
k=2
k-mean
clustering
k++
no
sk-sk-1 < t
yes
Bean Trader’s Coffee shop
< c1, c2 …, ck >
sk: the sum of
distance from
all pixels to their
(own cluster’s)
centroid.
t: convergence
threshold
Fingerprinting Color / Light

Similarity of fingerprints

Assume C1 = {c11, c12, …, c1n}; C2 = {c21, c22, …, c2m}
distance of centroid

Total size in C1 or C2
Fingerprint matching

The candidate list with maximum similarity is
declared to the matching fingerprint
Fingerprinting Wi-Fi

Wi-Fi fingerprint

Record MAC address from APs every 5 second
Fingerprint tuple:
<{AP1_MAC_Addr, AP1_fraction_time},
{AP2_MAC_Addr, AP2_fraction_time},
{AP3_MAC_Addr, AP3_fraction_time}>
Fingerprinting Wi-Fi

Similarity of fingerprints
fraction of time
M: union of MAC address of fingerprints f1 and f2

Use as filter/matching module


In the absence of light/color, we use it as matching
module.
Accuracy depend on location of shops.
Implementation

Client and server



Client: Nokia 95 phones using Python as client
Server: Matlab and Python code and some data
mining tools for fingerprinting algorithms.
Fingerprint database


Labor-intensive war-sensing at 51 stores
Store location: 46 business location in
university town, 5 location in India
Implementation
Evaluation

SurroundSense(SS) test environment



War-sensed 51 shops organized in 10 clusters
4 students visited the first nine clusters in university
town, while 2 students visited the tenth cluster in India.
4 localization models:




Wi-Fi only (Wi-Fi)
Sound, Accelerometer, Light and color ( Snd-Acc-Lt-Clr)
Sound, Accelerometer, Wi-Fi (Snd-Acc-Wi-Fi)
SurroundSense (SS)
Evaluation – Per-Cluster Accuracy
Similar hardwood
No Wi-Fi
floor in strip mall Same AP
False negative
Restaurant
Snd and Acc
Best represented
Evaluation – Per-Shop Accuracy

To understand the localization accuracy on a per-shop basis
30% shops
47% shops
SS: 92%
Snd-Acc-WiFi:
92%
Snd-Acc-Lt-Clr:
75%
WiFi: 75%
Evaluation – Per-User Accuracy

Simulate 100 virtual user, each assign 4~8 stores from cluster 1~9
Evaluation – Per-Sensor Accuracy

Hand-picked 6 samples to exhibit the merits and
demerits of each sensor
Number of shops left
after special filter
false negative
Percentage localized
using special sensors
Conclusion



Presented SurroundSense, a non-conventional
approach for logical localization.
Created fingerprints about ambient sound, light,
color, movement and Wi-Fi and match them with
fingerprint database to realize accurate logical
localization.
The evaluation achieved an average location
accuracy of over 85% using all sensors.
Discussion




The GPS 10 m, Wi-Fi and GSM 40m and 400m respectively.
Why not use Wi-Fi to get initial location instead of using
GSM?
Support vector machines (SVM), K-means clustering
algorithm are used in paper, do you have any better
machine learning methods? Such as Kalman filter, Particle
filter, and Wavelet Transform?
Can other sensors help? Such as compass and Bluetooth?
Energy consideration? Non-business location?