Transcript Location-Aware Computing John Krumm Microsoft Research Redmond, Washington, USA

Location-Aware Computing
John Krumm
Microsoft Research
Redmond, Washington, USA
Thank You
• Toshio Hori
• Takeo Kanade
• Chie Nakamura
Digital Human
Research Center
Location
Why Bother to Sense Location?
• Find conference room near me
• Who is in this meeting with me?
• Where are the people who are supposed to be here?
• If I’m in conference room, don’t allow cell phone, alerts, or IM
• How long will it take to get from here to next appointment?
• Route planning in strange buildings
No Really, Why Bother?
• Remind me when I’m near a certain customer
• Where are my kids, buddies, colleagues?
• Index my documents (email, photos) by location
• Change default printer and network settings based on location
• Electronic graffiti, e.g. “There’s a better Thai restaurant one
block north.”
Why Not Use GPS?
• Does not work indoors
• Needs view of satellites
Location Sensing
Rosum, promising
but where is it
going?
Indoor/outdoor,
coverage
spreading
Still a hard problem,
can give much more
than location (good
& bad)
Hazas, Scott, Krumm, “Location-Aware Computing”, IEEE Computer Magazine, February 2004.
Beyond Location
sensors → location → context
Patterson, Liao, Fox, Kautz, “Inferring High-Level Behavior from Low-Level
Sensors”, 2003
Use GPS tracking to infer user’s mode of
transportation as (car, bus, walk)
Sparacino, “Sto(ry)chastics: A Bayesian Network Architecture for User
Modeling and Computational Storytelling for Interactive Spaces”, 2003
Use indoor location sensing in MIT
museum to classify visitor into (greedy,
busy, selective)
Outline
•
•
•
•
•
•
•
•
Introduction
Video Tracking
Active badge – SmartMoveX
Coarse Wi-Fi location – “Here I Am”
Fine Wi-Fi location – LOCADIO
SPOT Wristwatch Location – RightSPOT
Wi-Fi proximity – NearMe
“Longhorn” Location Service – Sensor Fusion
Video Tracking
EasyLiving Project
Steve Shafer
Barry Brumitt
Steve Harris
Brian Meyers
Greg Smith
Mike Hale
John Krumm
Video Tracking Is Very …
•
•
•
•
•
•
… accurate (centimeters)
… easy for user (no devices to carry)
… hard to set up (camera calibration)
… hard to get right (live demos still rare)
… CPU intensive (one PC per camera)
… intrusive
Research can solve
Outline
•
•
•
•
•
•
•
•
Introduction
Video Tracking
Active badge – SmartMoveX
Coarse Wi-Fi location – “Here I Am”
Fine Wi-Fi location – LOCADIO
SPOT Wristwatch Location – RightSPOT
Wi-Fi proximity – NearMe
“Longhorn” Location Service – Sensor Fusion
SmartMoveX
Microsoft Research’s entry into the
active badge space (along with Xerox
PARC, UW, Intel, AT&T Cambridge, MIT,
etc.)
Receiver
Transmitter
Multiple receivers
for position
triangulation from
signal strengths
Hardware: Lyndsay Williams (Microsoft Research Cambridge UK)
Software: John Krumm & Greg Smith (Microsoft Research Redmond)
Receiver Network
RX
RX PC
RX PC
RX
RX PC
DB PC
RX
RX PC
RX
Graph Algorithm
Signal Strength vs. Time
190
180
Signal Strength
170
160
150
140
130
120
110
0
5
10
15
20
25
30
Tim e (sec)
35
40
45
50
55
60
• Compute path instead of single
locations
• Constrain path to allowable routes
• Process with Hidden Markov Model
(HMM – same as used for speech
recognition)
• Average error 3.05 meters
Constraints make things easier
SmartMoveX Evaluation
Good
• Cheap hardware
• Uses existing network infrastructure
• Graph algorithm imposes natural constraints on paths
Needs Improvement
• Privacy – depends on central server
• Convenience
– Extra device to wear
– No display on device
• Infrastructure – requires special receivers
Outline
•
•
•
•
•
•
•
•
Introduction
Video Tracking
Active badge – SmartMoveX
Coarse Wi-Fi location – “Here I Am”
Fine Wi-Fi location – LOCADIO
Active Badge
SPOT Wristwatch Location – RightSPOT
Wi-Fi proximity – NearMe
“Longhorn” Location Service – Sensor Fusion
“Here I Am” – Coarse Location
With Steve Shafer (Microsoft Research)
“Here I Am”
802.11 Access Point Data
MAC Address
Building
Floor
Room
0:40:96:29:e:7
18
2
2075
0:40:96:29:f:78
18
1
1407
0:40:96:29:1a:6c
18
2
2229
0:40:96:29:24:c7
18
1
1055
0:40:96:29:3f:c3
18
1
1212
0:40:96:29:48:37
18
2
2115
0:40:96:29:4a:40
18
2
2039
0:40:96:29:55:8
18
1
1081
0:40:96:29:57:a0
18
2
2498
0:40:96:29:58:2f
18
1
1265
0:40:96:29:59:16
18
1
1426
0:40:96:29:5b:1d
18
1
1137
0:40:96:29:63:1a
18
3
3367
“Here I Am” returns position of
strongest access point
Room Numbers Hand-Entered from Maps
Outline
•
•
•
•
•
•
•
•
Introduction
Video Tracking
Active badge – SmartMoveX
Coarse Wi-Fi location – “Here I Am”
Fine Wi-Fi location – LOCADIO
SPOT Wristwatch Location – RightSPOT
Wi-Fi proximity – NearMe
“Longhorn” Location Service – Sensor Fusion
Location from 802.11 with LOCADIO*
John Krumm & Eric Horvitz
Wi-Fi (802.11)
access point
• Mobile device measures signal strengths from Wi-Fi access points
• Computes its own location
*
Location from Radio
LOCADIO - Fine Location
Radio survey to get signal strength as a function of position
LOCADIO - Constraints
Make the client as smart as possible to reduce
calibration effort
A Posteriori Probability of Move
moving
actual
still
moving
unsmoothed
still
moving
smoothed
still
200
No passing through walls
No speeding
400
600
800
time (seconds)
1000
1200
We know when you move
LOCADIO - Results
relative frequency
Error Histogram
0.3
0.2
0.1
0
0
1
2
3
4
5
6
7 8 9 10 11 12 13 14 15 16
error in meters
relative frequency
Cumulative Error Distribution
1
0.5
0
0
1
2
3
4
5
6
7 8 9 10 11 12 13 14 15 16
error in meters
Hidden Markov model gives median
error of 1.53 meters
Outline
•
•
•
•
•
•
•
•
Introduction
Video Tracking
Active badge – SmartMoveX
Coarse Wi-Fi location – “Here I Am”
Fine Wi-Fi location – LOCADIO
SPOT Wristwatch Location – RightSPOT
Wi-Fi proximity – NearMe
“Longhorn” Location Service – Sensor Fusion
SPOT Watch
Commercial FM:
transmit new data
every ~2 minutes
Filter on watch to
take what it wants
weather
traffic
dining
movies
Watch displays
“personalized” data
Location-Sensitive Features
Nice to have
• Local traffic
• Nearby movie times
• Nearby restaurants
Need to know location of
device …
Use FM Radio Signal Strengths
88.5
89.3
89.3
89.5
89.7
89.9
89.9
90.1
90.3
90.7
90.9
91.3
91.7
92.5
93.3
94.1
94.5
94.9
95.7
96.5
97.3
98.1
98.9
99.9
100.7
101.5
102.5
103.7
104.5
104.9
105.3
106.1
106.9
107.7
Scan signal strengths of 32 FM radio stations at 1 Hz
RSSI Values in One Suburb
660
raw rssi
median filtered
650
640
630
620
RSSI
KPLU
KAOS
KASB
KNHC
KWFJ
KGHP
KGRG
KUPS
KEXP
KSER
KVTI
KBCS
KBTC
KLSY
KUBE
KMPS
KRXY
KUOW
KJR
KYPT
KBSG
KING
KWJZ
KISW
KQBZ
KPLZ
KZOK
KMTT
KMIH
KFNK
KCMS
KBKS
KRWM
KNDD
610
600
590
580
570
560
0
100
200
300
400
Sample Number
500
600
700
Redmond
Bellevue
Seattle
Woodinville
Measured RSSI
KMTT
Clustering Approach
Sammamish
KPLU
A
B
C
Input Power
But
• Each watch scales signal strengths
differently
• Impractical to calibrate every watch
Ranking Approach
Redmond:
Bellevue:
Issaquah:
…
KPLU < KMTT < KMPS
KMTT < KPLU < KMPS
KMTT < KMPS < KPLU
Any monotonically increasing
function of signal strength
preserves ranking
n radio stations  n! possible rankings
f = (f1, f2, f3, …, fn) = scanned frequencies
e.g.
s = (s1, s2, s3, …, sn) = signal strengths
r = (r1, r2, r3, …, rn) = ranks of signal strengths
R(r) = permutation hash code = [0, 1, 2, … n!-1]
s = (12, 40, 38, 10)
r = (2, 4, 3, 1)
Test
Six suburbs and six radio stations
1
0.8
0.6
0.4
0.2
0
0
1
2
3
4
5
Permutation Hash Code
81.7% correct from 8 radio stations
Sammamish
Issaquah
Bellevue
Kirkland
Woodinville
Redmond
Location
Avoid Manual Training
Seattle
KMPS 94.1 MHz
KSER 90.7 MHz
Classify Into Grid Cell
• Find location in grid
• Use predicted signal
strengths to avoid
manual training
≈ 8 kilometers
average error
Summer
intern Adel
Youssef, U.
Maryland
Outline
•
•
•
•
•
•
•
•
Introduction
Video Tracking
Active badge – SmartMoveX
Coarse Wi-Fi location – “Here I Am”
Fine Wi-Fi location – LOCADIO
SPOT Wristwatch Location – RightSPOT
Wi-Fi proximity – NearMe
“Longhorn” Location Service – Sensor Fusion
“NearMe”
Find people and
things nearby
printers
Person
people
reception desk
bathroom
conference
rooms
NearMe Basic Idea
802.11
Access
Points
NearMe Server
NearMe ≠ Location
Why compute absolute locations
when you only need relative
locations?
Hightower, Fox, Borriello,
“The Location Stack”, 2003
Tomasi, Kanade, “Shape and
Motion from Image Streams: a
Factorization Method”, 1991
Short Circuit
Get location
Compare
Get location
Get signal strengths
Compare
Get signal strengths
NearMe Screen Shots
1. Register with server
2. Report Wi-Fi signals
NearMe Server
• SQL Server
• .NET Web Service
3. Nearby people
4. Nearby printer(s)
NearMe Distance Estimate
Estimated Distance Between Clients
Distance in Meters
25
Nu
m
be
Estimate distance between
two clients by comparing “WiFi signatures”
20
15
10
15 meters rms error
5
ro
f A1
cc
es 2
sP
oin
ts 3
in
Co 4
m
mo
n, 5
n∩
-1
-0.75 -0.5
Distance = f(n∩,ρs)
-0.25 0.0
an Co
Spearm
0.25 0.5
rrelati on,
0.75 1.0
ρs
n∩ = number of access points seen in common
ρs = Spearman rank correlation of signal strengths
NearMe Applications
Device association
(with Ken Hinckley,
Microsoft
Research)
Look up URLs of
nearby people/things
Send email to
people
nearby
Outline
•
•
•
•
•
•
•
•
Introduction
Video Tracking
Active badge – SmartMoveX
Coarse Wi-Fi location – “Here I Am”
Fine Wi-Fi location – LOCADIO
SPOT Wristwatch Location – RightSPOT
Wi-Fi proximity – NearMe
“Longhorn” Location Service – Sensor Fusion
“Longhorn” Location Service
“Longhorn” PC knows its location
GPS
cell phone
Wi-Fi
your location
Location
Service
Bluetooth
other
unanticipated
AN/PLR-3 Helmet-Mounted Radar
(I am not making this up.)
other location resolvers
Longhorn Location Service
Mgmt App
AD
Resolver
Master
Resolver
WinFS
Fuser
User
Resolver
Cache
Plugin
Manager
LocProv API
OEM
Provider
Tracey Yao, PM
Florin Teodorescu, Dev
Vivek Bhanu, Dev
Jim Seifert, Test
Madhurima Pawar, Test
OEM
Service
Blue Tooth
Provider
BT
Configuration
Service
AD
Map Point
Resolver
Notification
Service
Location API
(Shell, OEM)
App
LocMgmt API
User Pref.
Db.
(shell, netxp, OEM)
MapPoint
Db / Serv.
802.11
Provider
Wireless (802.11)
Zero Configuration
Service
Fusion
Proper fusion of measurements depends on knowledge of uncertainty
Metric Measurements
0.12
Hierarchies
0.12
0.1
0.1
0.08
0.08
0.06
0.06
0.04
0.04
0.02
0.02
0
10
0
10
5
10
5
0
0
-5
Y
Building
Floor
Room
5
-10
X
40 (0.4)
41 (0.4)
3 (0.8)
3 (0.2)
1 (0.3)
3019 (0.2)
1502 (0.2)
10
5
0
0
-5
-5
-10
40 (0.9)
Y
Kalman Filter
-5
-10
-10
X
Weighted Hierarchical Voting
The End