Transcript Document

Your Life as a Movie
Sailesh Chutani
Director, Worldwide University
Relations
Microsoft Research
What if..
Everything you ever say, hear, observe,
do, and sense is recorded
Those recordings are stored permanently
and can be accessed easily
You could analyze or modify copies of
these recordings and share them
Some Questions
Is this scenario technically feasible?
What would such a future look like?
What new opportunities and risks will we
be confronted with?
Elements of such a future are already here
but just not uniformly distributed (to
paraphrase William Gibson)
Microsoft Research
Founded in 1991
Staff of over 700 in over 55 areas
Internationally recognized research teams
200+ WW university relationships
Research lab locations :
– Redmond, Washington
– San Francisco, California
– Cambridge, United Kingdom
– Beijing, People’s Republic of China
– Mountain View, California
A University CS Department
inside Microsoft
University organizational model
– Flat structure, critical mass groups
Open research environment
– Aggressive publication of results in peerreviewed literature
– Frequent visitors, daily seminars
Strong ties to University Research
– Nearly 15% of basic research budget directly
invested in Universities
Lab grants, research grants, fellowships, etc.
– Hundreds of interns and visitors
MSR Mission
Expand the state of the art in each of the
areas in which we do research
Rapidly transfer innovative technologies
into Microsoft products
Ensure that Microsoft products have a
future
Expanding the State of the Art
Thousands of peer-reviewed publications
13% of papers at 2001 CHI (UI)
20% of papers at 1996 SIGGRAPH (graphics)
25% of papers at 1999 PLDI (prog languages)
30% of papers at 2001 PLDI
More papers at SIGGRAPH 2003, SIGMOD 2004 , SIGIR
2004 and other key conferences than any other organization
Community leadership
Professional societies
Journals
Conferences
Segmenting the Problem
Information Capture
Storage
Query and Retrieval
Analysis and Processing
SenseCam
A Digital Memory for Everyone
Lyndsay Williams & Ken Wood @ MSRC and the MyLifeBits team @ BARC
SenseCam
Essentially a “Black Box” data and image recorder
for the human body
Current wearable prototype can be worn for a day
and captures up to 2000 images in that time
Image capture triggered by sensors, e.g. motion,
light, temperature, people in field of view, ...
Sensor data is also recorded for later presentation,
analysis, and correlation
Ultra Wide-Angle Lens
Captures everything regardless of exact camera orientation
Image Stabilisation
Accelerometer measures motion of the device
When any sensor indicates image capture, we wait a few
tens of milliseconds to capture image when movement is
less than 10 degrees/second if possible
Greatly reduces the number of blurred images
Before
After
A Short Walk Through Cambridge
How does SenseCam work?
Accelerometers for motion detection
Passive Infrared (PIR) sensors
triggered by people or other living
beings in front of camera
Digital sensors for light level and
temperature
Images captured automatically on
sensor events caused by, e.g.
• turning around, sitting down,
standing up, running, falling over, ...
• moving between locations (rooms,
car, bus, indoor/outdoor, etc.)
• meeting a friend in the street
Simple hand gesture allows
intentional image capture
Accelerometers also allow image
stabilisation
SenseCam Applications
Personal memory enhancement
– Where did I leave my umbrella?
– What was the name of that wine we had last week?
– What was written on the whiteboard?
Tourism
– Automatic photo journals or blogs of specific trips,
e.g. to the Edinburgh Festival
Clinical support for the memory-impaired
– We are working with a Cambridge neuropsychologist
on a trial for Alzheimer’s patients
Future
Now building next-gen SenseCams
– smaller, better power management, audio
trigger, wireless comms
Storage Needs
Number of seconds in 80 years
{60*60*24*365*80=2,522,880,000}
Data recording rate for MPEG-4 is 4Mb/s
Storage required for full motion video of 80
years of someone’s life is 10k tera bits or 10
peta bits (10^15)
Cost of such a storage in today’s terms is
approximately $10m
Storage capacity is doubling every year, in 10
years 10 peta bits may cost <$10k
Storage Trends
Source: Ed Grochowski, IBM Research Almaden
Moore’s Law
70
60
50
40
30
20
10
0
1998
2001
2004
2007
2010
“A threshold event will take place early in the 21st century: the emergence of machines
more intelligent than their creators. By 2019, a $1,000 computer will match the processing
power of the human brain - about 20 million billion calculations per second. Organizing
these resources - the “software” of intelligence - will take us to 2029, by which time your
average personal computer will be equivalent to 1,000 human brains.” -- Ray Kurzweil
Human Scale Storage
Terabyte ==
– $1600 today
– $400 in 2-3 years
Terabyte can hold
– A lifetime of all your conversations
– A year of what you can see
Querying
When did I meet the Dalai Lama?
How long did I live in Lausanne?
How many people did I encounter in my
life with whom I spend more than 5
minutes in conversation?
Where was I on my wife’s birthday during
the last 10 years?
What were the scariest moments in my
mountaineering trips?
Stuff I’ve Seen (SIS)
Support information re-use
Today:
– Many locations
– Many interfaces
With SIS:
– Unified index
– Fast, flexible
search
– Immediate update
– Rich UI
possibilities for
your content
SIS Usage Observations
Very short queries (1.6
words), quick filtering/sorting
Few searches for “best
matching” item
August, 1986!
3.5
– Know and use other attributes
– 50% more than one month old,
some up to 8 years
Importance of people
– 25% of queries involve people
Importance of time
– Date by far the most common
sort attribute
– “Useful” date varies by source
2.5
log(Frequency)
Age of items opened
All Items
3
2
1.5
1
0.5
f(x) = -1.111862*x + 3.809612
R^2 = 8.017632E-1
0
0
0.5
1
1.5
2
2.5
3
log(Days Since Item Creation)
3.5
4
Contextualizing Search
Search is not the end goal …
Goal is information management in the context
of ongoing work activities
– Search always available
– Search from within apps
– Show results in context of apps
MemoryLens
Harness models of human memory
to assist with navigating large
stores of personal content
MemoryLens
Search UI
Memory
landmarks
• Calendar events
• Images from store
• Key news stories
• Holidays
Hits over time
Search
hits
Modeling the interface after the
way people think & feel
Cognition
– Memory
associations vs. facts/details
– Deep History
objects recede vs. disappear
– Dynamic organization
multiple simultaneous categories vs. fixed
Personality & Emotion
– Know what people care about and use to help organize
– Watch users habits & behavior to customize
experience
Context
– Many people, places and events
MemoryLens
LifeBrowser
Adjustable threshold
for memorability
Walkthroughs
Re-live the past, explore the events as
they happened at your own pace
Analogous to “action replay” but at a large
scale
DEMO
WWMX (wwmx.org) Geography
for Photography
The World-Wide Media eXchange is
a centralized index of
the world’s image media
tagged by location (and other standard metadata).
Why Location for Photos?
Provides immediate context, hints at semantics
– E.g., photos taken at Disneyland, video shot in Hawaii…
Repeatedly appears in user studies regarding
organization
– Family trips strongly associated with location
– Event = time + location
– Subject as person doing something somewhere
Universal index (like date/time)
– Independent of ontology/taxonomy, language, culture, etc.
– Familiar visual browsing UI via maps
– Scales well
Technology to acquire location becoming common
– GPS, e911 services, radio-tower-based, 802.11-based…
Applications
Real estate tours
Travel and tourism
Shared travelogues
Driving directions with photos of
landmarks
Centralized media depository for
community events
Etc.
Creating Art
Recordings can serve as the raw material
for creating art
DEMO
Meta Questions
Who would regulate the recording of this
information?
Who will own this information?
Who would be able to access it?
Should copies and modifications be
allowed?
What would be the legitimate uses of this
information?
Meta Questions
What would it mean for us as human beings?
– Will human behavior change in light of the “permanent record”?
– How will we substitute for “forgetting” if memory becomes
perfect?
– Would we become more empathetic if we can literally see other
points of view?
– Would resolving conflicts be just a matter of comparing
recordings?
– Would we create a class of people who live vicariously?
– Would life be safer?
– Is video/experience editing the next big growth industry?
Clues for Answers
Current usage in forms of
– video surveillance,
– credit histories, public records
– policy on picture phones,
– popularity of reality TV
– use of web search to get background
Exploration in Art, movies that have
touched the themes
– Being John Malkovich, After Life, Slacker
Conclusions
Scientists and engineers create the
intellectual and physical building blocks;
Artists imagine the possibilities, but
It is the interplay of Society and Culture
that shape the actual usage and norms