Transcript Why surf alone? Exploring the Web with reconnaissance agents

Interaction Challenges for Agents
with Common Sense
Henry Lieberman
MIT Media Lab
Cambridge, Mass. USA
Http://www.media.mit.edu/~lieber
Common Sense Computing • MIT Media Lab
Agents with Common Sense
Some AI interfaces are now beginning to make use
of large knowledge bases of Common Sense
Explicitly collected
Cyc, Open Mind, ThoughtTreasure
Distilled from other sources
Semantic Web, Wikipedia, Web resources
Web mining, other resource mining
Common Sense Computing • MIT Media Lab
Interaction Challenges
Finding opportunities for applying Common Sense
in interfaces
Setting users' expectations
Making interfaces fail-soft
Taking advantage of user interaction
Common Sense Computing • MIT Media Lab
Interaction Challenges
Making better mistakes
Get lots of knowledge, but not too much
Common sense inference vs. mathematical
inference
Debugging
Evaluating Common Sense interfaces
Common Sense Computing • MIT Media Lab
Interaction challenges for
AI/Common Sense
Many interaction challenges for Common Sense
interfaces are the same as for AI in general
But some are unique or critical for Common
Sense…
Can't be sure what will be known
Reasonable, rather than right
Know a little about everything, not much about
anything
Don't miss the obvious
Try not to make stupid mistakes
Common Sense Computing • MIT Media Lab
Open Mind Common Sense
Asks the Web community to contribute English sentenc
expressing Common Sense knowledge
"The Wikipedia version of Cyc" (#2 after Cyc)
Launched in 2001 by Push Singh
Now contains ~800 Kilofacts
Freely available / Open Source
Some versions in other languages/cultures
Brazilian Portuguese, Korean, Japanese, Chinese
Common Sense Computing • MIT Media Lab
Open Mind Common Sense
English sentences parsed by POS Tagger
Pattern-directed mining of 22 relations
(isA, PartOf, UsedFor…)
Strong focus on easy integration with applications
Semantic Net (ConceptNet: Liu, Eslick)
Natural Language toolkit (MontyLingua: Liu)
Tools for: Context, Analogy, Affect and more
Common Sense Computing • MIT Media Lab
What do we mean by “Common
Sense”?
Simple statements about everyday life
Things fall down, not up
A wedding has a bride and a groom
You go to a restaurant to eat
And…the ability to use that knowledge when
appropriate
Common Sense Computing • MIT Media Lab
Open Mind Common Sense
- Push Singh & colleagues
Common Sense Computing • MIT Media Lab
ConceptNet - Liu, Singh, Eslick
Common Sense Computing • MIT Media Lab
Common Sense projects
Predictive typing
Speech recognition -- disambiguation & error correction
Storytelling with photo libraries
Searching social networks
Macro recording using Common Sense generalization
World construction for video games
Phrasebook for tourist information
Debugging problems in e-commerce interactions
Common Sense Computing • MIT Media Lab
Common Sense projects
Video editing based on story structures
Goal-oriented interfaces for consumer electronics
Mining Common Sense from the Web
Multi-lingual and multi-cultural Common Sense; translation
Games for acquiring, verifying and using Commonsense
knowledge
Commonsense "Captchas"
Understanding imprecise qualities such as affect
ShapeNet and Expectation-driven Image Recognition
Understanding sensor data using Commonsense
Common Sense Computing • MIT Media Lab
Opportunities for using Common
Sense
Find UI situations that are underconstrained
Ordinary system would either take no action or
do something arbitrary
Then, give user some reasonable choices
Provide intelligent defaults
Make the most likely thing easiest to do
Common Sense Computing • MIT Media Lab
Opportunities for using Common
Sense
Recognize users' likely goals
Help users map from goals to actions
Sanity checking
In the case of trouble, help users debug
Common Sense Computing • MIT Media Lab
Opportunities for using Common
Sense
Find situations where every little bit helps
A little bit of knowledge is better than none
A little bit of knowledge about a lot of things can
be more useful than a lot of knowledge about a
few things
Common Sense Computing • MIT Media Lab
Setting users' expectations
Avoid direct question-answer interfaces
Right or wrong. Only one shot.
Better to cast system in role of advisor
Making suggestions, help
Adapting interface to most likely uses
Remove unnecessary steps in the interface
The user only expects intelligent behavior only once
in a while
Common Sense Computing • MIT Media Lab
Take advantage of user
interaction
Repurpose input that the user gives you for other
reasons
Every time the user tells the interface something,
they're telling you what their interests are -learn from it
Common Sense Computing • MIT Media Lab
Make Common Sense interfaces
"fail-soft"
There should no dire consequences of being
wrong or not knowing what to do
Don't interfere if the user wants to use the
application without interaction with the agent
If the relevant knowledge is missing, incomplete
or wrong, the user is no worse off than without
the agent
Common Sense Computing • MIT Media Lab
Make better mistakes
Common Sense approaches have the advantage
that when they make mistakes, they tend to
make plausible mistakes
Statistical approaches can make arbitrary
mistakes
Better mistakes improve user trust in interfaces
Common Sense Computing • MIT Media Lab
Evaluation of Common Sense
interfaces
Evaluation is tough because
Depends on what's in the knowledge base
Depends on limited-depth and other kinds of
approximate inference
Standardized tasks don't test breadth of coverage
Try to relativize testing to coverage
Start with easier cases, then move to "typical" or
"hard" cases
Common Sense Computing • MIT Media Lab
Intelligent User Interfaces 2008
www.iuiconf.org
• Location: Canary Islands, Spain
• Dates: 13-16 January 2008
• Deadline: late September 2007