Transcript Letizia: An Interface Agent for Assisting Web Browsing

User Interface Issues for Agents
& Adaptive Software
Henry Lieberman
Media Laboratory
Massachusetts Institute of Technology
Cambridge, MA 02139 USA
[email protected]
http://www.media.mit.edu/~lieber/
Henry Lieberman • MIT Media Lab
Making it “easy” for machines to
understand the Web
Machines can’t read HTML intended for humans
Well, they can now, somewhat, heuristically
The Semantic Web will make it “easy” for
machines to understand knowledge on the Web
Yeah, but only if people know what knowledge to
represent and how to encode it
Yeah, but only if people know how to make
agents that use the knowledge
Henry Lieberman • MIT Media Lab
User interface problems
User interface problem for the user/Web author
Increasing number of knowledge engineers by 10000s of
times
Will user encode everything twice?
User interface problem for agent developer
Whole environment too !?#&*@ complex
Can’t encode procedural knowledge
Can’t debug
Henry Lieberman • MIT Media Lab
User interface for Web developers is
becoming unmanageable
How many languages do you need to know to
program the Web?
W3C has 19 languages on its home page
XML+extensions, Javascript, Java, CGI+Perl,
Php…
No defined interoperability
Unreasonably difficult to debug!
Henry Lieberman • MIT Media Lab
Static semantics of the Web
Structure of Web pages and links
Including meta-data [RDF, DAML+OIL, etc.]
Henry Lieberman • MIT Media Lab
Dynamic semantics of the Web
The user browses through Web pages
The user’s interests change
Programs “crawl” through Web pages
Web pages do stuff
Web pages change
Henry Lieberman • MIT Media Lab
Dynamic semantics of the Web
Water: Encoding dynamic semantics in Web
“pages”
User interfaces for understanding dynamic
behavior
(Program execution or Inference)
Instrumentation & Localization
Control of level of detail
History & Reversibility
Henry Lieberman • MIT Media Lab
Water (née Glue)
- Fry, Plusch, Lieberman
Embeds procedures in Web pages
Integrates content, code, program data
XML-compatible syntax
Prototype object system
Integrated development environment, debugger
Henry Lieberman • MIT Media Lab
Water
Henry Lieberman • MIT Media Lab
ZStep
Keeps complete history of computation
Reversible
Control of level of detail
Multiple, synchronized views of process state
User-interface and implementation-level views
Henry Lieberman • MIT Media Lab
ZStep
Henry Lieberman • MIT Media Lab
The old “information retrieval”
perspective
User issues the “perfect query” to a static
database
System returns the “perfect document”
(Keyword1 … KeywordN)?
Henry Lieberman • MIT Media Lab
What’s wrong with the IR view?
Users can’t formulate precise queries
Empirically: Users do 1-2 word queries, don’t use
advanced query languages
There is no “best document” in the Web
Web keeps growing, changing
Real goal: To make the best use of the user’s time
Consequence: Web browsing is a real-time activity
Henry Lieberman • MIT Media Lab
Theorem-proving view is similar
Query
Descriptions
and
assertions
Answer
Henry Lieberman • MIT Media Lab
What’s wrong with the theoremproving view?
Neither users nor programs can formulate precise queries
There is no “best answer” in the Web
Web keeps growing, changing
Real goal: To make the best use of the user’s and agent’s
time
Consequence: Web browsing and inference is a real-time
activity
Henry Lieberman • MIT Media Lab
The new “agents” perspective
Web browsing/search/inference should be a
cooperative activity between a human user and
[one or more] software agents
Each participant should do what they do best:
Users are good at evaluation
Agents are good at search/computation
Both are active in real time, communicate
Henry Lieberman • MIT Media Lab
Some semantics will be
discovered/computed by agents
Not all semantics of the Web will be statically
encoded
Agents will compute semantics dynamically from
natural language [info extraction]
Agents will compute semantics from relationships
[collab filtering, popularity]
How do we integrate statically declared semantics
with dynamically computed semantics?
Henry Lieberman • MIT Media Lab
Examples of Web agents
Letizia
Mindshare
Aria
Expert Finder
Apt Decision
Henry Lieberman • MIT Media Lab
Letizia: An Interface Agent for
Assisting Web Browsing
Letizia acts as an advance scout for Web
browsing:
• It watches your Web browsing to try to learn
what topics you are interested in
• Formulates “queries”
dynamically/incrementally
• While you are reading a Web page, Letizia
searches the neighborhood of the page to
discover other pages you might be interested
in
Henry Lieberman • MIT Media Lab
• Does “search” dynamically/incrementally
User’s Search [Depth-First]
User’s Search & Letizia’s Search
Henry Lieberman • MIT Media Lab
Advantages of Letizia
While you search “wide”, Letizia searches “deep”
Letizia uses the time that you spend reading a
page to anticipate what you might interested in
Letizia filters out “junk”
Letizia maintains persistence of interest
Letizia is good at discovering serendipitous
connections
Henry Lieberman • MIT Media Lab
Mindshare - Van Dyke & Lieberman
Henry Lieberman • MIT Media Lab
Mindshare - Van Dyke & Lieberman
Tool for developing collaborative ontologies of
Web pages
Browser/editor for personalized views of a
collaborative ontology
Decide which aspects of common ontology to
include in your personal ontology
Decide which aspects of your personal ontology
you wish to contribute to the common ontology
Henry Lieberman • MIT Media Lab
Aria: Annotation and Retrieval
Integration Agent - Lieberman
Aria = Email/Web editor + Photo database + Agent
"Last weekend, I went to Ken and Mary's wedding…"
Henry Lieberman • MIT Media Lab
Aria: Annotation and Retrieval
Integration Agent - Henry Lieberman
Agent uses the context of the message to infer relevance of
photos to text
Agent automatically retrieves relevant photos as message is
typed
Agent automatically annotates photos with relevant text
from message
Streamlined interaction: No dialog boxes, file names, cut and
paste, load and save, typed queries, multiple applications,
etc. etc. etc.
Henry Lieberman • MIT Media Lab
Expert Finder
- Vivacqua & Lieberman
Henry Lieberman • MIT Media Lab
Expert Finder
- Vivacqua & Lieberman
Agent to help locate someone who can answer a
question
Domain of [novice] Java programming
Matchmaking in context of user’s and helper’s
expertise
Agent reads users’ Java programs, relates them to:
• Java ontology [analyzed from Sun doc]
• Model of Java expertise [analyzed from “Java in 21
Days”]
Henry Lieberman • MIT Media Lab
Apt Decision - Sybil Shearin
Complex decision making for e-commerce
Looking for apartment rental
Simulates interaction style of real-estate agent
with customer
Proposals generated from a few criteria
User can react to any aspect of proposal at any
time
System infers preferences
Henry Lieberman • MIT Media Lab
Apt Decision - Sybil Shearin
Henry Lieberman • MIT Media Lab
Berners-Lee, Hendler, Lassila
Henry Lieberman • MIT Media Lab
Sci Am Scenario
Local devices
SemWeb for small devices
Volume Control
Describing appliances
Prescribed Treatment
Medical SemWeb
Providers
In-Plan
20-Mile Radius
Home
Rating
Directory services
Defining concepts
Geographic information
Domestic applications
Web services
Henry Lieberman • MIT Media Lab
Sci Am Scenario
Appointment
times
Personal info
spaces
Location, time
preferences
User Modeling &
adaptation
Trust
Security
Reschedule
Replanning
Miscategorization
Debugging
Details?
Explanation
Henry Lieberman • MIT Media Lab
Henry Lieberman • MIT Media Lab