Transcript Intelligent User Interfaces

Intelligent User Interfaces
Frank Shipman
Department of Computer Science
Texas A&M University
E-mail: [email protected]
What this is about



Designing, building, and evaluating
intelligent user interfaces
Particular technologies used to create
intelligent user interfaces
Issues concerning applicability of
intelligent user interfaces
Intelligent user interfaces (IUI)?




Systems that provide interactive support
based on embedded AI mechanisms
Interfaces to AI functionality and
knowledge representations
Adaptive systems
Cooperative problem-solving systems
Technologies I
User models,
Situation models, and
Programming by Demonstration
Adaptive Interfaces

Requirements:
– interface that can be adapted
– user or situation model
– adaptation strategy

Frequently used for providing
assistance or training to user
User Model

“any information which a program has
which is specific to a particular user.
The information itself could range from
a simple count of errors, to some
complicated data structure which
purports to represent relevant parts of
the user’s knowledge of the problem
domain.”
Stereotyping vs. Individual

Stereotyping (canonical user modeling)
– provide interfaces for classes of users
– classes might be based on skill (novices,
midrange users, experts) or role

Individual approach
– dynamically adapt to suit each user
– can be based on observed use of system
or self assessment
Representations for User Models

Descriptive method
– modeling the user’s observed behavior
– describes what system has seen user do

Skill-based or cognitive method
– attempt to model the internal cognitive
models and processes of the user
– represents background knowledge, goals,
plans, preferences, misconceptions
Acquisition of User Model

Model based on a combination of:
– Observations of system use
• statistical history
• chronological history
– Self-assessment
– Testing

How can model evolve over time?
– Any of the above
Berkeley UNIX Consultant

Goal: Provide help to new UNIX users

Generates user model based on
“successful” use of UNIX commands
– Explanations of difficult commands can
make use of student’s knowledge.
Intelligent Tutoring Systems

Task is generally well-known
– assignments given to student by system
– systems track partial completion

Systems keep record of student’s
success and failure.
– used to determine future assignments
– used to determine how to help when
student has difficulties
Situation Models

Components of situation:
users, system, environment

Users
– multiple user models

System
– hardware constraints and load
– device / resource availability
Representing the Environment

Identifying environmental influences
– anticipating use situations
– classes of use vs. detailed model of
environment

Monitoring environment
– direct input devices
– user description
Mars Medical Assistant



Goal: Provide medical support for
astronauts on three year trip to Mars
Consider educational, consultation, and
emergency situations
Models of user and patient
– limited highly-trained user community
– no new users joining during mission
Other Adaptive Systems

Typing completion
– suggests completions for partial terms
based on prior use

Emacs suggestions
– notifies user when more efficient method
available to complete task

Computer Chess Game
– determines quality of own play based on
perceived level of opponent
Programming by Demonstration


Generalizing from demonstrated action
and situation sequences to programs
Difficulties:
– knowing what must stay the same
– knowing what are variables and their types
– connecting to programmed application
code
Programming by Demonstration
Systems




Peridot -- demonstration of simple
interface
Marquise -- demonstration of graphical
editors including palettes and modes
DEMO -- demonstrating dynamically
created objects
DEMO2 -- refinement by system based
on multiple demonstrations
Pavlov


Focus on programming animation
Includes:
– graphical objects
– models of motion and time

Stimulus-response demonstration
– modes for creating objects and behaviors
– mode for demonstrating interaction
Technologies II
Presentation generation,
Design Environments, and
Interface agents
Presentation Generation

Generating dynamic links to information
– enabling user-controlled flow

Generating presentations based on
current situation and/or user
– Use of user or situation model

Generating rhetorical structure/transition
– Scripting events
– Media-based decisions
Presentations and Explanations

Examples:
– Explainer (Redmiles)
– Explainable Expert System (Moore)
– Story Presentation System (Sgouros, …)
Explainer



Domain: Graphical program explanation
for software reuse
Creates links between perspectives on
software including source code,
documentation, execution information,
application domain view
Provides user multiple points of access
to better inform about software
Explainable Expert System (EES)



Explains different outcomes in an expert
system / planner
Generates natural language to answer
user’s questions
Keeps dialog history to provide
differential descriptions
EES Architecture for Explanation
User
Question
Knowledge Base
Query analyzer
User goals
Plan Operators
Text planner
Dialog history
Response
Sentence generator
Focus Information
EES Example
User: “Describe Inderal”
System: “Inderal a drug that …”
User: “Describe Elavil”
System: “Like Inderal, Elavil is used …”
User: “Describe Cafergot”
System: “Cafergot is very different from
the drugs we have been talking about.
…”
Story Presentation System


A dynamic dramatization method for
narrative presentations
Architecture:
Symbolic Plot
Description
Plot
Analysis
Dramatic Effects
Library
Dramatization
Original Story
Material
Presentation
Manager
Story
Presentation
Story Presentation Analysis

Plot analysis models:
– physical and emotional state changes
– positive and negative interference among
characters

Dramatization uses plot analysis to
determine dramatic events in story
– Lifeline, Rising complication, Reversal of
fortune, Dramatic irony, Happy end
Story Presentation Results


Presentation manager adds dramatic
effects to original story material to
emphasize dramatic events in story
Effects include
– audio: selection of noises or music
– images and video: flashbacks,
flashforwards, images of other characters
Design Environments

Provide a software environment
supporting the activities part of design.
– specification, construction
– argumentation, documentation,
communication

Examples:
– Framer (Lemke, Fischer)
– JANUS (Morch, McCall, Fischer , ...)
Framer


Knowledge-based support for interface
design
Approach:
direct manipulation interface builder
Framer 1 -- construction kit approach
Framer 2 -- design environment
Design Environment
Components (1)

Checklist
– system provides decomposition of task,
– user identifies current focus

Palette & Workspace
– system provides primitive components
– user identifies components used and
organization of components in design
Design Environment
Components (2)

Specification sheets
– system brings design issues to user’s
attention, presents potential answers, and
explains significance and consequences of
design choices
– user symbolically specifies answers to
design issues
Design Environment
Components (3)

Critics
– system points out sub-optimal design
decisions, explains why this is believed,
and provides heuristics for making
decisions
– users may accept or reject the system’s
critique
Design Environment
Components (4)

Catalog
– system provides examples
– user selects designs to reuse and modify

Code generator
– system generates an executable
representation of designed interface
Other Design Environments

JANUS -- kitchen design
– designed for non-technical users

XNetwork -- computer network design
– identified need for simulation component

VDDE -- voice dialog design
– another type of interface design with
interesting constraints
Software Agents

One view:
Software processes that have non-trivial
tasks delegated to them which require
independent action and a report on the
results.
Issues for Software Agents (1)

Personification
– Should agents be represented as a living
or animated character?
– Does it improve adoption of software?
– Does it create inflated expectations?
– Is it just too annoying?
Issues for Software Agents (2)

Trust and Competence
– How does user develop an informed level
of trust?
– Can agent give self-assessment on likely
outcome of task?

Delegation
– How can user delegate tasks?
– How can user check on status of delegated
tasks?
Issues for Software Agents (3)

Control
– How does user set limits on the agent’s
activity?
– When does the agent get to interrupt the
user (mixed-initiative dialog)?

Dealing with multiple agents
– How can the user manage many agents?
– How can interactions between agents be
predicted?
Information Retrieval Agents


Watch user patterns to infer interests or
goals which are used to classify, rank,
or suggest new information
Examples:
– INFOSCOPE: patterns in Netnews use
– BASAR: patterns in Web access

Issue: the “cold start” problem
– must watch a while before suggesting
Social Filtering

Finding elements liked by others (with
similar preferences)
– requires some notion of preferences
– improves with more users

Examples
– Tapestry -- rating of documents
– GroupLens -- collaboration & user profiles
– Amazon.com and CD-NOW
Technologies III
Knowledge manipulation and
Using recognized structure
Interacting with Knowledge

User tasks
– Adding knowledge
– Editing rule bases and object hierarchies

Examples
– HITS Knowledge Editor (Terveen)
– Modifier (Girgensohn)
– Hyper-Object Substrate (Shipman)
Knowledge Representations

Informal
– text, graphics, audio, video

Semi-formal
– hypertext, argumentation

Formal
– frames, semantic nets, scripts, rules,
inheritance hierarchies,
HITS Knowledge Editor

Knowledge editor for CYC project

Difficulties of knowledge representation
formalization - articulation in precise detail
comprehension - complex vocabulary, size
modification - location and consistency
Terveen’s Design Principle #1

Provide a workspace in which users
and systems can jointly construct
and manipulate a context for
problem-solving, and in which the
state of the problem-solving is
represented visibly.
Terveen’s Design Principle #2

Deliver intelligent assistance through
critics.
Terveen’s Design Principle #3

Exploit the interactive potential of
computational media to manage
the user-system interaction
according to conventions that are
appropriate to the role of each
party in the interaction.
Support Provided by HKE



Inferences -- information inferred from
workspace and existing KB
Troubles -- inconsistencies between
workspace and KB
Suggestions -- relevant representational
issues for users to consider
Modifier

Support for End-User Modifiability
– Users are not knowledge engineers

Example:
– Adding new object class to existing system

Support: suggestions based on
similarity of features and efficiency of
representation
Hyper-Object Substrate

Formalization as difficulty
– cognitive load
– tacit knowledge
– premature structure

Supporting incremental formalization
– Flexible knowledge representation
– Suggestions based on informal content
– Automating knowledge acquisition?
Using Recognized Structure


When can it be used without user
acknowledging correctness?
Examples:
TileBars (Hearst)
• graphically presents results of text analysis
inferring topic changes within text corpus
Data Mining
• search for patterns within database
Non-Verbal Representations



Non-verbal communication allows
expression of emergent thoughts
Systems need to recognize implicit
structure
Examples:
– VIKI (Marshall, Shipman, …)
– PerSketch (Saund, Moran)
– Tivoli (Moran, van Melle, Chui, …)
VIKI: An Analyst’s Workspace


Task: collect, organize, and interpret
information
Characteristics:
– evolving understanding
– ephemeral structures
– emerging visual languages

Goal: Recognize visual structures and
languages
PerSketch: Adaptive Drawing Tool



Recognize multiple potential
connections between strokes in drawing
Enables cleaning up drawings, editing
drawn objects
Example:
TIVOLI: Meeting Room Support




Manipulating structures on an electronic
whiteboard
Recognize gestures, characters, etc.
Recognize lists, groupings
Map changes in visual structure to
underlying database
Design Considerations
Issues of applicability,
Evaluating interfaces, and
Discussion
IUI: Appearing Too Smart?

A central danger of creating IUI is, like
ELIZA, appearing too smart
– Creating unrealistic expectations
– Communication difficulties
• false alarm
• garden path

Design guideline: do not hide system’s
knowledge or lack thereof
Evaluation Methods







Interviews
Cognitive Walkthroughs
Questionnaires
Videotapes of use
Usage data from software monitoring
Comparative evaluations
Pre- and post- use testing
Evaluating Interfaces

Deciding between observational and
empirical methods
– Is there a specific question being asked?
– How mature is the technology?
– Is having real tasks in a natural setting
important?
Expectation Agents

Support for greater user participation in
software design
– Software designers encode expectations of
use which are matched against actual use.
– When expectations are not met, a dialog
can be started between designers and
users.