Transcript CULTH2 - The Knowledge Technology of I-MASS

The Knowledge Technology of
I-MASS (EC IST Research)
Communication, Computing and
Interactive Networks
Peter J. Braspenning
[email protected]
Local I-MASS group
Communications
Research &
Semiotics
(CR&S)
Peter-Paul Kruijsen, Gabriel
Hopmans & Peter J. Braspenning
15-01-2002
Communications Research & Semiotics
University of Maastricht
Assoc. Prof. Peter J. Braspenning, CR&S, UM
1
Signs & Meanings
Communications
Research &
Semiotics
(CR&S)
15-01-2002
2
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Content
Introduction
What is (Computational) Semiotics?
Semiotic Framework
Logo of CR&S
Communication Process among People
Perceive-Act Pathways
Multi-Modal User Interface:
Semiotic Engineering
Cognitive Engineering
Agent-Oriented Modeling
Communications
Research &
Semiotics
(CR&S)
15-01-2002
System’s Modeling via Multi-Agent Societies
Computational Semiotics for Agent Technology
Knowledge Landscape & VRR
Conclusions
3
Assoc. Prof. Peter J. Braspenning, CR&S, UM
What is Semiotics?
Communications
Research &
Semiotics
(CR&S)
15-01-2002
Semiotics: discipline of combining the theory of signs (representations), symbols (categories), and meaning extraction. It is an inclusive discipline which incorporates all aspects of dealing with
symbols and symbolic systems, starting with encoding and ending
with the extraction of meaning.
Mathematical tools of semiotics include those used in control sciences, pattern recognition, neural networks, artificial intelligence,
and cybernetics. Unified use within a computational semiotic
framework leads to better treatments of the complexities (communication and computation) inherent in advanced intelligent
systems. Semiotics is a strongly emerging multi-disciplinary field
of study around a new paradigm for surpassing the classical
mind-body dichotomy by focussing on all processes in which the
triad object-sign-interpreter plays an essential role.
The pervasive use of icons in the interaction with communicative
virtual environments (CVEs), is also part of Semiotics. A lot of
different kinds of signs are exchanged while communication
takes place.
4
Assoc. Prof. Peter J. Braspenning, CR&S, UM
What is Semiotics? (continued)
Communications
Research &
Semiotics
(CR&S)
15-01-2002
• Semiotics is devoted to studying COMMUNICATION: representations, their
interpretation and usage
• It investigates SIGNS and the processes by
which we take them to mean something to
us and expect them to mean something to
others
• It investigates the resolution of meanings
in conversations, collective discourse and
culture in general
• Semiotics also covers non-human communication processes such as that of animals
and machines/artificial systems
5
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Semiotic framework
Components of Symbol
System
[Schuyt]
Peirce: sign = “something
standing for something
else for somebody in one
or more respects”
interpretation system
interpretant
Communications
Research &
Semiotics
(CR&S)
groups
acts
& events
sign
object
15-01-2002
6
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Logo of CR&S
Communications
Research &
Semiotics
(CR&S)
15-01-2002
7
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Communication Process among People
interpretant
Unlimited
semiosis
interpretant
interpretant
coding
Communications
Research &
Semiotics
(CR&S)
medium
“I like guys
with dark hair”
decoding
message (signs)
15-01-2002
8
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Perceive-Act Pathways
Communications
Research &
Semiotics
(CR&S)
15-01-2002
9
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Semiotic Engineering
perspective of HCI
• design intention
• interaction principles
message
Communications
Research &
Semiotics
(CR&S)
system
designer
user
15-01-2002
10
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Cognitive Engineering
task model
+
user model
=
design
model
usage model
interaction
Communications
Research &
Semiotics
(CR&S)
designer
system
image
user
15-01-2002
11
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Cognitive Engineering x
Semiotic Engineering
Cognitive Engineering
context
Semiotic Engineering
context
context
medium
Communications
Research &
Semiotics
(CR&S)
designer
user
15-01-2002
12
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Semiotic Engineering and
Interface Evaluation
Communicability Concept
Communications
Research &
Semiotics
(CR&S)
15-01-2002
– Communicability is the property of software
that efficiently and effectively conveys to
users its underlying design intent and
interactive principles
– The communicability evaluation method
allows designers to appreciate how well users
are getting the intended messages across the
interface and to identify communication
breakdowns that may take place during
interaction
Assoc. Prof. Peter J. Braspenning, CR&S, UM
13
Assessing Communicability
Utterances
Communications
Research &
Semiotics
(CR&S)
Symptoms
User searches for a specific
Where is?
function but demonstrates
difficulty in locating it
User explores possibilities
What’s this?
of interaction
User performs an action,
Oops!
but the outcome is different
than expected / Cancels the
I can’t do it this way. sequence of actions &
chooses a different path
User expects an outcome
Why doesn’t it?
but doesn’t achieve it.
Insists on the same path
User ignores preferential
I can do otherwise. intended affordance present
in the interface
Classes of Interaction
Problems
Navigation
Meaning Assignment
Navigation
Meaning Assignment
Task Accomplishment
Missing of affordance
15-01-2002
14
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Agent-Oriented Modeling (issues)
Communications
Research &
Semiotics
(CR&S)
• Intelligent Agent: an assistant
that takes care of many gory
details of many mundane tasks
• Additional properties are
autonomy, sociability, and a
human-like communication
• Often able to adapt to user's
interests, habits and preferences
• Enabled to communicate with
other agents it is potentially
entering role-taking behavior
and social commitments with
other agents that allow it to
function in a society of agents


Multi-Agent System: bring
such agents together in a kind
of abstract society, wherein
coordination, cooperation
and/or collaboration are of
paramount importance in order
to solve problems that no
single agent could handle on its
own
FIPA specifications represent a
collection of standards, which
are intended to promote the
interoperation of
heterogeneous agents and the
services that they can
represent.
15-01-2002
15
Assoc. Prof. Peter J. Braspenning, CR&S, UM
System’s Modeling
via Multi-Agent Societies
Communications
Research &
Semiotics
(CR&S)
• One has to decide how to provide efficient inter-agent
communication support, what language should the
agents talk, should the agents be stationary or mobile,
and what technology should be used to build the
architecture
• At present, there are not much experience reports
• Architecture of a multi-agent system can naturally be
viewed as a computational organization
• Additional organizational concepts
– organizational rules,
– organizational structures, and
– organizational patterns
15-01-2002
16
Assoc. Prof. Peter J. Braspenning, CR&S, UM
System’s Modeling
via Multi-Agent Societies (continued)


Communications
Research &
Semiotics
(CR&S)
I-MASS uses MAS’s perspective
not just as a framework for interaction, but more as forming abstract
societies consisting of agencies
(comparable to societal institutions),
complex agents (in the sense of
consisting of simpler agents), and
agents (roughly comparable to
individuals in a societal context).
We try to deal with content interoperability issues at different
abstraction layers of syntactics,
semantics, pragmatics and social
world. These layers fit into a
coherent semiotics framework.
15-01-2002
17
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Computational Semiotics for
Agent Technology
Communications
Research &
Semiotics
(CR&S)
Coordination is cen- Therefore, agents have
tral to building
to have some autoMASs
nomy in performing
their actions.
Coordinating behaviors in MASs are
However, this autonooften realized by
my may lead to unforming groups in
coordinated activities
which both control
due to uncertainty
and data are distriabout the actions of
buted.
each of the agents.
15-01-2002
18
Assoc. Prof. Peter J. Braspenning, CR&S, UM
The relationship between
uncertainty and the situation that
the agents have to handle
Communications
Research &
Semiotics
(CR&S)
The uncertainty lowers as
the familiarity of the
situation that needs to be
handled increases!
Therefore, it makes sense
to develop a framework
in which agents know
how to handle routine,
familiar, and unfamiliar
situations
15-01-2002
19
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Co-ordination among agents:
guiding principles
Communications
Research &
Semiotics
(CR&S)
• Coordination among
agents is easier to establish in routine than in
unfamiliar situations;
• In general, communication between agents
will be more needed in
unfamiliar situations
than in routine situations.
15-01-2002
20
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Needed: an agent-architecture
in which three kinds of
interaction are adressed
• Conceptual models [J. Rasmussen, Information
Processing and Human-Machine Interaction: An Approach to
Cognitive Engineering, 1986]
– skills
– rules
– knowledge
Communications
Research &
Semiotics
(CR&S)
• The knowledge representation should be
adapted to these kinds of interactions.
15-01-2002
21
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Computational scenario
Communications
Research &
Semiotics
(CR&S)
15-01-2002
• First, perceived information from the environment
leads the agent to execute an action if the corresponding situation is perceived in terms of action.
• If this is not the case, the agent tries to recognize the
situation. It can recognize the considered situation in
terms of an action or in terms of a goal. In the first
case, it tries to execute the corresponding action, and in
the second case it invokes the planning module.
• Finally, if the agent faces an ambiguity and cannot
come to a decision, or faces many alternatives, then it
invokes the decisionmaking module (based on a
Cognitive Map) to make a decision in order to commit
to achieve a goal or an action. A goal leads an agent to
plan, that is to produce a sequence of actions that
achieve the chosen goal.
22
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Three levels of control of human
behavior
perception recognition decision planning execution
Knowledge
perception recognition decision execution
perception recognition planning a execution
Rules
Communications
Research &
Semiotics
(CR&S)
Skills
perception recognition planning b execution
perception recognition execution
perception execution
B. Chaib-draa & P. Levesque, Hierarchical Model and Communication by Signs, Signals and Symbols in
Multiagent Environments, 1998
a
15-01-2002
b
the planning process adapts old cases to the new situation, and the adaptation is significant
the planning process adapts old cases to the new situation, and the adaptation is generally minor
23
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Signals, Signs, and Symbols
Communications
Research &
Semiotics
(CR&S)
15-01-2002
Signals can be viewed as data
Signs indicate a state in the environrepresenting timespace
ment with reference to certain
variables from a dynamic,
norms for acts. In the case of
spatial configuration in the
communication by signs, the
environment and they can be
sender makes a sign which refers
processed directly by the
to some state of environment and
agents as continuous variawhich has the end of signifying, of
bles. In communication by
letting the receiver knows the
signals, the signal delivered
same reference. Of course, the
by an agent i has the end of
sender and the receiver should
simply being a releaser for
share a set of signs with their
the receiving agent j -- of
references in order to communisimply eliciting a reaction by
cate efficiently. For instance in urban
traffic, communication between a driver and a
j. That is, the signal generally
policeman at a crossroad is generally done by
invokes a stimulus or a
signs. The policeman makes a specific sign
which refers to a certain action and which is
reaction, without passing
addressed to certain driver(s). The addressee(s)
through the memory (a data
recognize(s) the reference of this sign and
activate(s) stored patterns of behaviors.
base in this model).
24
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Signals, Signs, and Symbols
Communications
Research &
Semiotics
(CR&S)
Information at knowledge and
Finally, agents can also communirule levels can act as symbols
cate by symbols. Symbols repredepending on the situation and
the language used for comsent variables, relations and
munication. In familiar situaproperties and can be formally
tions corresponding to the rule
processed. They are abstract conlevel, agents can use a specific
language (derived or not from
structs related to and defined by a
a natural language). This langformal structure of relations and
uage is generally constructed
processes, which according to
from repeated activities. When
unfamiliar situations occur,
convention can be related to
agents do not dispose of any
features of the external world.
operative knowledge nor of
any specialized language.
They must then make use of a
In urban traffic for instance, a dialogue between a
policeman and a driver in natural language
non specialized language (for
reflects a symbolbased communication. Another
example natural language),
example of symbolic communication is ``honk
which is less concise but
more flexible than their operthe car horn'', etc.
ative language used in familiar
situation.
15-01-2002
25
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Coordination: in summary
With signals and signs, agents
do not force their cognitive
control to a higher level
(i.e. the knowledge level)
than the demands of the
situation requires.
Communications
Research &
Semiotics
(CR&S)
15-01-2002
In contrast, agents have a
propensity for behaviors
based on skills and rules.
These behaviors are generally fast, effortless and
propitious to a better coordination between agents.
26
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Recap
Communications
Research &
Semiotics
(CR&S)
15-01-2002
• Communication and Semiosis are two
sides of the same coin;
• Knowledge Representation has traditionally only be treated in the context of solipsistic systems. However, communication
is constitutive for Knowledge (Representation) & Reasoning Reflection;
• Agent Technology as a modeling methodology allows us to treat rather complex
systems. Moreover, the semiotics perspective sheds new light on issues concerning
User Interfaces, Exploration Narratives,
and all emerging kinds of New Media
Assoc. Prof. Peter J. Braspenning, CR&S, UM
27
Knowledge Landscape & VRR
Communications
Research &
Semiotics
(CR&S)
• A systemic approach via
Agent-Oriented Modeling &
the development of agentbased tools, and
• An operational elaboration of
the new concept of the Virtual
Reference Room, by means of
which contextualized access to
heterogeneous objects can be
realized, and more knowledgebased navigation by means of
these contexts becomes
feasible.
15-01-2002
28
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Virtual Reference Room
Communications
Research &
Semiotics
(CR&S)
 Explanations about how to
explore the collections, products and services of the
institution;
 Delivery of information about
actual services of the Reference
Room that are in force;
 Pointers to where
bibliographical services may be
found;
 Pointers to relevant exhibitions
and other relevant events in
connection with their search
questions and associated
references;
 Orientation maps to more
autonomously explore the
facilities of the Reference
Room and around the particular
institutional collections
maintained;
15-01-2002
29
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Virtual Reference Room (cont.)
Communications
Research &
Semiotics
(CR&S)
15-01-2002
30
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Virtual Reference Room (cont.)
Communications
Research &
Semiotics
(CR&S)
15-01-2002
31
Assoc. Prof. Peter J. Braspenning, CR&S, UM
High-level OVERVIEW
I-MASS
1
MANIPULATIONS
Digital
Sources
4
User
Interface
Knowledge
Landscape
(KL)
5
Communications
Research &
Semiotics
(CR&S)
6
Query/Response and
KL-Building
2
3
Semantic & Syntactic
Inter-operability
15-01-2002
32
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Written in
Process Models
ABSTRACTION LAYER
AML
Constraints
Transformer
Content
Constraints
Increasing
Semantic
Expressiveness
KRL
Semantic
interoperability
Transformer
Content
O-O
Constraints
Transformer
Content
Constraints
RDF
Transformer
Content
Constraints
XML
Transformer
Content
Communications
Research &
Semiotics
(CR&S)
Syntactic
interoperability
Content providers
Query
Result
SQL
JDBC
Metadata
Metadata
Metadata
15-01-2002
33
Assoc. Prof. Peter J. Braspenning, CR&S, UM
THE VIRTUAL REFERENCE ROOM
P er
s pe
c ti
ve s
Projection
onto
user
screen
Knowledge Landscape Manipulation
- Tool 1
- Tool 2
- Tool 3
Knowledge Landscape
R
KO2
IS
R'
IS
KOcR
Reference' W ork
Reference' W ork
R''
KO3
KO1
Communications
Research &
Semiotics
(CR&S)
KOaR
KObR
Reference' W ork
IS
15-01-2002
34
Assoc. Prof. Peter J. Braspenning, CR&S, UM
The big picture
Real World
Virtual
World
Virtual Reference Environment
Heritage
Institutions
Virtual Reference Room
Query Specific
View of Structured
Reference Landscape
Communications
Research &
Semiotics
(CR&S)
Digitised
Artefact
Digitised Collections
Catalogues / Dictionaries / Artefacts
Encyclopaedias / Thesauri / Archives ...
User
Syntactic & Semantic
Interoperability
Uniform Structured
View of Digitised
Cultural Domain
Query
Digitised
Artefact
Web Sites
Web Pages
15-01-2002
35
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Conclusions w.r.t. I-MASS
Communications
Research &
Semiotics
(CR&S)
15-01-2002
• Short term research will address the precise kinds of
enabling technologies (e.g., from information retrieval,
ontological engineering and knowledge engineering) that
the I-MASS system should incorporate to synthesize
(configuring and presenting) pieces of information that
seem to fulfill an apparently existing need for information
at the side of the users;
• Also research about semantical (and pragmatical) interoperability will require much effort, especially as it
contributes quite a lot to the system’s ability to provide
good answers;
• Longer-term research must address how the cultural
domain may be modeled by means of process models that
capture the relevant insights of cultural processes (e.g.,
the rise and fall of the Roman Empire, or the Renaissance).
36
Assoc. Prof. Peter J. Braspenning, CR&S, UM
Conclusions
Communications
Research &
Semiotics
(CR&S)
15-01-2002
Leibniz’ (1646-1716) ambition was “to awake the
sleeping child in us all”
I-MASS forces us to use Knowledge Technology to the utmost and to use societal
metaphors as much as possible!
Assoc. Prof. Peter J. Braspenning, CR&S, UM
37