Transcript 1. Intriduction to Java Programming for Beginners, Novices

ABM: Decision making
Dr Andy Evans
Thinking in AI
Agent based systems and other AI can contain standard maths
etc.
But their real power comes from replicating how we act in the
real world: assessing situations, reasoning about them, making
decisions, and then applying rules.
Reasoning: “if a café contains food, and food removes hunger,
a café removes hunger”
Rules: “if my hunger is high, I should go to a café”
Reasoning
Reasoning is the remit of “brain in a box” AI.
Useful for:
Developing rulesets in AI.
Interpreting requests from people (Natural Language
Processing).
Developing new knowledge and replicating sentience.
Reasoning
Programming languages developed in the late 60’s / early 70’s
offered the promise of logical reasoning (Planner; Prolog).
These allow the manipulation of assertions about the world:
“man is mortal” and
“Socrates is a man” leads to
“Socrates is mortal”
Assertions are usually “triples” of
subject-predicate [relationship]-object.
There are interfaces for connecting Prolog and Java
http://en.wikipedia.org/wiki/Prolog#Interfaces_to_other_languages
Reasoning
This led to SHRDLU (MIT), which could do basic natural
language parsing and responses, based on a limited knowledge
of the world.
SHRDLU, however, was very limited.
Need more knowledge about the world.
Need, therefore, to get everyone involved in uploading
knowledge.
Led to projects like Cyc / OpenCyc (1984+; 306,000 facts) and
Open Mind Common Sense (1999+; 1,000,000+ facts).
Reasoning
An obvious source for information is the web.
However, parsing natural language is not easy – a lot of the
semantic content (meaning) relies on context.
It would be easier if language was embedded in an ontology
(structured collection of knowledge). For example “death”
[the event] was marked up as different from “Death” [the
medieval figure]
Obvious similarities to metadata – standards for which (eg
Dublin Core) led to work on structuring knowledge.
Semantic Web
1990’s Tim Berners-Lee and others pushed for a “Semantic
Web” marked up for meaning in context.
1999 Resource Description Framework (RDF) – way of marking
up web content so it relates to a framework of triples defining
the content. Very general.
Used to identify eg. initial Really Simple Syndication (RSS) feeds
(now not RDF).
But, not very specific, needs another language to extend it.
DARPA Agent Markup Language
1999 DARPA (US Military Research) wanted a machine-readable
knowledge format for agents, that could be used over the web.
Developed a RDF extension for this which led to the Web
Ontology Language (OWL).
This can be used to build more specific RDF ontologies.
http://www.schemaweb.info/schema/BrowseSchema.aspx
Example: Friend of a Friend (FOAF)
Issues
Easy to build self-contained ontologies, but what happens
when the same term is used by different groups. How do we
resolve conflicts?
Automated reasoning about knowledge is still not great
because ontologies don’t capture the estimation and flexibility
involved in real reasoning (eg. accepting paradoxes).
By concentrating on formal definitions rather than experience
they tend to be circular. Alternative: Folksonomies, but
difficult in modelling.
Hard to move from knowledge to action. For this we need
rules for when to act, and behaviours.
Thinking for agents
Building up rulesets is somewhat easier.
Though it is harder to represent them in a coherent fashion as
there’s an infinite number of things one might do.
While many standards for representing agents’ states, few for
representing rules, and most very very limited.
Usual for these to just be encoded into a programming
language.
Rulesets
Most rules are condition-state-action like:
“if hunger is high go to café”
Normally there’d be a hunger state variable, given some
value, and a series of thresholds.
A simple agent would look at the state variable and
implement or not-implement the associated rule.
How do we decide actions?
Ok to have condition-state-action rules like:
“if hunger is high go to café”
And
“if tiredness is high go to bed”
But how do we decide which rule should be enacted if
we have both?
How do real people choose?
Picking rules
One simple decision making process is to randomly choose.
Another is to weight the rules and pick the rule with the highest
weight.
Roulette Wheel picking weights rules then picks probabilistically
based on the weights using Monte Carlo sampling.
How do we pick the weights? Calibration? Do we adjust them
with experience? For example, with a GA?
We may try and model specific cognitive biases:
http://en.wikipedia.org/wiki/List_of_cognitive_biases
Anchoring and adjustment: pick an educated or constrained
guess at likelihoods or behaviour and adjust from that based on
evidence.
Reality is fuzzy
Alternatively we may wish to hedge our bets and run several
rules.
This is especially the case as rules tend to be binary (run / don’t
run) yet the world isn’t always like this.
Say we have two rules:
if hot open window
if cold close window
How is “hot”? 30 degrees? 40 degrees?
Language isn’t usually precise…
We often mix rules (e.g. open the window slightly).
Fuzzy Sets and Logic
Fuzzy Sets let us say something is 90% “one thing” and 10%
“another”, without being illogical.
Fuzzy Logic then lets us use this in rules:
E.g. it’s 90% “right” to do something, so I’ll do it 90% opening a window, for example.
Fuzzy Sets
Degree of membership
We give things a degree of membership between 0 and 1 in several
sets (to a combined total of 1).
We then label these sets using human terms.
Encapsulates terms with no consensus definition, but we might use
surveys to define them.
1
0.5
Cold
0
17° = 15% cold + 85% hot
Hot
20
Membership
function
40
Degrees
Fuzzy Logic models
We give our variables membership functions, and express the
variables as nouns (“length”, “temperature”) or adjectives
(“long”, “hot”).
We can then build up linguistic equations (“IF length long,
AND temperature hot, THEN openWindow”).
Actions then based on conversion schemes for converting
from fuzzy percentages of inputs to membership functions of
outputs.
Bayesian Networks
Of course, it may be that we see people in one
state, and their actions, but have no way of
replicating the rulesets in human language.
In this case, we can generate a Bayesian Network.
These gives probabilities that states will occur
together.
This can be interpreted as “if A then B”.
They allow you to update the probabilities on
new evidence.
They allow you to chain these “rules” together
to make inferences.
Bayesian Networks
In a Bayesian Network the states are linked by probabilities,
so:
If A then B; if B then C; if C then D
Not only this, but this can be updated when an event A
happens, propagating the new probabilities by using the new
final probability of B to recalculate the probability of C, etc.
Expert Systems
All these elements may be brought together in an “Expert
System”.
These are decision trees, in which rules and probabilities link
states.
Forward chaining: you input states and the system runs
through the rules to suggest a most useful scenario of action.
Backward chaining: you input goals, and the system tells you
the states you need to achieve to get there.
Don’t have to use Fuzzy Sets or Bayesian probabilities, but
often do.
How do we have confidence in our reasoning?
Expert systems may allow you to assess how confident you
are that a rule should be applied, though it isn’t always clear
how confidences add up.
For example:
“Man is mortal” (confidence 0.99)
“Socrates is a man” (confidence 0.5)
Final confidence that “Socrates is mortal” = ?
Dempster-Shafer theory allows us to deal with the confidence
in an event is happening (assigns a confidence to each
potential event totalling one for all possible events, and
allows us to assess multiple groups of evidence.)
Picking rules
However, ideally we want a cognitive framework to embed rulechoice within.
Something that embeds decision making within a wider model
of thought and existence.
Belief-Desire-Intention
We need some kind of reasoning architecture that allows the
agents to decide or be driven to decisions.
Most famous is the Belief-Desire-Intention model.
Beliefs – facts about the world (which can be rules).
Desires – things the agent wants to do / happen.
Intentions – actions the agent has chosen, usually from
a set of plans.
Driven by Events, which cascade a series of changes.
Decision making
BDI decisions are usually made by assuming a utility function.
This might include:
whichever desire is most important wins;
whichever plan achieves most desires;
whichever plan is most likely to succeed;
whichever plan does the above, after testing in
multiple situations;
whichever a community of agents decide on (eg
by voting)
Desires are goals, rather than more dynamic drivers.
The PECS model
Similar model is PECS – more sophisticated as it includes
internal drivers:
Physis – physical states
Emotional – emotional states
Cognitive – facts about the world
Social status – position within society etc.
On the basis of these, the agent plans and picks a behaviour.
Ultimately, though, these are decided between by a weighted
utility function.
Thinking for agents
Ultimately we have to trade off complexity of reasoning
against speed of processing.
On the one hand, behaviour developed by a GA/GP would be
dumb, but fast (which is why it is used to control agents in
games).
On the other, full cognitive architecture systems like Soar,
CLARION, and Adaptive Control of Thought—Rational (ACT-R)
are still not perfect, and take a great deal of time to set up.
Further reading
Michael Wooldridge (2009) “An Introduction
to MultiAgent Systems” Wiley (2nd Edition)
MAS architectures, BDI etc.
Stuart Russell and Peter Norvig (2010)
“Artificial Intelligence: A Modern Approach”
Prentice Hall (3rd Edition)
Neural nets, Language Processing, etc.