Transcript Document 7366813

CPE/CSC 481:
Knowledge-Based Systems
Dr. Franz J. Kurfess
Computer Science Department
Cal Poly
© 2002 Franz J. Kurfess
Introduction 1
Course Overview
 Introduction
 Knowledge

Semantic Nets, Frames, Logic
 Reasoning

with Uncertainty
Probability, Bayesian Decision
Making
 Expert

and Inference
Predicate Logic, Inference
Methods, Resolution
 Reasoning

Representation
System Design
 CLIPS

Overview
Concepts, Notation, Usage
 Pattern

Matching
Variables, Functions,
Expressions, Constraints
 Expert
System
Implementation

Salience, Rete Algorithm
 Expert
System Examples
 Conclusions and Outlook
ES Life Cycle
© 2002 Franz J. Kurfess
Introduction 2
Overview Reasoning and Uncertainty
 Motivation
 Dempster-Shafer

 Objectives
 Sources
of Uncertainty and
Inexactness in Reasoning



Incorrect and Incomplete
Knowledge
Ambiguities
Belief and Disbelief
 Probability

Certainty Factors
 \Approximate

Theory
Reasoning
Fuzzy Logic
 Important
Concepts and
Terms
 Chapter Summary
Theory
Bayesian Networks
© 2002 Franz J. Kurfess
Introduction 3
Logistics


Introductions
Course Materials


textbooks (see below)
lecture notes



handouts
Web page





PowerPoint Slides will be available on my Web page
http://www.csc.calpoly.edu/~fkurfess
Term Project
Lab and Homework Assignments
Exams
Grading
© 2002 Franz J. Kurfess
Introduction 4
Bridge-In
© 2002 Franz J. Kurfess
Introduction 5
Pre-Test
© 2002 Franz J. Kurfess
Introduction 6
Motivation
© 2002 Franz J. Kurfess
Introduction 7
Objectives
© 2002 Franz J. Kurfess
Introduction 8
Introductions
 reasoning
under uncertainty and with inexact knowledge
 heuristics


ways to mimic heuristic knowledge processing
methods used by experts
 empirical


associations
experiential reasoning
based on limited observations
 probabilities


objective (frequency counting)
subjective (human experience )
 reproducibility

will observations deliver the same results when repeated
© 2002 Franz J. Kurfess
Introduction 10
Dealing with Uncertainty
 expressiveness


can concepts used by humans be represented adequately?
can the confidence of experts in their decisions be expressed?
 comprehensibility


representation of uncertainty
utilization in reasoning methods
 correctness



probabilities
relevance ranking
long inference chains
 computational

complexity
feasibility of calculations for practical purposes
© 2002 Franz J. Kurfess
Introduction 11
Sources of Uncertainty
 data

missing data, unreliable, ambiguous, imprecise representation,
inconsistent, subjective, derived from defaults, …
 expert


inconsistency between different experts
plausibility


“best guess” of experts
quality



knowledge
causal knowledge
 deep understanding
statistical associations
 observations
scope

only current domain?
© 2002 Franz J. Kurfess
Introduction 12
Sources of Uncertainty (cont.)
 knowledge
representation
 restricted
model of the real system
 limited expressiveness of the representation mechanism
 inference
process
 deductive

the derived result is formally correct, but wrong in the real system
 inductive

new conclusions are not well-founded
 unsound
© 2002 Franz J. Kurfess
reasoning methods
Introduction 13
Uncertainty in Individual Rules
 individual
rules
 errors



domain errors
representation errors
inappropriate application of the rules
 likelihood



of evidence
for each premise
for the conclusion
combination of evidence from multiple premises
© 2002 Franz J. Kurfess
Introduction 14
Uncertainty and Multiple Rules
 conflict

resolution
if multiple rules are applicable, which one is selected
 explicit priorities, provided by domain experts
 implicit priorities derived from rule properties
» specificity of patterns, ordering of patterns creation time of rules, most recent usage, …
 compatibility


contradictions between rules
subsumption




one rule is a more general version of another one
redundancy
missing rules
data fusion

integration of data from multiple sources
© 2002 Franz J. Kurfess
Introduction 15
Basics of Probability Theory
 mathematical

sample space set
X = {x1, x2, …, xn}



approach for processing uncertain information
collection of all possible events
can be discrete or continuous
probability number P(xi)
likelihood of an event xi to occur





non-negative value in [0,1]
total probability of the sample space is 1
for mutually exclusive events, the probability for at least one of them is the
sum of their individual probabilities
experimental probability
 based on the frequency of events
subjective probability
 based on expert assessment
© 2002 Franz J. Kurfess
Introduction 16
Compound Probabilities
 describes
 do
independent events
not affect each other in any way
 joint
probability of two independent events A and B
P(A  B) = n(A  B) / n(s) = P(A) * P (B)
where n(S) is the number of elements in S
 union
probability of two independent events A and B
P(A  B) = P(A) + P(B) - P(A  B)
=P(A) + P(B) - P(A) * P (B)

where n(S) is the number of elements in S
© 2002 Franz J. Kurfess
Introduction 17
Conditional Probabilities
 describes
 affect
dependent events
each other in some way
 conditional
probability
of event a given that event B has already occurred
P(A|B) = P(A  B) / P(B)
© 2002 Franz J. Kurfess
Introduction 18
Advantages and Problems of
Probabilities
 advantages
 formal
foundation
 reflection of reality (a posteriori)
 problems
 may

be inappropriate
the future is not always similar to the past
 inexact

or incorrect
especially for subjective probabilities
 knowledge
© 2002 Franz J. Kurfess
may be represented implicitly
Introduction 19
Bayesian Approaches
 derive
the probability of a cause given a symptom
 has gained importance recently due to advances in
efficiency
 more
computational power available
 better methods
 especially
useful in diagnostic systems
 medicine,
 inverse
computer help systems
or a posteriori probability
 inverse
to conditional probability of an earlier event given
that a later one occurred
© 2002 Franz J. Kurfess
Introduction 20
Bayes’ Rule for Single Event
 single
hypothesis H, single event E
P(H|E) = (P(E|H) * P(H)) / P(E)
or
 P(H|E) = (P(E|H) * P(H) /
(P(E|H) * P(H) + P(E|H) * P(H) )
© 2002 Franz J. Kurfess
Introduction 21
Bayes’ Rule for Multiple Events
 multiple
hypotheses Hi, multiple events E1, …, Ei,
…, En
P(Hi|E1, E2, …, En) = (P(E1, E2, …, En|Hi) * P(Hi)) /
P(E1, E2, …, En)
or
 P(Hi|E1, E2, …, En) = (P(E1|Hi) * P(E2|Hi) * …*
P(En|Hi) * P(Hi)) /
k P(E1|Hk) * P(E2|Hk) * … * P(En|Hk)
* P(Hk)
with independent pieces of evidence Ei
© 2002 Franz J. Kurfess
Introduction 22
Advantages and Problems of
Bayesian Reasoning
 advantages


sound theoretical foundation
well-defined semantics for decision making
 problems

requires large amounts of probability data






sufficient sample sizes
subjective evidence may not be reliable
independence of evidences assumption often not valid
relationship between hypothesis and evidence is reduced to a number
explanations for the user difficult
high computational overhead
© 2002 Franz J. Kurfess
Introduction 23
Dempster-Shafer Theory
 mathematical
theory of evidence
 notations
 frame

power set of the set of possible conclusions
 mass


of discernment FD
probability function m
assigns a value from [0,1] to every item in the frame of discernment
mass probability m(A)

portion of the total mass probability that is assigned to an element A
of FD
© 2002 Franz J. Kurfess
Introduction 24
Belief and Certainty
 belief
Bel(A) in a subset A
 sum
of the mass probabilities of all the proper subsets of A
 likelihood that one of its members is the conclusion
 plausibility
 maximum
 certainty
Pl(A)
belief of A
Cer(A)
 interval
[Bel(A), Pl(A)]
 expresses the range of belief
© 2002 Franz J. Kurfess
Introduction 25
Combination of Mass Probabilities
 m2 (C) =  X  Y=C m1(X) * m2(Y) /
1- X  Y=C m1(X) * m2(Y)
where X, Y are hypothesis subsets and C is their
intersection
 m1
© 2002 Franz J. Kurfess
Introduction 26
Advantages and Problems of
Dempster-Shafer
 advantages
 clear,
rigorous foundation
 ability oto express confidence through intervals

certainty about certainty
 problems
 non-intuitive
determination of mass probability
 very high computational overhead
 may produce counterintuitive results due to normalization
 usability somewhat unclear
© 2002 Franz J. Kurfess
Introduction 27
Certainty Factors
 shares
some foundations with Dempster-Shafer
theory, but more practical
 denotes the belief in a hypothesis H given that some
pieces of evidence are observed
 no statements about the belief is no evidence is
present
 in
contrast to Bayes’ method
© 2002 Franz J. Kurfess
Introduction 28
Belief and Disbelief
 measure
of belief
 degree
to which hypothesis H is supported by evidence E
 MB(H,E) = 1 IF P(H) =1
(P(H|E) - P(H)) / (1- P(H)) otherwise
 measure
of disbelief
 degree
to which doubt in hypothesis H is supported by
evidence E
 MB(H,E) = 1 IF P(H) =0
(P(H) - P(H|E)) / P(H)) otherwise
© 2002 Franz J. Kurfess
Introduction 29
Certainty Factor
 certainty
factor CF
 ranges
between -1 (denial of the hypothesis H) and 1
(confirmation of H)
 CF
= (MB - MD) / (1 - min (MD, MB))
 combining antecedent evidence
 use



of premises with less than absolute confidence
E1  E2 = min(CF(H, E1), CF(H, E2))
E1  E2 = max(CF(H, E1), CF(H, E2))
E =  CF(H, E)
© 2002 Franz J. Kurfess
Introduction 30
Combining Certainty Factors
 certainty
factors that support the same conclusion
 several rules can lead to the same conclusion
 applied incrementally as new evidence becomes
available
 Cfrev(CFold,
CFnew) =
 CFold
+ CFnew(1 - CFold) if both > 0
 CFold + CFnew(1 + CFold) if both < 0
 CFold + CFnew / (1 - min(|CFold|, |CFnew|)) if one < 0
© 2002 Franz J. Kurfess
Introduction 31
Advantages and Problems of
Certainty Factors
 Advantages


simple implementation
reasonable modeling of human experts’ belief



expression of belief and disbelief
successful applications for certain problem classes
evidence relatively easy to gather

no statistical base required
 Problems

partially ad hoc approach

theoretical foundation through Dempster-Shafer theory was developed later
combination of non-independent evidence unsatisfactory
 new knowledge may require changes in the certainty factors of existing
knowledge
 certainty factors can become the opposite of conditional probabilities
for certain cases
 not suitable for long inference chains
© 2002 Franz J. Kurfess

Introduction 32
Fuzzy Logic
 approach
to a formal treatment of uncertainty
 relies on quantifying and reasoning through natural
language
 uses
linguistic variables to describe concepts with vague
values

tall, large, small, heavy, ...
© 2002 Franz J. Kurfess
Introduction 33
Get Fuzzy
© 2002 Franz J. Kurfess
Introduction 34
Fuzzy Set
 categorization
 described

of elements xi into a set S
through a membership function m(s)
associates each element xi with a degree of membership in S
 possibility
measure Poss{xS}
 degree
to which an individual element x is a potential
member in the fuzzy set S
 possibility refers to allowed values
 probability expresses expected occurrences of events
 combination of multiple premises


Poss(A  B) = min(Poss(A),Poss(B))
Poss(A  B) = max(Poss(A),Poss(B))
© 2002 Franz J. Kurfess
Introduction 35
Fuzzy Set Example
membership
short
1
tall
medium
0.5
0
0
© 2002 Franz J. Kurfess
50
100
150
200
height
250 (cm)
Introduction 36
Fuzzy vs. Crisp Set
membership
short
1
tall
medium
0.5
0
0
© 2002 Franz J. Kurfess
50
100
150
200
height
250 (cm)
Introduction 37
Fuzzy Inference Methods
 how
to combine evidence across rules
 Poss(B|A)

= min(1, (1 - Poss(A)+ Poss(B)))
implication according to Max-Min inference
 also
Max-Product inference and other rules
 formal foundation through Lukasiewicz logic

extension of binary logic to infinite-valued logic
© 2002 Franz J. Kurfess
Introduction 38
Example Fuzzy Reasoning
© 2002 Franz J. Kurfess
Introduction 39
Advantages and Problems of Fuzzy
Logic
 advantages
 general
theory of uncertainty
 wide applicability, many practical applications
 natural use of vague and imprecise concepts

helpful for commonsense reasoning, explanation
 problems
 membership
functions can be difficult to find
 multiple ways for combining evidence
 problems with long inference chains
© 2002 Franz J. Kurfess
Introduction 40
Post-Test
© 2002 Franz J. Kurfess
Introduction 41
Use of References
 [Giarratano
& Riley 1998]
 [Russell & Norvig 1995]
 [Jackson 1999]
 [Durkin 1994]
[Giarratano & Riley 1998]
© 2002 Franz J. Kurfess
Introduction 43
Important Concepts and Terms













agent
automated reasoning
belief network
cognitive science
computer science
hidden Markov model
intelligence
knowledge representation
linguistics
Lisp
logic
machine learning
microworlds
© 2002 Franz J. Kurfess







natural language processing
neural network
predicate logic
propositional logic
rational agent
rationality
Turing test
Introduction 44
Summary Chapter-Topic
© 2002 Franz J. Kurfess
Introduction 45
© 2002 Franz J. Kurfess
Introduction 46