Transcript Chapter 3 Knowledge Acquisition 知識擷取 3.1 INTRODUCTION • The goal of knowledge acquisition（知識擷取） is to elicit expertise（專業知識） from domain experts（領 域專家）. Expertise Transfer Knowledge base Computerized Representation Expert 3.

Chapter 3
Knowledge Acquisition
知識擷取
3.1 INTRODUCTION
• The goal of knowledge acquisition（知識擷取） is to
elicit expertise（專業知識） from domain experts（領
域專家）.
Expertise Transfer
Knowledge base
Computerized
Representation
Expert
3. 知識擷取
G.J. Hwang
2
Advantages of Employing Knowledge
Acquisition（知識擷取） Systems：
1.
They does not only depend on the training cases（訓
練範例）.
2.
Real-time analysis is possible.
3.
Real-time consistency checking is possible.
4.
They can be integrated with KE tools.
5.
Knowledge bases（知識庫） can be automatically
generated.
3. 知識擷取
G.J. Hwang
3
REVIEWS OF PREVIOUS WORKS
Substantive Knowledge：
To identify current state
“Am I in danger of being attacked”
Strategic Knowledge：
To determine what to do next
“Climb to 30000 feet”
3. 知識擷取
G.J. Hwang
4
Knowledge Acquisition
（知識擷取） System
Substantive
Knowledge
Strategic
Knowledge
Classification
Decision making
Repertory Grid
Approach
Control
Planning
MORE
SALT
MOLE
ASK
Other
Approach
TEIRESIAS
ETS
NeoETS
KITTEN
AQUINAS
3. 知識擷取
KSSO
KNACK
KRITON
RuleCon
G.J. Hwang
5
 The Acquisition of Substantive Knowledge
• Repertory Grid（知識表格）-Oriented Methods：
Step 1. Elicit elements to be classified.
Step 2. Elicit constructs from experts.
Each time three elements are selected.
The expert is asked to give a construct
to distinguish one element from the other two.
Step 3. Rate the grid by filling a rating (1-5) to each entry.
Step 4. Generate implication graph.
3. 知識擷取
G.J. Hwang
6
Step 1：Elicit elements from experts.
Measles
German
Measles
Dangue
Fever
Chickenpox
Smallpox
Step 2：Elicit constructs from experts.
Measles
high
fever
red
purple
4
German
Measles
1
1
Dangue
Fever
5
5
4
headache
3. 知識擷取
G.J. Hwang
Chickenpox
5
1
4
Smallpox
1
2
no high
fever
not red
no purple
no
headache
7
Step 3：Rate each entry of the grid.
Measles
high
fever
red
purple
headache
4
5
4
1
German
Measles
1
1
1
2
Dangue
Fever
Chickenpox
Smallpox
5
5
4
4
4
5
1
4
2
4
1
2
no high
fever
not red
no purple
no
headache
Step 4：Generate the implication graph.
3. 知識擷取
headache
red
purple
high fever
G.J. Hwang
8
Rules generated from the grid：
First column：
IF
high_fever and red and purple and (not headache)
Then
Disease = Measles
CF = MIN (0.8,1.0,0.8,0.8)
= 0.8
Second column：
IF
(not high_fever) and (not red) and
(not purple) and (not headache)
Then
Disease = German Measles
3. 知識擷取
G.J. Hwang
9
Advantages of applying repertory-grids
（知識表格）
Easy to analyze the elicited knowledge：
1.
Similarity analysis of constructs.
2.
Similarity analysis of elements.
3.
Analysis of the relationships among constructs.
4.
Detection of missed elements.
5.
Detection of logical errors.
3. 知識擷取
G.J. Hwang
10
3.2 ELICTATION OF SUBSTANTIVE
KNOWLEDGE
 Knowledge Representation（知識表示法）
dog
4-legs
2-legs
no-legs
# of
legs
bird
5
1
1
1
5
1
1
1
5
not 4-legs
not 2-legs
has-legs
dog
bird
fish
4,2
2,2
0,2
A dog has 4 legs
3. 知識擷取
fish
being very sure
G.J. Hwang
11
 An acquisition table is a repertory grid（知識
表格） of multiple data types：
Boolean ：true or false
Single value：an integer, a real, or a symbol
Set of value：a set of integers, real numbers or symbols.
Range of values：a set of integers or real numbers.
‘X’：no relation.
‘U’：unknown or undecidable.
 Ratings：
2：very likely to be.
1：maybe.
3. 知識擷取
G.J. Hwang
12
3.3 Some Problems of Repertory Grids （知識表
格）
 Problem of Element Selection
E1
E2
E3
E4
E5
C1
1
5
5
4
2
C’1
C2
1
5
1
1
5
C’2
C3
1
5
1
2
2
C’3
C4
1
5
1
1
4
C’4
3. 知識擷取
G.J. Hwang
13
 Problem of Multi-Level Knowledge and
Acquirability
INPUT DATA
SUBGOAL
INPUT DATA
SUBGOAL
SUBGOAL
INPUT DATA
GOAL
3. 知識擷取
G.J. Hwang
14
• The Concept of Acquirability：
The value of a terminal attribute of a decision tree must either
be a constant or be acquirable from users.For example：
IF
(leaf-shape = scale) and
(class = Gymnosperm)
THEN
family = Cypress.
Class is not an acquirable attribute.
3. 知識擷取
G.J. Hwang
15
？
Leaf Shape
？
？
Class
Family
3. 知識擷取
G.J. Hwang
16
Domain basis and classification knowledge：
Diseases
Domain
basis
Other
diseases
Acute
Exanthemas
Classification
knowledge
Measles, German measles, Dangue fever,…
3. 知識擷取
G.J. Hwang
17
 Problem of Missing Embedded Meanings（隱含
知識）
• When a diagnostician expresses the features of catch cold
are
headache, feel tired, cough, sneeze,…,
he means “if a person catches cold, he may
have those features”
•
We usually represent the expertise as the following rules:
(Headache = yes) and (Feel_tired = yes) and
(cough = yes) and …,
--> Disease = Catch_cold
3. 知識擷取
G.J. Hwang
18
• The embedded meaning（隱含知識）of the
diagnostician
“if one or some features do not appear, it is still possible that
the patient catches cold.”
Is ignored.
3. 知識擷取
G.J. Hwang
19
3.4 EMCUD：A New Model for Eliciting
Knowledge Representation（知識表示法）：
Conventional Repertory grid（知識表格）or
Acquisition Table
+
Attribute Ordering Table （屬性序列表格） (AOT)
3. 知識擷取
G.J. Hwang
20
 Eliciting embedded meanings（隱含知識） by
constructing the Attribute Ordering Table（屬性
序列表格）
A1
A2
A3
Obj1
Obj2
Obj3
D
1
X
D
1
X
2
1
D
Obj4
Obj5
1
D
1
D
D
D
• Value in an AOT may be：
‘D’：The attribute dominate（主導權） the object.
‘X’：The attribute has no relation with the object.
an integer：The attribute is of some degree of importance to
the object.(A smaller integer means less important.)
3. 知識擷取
G.J. Hwang
21
An example of Repertory Grid（知識表格）：
A1
A2
A3
Obj1
Obj2
Obj3
Obj4
Obj5
{9,10,12},2
YES,1
X
20,2
NO,2
X
(13-16],2
YES,1
4.3,2
17,2
YES,2
2.1,2
3,2
NO,2
6.0,2
The rule generated from first column：
RULE3： (13<A116)(A2=YES)  (A3=4.3) → GOAL = Obj3
Where
F(confidence) = 1.0 if confidence = 2
= 0.8 if confidence = 1
and
Certainty Factor CF = MIN(F(2),F(1), (F(2)) = 0.8
3. 知識擷取
G.J. Hwang
22
An example of constructing AOT.
EMCUD：If A1  {9,10,12}, is it possible that GOAL =Obj1 ?
EXPERT：No. /*This implies that A1 dominates Obj1 and
AOT<Obj1,A1> = ‘D’ */
EMCUD：If A2  YES,is it possible that GOAL = Obj1?
EXPERT：Yes. /*A2 does not dominate Obj1 */
EMCUD：If A1 > 16 or A1  13, is it possible that GOAL = Obj3?
EXPERT：Yes. /* A1 does not dominate Obj3 */
EMCUD：If A2  YES, is it possible that GOAL = Obj3 ?
EXPERT：Yes. /* A2 does not dominate Obj3 */
EMCUD：If A3  4.3 , is it possible that GOAL = Obj3 ?
EXPERT：No. /* A3 does dominate Obj3 */
3. 知識擷取
G.J. Hwang
23
EMCUD：Please rank A1 and A2 in the order of importance to
Obj3 by choosing one of the following expressions：
1)A1 is more important that A2
2)A1 is less important that A2
3)A1 is as important as A2
EXPERT：1 /* A1 is more important to Obj3 than A2, hence
AOT < Obj3,A1> = 2 and AOT <Obj3,A2> = 1 */
A1
A2
A3
3. 知識擷取
Obj1
Obj2
Obj3
Obj4
Obj5
D
1
X
D
1
X
2
1
D
1
D
1
D
D
D
G.J. Hwang
24
Elicit Embedded Meanings（隱含知識）
From RULE3, the following embedded rules（隱含規則） will
Be generated by negating the predicates of A1 and A2：
RULE3,1：NOT(13<A116)(A2=YES)  (A3=4.3)
→ GOAL = Obj3
RULE3,2： (13<A116)NOT(A2=YES)  (A3=4.3)
→ GOAL = Obj3
RULE3,3：NOT(13<A116)NOT(A2=YES)  (A3=4.3)
→ GOAL = Obj3
3. 知識擷取
G.J. Hwang
25
Certainty Sequence(CS)：
Represents the drgree of certainty degradation.
CS(RULESij) = SUM(AOT<Obji,Ak>)
for each ak in the negated predicates of ruleij
For example：
CS(RULE3,3) = AOT < Obj3,A1 + AOT<Obj3,A2>
=2+1=3
The embedded rules（隱含規則） generated from RULE3：
RULE3,1：NOT(13<A116)(A2=YES)  (A3=4.3)
→ GOAL = Obj3
CS = 2
RULE3,2： (13<A116)NOT(A2=YES)  (A3=4.3)
→ GOAL = Obj3
CS = 1
RULE3,3：NOT(13<A116)NOT(A2=YES)  (A3=4.3)
→ GOAL = Obj3
3. 知識擷取
CS = 3
G.J. Hwang
26
Construct Constraint List
1.
2.
Sort the embedded rules according to the CS values：
RULES3,2
CS = 1
RULES3,1
CS = 2
RULES3,3
CS = 3
A prune-and-search algorithm：
EMCUD：Do you think RULE3,1 is acceptable?
Expert：Yes. /* then RULE3,2 is also accepted*/
EMCUD：Do you think RULE3,3 is acceptable?
Expert：No. /* then CS=3 is recorded in the constraint
list */
3. 知識擷取
G.J. Hwang
27
Calculate Certainty Factors（確定因子）
Confirm：1.0
Strongly support：0.8
Support：0.6
May support：0.4
CFij= Upper-Boundi- (Csij/MAX(Csi))  (Upper-Boundi – Lower-Boundi)
MAX(Csi)：maximum CS value of the embedded
rules generated from RULEi.
Upper-Boundi：certainty factor of embedded
Lower-Boundi：certainty factor of embedded
rule with MAX(Csi) /* The rule with least confidence*/
3. 知識擷取
G.J. Hwang
28
An example of calculating certainty factors（確定因
子）
For the embedded rules（隱含規則） from RULE3：
1. Upper – Bound = CF(RULE3) = 0.8
2. Since RULE3 is not accepted, the embedded rule with MAX(CS)
is RULE3,1：
EMCUD：If RULE3 strongly supports GOAL = Obj3 ,
what about RULE3,1 ?
Expert：1. /*The Lower-Bound = 0.6*/
CF3,1 = 0.8 – (2/2) * (0.8 – 0.6) = 0.6
CF3,2 = 0.8 – (1/2) * (0.8 – 0.6) = 0.7
3. 知識擷取
G.J. Hwang
29
• The process of eliciting embedded meanings（隱
含知識）：
repertory grid
original rules
possible
embedded rules
eliciting
embedded
rules
thresholding
Attribute-Ordering
Table
Constraint List
accepted
embedded rules
mapping
mapping function
certainty factors
of
the embedded rules
3. 知識擷取
G.J. Hwang
30
ACQUISITION TABLE
肺
咳嗽
疲倦
頭痛
炎
YES,2
YES,2
YES,1
AOT
肺
咳嗽
疲倦
頭痛
3. 知識擷取
炎
D
2
1
G.J. Hwang
31
Conventional Repertory Grids（知識表格）：
IF (咳嗽=YES)&(疲倦=YES)&(頭痛=YES)
THEN DISEASE=肺炎
CF=0.8
EMCUD：
IF (咳嗽=YES)&(疲倦<>YES)&(頭痛=YES)
THEN DISEASE=肺炎
CF=0.67
IF (咳嗽=YES)&(疲倦=YES)&(頭痛<>YES)
THEN DISEASE=肺炎
CF=0.73
IF (咳嗽=YES)&(疲倦<>YES)&(頭痛<>YES)
THEN DISEASE=肺炎
3. 知識擷取
CF=0.6
G.J. Hwang
32
 OBJECT CHAIN：A METHOD FOR
questions selection：
• For the grid with 50 elements (or objects), there are 19600
possible choices of questions to elicit constructs (or attributes).
• Initial repertory grid（知識表格） and the object chains：
OBJECT CHAIN
Obj1 --> 2,3,4,5
Obj2 --> 1,3,4,5
Obj1
Obj2 Obj3
Obj4
Obj5
Obj3 --> 1,2,4,5
Obj4 --> 1,2,3,5
Obj5 --> 1,2,3,4
3. 知識擷取
G.J. Hwang
33
• The expert gives attribute P1 to distinguish Obj1 and
Obj2 from Obj3
OBJECT CHAIN
Obj1
Obj1 -- > 2,5
Obj2 -- > 1,5
P1
T
Obj2 Obj3
T
F
Obj4
F
Obj5
T
Obj3 -- > 4
Obj4 -- > 3
Obj5 -- > 1,2
3. 知識擷取
G.J. Hwang
34
• The expert gives attribute P2 to distinguish Obj2 and
Obj5 from Obj1
OBJECT CHAIN
Obj1
Obj1 -- > NULL
Obj2 -- > 5
Obj3 -- > NULL
P1
P2
T
T
Obj2 Obj3
T
F
F
T
Obj4
F
F
Obj5
T
F
Obj4 -- > NULL
Obj5 -- > 2
3. 知識擷取
G.J. Hwang
35
• The expert gives attribute P3 to distinguish Obj2
from Obj5
OBJECT CHAIN
Obj1
Obj1 -- > NULL
Obj2 -- > NULL
Obj3 -- > NULL
Obj4 -- > NULL
Obj5 -- > NULL
3. 知識擷取
P1
P2
P3
T
T
F
G.J. Hwang
Obj2 Obj3
T
F
T
T
T
T
Obj4
F
F
F
Obj5
T
F
F
36
• Advantages：
1.
Fewer questions are asked(log2n to n-1 questions).
2.
All of the objects are classified.
3.
Every question matches the current requirement of
classifying objects.
•
Disadvantages：
1.
It may force the expert to think a specific direction.
2.
Some important attributes may be ignored.
3. 知識擷取
G.J. Hwang
37
 Eliciting hierarchy of grids：
• For the expert system（專家系統） of classifying families
of plants
Goal is FAMILY (科)
Cypress
Pine
Bald Cypress Magnolia
scale
needle
needle
scale
Leaf shape
X {random,cvenline} evenline
X
Needle pat.
Class (綱) Gymnosperm Gymnosperm Gymnosperm Magnolia
Silver band
X
T
F
X
3. 知識擷取
G.J. Hwang
38
• Since class is not acquirable, it becomes the goal of a
new grid.
Goal is CLASS
Gymnosperm Magnolia
Type (種)
Flate
3. 知識擷取
Tree
F
Herb
T
G.J. Hwang
Angiosperm
Tree
T
39
• Since class is not acquirable, it becomes the goal of a
new grid.
Goal is TYPE
stem
position
one trunk
3. 知識擷取
Herb
Vine
Tree
Shrub
green
X
F
woody
creeping
T
woody
upright
T
woody
upright
F
G.J. Hwang
40
Decision tree of the hierarchy of grids：
FAMILY OF PLANT
LEAF SHAPE
NIDDLE PATTERN
CLASS
TYPE
STEAM
3. 知識擷取
POSITION
G.J. Hwang
FLATE
ONE TRUNK
41
3.5 An Application and Performance Evaluation
of EMCUD
 Application Domain：
Diagnosis of Acute Exanthema
 Hardware：
Personal Computer
 Software：
Personal Consultant Easy
3. 知識擷取
G.J. Hwang
42
case number
1
2 3 4 5 6
7
8 9 10 11 12 13
physician
12
3
2
1
14
2
6
5
5
3
1
old prototype
12 X X X X
X
14 X
6
X
X
3
1
new prototype
12
1
14
6
5
5
3
1
16 17
18 19 20 21 22
23
24
25
case number
3
14 15
3
3
1
1
2
2
physician
6
6
12
5
8
9
14 13
4
1
2
14
old prototype
X
X
12
5
X
9
14 13
4
1
2
14
new prototype
6
6
12
5
8
9
14 13
4
1
2
14
The codes of diseases and their translations:
1-Measles
8 - Meningococcemia
2-German measles
9 - Rocky Mt. Spotted fever
3-Chickenpox
10 - Typhus fevers
4-Smallpox
11 – Infectious mononucleosis
5-Scarlet
12 – Enterovirus infections
6-Exanthem subitum
13 – Drug eruptions
7-Fifth disease
14 – Eczema herpeticum
Table 3.3：Testing results of the old and new prototypes.
3. 知識擷取
G.J. Hwang
43
3.6 Knowledge integration（知識整合） from
multiple experts
 To build a reliable expert system, the cooperation of
several experts is usually required.
 Difficulties：
• Synonyms of elements (possible solutions)
• Synonyms of traits (attributes to classify the solutions)
• Conflicts of ratings
3. 知識擷取
G.J. Hwang
44
Habitual domain
of Expert 1
Each expert has
his own way to
do some works.
Habitual domain
of Expert 2
Integrated Knowledge
Use more attributes to make choices
from more possible decisions
3. 知識擷取
G.J. Hwang
45
Expert 1
Expert 2
Expert N
Busy
Busy
Busy
Far away
Far away
Knowledge
Engineer
It is difficult to have all of the experts
work together
3. 知識擷取
G.J. Hwang
46
Expert 1
…
Expert 2
Repertory Grid 1
Repertory Grid 2
Expert N
Phase 1
interview
Repertory Grid N
The unions of element sets and construct sets
Common Repertory Grid
Expert 1
Expert 2
…
Phase 2
interview
Expert N
Eliminate some redundant vocabularies
Common Repertory Grid
3. 知識擷取
G.J. Hwang
47
Expert 1
Rated Common
Repertory Grid 1
…
Expert 2
Rated Common
Repertory Grid 2
Expert N
Phase 3
interview
Rated Common
Repertory Grid N
Knowledge Integration
Integrated Repertory Grid
Rule Generation
3. 知識擷取
G.J. Hwang
48
Repertory Grid 1
Repertory Grid 2
Repertory Grid N
The unions of element sets and construct sets
Common Repertory Grid
Expert 1
Expert 2
…
Phase 2
interview
Expert N
Eliminate some redundant vocabularies
Common Repertory Grid
Phase 3
interview
Expert 1
3. 知識擷取
Expert 2
Expert N
G.J. Hwang
49
Rated Common
Repertory Grid 1
Rated Common
Repertory Grid 2
Rated Common
Repertory Grid N
Knowledge Integration
Integrated Repertory Grid
Flat Repertory Grid
Generate AOT
AOT
Filled AOT 1
…
Filled AOT 2
Filled AOT N
Integration or AOT’s
Integrated AOT
Rule Generation
3. 知識擷取
G.J. Hwang
50
Eye pain
Expert 1
E1 E2 E3 E4 E5
Expert 2
E1 E2 E3 E4 E5
5
4
1
4
5
Eye pain
5
3
1
5
4
1
1
5
1
1
1
2
4
1
1
4
4
5
3
1
3
4
5
2
1
5
5
5
4
3
Pupil size
headache
Cornea
5
5
5
3
2
4
1
1
5
4
5
1
1
5
4
4
1
1
5
5
4
1
1
4
5
5
1
1
5
4
5
1
1
5
5
1
4
5
1
1
1
3
4
1
1
5
2
2
5
5
5
2
1
5
5
Both Side 5
1
4
1
1
5
1
3
1
1
Pupil size
headache
Cornea
Inflame of Eye
Tears
Redness
Vision
Papillary light
response
Inflame of Eye
Tears
Redness
Vision
Papillary light
response
Both Side
Knowledge Integration
3. 知識擷取
G.J. Hwang
51
Eye pain
5
4
1
5
5
1
1
5
1
1
4
4
5
2
1
5
5
5
4
2
5
1
1
5
4
4
1
1
5
5
5
1
1
5
5
1
4
5
1
1
5
2
1
5
5
Both Side 5
1
4
1
1
Pupil size
headache
Cornea
Inflame of Eye
Tears
Redness
Vision
Papillary light
response
3. 知識擷取
Expert 3
E1 E2 E3 E4 E5
G.J. Hwang
52
Results of the first experiment
Differential Diagnosis for Common Causes of Inflamed Eyes.
60 test cases are used to evaluate the knowledge base from
Expert 1, the knowledge base from Expert 2, and the
integrated knowledge base.
Knowledge
base
Expert 1
Expert 2
Integrated
3. 知識擷取
Ratio of Correct Diagnosis
0.67
0.64
0.8
G.J. Hwang
53
Results of the first experiment
Differential Diagnosis for Common Causes of Inflamed Eyes.
336 test cases are used to evaluate the knowledge base from
Expert 1, the knowledge base from Expert 2, and the
integrated knowledge base.
Knowledge
base
Expert 1
Expert 2
Integrated
3. 知識擷取
Number of
Correct
Diagnosis
255
243
306
G.J. Hwang
Ratio of
Correct
Diagnosis
0.759
0.723
0.911
54
3.7 Machine Learning（機器學習）
Building computer programs able to construct new
knowledge or to improve already possessed knowledge
Application：
Expert Systems
Cognitive Simulation
Problem Solving
Control …
Example：
Perceptron [Rosenblatt, 1961]
Meta-Dendral [Bucmanan, Feigenbaum, Sridharan, 1972]
AM
[Lenat, 1976]
LEX…
[Mitchell, Utgoff, Banerji, 1983]
3. 知識擷取
G.J. Hwang
55
傳統專家架構系統
知識源
知識編輯介面
推理機置
知識庫
使用者介面
使用者
3. 知識擷取
G.J. Hwang
56
具歸納式機器學習能力之專家系統架構
知識源
範例編輯介面
範例庫
學習機置
推理機置
知識庫
使用者介面
使用者
3. 知識擷取
G.J. Hwang
57
Machine Learning
 Machine Learning  Central to A. I.
 Learning from training cases.
2 2
22 2
1
1 1
1 1
1
3 3
3 3
3
3 3
2 2
3
2
2
3. 知識擷取
G.J. Hwang
11
11 1 1
1 1
2
2 2
2 2
2 2 2
58
Taxonomy
[Michalski, 1983]
Learning
Rote
Learning
Learning
by Analog
Learning by
Instruction
Learning from
Observation and
Discovery
Learning from
Examples
3. 知識擷取
Learning by
Induction
G.J. Hwang
59
Classification
Learning
Strategies
Symbolic
Learning
Neural
Learning
Incremental
Learning
e.g.
Version
Space
Batch
Learning
e.g.
Perceptron
e.g.
ID3
e.g.
PRISM
3. 知識擷取
G.J. Hwang
60
Symbolic Learning
1.Attributes
3. Hypothesis
Space
Learning Unit
3. 知識擷取
2. Matching
Predicates
G.J. Hwang
4.Training
Instances
61
Review of some data-driven learning
strategies [T.M. Mitchell 1979]
1. Depth-first search
2. Specific-to-general breadth-first search
3. ID3
4. Version space
3. 知識擷取
G.J. Hwang
62
Problem Description
Description of instances:
an unorder pair of simple objects, characterized by
three attributes(size, color, shape)
Three instances:
{(Large,Red,Triangles)(Small,Blue,Circle)}
{(Large,Blue,Circle) (Small,Red,Triangle)}
{(Large,Blue,Triangle)(Small,Blue,Triangle)}
3. 知識擷取
G.J. Hwang
+
+
63
Depth-First Search
{(Large,Red,Triangle)
(Small,Blue,Circle)}
1.{(Large,Red,Triangle)
(Small,Blue,Circle)}
{(Large,Red,Triangle)
(Small,Blue,Circle)}
2.{(Large,Blue, Circle)
(Small,Red, Triangle)}
{(Large,?,?)
(Small,?,?)}
3.{(Large,Blue, Triangle))
(Small,Blue, Triangle)}
{(Large,Red,Triangle)
(Small,Blue,Circle)}
{(?,Red,Triangle)
(?,Blue,Circle)}
3. 知識擷取
G.J. Hwang
{(Large,?,?)
(Small,?,?)}
64
Disadvantages of Depth-First Search:
1. Needs backtracking
2. Needs additional cost of maintaining
consistence with past instances
3. 知識擷取
G.J. Hwang
65
Specific-to-general breadth-first search
1.{(Large,Red,Triangle)
(Small,Blue,Circle)}
{(Large,Red,Triangle)
(Small,Blue,Circle)}
2.{(Large,Blue, Circle)
(Small,Red, Triangle)}
{(Large,Red,Triangle)
(Small,Blue,Circle)}
{(?,Red,Triangle)
(?,Blue,Circle)}
3.{(Large,Blue, Triangle))
(Small,Blue, Triangle)}
{(Large,Red,Triangle)
(Small,Blue,Circle)}
{(?,Red,Triangle)
(?,Blue,Circle)}
3. 知識擷取
{(Large,?,?)
(Small,?,?)}
G.J. Hwang
{(Large,?,?)
(Small,?,?)}
66
Disadvantages of breadth-first search:
Needs to check past negative instances to assure
that the revised generalization is not overly general
3. 知識擷取
G.J. Hwang
67
Problem Description
1. A set of attributes = { A : the age of the patient,年齡
B : spectacle prescription,視力
C : astigmatic, 亂視
D : tear production rate 淚量}
2. Matching Predicates:
A= { A1 : young , 青年, A2 : pre-presbyopic, 中年, A3 : presbyopic 老年}
B= { B1 : myope, 近視, B2 : hypermetrope 遠視}
C= { C1 : no無, C2 : yes有}
D= { D1 : reduced ,較少, D2 : normal正常 }
3. A set of classes (Hypothesis Space)
= { DEC1 : hard contact lenses, 硬式隱形眼鏡
DEC2 : soft contact lenses, 軟式隱形眼鏡
DEC3 : no 不適合戴隱形眼鏡 }
3. 知識擷取
G.J. Hwang
68
4. Training Instances 訓練範例
範 A B C D Dec
範 A B C D Dec
範 A B C D Dec
例
例
例
1 A1 B1 C1 D1 Dec3
9 A2 B1 C1 D1 Dec3
17 A3 B1 C1 D1 Dec3
2 A1 B1 C1 D2 Dec2
10 A2 B1 C1 D2 Dec2
18 A3 B1 C1 D2 Dec2
3 A1 B1 C2 D1 Dec3
11 A2 B1 C2 D1 Dec3
19 A3 B1 C2 D1 Dec3
4 A1 B1 C2 D2 Dec1
12 A2 B1 C2 D2 Dec1
20 A3 B1 C2 D2 Dec1
5 A1 B2 C1 D1 Dec3
13 A2 B2 C1 D1 Dec3
21 A3 B2 C1 D1 Dec3
6 A1 B2 C1 D2 Dec2
14 A2 B2 C1 D2 Dec2
22 A3 B2 C1 D2 Dec2
7 A1 B2 C2 D1 Dec3
15 A2 B2 C2 D1 Dec3
23 A3 B2 C2 D1 Dec3
8 A1 B2 C2 D2 Dec1
16 A2 B2 C2 D2 Dec1
24 A3 B2 C2 D2 Dec1
3. 知識擷取
G.J. Hwang
69
基本的歸納決策樹學習演算法推導決策樹
A= A1
B=B1
C=C1
C=C2
B=B2
C=C1
C=C2
訓練
範例
A= A2
B=B1
C=C1
C=C2
B=B2
C=C1
D=D1
D=D2
D=D1
D=D2
D=D1
D=D2
D=D1
D=D2
[Dec3]
[Dec2]
[Dec3]
[Dec1]
[Dec3]
[Dec2]
[Dec3]
[Dec1]
D=D1
D=D2
D=D1
D=D2
D=D1
D=D2
[Dec3]
[Dec2]
[Dec3]
[Dec1]
[Dec3]
[Dec2]
[Dec3]
C=C2
A= A3
B=B1
B=B2
3. 知識擷取
C=C1
C=C2
C=C1
C=C2
G.J. Hwang
D=D1
D=D2
D=D1
D=D2
[Dec3]
[Dec3]
[Dec1]
[Dec3]
[Dec2]
[Dec3]
70
ID3
Assumption:
C
p個正例
n個反例
1. C中所含正反例的個數可以反映出
一般正反例的比例。
p
n
:
 p:n
正：反 =
pn pn
2. 表達 此資訊的最小bit期望值(所含資訊量)為
I ( P, N )  
3. 知識擷取
p
p
n
n
log2

log2
pn
pn pn
pn
G.J. Hwang
71
以 Data Compression 的觀點：
越少出現的 information 用越多 bits 表示
越常出現的 information 用越少 bits 表示

可用最少的 memory 表達最多的 information

若出現機率是
p
則用
pn
n
則用
pn
1
p
p n
1
n
p n
 K memory表示
K
memory表示
所以P個正例，n個反例期望的 bits 數為
p
K
n
K
log 2 p 
log 2 n
pn
pn
p n
p n

p
p
n
n
log 2

log 2
 K'
pn
pn pn
pn
bits 數和 information 量成正比
3. 知識擷取
I(P,n)
G.J. Hwang
72
C
P+
n-
A1, A2, A3, ......
(Attributes)
···
C1
P1 +
n1 E( A ) 
C2
v

i 1
C3 P3 +
P2 +
n3 n2 -
Cr Pr +
···
nr -
pi  n i
I( p i , n i )
pn
gain (A) = I(P, n) - E(A)
A所含的
information
原來的
information量
用A分類之後剩餘的
information量
取 Gain(A)最大的attribute來分類
3. 知識擷取
G.J. Hwang
73
An Alternative point of view :
p
p
n
n
I( p, n)  
log2

log2
pn
pn pn
pn
Entropy (亂度)
C
+
+ + +
C
A1
+
+
+
A2
+ +
－－
－－
++
++
+ +
亂度低
亂度高
3. 知識擷取
+
G.J. Hwang
74
ID3歸納決策樹學習演算法推導決策樹
訓練
範例
D=D1
D=D2
C=C1
C=C2
3. 知識擷取
G.J. Hwang
A=A1
A=A2
A=A3
B=B1
B=B2
B=B1
B=B2
A=A1
A=A2
A=A3
[Dec3]
[Dec2]
[Dec2]
[Dec3]
[Dec2]
[Dec1]
[Dec1]
[Dec3]
[Dec3]
75
歸納法決策樹可以轉換成決策法則，例如在上圖的第
二個分枝可以表示成如下之法則:
若
患者的淚量=正常
AND C=C1
且
患者有亂視=有
AND A=A1
且
患者的年齡=青年
THEN Dec=Dec2
則
隱形眼鏡決策=軟式隱形眼鏡
IF
D=D2
3. 知識擷取
意為
G.J. Hwang
76
Problem Description
assume only one attribute exists
transc
trig
sin
cos
explog
tan
ln
exp
Instance space: terminal nodes,Hypothesis space: all nodes
Predicates: predecessor-successor relations
Positive Training Instances：sin and cos
Negative Training Instance ：ln
→ Concept：trig
3. 知識擷取
G.J. Hwang
77
Terminology
• An Instance Space：
a set of instances which can be legally
described by a given instance language
．Attribute-based Instance Space
．Structured Instance Space
• A Hypothesis Space：
a set of hypotheses which can be legally described
by a generalization language
5 kinds of expressions
Conjunctive Form
Disjunctive Form
e.g.
Color=red and shape=convex C1 or C2 or C3…
(most prevalent form)
conjunctive form
3. 知識擷取
G.J. Hwang
78
Terminology
 Predicates：
required for testing whether a given instance is contained in
the instance set corresponding to a given hypothesis
• Powerful basis for organizing a search
• Two partial ordering relations exist：
 A is more specific（特殊） than B：
B is more general（泛化） than A：
If each instance contained in A is also
contained in B
3. 知識擷取
G.J. Hwang
79
Version Space
 Incremental Learning（逐漸式學習演算法）
 For Conjunctive Hypothesis
 Idea：
Be represented by two sets hypotheses
more general
-
G
+
S
more specific
 S：the most specific set（最特殊規則集） consistent
with the training instances
G：the most general set（最泛化規則集） consistent
with the training instances
3. 知識擷取
G.J. Hwang
80
Example of Version Space
transc
trig
sin
explog
cos
tan
ln
exp
( sin + )
S：sin
G：transc
( ln - )
S：sin
G：trig
( cos + )
S：trig
G：trig
Concept：
Lemma： a

S,
trig
b

G,
a is more specific than b
3. 知識擷取
G.J. Hwang
81
1.{(Large,Red,Triangle)
(Small,Blue,Circle)}
S： {(Large,Red,Triangle)
(Small,Blue,Circle)}
G： {(?,?,?)
(?,?,?) }
2.{(Large,Blue, Circle)
(Small,Red, Triangle)
S： {(Large,?,?) {(?,Red,Triangle)
(Small,?,?)} (?,Blue,Circle)}
G： {(?,?,?)
(?,?,?) }
3.{(Large,Blue, Triangle))
(Small,Blue, Triangle)}
S： {(?,Red,Triangle)
(?,Blue,Circle) }
G： {(?,?,Circle)
(?,?,?) }
3. 知識擷取
G.J. Hwang
{(?,Red,?)
(?,?,?) }
82
Check contradiction between S and G
• Step1: Take a generalization s in S and a
generalization g in G. Check s with g, if g is not
more general than s , mark s and g.
• Step2: Repeat step1 until each in S and G are
processed.
• Step3: Discard those generalizations in S with |G|
marks and those in G with |S| marks.
3. 知識擷取
G.J. Hwang
83
Advantage of Version Space:
Needs not check past instances---the reason to
apply it in our parallel learning algorithm
3. 知識擷取
G.J. Hwang
84
Exercise
1. 試以動物分類為例，建立一個Repertory
Grid（知識表格）及產生對應的推論規
則。
2. 分析產生的動物分類推論規則中是否有
遺漏的Embedded Meanings（隱含知識）。
3. 知識擷取
G.J. Hwang
85
3. Use Depth-First-Search to learn from the
following training cases.
COLOR
black
brown
brown
black
brown
black
brown
brown
brown
black
black
black
3. 知識擷取
SIZE
large
large
medium
small
medium
large
small
small
large
medium
medium
small
COAT
shaggy
smooth
shaggy
shaggy
smooth
smooth
shaggy
smooth
shaggy
shaggy
smooth
smooth
G.J. Hwang
COLOR
+
+
+
+
+
+
86
4. 已知有一分析型領域問題的屬性與決策如下：
屬性 A = {A1, A2, A3} 屬性 B = {B1, B2}
屬性 C = {C1, C2}
屬性 D = {D1, D2, D3}
決策 Dec = {Dec1, Dec2, Dec3}
(1) 用屬性 A, B, C, D 作順序所產生之決策樹。
(2) 用 ID3 演算法時所產生之決策樹。
範
例
1
2
3
4
5
6
7
8
9
10
11
12
A B C D Dec
A1
A1
A1
A1
A1
A1
A1
A1
A1
A1
A1
A1
B1 C1 D1 Dec1
B1 C1 D2 Dec1
B1 C1 D3 Dec1
B1 C2 D1 Dec1
B1 C2 D2 Dec1
B1 C2 D3 Dec1
B2 C1 D1 Dec2
B2 C1 D2 Dec2
B2 C1 D3 Dec2
B2 C2 D1 Dec2
B2 C2 D2 Dec3
B2 C2 D3 Dec2
3. 知識擷取
範 A B C D Dec
例
13 A2 B1 C1 D1 Dec1
14 A2 B1 C1 D2 Dec1
15 A2 B1 C1 D3 Dec1
16 A2 B1 C2 D1 Dec1
17 A2 B1 C2 D2 Dec1
18 A2 B1 C2 D3 Dec1
19 A2 B2 C1 D1 Dec2
20 A2 B2 C1 D2 Dec2
21 A2 B2 C1 D3 Dec2
22 A2 B2 C2 D1 Dec2
23 A2 B2 C2 D2 Dec3
24 A2 B2 C2 D3 Dec2
G.J. Hwang
範
例
25
26
27
28
29
30
31
32
33
34
35
36
A B C D Dec
A3
A3
A3
A3
A3
A3
A3
A3
A3
A3
A3
A3
B1
B1
B1
B1
B1
B1
B2
B2
B2
B2
B2
B2
C1 D1 Dec1
C1 D2 Dec1
C1 D3 Dec1
C2 D1 Dec1
C2 D2 Dec1
C2 D3 Dec1
C1 D1 Dec2
C1 D2 Dec2
C1 D3 Dec3
C2 D1 Dec2
C2 D2 Dec3
C2 D3 Dec3
87