Transcript ICT619 Intelligent Systems - Topic 7

ICT619 Intelligent
Systems
Topic 7: Case Based
Reasoning
Case Based Reasoning




Introduction
How CBR works
Business Applications of CBR
CBR Development methodology and
Tools
 Advantages of CBR systems
 Case Study
ICT619
2
What is Case Based Reasoning?
 An intelligent systems methodology based on using
stored past problem solving experience to solve a
current problem
 Similarity with human problem solving - analogical
reasoning or memory-based reasoning
 CBR draws on similarities and differences between a
given problem and a similar problem solved in the past
 CBR learns from experience
 Adds a solved problem to the case base for use in
future
 Difference with ANN learning - does not generalise
 Difference with rule-based systems - cases are not
rules
ICT619
3
Case Based Reasoning Terms
 A case denotes a problem situation, experienced or
learned in the past and retained in a case base
 A case may also be an unsolved case – a problem to
be solved at present or in the future
 Note that some aspects of the law are like this - based
on a library of precedents, expressed as cases
ICT619
4
CBR History and Current Status
 1980s: Early work done by Roger Schank - Yale University
 Funding support from the US Defence Advanced Research
Projects Agency (DARPA)
 Started later in Europe - Germany most active
 1990: Commercial applications appear
 1998: Research activity in >35 institutions around the world
 15 reported CBR commercial tools
 Many applications in daily use
ICT619
5
How CBR works
A Case
Attributes
…
…
…
Solution
…
 The knowledge base of a CBR consists of cases units of experience consisting of problems and
solutions
 Each case in case base is defined in terms of its
attributes and the solution found for it
ICT619
6
Steps in the case-based
reasoning cycle:
1. Retrieve a case matching given problem
2. Adapt the matching case’s solution to produce desired
solution
3. Test and revise suggested solution
4. Retain confirmed solution by adding it to case base for
future use.
ICT619
7
Steps in the CBR Cycle
Problem
New
Case
RETRIEVE
Learned
Case
Case Base
RETAIN
Tested/
Repaired
Case
Previous
Cases
General
Knowledge
Retrieved
Case
New
Case
REUSE
Solved
Case
REVISE
Confirmed
Solution
Suggested
Solution
The CBR cycle (Aamodt 1994)
ICT619
8
A Simple Example of Problem
Solving Using CBR (Bergmann 1998)


Problem: Find the cause for a fault in a car and a
suggest a repair strategy
Case base consists of cases containing:
 A description of symptoms
 A description of the cause
 A description of the repair strategy



Each case describes one particular diagnostic
situation
It records several features and their specific values
found for that situation
Each case describes one particular situation
independently of any another
ICT619
9
A Simple Example (continued)
Case 1
Symptoms (Attribute: Value)
Fault : Front lights not turning on
Car : VW Golf 1.6L
Year : 1993
Battery voltage : 13.6 V
State of lights : OK
State of light switch : OK
Case 2
Symptoms (Attribute: Value)
Solution
Fault : Front lights not turning on
Diagnosis : Front light fuse blown
Car : Audi A6
Repair
: Replace front light fuse
Year : 1995
Battery voltage : 12.9 V
State of lights : Surface damaged
State of light switch : OK
Solution
Diagnosis :
Repair
:
Bulb defect
Replace front light
ICT619
10
A Simple Example (continued)
To solve a specific fault finding problem
 A case for it is built up without the solution part
 Attribute values are gathered by observing symptoms (eg,
engine not starting) and measuring values (eg battery
voltage = 6.3V)
 Not all attributes values may be known for a problem.
 In other problem domains the unsolved problem may have
all or part of the solution, but one or more of the attributes
missing.
Case
to be
solved
Symptoms (Attribute: Value)
Fault : Brake lights not turning on
Car : Audi 80
Year : 1989
Battery voltage : 12.6 V
State of lights : OK
ICT619
11
Similarity computation in simple example
 The degree of similarity is expressed using a number in
the range 0.0 (no similarity at all) to 1.0 (complete
similarity)
Symbolic Attribute: Fault
Front lights not turning on
0.8
Brake lights not turning on
Front lights not turning on
0.4
Engine not starting
Numeric Attribute: Battery Voltage

12.6 V
0.9
13.6 V
12.6 V
0.1
6.7 V
Attributes are given weights reflecting their significance
In this example:
weight 6 given for higher importance and
ICT619
weight 1 for lower importance
12
Adaptation of matching case
in example
 Once a matching case has been found its solution part is adapted
 The adaptation is based on
 Differences and similarities between the given problem and the
matched case (Case 1)
 How the differences should affect the solution
 One heavily weighted attribute is fault
 In the example, fault has values “Brake lights not turning on” and
“Front lights not turning on”
 Significant similarity in nature (light problem), but difference in
specifics (type of light)
 Differences and similarities in other attributes (eg, make and year
of manufacture) are not significant
 Adaptation done in matching case’s solution part “front lights” changed to “brake lights”.
ICT619
13
Adaptation of matching case
in example
Problem to be solved (Symptoms)
- Fault : Brake light not turning on
- Car : Audi 80
- Year : 1989
- Batt. Voltage : 12.6 V
- State of lights : OK
Important attribute: weight = 6
Less important attribute: weight = 1
0.8
0.4
0.6
0.9
1.0
Case 1 (Symptoms)
- Fault
: Front light not turning on
- Car
: VW Golf
- Year
: 1993
- Batt. Voltage : 13.6 V
- State of lights : OK
- State of light switch : OK
Solution
- Diagnosis : Front light fuse defect
- Repair
: Replace front light fuse
Similarity Computation by Weighted Averaging
Similarity (problem to be solved, case 1) = 1/20 * [6*0.8 = 1*0.4 + 1*0.6 + 6*0.9 +
6*1.0] = 0.86
ICT619
14
Adaptation of matching case
in example
Case 1 (Symptoms)
- Fault : Front light not turning on
-…
Solution
- Diagnosis : Front light fuse defect
- Repair
: Replace front light fuse
Problem to be solved (Symptoms)
- Fault : Brake light not turning on
- Car : Audi 80
- Year : 1989
- Batt. Voltage : 12.6 V
- State of lights : OK
Adapted Solution
- Diagnosis : Brake light fuse defect
- Repair
: Replace brake light fuse
ICT619
15
Reuse (by adaptation) of the Solution in
Case 1
Case 3 (Symptoms)
- Fault : Brake light not turning on
- Car : Audi 80
- Year : 1989
- Batt. Voltage : 12.6 V
- State of lights: OK
Solution
- Diagnosis
- Repair
: Brake light fuse defect
: Replace brake light fuse
Storage of new Experience
ICT619
16
Representation of Cases




CBR heavily dependent on structure and
content of case base
Case search and retrieval of matching
processes needs to be efficient
Actual case representation depends on
domain and task requirements
Also influenced by the structure of the
already available case data
ICT619
17
Some of the representation
approaches
Flat feature-value list

A simple structure

Sometimes sufficient for solving problems in a given domain

Allows relatively easy storage and retrieval in a CBR system
Object-oriented (OO) representations

Reflect the case structure in a hierarchical fashion

A case consists of a set of objects

Objects described by a set of attributes

Each object belongs to an object-class. Object-classes are
organised in an inheritance
ICT619
18
Some of the representation
approaches
Object-oriented (OO) representations (con't)
Car
…
…
Brake system
Ignition system
Spark coil
Transmission system
Motor
Fuel injection system
Spark plug
ICT619
19
Some of the representation
approaches
Graph representations

Graph representations consist of

a set of nodes

arcs joining the nodes

A more flexible but complex structure compared with a
hierarchical representation
ICT619
20
Some of the representation
approaches
Graph representations (con't)
3
Concrete
Number
of lanes
Performance/
Cost attributes:
Cost < $20 million
Deck type
Length
Purpose
Medium
Auto
ICT619
Graph representation of a case (Dhar & Stein 1997)
21
Computation of Similarity for
Matching




Different attributes usually carry different levels of significance
So simple comparison for retrieving similar cases not useful
Attribute values assigned weight values to reflect their significance
This may be done
 A priori based on user experience
 Depending on importance they assume for a specific problem
instance - “discriminating power”
 Attributes may be numeric or symbolic
Numeric valued attributes
 The nearest-neighbour algorithm is commonly used for measuring
inter-case distances
 But the nearest-neighbour algorithm becomes less and less
reliable with increasing number of attributes
ICT619
22
Computation of Similarity for
Matching
Symbolic attributes
 Similarity may be measured heuristically by using
set/subset relationships
 Cases represented hierarchically depending upon their
attributes
 For example, two cases B and C with symbolic
attributes x’ and x’’ have the case A as parent if x’ and
x’’ are special instances (subsets) of A’s attribute x.
 Distance between two symbolic cases may be
measured by their distances to their common parent.
ICT619
23
Computation of Similarity for
Matching
A
C
B
D
E
Case D is more similar to case E (common parent B) than to case
C (a more distant common parent A)
 The distance calculation module of a CBR system may take the
form of a statistical model, a rule-based system or a neural
network.
ICT619
24
Case Storage

Efficient storage and retrieval of cases is essential for large
case bases
Storage method depends on the case representation scheme,
and the size of the case base.The two main approaches are:






Linear lists, for small case bases
Index structures consisting of trees or nets, for large case bases.
Internal vs. External Storage
For small case bases and non-shared data, the main memory
stores the case base.
Databases if the case base is large, or if the data is shared with
other applications
ICT619
25
Case Retrieval
(Partial) Problem
Description
Identify Attributes
Initially match
Search & Select
Best Matching
Case
ICT619
26
Case Retrieval
Identification of Attributes
 Generates relevant problem descriptors from the user input
 Unknown descriptors disregarded or asked to be explained by
user
 Descriptors may be inferred by using contextual general
knowledge
Initial Match
 Cases that match all input attributes are good candidates for
selection
 Cases that match a given fraction of problem features may also be
retrieved
 Similarity assessment may be more knowledge-intensive
Select
 Best match chosen from good candidates found in initial match
 Involves closer inspection and ranking
 Knowledge-intensive selection methods typically generate
ICT619
explanations
27
Case Reuse
 The principal issue for reuses is how to adapt the
solution part from the best matching case to make it
suit given problem
 Possibilities:
 No modification of the solution: simply copies matching
case’s solution part!
 Manual/interactive solution adaptation by the user
 Automatic solution adaptation
 Automatic solution adaptation carried out by
 Using the past case solution – known as transformational
analogy
 Using the method that constructed the past case solution –
derivational analogy.
ICT619
28
Case Reuse
 In transformational analogy, rules or operators
are used to adjust the past case solution with
respect to differences in the two situations
 In derivational analogy, the retrieved case
holds information about the method used for
solving the retrieved problem. Applies retrieved
method to the new case.
ICT619
29
Case Revision

Consists of two tasks:
(1) evaluation of case solution generated by the reuse, and if it fails
evaluation,
(2) repair of the case solution using domain-specific knowledge

The evaluation task may take the form of
 Applying the solution in the real environment to verify its correctness,
quality & user acceptance
 Computer simulation (try out before you commit)

The case repair task involves
 detecting errors in the current solution and
modifying the solution so that failures do not occur

Case retention
 Tested, verified and (if necessary) revised solutions accepted as a correct
solution
 and retained by adding it to the case base
ICT619
30
Business Applications of CBR
(Allen 1994)
 Many focused on case retrieval for decision support
 Case retrieval avoids the step of case adaptation
 Aid decisions to be based on the most similar available
precedents
 Customer service help desk
 Volatile nature of the problem domain
 Knowledge acquisition and maintenance too expensive for
traditional expert systems
 Example - Compaq Computers’ SMART system
 Automation of business processes
 Wide-scale distribution of technical and managerial expertise
 Example - NEC’s SQUAD a corporate-wide system for capture
and distribution of software quality control knowledge
 Some 3000 cases added to the system per year since 1982
ICT619
31
Business Applications of CBR




Design and configuration
Support reuse and modification of standard designs
Used by Nippon Steel and Lockheed.
Applications of CBR systems have also been reported
in the following areas:





Technical fault diagnosis
Classification and prediction
Control and Monitoring
Planning
Bank loan analysis
 The CBR approach to automating planning and
scheduling is an active area of research
ICT619
32
Phases of CBR Development
 Methodologies for CBR system development share the following
phases:
Case-base Design
 A general representation for cases developed using source
materials - written documentation and expert accounts, and
database records
 Involves a coordinated effort by user, managers and system
developers
 Tasks:




Compilation of a lexicon of terms used to describe problem attributes
Selection of appropriate attributes for indexing cases
Specification of database schemas used to store cases
Definition of case base authoring standards.
ICT619
33
Phases of CBR Development
Initial Case-Base Development
 A “seed” case base developed as a baseline
 This case base reviewed and refined by users and
developers until a valid case base covering an
adequate part of the case space developed
Ongoing Development and Maintenance
 Initial case base further refined through execution of
the revise and retain steps during use
 Case accuracy and utility are monitored
 Case base managed like an organisational database
ICT619
34
CBR Tools
 Commercial tools like expert system shells, available for building
CBR systems
 Facilitate quick development of applications
 A typical CBR development environment, as reported in 1994,
provides




Default database schemes for case representation
Problem solving tool for case-based decision support
Forms used for editing cases, attributes, and solutions
Utilities include those for
 manual and automated indexing of cases
 automatic import of cases from records in relational database tables
 conceptual clustering of cases for analysis.

Some examples of CBR tools





ReMind from Cognitive Systems Inc.
CBR Express from Inference Corporation
Esteem from Esteem Software Inc
CasePower from Inductive Solutions Inc.
ReCall from Isoft
ICT619
35
Advantages of CBR systems
 Solves difficult to model problems
 In many application areas, eg in business, problems are often
unstructured and difficult to model
 Reduced knowledge acquisition effort
 Reliance on experts modest, especially if good data already available
 Easier for experts to describe case attributes rather than providing
heuristic rules for solution
 Reduced maintenance effort
 Carried out by addition or deletion of cases
 Reliance on experts is modest, cases easy to understand (cf. rules
in a rule base)
 High scalability and flexibility
 Case base easy to expand and refine
 Enhancement and refinement happens as part of the overall
operation and use
 Mistakes corrected relatively easily by adapting cases
 Performance improves over time through refinement
 Changes in environment get reflected through the addition of new
ICT619
36
and/or deletion of outdated cases
Weaknesses of CBR systems
 Critics of CBR complain that it uses ad hoc or
anecdotal evidence as its main operating principle weak for the reasons human ad hoc adoption of cases
from memory is weak
 Without statistically relevant data for backing and
implicit generalization, there is no guarantee that the
generalization is correct
 Some of the work in CBR may be 'handed off' to
human operator - eg case database needs begin by
hand-crafted cases, which could be difficult to write
 Response time may suffer as number of cases in the
case base grows (depends on indexing method)
ICT619
37
Case Study - Customer Support
(Dhar & Stein 1997, pp.236-243)
 A very successful intelligent system for customer support system
developed at Compaq Computer Corporation
 Given the complexity of a modern personal computer system,
effective customer support is a major undertaking
 The customer support engineer must enter a call for support into a
logging system, analyse the customer’s data, resolve the problem,
and deliver the solution.
 This task is particularly challenging due to
 Dynamic nature of the problem domain - with an increasing range of
products, support staff faces an increasing variety of questions.
 Widening scope of problem domain - the use of more and more third
party hardware and software that must be integrated with Compaq’s
products
 Distributed expertise - due to the width and diversity of the problem
domain, few support staff ever experience the full range of problems
ICT619
38
Case Study (continued)

Compaq had to either increase its support staff number or reduce the
number of incoming calls and the time taken to resolve a call, while
maintaining a high support standard

Compaq needed to develop an intelligent support system that would be a
central repository of problem solving expertise.
The requirements
 The system must integrate knowledge that was highly distributed in
nature.
 The system must handle a large and changing array of models, products
and configurations.
 Users (service engineers, dealers) must be able to find solutions quickly
so that the customer could be got back to within a few minutes of time.
 Users need not require deep knowledge about all problem areas.
 The system must be able to handle incomplete or inexact input since
many customers may not be able to describe the problem fully or
accurately.
 The system must be accessible to many users at different locations.
 To gain user confidence, some explanation capability was needed.
 The system must be easily expandable
over time to allow new kinds of
ICT619
problems to be added.
39
Case Study (continued)

Some positive aspects of the system development
problem were:


Most problems were relatively independent in nature and
non-interacting with other problems.
The system did not need to provide an exact diagnosis, just
the likely problem area.
Possible solutions

Apart from a CBR system, there were two other
possible solution choices:

A standard database system - a DBMS would lack
the rich structures needed for representing the
problems. Also, such a system would be too rigid to
allow users access to open-ended data.
ICT619
40
Case Study (continued)

An expert system - this choice faced the following difficulties




The problem domain was not stable enough for experts to express
solution heuristics with a high degree of confidence
The rule base would require frequent updating due to continuing
changes in the problem domain
Due to the distributed nature of the expertise, the knowledge
extraction process would be difficult
A case-based reasoning system




Required enough prototypical cases to cover the problem domain
This had to be done either by using the experts or from data
gleaned from customer calls
One weakness of a CBR system is the response time as the size of
the case base grows
However, it could deal with noisy or partial data, as missing
attributes tend not to affect similarity computation too adversely
ICT619
41
Case Study (continued)
The solution implemented
 The SMART system based on case-based reasoning and
integrated with the call-logging system
 The following year Compaq developed a second CBR system
called QUICKSOURCE, which was also aimed at the customer
apart from dealers and internal staff
 Case bases for both systems were built using prototypical cases
of previous problems
 The structure of each case was as follows:
 A description of the problem in English
 A set of questions. The answers of the questions could be of the form
yes/no, numeric, or an item from a list. Each question also has a
match weight and a mismatch weight to reflect its importance
 A set of actions (the solution part)
ICT619
42
Case Study (continued)
System operation
 A customer support staff collects the problem information with a
textual description
 Simple problems are resolved by him/her straightaway
 Unresolved problems cause the invocation of the CBR system.
 The system performs an initial search for similar cases using
keywords from the textual description
 A list of matching cases along with their distance scores is
displayed on the screen
 A list of relevant questions pops up on the screen
 As the user answers the questions, the list of cases and their
scores change
ICT619
43
Case Study (continued)
System operation (continued)
 If a perfectly matching case is found, the problem is solved
 If a perfectly matching case is not found, the case is marked
“unresolved” and passed on to the case builder experts. It is then
reviewed and solved
 The system gets refined with each new problem solved added to
its case base
 To keep the growth of the case base under control, new cases are
only added if they are judged to be unique
ICT619
44
Case Study (continued)
Results
Some of the benefits achieved by the CBR system were:
 A larger number of problems could be resolved during the customer’s
interaction with the support system than it was previously possible.
 The success rate on test cases was 50% higher than that without the CBR
system
 Ready access to the case base meant, support staff was more likely to
search for relevant information more actively instead of passing problems on
to specialists
 QUICKSOURCE resulted in 20% fewer calls to Compaq support centre from
dealers and customers as only the harder problems got directed to it
 The case base served as a valuable repository of product performance
information for Compaq
 The company became less susceptible to departures of experienced support
staff
ICT619
45
REFERENCES
 Aamodt, A., & Plaza, E., Case-Based Reasoning: Foundational
Issues, Methodological Variations, and System Approaches,
AICom – Artificial Intelligence Communications, IOS Press,
Vol.7:1, 1994, pp.39-59.
 Allen, B. P., Case-Based Reasoning Business Applications,
Communications of the ACM, Vol.37, No.3, March 1994, pp.40-42.
 Bergman, R., Introduction to Case-Based Reasoning¸
URL http://www.cbr-web.org/CBR-Web/cbrintro.
 Dhar, V., & Stein, R.,”Solving Problems by Analogy” in Seven
Methods for Transforming Corporate Data into Business
Intelligence., Prentice Hall 1997, pp. 149-166, 236-243.
ICT619
46