Transcript No Slide Title

Intelligent Knowledge Extraction from Service
Notes for Equipment Diagnostics Support
Anil Varma
Vipin Ramani
Information Technology Lab
GE Corporate Research & Development
Correspondence to : [email protected]
Introduction
• GE Medical Systems - Health Care Services - Multi Vendor
Equipment Maintenance
• > 30 OEMs. > 50 Equipment Types > 100 Models
• Laptop based Problem-Solution Knowledge Base
• Improved Dissemination of Diagnostic Knowledge
• Corporate knowledge base
• Data Source
• Text Dispatches describing observed symptoms and
solution process attempted.
Little AI goes a long way..
• > 100 Meg of data/year.
• Need to automatically identify dispatches with useful
diagnostic information
• Classification problem with supervised learning
• Results
Project Overview
Dispatch
Database :
Dispatch Data
Field
Engineer
OLC Service
Engineer
Field
Engineer
Dispatch Data
Field
Engineer
EXAMPLE DISPATCH
REPORT DISTRIBUTION LOCATION 3017
DISPATCH NO: 0650 170729
DISPATCHER: A5569
GROUP CONTROL NO: 0650170729
COLUMBIA ROSE MEDICAL CENTER
ACCOUNTS PAYABLE
4567 EAST 9TH AVENUE
DENVER
CO 80220-3908
START
END
EQUIP STATUS
970725 970803
STARTING: 92
18 00
18 00
FINAL: 90
S E R V I C E
R E C O R D
H I S T O R Y
D E T A I L
01/19/98
14:27:50
TYPE SERVICE: EMG
BILL ACCT : 650742935
DATE: 970725
ACCT TYPE : 0
AREA CODE: H311
TIME: 14 23
PHONE
: 3033202815
ROOM ID
: CT
ISC ENT: N
STATUS
: 045
ROOM DESC : SOMATOM PLUS
ISC ASG: N
MODALITY : CT
ROOM SYS-ID: 303320CT
FE ASSIGNED: 12467 COYLE
H311 WHEN ASSIGNED: 970725 14 27
--ACCUM CONTACT HRS----ACCUM TRAVEL HRS--TRAVEL EXPENSE SVC-CLS CONT-CLS
STD
19.5
18.5
STD
4.3
4.3
.00
005
01
ORDER NO.
MODEL NUMBER
SERIAL NO.
FMI NO
FMICD MA
CM
PT
9999 999999
94
Y
CEPMPP3
CONT-NO.
116493
SYMPTOMS: HAVING RING ARTIFACT ON SCANNER
SVC
IS
PRF
EI
CDS
NT
HRS
38.0
DIAGNOSIS AND SERVICE PERFORMED DESCRIPTION
PM CODE:
TROUBLESHOT FOUND BAD PREAMP BOARD #11 IN DAS MOVED IT TO #2 SLOT
Effective utilization of FE Dispatch Data (110,000+). Creation of a Corporate Knowledge Base
4
Raw Problem-Solution Data
Dispatch_No Symptom_text
220241936
KEYBOARD IS NOT WORKING/TIME OUT
MEASUREMENT ERROR QUE OVERRUN ERROR
220241940
Magnet Monitoring Alarm Site stuck in
fill mode- check key/pwr
220241956
HAVING CHEMICAL PROBLEM WITH THE
PROCESSOR
220241963
AUTO COLLIMATOR NOT WORKING
220241969
THE TUBE STAND COMPUTER IS LOCKED UP
220241970
220241972
POOR RESOLUTION ON BARS
THE A-DAC IS LOCKED AND THEY ARE DOWN
220241974
X-RAY TUBE IS OVERHEATING/COOLING UNIT
IS NOT WORKING
PROBLEM WITH FILM MAGAZINE
TOMO NOT WORKING
COLLIMATION IS NOT OPENING UP
COMPLETELY
THE BUCKY TRAY IS LOOSE
COLLIMATION IS NOT OPENING UP
COMPLETELY
POOR GAIN
OPTICAL DISK--CRASHED OPTICAL DISK
NO EXPOSURES
HEAD COIL IS BROKEN
TUBE CHANGE
BAD H2O FLOW SENSOR ON MAGNET
MONITORING KIT
IIE CAMERA TAKES TOO LONG TO EXPOSE
HAVING TROUBLE WITH THE BACK UP TIMER,
AND DARK FILMS
THYROID UPTAKE PROBE IS BLOWING FUSES
LIKE CRAZY
220241979
220241981
220241983
220241997
220241999
220242004
220242008
220242010
220242015
220242023
220242027
220242035
220242038
220242056
Dispatch_No Solution_text
220241936
SYSTEM HAD COMMUNICATION INTERFERENCE LIKE
BEFORE BETWEEN FUNCTION KEYS AND HOST
COMPUTER SWAPPED FIBER OPTIC COMM LINES
BETWEEN KEYBOARD AND FUNCTION KEYS.
PROBLEM HAS DISAPPEARED.
220241963
RESTRUNG CASSETTE SIZE SENSING MECHANISM
IN BUCKY TRAY, ALSO STRAIGHTENED
COLLIMATOR ON TUBE
220241969
CONTROL PROBLEM INTERMITTENT SWITCHED
CONTROL WITH NORTHROOM.
220241970
FOUND CO57 MAPED TO TL201REMAPPED TO TC99
RAN FLOODS CHECKED OK. KEN REQUESTED WE
RUN CO57 FLOOD CAL. ATTEMPTED TO DO SO BUT
SHEET SOURCE TO WEEK. MAPPED TO TC99 PER
KEN.
220241972
PHONE FIX
220241979
MAGAZINE BINDS AND WILL NOT RUN
220241981
REPLACED TOMO FULCRUM DRIVE FUSE, RESEATED
TUBESTAND DRIVE ARM
220241983
FOUND MANUAL COLLIMATOR CONTROLS IN MANUAL
OPERATION OPENEDSHUTTERS. CHECKED
OPERATION IN FLUORO AND
RADIOGRAPHIC.ADJUSTED MANUAL KV/MA FOR
BETTER IMAGE ON MONITOR.
220241997
WAITED TILL 12:30 FOR ROOM CASSETTE
HOLDING FOOT BROKEN OFF. DRILLED, TAPPED
AND RE-FASTENED FOOT.
220242004
REPLACED RECIEVER BOARD
OPS CHECKED
GOOD.
5
Examples of Varying Dispatch Content
•PROBLEM FOUND TO BE WITH "MOLEX" CONNECTOR FROM POWER SUPPLY CONNECTOR
CHANGED BY WAYNE TYSAK ON 5-2-97, PROBLEM CORRECTED.
•PERFORMED EXTENSIVE TESTING TO PROVE THAT CUT-OFF IS CAUSEDBY MISALIGNMENT OF FILMSCREENS INSIDE KODAC 18X24 CASSETTES NOT AN OBSTRUCTION
IN THE X-RAY FIELD.
•FILM APPEARS TO BE HANGING ON THE NIP ROLLER AS IT ENTERS THE RECEIVER
MAGAZINE ORDERED NIP ROLLER AND DRIVE BELTS. INSTALLED NIP ROLLER AND
DRIVE BELTS. OK FOR NOW BUT BELIEVE SPRING STEEL TENSIONER IS WEAK AND
NEEDS REPLACEMENT.
•DID NOT FIND ANY PROBLEMS
•TESTED TOMO BUT COULD NOT FIND ANY PROBLEMS RETURNED TO SERVICE.
•CHECKED SYSTEM OPERATION AND DID NOT FIND ANY PROBLEM
•CHECKED IMAGE
DID NOT FIND ANY PROBLEMS.
•THIS IS A BOGUS DISPATCH
•COULD NOT REPRODUCE PROBLEM
ADJUSTED TABLE BUCKY CONTACTS.
•COULD NOT REPRODUCE PROBLEM TURNED PROBLEM OVER TO C MCCALLISTER /
R.RUSH
•COULD NOT REPRODUCE PROBLEM TESTED SYSTEM AND COULD NOT REPRODUCE.
•TABLE LOCKS WERE WORKING AND COULD NOT REPRODUCE PROBLEM
•PERFORMED PMBILL NATIONAL MD FOR 6 5 HRS LABOR & 1 HR TRAVEL TOTAL CHARGE
= $1,200.00
6
Data Quality
Bad!
Spellng errors, technical terms, abbrev., spacingerrors,
a.c.r.o.n.y.m.s
NLP
Don’t try to understand,
just learn patterns in data
Process for Supervised Learning
Mine for
diagnostically useful
dispatches
100-X%
X%
Manually
judge
relevance
Useful
Dispatches
extracted typically 10%60% of raw
input data
Rejected Dispatches
Accepted Dispatches
Algorithm for learning
diagnostically useful
patterns
Text Matching Features
FILM APPEARS TO BE HANGING ON THE NIP ROLLER
Sentence
Words
Word parts
Increasing
similarity
possibility,
decreasing
context
Letters
FILM APPEARS TO BE HANGING ON THE NIP ROLLER
WATER FILM ON THE NIP ROLLER
FILMAPPEARS TOBEHANGING ONTHENIPROLLER
AAAEEEFGGHIILLLMNNOOOPRRRST
N-gram Matching Approach
• N - grams
diagnosis = dia, iag,agn, gno,nos,osi,sis (trigram)
diagnsis = dia, iag,agn, gns,nsi,
sis
• Have user identify seed accepted and rejected dispatches
• Break dispatches into N- Grams
• Use N-Grams as features
+1
+ve
-1
VOI
THE
-ve
0
+ve
-ve
FIX
+ve
-ve
Case Based Matching Approach
New Dispatch
Accepted Dispatches Bin
Rejected Dispatches Bin
Similarity Function = [ Weights of common datagrams]2
---------------------------------------- Weight of datagrams in Classified phrase X  Weight of datagrams in New phrase
Too Slow!
Alternate Approach
•Try to come up with aggregate decision by looking at
constituent datagram weights and signs.
OPTICAL DISK IS MECHANCIALLY JAMMED
WILL NOT ACCEPT DISK.REPLACE WITH PART FROM INVENTORY
HP UNIT. TEST PASSED.
•# +ve versus # -ve datagrams
•Sum of +ve weights vs. sum of -ve weights
•Highest weight datagram
•Highest weight ranking if weight > W else
voting/aggregation.
Accept
Neutral
Reject
CONNECTOR
2.6264
GREASE
0.4797
BEAR
0.0000
ALLOW
0.0000
COST
0.0000
TRANSMISSION 2.2856
DETECTOR
2.3476
BILLING
-3.4435
VOID
-1.7765
Collect features of high +ve or -ve weights - adaptive
datagram size
1. Vary datagram size from 3 to N.
2. For each run, retain datagrams with weight > W
3. Look for neighboring datagrams with similar
weights/signs, merge as new test phrase pre rev eve ven ent nti ive -> preventive
average wt. = -0.51
-0.87
System Operation
@0470393440@ PERFORMED COMPLETE PREVENTIVE MAINTENANCE CALIBRATION
CHECK=SATISFACTORY, KV, MA,& EXPOSURE TIMES ALL CH.SAT. IMAGAE
QUALITY CH.SAT. MAX. FLOURO RAD. DOSE. CH.SAT. HV CABLES CLEANED&LUB. ALL
CONTROLS AND LOCKS OP.CH.SAT. OVERHEAD TUBE SAFETY CABLE C
H.SAT. ENTIRE SYSTEM CLEANED .@
Recommendation : -0. 54
Automatically Accept
User varies
Forward to
expert for
decision
Automatically Reject
Service Center
email
Dispatch
classification
Periodic updates
Field
Engineer
Some Results on Test Data
Trial
Size of
N-gram
Overall
accuracy
%
%
of
accepted
notes
correctly
classified
%
of
rejected
notes
correctly
classified
% training
set
1
2
3
4
3
4
5
4
84
87
81
88
90
94
88
95
80
83
77
84
70
70
70
80
Table 1. Service note classification performance
Tool released to 450 field service engineers in US & Europe
Conclusions
• Lots of text diagnostic information in equipment
service operations
• Quickly deployable
• Learning approaches to text
• Clustering different failure modes for particular equipment
Locomotive example
Text repair actions
for same problem
code
REPORTED PENALTY BRAKE APPLICTION
WATER IN AC OIL- CHANGED OIL
28WATER IN AC OIL-REPL OIL
PENALTY BRAKE APPLICATION TESTED
APPLIED WABCO MODS
11WATER IN AC OIL CHANGED OIL
CO CCC FOR NOT BOOTING UP
CO BCCM FOR SLOW BC APPLICATION
EAB TROUBLE - NTF
DID WABCO EAB MODS