Transcript Use of Online Resources While Using a Clinical Information

Use of Online Resources While
Using a Clinical Information System
James J. Cimino, MD; Jianhua Li, MD;
Mark Graham, PhD, Leanne M. Currie, RN, MS;
Mureen Allen, MB BS; Suzanne Bakken, RN, DNSc,
Vimla L. Patel, PhD
Department of Biomedical Informatics
Columbia University
Information Needs Study
Method
Development
Study 2
Identify information resources
currently being used
Study 1A
Identify information
needs events
Observational
Study
Characteristics of
Information Needs Events
2A
Log file
analysis
2B
Survey
Characteristics of Resource Use
Study 1B
Classify Information
Needs Events
Classification
Categories of
Generic Question
Understanding Resource Use
• Are clinicians using resources to answer
patient-related questions?
• What resources do clinicians use?
• Do they use resources differently
depending on the context?
Methods
• Log file captures clinician use of resources
– Records user, function, location
– User type determined from user directory
– Context of use can be inferred
Logging Information Resource Use
WebCIS
Logon
Screen
WebCIS
Functions
Health
Resources
Page
(anonymous)
Health
Resource
System
Log
Health
Resources
Page
Health
Resource
Log File Examples
16:14:38|Resource|nobody|156.1.5.1|salk.cpmc.columbia.edu/mdxdocs
16:16:46|WebCIS|ciminoj|156.1.5.1|lab^2003-03-02-05.19.00.819650
16:18:55|WebCIS|ciminoj|156.1.5.1|healthresource
16:21:18|Resource|ciminoj|156.1.5.1|salk.cpmc.columbia.edu/mdxdocs
16:24:30|WebCIS|ciminoj|156.1.5.1|lab^1997-09-02-15.54.00.359121
16:25:12|WebCIS|ciminoj|156.1.5.1|pubmed
16:25:48|WebCIS|ciminoj|156.1.5.1|micromedex
16:32:38|WebCIS|ciminoj|156.1.5.1|phar
16:25:48|WebCIS|ciminoj|156.1.5.1|micromedex
Log File Results
• Six months of log files
• 38,763 health resources
– 14,036 anonymous health resources
– 19,913 health resources from WebCIS
– 4,814 infobuttons
• 2,607 users
– 51% resident physicians
– 34% attending physicians
– 5% nurses
– 10% other (pharmacists, administrators, etc.)
Information Resource Usage
12% IBs
10% 1% 1%
6%
51%
6%
4%
4%
4%
3%
7% 3%
Micromedex
Pharmacy IB
Sensitivity IB
Culture IB
Ovid Medline
Online Formulary
Harrison's
PubMed
Journals
MedlinePlus
ICD9-CM
Other Resources
Information Use Contexts
9%
Laboratory
7%
3%
51%
Radiology
2%
Clinic Notes
2%
Pathology
2%
Visit Lists
Pharmacy
Discharge Sum.
24%
Other
Context-Specific Resource Use
100%
Sensitivity IB
Culture IB
80%
Pharmacy IB
Medlineplus
60%
ICD9-CM
Journals
PubMed
40%
Harrison's
Formulary
20%
Ovid Medline
Micromedex
Ph
ar
m
ac
y
t
it
Li
s
S
ge
ch
ar
D
is
Vi
s
um
.
gy
ho
lo
Pa
t
ol
og
y
C
ar
di
C
lin
ic
gy
R
ad
io
lo
La
bo
ra
t
or
y
0%
Methods
• Log file captures clinician use of resources
– Records user, function, location
– User type determined from user directory
– Context of use can be inferred
• Questionnaire to determine:
– Are they using resources through our system?
– Why do they use resources?
– What resources do they use?
– What resources do they like?
– What other resources do clinicians use?
– Do survey results match log files?
Survey Population
• 1055 resources users in 01/03
• 250 randomly surveyed
• 45 (18%) returned
– 23 attendings
– 19 housestaff
– 3 nurses
– 1 pharmacist
Using Resources through WebCIS
•
•
•
•
14 anonymous only *
17 in WebCIS only
11 both
3 neither *
* All used resources in WebCIS
Why do they use resources?
• 43 for patient-specific issues
• 23 for general knowledge
• 3 for other reasons
– writing
– teaching
– studying
ro
m
e
P u dex
b
H Me
a
O rri d
vi so
d
M n's
ed
l
H
ea Fu ine
l
lth l T
Jo ex
u t
C r na
ul
ls
Le ture
x
Ph i-C IB
ar om
m p
ac
y
IB
P
Fo IE
M rmu R
ed l a
lin ry
eP
ST lus
S
S c en A T
ie sit re
nc i v f
e ity
Jo IB
ur
n
O ICD als
th
er 9-C
Jo M
ur
n
In als
de
Su Me xe
b d s
T u j e c i ca
be t G re
Pr N rcu uid
eg urs los es
na in is
nc g P inf
o
y
Ca olic
l c i es
ul
at
or
s
M
ic
What resources do they use? Like?
40
35
30
25
Not Helpful
20
Somewhat Helpful
15
Very Helpful
10
5
0
• No marked difference between user types
• Log files corroborated usage claims
• 26/45 said “do not use” (although 38/45 used)
What about other resources?
• Reources used:
– Up To Date
– MD Consult
– New England Journal of Medicine
– Epocrates
– Google
– Books
Reasons for use:
28 for patient-specific issues
25 for general knowledge
1 for teaching
Lessons Learned
• Log files representative of resource use:
– Our users tend to use “Health Resources” page
– Surveys reflect observed usage
•
•
•
•
•
•
•
•
Micromedex is most popular resource
Other resources needed (e.g. UpToDate)
Resource use is for patient-specific reasons
Most resource use is within patient context
Laboratory results are most popular context
Context does influence resource use
Infobuttons actually get used!
Infobuttons can be preferred!
Information Needs Study
Method
Development
Study 2
Identify information resources
currently being used
Study 1A
Identify information
needs events
Observational
Study
Characteristics of
Information Needs Events
2A
Log file
analysis
2B
Survey
Characteristics of Resource Use
Study 1B
Classify Information
Needs Events
Classification
Categories of
Generic Question
Infobutton
Manager
Where Do We Go From Here?
• Up To Date has been added
• Infobutton Manager matches context to questions
• Focus on laboratory-related questions
• Focus on ways to exploit Micromedex
• Extend Infobutton Manager to other institutions
www.dmi.columbia.edu/homepages/ciminoj/Infobuttons.html
Conclusions From the Three Studies
•
•
•
•
•
•
•
Information needs arise while using clinical systems
Needs can be detected and characterized
Information needs are context-specific
Information needs are stereotypic
Satisfaction is often unsuccessful or deferred
Resource use is context-specific
Infobuttons are a viable way to resolve needs
– Offer context-specific retrievals for specific needs
– Provide shortcut to popular resources
• Infobuttons are an institution- and applicationindependent way to improve resolution of
information needs and reduce medical errors
Acknowledgments
• National Library of Medicine research grant
• National Library of Medicine training grant
• Rick Gallagher for log file support
www.dmi.columbia.edu/homepages/ciminoj/Infobuttons.html