Transcript Slide 1

Sharing guidelines
knowledge: can the
dream come true?
Medinfo panel
Cape Town, September 15, 2010
Motivation

The vision of sharing executable clinical
knowledge can be achieved only if we:





2
Standardize platforms for deploying scalable
knowledge based services
Ensure services are mutually compatible and
interoperable and free of institution-specific details
Develop reusable content and service components
Support automated cross-verification for quality and
safety
Establish communities of practice who share,
maintain, update, and improve content
Objectives

Raise awareness of the practical challenges
involved in maintaining repositories of sharable
executable clinical knowledge




3
Challenges with maintaining a repository
Defining what knowledge can be shared and how
Challenges in piecing together knowledge into a
care plan and integrating it with EHR data
If the knowledge if free, what’s the business model
and incentives for contributing knowledge?
Panel participants




4
John Fox, Department of Engineering Science,
University of Oxford, UK
Robert Greenes, Ira A. Fulton Chair of the
Department of Biomedical Informatics, Arizona
State Univercity, Phoenix
Sheizaf Rafaeli, Head of the Graduate School of
Management and Sagy Internet Research Center,
University of Haifa, Israel
Mor Peleg, Head of the Department of
Information Systems at the University of Haifa
What shall we discuss?
John Fox 
Bob Greenes 
Mor Peleg 
Sheizaf Rafaeli 
5
Life-cycle approach for sharable
knowledge-based patient-care services
Methodology for distilling sharable
knowledge from business/ implementation
considerations
Methods for weaving medical knowledge
services into an application and for
mapping clinical abstractions into EHRs
Incentives and business models for a
knowledge-sharing community
John Fox
20th Anniversary
Gold Medal Award
Options for addressing open-source publishing of medical knowledge,
drawing on lessons learned in the OpenClinical project
6
OpenClinical: Open Source?
John Fox
University of Oxford (Engineering Science)
UCL (Oncology, Royal Free Hospital)
www.cossac.org
www.OpenClinical.org
• Goal: To promote awareness and use of
decision support, clinical workflow and other
knowledge management technologies for
improving quality and safety of patient care
and clinical research.
• A resource and portal for technologists,
clinicians, healthcare providers and suppliers
• Currently about 200,000 visitors a year (80%
growth in 2010)
www.OpenClinical.net
• Experimental project to explore how to
develop content for high quality clinical
decision support and workflow services at
the point of care
• Goal is to build a community of users,
researchers and content providers who are
willing to contribute to the development of a
repository of open content, including
applications and application components
OpenClinical.net test site
pro tem: modx.openclinical.net
Content development lifecycle
• Prototype development model for open
source content repository on
www.OpenClinical.net
• Currently limited to PROforma decision
and process modelling language
• Intended to eventually multiple
representations (e.g. GLIF, ASBRU,
GELLO, OWL ...)
Load from, save to repository
Download tools
www.cossac.org/tallis
Web publishing (“publets”)
Integrate and Deploy
Key questions for open content
• Quality and Safety
– Quality lifecycles, safety culture, who is liable?
• Reusability and interoperability
– Open technical standards, who is developing them?
• Functioning community (Sheizaf Rafaelli)
– What will sustain the open source ethic?
• Facilitating infrastructure (Bob Greenes)
– Three organisations; too little? too much?
• Sustainable business models
– How do the proprietary/open source worlds coexist?
Sustainable business models (1)
• Traditional standalone apps?
• Issues of integration and localisation
• Likes fragmentation; hates interoperability
• Pay per patient (analogous to pay per
view)
• Who would/should actually pay?
• No-one pays for Adjuvant! Online
Sustainable business models (2)
• Standard medical publishing model
• Commercially viable on a publishing model?
(Clinical Evidence)
• Discussion on www.berkerynoyes.com/
pages/innovations_in_evidence_based_medicine.aspx
• Open Source with value-adding services? (c.f.
Linux model)
• Attractive model but how can we achieve critical
mass of a content development community?
Towards an open content lifecycle?
Ioannis Chronakis
Vivek Patkar
Richard Thomson
Matt South
Ali Rahmanzadeh
Thank you
Robert Greenes
MUMPS
Morningside Initiative
Morris Collen Award
Sharing medical knowledge involves separation between the
medical content and the business/applications considerations
22
Toward sharing of clinical
decision support knowledge
- A focus on rules
Robert A. Greenes, MD, PhD
Arizona State University
Phoenix, AZ, USA
Purpose of this talk
• Identify key challenges to CDS adoption
with focus on rules
– Expressed in terms of 3 hypotheses:
1. Sharing is key to widespread adoption of CDS
2. Sharing of rules is difficult
3. Sharing can be facilitated by a formal approach
to rule refinement
Hypothesis 1: Sharing is key
to widespread adoption of
CDS
• We know how to do CDS!
– Over 40 years of study and experiments
• Many evaluations showing effectiveness
Rules as a central focus
• Importance of rules
–
–
–
–
Can serve as alerts, reminders, recommendations
Can be run in background as well as interactively
Can fire at point of need
Same logic can be used in multiple contexts
• e.g., drug-lab interaction rule can fire in CPOE, as lab alert, or as
part of ADE monitoring
– Can invoke actions such as orders, scheduling, routing of
information, as well as notifications
• Relation to guidelines
– Function as executable components when GLs are
integrated with clinical systems
• Poised for huge expansion
– Knowledge explosion – genomics, new technologies, new
tests, new treatments
– Emphasis on quality measurement and reporting
Yet beyond basics, there is
very little use of CDS
• Positive experience not replicated and
disseminated widely
–
–
–
–
Largely in academic centers
<30% penetration
Much less in small offices
Pace of adoption barely changing
• Only scratching surface of potential
uses
–
–
–
–
drug dose & interaction checks
simple alerts and reminders
personalized order sets
Narrative infobuttons, guidelines
Adoption challenges
• Possible reasons
1.
2.
Users don’t want it
Bad implementations
• Time-consuming, inappropriate
• Disruptive
3.
Adoption is difficult
•
•
•
•
Finding knowledge sources
Adapting to platform
Adapting to workflow and setting
Managing and updating knowledge
• But new incentives and initiatives rewarding quality
over volume can address #1
– Health care reform, efforts to reduce cost while preserving
and enhancing safety and quality
• And #2 AND #3 can be addressed by sharing of best
practices knowledge
– Including workflow adaptation experience
Hypothesis 2: Sharing of
rules is difficult
• Rules knowledge seems deceptively simple:
– ON lab result serum K+
– IF K+ > 5.0 mEq/L
– THEN Notify physician
• Even complex logic has similar EventCondition-Action (ECA) form
– ON Medication Order Entry Captopril
– IF Existing Med = Dyazide
AND proposed Med = Captopril
AND serum K+ > 5.0
– THEN page MD
Why is sharing not done?
• Perception of proprietary value
– Users, vendors don’t want to share
– Non-uptake even with:
• Standards like Arden Syntax for 15 years, GELLO for 5 years
• Knowledge sources such as open rules library from Columbia since 1995,
and guidelines.gov, Cochrane, EPCs, etc., - most not in computable form
• Failure of initiatives such as IMKI in 2001
• Lack of robust knowledge management
– To track variations, updates, interactions, multiple uses
• Same basic rule logic in different contexts
• Beyond capabilities of smaller organizations and practices to undertake
• Embeddedness
–
–
–
–
–
In non-portable, non-standard formats & platforms
in clinical setting
in application
in workflow
in business processes
Example of difficulty in
sharing
• Consider simple medical rules, e.g.,
– If Diabetic, then check HbA1c every 6 months
– If HbA1c > 6.5% then Notify
• Multiple translations
– Based on how triggered, how/when interact,
what thresholds set, how notify
– Actual form incorporates site-specific
thresholds, modes of interaction, and workflow
• Multiple rules have similar intent
• Differences relate to how triggered, how delivered,
thresholds, process/workflow integration
• Challenge is to identify core medical knowledge and
to develop a taxonomy to capture types of
implementation differences
Setting-specific factors
(“SSFs”)
•
Triggering/identification modes
– Registry, encounter, periodic panel search, patient list for day, …
– Inclusions, exclusions
•
•
Interaction modes, users, settings
Data mappings & definitions, e.g.,
– What is diabetes - code sets, value sets, constraint logic?
– What is serum HbA1c procedure?
•
Data availability/entry requirements
– Thresholds, constraints
•
Logic/operations approaches
– Advance, late, due now, …
•
Exceptions
– Refusal, lost to follow up, …
•
Actions/notifications
– Message, pop-up, to do list, order, schedule, notation in chart, requirement
for acknowledgment, escalation, alternate. …
Hypothesis 3: Sharing can be
facilitated by a formal approach
to rule refinement
• Develop an Implementers’ Workbench
• Start with EBM statement
• Progress through codification and
incorporation of SSFs
• Output in a form that is consumable
“directly” by the implementer site or
vendor
Life Cycle of Rule Refinement
Start with EBM statement
Stage
1.
Identify key elements and logic – who, when, what to be done
–
–
2.
Structured headers, unstructured content
Medically specific
Formalize definitions and logic conditions
–
–
3.
Structured headers, structured content (terms, code sets, etc.)
Medically specific
Specify adaptations for execution
–
–
4.
Taxonomy of possible workflow scenarios and operational
considerations
Selected particular workflow- and setting- specific attributes for
particular sites
Convert to target representation, platform, for particular
implementation
–
–
–
Host language (Drools, Java, Arden Syntax, …)
Host architecture: rules engine, SOA, other
Ready for execution
Four current projects addressing
this challenge
EBM statement
1. Identify key elements and logic – who, when,
what to be done
2. Formalize definitions and logic conditions
3. Identify possible workflow scenarios – model
rules, defining classes of operation
4. Convert to target representation, platform, for
particular implementation
Idealized life cycle / Morningside / KMR / AHRQ SCRCDS/ SHARP 2B
What we hope to accomplish
•
•
•
•
Implementers’ Workbench (IW)
Taxonomy of SSFs
Knowledge base of rules
Approach
– Vendor, implementer, other project input, buy-in,
collaboration
– Taxonomy as amalgam of NQF expert panel,
Morningside/SHARP/Advancing-CDS workflow studies,
SCRCDS implementation considerations
– Diabetes, USPS Task Force prevention and screening
A&B recommendations, and Meaningful Use eMeasures
converted to eRecommendations as initial foci
– Prototyping, testing, and iterative refinement of IW
What we expect to share
•
•
•
•
Experience/know-how
Knowledge content
Methods/tools
Standards/models
Standards/models
•
•
•
•
•
•
Representation
Data model/code sets
Definitions
Templates
Taxonomies
Transformation processes
Where CDS should go
from here?
• Need for coordination
– Multiple efforts underway
– Need to coalesce and align these
• Need sustainable process
– Multi-stakeholder buy-in, participation, support,
commitment to use
• Need to demonstrate success
– Small-scale trials
– Larger-scale deployment built on success
• Expansion to other kinds of CDS
Comments? Questions?
Mor Peleg
GLIF3
Process
Mining
New Investigator Award
Biomedical Ontologies
K
KDOM
Data
42
Weaving medical knowledge services into applications.
Using a mapping ontology to map medical knowledge into institutional data
Implementing decisionsupport systems by
piecing sharable
knowledge components
Mor Peleg, University of Haifa
Medinfo panel, Cape Town, September 15, 2010
Motivation




Computerized guidelines have shown
positive impacts on clinicians but they take
time to develop
Solution: Share executable GL components,
stored in Medical Knowledge Repository
Assemble computerized GLs from
components
Map the GL’s medical terms into institutional
EHR fields
Examples of medical resources that
could be shared and assembled





Medical calculators
Risk-assessment tools
Drug databases
Controlled terminologies (e.g. SNOMED)
Authoring, validation, and execution
tools for computer-interpretable GLs
Component
interface
Peleg, Fox,
et al. (2005)
LNCS 3581
pp.156-160.
The interface can be used for


Sharing components
Indexing and searching for components


Using the attributes: clinical sub-domain,
relevant authoring stages, and goals
Assembling components into a GL

Specifying the guideline's skeleton
language (e.g., GLIF, PROforma) into
which components can be integrated
Example: providing advice on
regimens for treating breast cancer
Get patient
Data
Prescribe
regimen
Is patient
eligible for
evaluating
therapy
choices?
Calculate
regimen
Adjuvant's lifeexpectancy
calculator
Choose option
Filter out nonbeneficial and
contraindicated
therapies
Present
choices to
user
Using Standards



Skeleton can be any GL formalism
Eligibility criteria expressed in GELLO standard
Referring to the HL7-RIM patient data model
Integrating assembled
guideline with EHR data

Encode once but link to different EMRs


Global-as-View
Mapping Ontology + SQL Query Generator
RIM
50
Peleg et al., JBI 2008 41(1):180-201
Knowledge-Data Ontology
Mapper (KDOM)
Patient has Palpable Breast Mass or
Hard_Breast_Mass
GELLO
interpreter
anyEMR
Guideline Expression
(need not use EMR’s terms)
“Breast Mass = true”
KDOM mapping
classes:
Direct, Hierarchical,
Logical, Temporal
KDOM mapping
instances
SQL
query
generator
SQL query
Evaluated
expression
Query result:
RIM view
Palpable Breast Mass is-a Breast Mass.
Palpabale
51 Breast Mass is stored in the Problems table
true
Observation of Breast Mass
Summary



52
A repository of tested executable
medical knowledge components that
would be published on the Web
Framework for specifying the interface
of components so that they could be
searched for and integrated within a
Computerized GL specification
KDOM used to integrate the medical
knowledge with institutional EMRs
Thanks!
[email protected]
Hope to see you at AIME 2011, July 2-6, 2011, Bled, Slovenia
53
Sheizaf Rafaeli
survey and contrast social, technical, hierarchical and
market-based models for motivating and maintaining the
sharing of information and processing tools
54
Sharing Guidelines Knowledge:
can the dream come true?
Sheizaf Rafaeli
[email protected]
http://rafaeli.net
MedInfo 2010
Bits Replacing Atoms
56
[email protected], http://rafaeli.net
Moore, Gilder, Metcalfe, Reed
utility
users
57
[email protected], http://rafaeli.net
Information Overload
Economics of Scarcity vs.
Economics of Abundance?
58
[email protected], http://rafaeli.net
What’s really new?
• Access has become widespread
• Information as a commodity; IT as a commodity
“Does IT matter “? Transmission has been solved
• Information is an experience good
• The impossible ease of copying
• Disintermediation
• Free information has become commonplace,
normative, expected. Both free and for-fee
information occupy the same net
59
[email protected], http://rafaeli.net
“Free” as in free speech, or as in free beer?
• New Rules for the New Economy : 10 Radical
Strategies for a Connected World
by Kevin Kelly
• “Information Rules : A Strategic Guide to the
Network Economy
by Carl Shapiro, Hal R. Varian
60
[email protected], http://rafaeli.net
61
[email protected], http://rafaeli.net
How is UGC motivated?
62
[email protected], http://rafaeli.net
The Value of Information
•
•
•
•
•
•
Public source
Commodity
Overload
History
Technology
Psychology?
•
•
•
•
•
•
•
Private source
Uniqueness
Timing
Presentation
Tailoring
Technology
Network effects
Emphasis on distinction between Private and Public
Suggesting the Subjective Value of Info
63
[email protected], http://rafaeli.net
Wikipedia: a system that shouldn’t work, but does.
Participation Power Laws and Long Tail
‫כלים מוזרים לתיגמול‬
Wiki “barnstars”
Web 2.0
UGC
and
Coproduction
65
[email protected], http://rafaeli.net
Further personal stakes in info value
• Information markets http://answers.google.com
• Online Scientific Journals
http://jcmc.indiana.edu
• Citizens’ Advice Bureaus
http://shil.info
• Wikis http://misbook.yeda.info
• Online Higher Ed systems
http://qsia.org
• Games and Serious Games
66
[email protected], http://rafaeli.net
67
[email protected], http://rafaeli.net
68
[email protected], http://rafaeli.net
SHIL





(‫שרות יעוץ לאזרח )שי"ל‬
Citizen Advice Bureaux (CABs) Established
1957
55 “Brick and Mortar” offices
Telephone hot line & Internet web site,
operated at the Univ. of Haifa Sagy Center
Operated by Volunteers, coordinated and
funded by the Israeli Ministry of Social
Affairs and Social Services in collaboration
with municipalities.
Ownership…
• Legal Perspective
Vs. Open Source,
Peer-to-Peer,
UGC, Web 2.0, etc.
• Apply 19th century
property law to
21st century reality?
• Legality: "fair use" "first sale"
"prior art" doctrines
• Open Innovation
70
[email protected], http://rafaeli.net
Discussion
• Still LOTS to study and learn…
• Interactivity and Social Motivations seem to be king
• A high (too high?) overall subjective value for information.
• As predicted by the Endowment Effect theory, WTA for
information was significantly larger than WTP for information
• This predicts undertrading. Implications for system design
73
[email protected], http://rafaeli.net
Discussion (2)
• Information is a commodity.
Nevertheless, information is still easier to duplicate, easy to share, and
ownership of it proves more difficult to enforce
• Society has not yet adjusted its information consumption patterns to the
present situation of information abundance
• Scoring and Governance Rules!
74
[email protected], http://rafaeli.net
Thank you
[email protected]
http://rafaeli.net
76
[email protected], http://rafaeli.net
Provocative statements
77
Statement 1

78
A national or international effort can
be put together to create a repository
of implementable knowledge.
Statement 2

79
Guideline sharing could be achieved
within 10 years
Statement 3

80
Guideline sharing at the
implementation level requires
separation into component steps
that can be individually implemented,
because of differences in
process/work flow that prevent the
guideline from being adopted in its
entirety
Statement 4

81
True sharing of executable medical
knowledge could never be achieved
because knowledge could not be
separated from institutional
adaptations
Statement 5

82
Guideline formalization activities do
not typically address implementation
settings and requirements
Statement 6

83
The benefits of formalizing and
sharing clinical knowledge are beyond
dispute: the challenge now is to
establish principles of safe
deployment and use in clinical service
design
Statement 7

84
As in so many other fields of
engineering, one of the keys to
effective and safe deployment will be
open technical standards
(covering medical concepts, clinical
vocabulary, task models for example)
Statement 8

85
Adoption of standards will be
necessary but will not be sufficient for
success: another vital challenge is to
persuade the commercial world of
medical IT, publishing, etc. to
develop business models that
accept and build on open standards
Statement 9

86
If information “wants to be free” why
discuss incentives for sharing
anyway?
Statement 10

87
The only types of incentives for
sharing are material, social, or egooriented.
Statement 11



Which of these incentives is more
available (material, social, or ego-oriented)
Which is more likely to generate
results (material, social, or ego-oriented)
Which has more leverage for potential
participating scientists? (material, social,
or ego-oriented)
88
Statement 12

89
Ever since Fred Brook’s “Mythical
Man-Month” vs. Eric Raymond’s “The
Cathedral and the Bazaar”, we’ve
seen a conflict between orderly design
and sharing. Following Brook’s recent
“Design of Design”, should the notions
of iterative design be applied to
sharing; or is the Open Code
approach the way to go?
Discussion
90
Thank you!
91
Rafaeli, S. and Raban, D. (2003) , The Subjective Value of Information: An
experimental comparison of willingness to purchase or sell information,
JAIS: The Journal of the Association for Information Systems (AIS). Vol. 4:5
pp. 119-139
Rafaeli, S. & Raban, D.R. (2003 )
The Subjective Value of Information : Trading expertise vs. content, copies vs.
originals in E-Business, The Third International Conference on Electronic
Business (ICEB 2003), pp. 451-455.
Rafaeli, S. and Raban, D.R. (2005) Information Sharing Online:
A Research Challenge, in the International Journal of Knowledge and
Learning, (inaugural issue), Vol. 1, Issue 1-2, pp. 62-80. ,
Raban, D.R. and Rafaeli, S. (2006) , The Effect of Source Nature and Status
on the Subjective Value of Information , Journal of the American Society for
Information Science and Technology ( JASIST ), Volume 57, Issue 3 (p 321329)
Rafaeli, S., Raban, R.D., & Ravid, G., (2005). Social and Economic Incentives
in Google Answers. ACM Group 2005 conference, Sanibel Island, Florida,
November 2005.
http://jellis.net/research/group2005/papers/RafaeliRabanRavidGoogleAnswers
Group05.pdf
M. Harper, D. Raban, S. Rafaeli, J. Konstan, Predictors of Answer Quality in
Online Q&A Sites. CHI 2008.
D. Raban, M. Harper, Motivations for Answering Questions Online. Book
chapter in New Media and Innovative Technologies (Caspi, D., Azran, T. eds.),
2007.
Rafaeli, S., Raban, D.R. and Ravid, G. (2007) 'How social motivation enhances
economic activity and incentives in the Google Answers knowledge sharing
market', International Journal of Knowledge and Learning ( IJKL ), Vol. 3, No.
1, pp.1-11.