Transcript Enterprise Access to Evidence-Based Literature
“Modules of Workforce Development: Pragmatic Approaches to Knowledge Acquisition, Application, and Information Dissemination”
Enterprise Access to Evidence-Based Literature: A Dynamic Digital Library for Public Health
Enterprise Access to Evidence-Based Literature: A Dynamic Digital Library for Public Health Karen Dahlen, Project Consultant Elaine Martin, Director New England Region NN/LM Lamar Soutter Library University of Massachusetts Medical School This project is currently funded by the NLM via contract through the NN/LM, New England Region
Enterprise Library Approach to Knowledge Acquisition, Application & Information Dissemination
Measure Importance of Trusted Information Access Ensure Direct Access from Intranet-Based Site Access (Without Name/Password Required) Support Core Competencies/Training Use Survey Instruments Adapted to the Location Relate Cost Efficiencies to Enterprise Licensing/Creative Partnerships Document Institutional Change in a Dynamic Environmen t Evaluate Over Time (Illustrate Efficiencies)
Shifting the Culture to Evidence-Based Practice
• Culture of evidence-based practice was affected when funding for public health libraries was eliminated. • State Libraries (in many states) were mandated to support state agencies, but had no collections to support public health.
• Streamlined access to resources through the Digital Library Platform- along with trust in new relationships--prompted resurgence in use of resources and has facilitated this cultural shift.
• Training stimulated access to resources reviving evidence-based interest and relationship to practice.
• Project has the ability to leverage government resources (NLM, CDC, NAL) to improve evidence-based practice.
Background (Why this Project) Project Rationale Partnerships ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Council on Linkages: Academia & Public Health Practice Public Health Competencies
• • •
Competencies: Council on Linkages Analytic/Assessment Skills
– – – References sources of public health data and information.
Information technology to collect, store, retrieve data
.
Utilizes data to address scientific, political, ethical, and social public health issues.
Policy Development/Program Planning Skills
– Analyzes information relevant to specific public health policy issues
Public Health Sciences Skills
– Conducts a comprehensive review of the scientific evidence related to : a public health issue, concern, or intervention. – Retrieves scientific evidence from a variety of text and electronic sources.
• •
Project Tied to Competencies Knowledge Informatics
– Access trusted resources directly from the desktop.
– Describe relationships and functionality of e resources to core public health interests.
– Introduce specific types of resources: coding and classification tools.
Informatics Skills
– Train on interoperable features that support public health work, including connectivity to trusted sites; advanced search filters, use of topical alerts and information mgt software.
Obstacles to Getting Information
Gathering and Analyzing EB Information Rapid Detection of Epidemics
Detecting an anthrax epidemic one day earlier would save $1-7 billion
CDC Presentation by C. Safran citing:
http://www.cdc.gov/ncidod/eid/vol3no2/kaufman.htm
Kaufmann AF, EID, V3, N2
Library Without Signature Building Goal of the Project Who Participates Objectives & Data Sets Development of Digital Library
ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
PHIA: Innovative Business Model Not Tied to Traditional Building
Madison, WI Atlanta, GA
Goal of PHIA
Public Health Information Access
To determine what resources are core, useful, evidence-based to advance public health practice.
Full text directly available through DL
Core to public health
Full text available through LP
Unique Aspects of Digital Library
• Resides on PHD Intranet ( outside a signature building ) without need for id and password.
• Relies on collaboration, interoperability & funding.
• Move toward analysis; less time on access • Skills & training support personalized, interactive styles tied to continuum of learning and institutional specific priorities and projects.
• Collection of evidence occurs in many aspects of Brownson wheel not just at the literature review stage.
• Fundamental change in the way resources are distributed and utilized.
Public Health Departments & Library Partners
State
AK AR CO CT HI IN KY ME MA MA MD NH RI VT WI
WV WA Public Heath Department Library Partners
Alaska Division of Public Health (AKDH) Arkansas Department of Health (ADH) Univ of Alaska (Anchorage) Medical Library Univ of Alabama Medical Sciences Library Colorado Dept of Public Health & Environment Poudre Valley Health System Library Connecticut Public Health Department Hawaii Department of Health Indiana State Department of Health Kentucky Department for Health Maine CDC Boston Public Health Commission* Massachusetts Public Health Department Maryland Dept of Health & Mental Hygiene New Hampshire DHHS Rhode Island Department of Health (RIDH) Vermont Department of Health (VDH) Wisconsin Department of Health Services
West Virginia Bureau of Public Health Washington State Department of Health
University of Conn Maynard Stowe Library University of Hawaii Medical Library Ruth Lily Medical Library, Indiana University University of Kentucky Medical Library Maine Medical Center Library University of Massachusetts Soutter Library Lemuel Shattuck Hospital Library University of Maryland HS Library New Hampshire DHHS Library Rhode Island Hospital/Lifespan UVM Dana Medical Library Ebling Library of the Health Sciences
West Virginia University Libraries University of Washington HS Library
PHIA Evaluation Process Tied to Data Model
To advance access to evidence-based resources in support of improved public health practice
.
What resources are high use? What technical issues arise?
What resources are missing? How can training be improved?
What project components lead to improved efficiencies?
What cost efficiencies are involved with an enterprise approach to digital library implementation, and training. • •
Logic Model
– Resources, activities, outputs, outcomes (minutes, visits, trainings, people lists, IP management, levels of troubleshooting, RML reports.
Data Tools
– Overarching questions – Journal SurveyMonkey – Pre-Post Training Surveys – Just-in-time Information Checks – Formal Evaluation (interviews & focus group sessions
Data Sets & Sources
• • • • • • • • Resource Metrics Collected
Journal Survey Monkey Identifies Perceived Need & Library Use
.
Document IT technical issues. Licensing Issues Related to Journals and Databases. Training Surveys Collected Preliminary Meetings/Updates Held.
Trainings Held Year End Evaluation --Interviews & Focus Groups
View of PHIA Digital Libraries
Connecticut Digital Library Arkansas Digital Library
Resources on Digital Library Use of Resources Article Delivery Use & Costs
ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Databases & Major Publishers of Resources
PubMed , Other NLM Databases NAL, CDC & Other Gov’t Resources CLSI (Clinical Laboratory Standards) Global Health Cochrane Library –
Systematic Reviews
Stat!Ref
—E Books (35) -- Coding & Classification ASABE –Health Linked to Agriculture
Publisher Mass Medical Society (1) AAP(1)
Oxford University Press (20) Springer (18) ASM Journals (12) Annual Reviews (9) Sage (11) BMJ (9) Mary Ann Liebert (11) University of Chicago Press (6) Lippincott/Ovid (6)
Total Use 14,136 9,888
23,369 2, 485 3,518 697 939 2932 1798 955 122
Identifying Core, Useful, & Evidence-Based Journals
Journal Title New England Journal of Medicine Pediatrics
Clinical Infectious Diseases Maternal & Child Health Journal of Clinical Microbiology Annual Review Public Health The Diabetes Educator British Medical Journal Breast Feeding Medicine Infection Control & Hospital Epidemiology J Public Health Mgt Practice
Use 14,136 9888
3147 474 1729 394 135 1832 355 749 55
7000 6000 5000 4000 3000 2000 1000 0
NEJM Use 2010-2013
2010=2087 2011=11,194 2012=14,022 2013=14,136 4864 1772 982 554 314 527 337 311 890 308 1183 1178 494 422
ASM: Journal of Clinical Microbiology
Total Use 2013=1729
298 287 233 175 135 94 90 89 82 78 71 42 32 18 5
Total Use by Each PHD N=13,827 PHD
Alaska DPH Arkansas DH BPHC CDPHE CT DPH HI SDH ISDH KYDPH Maine CDC MDHMH MDPH NH DHHS RIDH VDH WI DHS
Total
1245 1483 182 608 1234 638 700 1134 757 1427 1817 157 620 709 1116
Oxford University Press Use Jan-Dec 2013 High Use Journal Titles Journal Title
Clinical Infectious Disease American Journal of Epidemiology Schizophrenia Bulletin The Journal of Infectious Diseases ICES: Journal of Marine Science JNCI: Journal of the National Cancer Institute Journal of Antimicrobial Chemotherapy International Journal of Epidemiology Alcohol and Alcoholism Journal of Analytical Toxicology Health Promotion International Rheumatology Age and Ageing
Total Use
3147 941 771 627 474 434 297 288 274 262 252 246 240
STAT!Ref E-Books/Tools Provide Basic Understanding of Topic
Filters & Tools
–
Codes[5]
• CPT with RVUs Data File, INGENIX® (2011)[1] • ICD-9+CM - Volumes 1, 2 & 3 (2012) – Titles By Discipline[344] Searchable Alerts (by topic) –
Functionality with NLM
– Chapters link to PubMed – TOXNET linked to Search – Related Concepts/Meta… –
Functionality with CDC
MMWR, Community Guide;
STAT!Ref E- Books for Public Health
Titles Containing PH Content E-Book Use
ACP PIER, Journal Club & AHFS DI Essentials (2013) Oxford Textbook of Public Health - 5th Ed. (2009) Manual of Clinical Microbiology - 10th Ed. (2011) AHFS Drug Information (2013) Red Book: 2012 Report of the Committee on Infectious Diseases - 29th Ed. (2012) Emerging Infections Series (2008 - 2010) 482 407 403 305 241 216 Textbook of Modern Toxicology, A - 4th Ed. (2010) 209
Coding & Classification Books E-Book
ICD-9-CM - Volumes 1, 2 & 3 (2013) CPT with RVUs Data File, INGENIX (2013) ICD-10-PCS: Procedure Coding System (2014) ICD-10-CM: Clinical Modification (2014) Diagnostic and Statistical Manual of Mental Disorders 4th Ed. (DSM-IV-TR, 2000)
Use
881 474 12 15 107
CLSI (Clinical & Laboratory Standards) 20 Documents Used 790 Times April 21, 2014
Doc # Document Name
M100 -S24 M100-S24: Performance Standards for Antimicrobial Susceptibility Testing; Twenty Fourth Informational Supplement EP09 A3 Measurement Procedure Comparison and Bias Estimation Using Patient Samples; Approved Guideline—Third Ed. EP12 A2 User Protocol for Evaluation of Qualitative Test Performance MM0 9-A2 Nucleic Acid Sequencing Methods in Diagnostic Laboratory Medicine; Approved Guideline—Second Edition
#
248 102 55 54
Enterprise Licensing is based on 10% of the Total FTE level to capture baseline information and measure use/interest of resources.
Calculations Related to Enterprise Licensing
State PHD ADH AKDPH BPHC* CDPHE CT DPH Hawaii DPH ISDH Kentucky BPH MDHMH Maine CDC MDPH NH DHHS RIDH VDH WDPHS VW DHS
Washington
Total PHIA FTE FTE 2781/1390 525 1100/550 1200/600 780 3000/1500 780 400 8000/4000 395 3000/1500 279 400 300 400 780
1600/800
10,999 10% Rule 139 52 55 60 78 150 78 40 400 39 150 30 40 35 40 78
80
1546
Article Delivery Costs Via Library Partnerships May 2011 – April 2012; Cost=$18,384 May 2012 --- April 2013; Cost=$12,241 STATE
Connecticut Maine Massachusetts (BPHC) Massachusetts (MDPH) New Hampshire Rhode Island Vermont Colorado
Total ILL
0 560 8 1 0 47 21 323
960 DD
83 364 3 1 0 36 191 156
834 Total 83 924 11 2 0 83 212 479 1794 STATE
Connecticut Maine Massachusetts (BPHC) Massachusetts (MDPH) New Hampshire Rhode Island Vermont Colorado Arkansas Kentucky Wisconsin
Total
0 82 54 112 32 8 0
462 ILL
2 154 9 7 0 79 243 40 73 3 0
593 DD
20 90 10 6
Total 22 244 19 13 0 161 297 152 105 11 0 1055
Training Topics Embrace National and State Interests
• • • • • • Maryland DHMH Health Initiatives Health Disparities(sexual minorities) Preventive Services ACA* HIV Screening Immigrant Health Undocumented individuals under ACA Drug Resistant Disease which impacts Impact on TB, STI (plus others) Health Care Reform and Interpersonal Violence/Domestic Violence Infectious Diseases; Hepatitis C Virus HIV Infections --HIV and HCV Co-Infection
Number of Trainings/Diversity of Workforce
STATE RI VT ME MA MA NH CO CT KY AR WI AK HI IN MD TOTAL PHD RIDH VDH MAINE CDC* BPHC* MPHD* NH DHHS CDPHE* CT DPH* KDPH ADH WI DPH AK DPH HI DPH ISDH MDHMH 15 Trainings 2 3 2 2 2 41 3 4 3 2 2 2 5 5 2 2 Attendance 30 46 52 58 32 24 100 101 46 31 33 50 68 60 38 769 Maryland
Nutrition Specialist (2)
Indiana
State Epidemiologist Research Statistician Supervisor – Labs + (3) Policy Analyst (2) Program Mgr (2) Epidemiologist Medicare Specialist/Advisor (2) Policy Advisor (2) Psychiatric VT Executive Ass’t Deputy Secretary Director VSA Injury Prevention Epidemiologist Staff Attorney Microbiologist ( 2) Regional Program Director Director of Prog Evaluation Director of Prog Development Field Epi Director Access Services Mgr IT Supervisor
Hawaii
Education Coordinator (2) Asthma Coordinators(2 Legislative Liaison Epidemiologist Research Analyst (2) Program Coordinator, SAPB Epidemiologist (3) Genetic Counselor Tobacco Prevent Mgr Informatics Analyst Nutritionist
Enterprise Efficiencies Unique Aspects of the Business Model Evaluation Processes
ENTERPRISE ACCESS TO EVIDENCE-BASED LITERATURE: A DYNAMIC DIGITAL LIBRARY FOR PUBLIC HEALTH
Cost Efficiencies: Article Analysis
Alternative Article Delivery Direct Access to Articles
Contract Year May 2013 –April 2014 Total # Amount $ 17+ 285,409 May 2012-Apr 2013 May 2011-Apr 2012 May 2010-Apr 2011 15 11 9 200,352 156,280 96,252 Contract Year May 2013-March 2014 1038 May 2012-Apr 2013 Total # 1055 Amount $ 10,102 12,241 May 2011-Apr 2012 May 2010-Apr 2011 1794 672 18,385 8,263
Enterprise Licensing Efficiencies: Single Titles Contract Year May 2012-April 2013
Journal Title # Uses Cost Per Use Replacement Cost Cost of License Am J Tropical Med & Hygiene 410 $1.58
$4,510 $ 650 Am J Respiratory & Critical Care Med 189 $14.28
$2,079 $2700 Health Affairs Infection Control & Hospital Epidemiology Public Health Reports 2038 749 658 1.81
1.50
4.69
$22,418 $8,239 $7,238 $3690 $1125 $3085 Intern’l J of Tuberculosis & Lung Diseases Annual Review Series Pediatrics 192 595 7846 5.83
12.60
.68
$2112 $6545 $86,306 $1120 $7500 $5300
PHIA: Innovative Business Model
• • • • • • Enterprise licensing saves time, money, and effort.
Central management of IP addresses ensures quick turn-around.
Ability to understand change (System, personnel, environment).
Resources are identified via benchmarking,metrics of direct use through Digital Library, and requests from PHDs.
Journals are identified via “Article Delivery” on Library Partner side.
Alternative delivery of resources strengthens state relationships with immediate access (within 4-24 hours).
EVALUATION PROCESSES
Levels of Data Collection
• • • • • • • • • • Journal SurveyMonkey (baseline data for PHIA and PHD) Enterprise Licensing (cost effectiveness over time) Vendor statistical reports (validates “use of resources”) Monthly reports from “library partners” (measures use and identify resources to develop Digital Libraries). Ongoing capture of suggested e-books, journals, and databases to enhance collection Pre-Post training links – (data related to knowledge of resources) Feedback from hands-on training Interviews with leaders and workforce Focus group sessions (after one year; subsequently) Quarterly/Annual Reports (submitted to NLM)
Results
• Digital libraries will be in place in 17 PHDs by July.
• Full-text access to more than 150 e-journals, 5 databases, reports and more.
• Introductory sessions have introduced PHD leadership to the project and explained obligations.
• More resources have been added to “digital library.” • Structured training has been held in 15 PHDs.
• More than 600 people have been trained with backup models currently in test stage.
NACCHO Award Brings Recognition to Project
• March 2012 and 2013, the PHIA PHD Digital Library Project received a “promising practice award” from NACCHO.
• Presentations have been held at national and local meetings, e.g., APHA in Washington, DC in 2011 and Boston in November 2013.
• Article was published in AJPH in January 2014. Project was cited in August 2012 EID (CDPHE). • Presentations have been made at national public health meetings, national and regional library meetings.
Testimonial
“Great recognition for this very valuable initiative. Kudos and thank you.”
Jewel Mullen, MD, MPH, MPA,
Commissioner, Connecticut Department of Public Health