Transcript ACT Evaluation Engine – technology scouting

ACT Project
Methodologies for assessment of
coordinated care and telehealth - IFIC
2014
Cristina Bescos, Helen Schonenberg
7/20/2015
ACT Programme
1
Description
“Identify ‘best practice’ organisational and structural processes
supporting integration and implementation of telehealth in a care
coordination context for routine management of chronic patients”
• The ACT program is an assessment and evaluation
project for Care Coordination & Telehealth Deployment
• Collect and share best practices between members
7/20/2015
ACT Programme
2
Maturity Map
Phase 1: Planning of change
• This phase covers landscaping of the situation, knowledge acquisition, business case analysis and
planning and preparation activities to the level of definition of technical and functional requirements,
service specifications, selection of providers and users, setting the procurement system and staff/user
capability planning.
Phase 2: Adaptation phase
• The second phase includes an initial introduction of changes in a geographical/organisational
environment (one or several projects) for a limited duration pursuing a detailed review of the
intervention and impact before decision to large scale long-term commitment. This phase includes
refinements of the outcomes of the planning phase based on the experience accumulated. The new
models/services could still be stopped without major impact to the rest of the social and HC
organization.
Phase 3: Full scale implementation phase
• Point of non-return reached. This phase covers a full deployment of new services under a sustainable
infrastructure and organization. Services are adequately monitored and evaluated. The services are
running, performing with certain stability.
Phase 4: Continuous improvement after deployment
• The performance of the healthcare system is reviewed and some further changes are introduced to
improve efficiency and efficacy or enlarge the scope/coverage of integrated care. Refinements of
services and new solutions are incorporated in a non-disruptive manner.
3
Phasing
Projects
Healthcare services
Healthcare &
social services
Northern Netherlands
Catalonia
Basque Country
Lombardy
Scotland
Maturity map
(Outcome Barcelona &
Brussels workshop)
Planning of change
Organisational model
NN
Sc
Lo
BC
NN
Sc
Lo
BC
NN
Sc
Lo
Ca
Lo
Ca
Lo
Ca
Ca
Lo
Workforce development
Development of population
stratification tools
Early adoption
Full scale implementation
phase
Continuous
improvement after
deployment
Lo
Ca
Sc
Lo
NN
Integrated care pathways
BC
Sc
BC
Sc
User involvement/Patient
engagement
Support of technology for the
new care model
4
BC
Sc
NN
Sc
Lo
Sc
BC
Lo
Ca
Methodology
• Based on literature, clinical & regional experts,
– Definition, collection, monitoring & analysis of
• Key Performance Indicators that measure CC&TH deployment
success
• Drivers indicators that influence Key Performance Indicators
– ACT Evaluation engine
• Systematic automated way to collect, store and aggregate data
• Provides interactive access for regions and clinical experts
– Analysis results
– Benchmarking within or across regions
7/20/2015
ACT Programme
5
Evaluation Frameworks
Description
Application
INAHTA
EUnetHTA
MAST
NTOIP
International network of
agencies for health
technology assessment
Consortium of 57 non-profit
organizations
European network for health
technology assessment (HTA)
Model for assessment of
Telemedicine applications
National Telehealth outcome
indicators project
Includes 64 HTA Organizations
MethoTelemed study
Whether or not to fund a
Telemedicine application
Assessment of medical and
surgical interventions ad
diagnostic technologies
Decide whether or not to use
new Telemedicine applications
Should we replace an
existing healthcare
application by Telemedicine?
Goal
ACT
• Assess Telehealth
technology
• Explore deployment of
Telemedicine
• Development of a
framework for the
assessment of
Telemedicine applications
Application
Evaluate care coordination
and Telehealth deployment
7/20/2015
Choose the most appropriate
technology to be used
• Develop/ promote good
practice in HTA methods and
processes
• Facilitate local adaptation of
HTA information
• Help reduce unnecessary
duplication of HTA activities
• Publish book with guidelines
Goal
Identify CC&TH best practices
Publish best practicesACT Programme
Facilitate deployment of best practices
• Provide framework for
assessing Telemedicine
applications
• Model for new studies
• Checklist for inclusion of
domains and outcomes in
new studies
• Model for assessment based
on literature reviews
Identification and definition
of a small number of simple
and meaningful outcomes
for consistent application in
Telehealth evaluation
Move focus from easy input
measures and process
outcomes to more complex
challenges
Clarify hierarchical
relationships between
outcome indicator elements
6
Evaluation Frameworks
Framework
INAHTA
EUnetHTA
MAST
NTOIP
• General considerations:
technical assessment,
effectiveness, study design,
quality of life measures,
other outcomes
• Costs: direct, indirect,
intangible
• Economic evaluation
• Sensitivity evaluation
9 domains: health
problem, description of
technical characteristics,
safety, effectiveness, cost
and economic evaluation,
ethical, organizational,
social and legal.
Describes purpose, relevant
alternatives, scope of
assessment and application
maturity
5 domains, outcome
• Category: health status, nonmedical determinants, health
system performance,
community and health
system characteristics
• Theme: quality, accessibility,
acceptability, cost
• Indicator: the variable to
assess
• Measure: the visible effect of
a TH related intervention
• Tool: instruments to collect
data for any single outcome
measure
7 domains: health problem &
characteristics of the
application, safety, clinical
effectiveness, patient
perspectives, economic,
organizational, socio cultural,
ethical and legal aspects
Describes transferability of
results
Notes
ACT
Suitable for development,
initial assessment and long
term assessment
Only suitable for mature
applications.
Outcomes focused, processes
are not included
Framework
Notes
Per CC&TH area: (1) stratification, (2) care coordination &
• Process and outcomes oriented framework
workflow management, (3) patient adherence & staff
• Different maturity levels of CC&TH implementations
engagement, (4) efficiency & efficacy outcomes
Domains based on INATHA, EUnetHTA, MAST, NTOIP and
7/20/2015
ACT Programme
expertice from clinical experts and regions
7
Evidence
Population Stratification
Method
Diseases
Age
Deprivation Index
Past use of HC resources
Drivers
Care Coordination and WF
management
Coordination of care
Org. structure & function
Technology
Care Pathways
Method
Health status
Frailty
Capabilities
Outcomes
7/20/2015
Case ascertainment
Health outcomes
Clinical management goals
Process outcomes
Service utilization
Economic outcomes
Introduction
Belief
Experience with TH technology
Staff Engagement
Leadership
Awareness
Motivation
Workforce development
Creating psychological ownership
Organizational change
Patient Stratification
Efficiency & Efficacy
Patient Adherence
Patient Adherence
Adherence
Satisfaction
Acceptance
Benefits
ACT Programme
8
Challenges
• CC&TH implementation is not led by us,
– Variations: implementation, maturity level, evaluation goals
• Lack of data collection standards / methods
– Variations in interpretation reported values,
– Variations in reporting level (regional, program, org. unit)
• Access to population-level data only
– Limits analysis possibilities
– Due to: lack of direct access, trust, or high effort
• Attitude is changing
• Acceptance of results in clinical domain (no RCT)
7/20/2015
ACT Programme
9