Transcript Initiating International Collaborative Research Projects

Initiating International
Collaborative Research
Projects in Mental Health:
A Progress Report on the
International Project on the
Effectiveness of Psychotherapy and
Psychotherapy Training (IPEPPT)
International Project on the
Effectiveness of Psychotherapy and
Psychotherapy Training (IPEPPT)

Formally initiated, June 2004, by:


Italian Coordinamento Nazionale Scuole di Psicoterapia
(CNSP; >5,000 therapists)
Italian Federation of Psychotherapy Associations
(FIAP; 21 psychotherapy associations: >10,000
therapists)

General Goal: To improve psychotherapy and
psychotherapy training in a broad range of
theoretical approaches by encouraging systematic
research in therapy training institutes and
university-based training clinics.

Political purpose: Increase status of involved organizations
IPEPPT General Scientific
Steering Committee







Robert Elliott, Scientific Director (University of ToledoUSA)
Alberto Zucconi, Coordinator (University of Siena-Italy)
David Orlinsky (University of Chicago-USA)
Franz Caspar (University of Freiburg)
Louis Castonguay (Pennsylvania State University-USA)
Glenys Parry (University of Sheffield-UK)
Bernhard Strauss (Friedrich Schiller University JenaGermany)
IPEPPT: Current Status





Still in formation stage
Not a single study
The “Project” = Promoting practice-based
research in Europe, North American and
elsewhere
Finding partners/applying for EU funding
Creating/finding tools



E.g., conceptual/organizing concepts
Finding/translating instruments
KU Leuven piloting Dutch-language systematic
case study protocol
IPEPPT Immediate Objectives

1. To construct a list of agreed-upon general
pantheoretical recommendations for evaluating:




2. To facilitate the development of specific treatment
and training outcome protocols for particular:
Therapy approaches (e.g., Systemic therapy)
 Client populations (e.g., people living with schizophrenia)
 Linguistic/national groups (e.g., Italy)
3. To facilitate national/international collaborations


Key aspects of therapy, especially in training centers
Key aspects of therapy training outcome
Not a “Core Battery”
IPEPPT Draft Research
Framework



Such a project requires a guiding conceptual
framework for determining what to measure
and how to measure it
Work-in-progress
8 measurement domains:


4 Research themes
2 Levels (Star design)
Framework: Eight Therapy Measurement
Domains, with examples of key concepts
Research Theme
A. General/
Pantheoretical
B. TreatmentSpecific
I. Therapy
Outcome
e.g., general problem e.g., theory-based
severity
dysfunctional
processes
II. Therapy
Process
III. Client/Therapist Background
e.g., therapeutic
alliance
e.g., demographics
IV. Training
Outcome
e.g., productive vs.
e.g., therapist skill
unproductive practice development
pattern
e.g., therapist
techniques
e.g., preference for
type of therapy
Structure: (1) “Star” Design

Main body of the star = General outcome/
process protocol



Shared by all orientations (General/
Pantheoretical)
Provides common metric
Star rays = Specialized protocols for
different therapy approaches and different
countries (Treatment/Population/Language
Specific)
“Star” Design for Sample Concepts within
Therapy Outcome Domain for Studies of Four
Different Therapies
CBT
Dysfunctional Attitudes
Self-Ideal Discrepancy
Target Problems
Implicit
Cognitive
Biases
Experiential
Experiential
Access
Self-Esteem
General problem
severity
Interpersonal/
relational issues
CCRT
Change
Maturity of
Defenses
Psychodynamic
Level of Object
Relations
Qualitative
perceptions of
change
Relational
Satisfaction
Family
Environment
Interpersonal Empathy
Family/
Couples
Structure: (2) Nested Priority
Lists



Not a single “core battery”
Allow flexibility while encouraging
consistency within & across approaches
Three Levels of Priorities:
1)
2)
3)
Measurement domains are prioritized
Within each measurement domain, key concepts
are ranked by approximate importance
For each concept, available instruments are also
described (researchers prioritize)
Framework: Eight Therapy Measurement
Domains, with examples of key concepts
Research Theme
A. General/
Pantheoretical
B. TreatmentSpecific
I. Therapy
Outcome
e.g., general problem e.g., theory-based
severity
dysfunctional
processes
II. Therapy
Process
III. Client/Therapist Background
e.g., therapeutic
alliance
e.g., demographics
IV. Training
Outcome
e.g., productive vs.
e.g., therapist skill
unproductive practice development
pattern
e.g., therapist
techniques
e.g., preference for
type of therapy
Example: General Therapy
Outcome Domain

Key concepts in a possible recommended
priority order: (“Star”)

(1) General problem severity (quantitative)






Give every 2 sessions to reduce data loss from drop-out
(2) Interpersonal/relational functioning
(3) Qualitative perceptions of change
(4) Individualized problems/goals
(5) Health care utilization/costs
(6) Quality of life/life satisfaction/well-being
Common Client Problem Severity Instruments
In strumen t (length )
Refe rence
Symptom Checklist 90-Revised (SCL-90R) (90 items)
Clinical Outcomes in
Rout ine Evaluation
Outcome Measure
(CORE-OM; 34
items)
Outcome
Quest ionnaire (OQ45) (45 items)
Treatment Outcome
Package (TOP) Adult
Clinical Scales v4.0
(58 items)
S horter
Forms
(length )
Scale Basi s,
Poin ts &
Ti me Frame
C ost
Non Engl ish
Translati on s
10 (incl.
Norwegian)
Derogat is
Brief
et al., 1976 Symptom
Inventory (53
items)
Evans et
CORE-SF
al., 2002
(18 items)
Dist ress
5 point s
Past week
$2 each
(scoring
ext ra)
Frequency
5 point s
Past week
Free
Italian,
Slovak,
Norwegian
Lambert et
al., 1996
OQ-30 (30
items); OQ10 (10 items)
Frequency
5 point s
Past week
German,
Dutch,
Spanish,
Italian
Kraus et
al., 2005
TOP 37 (37
items)
Frequency
6 point s
Past 2 weeks
One-t ime
licensing
fee (e.g.,
$30 for a
student)
Free
Spanish
Framework: Eight Therapy Measurement
Domains, with examples of key concepts
Research Theme
A. General/
Pantheoretical
B. TreatmentSpecific
I. Therapy
Outcome
e.g., general problem e.g., theory-based
severity
dysfunctional
processes
II. Therapy
Process
III. Client/Therapist Background
e.g., therapeutic
alliance
e.g., demographics
IV. Training
Outcome
e.g., productive vs.
e.g., therapist skill
unproductive practice development
pattern
e.g., therapist
techniques
e.g., preference for
type of therapy
Example: General Therapy
Process Domain

Key concepts in possible recommended priority
order:




(1) Therapeutic alliance
(2) Perceived helpful aspects of therapy
(3) Therapist and client response modes
(4) Perceived session effectiveness
Common Therapeutic Alliance Instruments
In strumen t
(items)
Refe rence
S ubscales
In formant
Non -Engl ish
Trans lation s
Horvath &
Greenberg,
1989;
Hatcher,
2005a
Scale B asi s,
Poin ts &
Ti me Frame
Frequency
7 point s
(revised short
form: 5
points)
W orking
Alliance
Inventory
(W AI) (36
items; 12 item
short form)
California
Psychotherapy
Alliance Scale
(CALPAS; 24
items)
Bond
Task
agreement
Goal
agreement
Client
Therapist
Observer
Dutch
Danish
French
Italian
Norwegian
Gaston &
Marmar,
1994
Agreement Disagreement
6 point s
Pat ient
W orking
Capacity
Pat ient
Commit ment
Therapist
Understanding
and
Involvement
W orking
St rategy
Consensus
--
Client
Therapist
French
Portuguese
Italian
Penn Helping
Alliance
Quest ionnaire-II
(Haq-II; 19
items)
Luborsky
Agreement
et al., 1996 7 point s
Client
Therapist
German
French
Norwegian
Dutch
Different Levels of Research
Protocol are Possible
I. Minimum Protocol
 II. Systematic Case Study Protocol
 III. Maximum Protocol
Other Protocols:
 IV. General Training Protocols
 V. Specific Treatment Protocols

I. A Recommended Minimum
Protocol: Applications




Easy to use: Limited to one measure from
each of the first three research domains
Can use with own clients
Provides basic treatment monitoring for
individuals & agencies
Other versions are possible (e.g., different
outcome or process measures)
I. A Recommended Minimum
Protocol: Elements

(1) General therapy outcome instrument



(2) General therapy process


Client problem severity
Give at odd-numbered sessions (short form)
Therapeutic alliance (use short from)
(3) Client/therapist background measure




Standard practice:
Client/ therapist demographics
Client diagnosis, presenting problems
Type of therapy
II. Systematic Case Study
Protocol: Applications
Use for student case study requirements
 Meets emerging standards for systematic single
case research


New online journal: Pragmatic Case Studies in
Psychotherapy (Rutgers University, Editor: Fishman)
II. Systematic Case Study
Protocol: Elements

A. Therapy Outcome:
 (1) Weekly/biweekly outcome measure
+ (2) At least one other quantitative outcome measure
+ (3) Qualitative outcome assessment (e.g., post-therapy interview)

B. Therapy Process
 (1) Therapeutic alliance
+ (2) Detailed record of therapy (process notes and/or recordings)
+ (3) Qualitative perception of helpful aspects (post-session and/or posttherapy)

C. Client/therapist background
 Client/ therapist demographics; client diagnosis, presenting problem;
type of therapy
II. Systematic Case Study
Protocol: Research Questions



(1) Did the client change substantially
over the course of therapy?
(2) If the client changed, did therapy
make a substantial contribution?
(3) What brought about the client’s
changes?
II. Systematic Case Study
Protocol: Emerging Evidence
Standards



(1) Rich case record, including both quantitative &
qualitative data
(2) Replication/convergence across methods
(3) Critical examination of alternative views (e.g.,
Hermeneutic Single Case Efficacy Design, Elliott, 2002):



Non-change explanations (e.g., measurement error)
Non-therapy explanations (e.g., extra-therapy events)
(4) Narrative coherence


Narrative model of predisposing and process factors
Use for generalizing to other cases
III. Maximum Protocol



Include measures of at least one concept in
each of the eight domains
Appropriate for research centers (e.g.,
Center for the Study of Experiential
Psychotherapy)
Also consortia of cooperating centers:

Each center measure some variables
IV. General Training Research
Protocols: Issues


Outcomes of therapy training not well understood
Difficulties:



Lack of agreed-upon measures of therapist functioning
and skill
Must measure therapist change longitudinally over
several years of training
Possible applications:


Use research to improve training
Meet requirements of accrediting and funding agencies
IV. General Training Research
Protocols: Promising Concepts

Quality of therapist professional involvement
and growth (e.g., Orlinsky &Rønnestad;
Collaborative Research Network [CRN])

Qualitative perceptions of effects and
important aspects of training (e.g., qualitative
interviews)


General therapist facilitative interpersonal
skills (e.g., coping with common difficulties)
Change in therapist self concept (e.g., Scilligo,
SASB Introject scales)
V. Specific Treatment Protocols



= Star rays
Applications: For specific theoretical
approaches, client populations, or language
groups
Requires working committee for each group



Identify relevant therapy outcomes, processes,
background variables (or training outcomes)
Do protocol and measure development research
Establish virtual communities for exchanging
V. Specific Protocol Example:


Person-Centered and Experiential Psychotherapy
International Research Group (PCEP-IRG)
Current core members:




University of Toledo (Elliott & team)
Ohio University (Anderson & team)
Katholieke Universiteit Leuven (Leijssen & team)
Universities of Strathclyde & Abertay, Scotland
(McLeod, Cooper)
V. PCEP-IRG Outcome Protocol:
Promising Developments


Center for the Study of Experiential Therapy Research
Protocol (CSEP- 2):
Self-determined problems/goals:


Self-concept (content & coherence)



Qualitative Self-Description interview
Tennessee Self-Concept Scale 2 (long, short forms)
Experiential processing:


Personal Questionnaire (PQ-10)
Toronto Alexithymia Scale (TAS-20)
Need positive mental health measures, self-coherence, etc.
V. Specific Training Research
Protocols

Some Possible Types of Specific Training
Outcomes:

Mastery of therapy theory/knowledge

Case formulation skill (e.g., use clinical
vignettes)
Treatment-specific intervention skill
 Therapist personal development (e.g.,
maturity, identification with orientation,
values)

Promising New Therapy Research
Methods Make this Work Possible





Systematic qualitative research methods
Interpretive single case designs (Fishman,
Elliott)
Using early outcome to identify & repair
problems (Lambert: Signal alarm methods)
New, powerful psychometric methods
(Rasch analysis/Item Response Theory)
Virtual communities (Community Zero)
Lessons Learned So Far







Importance of vision and leadership
Leading by example is important
Both immediate and longterm payoffs offs are
important
These things eat up a lot of time
Grant funding is not necessary and can
complicate broad collaborations
Clinically interesting examples are useful to
persuading skeptical possible participants
Progress is possible in creating a research
climate in training institutes and schools
Invitation to Dialogue - 1
1) Provide comments and suggestions on the
framework & concepts presented here:
[email protected]
2) Form or join online discussion groups or virtual
communities
Closed sites; must apply for membership




General info: www.communityzero.com/ipepp
Example: www.communityzero.com/pcepirp
Open site under development: www.ipeppt.net
3) Begin implementing the minimum protocol design
with your own clients and in your own training
setting.
Invitation to Dialogue - 2
4) Convert traditional case presentation training
requirements into systematic case study exercises
5) Help with translations of key research instruments
6) Contribute to psychometric research:


Improve existing instruments
Equate different instruments for same concepts
7) Collaborate with groups with similar interests to
generate data for pooling.