Transcript BFB 1.3 (2) basic concepts - The Behavioral Medicine

Why Should You Take a Clinical Research Course?
Audiovisual set-up slide:
If you are hearing my voice now, know how to advance slides, and how to
end the slide show, you can skip this slide.
To hear my voice and to see the slides best, you need to view the slides
using Power Point’s “view slide show” setting. When in Power Point, go
to “view” on the top bar of the power point display. Select “slide show” so
the slide occupies your entire screen.
To advance slides, double click the down arrow key on your key board
(not the arrow on the “2” key). You can see and hear the preceding slide
by pressing the up arrow key. The slides are NOT set to advance
automatically so you can spend as long on each as you wish.
To leave the slide show, move your cursor around slowly near the lower
left edge of the screen until a small, translucent box appears on the slide.
Click it for options including “end show”.
If you are in the full screen “view slide show” and your computer’s sound
is on, you should be hearing my voice right now. If you can’t hear me,
something is wrong with your sound system.
Why should you go to all the time and effort to take a
course in Clinical Research??
I’m Rich Sherman.
I have been president of AAPB
and have been involved in
biofeedback for over 30 years
so am familiar with many of
the problems which happen
when people try to perform
biofeedback without an
adequate background. I’m
going to tell you why you need
to know just a bit about
research and the placebo
response to perform
biofeedback effectively.
Richard A. Sherman, Ph.D.
Dean, College of Psychophysiology,
University of Natural Medicine.
[email protected]
(This slide appears in all 4 basic skills talks so skip it if you already read it.)
You are attending one of four brief talks in the series on
“what you need to know to perform biofeedback effectively”.
The series contains talks on why:
1.
You need to know how biofeedback devices actually record physiology and
how to set the devices to produce useful recordings (Biofeedback
Instrumentation).
2. You need to know how the body works in order to use biofeedback effectively to
change physical and mental functioning. Otherwise, you may try to do
training which can’t work well or can’t be done at all (A&P / Human Biology).
3. You need to be able to tell if there is enough solid evidence supporting the use of
a technique for you to even try it, let alone charge for it, and to tell whether
the results you see or hear about are due to placebo or treatment effects
(Basics of clinical research).
4. Everybody needs a basic biofeedback course before doing biofeedback. You
need to know when an introductory level general biofeedback or
neurofeedback course doesn’t provide the depth you need to effectively
incorporate biofeedback into interventions for the kinds of disorders you treat
(e.g. chronic pain, incontinence, muscle rehabilitation, etc.).
(This slide appears in all 4 basic skills talks so skip it if you already read it.)
Before we get started, let’s be sure you and I have the same general
concept of what biofeedback is.
Biofeedback includes (a) performing accurate psychophysiological recordings
using standard techniques to evaluate known problems with levels and patterns of
function of one or more known physiological systems and then (b) immediately
showing these levels and patterns to a client as they are recorded so he or she can
learn to recognize and control them. Numerous placebo controlled studies with
long term follow-ups have proven that biofeedback is effective for many disorders.
The crucial points here are (1) that biofeedback includes actual learning by the
client and (2) that widely recognized problems in function are corrected.
Biofeedback devices are NOT magic machines recording unverified physical
problems using methods not explainable through normal scientific means. They do
not magically cause the body to change its functioning through unproven methods
to fix problems which can not be demonstrated through standard tests. No
scientifically acceptable studies have shown devices making such claims to be
effective.
If you are not familiar with psychophysiological recording and biofeedback, please
see the introductory lecture in the “Want to know what biofeedback and
psychophysiology are?” section on this site.
Credibility
How do you know that the technique you were taught, watched,
heard about, etc. actually works?
You need to know how to evaluate clinical research.
Is the common use of a technique an indication of its efficacy?
No! Increasing numbers of popular techniques are being
shown to be ineffective.
If a manufacturer says it works – does that mean it does?
(Doesn’t the FDA stop anybody from selling a device
that doesn’t work?)
No and NO!!!
If the big person on campus uses it and pushes it, does that mean it
works or just adds to its credibility? Just adds credibility!
Does the FDA control quality of behavioral interventions? NO!
You need to be able to balance all the factors
to make your own decision about whether a
technique is efficacious or even worthy of a
trial
Keeping an
open mind
The weight
of evidence
Many of the techniques currently in use will change over the next
few years. More studies supporting or detracting the use of a myriad
of techniques will be published.
Dozens of new techniques for assessment and control will be
promulgated by your colleagues, clinicians in other professions,
patients, and “superiors”.
You need to know how to read the clinical literature defensively in
order to separate the few gold nuggets from the mass of useless,
misleading chaff.
Thus, you need to be able to recognize a good study when you see
one and - at least as important - recognize the crucial flaws in a
clinical study.
You need to have sufficient skills in searching the literature and
assessing quality to tell when it is worth trying a technique and when
it is worth discouraging the use of others.
Do you actually know how effective you are as a clinician?
Droves of studies conducted by neutral observers, including several performed by
my group, have shown that nearly all clinicians from every field think they are far
more effective than they actually are. It is human nature to remember the
successes and not realize that patients who don’t come back are frequently failures
even through they tell you that they are better as they walk out your door.
My group has the advantage of not only being able to conduct large surveys but
also of having longitudinal records of virtually all of the patients seen in our
medical system (the US military) over decades. Thus, we can easily trace the
progress of large numbers of patients having various disorders through numerous
treatments.
Some years ago, we audited a biofeedback group who had spent nearly a decade
treating a variety of patients with medical conditions. They felt that they were very
successful in treating patients with medical problems such as headache. However,
our audit showed that few of their patients showed improvement with biofeedback
beyond the placebo level and that any changes were usually brief.
Biofeedback has been proven effective for many of the conditions they were
treating. They simply didn’t know how and didn’t recognize their own limitations.
Lack of Credibility
A huge problem for acceptance of biofeedback is the prevalence of
unsubstantiated (unproven, unvalidated) interventions
incorporating an incredible variety of techniques which are talked
about, used, and charged for as though there was a reasonable body
of clinical evidence supporting their efficacy.
The combination of (a) there not being many properly designed
studies with adequate numbers of subjects and sufficiently long
follow-ups to be convincing and (b) the prevalence of
unsubstantiated techniques which are being charged for has led
many health care providers to be very wary of biofeedback.
In this time when previously accepted medical and surgical
techniques are falling by the way-side in droves because good studies
and patient audits are proving their ineffectiveness, the lack of
credibility truly harms the acceptance of biofeedback.
Key elements of a credible clinical study / publication:
1. Adequate diagnosis of the subjects
2. Adequate pre-treatment baseline to establish symptom variability.
3. Objective outcome measures relevant to the disorder.
4. Intensity of the intervention sufficient to produce an effect.
5. Way to check whether the intervention was successful (drug taken
properly, behavioral technique successfully learned and then used).
6. Sufficient patient-subjects so result is credible.
7. Appropriate design for the question (e.g. single group, controls,
believable placebo, etc.).
8. Sufficient descriptive statistics so results are clear.
9. Long enough follow-up so duration of results can be established.
Criteria for accepting a new technique as efficacious:
Many organizations such as the American Psychological
Association & AAPB have adopted requirements such as the
following for determining that a treatment has been shown to
be efficacious:
a. Two studies with appropriate design and controls
(group design or a series of single case design studies).
b. Studies conducted by different researchers.
c. Studies demonstrate clinical efficacy. The new
treatment must be shown to be efficacious in comparison with
medication, a placebo, or another treatment. The treatment
must be shown to be equally effective to an established
treatment for the same problem.
d. Waiting list controls are not sufficient for
demonstrating efficacy. (No expectation effect.)
e. The diagnostic criteria must be clearly specified.
Fatal Flaws in conducting research
Fatal flaws happen when the investigators don’t know enough
about (1) the disorder they are working with, (2) their
recording techniques, (3) assessing outcome measures to ask
the right question, and (4) the basic elements of research
design!
You need to look for these fatal flaws when you read a study
or hear about an idea. If you are going to incorporate a
technique you read or hear about into your practice, you need
to know enough about the above factors to evaluate the idea
and to apply it.
A common fatal design flaw is Failure to anchor the study outcome measures to the
population having the problem being investigated
A good example is an early study on behavioral treatment of cancer.
The design compared a group therapy intervention to the records of similar patients.
The outcome measure was years of survival with significance determined by the
difference in average number of years of survival of each group.
The investigators reported that the group receiving group therapy survived
significantly longer than the control group and concluded that group therapy was
probably the reason for the longer survival.
The investigators did not compare the survival rates of their tiny groups to the huge data
base of similar cancer patients starting at the same stages of the same type of cancer. In
fact, (a) their groups were so small that known variability in survival would have made
it very likely that one group would have an average survival time far longer than the
other just by chance and (b) their failure to review well known life table data on
survival times caused them to miss the crucial point that the apparently longer survival
of several participants was not out of line with the population.
Sadly, the controls died earlier than would be expected.
Numerous studies have now shown behavioral interventions to be useless for
cancer survival.
Failure to analyze the data correctly because the investigators don’t
understand the outcome measures or human variability!
For example, typical ESP studies have participants try to guess which one of five
possible shapes – such as those on the cards shown below – a sender is thinking
about. Each shape is on a card which can’t be identified from behind and there are
usually 25 cards in the deck. Thus, the participant has a one in five chance of
guessing a card correctly. If there is no ESP, the participant would be expected to
randomly guess five of the 25 cards.
If somebody guesses far more or less than 5 they may be declared to have ESP.
Regression to the mean tells us that a greater or lesser number occurs by chance if
sufficient people are tested and that the person guessing unusually more or less
than chance is likely to guess at about chance level at the next test. Unfortunately
for proponents of ESP, this is just what happens nearly all the time.
Know the tools and equipment – the recording methodology!
a. Use of the Minnesota Multiphasic Personality Inventory (MMPI)
to evaluate low back pain.
Ludicrous use pointed out by an orthopedic surgeon who read
the test! Many questions relate to actual pain symptoms!
b. "Alpha" EEG for anxiety: Alpha frequency waves look just like
muscle tension from the eyes! The early biofeedback devices couldn’t tell the
difference and neither could the early clinicians so all they did was teach people to
increase eye muscle tension by rolling up their eyes. The treatments worked because
(1) people learned to sit quietly in a dim room and (2) effective relaxation exercises
were given as homework!
Both “EOG” and
alpha waves look
like this.
Know the physiology and the disorder being worked with!
a. Lack of repeated measures in hypertension studies.
Habituation controls
Untreated controls
BP
Treated patients
1
8
Weekly Sessions
b. Finger temperature baseline instability
(stressed college kids doing class demonstration gradually relaxing – blood flow and temp down with
stress – up with relaxation!)
Temp
Time in minutes
Stressed
relaxed
Know the difference between learning a technique and changes
in intensity of a clinical problem:
The table shows the results of a clinical study which used biofeedback to treat a
disorder. The authors sent only the left column of the table in with their paper for
review and concluded that biofeedback did not work for the disorder. However, I
was able to find sufficient data in the paper to construct the middle and right
columns. The middle column shows the number of subjects who learned the skill vs.
improvement while the right hand column shows the number of subjects who did
not learn the skill vs. improvement. It is obvious that most subjects did not learn the
skill but that most of those who did learn the skill showed clinical improvement
while most of those who did not learn the skills did not show clinical improvement!
Original results of the study
Subjects who learned the skill
Subjects who did not learn
the skill
Subjects who showed no
improvement
30
6
24
Subjects who improved
18
3
21
Non-specific / Placebo effects
The bane of uncontrolled clinical studies!
or
why all the “new” treatments work.
New treatments are initially tested using the pretreatment baseline – intervention –
post treatment baseline (A-B-A) design with no control group. So, changes due to
natural fluctuations in the disorder’s intensity (as with acute low back pain) and
non-specific / placebo effects are missed.
(30% placebo cure rate for headache)
Non-specific effects include patient-therapist bonding, placebo effects, changes
with time, expectations, etc.
Follow-ups are usually too short to observe the placebo effects wearing off.
A good placebo control includes
at least:
1. Treatment expectation
effects.
2. Placebo effect from the belief
that the treatment can work /
is working.
3. Habituation to the treatment
environment and sufficient
duration to elucidate changes
with time.
4. Good therapist-patient
bonding with the therapist
giving general support and
expectation that the treatment
will work.
PL
Rx
The Crucial need for Controlled Studies
Controls - especially realistic placebo controls - are critical because
of (1) the placebo effect and (2) changes in problems with time.
Ex: Dohrman and Laskin (1976) conducted a study of 24 patients
diagnosed as having jaw area pain related to muscle tension.
Eight patients were in a placebo group and sixteen were given
biofeedback.
Three quarters of the patients treated with biofeedback showed
“significant improvement of clinical symptoms and required no
further treatment”.
Sounds great right?
continued
Need for placebo controls - continued
Unfortunately for people who say that controls aren’t needed,
half of the controls had just as good a result!!
There was no long term follow-up so there was no opportunity to
know whether the placebo effect wore off (it usually lasts six months
or so) or whether the pain simply returned on its own.
I strongly recommend that readers unfamiliar with the great power of the
placebo effect on pain read a review of this phenomenon (e.g. Elton et al
1983).
An especially good article is by Finniss & Bendetti (Pain 114, 3 – 6, 2005)
which discusses mechanisms of the placebo response and their impact
on clinical trials and clinical practice.
A good article demonstrating the power of the placebo response is the
one by Price et al (Pain 127, 63 – 72, 2007) which shows that placebo
analgesia results in large reduction in pain-related brain activity in
irritable bowel syndrome patients. This is one of many studies beginning
to come out which show that changing how the brain processes pain
changes pain perception regardless of the source of the pain or the
method used to change the brain’s processing.
Discover and Scientific American magazines have had several articles on
brain scans showing that placebos stop pain by changing processing of
the signals (e.g. Epstein in Discover, page 26, January 2006; Choi in
Scientific American page 36 November 2005; and Ruvin in Discover April
2006).
For more on the brain and pain, you may want to look at Nicoll and Alger’s
article on the brain making its own pain relievers (Scientific American,
December, 2004).
Why open studies don’t show that a technique is actually effective:
Single subject and single group designs are important to demonstrate that a
change in important outcome measures such as pain intensity, ability to walk
further, etc. takes place from beginning to end of the study period so it is worth
continuing to a much more expensive, complex design.
Evaluation of a technique’s efficacy can’t stop at open studies because the change
could just as easily be due to time alone or placebo effects.
This is why single group studies showing that interventions for low back pain such
as (a) chiropractic manipulation and (b) low back surgery help low back pain
patients have little to no credibility in the medical community.
As soon as a control for change with time is introduced, it turns out that subjects
receiving chiropractic do no better that those not receiving any treatment. (Most of
the comparison studies are “population change” based in which changes during a
chiropractic study are compared with changes expected of the population.)
As soon as a comparison control with other treatments is included, it turns out that
surgery for low back pain is no better than well designed, intense behavioral and
strengthening programs. When both are compared with no intervention, their
results are less than impressive.
Fatal flaws in typically tiny “controlled” studies
of behavioral interventions
(1) waiting list control groups (a) do not have expectation / nonspecific / placebo effects and (b) the people on the waiting lists are
frequently very different from those in the study. They have sometimes
turned down participation in the behavioral study, are too poor to make
the required trips, or are involved in other situations.
(2) sample sizes too small to adequately represent true patterns
in the general population of people with the disorder are common. The
typical 10 subject behavioral study is too small to either represent typical
patterns in the population of sick people or to avoid looking good or bad
because of unusual reactions by a few people very sensitive or
insensitive to the treatment.
(3) follow-ups are generally, but not always, too short to observe
placebo effects wearing off.
Why you should not trust the
“weight of clinical experience”
“If the technique is in use now, it must be efficacious” = bunk!
When the idea of washing hands before surgery came out,
“everybody” knew it was dumb and certainly wouldn’t help. It was only
accepted after a comparative controlled study counting surgical survival.
Virtually all of the techniques which were in use around WWI are
gone. Everybody “knew” they worked and laughed at the new ideas.
Nearly all of the drugs and techniques which everybody knew worked
during the WWII era are gone.
How about now? The FDA’s insistence on proving a new drug to
be efficacious is NEW and unique on the face of the planet. It doesn’t
cover surgical or behavioral techniques unless they incorporate devices.
continued
Weight of evidence - continued
Many of today’s accepted techniques espoused by “experts” have
never been subjected to controlled study so nobody knows how many
are actually useless, doing harm, or are relatively worse than
competing techniques.
At least some of the “alternative and complimentary” techniques of
today will survive to be the standard techniques of tomorrow and
we’ll all look back and laugh at the old techniques which we swear
by today.
With a bit of luck, better education of clinicians, and enforcement of
the law, more and techniques will have to prove themselves effective
before they reach wide use.
Weight of evidence - continued
Ex:
Continued use of biofeedback for stroke rehabilitation without
concurrent training to rectify learned non-use.
At least two excellent controlled studies (Basmajian et al 1987, Inglis
et al 1984) and a major review of controlled studies (de Pedro-Cuesta
et al 1992) demonstrated that biofeedback does not add anything to
standard physical therapy for stroke rehabilitation.
One of the studies was done by a world renowned expert in
rehabilitation medicine who is also the father of rehabilitation
biofeedback (Basmajian), so it is very likely that he knew what he
was doing.
Thus, “clinical experience” is not a guide to deciding whether a
treatment is effective.
Credibility alert
While you are listening to discussions of alternative techniques,
please keep credibility in mind.
Positive signs of credibility to watch for during this discussion
include:
(1) numerous articles by different authors supporting use of a
technique
(2) when many articles have good, realistic, placebo controls
(3) the articles are double blind – not single blind and the
evaluations are done by a neutral team
(4) the patients were randomly assigned to the alternative technique
(not people who showed up wanting it)
(5) the articles are published in mainstream journals with high
reputations rather than only in a journal supported by
practitioners of that technique
Well, I certainly hope I’ve provided some convincing support for the
idea that people need a course in clinical research to do a good job at
biofeedback – or at least given you enough hints so you can do your
job better after attending this brief talk.
Just a note on typical clinical research courses: Most are designed
for people who are going to be biologists or perform psychological
studies. They don’t concentrate on how to recognize or perform good
studies in the real biofeedback clinical environment without a ton of
money and time. So, be careful to take a course relevant to your
interests.
If you have questions – especially about the best ways to learn this
material if you are going to be doing biofeedback rather than basic
research, please feel free to contact me.
Thanks for listening and, especially, for keeping an open mind!
Rich Sherman
[email protected]