Transcript Clinical Trials in Surgery Mazen S. Zenati, MD, MPH, Ph.D.

Clinical Trials in Surgery
Mazen S. Zenati, MD, MPH, Ph.D.
University of Pittsburgh,
Department of Surgery and Epidemiology
Modern Clinical Trials in
History




Clinical trials began in 1800 onward to
proliferate and more attention was paid to
study design.
Placebos were first used in 1863
The idea of randomization was introduced
in 1923.
The first trial using properly randomized
treatment and control groups also featured
double blind assessment was carried out in
1948 by the Medical Research Council, and
involved the use of streptomycin to treat
pulmonary tuberculosis..
The Need for Clinical Trials
in Surgery


In surgery many plausible clinical theory are
institutionalized into practice before as
systematic risk-benefit, cost-effectiveness and
long term quality of life have been measured
and evaluated
Many surgical procedures implemented for
years before we discovered the ineffectiveness
to eventually discredited:




Halsted radical mastectomy
Kidney decapsulation (for hypertension)
Uterine suspension
Legation of internal mammary arteries (for
coronary insufficiency)
The Need for Clinical Trials
in Surgery.. cont.






New surgical procedures are not regulated in the
same way new drugs or devices are
New procedures can be approved in the hospital
based on the surgeon ability to convince the
patients and some time upper management of the
potential benefit of the new idea or the technique.
No enough cases or controls,
Protocols and IRB in many cases are often not
required
No proper documentation of the outcome in
systematic manner
No real plan for study or follow up
The Unique Nature of Surgical
Trials





Blinding of patients or Surgeon is almost
infeasible
Placebo-controlled procedures are ethically
controversial specially if invasive
The placebo effect is powerful in surgical
procedures and should be identify
The bias of the investigator due to the need
for that procedure or program to be
successful
Patient selection, extra skills in performance,
and extra care and attention (efficacious vs.
effective)
The Unique Nature of Surgical
Trials.. cont.



Difficulty in selecting centers to represent
the practicing community (VA sites vs.
academic centers)
Standardization across the sites is difficult
When we compare a well-established
technique with an alternative, the skills and
experience in the first might bias the
comparison (the need for video tapes and
operative reports for QC)
Clinical trial in Surgery
Necessary for




To test the safety and efficacy of a new
technology in the short and long term
outcome
To test if the new technology is Cost
effective
To test if the new technology is at least as
effective and/or safe as existing proven
techniques
To test for the feasibility of implementing
the new technology in relation to the
needed skills/training, equipment, and
other factors.
Barriers to Clinical Trials in
Surgery




Underutilized due to difficulties in design and
recruitments
The high cost: hospitalization, operation,
personnel, and related special method of data
collection and analysis
Funding is difficult to obtain from public and
agencies
Peer-reviewers, IRB, and other over criticisms
(protecting patients right and safety) delay the
process and takes years that the timeliness of
introduction may be lost
Barriers to Clinical Trials in
Surgery.. cont.




Multicenter trials might starts in different
point with unbalance recruitments)
Funding fro the industry carries its own risk
of design and ownership
Mastering complex procedures related to
learning curve
Consent documents very sensitive (the risk
is more of an open ended format)
Forces Opposing Clinical Trials
in Surgery



The magnitude of potential benefits might
subside the necessity for wait and for the
total prove through a clinical trial
The high risk might hold recruitment for a
control clinical trial
The wide range and high number of
potential recipients also a factor in
determine the need compared to a rare
disease that may require special surgical
procedure.
Exception
When we deal with disease that carry
very bad prognosis as in oncology,
cardiology, and transplant where the
intervention is highly morbid and
invasive
 The evidence-based medicine
appraised by patients themselves and
third-party payers might modify a
total embracement traditionally done
by the surgical community

Randomized Control Trial


Can we really have a controls in surgical
clinical trials?
Controls is it:



Not to operate on half of the patients?
Mock operation not related to the indication?
Can we really deny half of the patients in
need for surgical interventions just for the
sake of research?



Are we then doom half of them to their death?
But which half is that? (will be really exposed to a greater
eventful hazard?)
The interventions one or the controls one… do we
really know?
The Basic design of randomized
Controlled Clinical Trial
Population
Eligibility
Criteria
Ineligible
Eligible
Decline to
Participate
Recruitment
Agree to Participate,
Informed Consent
Randomization
Controls
Intervention
Current standard
No therapy/placebo
New surgical
technique
Good
Outcome
Bad
Outcome
Good
Outcome
Bad
Outcome
Barriers to Performing
Randomized Trials
Surgeon bias for or against specific
procedures
 Morbidity associated with surgery
 The for granted acceptance of lesser
forms of clinical evidence by the
surgical community
 Difficulties in recruitments
 Ethics and legality … consents

Threats of the Lack of
Randomized Control Group
Internal validity related to the
temporal trends
 Surgical learning curve effects
 Regression to the mean
 The frequent lack of equipoise among
surgeons
 Amplifying the fact that the clinical
practice outpaces research evaluation

RCT vs. Quasi-experimental
Study Design




RCT minimize the measurable confounding that
potentially biasing the study's conclusions.
RCT reduces selection bias
RCT however, restricts the generalizablity of the
results because of the strict inclusion and
exclusion criteria
Some worthy to consider designs in case of
unlikely to achieve RCT


The use of properly selected nonrandomized control
groups
Using pre and post comparisons
Bias and Validity



Bias: Systematic error within the study that
results in mistaken estimate of the effect of
therapy on disease
Internal validity: The ability of a trial to
come to the correct conclusion regarding
the question being studied
External validity: The ability of trial to
produce results that are generalizable to a
larger population of patients with disease
Formulating Research Question


RCT to answer as narrow as a possible clinical research question
To answer an important question justifying:



Recruitment of large number of patients,
The high expenses of the trial
The need to compare: the exist of legitimate uncertainty between the
effective of at least two therapies of the same disease (clinical
equipoise)



In surgery there are many patients avoid surgery
Many surgeons are and perhaps patients reluctant to relinquish the control
of choice of treatment of deeply held beliefs of most appropriate treatment
Ideal problems to study by RCT:



Similar morbidity of the procedure using for example similar incisions
The disease has high morbidity, new procedure promise to improve
outcome
Disease with current treatment result in potential long term morbidities
that will may justify the risk for exploring a new choice
Issues in Formulating
Research Question
What are the end points
 Can these end points be accurately and
reliably measured
 What potential surrogate end points are
available and at what expenses to the
internal validity
 All the above in consideration to
practicality, expenses, and feasibility of RCT
 Pragmatic trials vs. explanatory trials

Pragmatic vs. Explanatory


Pragmatic: attempt to stimulate clinical realities
more accurately. The outcomes can be easily
generalized and accepted into clinical practice
Explanatory: attempts to answer more specific and
narrow question (strict inclusion and exclusion
criteria, homogeneous population)
Eligibility Criteria






Inclusions and exclusions
Important to determine the external
validity
The study population should not be too
narrow or to broad
It is a balance between statistical
advantage and the generalizability of the
results
Pragmatic trial tend to be broader in
inclusion
Explanatory trials are narrower in focus
Outcome Measurement in
Surgery

Traditional outcome measures:
 Mortality
 Measures


Mortality rate, crude mortality rates, specific
mortality rates, adjusted mortality rates,
standardized
Morbidity
 Measures




Prevalence, point prevalence, period prevalence,
incidence, cumulative incidence, attack rate,
incidence rate
Speed of recovery: ICU LOS, Hospital LOS…
Health related quality of Life
Measuring patients satisfactory
Example on Eligibility

A study in vascular surgery to compare between a newly
MIT and the conventional approach in treating occlusive
peripheral vascular disease of lower extremities





Inclusion criteria patients< 65 yr with clinical and radiographic
evidence
Randomly allocated for the above interventions
End points: angiographic resolution of the occlusion
Typical patients may be older than 65 yr, lack of generalizability
lower the external validity
Eligibility criteria should be more objectively defined


Severity of lesion, symptoms
Endpoint might not be the best to determinant of therapeutic
success:

Patient symptoms, activity levels, and wound healing (more
appropriate and more relevant clinically)
Example on selecting
Endpoint


A RCT to comparing between prothrombin complex
concentrate and FFP in emergency need (surgery/
trauma) for reversing the effect of warfarin. The INR has
been suggested as an endpoint but the FDA did not
accept the proposal and asked for more clinically related
outcome. Any suggestion?
A RCT to test the effect of preoperative oral Citrulline on
surgical wound healing . The speed of wound healing,
strength of the healing or LOS was though of as endpoint
outcomes. Anything wrong with these?
Sample Size




We might need in surgery to conduct first a pilot
nonrandomized observational study on
homogenous population to determine the
differences anticipated in outcome between the
randomized groups in case we do not have
enough data
The above will be also a good source to estimate
the variability in the outcome data: i.e. standard
deviation, or standard error
What is clinically relevant is more important of
just statistically significant in power analysis
Statistical consultation in this regard is very
essential and should be obtained early in the
course of study preparation
Controls and Level of Evidence




Controls to increase the level of evidence without
controls is just as case series study which is important
in surgery to obtain all the descriptive analysis of
outcome but will not provide evidence of superiority
(biased)
Without control we can not account for the placebo
effect
Without control we can not account for confounding
variables
Historical controls might be used frequently in surgery
when a prospective controls is beyond the reach,
specially when performed by the same surgeon and in
the same institution

Population and the time change in technology still differ
Controls and Level of Evidence

Variables that we usually match with in
selecting controls in surgical trials beside all
inclusion and exclusion criteria:





Age, sex, BMI,
Comorbidities, preexisting illness
Disease stage, duration, severity, ISS, mechanism
of injury, site
Surgeon, procedure
Others: smoking, alcohol use, depression, insurance
coverage, etc
“Still the superior level of evidence attributed to RCT is
contingent on a successful randomization”
Randomized Trial in Surgery



Random allocation of the subjects to the trial groups
Breaks the link between any unmeasured confounding
variables and treatment status
Assume all differences in effect between treatment groups
to be a result of the differences in treatment (absent
confounding)

Randomization is the major challenge clinical trials in
surgery and causes considerable weakness the design
when not satisfactory attained


The Challenge is convincing patients to relinquish control of their
care to random process by whether or not they receive surgery or
even which kind of surgery they actually received
Randomized trials in surgery can be either Pragmatic or
Explanatory
Pragmatic vs. Explanatory

It is similar to effectiveness vs. efficacy in
drug trials where:


Effectiveness: is the ability of an intervention to
accomplish its intended outcome in population under
real-life circumstances
Efficacy: Is the ability of an intervention to obtain its
intended outcome under ideal situation
Most randomized trials in surgery tend to be
categorized as pragmatic “effectiveness”,
comparing surgical technique and outcome under
usual clinical conditions
Randomization


Stabilizes the internal validity of the study
Randomization process can be predicted by specific
logistics and practical constraints of the study. The strong
desire to operate or not “by a resident” my tempt for the
selection bias.


The process must be truly unpredictable and concealed from the
investigator. Allocate the patient to randomization after obtaining the
consent. Using random number table/a computer generated and then
enclosed in opaque, serially numbered enveloped.
Investigator selection bias in selecting the intervention
to subjects whenever more than one patient is presented at
the same time as in a busy trauma center for instance

Use distance randomization by phone or internet
“Randomized Trials appear to annoy human nature-if properly conducted, indeed they should”
Randomization, cont..
Block Randomization: to result
equal distributed patients between
the arms of the study
 Stratified Randomization: to
assure certain traits are equal among
treatment groups

Example on Randomization

In a busy trauma center, RCT took place to assess
whether abdominal ultrasound or diagnostic
peritoneal lavage was more accurate in diagnosing
intra-abdominal injury after blunt trauma. In the ER
a clinical indicator was used to determine the need
for rapid abdominal assessment. If met, a resident
would open the next opaque, serially numbered
envelope stored in a designated folder in the ED to
indicate which assessment would be used. In such
busy trauma centers, two or more patients may
presented simultaneously. At the end of trial, it was
noted that, more often than not, diagnostic
peritoneal lavage was performed on thinner
patients!
Example on Randomization


Two cases or more at the same time might
offer the busy resident an opportunity to
open more than one envelope at once and
subsequently decide which patients receive
which assessment.
The process was corrupted by a third part
motivated by self interest. Likely/hopefully
not understanding the potential effects on
the outcome of the trial.
Masking…. Blinding






To prevent or reduce observation bias and placebo effect.
Observation bias: is systematic variation between the true
outcome and the outcome observed (consciously or unconsciously)
Placebo effect: exaggerate the effects of treatment based on
their enthusiasm and expectations
Single blinded: treatment assignment is concealed from subject only
Double-blinded: treatment assignment is concealed from both subject
and the investigator who ascertains the study end point
Contrary to studies of pharmacologic therapies for a
disease, blinding in surgery is not that easy:


Logistic Challenges: we do not have placebo surgery similar to
placebo tablets. We have to use sham surgery
Ethical Challenges: many of the sham-surgery interventions are
considered unethical by today’s standards
Masking…. Blinding.. Cont.


Using blinded third party (not the operating
surgeon) might be necessary
We could verify the effectiveness of
blinding by asking all study participants to
guess which treatment they had received.


If the patient’s guesses were no better than
random chance.. indicated that the blinding
remained successful
Oftentimes, it is not possible to perform a
randomized trial in blinded fashion: different
surgical techniques, therapies, approaches, or
incisions
Example on Blinding

In 1959, Cobb et al. reported a RCT designed to determine
if the internal mammary artery ligation was effective in the
treatment of angina. The researchers enrolled 17 patients
with classic symptoms and recorded preoperative severity
of symptoms as well as stress test electrocardiogram data.
The patients then underwent surgery. After the internal
mammary arteries were isolated, an envelope was opened
and directed to either ligate or simply end the operation.
The patients were informed preoperatively that the ligation
procedure was experimental, but they unaware of the
randomized nature of the study. Post operatively data were
collected and found that the patients undergoing the sham
procedure had greater symptomatic relief than those
undergoing ligation, and one patient in the sham group
had reversal of stress test electrocardiogram abnormalities.
Overall, patients in both groups showed only modest
improvement.
Example on Blinding.. cont.


The importance of sham surgery, without it
(as case series design) the author could
incorrectly concluded that intervention had
a merit
The differences might be related to:



Placebo effect caused the improvement in both
groups
Something beside the trial intervention caused
the improvement
The observed benefit was part of the natural
history of the disease
Patient Crossover and
Intention-To-Treat Analysis


Intention-To-Treat Analysis: Analyze the outcome
based on the original randomization assignments,
regardless of treatment actually received
The alternative: Analyze the data based on the
treatment actually received not by random allocation
(as if here we are breaking the randomization introducing
selection bias since crossover does not occur by chance)



A high crossover rates questioned the validity of the
study
High crossover rates in surgical clinical trials is
common
Especially in trials comparing surgical with medical
interventions
Patient Crossover and
Intention-To-Treat Analysis.. cont.




There is a specific need to multiple
statistical methods and mixed modeling
techniques to deal with crossover
We stick with Intention-To-Treat Analysis
approach whenever randomization is
preserved and in order to keep the
statistical assumptions valid for the analysis
approaches
What is the nonrandom factor that caused
patients to crossover
These nonrandom factors will biases the
study results
Example on Crossover

From 1972 through 1974, the Veterans Administration
Coronary Artery Bypass Surgery Cooperative Study
Group enrolled patients into a randomized study
comparing coronary artery bypass surgery with
medical therapy in patients with angina and
radiographic evidence of CAD. Patients were
randomized to receive either surgical or medical
therapy. After 14 yr of follow up, 55% of patients
assigned to receive medical therapy had crossed over
to receive surgical treatment, whereas 6% of the
patients assigned to receive surgical treatment
decided to not undergo the procedure and were thus
managed clinically. Should we analyze the data based
on the intention-to-treat (original randomization) or
analyze the data based on the treatment actually
received (not by random allocation)?
Example on Crossover.. Cont.


Should we analyze based on the intentionto-treat with such large cross over. Is not
that seem counterintuitive of the
randomization process?
Should we accept to break the
randomization and allowing the
introduction of selection bias by analyzing
according to the actual treatment received?
Conclusions

Surgeons have fallen behind the rest of medical community partly
because of the challenges required to complete a valid randomized trial of
surgical therapy


Recent era of increase in medical evidence demand from policy maker
and caregivers and even patients resulted in an increase in the demand
of randomized clinical trials for publications and quality assurance. And
partly due to our acceptance and reliance on lesser forms of evidence





Clinical Trials in surgery is underutilized
Clinical Trials in surgery is urgently needed
Randomized clinical trials in surgery are not easy but still can be
accomplished with the appropriate planning and selecting suitable design
Good controls, selection of end point, randomization, blinding are the
keys for successful CT
We need to standardized the ascertainment of outcome, attempting to
quantify outcome as objectively as possible
We also should minimize the potential observation bias and placebo effect
by selecting the most appropriate design
Final Thoughts




Only until clinical trails in surgery became the gold
slandered to prove safety and effectiveness of the
treatment, the current practice will continue to be
dominant by opinions or the enthusiasm for a procedure.
Still however the surgeon's Judgment will never be a
dispensable
The results still need to be interpreted in light of the study
design and clinical reasonableness
The better commitment to evidence-based medical
practice and the persistence and the application of novel
approaches to overcoming difficult methodological hurdles
will be continuously necessary to advance medicine
References and recommended
Reading:










John Gallin, Principles and Practice of Clinical Research, Academic
Press, 2002
H. Troidl, Surgical Research, Springer, 1987
David Machin, Text book of Clinical Trials, Wiley, 2006
John Wei, Clinical Research Methods for Surgeons, Humana Press,
2006
Friedman LM, Fundamentals of clinical Trials-Third Edition, PSH
Inc, 1998
Ovid MEDLINE:
http://www.hsls.pitt.edu/resources/ovid/
PubMed:
http://www.ncbi.nlm.nih.gov/sites/entrez?holding=upittlib
FDA: http://www.fda.gov/cder/guidance/iche6.htm
Clinical Research Office for Surgery and Trauma (CROST),
University of Pittsburgh, Department of Surgery:
http://www.surgery.upmc.edu/General/research.htm
Clinical Trials in Surgery