Slide 1

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 2

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 3

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 4

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 5

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 6

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 7

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 8

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 9

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 10

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 11

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 12

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 13

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 14

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 15

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 16

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 17

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 18

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 19

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 20

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 21

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research


Slide 22

What Randomized Clinical Trials Are
Possible / Necessary
In Phlebology

Mark H. Meissner, MD
Professor of Surgery
University of Washington School of Medicine

Levels of Evidence for Therapeutic Studies
Straus SE, Evidence-Based Medicine 3rd Ed

Level of
Evidence

therapy, prevention,
etiology,
harm
What Do Studies
WeofReally
Care
About?



1a bestSystematic
with of
homogeneity
of RCTs
The
availablereview
estimate
benefits and
harms
1b
Individual
RCT with
narrow confidence intervals
(estimate
of treatment
effect)
1c



Application of the evidence to the individual patient
2a
Systematic review with homogeneity of cohort studies
(generalizeabilty)
2b



All or none

Individual cohort study or low quality RCT

Incorporation
of societal
values
3a
Systematic
review with
homogeneity of case-control studies
 Societal costs
3b
Individual case-control study
 Comparative effectiveness of different technologies
4

Case series

5

Expert opinion without explicit critical appraisal

Where Does Clinical Evidence Come From?
How Do We Measure the Magnitude of Effect?




Semi – experimental


Comparison with historical controls



Fatally biased

Observational studies (all with concurrent controls)








Cross sectional - Compares proportion with disorder based
on exposure at one point in time
Cohort studies – Prospective evaluation of outcome based on
exposure
Case - control studies – Retrospective evaluation of exposure
based on outcome

Randomized, controlled clinical trials

Determinants of Evidence Quality
Determinant

Definition
Systematic Review of RCTs

Quality

Bias

Treatment
Effect

High

Low

Precise

Randomized Clinical Trials
Observational Studies

Methodology

Cohort Studies
Case-Control Studies
Case Series
Expert Opinion

Unknown
Low

High

Unknown

Consistency Similarity of treatment effect across studies
Directness

Appropriateness of groups and outcomes

RCTs – The Holy Grail


Why are RCTs the holy grail?
 Comparison to standard of care
 Minimizes bias & confounders
 Provides a precise estimate of effect

But …








Require true clinical equipoise (RR 0.4 – 0.9)
Difficult to justify if observational studies show
 Large harmful effects
 Large (risk ratio < 0.4) beneficial effects
 Small beneficial effects (risk ratio 0.9 - 1.0)
Are expensive
May be difficult to generalize (Restrictive inclusion criteria)
Usually not powered to detect harms of treatment
May be better, worse, or complimentary to observational
studies

Not All Questions Require RCTs

This
is Nonsense
“We think
that everyone
might benefit if the most
radical protagonists of
evidence-based
Magnitude of medicine
effect is
organised important
and participated in
a double blind, placebo
controlled, crossover trial of
parachute”
All orthe
None
Phenomenon

Nor Is There An RCT For Every Question
Ioannidis et al: JAMA 2001





48 interventions with randomized and observational trials
Results highly correlated (correlation coefficient - 0.83)
Larger treatment effect in nonrandomized trials

Trial Design
A Continuum Rather Than A Hierarchy
Treatmen
t Effect

Example

Huge
(All or
None)

Parachutes
Epinephrine/An
aphylaxis
UFH/DVT

Standard of Care
Established

Case Series
Standard of Care
Established

Observational
Studies
Case Series
Standard of Care
Established

Large

Bypass for CLI

RCTs

Case Series
Moderate

Statins
HCSE

Observational
Studies

What Are The Important Questions?
Chronic Venous Disease



Is the use of compression prior to intervention cost effective ?
What is the best treatment for C2 & C3 disease?
 Interventions
Compression
Pharmacotherapy (HCSE, MPFF)

Ablation (RF, laser, foam)


Outcomes
Patient important benefits – Pain, quality of life, recurrence
Costs to health care system






Perforating veins
 The pathological perforator – Which are clinically important?
 C5, 6 disease – Healing and recurrence
Is 1st rib resection after a first effort thrombosis warranted?
What is the accuracy of CTV / MRV for iliac obstruction
… And Many Others

What Are The Important Questions?
Chronic Venous Disease


Is there a role for extended prophylaxis other than THR
and malignancy?



Are there ANY prophylactic indications for IVC Filters?



The treatment of acute DVT


Pharmacomechanical thrombolysis

Iliofemoral DVT
Femoropopliteal DVT



Isolated calf vein thrombosis

Is there any role for U/S (using US protocols) in
determining the duration of anticoagulation?
… And Many Others

How Do We Answer the Questions?
Clinical Question

RCT

Observational

Outcomes

Value of Compression (C2)

√

√

QoL, Cost (ICER)

Comparative effectiveness
of different technologies

√

QoL, Cost (ICER,
cost-consequence)

Definition of the pathologic
perforator

√

Ulcer healing /
recurrence

Role of 1st rib resection in
effort thrombosis

√

Recurrent
thrombosis

Extended prophylaxis

√

Recurrent DVT,
Bleeding

Pharmacomechanical lysis

√

QoL, Bleeding, Cost

Calf vein thrombosis

√

Recurrent
thrombosis,
Bleeding, cost

U/S & anticoagulation

√

Recurrent
thrombosis

The CLASS Trial







HTA (UK) funded randomized clinical trial
1000 C 2-6 patients (6 centers)


Saphenous surgery



Foam sclerotherapy



Laser ablation with adjuvant foam sclerotherapy

1º outcomes (6 months, possible 5 yr)


Disease specific – Aberdeen VV Questionnaire



Generic – EuroQol, SF-36

2º outcomes


Validated return to function instrument



Incremental cost effectiveness

ATTRACT TRIAL


692 patients



28 North American centers



Randomized to


Best medical therapy



Pharmacomechanical lysis
Trellis 8
Angiojet powerpulse



Iliofemoral & femoropopliteal arms



Clinically relevant endpoints


Objective PTS (Villalta)



Quality of life

The DiVeTAS Trial – Specific Aims
DIstal VEnous Thrombosis: Anticoagulation vs Surveillance








To compare the short-term efficacy and safety of standard anticoagulation
versus duplex ultrasound surveillance for a first episode of acute
symptomatic DVT confined to the calf veins. The primary endpoint will
be a composite of proximal propagation, symptomatic pulmonary
embolism (PE), major bleeding, and all-cause mortality occurring during
the first 3 months of treatment.
To evaluate the relationship between baseline characteristics, including DDimer and other biomarkers, and the risk of proximal propagation and
other endpoints, with the goal of identifying high risk and low risk subgroups which may differ in treatment efficacy.
To compare long-term outcomes of calf DVT after treatment with
standard anticoagulation versus duplex ultrasound surveillance with
respect to the development of objectively defined PTS and quality of life.
To compare the cost and cost-effectiveness of standard anticoagulation
versus duplex ultrasound surveillance for the management of isolated calf
vein thrombosis.

Comparative Effectiveness Research
The “New” Holy Grail





Background
 Interventional technology – 50% of healthcare resources
(50 million procedures / yr)
 Clinical data in < 15% of 510k approvals
 Adoption after only 10-20% perceived implementation
 Practice integration before value, risks, and costs established
Comparative effectiveness
 “a rigorous evaluation of different treatment options”
(Congressional Budget Office)
 May focus on benefits/risks or cost/benefit
 > $1 billion dollars appropriated by Congress

CDRH Device Classification






Class I
 Low risk devices (tongue depressors, scalpels)
 General controls
Good manufacturing practices
Quality systems regulation
Class II
 Venous lasers, RF devices
 Special controls - Performance standards, registries,
postmarket surveillance
 Most approved through Premarket Notification (510k)
 Safety / effectiveness equivalent to predicate device
Class III
 Insufficient information to ensure safety & effectiveness
 Most approved through Premarket Application (PMA)

Growth in Varicose Vein Treatment

Courtesy of John Mauriello

Economic Analysis*
Model
Economic
Costconsequence

Model-based

Description

Pros / Cons

Quantitative, statistical
Simple, but neglects
analysis of economics
clinical outcomes
only
Economic & clinical
Allows evaluation of
outcomes evaluated in
“trade offs”
common
Previously reported
Flexible, but relies
data used as model
on high quality data
input

* All require data from comparative trials

The REACTIV Trial
Ratcliffe , Br J Surg 2006
Conservative

Surgery

Mean Difference

Mean NHS Cost

£344.53

£733.10

£388.57

AUC SF-6D

1.42

1.50

0.083

ICER *

£4682

* Incremental cost effectiveness ratio





246 patients extensive vv and saphenous reflux randomized to
 Conservative measures (n = 122)
 Saphenous stripping / phlebectomy (n = 124)
24 mo cost effectiveness of £4682 per QALY gained
Below NHS threshold of £20,000 per QALY

Conclusions


The questions are important and need prioritization

But …


The goals, not the methods, are most important






Precise estimates of harms, risks, and benefits
Minimizing bias and unknown confounders

Every question requires a comparison group
An RCT is not necessary, feasible , or even desirable
for every question

Developing Phlebology as a Clinical Science






Demands for industry


Clinical evidence prior to marketing



Research with patient important endpoints

Demands for ourselves


Avoid herd mentality in the absence of data



Pay attention to costs to the health care system



Consider comparative effectiveness of technology

Demands for phlebology


Raise the bar for presentation / publication



Fellowships in epidemiology & health systems research