Transcript Secondary Database Analysis II

Secondary Database Analysis II
Case-Mix Adjustment
What is the Purpose of Health
Services Research?




Income
Ethnicity
EMR
CPOE
Utilization
A1c
Immunization
Medical Errors
Last Time – Uses
Secondary Data:

Can provide new information on health care delivery
(quality; geographic variation; post-marketing adverse
events; cost), on the natural history of disease, and on
regulatory matters

Can save data-collection resources: time, money,
personnel, participant burden

Can save researcher resources: your time, your money

May permit you to build your CV without anyone’s help
or funds – just your time (but co-authors are a good
thing)

Can provide preliminary data for a grant proposal

May enable research on rare events or difficult
populations
Last Time – Precautions


Take note of the study design; is it
suitable?
Consider: What are inclusion & exclusion
criteria? Why? How does that affect the
sample and generalizability?
This Time – Precautions




What bias may be inherent in the database –
perhaps from the population or from the method
of measurement?
Selection bias: who got included
Migration bias: who was lost/gained and why
Other sources of systematic error (bias)?
This Time – Precautions &
Solutions





Control for bias –
Restrict
Match
Stratify
Adjust with covariates
Risk
Adjustment
Validity
infer
Truth in the
Universe
Truth in the
Study
ERRORS
RESEARCH
QUESTION
infer
Nesting/Clustering of
data
Findings in
the Study
ERRORS
STUDY
PLAN
ACTUAL
STUDY
Target
Population
Intended
Sample
Actual
Subjects
Phenomenon
of Interest
Intended
Variables
Actual
Measurements
EXTERNAL
VALIDITY
INTERNAL
VALIDITY
Bias

Systematic error in measurement or a
systematic difference (other than the
one of interest) between groups

Selection



For cohorts, assembly, migration,
contamination, and referral bias
Measurement
Confounding
Bias: Anticipate and Control






Restriction (may lose generalizability)
Matching (limited # factors)
Stratification
Standardization
Multivariate Adjustment
Assuming the worst (sensitivity analyses)


If needed, conduct sensitivity analyses on
multivariable models
>> Always discuss potential impact of
uncorrected bias on your results
Bias: Systematic Error

Selection Bias





Case-Mix/Disease Severity
Nesting/clustering
Measurement Bias
Recall Bias
Investigator Bias
Example: Hospital Mortality
Report Cards



Originally unadjusted
Hospitals without trauma centers,
doing primarily elective surgery, etc.,
looked really good
Made hospitals who took care of the
sickest of the sick look bad
11/05/03
Quality Assessment

Data Quality: Garbage in, garbage out

Risk Adjustment: To remove the
confounding effect of different
institutions providing care to patients
with dissimilar severity of illness and
case complexity
Risk Adjustment
 Controls for patient characteristics that
are related to the outcomes of interest
 Removes the confounding effect, e.g., of
different institutions providing care to
patients with dissimilar severity of illness
and case complexity
 Addresses regional variations
 Inadequate case-mix adjustment can
lead to misclassification of outlier status
Essential Elements of Risk Adjustment




Outcome-specific
Contains specification of the principal
diagnosis
Contains demographics as proxies for
preexisting physiological reserve
Measures # of comorbidities and
allows all the most important
comorbidities to assume their own
empirically derived coefficients
Additional considerations





– outcome-specific: is one available or do you need to
construct & validate one yourself?
– an index incorporates many predictors – do you need
to study some separately?
– established comorbidity indices for case-mix
adjustment include Charlson, Elixhauser, Selim,
developed on various patient pops – suitable for yours?
– other predictors based on your reading of the literature
– do you need omitted constructs?
– propensity scores (for treatment choice; good for small
datasets, for non-randomized studies of treatment effect)
Risk Adjustment & Outcomes




Primary data collection vs.
administrative data
Disease-specific vs. generic
Commercial vs. developed for your
study
Predictors vary by outcomes being
predicted
Classification of Disease States



ICD-9: too many specific codes (n~10,000)
Clinical Classifications for Health Policy
Research (CCHPR): good for chronic illness
and longitudinal care
[http://www.ahrq.gov/data/hcup/his.htm]
Primary diagnosis: good for studies that
focus on a single episode of care
Famous Methods of Risk
Adjustment

DRGs: diagnosis related groups


APACHE III


Adult ICU
PRISM


Used by Medicare to set hospital
reimbursement
Pediatric ICU
Charlson Score

Adult 1 year survival after hospitalization
Risk Adjustment: Charlson

Advantages:


Commonly used case-mix classification
system in the health care industry
System with which most clinicians and
reviewers are familiar
Risk Adjustment: Charlson

Disadvantages




Principal diagnosis not differentiated
Original work did not specify ICD-9 codes
that went into the disease categories
Developed on inpatients predicting
mortality; may not be well suited to
outpatients at low risk of death
Not good for longitudinal care / chronic
illness
Demographic Factors in Risk Adjusters


Age (e.g., age-adjusted Charlson)
Proxies of Social Support





Marital status
Race
Gender
SES (occupation, employment status,
education)
Proxies of Socioeconomic Status


Health insurance status
Home address zip code average income
Race and Gender


Don’t adjust for automatically
Ideally adjust for variation in the
patients’ physiological reserve and
disease burden but not for variation in
care rendered to patients
Propensity Scores






Useful when sample size is small, to conserve
power
An alternative to including a lot of covariates
Ask: propensity for what?
Include as many predictors as possible to get
predicted probability of group membership
(Rosenbaum & Rubin)
Published schema may include predictors you want
to study separately
Best for non-randomized studies of treatment effect
where you want to adjust for the factors that may
have influenced the treatment choices
Study Design: Minimize Bias
Decision #1:
Experimental Study
Apply intervention,
observe effect on
outcome
yes
Alter events
under study?
no
Observational Study
no
Decision #2:
Make measurement
on more than one
occasion?
yes
Cross-Sectional Study
Each subject examined
on only one occasion
Longitudinal Study
Each subject followed
over a period of time
Case-control
Cohort
Risk adjustment is needed…


When subjects are not randomly
assigned
People do not randomly distribute by



Setting
Provider
Risk Adjust

Outcomes = f (intrinsic pt factors;
treatment applied; quality of treatment;
random chance)
On What Factors Should We Risk
Adjust?

Risk of what outcome?




For what population?




Ejection Fraction
Readmission
Activities of Daily Living (feeding, dressing, etc.)
Inpatient
Outpatient
Nursing Home
For what purpose?



Clinical
Quality
Payment
Classes of potential risk
factors







Demographics: age, sex, etc.
Physiologic status
Number and type of medical diagnoses
Cognitive and mental status
Sensory function
Social, economic, environmental factors
Functional/overall health status
An Important Distinction

Disease Severity
Case-mix or co-morbidity

What is the difference?

Disease Severity v. Case Mix
70
60
50
disease 40
severity 30
20
10
0
Patient A
Patient B
HTN
DM
COPD
case-mix
Arthritis
Nesting or Clustering of
Observations
A threat to validity
Nesting/Clustering of Observations


Traditional methods of analysis assume
observations are independent: people who see
same MD are not!
By setting




Multicenter studies
By clinic
By clinician
Key issue is understanding SOURCE of variance in
the observed data


Within group (clinic)
Between groups (clinics)
Survival by OR Team
10 yr survival
60
50
40
OR 1
OR 2
OR 3
30
20
10
0
0
2
4
6
cases per month
8
10
Common Terminology

Intra-class Correlation Coefficient
(ICC)


The extent to which individuals within the
same group are more similar to each
other than they are to individuals in
different groups
The proportion of the true variation in the
outcome that can be attributed to
differences between the clusters
ICC Values




20 primary care clinics
30-32 patients with type 2 DM per
clinic
For process of care measures (foot
exam, labs ordered etc.) ICC = 0.32
For A1c values, ICC = 0.12
Question: Which is more dependent upon site of care:
processes of care or outcomes of care (A1c control)?
Sample Size Implication

Design Effect = 1 + (n-1)ICC
(n= number of subjects per cluster)

So if estimated sample size of 300 per
group for an intervention study, what
sample size would you need for 30
subjects per cluster (10 clinics) and
ICC of 0.12?
Assignment

Design a study:






Experimental
Cross-sectional
Prospective cohort
Case-Control
What factors you would measure to “risk
adjust” and how measure them?
How would you would adjust for
nesting/clustering of data?
Questions?
Common Research Design
Issues in Health Services
Research