Transcript S8_Cohen

The Impact of Survey Design Modifications
on Health Care Utilization Estimates in a
National Longitudinal Health Care Survey
Steven B. Cohen, Ph.D.
Trena Ezzati-Rice, M.S.
Marc Zodet, M.S.
Presentation
 Need for essential data on health care utilization to






inform health care policy and practice
Description of the Medical Expenditure Panel Survey
(MEPS): purpose, design and analytical capacity
Nonresponse and post-stratification adjustments
Recent survey design modifications:
(1) CAPI upgrade; (2) Sample Redesign
Evaluation of impact of design modifications on health
care utilization estimates
Impact of design modifications on model-based
analyses of health care use
Discussion
Medical Expenditure Panel
Survey (MEPS)
Annual Survey of 14,000 households:
provides national estimates of health care
use, expenditures, insurance coverage,
sources of payment, access to care and
health care quality
Permits studies of:
 Distribution of expenditures and sources of
payment
 Role of demographics, family structure,
insurance
 Expenditures for specific conditions
 Trends over time
Key Features of MEPS-HC
 Survey of U.S. civilian noninstitutionalized population
 Sub-sample of respondents to the National Health
Interview Survey (NHIS)
 Oversample of minorities and other target groups
 Panel Survey – new panel introduced each year
– Continuous data collection over 2 ½ year period
– 5 in-person interviews (CAPI)
– Data from 1st year of new panel combined with
data from 2nd year of previous panel
MEPS Overlapping Panels
(Panels 13 and 14)
MEPS Household
Component
MEPS Panel 13 20082009
1/1/2008
NHIS
2007
Round 1
1/1/2009
Round 2
Round 3
NHIS
2008
Round 4
Round 1
Round 5
Round 2
Round 3 Round 4
MEPS Panel 14
2009-2010
Round 5
HC - Purpose
 Estimates annual health care use and




expenditures
Provides distributional estimates
Supports person and family level
analysis
Tracks changes in insurance coverage
and employment
Longitudinal design; linkage to National
Health Interview Survey (NHIS)
Tool Chest of Methods to Maximize
Survey Response
 Recruitment of experienced interviews and bilingual
 10+ days training (including procedures for obtaining







signed consents)
Uses of MEPS data as reference materials for
interviewers
Periodic retraining and special trainings (e.g. methods
to improve response rates)
Respondent remuneration
Advance mailings from co-sponsors of survey
Monthly planning calendar and MEPS DVD
Daily emails to interviewers regarding interviewing
progress
Multiple contacts for refusal conversions
MEPS Response Rates
 Multiplicative response rates (RR): product of
 NHIS RR and
 MEPS RR (multiplicative function of round specific
RR):
 MEPS rounds 1-3 of new panel (YR1 estimates)
 MEPS rounds 3-5 of old panel (YR2 estimates)
MEPS Response Rates
(RR)
 Overall annual RR (~65%)
 Highest RR 1st year, new panel (~66-71%)
 Lowest RR 2nd year, old panel (~63-65%)
 Post-survey nonresponse adjustments
– Dwelling unit level
– Person level survey attrition
NHIS variables used as potential
covariates in forming DU
NR adjustment cells
SocioEconomic
Status
Geographic
Demographic
Household
Characteristic
Age
DU size
Poverty status
Census region
Health status
Race/ethnicity
Has phone
Education
MSA size
Need help
Marital status
Working/reason
not work (e.g.,
attending
school, retired,
etc,)
Income
MSA/nonMSA
Gender
Type of PSU
Employment
status
Urban/Rural
Any Asian
Any Black
Health
NEW NHIS variables added as potential
covariates in forming DU NR adjustment cells
SocioEconomic
Status
Demographic
Household
Characteristic
Interview
language
Type of home – Category of
house, Apt., etc. medical
expense
Number of
nights in
hospital
U.S. Citizenship
Time no phone
Healthcare
coverage
Born in US
Home
ownership
Health
Adjustment factor

Within each adjustment cell:
A(c) =

i c
E ( i )W 1( i )

i c
R ( i )W 1( i )
ratio of the sum of weights of all eligible (E) units
in the cell to the sum of weights of only the
respondents (R) in the cell
Person Level Adjustments:
Annual Estimates
 Each panel weighted separately
 Nonresponse adjustment for survey
attrition
 Final Poststratification adjustment –
CPS 12/31:
age, race/ethnicity, sex, region, MSA
status, poverty status
Person Level (survey attrition)
Nonresponse Adjustment Covariates

Factors associated with survey attrition (after R1)
– Indicator for initial refusal to R1interview
– Family size
– Age
– MSA, census region
– Marital status (family reference person)
– Race/ethnicity
– Education of reference person
– Employment status
– Health insurance status
– Total expenditures (in yr 1 for yr 2 adj.)
– # doctor visits (in yr 1)
– Self reported health status
Longitudinal Estimation Strategy
2009
Round 1
Round 2
Individuals in the
2009 sample with
positive weights that
left the civilian
population prior to
2010, with no return
2010
Round 3
&
Round 4
Round 5
2009 sample also
responding in
2010 with
complete
information for
both 2009 and
2010
MEPS Redesign in 2007
 Re-engineered CAPI Interview: Windows-based
Platform replaces DOS-based system for Panel 12
 New NHIS Sample Design Introduced in 2006: MEPS
Panel 12 selected from redesigned NHIS sample
 Year 2 of MEPS Panel 11 based on original MEPS
survey design
 The overlapping panel structure in MEPS allows for a
comparison of survey estimates across the alternative
designed for the same time period
Evaluation of Concordance of Healthcare
Utilization Estimates: Comparison of results
from new and original designs
 MEPS has overlapping panel design: 1st year of new
panel combined with data from 2nd year of previous
year’s panel to yield annual data
 Multiplicative response rates: product of NHIS RR and
MEPS RR (multiplicative function of round specific RR:
3 rounds for new panel/5 rounds for old panel)
 Detailed adjustments for survey nonresponse and
poststratification:
 Compare 2007 health care utilization estimates based
on new design (MEPS Panel 12 – Year 1) with original
design (MEPS Panel 11-Year 2)
Testing for Survey Redesign Effects
Comparisons of panel specific national health care utilization
estimates derived from the MEPS for the following health care
services:
 ambulatory visits (office- based visits and outpatient facility
visits)
 in-patient stays
 ER visits
 dental visits
 prescribed medicine purchases
For the overall population, and further subset by age
classification (0-17, 18-64, 65+)
 Model-based tests for survey redesign effects
Capacity of MEPS to Produce
Comparable NHIS Estimates of
Health Care Utilization
The following NHIS measures of health care utilization
were selected in support of these analyses:






Have you been hospitalized OVERNIGHT in the past 12 months?
(yes; no; refused/not ascertained/DK)
How many different times did you stay in any hospital overnight or
longer DURING THE PAST 12 MONTHS? (#; refused/not
ascertained/DK)
Altogether how many nights were you in the hospital DURING THE
PAST 12 MONTHS? (#; refused/not ascertained/DK)
During the past 12 MONTHS did you receive care from doctors or
other health care professionals 10 or more times? Do not include
telephone calls. (yes; no; refused/not ascertained/DK)
DURING THE PAST 12 MONTHS, have you delayed seeking medical
care because of worry about the cost? (yes; no; refused/not
ascertained/DK)
DURING THE PAST 12 MONTHS, was there any time when you
needed medical care, but did not get it because you/the family
couldn't afford it? (yes; no; refused/not ascertained/DK)
Options for aligning redesign-based estimates with the original design
Period of
Applicability
Constraints
Restrict time trend
analyses to sample
with old design
Year(s) with
overlap between
redesign and prior
design
Loss in precision
Implementation of
measure specific
adjustments to a
set of estimation
weights
When the survey
redesign is
implemented and
subsequent years
Introduction of greater variability in resultant
survey estimates ; complicates model based
analyses
Implementation of
“bridging”
adjustments to the
primary survey
estimation weight
When the survey
redesign is
implemented and
subsequent years
Dependence on within survey adjustments or
availability of comparable external data source
for national control totals
No additional
adjustments for
redesign
When the survey
redesign is
implemented and
subsequent years
Possibility of differences detected in trends
partially attributable to redesign
Option
Summary
 Need for accurate and reliable national data
on health care utilization to inform policy and





practice
MEPS design features and analytical capacity
Statistical, methodological and operational
design features to adjust for nonresponse and
attrition
Evaluation of impact MEPS redesign on health
care utilization estimates
Impact on model based studies
Some evidence of redesign effect
Strategies to Improve
Accuracy
 MEPS includes a linked survey of medical
providers for expenditures: use of medical
event information to evaluate household reports
of health care use
 MEPS data periodically linked to Medicare
claims data for evaluations: permits
examination of accuracy of household reported
data
 Implement additional improvements to the CAPI
interview and enhanced post-survey adjustment
strategies