Transcript Survey of Tobacco Retail Outlets in 50 United States, DC, and Territories Lily Trofimovich, Karol Krotki RTI International RTI International is a trade name.

Survey of Tobacco Retail Outlets in 50 United
States, DC, and Territories
Lily Trofimovich, Karol Krotki
RTI International
RTI International is a trade name of
Research Triangle Institute
Outline
I. Introduction
1. Synar Amendment and Regulation, Key Players, GAO Report
2. Synar Survey
II. Methodology of the Survey
1. Sampling Design
2. Sample sizes and precision
3. Inspector Characteristics (effect of gender and age on the survey results)
III. Coverage Study
1. Introduction
2. Guidelines
3. State’s coverage studies: Results and Challenges
IV. Results
1. RVRs and SE
2. Trends
V. Conclusions
2
THE SYNAR AMENDMENT
3

In July 1992 Congress enacted the Alcohol, Drug Abuse, and
Mental Health Administration Reorganization Act, which included
the Synar Amendment aimed at decreasing minors’ access to
tobacco products

Synar Amendment: Section 1926 of Title XIX of the Federal
Public Health Service Act

Sponsored by Mike Synar (1950-1996) – late Representative (D)
of Oklahoma

It required states to pass and enforce laws that prohibit the sales
of tobacco to individuals 18 years of age or younger
THE SYNAR AMENDMENT – cont.
To determine compliance, the Amendment requires
each State to:
4

Enact State youth tobacco access control law

Enforce State law

Conduct annual random, unannounced inspections
of retail tobacco outlets

Report these findings to the Human Health and
Services Secretary
THE SYNAR AMENDMENT – cont.
Possible penalty for noncompliance with the
Amendment:
Potential loss of a portion of SAPT Block Grant funding.
5
Players
6

The States

Center for Substance Abuse Prevention
(CSAP)/Substance Abuse and Mental Health
Services Administration (SAMHSA)

RTI provides technical assistance (statistical
consulting) to CSAP:
 Verifying State’s results
 Evaluating sampling designs and coverage studies
 Other
THE SYNAR REGULATION
January 1996:
SAMHSA issued the final Synar Regulation to
provide guidance to States on implementing the
Synar Amendment.
7
General Accounting Office (GAO) Review


GAO examined implementation of the Synar Amendment in 2000 in two
key aspects:

factors that can affect the quality and comparability of the retailer
violation rates reported by the States

whether the States are seeking penalties from retailers as part of
their enforcement strategies
GAO made 3 recommendations to improve Synar implementation and
oversight:
1. Help States improve quality and comprehensiveness of tobacco outlet
lists
2. Develop a more standardized inspection protocol that promotes more
uniform implementation across States and that better reflects research
results
3. Ensure that RVRs exclude invalid inspections
8
SAMHSA Actions to Improve Synar
implementation and oversight
1.
Help States improve quality and comprehensiveness of tobacco outlet lists by:
•
•
•
9
Assisting States assess the accuracy and completeness of lists
through site visits
Requiring coverage studies every 2 years for States using list
frames
Develop new guidance document on conducting coverage survey
(2005)
SAMHSA Actions to Improve Synar
implementation and oversight
2. Develop a more standardized inspection protocol that promotes more uniform
implementation across States and that better reflects research results by:
10
•
Recommending that all States use youth inspectors aged 15 and 16
and monitoring age and gender breakdowns each year as reported
in the Annual Synar Report
•
Requiring States use a standard format for reporting inspection
protocol activities
SAMHSA Actions to Improve Synar
implementation and oversight
3.
Ensure that RVRs exclude invalid inspections by:
•
Instituting review protocols that require States to exclude inspections
that do not include age or gender of minor inspectors or outcome of
inspection
•
Developing a Synar Survey Estimation System (SSES) to assist
States estimate and report their annual RVRs
(FYI: In FFY07, 48 States utilized the SSES)
11
Who is Required to Comply with the Synar
Regulation?
All 50 States, DC and 8 Territories:
12

American Samoa

Commonwealth of the Northern Mariana Islands

Federated States of Micronesia

Guam

Marshall Islands

Palau

Puerto Rico

US Virgin Islands
Objective of the Synar Survey
To determine the retailer violation rate (RVR) for each
State based on random, unannounced inspections of
a sample of tobacco outlets accessible to youth.
The sample of outlets inspected must be representative
of the geographic distribution of tobacco outlets in the
State. The RVR obtained over a number of years will
be used to assess the State’s progress towards
achieving the overall Synar goal of RVR<=20%
13
Sampling implementation steps
14

Develop a sampling frame (which includes over-the counter and
vending machines and includes at a minimum, 80% of the tobacco
outlets in the state) and asses its quality (COVERAGE STUDY)

Determine appropriate sample size for Synar survey to meet SAMHSA’s
precision requirement of having an RVR with no more than 3% margin
of error with 95% confidence

Select a random sample of outlets that reflects the geographic
distribution of outlets accessible to youth

Implement and monitor the Synar Survey within the federal fiscal year
(FFY)

Analyze the results of Synar Survey

Report results to SAMHSA before January 1 of the same fiscal year
Eligibility Requirements
An over-the-counter or vending machine outlet is
eligible for the Synar Survey if it sells tobacco
products and is accessible to youth under age 18.
Ineligible tobacco outlets are bars, taverns, or other
adult-only clubs that have enforced minimum age
restriction for entry of 18 or older.
15
Sampling Frame Considerations

Type of Frame

List Frame (used by the majority of states)
 Business List
 License List

Area Frame (used by two states: MA and IL)

List – Assisted Area Frame (Hybrid Frame)
(implemented by two states)

16
Accuracy/Coverage
Synar Survey Design Options
If SSES is used,
17

Stratified Simple Random Sample with and
without FPC (for census, simple or systematic
random sample)

Stratified Cluster Sample with and without FPC
Sample Size Determination
Three sample size calculations are performed based
on previous year’s survey
 Effective sample size
 Target sample size
 Original sample size
18
Effective Sample Size
Effective sample size:
ne 
1
 ( s.e.) 2
1

 
 P(1  P) N 
where P is the previous year’s RVR
s.e is the standard error of the estimate for 3% margin
of error for one-sided confidence interval (0.0183 for
one-sided)
N is the total number of outlets in the sampling frame.
19
Target Sample Size
Target Sample Size = Effective Sample Size * Design
Effect
Where design effect is the ratio of the variance under
the chosen design over the variance under the SRS
design
The design effect is taken either from the previous
year’s survey (SSES calculates the design effect) or
hypothesized value is used
20
Original Sample Size
The original sample size is determined by:
nt
no  (1  s)
rl rc
where s is a safety margin (10% is recommended), rl is
the expected eligibility rate, and rc is the expected
completion rate.
21
Analysis of Survey Results
Analysis of the inspection results

22
Estimation of three essential items
 Retailer violation rate (RVR = number of
violations over number of eligible completed
inspections)
 Variance or standard error of RVR estimate
 95% confidence interval
Analysis of Survey Results
Analysis of the Inspection Results (continued)

23
Estimation of other required items
 Coverage rate of the survey
 Accuracy rate of the list frame if appropriate
 Completion rate
Coverage Study - Background


24
States can choose

List frame

Area frame

Hybrid (lists within selected PSUs)
HOWEVER, IF LIST FRAME IS ADOPTED, STATE
MUST PROVIDE EVIDENCE FOR QUALITY OF
LIST, ESPECIALLY COMPLETENESS, THAT IS,
COVERAGE.
Characteristics of a List Frame

Coverage - Crucial
A low coverage rate is a potential source of bias,
because the unlisted outlets might be different
from outlets on the list with respect to their
likelihood of selling tobacco to youth.

25
Accuracy – Less important
SAMHSA Requirements
26

Synar sampling frames must conform to the minimum
coverage requirement of 80%.

A frame coverage study should be repeated every 3
years. If a State provides compelling evidence to
SAMHSA that the coverage rate is +90% and the list
frame used is comprehensive and stable, the State
may be allowed to implement the coverage study at
5-year intervals.
Coverage Study Guidelines
27

Area frame - list of easily identifiable and manageable
geographically designed areas that represent the full geographic
extent of the State, without gaps or overlaps

SAMHSA recommends using census tracts (ZIP Codes and
census blocks/block groups could also be used)

Rule of thumb: Area should contain 720 outlets on average

It is important to select a coverage study sample and implement
the sample as close as possible to the time when the Synar
survey is conducted, because the outlet population keeps
changing over time
Sample Design for a Coverage Study – Cont.

To select a sample of areas from the area frame, a
State must decide on the sample design:


28
Design can range from SRS to a more complex
design (stratified or a multi-stage design)
depending on a State’s specific situations.
Stratification always improves the quality of the
design.
Corrective Actions for Low Coverage
29

Improve frame to reach the minimum coverage rate
prior to drawing the Synar sample

Use additional data source in multi-frame mode

Use area frame for the Synar sample itself
State-Level Statistics
FFY 2007 Retailer Violation Rates for 59 Jurisdictions
40.00%
35.00%
30.00%
RVR
25.00%
20.00%
15.00%
10.00%
5.00%
0.00%
59
58
57
56
55
54
53
52
51
50
49
48
47
46
45
44
43
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
Jurisdictions
30
State-Level Statistics
FFY 2007 Standard Errors for 59 Jurisdictions
2.50%
Standard Errors
2.00%
1.50%
1.00%
0.50%
0.00%
1
3
5
7
9
11
13
15
17
19
21
23
25
27
29
31
33
Jurisdictions
31
35
37
39
41
43
45
47
49
51
53
55
57
59
61
National Weighted* Average Retailer
Violation Rate, FY 97–05
Retailer Violation Rate (%)
45
40.1
40
35
30
25.4
25
20.5
20.0
20
17.5
16.3
14.1
15
12.8
12.0
2004
2005
10
5
0
1997
1998
1999
2000
2001
2002
2003
Fiscal Year
*Weighted by State population
Note: All figures include data from the 50 States and the District of Columbia.
32
States are Achieving Very Low Retail
Violation Rates
Breakdown of States Achieving the
20% Retailer Violation Rate, FY05
30
Number of States
22
20
17
10
6
4
2
0
≤ 5%
6-10%
11-15%
Retailer Violation Rate
33
16-20%
> 20%
Coverage Rates as of FFY07
<80% - 6 States/Territories
[80-90)% - 9 States/Territories
[90-99)% - 20 States/Territories
99-100% - 10 States/Territories
No results – 14 States/Territories
34
Conclusions/Next Steps

The National Synar Retailer Violation Rate has significantly dropped since
FY97; almost all States have achieved the 20% target

Most States are verifying the coverage of the frames

Most States implement efficient designs and analyze data using SSES
Next Steps:
35

Accuracy and Coverage of Frames should be assessed by all states
with list frames

Correct and efficient implementation and reporting of the survey
Acknowledgements
Mel Tremper, Jennifer Wagner, Patty Martin
JBS International
RTI International is a trade name of
Research Triangle Institute