Launching Adaptive Design at the US Census Bureau

Presented by Michael Thieme, Chief, Center for Adaptive Design Peter Miller, Chief Scientist, Center for Adaptive Design Discussion with the Council of Professional Associations on Federal Statistics December 7, 2012 U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

Center for Adaptive Design Overview

This new Center works in three areas:

Research

Through research, experimentation, and testing, prove that adaptive design principles can be effective at increasing survey and census efficiency and timeliness

Outreach and Education

Using results from research and testing, communicate and demonstrate to survey sponsors and stakeholders that implementing adaptive design is a viable tool for increasing survey and census efficiency and timeliness

Design and Build

Develop a solution architecture, as well as a strategy for building and deploying the systems that enable near-real-time cost–error trade-off decisions in data collection operations U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

2

CAD Research Activities for 2012-2013 •

1. Assembling and assessing data requisite for adaptive design

–

Auxiliary frame data

–

Paradata

–

Survey response data

U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

3

CAD Research Activities for 2012-2013 • 2. Test uses of Frame Enhancements and Paradata in Survey Management – Propensity models in simulation – Business rules in simulation – Business rules in the field – Testing grounds: National Health Interview Survey (NHIS), National Survey of College Graduates (NSCG), others U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

4

CAD Outreach and Education •

Trainings and follow-up for survey teams

•

Development of custom dashboards to display key survey metrics

•

Work with Field on implementation issues

•

Publicize activities and lessons

U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

5

Design and Build • • • Documenting our current architecture to inform where Adaptive Design can be applied Developing adaptive design overall conceptual solution architecture, as well as initial baseline architecture (Spring 2013) Target for initial baseline is to have a functioning system in place for the 2015 American Community Survey (ACS) and the 2014 Company Organization Survey/Annual Survey of Manufacturers (COS/ASM) U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

6

Communication About Adaptive Design • Need to show that adaptive design is not cutting corners • Need to show that it saves money • Need to show that it does not adversely affect estimates, time series, or the job of the analyst • Need to show a comparable product for the money expended U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

7

Precursors to Adoption • Flexibility to give adaptive design a chance to show whether it works or not • Openness to consideration of alternative quality measures (quality of estimates vs. response rate) • Collaborative strategies for testing adaptive design in a variety of surveys across government U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

8

Adaptive Design Challenges •

Legislative and political considerations

• Use of administrative data and paradata • Alternative contact methods • Stopping data collection early • The Decennial Census, the American Community Survey, etc.

U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

9

Adaptive Design Challenges • Transition between fixed and adaptive survey designs at Census: • Mode-switching based on analysis of paradata and modeling • 2015 – American Community Survey (ACS), Company Organization Survey/Annual Survey of Manufacturers (COS/ASM) • Maturing and implementing Adaptive Design business rules (including stopping rules) • 2016 – ACS, testing for 2020 Census, 2017 Economic Census • • 2017 – Economic Census 2020 – Decennial Census U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

10

Moving Forward • Take risks now • Adaptive design is a potential remedy for declining budgets and rising nonresponse • Explore the potential so that when budgets decline we have an understanding of how use it U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

11

Questions and Discussion U.S. Department of Commerce Economics and Statistics Administration U.S. CENSUS BUREAU

12