Transcript Slide 1
2007 Annual Conference “Get Ready for Continuous Computer Based Testing” Your psychometrician can be used as a flotation device Troy Elliott Association of Social Work Boards Today’s Presentation • • • • I. Idle Ruminating II. Rudderless blathering III. Nebulous opining IV. Insipid Generalizing Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Who We Are • Association of Social Work Boards (ASWB) • 59 members (49 States, D.C., USVI, 8 Provinces) • 4 Examinations for licensure • About 27,000 administrations annually • Other services (model law, trainings, etc.) Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia About the Tests • 170 item (150 scored), 4 option m.c., 4 hrs. • Computer based, continuous • National passing scores • Pre-equated— “unofficial score” available to candidate at test site • ACT, Inc.—psychometric and administrative support (200+ test centers) Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Our Fascinating History, or “Wake Me at the Next Slide” • • • • • • • First test, 1983 Paper and pencil until mid ‘90s First non-p&p – portable device Contract with ACT began in 1997 Continuous CBT began 1998 Blah blah blah Etc. Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia What We Had Going for Us •Relatively short test (170 items) •Relatively simple format (m.c.) •National passing scores; IRT already in use •Steady – growing test population •Budget already “exam heavy” (70 – 80%) •Customer demand (social workers and boards) Orson Welles: someone who had a lot going for him. Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia What we had going against us •Item bank work needed •Generous testing times •Customer apprehension •Culture of accommodation, responsiveness •General naivete Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia One additional challenge - CRC ASWB Candidate Registration Center • All exam registrations come to us • We screen for eligibility to take exam (except in 4 states) • Forward data to ACT, candidates • Candidates schedule appt. with ACT Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Problems and Solutions Item Exposure 3 forms online at all times 3 forms online Pre-equating Pre-equating Aggressive pretesting Aggressive pretesting Strong item production system Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia What it’s All About: ITEMS What it’s NOT all about: The Hokey Pokey Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Getting Items Lots of ways to do this: our process Our words of advice • Be realistic – account for high item attrition Bottom line • A bunch of items isn’t enough Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Test Administration: The Seamy Underbelly TEST SITES Enough? Locations? Multiple users okay? ASWB: 27K administrations = 108,000 hrs./year, about 230 test sites Quality of test sites? Security ? ASWB: extensive security measures through ACT Technology at test sites? Score reporting? ASWB: tests downloaded at appt. time; “unofficial scores”, including diagnostics, at conclusion of test Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Test Administration: The Seamy Underbelly The Front End – Registration Who’s going to do it? You? Your contractor? Someone else? PROS and CONS Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Test Administration: The Seamy Underbelly Ongoing Issues •Continuous testing and the ADA •Continuous testing and ESL •Problems along the way: cancellations, changes, refunds •Problems at the test site •Retakes and waiting periods Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Test Administration: The Seamy Underbelly After the test •Score reporting to boards •“Unofficial” vs. “Official” •Failing score reports •Complaints and appeals Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Hindsight The Good Service to members, service to candidates, steady revenue stream, greater understanding/familiarity with your organization over time The Bad Security risks, infrastructure demands, item demands, costs, dependence on vendors The Ugly Managing expectations, anticipating problems, searching for loopholes Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Hindsight Are YOU ready to take the plunge? •Items—now and ongoing •Equating •National passing score •Registration/scheduling system •Adequate test sites, security •Enforceable policies on retakes, appeals, ADA, ESL, technical problems, etc. Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Speaker Contact Information For more information Troy Elliott Communications Director ASWB [email protected] 1-800-225-6880, ext. 3005 Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia