Transcript Slide 1
2007 Annual Conference “Windows and Continuous CBT: a Brief Overview” George T Gray, EdD ACT, Inc Continuous CBT vs. Testing in Windows • Primary reason for continuous testing • Convenience for candidates • The test can be taken any time Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Reasons for testing in windows instead of continuous testing • Inability to calculate passing score until candidates have tested (new items) • Testing timed to coincide training program graduation • Item exposure - Items seen by same number of candidates but over shorter interval of time • Administrative considerations (updating item pools, registration windows) • Focus on development of a single annual test form • Candidate motivation – less procrastination if there is a time-limited testing event Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Two Exposure Scenarios • Testing one day every three months, item seen one day every 18 months, 4000 candidates, 33% exposure • Testing every day, item seen every day, 1500 candidates over a period of years, <15% exposure Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Where does the balance lie? • No shortage of opinion, but probably no consensus Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Are large item pools necessary for continuous CBT? • Psychometric opinion differs, and some organizations are more concerned about compromise of their item pool by cheating than others Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia My minimum recommendations for continuous testing • Multiple test forms for candidates who retest or pool-based item selection method •Items must have statistics from previous use or at least passing score information associated with them (eg. Angoff ratings) Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia What are certification organizations doing? • An informal survey of certification programs: • 14 organizations offering continuous testing • 4 organizations offering windows of a month or more • 1 organization offering a one-week window • 2 organizations offering a one-day window • 1 organization offering one examination with continuous testing and others with one day windows Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia continuous or windows testing not necessarily related to size of program. • One windows testing program tests tens of thousands of candidates: 2 months online and one month offline. • Some programs that have annual candidate volumes in the hundreds offer continuous testing Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia Speaker Contact Information • George T. Gray, EdD • ACT, Inc. • [email protected] Council on Licensure, Enforcement and Regulation 2007 Annual Conference Atlanta, Georgia