Transcript Grady Barnhill
Slide 1
Slide 2
What is CCT?
CBT = Computer-Based Testing
CCT = Computerized Classification
Test
•
•
•
Competent or Not Competent?
Focus on measurement precision
near cut score
Focus on reliability of decision
2007 Annual Conference
Council on Licensure, Enforcement and Regulation
Atlanta, Georgia
Slide 3
The Testing Programs
(CBT since 9/1/99)
Program
# Items
# Pretest
Items
Time
allowed
RD
100-120
25
3 hours
DTR
80-100
30
3 hours
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 4
The Testing Programs
Program
Total #
Items
Minimum Avg. Item
sec./item Latency
RD
125-145
74 sec.
46 sec.
DTR
110-130
83 sec.
51 sec.
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 5
The Testing Blueprints
Content
Food & Nutrition
Nutrition Srvcs:
Commun./Clinical
Educ. (&
Research)
Foodservice Sys.
Management
RD
15%
DTR
10%
40%
38%
7%
7%
18%
20%
25%
20%
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 6
The Testing Programs
Program
Total #
Examinees/yr % Passing
.
RD
4,221
72%
DTR
351
57%
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 7
Sequential Probability
Ratio Test (SPRT)
RD
DTR
Min. Ability Level
-0.69
-0.34
Zone of Indifference
+/-0.20
+/-0.20
Acceptable Error Rate
5%
5%
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 8
The Pros
Candidates love instant results!
Helps control item exposure
(esp. with no review)
Year-round testing provides
more flexibility - more locations
Computer testing provides more
potential for creativity - fidelity
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 9
The Cons
Quality control usually = once/yr.
Item analysis/review corrective
“for next time” - not linked to
candidate score
Expense
Sometimes variable-length test
seen as disadvantage
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 10
Security
Considerations
Innocent Conversations
Internet Brain Dump sites
Test Prep Agencies
Technology (text message,
camera phones, tie cams)
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 11
Item Exposure
Over-exposure of items at popular ability levels
Computer routines to control item overexposure (Simpson-Header Control)
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 12
Challenges
• Small sample size = Hard to
get sufficient N for item
analysis
• Small sample size = Hard to
get sufficient N for pretest
items
• Small sample size = Hard to
get sufficient $ for program
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 13
Future?
Text
• More candidates!
• Dietetics Education Task Force
• Minimum for 3 Parameter = 300?
• Rasch?
Slide 14
Slide 2
What is CCT?
CBT = Computer-Based Testing
CCT = Computerized Classification
Test
•
•
•
Competent or Not Competent?
Focus on measurement precision
near cut score
Focus on reliability of decision
2007 Annual Conference
Council on Licensure, Enforcement and Regulation
Atlanta, Georgia
Slide 3
The Testing Programs
(CBT since 9/1/99)
Program
# Items
# Pretest
Items
Time
allowed
RD
100-120
25
3 hours
DTR
80-100
30
3 hours
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 4
The Testing Programs
Program
Total #
Items
Minimum Avg. Item
sec./item Latency
RD
125-145
74 sec.
46 sec.
DTR
110-130
83 sec.
51 sec.
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 5
The Testing Blueprints
Content
Food & Nutrition
Nutrition Srvcs:
Commun./Clinical
Educ. (&
Research)
Foodservice Sys.
Management
RD
15%
DTR
10%
40%
38%
7%
7%
18%
20%
25%
20%
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 6
The Testing Programs
Program
Total #
Examinees/yr % Passing
.
RD
4,221
72%
DTR
351
57%
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 7
Sequential Probability
Ratio Test (SPRT)
RD
DTR
Min. Ability Level
-0.69
-0.34
Zone of Indifference
+/-0.20
+/-0.20
Acceptable Error Rate
5%
5%
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 8
The Pros
Candidates love instant results!
Helps control item exposure
(esp. with no review)
Year-round testing provides
more flexibility - more locations
Computer testing provides more
potential for creativity - fidelity
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 9
The Cons
Quality control usually = once/yr.
Item analysis/review corrective
“for next time” - not linked to
candidate score
Expense
Sometimes variable-length test
seen as disadvantage
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 10
Security
Considerations
Innocent Conversations
Internet Brain Dump sites
Test Prep Agencies
Technology (text message,
camera phones, tie cams)
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 11
Item Exposure
Over-exposure of items at popular ability levels
Computer routines to control item overexposure (Simpson-Header Control)
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 12
Challenges
• Small sample size = Hard to
get sufficient N for item
analysis
• Small sample size = Hard to
get sufficient N for pretest
items
• Small sample size = Hard to
get sufficient $ for program
Council on Licensure, Enforcement and Regulation
2007 Annual Conference
Atlanta, Georgia
Slide 13
Future?
Text
• More candidates!
• Dietetics Education Task Force
• Minimum for 3 Parameter = 300?
• Rasch?
Slide 14