Transcript Document 7607186

Migrating your Paper-n-Pencil
Test to the Computer with
Security in Mind
Jamie Mulkey
Caveon Test
Security
Laura Culver-Edgar
NCCAOM
® Caveon, 2006
Secure Test Administration
Prevention
Detection
Remediation
® Caveon, 2006
Agenda
 Why move to CBT?
 Prevention (Before)
 Policy and process considerations
 Detection (During)
 Monitoring candidates
 Remediation (After)
 Candidate, record, and data
management
® Caveon, 2006
Making the decision to move to
CBT
 Better access and distribution of
testing program
 More efficient testing
 More flexible testing
 Ability to develop/deliver innovative
item types
 Better test security…
® Caveon, 2006
Security Advantages
 Ability to turn items on and off as needed.
 Bad items, compromised items, overexposed
items.
 Unlikely that people will get the same
items
 CBT, especially CAT can help in this regard
 Ability to investigate item-level data
better
 Response time, unexpected change in ability
during the test, etc.
 Faster reporting mechanisms
 Important in cases of irregular behavior
® Caveon, 2006
Test Administration (Before)
 Proper identification procedures
 Define authorized/unauthorized
devices
 Process to report any discrepancies
 Proctors and other test administration
personnel trained
 Interventions are used when suspicious
activity occurs
® Caveon, 2006
Security Measures - Biometrics
® Caveon, 2006
Testing continuously vs. testing
windows
 Size of testing population and the availability of
testing sites/seats
 Length of test and availability of testing
sites/seats
 Monitoring of item exposure through internet
braindumping/forums/chatrooms
 Monitoring of possible item exposure in various
time zones
 Results to candidate are released immediately or
after a period of time
 Size of your item pool
 Test administration: form-based or CAT
® Caveon, 2006
Test Administration (During)
 Managing candidate identification
 biometrics
 Continuous monitoring or examinees
 Personnel, video/audio monitoring
 Physical control over unauthorized
material and devices
 Physical space to hold personal items
 Taking advantage of computer technology
 Managing an intervention when suspicious
activity occurs
 Halt testing, notification and escalation
® Caveon, 2006
Confirming exam exposure
with Data Forensics™
Helps validate level of
exposure
 Has a test been compromised
and to what extent?
 When and where was it
compromised?
 Who are the potential culprits?
® Caveon, 2006
Caveon Case Study: Global Certification
Exams - Effect on Test Performance
® Caveon, 2006
Incidence of Test Abuse by Exam
Exam 425 has experienced the
greatest incidence of test abuse with
suspected test abuse occurring in
~180 of every 1000 tests.
High instance
of collusion
and cheating
indicate
probable sale
and/or sharing
of test content
on the Internet.
® Caveon, 2006
Test Administration (After)
 Relevant data immediately forwarded
to the test sponsor
 Score validation and reporting
 Using Data Forensics to Identify and:
 Invalidate scores
 Manage rogue test centers
 Manage item exposure
® Caveon, 2006
Remediation
 Execution of policies
 Incident response matrix
 Sanctions for poor-performing sites
 Investigation
 Warning
 Closure
 Cease & Desist / Take down
® Caveon, 2006
Tier 1
® Caveon, 2006
Security Planning




Roles and Responsibilities
Security Budget
Policies and Procedures
Putting an overall plan together
® Caveon, 2006
Questions & Answers
® Caveon, 2006
Thanks for attending!
Please contact us:
Jamie Mulkey, Ed.D.
[email protected]
916 652-4017 phone
www.caveon.com
Laura Culver-Edgar, MBA, MT(ASCP)
703 299-5891
[email protected]