Transcript Document

Expect the Unexpected: Are We Clearly Prepared?
Leveraging the Internet in Exam
Development and Delivery
Tony Zara, PhD
Ho J. Mun
Pearson VUE / Promissor
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Agenda
• Introductions
• Exam Development
–
–
–
–
Requirements for virtual meetings
Guidelines for success
Disadvantages/advantages of virtual meetings
Regulatory client experiences
• Examination Delivery
• Credentialing Process
• Discussion
– The next big thing
• Conclusions
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Exam Development
• How to coordinate geographically diverse panels
committees, and/or employees?
–
–
–
–
High cost of travel
Time involved in travel
Multiple schedules to coordinate
Jet lag
Sounds like an opportunity for virtual meetings
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Requirements for Virtual Meetings
•
Need to provide an experience more rich than
phone/email
•
Evaluate each meeting type for appropriateness
(e.g., training, item review, standard setting, job
analysis, item writing etc.)
•
Must evaluate capabilities of different
software/systems
•
•
•
•
Bandwidth
Sharing applications
Messaging (public/private)
Plug-ins
•
•
•
•
Security
Asynchronous viewing
Firewall issues
Host/guest connectivity
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Running Successful Virtual Meetings
• Allow extra time
• Decide in advance on the format for each part of the
presentation (self-directed vs. host-controlled)
• Participants login ahead of time
• Have a wingman – multiple hosts allow you to deal
with participants individually
• Distribute necessary print materials (e.g., content
outlines, checklists) prior to meeting
• Review the different software options
• Ensure support is readily available
If this is your first time, conduct a dry run internally
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Advantages of Virtual Meetings
•
•
•
•
Lower costs
Faster turnaround
Shorter, more focused meetings
Access to wider range of experts (allows for
geographic diversity)
• May allow small clients to perform work that they
otherwise couldn’t afford
• May allow side conversations (chat/IM) that are less
disruptive to the whole group
• Easier to include large groups for training
Supplement – not replacement for – face to face (FTF) meetings;
also allows for FTF meetings to be more productive
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Disadvantages of Virtual Meetings
•
•
•
•
•
Technology issues
Lack of face time
No body language/ nonverbal cues
Participants “zoning-out”/multi-tasking
Travel perk for participants
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Client Experiences
• Benefiting Industries
–
–
–
–
–
–
Insurance departments
Real Estate commissions
Appraisers
Nurse Aide boards
Contractor boards
Certification clients
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Examination Delivery
• Types of computer-based testing
– Web-based testing
• Browser-based (connected)
• Internet-based (disconnected)
– Center-based testing
• Third party sites
• Private sites
• Owned/operated sites
– Independent testing
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Web-based Testing
• Browser-based
–
–
–
–
Live, connected testing across the Internet
Either center-based or independent
No standardization of testing stations
Internet performance issues
• Internet-based
–
–
–
–
Internet used to distribute testing information
Center-based, disconnected
Client-server testing center model
Standardization of testing stations possible
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Center-based Testing
• Third party sites
– Can deliver web-based or internet-based testing
– Standardized procedures
– Proctoring
• Private sites
– Corporate environments
• Owned/operated sites
– Most standardized environment
– Optimized for testing
– Professional motivated staffing
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Independent Testing
• Generally browser-based
• Broad distribution of testing
• Standardization issues
• Security challenges
– Unproctored delivery
– Examinee identification
– Testing intellectual property
• Inexpensive assessment events
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Appropriate Applications
• Independent testing
–
–
–
–
Connected, browser-based
Low-stakes assessments
Diagnostic assessments
Practice testing
• Center-based testing
– Disconnected, client server
– High-stakes examinations (licensure, certification,
academic)
– High-complexity examinations
Use the tool appropriate for the assessment situation
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Credentialing Process
•
•
•
•
•
Enhanced communications
Eligibility process
Data transfer
Results processing
State systems
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Discussion
The Next Big Thing?
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Conclusions
• As with all technological advances, the internet is
here to stay in testing
• The Internet is a useful tool for many contexts
• Just because you have a hammer, not all problems
are nails
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia
Contact Information
Anthony R. Zara, Ph.D.
Pearson VUE
952-681-3811
[email protected]
www.pearsonvue.com
Ho J. Mun
Promissor
610-617-5148
[email protected]
www.Promissor.com
Presented at the 2006 CLEAR Annual Conference
September 14-16 Alexandria, Virginia