Transcript Cut Score Studies – Case for updating the Process

ASQ Certification Programs
What is ASQ Certification?
A formal recognition by ASQ that an
individual has demonstrated a
proficiency within, and comprehension
of, a specified Body of Knowledge at a
point in time
What is ASQ Certification?
• Internationally accepted and
recognized
• Recognition of excellence for 44 years
• Endorsed by over 125 corporations
ASQ Certification Programs

Certifying individuals for over 44
years

17 various certification programs
from basic to advanced

Certified over 120,000 individuals
globally
Key Benefits to Members
• Gain new skills and upgrade
proficiency
• Source of professional and personal
pride
• Investment in career
Key Benefits to Organizations
• Investment in the company’s future
• Perfect and share new techniques in
the workplace
• Knowledgeable employees that assure
product/service quality
Exam Development Process
• ASQ uses ISO 17024 to develop our
programs.
– International standard that sets criteria for an
organization's certification program for individual
persons.
• Standard used by Certifying Agencies
• Ensures defensibility and reliability
• BoK reviewed every five years
ASQ Certifications
•
•
•
•
•
•
•
•
Quality Auditor
Biomedical Auditor
HAACP Auditor
Calibration Technician
Master Black Belt
Six Sigma Black Belt
Six Sigma Green Belt
Quality Improvement
Associate
• Quality Process Analyst
• Software Quality Engineer
• Manager of
Quality/Organizational
Excellence
• Quality Engineer
• Quality Technician
• Quality Inspector
• Reliability Engineer
• SME/AME/Shingo
Prize/ASQ Lean
Certification
• Pharmaceutical GMP
Professional
History of Certification
Certified Quality Engineer
1968
Certified Quality Technician 1970
Certified Reliability Engineer 1972
Certified Quality Inspector
1984
Certified Quality Auditor
1987
Certified Manager of Quality/
Org Excellence
1995
History of Certification
Certified Software Quality
Engineer
1996
Certified HACCP Auditor
2000
Certified Quality Improvement 2001
Associate
Certified Six Sigma Black Belt 2001
History of Certification
Certified Biomedical Auditor
2002
Certified Calibration Technician
2004
Certified Quality Process Analyst 2005
Certified Six Sigma Green Belt
2006
Certified Pharmaceutical GMP
2009
Certified Six Sigma Master Black
Belt
2010
Steps in Exam Development
New Certification
Job Analysis
Workshop
Test Specifications
Workshop
Body of Knowledge (BOK)
published for Exam
Preparation
Item-Writing
Workshop
Item-Review
Workshop
Exam-Review
Workshop
Review
Camera-Ready
Version
Administer Exam
Item Analysis
& Scoring Changes
Cut-Score Study
Score and
Notify Candidates
Exam Development Process
Exam Development Meetings:
• Job Analysis Meeting (every 5 years)
• Test Specification (every 5 years)
• Item Writing
• Item Review
• Exam Review (prior to each administration)
• Cut Score (after 1st administration)
Job Analysis Workshop
• The results of the Job Analysis is used to
develop a Survey, which is sent to current
holders of the certification.
• Their responses are compiled for the next
step in the process: Test Specifications
12 Subject Matter Experts
– 2 day workshop
Test Specification Meeting
To identify various Tasks and Knowledge
areas that are essential to an individual in
the performance of their professional
duties.
Develop the
Body of Knowledge (BoK) and subtext
12 Subject Matter Experts
– 2 day workshop
Item Writing Workshop
To write Valid and Reliable items, for use in
evaluating a candidate’s knowledge and
ability or skill level on a variety of topics.
16 Subject Matter Experts
– 2 day workshop
Item Review Workshop
To ensure the quality of newly created
questions. Each reviewed for accuracy and
to ensure they are linked to the correct area
of the BoK
12 Subject Matter Experts
– 2 day Workshop
Exam Review Workshop
To produce a final exam that meets test
specifications and accepted assessment
standards
12 Subject Matter Experts
– 2 day workshop
After Exam Administration
• Each examination question is reviewed
using a variety of statistical methods
• Examinee comments are also reviewed
• Questions that did not perform as expected
are reviewed with the appropriate subject
matter experts
• A determination is made on these flagged
items in terms of how to score them
Assessment Concerns: Validity
• Evidence that the test is measuring what it
says it will measure.
• Processes that provide test validity
evidence include:
Test Specifications 
Body of Knowledge (BOK) 
Test Question Classification 
Reference Text & Rationale 
Assessment Concerns: Reliability
Is the behavior being measured consistently?
• Consistency is based on
– Test Length
– Consistency in Test Administration
– Clarity of Questions
– Appropriateness of Questions
Cut Score Study
• Develop a statement describing the
expected performance of a borderline
candidate
• Estimate the difficulty level of each
examination question
• Recommend a Passing Score
12 Subject Matter Experts
– 2 day workshop
Exam Development Overview
• A rigorous process that ensures legal
defensibility and face validity
• Involves 64 Subject Matter Experts for first
examination
• Utilizes customer feedback as well to
continuously improve the process and
examinations
Exam Development Overview
After first administration additional workshops
may be required to ensure appropriate
number of questions in the exam bank:
- Item Writing
- Item Review
Prior to each administration:
- Exam Review
Recertification
Recertification verifies that you have
maintained the same knowledge level
you demonstrated when you originally
passed the exam
Who Has to Recertify?
Manager
CQE
CQA
CBA
CCT
CSQE
CHA
CRE
CPGP
Six Sigma Black Belt
Six Sigma Master Black Belt
Recertification Requirements
18 RU credits over three years
Credits earned through:
Employment
Professional Development
Education
Society Involvement
Certifications
Publishing
Recertification Requirements
Professional Development:
Seminars /
Workshops/Conferences
Society Involvement:
Exam Proctor
Meeting/workshop attendance
Committee member
Officer