Transcript Evaluate and Select an Assessment Instrument or Strategy

Chapter 7
Selecting, Administering, Scoring, and
Interpreting Assessment Results
Assessment Procedures for Counselors and Helping Professionals, 7e
© 2010 Pearson Education, Inc.
All rights reserved.
Selecting Assessment Instruments & Strategies
1. Identify the type of information needed
2. Identify available information
3. Determine the methods for obtaining
information
4. Search assessment resources
5. Evaluate and select an assessment instrument
or strategy
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-2
© 2010 Pearson Education, Inc.
All rights reserved.
Identify the Type of Information Needed


Any assessment instrument or strategy is
useful only if it provides the information
required for the assessment.
Counselors determine what information to
collect based on the purpose for assessment.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-3
© 2010 Pearson Education, Inc.
All rights reserved.
Identify Available Information
Pre-existing assessment information about
clients can include:





intake questionnaires
biographical data
preliminary diagnoses
school grades
previous assessment results
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-4
© 2010 Pearson Education, Inc.
All rights reserved.
Determine the Methods for
Obtaining Information

Counselors can choose from both formal
and informal assessment instruments and
strategies
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-5
© 2010 Pearson Education, Inc.
All rights reserved.
Search Assessment Resources
References sources




Mental Measurement Yearbook
Tests in Print
Test Critiques
Tests
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-6
© 2010 Pearson Education, Inc.
All rights reserved.
Test Publisher Catalogs

Publisher catalogs provide important test
information including:



Cost of Materials and Scoring
Types of Scoring Services
While publisher catalogs are a valuable source
of information, bias may exist. Catalogs
should be used along with other sources of
information.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-7
© 2010 Pearson Education, Inc.
All rights reserved.
Test Manuals


Test manuals are a primary source of
information when selecting tests
The Standards for Educational and
Psychological Testing (1999) provides a
list of key components that should be
included in test manuals.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-8
© 2010 Pearson Education, Inc.
All rights reserved.
Research Literature


Academic journals dedicated to the
exploration of testing and assessment
can be valuable resources in test
selection.
Individual journal articles may explore
the use of specific tests in various
settings.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-9
© 2010 Pearson Education, Inc.
All rights reserved.
Professional Organizations






American Educational Research Association
Association for Assessment in Counseling &
Education
National Center for Research on Evaluation,
Standards, and Student Testing (CRESST)
National Council on Measurement in
Education (NCME)
American Counseling Association
American Psychological Association
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-10
© 2010 Pearson Education, Inc.
All rights reserved.
Internet Resources



Online test locator services can be valuable
resources for selecting tests.
Buros Institute
(http://www.unl.edu/buros/)
Educational Testing Service (www.ets.org)
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-11
© 2010 Pearson Education, Inc.
All rights reserved.
Directories of Unpublished Instruments


Directory of Unpublished Experimental
Measures (2007)
Measures for Psychological Assessment: A
Guide to 3,000 Original Sources and Their
Applications (1974)
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-12
© 2010 Pearson Education, Inc.
All rights reserved.
Evaluate and Select an Assessment
Instrument or Strategy
1. What is the purpose of the instrument? Who is the intended
2.
3.
4.
5.
6.
7.
population?
What sample of the population was used in instrument development,
validation, and norming?
Are the results of the instrument reliable?
Do the instrument’s results have evidence of validity?
Does the instrument’s manual provide clear and detailed instructions
about administration procedures?
Does the manual provide sufficient information about scoring,
interpreting and reporting results?
Is the instrument biased?
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-13
© 2010 Pearson Education, Inc.
All rights reserved.
Evaluate and Select an Assessment
Instrument or Strategy cont.
8. What level of competency is needed to use the
instrument?
 Test User Qualifications
A-level: no advanced training required
B-level: master’s degree in counseling, psychology,
education or related area, or have completed
specialized training or expertise
C-level: doctorate in psychology, education, or a related
field, or licensure.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-14
© 2010 Pearson Education, Inc.
All rights reserved.
Evaluate and Select an Assessment
Instrument or Strategy cont.
9. What practical issues should be
considered for this instrument?
Time required for administering the instrument.
b. Cost of the instrument
c. Format
d. Readability
e. Ease of Administration
f. Ease of Scoring
g. Ease of Interpretation
a.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-15
© 2010 Pearson Education, Inc.
All rights reserved.
Administering Assessment Instruments
Modes of Administering Instruments:








Self-Administered
Individually Administered
Group Administered
Computer Administered
Video Administered
Audio Administered
American Sign Language
Nonverbal
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-16
© 2010 Pearson Education, Inc.
All rights reserved.
Before Administration







Counselor should be familiar with the instrument.
Review manual, forms, other materials.
Be familiar with the content of the instrument.
Review directions for administering the instrument.
For standardized tests: schedule date, room, facilities;
obtain booklets, answer sheets, pencils, other
materials
Obtain informed consent from examinees
Provide orientation for examinees
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-17
© 2010 Pearson Education, Inc.
All rights reserved.
During Administration




Final check of materials, lighting, ventilation,
seating arrangements, work space, pencils,
etc.
For standardized instruments, follow
administration procedures and deliver
instructions verbatim.
Establish rapport with client.
Be alert to problems and client behaviors
during administration.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-18
© 2010 Pearson Education, Inc.
All rights reserved.
After Administration




Collect materials
Count booklets and answer sheets
Pack materials according to stated
procedures
Record any incidents.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-19
© 2010 Pearson Education, Inc.
All rights reserved.
Scoring Assessment Instruments




Hand-scoring
Computer-scoring
Sent to publisher for scoring
Self-scored by client
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-20
© 2010 Pearson Education, Inc.
All rights reserved.
Scoring Performance Assessment



Performance assessment: portfolios,
projects, tasks, open-ended exercises.
No clear-cut right or wrong answers.
Scored using a scoring rubric.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-21
© 2010 Pearson Education, Inc.
All rights reserved.
Scoring Errors
Errors in Hand scoring:




Assigning incorrect values to individual responses.
Incorrectly recoding responses
Incorrectly converting raw scores to standard
scores
Calculation errors
Errors in Computer scoring:


Inputting data incorrectly
Incomplete erasures or stray marks (when using
optical scanners)
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-22
© 2010 Pearson Education, Inc.
All rights reserved.
Interpreting Assessment Results



Determine if score is norm-referenced or
criterion-referenced.
Explain the meaning of various standard scores
and their implications to the test taker.
Understand factors that can affect assessment
interpretation:
1. Psychometric factors
2. Test taker factors
3. Contextual factors
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-23
© 2010 Pearson Education, Inc.
All rights reserved.
Interpreting Assessment Results
cont.
Computer-generated reports or narratives:
 Canned interpretations based on test results.
 May be very complex
 Not stand-alone interpretations: computers
cannot incorporate uniqueness of the client
 Counselors ultimately accountable for accuracy
of interpretations, even when using computergenerated reports.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-24
© 2010 Pearson Education, Inc.
All rights reserved.
Standards for Scoring and Interpreting
Tests






Consider any major differences between the norms or comparison groups
and the actual test takers and any differences in test administration
practices or familiarity with the specific questions on the tests.
Avoid using a single test score as the sole determinant of decisions about
test takers.
Avoid using tests for purposes not specifically recommended by the test
developer unless evidence is obtained to support an alternate use.
Report test results accurately and clearly and in a manner than is
understood by the test taker.
Routinely review a sample of the test answer sheets to verify the
accuracy of the initial scoring.
Employ systematic procedures to verify the accuracy and consistency of
computer scoring of answer sheets.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-25
© 2010 Pearson Education, Inc.
All rights reserved.





Obtain a separate and independent verification that appropriate scoring
rules and normative conversions are used for each person tested.
Verify as accurate the computation and conversion of raw scores and the
conversion of raw scores to normative or descriptive scales prior to release
of such information to the test taker or to users of the test results.
Check routinely for accuracy of manual or computer reports of test results.
The person performing this task must be qualified to recognize
inappropriate or impossible scores.
Develop and use systematic and objective procedures for observing and
recording the conditions and behaviors of persons being tested and make
this part of the scores or test results that are reported.
Label clearly the scores that are reported and the date that a particular
test was administered.
Assessment Procedures for Counselors and Helping
Professionals, 7e
Drummond/Jones
1-26
© 2010 Pearson Education, Inc.
All rights reserved.