Transcript Stages of Test Development

Stages of Test Development
By Lily Novita - 69090007

Make a full and clear statement of the testing ‘problem’.

Write complete specifications for the test.

Write and moderate items.

Trial the items informally on native speakers and reject or modify
problematic ones as necessary.

Trial the test on a group of non-native speakers similar to those for whom
the test is intended.

Analyse the results of the trial and make any necessary changes.

Calibrate scales.

Validate.

Write handbooks for test takers, test users and staff.

Train any necessary staff (interviewers, raters, etc.).
1. Stating the Problem

The essential initial step in any testing is to make oneself perfectly
clear what one wants to know and for what purpose
 What kind of test it is constructed for?
 What is the precise purpose?
 What abilities are to be tested?
 How detailed must the results be?
 How accurate must the results be?
 How important is backwash?
 What constraints are set by unavailability of expertise,
facilities, time ? (for construction, administration and
scoring)
2. Writing specifications
for the test

Content
 Operations
 Types of text
 Addresses of texts
 Length of text(s)
 Topics
 Readability
 Structural range
 Vocabulary Range
 Dialect, accent, style
 Speed of processing
2. Writing specifications
for the test
Structure , timing,
medium/channel and techniques
 Test structure
 Number of items
 Medium / channel
 Timing
 Techniques
2. Writing specifications
for the test

Criterial levels of performance
 Accuracy
 Appropriacy
 Range
Flexibility
 Size
2. Writing specifications
for the test
 Scoring procedures
 Subjectivity
 Achievement of high reliability &
validity in scoring
 Rating scale to be used?
 No. of people rating each piece of
work?
 Solutions on disagreements between
raters
3. Writing and moderating
items
Sampling
Writing
items
Moderating items
4. Informal trialling of
items on native speakers
Moderation of grammar test
1. Is the English grammatically correct?
2. Is the English natural and acceptable?
3. Is the English in accordance with the specifications?
4. Does the item test what it is supposed to test, as specified?
5. The correct response cannot be obtained without the appropriate
knowledge of grammar (other than random sampling)
6. Is the item economical?
7. (a) Multiple choice – is there just one correct response?
(b) Gap filling – are there just one or two correct responses?
8. Multiple choice : Are all the distractors likely to distract?
9. Is the key complete and correct?
Yes
No
5. Trialling of the test on a group of non-native
speakers similar to those for whom the test is intended
trials are designed to help ensure that
the items function appropriately and
are not confusing for the students.
 this is accomplished by embedding
field test items in the operational test,
to ensure that the items are taken by a
representative group of motivated
students under standard conditions.

6. Analysis of the results of the trial
– making of any necessary changes

2 kinds of analysis should be carried out :
 Statistical analysis : reveals qualities (reliability)
as a whole and individual items – how difficult
they are , how well they discriminate between
stronger and weaker candidates.
 Qualitative analysis : responses are examined to
discover misinterpretations, unanticipated but
possibly correct answers and indicators of other
faulty items.
7. Calibration of scales

It means collecting samples of performance which cover the
full range of the scales.

A calibration test is a procedure in which an instrument,
tool, or device is tested to confirm that it conforms with the
standard. Calibration is very important, as it ensures that
objects are working properly. There are a number of reasons
to conduct a calibration test, ranging from concerns that
something is not working as it should to preparations for an
event in which very precise calibration is desired, and there
are a number of ways to perform a calibration.
8. Validation
Essential
validation – for high
stakes or published tests
Small-scale
validation – for
low stakes used within an
institution
9. Writing handbooks for test
takers, test users and staffs (contents)

The rationale for the test;

An account of how the test was developed and validated

A description of the test

Sample items

Advice on preparing for taking the test

An explanation of how test scores are to be interpreted

Training materials

Details of test administration
10. Training Staff
All
staffs who will be involved
in the test process should be
trained : interviewers, raters,
scorers, computer operators,
and invigilators.