Part #3 - Sullivan University

Download Report

Transcript Part #3 - Sullivan University

Part #3

Assembling a Test

•

First page = easier questions

•

Test should progress from easier to more difficult (Nitko, 2004)

Assembling a Test

•

Unnecessary to group content topics

Assembling a Test

•

Avoid the correct answer to be the same letter for more than 3 or 4 questions in a row

•

Vary the difficulty level within a content area

Assembling a Test

•

Edit w/ having a colleague proofread for punctuation, grammar, reading level, ‘makes sense’, clarity and content

•

Ensure stem and options are on same page

Assembling a Test

•

Print on one side of the page – if two sided use 2 staples like a book

•

Provide margins for making notes – these can be reviewed for areas that students may have ‘fuzzy’ thinking

Are you tense?

Test Construction

• Curriculum = 100, 200, 300, 400 • Within a Course – Formative Evaluation = – Summative Evaluation = – Exam Matrix for curriculum/course – Exam Blueprint • Revisions – Verbs for Objectives – Types of questions = structure/level • Decision Points © 2014 Rollant Concepts, Inc.

Test Analysis

Thought for the day… “There are three kinds of lies: lies, damned lies and statistics.”

» Mark Twain

Statistical findings in isolation are meaningless . Informed interpretation is the goal without distortion.

Where to Analyze?

• #1 Exam – overall [KR-20] • #2 Items for Correct Responses • #3 Items for discrimination • #4 “Option A” as incorrect select • #5 OPTIONAL – Option discrimination © 2014 Rollant Concepts, Inc.

Summary – Steps to Analysis

1. Exam – desired KR-20 of 0.65 to 0.85 2. & 3. Items ** – mark & review ? that have



P-value of < 0.30 or > 0.80



Point Biserial Index [PBI]

• •

If negative check first [may be keyed incorrectly] Then if time review items with <0.09

**Make decisions to reject ? or accept more than one answer 4. Option Choices **



– mark & review PBI if Correct option [-] negative



Incorrect option [+] positive

What to Analyze – the EXAM?

STEP #1 Look the Exam – overall

– Reliability coefficient [KR – 20] – desired -

KR-20 of 0.65 to 0.85

– OPTIONAL: Score distribution –

histogram

• Review raw scores and percentages • Positive or negative sided =

red flag

What to Analyze - the ITEM?

STEP #2 Look at the Items – Are the questions acceptable?

– Item difficulty level [

P-value responses to an item] = # of correct

– P-value of < 0.30 or > 0.80

STEP #3

– Discrimination •

Item discrimination ratio [IDR] = % of upper 27% minus the lower 27% who answered item correctly

• Point Biserial Index [PBI

] = considers the total group variance

Step #2

Step #3

What to Analyze – in Options?

STEP #4 Options – Which were selected by test-taker?

Distractor analysis

– Response frequencies & patterns

option = # number of students who selected each

– Who selects option “A” as a pattern for

incorrect items?

Step #4

What to Analyze – in Options?

STEP #5 Options – Which were selected by test-taker?

Distractor analysis - Point biserial for options the option = indicated the average performance of those selecting

 

Positive [+] correlates w/ correct option Negative [-] correlates w/ other options

Summary – 3 Steps to Analysis

1. Exam – desired KR-20 of 0.65 to 0.85 2. Items ** – mark & review ? that have



P-value of < 0.30 or > 0.80

 • •

Point Biserial Index [PBI] negative first [check for being correctly keyed] Then <0.09 if time 3. Options **



– mark & review PBI if Correct option [-] negative



Incorrect option [+] positive **Make decisions to reject ? or accept more than one answer

Part #3 - Sullivan University

Transcript Part #3 - Sullivan University

Part #3

Assembling a Test

Assembling a Test

Assembling a Test

Assembling a Test

Assembling a Test

Assembling a Test

Are you tense?

Test Construction

Test Analysis

Where to Analyze?

Summary – Steps to Analysis

What to Analyze – the EXAM?

What to Analyze - the ITEM?

What to Analyze – in Options?

What to Analyze – in Options?

Summary – 3 Steps to Analysis

Directory