Transcript DC CAS GTI Workshop - CTB/McGraw-Hill

Guide to Test Interpretation
Using DC CAS Score Reports to Guide
Decisions and Planning
District of Columbia
Office of the State Superintendent of Education
CTB/McGraw-Hill
September 26–27, 2013
Introductions
OSSE
CTB
 Jessica Enos
Data Analysis Manager
 Jeff Noel
Director of Data Management
 Tommy Shen
Senior Data Analyst
 Amy Dement-Dorey
Senior Program Manager
 Glenn Gage
Program Associate
 Gabriel Martinez
Content Development Lead
 Rick Mercado
Standard Setting Specialist
Copyright © 2013 CTB/McGraw-Hill LLC.
2
Objectives
 Participants will be prepared to
– describe the information found in DC CAS score
reports accurately;
– share this information with their colleagues and
stakeholders; and
– use the information on those reports to make wellinformed, practical decisions.
Copyright © 2013 CTB/McGraw-Hill LLC.
3
About This Presentation
 Overview the Guide to Test Interpretation (GTI)
 Describe the reports available in the GTI
 Discuss best practices for interpreting and using
the reports
 Provide hands-on experience and discussion
with colleagues
Copyright © 2013 CTB/McGraw-Hill LLC.
4
Agenda
 About the DC CAS and the GTI
 Discuss the tools and information needed to
make good decisions
 Discuss the types of score reports available
 Provide hands-on experience using score
reports
 Discuss how to use the information to effect
positive changes in the classroom
 Closing discussion
Copyright © 2013 CTB/McGraw-Hill LLC.
5
About the DC CAS
 Statewide standards-based tests for:
–
–
–
–
–
Reading (Grades 2–10)
Mathematics (Grades 2–8 & 10)
Science (Grades 5 & 8), Biology (High School)
Composition (Grades 4, 7, & 10)
Health (Grades 5, 8, & High School)
Copyright © 2013 CTB/McGraw-Hill LLC.
6
What does the DC CAS measure?
 DC CAS:
– is aligned to the DC content standards
– assesses students’ proficiency in the
knowledge and skills specified in the DC
content standards
– communicates student proficiency using
performance levels and performance level
descriptors (PLDs)
Copyright © 2013 CTB/McGraw-Hill LLC.
7
Performance Level Descriptors
 Abbreviated PLDs are printed on the backs of
score reports.
 Full PLDs can be found:
– in Appendix A of the GTI
– on the OSSE website at
osse.dc.gov/service/dc-cas-performance-level-descriptors
Copyright © 2013 CTB/McGraw-Hill LLC.
8
Guide to Test Interpretation
 The GTI provides
information about how
results are reported and
how those results can be
interpreted and acted
upon.
 It arrives in schools in the
fall, following the spring
test administration.
Copyright © 2013 CTB/McGraw-Hill LLC.
9
Inside the GTI
 Sample DC CAS score
reports
 PLDs for each tested
grade and content area
 Accommodations that are
allowable for Students with
Disabilities (SWDs) and
English Language
Learners (ELLs)
Copyright © 2013 CTB/McGraw-Hill LLC.
10
Important Companion Document
 The DC CAS
Resource Guide
includes:
– Descriptions of the
reporting categories
– Content standards that
are targeted in DC CAS
– Scoring rubrics and
scoring guides
Copyright © 2013 CTB/McGraw-Hill LLC.
11
DC CAS Resource Guide
 The 2013 DC CAS
Resource Guide is now
available on the CTB
website:
www.ctb.com/dc-cas
– Then click on “Training
and Ancillary Materials”
 The 2014 DC CAS
Resource Guide is
scheduled to be posted
later in the year.
Copyright © 2013 CTB/McGraw-Hill LLC.
12
Reading
Mathematics
Science
Biology
Composition
• Roster
Report
• Content
Strands
Roster
Report
• Scoring
Categories
Roster
Report
School
•
•
•
•
•
Classroom
Student
Three Types of Score Reports
• Content
Strands
Summary
Report
• Item Analysis
Summary
Copyright © 2013 CTB/McGraw-Hill LLC.
13
Score Report Terminology
 Raw scores
– Number (or percentage) of score points achieved
 Scale scores (SS)
– Derived from raw scores
– Enable comparison across
school years, in the same grade
– Do not enable comparison
across grades or content areas
Copyright © 2013 CTB/McGraw-Hill LLC.
14
Score Report Terminology (cont.)
 Percent index (PI) scores
– PI scores can be interpreted very much like percent
correct scores and range 0–100.
– They have been shown to
be more reliable than
percent correct scores,
representing an expected percent correct if students
had been given all possible items measuring the
content area.
– PI scores are computed using the scale scores
derived from item response theory (IRT).
Copyright © 2013 CTB/McGraw-Hill LLC.
15
Score Report Terminology (cont.)
 Content strands
– Subcategory of related content standards in a content area
– Usually synonymous with “reporting category”
 Performance levels
– Levels of performance on
DC CAS, with a focus on
“proficiency”
 Performance level descriptors (PLDs)
– Summarize the knowledge, skills, and abilities expected of
students in each performance level
Copyright © 2013 CTB/McGraw-Hill LLC.
16
Score Report Terminology (cont.)
 Roster
– List of student scores organized by class or grade
 Disaggregated results
– Test results reported for separate student subgroups
 Item analysis summary
– Information on student performance for each test item
Copyright © 2013 CTB/McGraw-Hill LLC.
17
Performance Level Descriptors
Example of the back of a Grade 2 student report.
Also shown on page 11 of the 2013 GTI.
18
Student Report, Reading/Math
Example of a Grade 2 Student Report for Reading and Mathematics.
As shown on page 10 of the 2013 GTI.
19
Using the Student Report
 Parents and teachers can use this report to:
– Assess a student’s proficiency (i.e., knowledge and
skills) in the DC CAS content standards using the
PLD for the student’s performance level
– Compare the student’s content strand scores to state
average content strand scores
Copyright © 2013 CTB/McGraw-Hill LLC.
20
Student Report Information
 Scale scores and performance levels
– Scale scores and performance levels are comparable
across years for that content area and grade level.
– Scale scores and performance levels are not
comparable across grades.
 Content strand percent correct scores
– Students’ content strand performance can be
compared with the average State performance.
Copyright © 2013 CTB/McGraw-Hill LLC.
21
Student Report, Composition
Example of a Grade 4 Student Report for Composition.
As shown on page 13 of the 2013 GTI.
22
Content Strands Summary Report
Example of a Grade 2 Mathematics Content Strands Summary Report.
As shown on page 17 of the 2013 GTI.
23
Using the Content Strands Summary
Report
 Teachers and principals can use this report to:
– Evaluate grade-level performance on the content
strands by comparing the school’s performance to
state performance on each content strand.
– Identify content strands where instructional
approaches, instructional materials, or emphasis
(e.g., number of instructional hours, homework) may
need to be reconsidered.
Copyright © 2013 CTB/McGraw-Hill LLC.
24
Content Strands Summary Report
Information
 PI score for the school and state, for each
content strand
– You cannot compare these PI scores across content
strands or across years.
– PI scores are not adjusted for the varying difficulty
levels of the items in each strand.
Copyright © 2013 CTB/McGraw-Hill LLC.
25
Student Roster Report
Example of a Grade 6 Roster Report.
Also shown on page 15 of the 2013 GTI.
26
Using Student Roster Reports
 Roster reports list and summarize the
information found in other reports.
 The student roster report provides:
– Alphabetical list of students, by class and grade.
– For multiple grades in one class, a separate report is
produced for each grade.
Copyright © 2013 CTB/McGraw-Hill LLC.
27
Content Strands Roster Report
Example of a Grade 8 Mathematics Content Strands Roster Report.
Also shown on page 19 of the 2013 GTI.
28
Content Strands Roster Reports
Information
 Part 1: Roster
– Alphabetical list of students, by class and grade.
– Scores for the total test: scale score, raw score, percent
correct, scale score error range, performance level, PI.
– Scores for the content strands: number of items (and #
points possible), minimum PI for proficiency, and
performance indicator/PI score.
• Performance indicators allow for comparison of student
proficiency across content strands
 Part 2: Summary
– Summary by class and grade (i.e., mean PI, mean scale
score, mean raw score).
Copyright © 2013 CTB/McGraw-Hill LLC.
29
Content Strands Roster Report
Summary
Example of a Grade 8 Mathematics Content Strands Roster Report Summary.
Also shown on page 21of the 2013 GTI.
30
Item Analysis Summary
Example of a Grade 8 Mathematics Item Analysis Summary.
Also shown on page 25 of the 2013 GTI.
31
Using the Item Analysis Summary
 Teachers and principals can use this report to:
– Evaluate grade-level performance on the objectives
within each content strand
– Identify objectives where instructional approaches,
instructional materials, or emphasis (e.g., number of
instructional hours, homework) may need to be
reconsidered
– Compare the school’s performance to state
performance on each objective
– Compare performance on multiple choice and
constructed response items
Copyright © 2013 CTB/McGraw-Hill LLC.
32
Item Analysis Summary Information
 Percent correct score for the school and state,
for items in each tested standard
– Do not compare percent correct scores across items
or across years; they are not adjusted for difficulty.
Copyright © 2013 CTB/McGraw-Hill LLC.
33
Transition to the Common Core
2011
2012
2013
2014
• Transition to the Common Core State Standards began
• DC CAS Reading and Writing were aligned to Common Core
• DC CAS Mathematics focused on essential knowledge and skills students
needed to be successful with Common Core
• Mathematics was aligned to Common Core
• All DC CAS assessments are aligned to Common Core
Copyright © 2013 CTB/McGraw-Hill LLC.
34
Practice with Reports
 In your own or in pairs, use the reports in the
GTI to answer the three questions on the
Practice with Reports worksheet.
– We’ll review the answers in several minutes.
Copyright © 2013 CTB/McGraw-Hill LLC.
35
Practice with Reports
1. How many students in my class scored Proficient or
above in Math overall?
a. Which report(s) can I use to answer the question?
• Roster Report (p. 15); or
Content Strands Roster Report, Part 2 (p. 21).
b. What is the answer using the report in the GTI?
• Roster Report, 4 students; or CSRR2, 26 students.
Copyright © 2013 CTB/McGraw-Hill LLC.
36
Practice with Reports
2. For which content strand(s) did my class have the
most difficulty reaching proficiency?
a. Which report(s) can I use to answer the question?
• Content Strands Roster Report, Part 1 (p. 19)
b. What is the answer using the report in the GTI?
• Geometry (5 students did not score proficient)
Copyright © 2013 CTB/McGraw-Hill LLC.
37
Practice with Reports (cont.)
3. For which content strand(s) is my school’s performance
higher than the state (DC public schools, including
charters), for the grade level shown?
a. Which report(s) can I use to answer the question?
• Content Strands Summary Report (p. 17)
b. What is the answer using the report in the GTI?
• Measurement & Data shows the school ahead of State.
Copyright © 2013 CTB/McGraw-Hill LLC.
38
Small Group Discussion
1. On the Data Exploration Worksheet, identify
one or two instructional challenges you face this
school year with a class or grade level of
students.
2. Identify specific information in the reports that
might help you examine these challenges.
3. Use the information to sketch out a plan for
addressing the challenges this year, and what
information would be expected to change if
there is improvement.
Note: Some types of questions (e.g., growth) are computed separately from the
reports, and are not able to be answered using these reports.
39
Large Group Discussion
 Share your insights, comments, and
unanswered questions:
– What challenges did you identify?
– What information in the score reports was useful?
– What thoughts and questions do you have about the
score reports?
– What actions, interventions, or other next steps are
you considering?
– What additional information or support do you need?
Copyright © 2013 CTB/McGraw-Hill LLC.
40
Closing Discussion
 Planning:
– What are your next steps?
– Will DC CAS reports help with your planning?
– Will additional information be needed to plan for the
2013–2014 school year?
 Please give feedback on today’s presentation on
the workshop evaluation.
Copyright © 2013 CTB/McGraw-Hill LLC.
41
For Further Information
 For further assistance regarding score reports or
the Guide to Test Interpretation, contact:
 For more DC CAS information and resources,
visit the CTB website at:
www.ctb.com/dc-cas
Copyright © 2013 CTB/McGraw-Hill LLC.
42
Thank you!
 Rick Mercado
 [email protected]
 Gabe Martinez
 [email protected]
 Glenn Gage
 [email protected]
 Amy Dement-Dorey
 [email protected]
Copyright © 2013 CTB/McGraw-Hill LLC.
43