Transcript www.caaspp.org
California Assessment of Student Performance and Progress (CAASPP)
Introduction to the 2015 CAASPP Online Reporting System (ORS)
1 – 2 p.m.
May 4, 2015
•
Purpose
To provide an introduction to the Online Reporting System (ORS), which is the reporting system local educational agencies (LEA) will use to view
preliminary
student results on the following CAASPP assessments: – Smarter Balanced Summative Assessments in English language arts/literacy (ELA) and mathematics – California Standards Test (CST) for Science – California Modified Assessment (CMA) for Science – California Alternate Performance Assessment (CAPA) for Science – Standards-based Tests for Spanish (STS)
Introduction to the 2015 CAASPP Online Reporting System
2
Agenda
• Overview • Accessing the ORS • Understanding the ORS Interface • Viewing Score Reports in the ORS • Live Demonstration • Questions • Updates and Announcements
Introduction to the 2015 CAASPP Online Reporting System
3
Overview
What is the Online Reporting System?
The ORS is a
web-based
system that displays
preliminary
score reports and participation data for each student who takes part in the CAASPP assessments.
Introduction to the 2015 CAASPP Online Reporting System
5
Why Online Reporting?
• Timely data is more useful for affecting learning and instruction.
• Move to computer-based assessments facilitates faster reporting.
• Policymakers made timeliness a priority (Education Code 60641(b)).
Introduction to the 2015 CAASPP Online Reporting System
6
Important Reminder
The data available in the CAASPP ORS represents partial and preliminary results that are not appropriate for public release. As a real-time system, results will change as additional data is received and relevant appeals and rescores are processed. These changes may result in final scores being higher or lower than the preliminary results posted to this system. The California Department of Education (CDE) recommends that data from the ORS only be released publically following the state level release of assessment data that occurs in August.
Introduction to the 2015 CAASPP Online Reporting System
7
Process for Release of Preliminary Test Results
• Hand score • Machine score • AI Score
Score Check
• Confirm accuracy by grade and content area •
Preliminary
scores available in the ORS
Report
Introduction to the 2015 CAASPP Online Reporting System
8
Reporting Timeline for Online Smarter Balanced Summative Tests Week 0:
Student completes a content area
Weeks 1-3:
Student responses are scored and merged; preliminary results are checked
Week 4:
LEA accesses ORS to view
preliminary
results
Introduction to the 2015 CAASPP Online Reporting System
9
Why Preliminary Results?
• Test administrations may still be in progress • Factors that could change results – Security breaches – Problematic test questions – Appeals processed and test rescores
Introduction to the 2015 CAASPP Online Reporting System
10
Preliminary Test Results
• Provides information to support teachers in: – Focusing instruction – Improving learning • Preliminary results include: – Student and aggregate results – Overall scores reported within one of four achievement levels
Achievement Levels LEVEL 4 Exceeded the Standard LEVEL 3 LEVEL 2 LEVEL 1 Met the Standard Nearly Met the Standard Has Not Met the Standard
Introduction to the 2015 CAASPP Online Reporting System
11
Estimated Timeline for Release of Preliminary Results
Grade and Content Area Smarter Balanced Grade 3 mathematics Smarter Balanced Grade 5 mathematics Smarter Balanced Grade 7 ELA Smarter Balanced Grade 6 ELA and mathematics Smarter Balanced Grade 7 mathematics Smarter Balanced Grade 8 ELA and mathematics Smarter Balanced Grade 3 ELA Smarter Balanced Grade 4 ELA Smarter Balanced Grade 5 ELA Smarter Balanced Grade 4 mathematics Smarter Balanced Grade 11 ELA and mathematics CST/CMA/CAPA Science Grades 5, 8 and 10 STS RLA Grades 2-11 Estimated Date
May 3 (today) for students completed testing on or around April 1 TBD TBD TBD TBD
Introduction to the 2015 CAASPP Online Reporting System
12
Reporting Timeline for Final Results
Week 0:
Student completes a content area
Weeks 1-3:
Student responses are scored and merged; preliminary results are checked
Week 4:
LEA accesses ORS to view
preliminary
results
4 Weeks After Test Administration Window Closes:
LEAs accesses and downloads final student data file from TOMS
Introduction to the 2015 CAASPP Online Reporting System
13
Examples of Recommended Uses
• Use preliminary overall scores and claim information to focus instruction • Combine with other academic information about the student to identify learning opportunities
Introduction to the 2015 CAASPP Online Reporting System
14
Not Recommended
• Adding these results to your local student information system • Presenting these results as final scores • Using these results as the sole basis for determining a student’s learning opportunities • Comparing these results to last year’s STAR results
Introduction to the 2015 CAASPP Online Reporting System
15
Accessing the ORS
User Roles and Access in the ORS
• ORS operates in a secure, role-based structure • Access to reports and data in ORS depends on your user role in the Test Operations Management System (TOMS) –
Example:
LEA-level users can view data for all schools and students in their LEA; school-level users can view data only for students in their school.
Introduction to the 2015 CAASPP Online Reporting System
17
User Roles and Access in the ORS (cont.)
Access Level School Listing Student Listing Preliminary Student Results Report Summary Statistics Retrieve Student Results Plan and Manage Testing Test Completion Rates Search Students LEA CAASPP Coordinator
Test Site Coordinator (SC)
Test Administrator (TA)
Introduction to the 2015 CAASPP Online Reporting System
18
Accessing the ORS
•
Note:
To log onto ORS, users must have an authorized TOMS username and password.
1. Select the [
Online Reporting System
] button on the CAASPP portal at http://caaspp.org
.
2. Log onto ORS using your TOMS logon credentials.
Introduction to the 2015 CAASPP Online Reporting System
19
Accessing the ORS
3. Upon logging on, the Welcome page will appear.
Introduction to the 2015 CAASPP Online Reporting System
20
Accessing the ORS
•
Note:
ORS contains students’ personally identifiable information (PII). For security purposes, it is important that you log out to ensure that the reports cannot be accessed by unauthorized users following your session.
• ORS has a timeout feature that automatically logs you out if you have been inactive for 30 minutes.
Introduction to the 2015 CAASPP Online Reporting System
21
Understanding the ORS Interface
Understanding the ORS Interface
• Two primary features of ORS: 1. Test Management Center (Completion Status System) 2. Score Reports
Introduction to the 2015 CAASPP Online Reporting System
23
ORS Welcome Page
Takes you to the Completion Status System Takes you preliminary student test scores
Introduction to the 2015 CAASPP Online Reporting System
24
Test Management Center
• Same as the Completion Status System • Provides participation data for students taking tests • Can be used to determine which students need to complete and/or start testing • Can be used to view a summary statistics report and download student data files • A detailed overview of accessing and using student participation reports is provided in the
Completion Status User Guide Introduction to the 2015 CAASPP Online Reporting System
25
Score Reports
• Provide score data for each administered test • Compare score data between individual students and the school, LEA, or overall state average scores • Provide information about performance on claims
Introduction to the 2015 CAASPP Online Reporting System
26
ORS Banner
Completion Status Reports Opens a pop-up window where you can search for a student and access his/her preliminary scores Preliminary student test results Displays Retrieve Student Results page, where you can access student performance data files Links to the ORS User Guide (forthcoming) Exports the data displayed on the page in a .xls file
Introduction to the 2015 CAASPP Online Reporting System
27
Viewing Score Reports in the ORS
Viewing ORS Reports
1. From the
Select
drop-down list, select the LEA or school whose reports you want to view. – A list will appear only if you are associated with more than one school or LEA 2. Select [
Score Reports
].
Note:
All score reports data, except for individual students’ score reports, can be disaggregated into subgroups for detailed analysis. For example, you can view a Grade 5 Mathematics report for a for an entire school or for a LEA.
Introduction to the 2015 CAASPP Online Reporting System
29
Available Preliminary Score Reports in the ORS
Report Home Page Dashboard
Summary of performance (to date) across grades and subjects or courses for the current administration.
Subject Detail
Data for a subject within a particular grade or course for the current administration.
Claim Level Detail (Smarter Balanced)
Claim/Block-level percentage at each performance level for a subject within a particular grade or course for the current administration.
Student Listing
Roster of all students who belong to a school with their associated subject or course (and claim, if applicable) scores for the current administration.
Student Detail
Detailed information about a selected student’s performance in a specified subject or course (and claim, if applicable).
District Level
School Level Student Level
Introduction to the 2015 CAASPP Online Reporting System
30
Home Page Dashboard Report
• Displays the overall summary of score data and testing progress for your LEA and/or school • Starting point for data analysis • Define the students whose aggregated scores you • want to view • Navigate to more detailed score reports from the Home Page Dashboard
Note:
The score data you see are dependent on your role
Introduction to the 2015 CAASPP Online Reporting System
31
Home Page Dashboard Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
32
Subject Detail Report
• The aggregation tables on the Home Page Dashboard display score data for students by grade and subject and provide access to Subject Detail Reports.
Introduction to the 2015 CAASPP Online Reporting System
33
Subject Detail Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
34
•
Subject Detail Report (cont.)
Each Subject Detail Report consists of the following components: Report descriptor Test name (subject and grade) Administration year Entity (e.g., LEA, school) The title of the score report table Name, number of students, average scale scores, percent in each achievement level All data are based on the total number of students who have taken and completed the test and submitted it for scoring
Introduction to the 2015 CAASPP Online Reporting System
35
1
Claim Level Detail Report
2
Introduction to the 2015 CAASPP Online Reporting System
36
Claim Level Detail Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
37
Claim Level Detail Report (cont.)
• The Claim Level Detail Report consists of the following components: – Report name [Entity] Performance for Each Claim.
What are my [entity’s] strengths and weaknesses in [Subject or Course]?
– Test name (subject and grade) – Administration year – Entity (e.g., LEA, school, or roster) – The title of the score report table – Name, number of students, average scale score, claims, and percentage in each claims achievement level
Introduction to the 2015 CAASPP Online Reporting System
38
1
Student Listing Report
2
Introduction to the 2015 CAASPP Online Reporting System
39
Student Listing Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
40
Student Listing Report (cont.)
• Students’ SSIDs are displayed • Scores for individual students are
not
mean scale scores • Different procedure for viewing scale score data by demographic subgroup • Print on the Student Listing Report prints the current page
and
also generates a PDF file of individual preliminary student results for all the students in the roster
Introduction to the 2015 CAASPP Online Reporting System
41
1
Student Detail Report
2
Introduction to the 2015 CAASPP Online Reporting System
42
Student Detail Report (cont.)
Introduction to the 2015 CAASPP Online Reporting System
43
Student Detail Report (cont.)
• Displays the breakdown of the student’s preliminary scale score, achievement level for the selected subject, and performance and claim description for each claim • Includes average scale scores for the state and LEA for comparison purposes –
Note:
State-level scale score averages will not be available until formally released by the CDE.
Introduction to the 2015 CAASPP Online Reporting System
44
System Enhancements under Consideration
• Target reports • Creating roster groups
Introduction to the 2015 CAASPP Online Reporting System
45
Live Demonstration
Questions?
Submitting Feedback on the ORS
New systems require patience, we appreciate your understanding and welcome your feedback: https://etsforms.formstack.com/forms/ors_feedback
Introduction to the 2015 CAASPP Online Reporting System
48
Updates and Announcements
•
Updates and Announcements
• The
Online Reporting System User Guide
is forthcoming.
Next Webcast:
2015 CAASPP Post-Test Workshop – –
Date:
Wednesday, May 20, 2015
Time:
1 –3:30 p.m.
Introduction to the 2015 CAASPP Online Reporting System
50
Resources and Support
Help Desk Support
The California Technical Assistance Center (CalTAC) is here to support all LEA CAASPP coordinators!
Monday –Friday from 7 a.m.–5 p.m. PT
E-mail:
Phone:
800-955-2954
Web site:
http://caaspp.org
Introduction to the 2015 CAASPP Online Reporting System
52