Transcript 2012 STAR Pre-Test Workshop

California Assessment of Student
Performance and Progress
(CAASPP)
CAASPP Test Site Coordinator Training
SBAC Online Test Administration
February 25, 26 2015
March 4, 5 2015
Presented by
Stephanie Hearne, Assessment Supervisor
Agenda










Overview- Student Participation
Roles and Responsibilities
Technology
Test Administration
Using the TOMS System
Student Accessibility Supports
Creating and Administering an Online Test Session
Test Security- Improprieties, Irregularities, Breaches
Resources and Site Support
Questions
2
Objective

By the end of this training, attendees will understand:


3
The administration of the SBAC summative assessments using
appropriate procedures
The navigation and uses for Test Operations Management System
(TOMS)
Overview
4
Overview
Student Participation
Q: Who will take the online SBAC summative
assessments?
A: All students in grades 3-8 and 11 are required to take the
summative assessments.
The only students exempt from participating in the SBAC
summative assessments are:
 Students participating in the California Alternate Assessments
(CAA)
 English learners (ELs) who have been attending school in the
United States for less than 12 months (for the ELA test only)


5
Students will be required to take the math assessment
Overview
Pgs. 32-33 TAM
Student Participation: Early Assessment Program
The Early Assessment Program (EAP) will be using the
grade 11 SBAC summative assessments in ELA and
mathematics in place of the grade 11 CSTs in ELA and/or
mathematics, additional 15 questions and the ELA essay
Grade 11 students taking the SBAC summative
assessments may choose to release their results to the
CSU and participating CCCs at the end of the computer
adaptive test
Students who do not authorize the release of their test
results at the time of testing may need to provide a copy
of the score report upon request of the campus



6
Overview
About the Summative Assessments

Content areas;



English language arts/literacy (ELA)
Mathematics
Two components per content area:


Computer adaptive test (CAT)
Performance task (PT)
Test Item Types: Page 59 in the TAM
7
Overview
About the Summative Assessments (Cont.)
Q: What is the CAT component of the online
summative assessment?
A: The computer adaptive portion (CAT) of the online test
includes a wide variety of item types. As students progress
throughout a CAT, the computer program adjusts the difficulty of
questions throughout the assessment based on their responses.
For example, a student who answers a question correctly will
receive a more challenging item, while an incorrect answer
generates an easier question.
8
Overview
About the Summative Assessments (Cont.)
Q: What is the PT component of the online summative
assessment?
A: The PT is designed to provide students with an opportunity to
demonstrate their ability to apply their knowledge and high-order
thinking skills to explore and analyze a complex, real world
scenario. Prior to the PT, teachers or instructional staff conduct a
Classroom Activity for all students to ensure that a lack of
understanding of the context of the task does not interfere with a
student’s ability to address the content of the task.
9
Overview
Systems and Processes for Online Administration
1
2
3
CALPADS
TOMS
Test Administrator
Interface & Student
Testing Site
(California Longitudinal
Pupil Achievement Data
System)
•Manage student
demographic data
10
• Verify student demographic
data
• Configure online test
settings (designated
supports and
accommodations)
• Administer online
Smarter Balanced
assessments
Roles and Responsibilities





LEA CAASPP Coordinator
Test Site Coordinator
Test Administrator
Technology Coordinator
CALPADS Administrator
11
Roles and Responsibilities
LEA CAASPP Coordinator (DC)

Add SCs and/or TAs in the TOMS

Ensure that the SCs are appropriately trained regarding the
state and SBAC assessment administration as well as security
policies and procedures
Approve testing schedules and security plans for the schools


Report all test security irregularities and/or breaches that
occur before, during, and after test administration to the CDE

Oversee all CAASPP administration activities in schools
12
Roles and Responsibilities
School Coordinator (SC)











Identify TAs, ensure that they have submitted signed CAASPP Test
Security Affidavits
Add TAs into TOMS
Ensure TAs are properly trained so they can administer the assessments
Enter and/or verify students test settings in TOMS
Ensure that testing in the school is conducted in accordance to test
security policies
Create/approve test schedules and security procedures for the school
Work with technology staff to ensure that necessary secure browsers
are installed and any other technical issues are resolved
Monitor testing progress and ensure that all eligible students participate
Report test security incidents to the DC
Relay issues/questions to the DC
Oversee CAASPP administration activities in the school
13
Roles and Responsibilities
Test Administrator (TA)









Read and sign the CAASPP Test Security Affidavit
Carefully and completely read and sign security affidavit
Complete training and review all SBAC policy and administration
documents prior to administering any SBAC assessments
View student information prior to testing to ensure it is correct
Ensure that appropriate designated supports and accommodations
have been set prior to testing
Administer the SBAC assessments
Report all potential test security incidents to the SC
Relay issues/questions to the SC
Encourage students to do their best
14
Roles and Responsibilities
Technology Coordinator

Assist District in setting up and overseeing technology
infrastructure

Support each school site

Work closely with the LEA CAASPP Coordinator to ensure
network and testing devices are properly prepared by installing the
most current secure browser

Help troubleshoot during time of testing
15
Roles and Responsibilities
CALPADS Administrator


Ensures student demographic, enrollment, and program
participation data is correct in the CALPADS
Work closely with the District CAASPP Coordinator
16
Technology Infrastructure
17
Technology Infrastructure

Prior to the test administration, District CAASPP Coordinator
with Technology Coordinator should review the technology
infrastructure





Testing devices are working
Ensure that secure browsers are properly installed a minimum of two
weeks before testing
Ensure the necessary hardware is available and working (i.e., headphones,
external keyboards, printers)
Content filters, firewalls, and proxy servers configured
Stable (wired or wireless) internet connection
18
Secure Browser

Students must use the secure browser to log in to the Student
Interface and access the summative test


Prevents students from accessing other hardware and software functions
Prior to testing, TAs should verify that:



Pgs. 74-75 TAM
External user applications are closed, secure browser will not work if the
computer detects that a forbidden application is running
Volume must be tested before launching browser
Test Administrators (TA) will use a standard browser

No testing on computers with dual monitors

19
Students must not take the test on computers connected to more than one
monitor
Hardware Requirements

Headphones are required for each test station




English language arts/literacy (ELA) tests
Text-to-speech accommodation
Job Access with Speech (JAWS)
Microphones are not required
20
Test Administration
21
Preparing Students and Staff
Practice and Training Test

There are multiple ways to prepare students and staff for
the SBAC summative assessments:



Practice Test
Training Test
Purpose of the Practice Tests

Allows students and administrators to become familiar with
the format and structure of the operational assessments

Scoring rubrics and answers are available on the TA
Resource Web page at http://caaspp.org/taresources/index.html

All universal tools, designated supports, and
accommodations are available
22
Preparing Students and Staff
Practice and Training Test

(Cont.)
There are two ways for accessing the Practice and
Training Test:


Through caaspp.org on a Web browser
Through the secure browser


23
Simulates operational online testing environment
TAs may create test sessions for the Practice or Training Tests
on the TA Practice and Training Site
Test Administration
Pgs. 27-31 TAM
Scheduling Testing Time
Computer
Adaptive Test
(CAT) items
Content Area
English
Language
Arts/Literacy
Mathematics
Both
24
Grades
hrs : mins
Performance
Task (PT)
Total
Classroom
Activity
(administered
prior to the
PT)*
hrs : mins
hrs : mins
hrs : mins
hrs : mins
Total
3–5
1:30
2:00
3:30
:30
4:00
6–8
1:30
2:00
3:30
:30
4:00
11
2:00
2:00
4:00
:30
4:30
3–5
1:30
1:00
2:30
:30
3:00
6–8
2:00
1:00
3:00
:30
3:30
11
2:00
1:30
3:30
:30
4:00
3–5
3:00
3:00
6:00
1:00
7:00
6–8
3:30
3:00
6:30
1:00
7:30
11
4:00
3:30
7:30
1:00
8:30
Test Administration
Scheduling Testing Time (Cont.)

The estimates do not account for any time needed to
start computers, and log in students

Recommended: CAT first, PTs second

The classroom activity should be completed within 3 days
prior to administering a PT



ELA PTs have 2 parts and can be completed over 2 days
Mathematic PTs have 1 part and can completed in 1 day
PTs should not be administered on the same day as the classroom
activity or the CAT
CAT
25
Classroom
Activity
Performance
Task
Test Administrator Training

Ensure that TAs:





Sign Security Affidavits
Review the TA Reference Guide
Review Online Test Administration Manual Section 2.1 and 10.0
SBAC Usability, Accessibility, and Accommodations Guidelines
Review Training Modules



What is a CAT
What are Universal Tools
TAs should know testing procedures:


26
they have school site schedules
they are familiar with security procedures, and conduct
associated with the administration of standardize assessments
Test Administration
Establishing Appropriate Testing Conditions

Testing students in classroom-sized groups is preferable


Remove or cover displayed information that might be used by
students to help answer questions



Rubrics, vocabulary charts, student work, graphs, charts, etc.
Establish procedures to maintain a quiet testing environment


Facilitates monitoring and control for TAs
Remember some students will finish more quickly than others
Cell phones or other electronic devices are not allowed to be
used during testing
Post an “Unauthorized Electronic Devices” sign so that it is
clearly visible to students and place a testing “Do Not
Disturb” sign on doors to testing rooms
27
Test Administration
General Testing Information

Students must answer all test items on a page before
moving on to the next page

Students may not return to a test segment once it has
been completed and submitted

Students must answer all test items before the test can
be submitted

TAs must follow the “SAY” script exactly each time a test
session is administered

28
Refer to the “SAY” boxes of the Online Test Administration
Manual or Online Directions for Administration
Using the Test Operations
Management System (TOMS)

Task in Toms

Add and manage TAs


Review student data



Demographic Data from CALPADS
Test enrollment
Assign online test setting to
students


Gives them access to TA interface
Designated Supports and
Accommodations
TOMS can be accessed from the
CAASPP Portal at
http://caaspp.org
29
TOMS and CALPADS





CALPADS is the sole source for student demographic and
program participation data for TOMS
Student accommodations and supports will not be uploaded
in CALPADS
LEAs cannot update student demographic or program
participation data in TOMS; must be updated in CALPADS
Processing time from CALPADS to TOMS is 48 hours
SCs must set designated supports and accommodations for
the students in TOMS
30
Reviewing Student Data

Each student profile in TOMS includes:

Demographic data


Test enrollment


All students in grades 3-8 and 11 are automatically enrolled in the
online SBAC summative assessments
Online test settings

31
Read-only; must be modified in CALPADS
Universal tools, designated supports, and accommodation
Reviewing Student Data (Cont.)

Students’ grade levels must be correct in TOMS before
testing

Grade levels are only correctable in CALPADS

All students who are going to test must appear in TOMS

If a student’s designated supports and accommodations
information is incorrect, it must be updated in TOMS
before he or she tests
32
Student Profile in TOMS:
Demographics
All
demographic
data is readonly
33
Student Profile in TOMS:
Test Mode
All students in grades
3–8 and 11 are
automatically
enrolled in the online
Smarter Balanced
summative tests
34
Student Profile in TOMS:
Test Settings
35
Assigning Accessibility Supports in TOMS

Student online test settings must be configured at least 24
hours prior to testing

Allow up to 24 hours from when you configure test settings in
TOMS to when the changes appear in the TA and Student
Testing Interface

LEA and School Coordinators are authorized to set
accessibility supports for students in TOMS. TAs do not have
the ability
1.
2.
3.
36
One-by-one through student profile
Special Ed will initially upload students settings from SEIS
Coordinators will verify that information is correct
What are Accessibility Supports?

Universal Tools:


Designated Supports:


Available to all; based on
student preference and
selection (i.e., strikethrough,
highlighter)
Need to be indicated by an
educator or guardian (i.e.,
color contrast, read aloud,
glossary, magnification)
Accommodations:

37
Based on individualized
education programs (IEPs)
or Section 504 plans (i.e.,
American Sign Language,
scribe)
What are Accessibility Supports? (Cont.)

The online assessments include embedded and non-embedded
accessibility supports

Embedded: part of the computer administration (i.e., digital
notepad, calculator)

Non-embedded: provide outside the computer administration
(i.e., scratch paper, multiplication table)
38
What are Accessibility Supports? (Cont.)
Embedded
Designated Support
Color Contrast
Accommodation
American Sign Language (ELA
listening and math items)
Non-Embedded
Designated Support
Accommodation
Bilingual Dictionary (ELA
Abacus
performance task full writes)
Alternate Response
Color Contrast
Options
Masking
Braille
Text-to-Speech (math and
ELA items)
Translated Test Directions
(math items)
Translations (glossaries)
(math items)
Translations (stacked) (for
math items)*
Closed Captioning (ELA
listening items)
Color Overlays
Calculator
Streamlined Interface
Magnification
Multiplication Table
Text-to-speech (ELA reading
passages)
Noise Buffers
Print on Demand**
Read Aloud (math and ELA
items)
Scribe (ELA non-writing and
math items)
Separate Setting
Translated Test Directions
Translation (glossaries) (math
items)
Special Lighting or Acoustics
Translated Test Directions
(math items)
Read Aloud (ELA reading
passages)
Turn off any universal tools*
Permissive Mode
Print Size
*Must be activated in the test delivery system.
**Print on demand for stimuli only can be
activated in TOMS. If a student(s) need print
on demand for items and/or items and
stimuli, contact CalTAC at least 24 hours
prior to testing.
39
Scribe
Speech-to-Text
Individualized Aids
Administering a Test Session
Pg. 41 TAM

The TA logs on to
the TA Interface from
http://caaspp.org on a
standard Web
browser

The TA logs on using
his/her username and
password
40
Administering a Test Session (Cont.)

The TA selects the test category or the grade and
content area
41
Administering a Test Session (Cont.)

The TA selects the tests to administer


Additional test can be added to a session; you will receive a
confirmation message box
The TA starts a test session
42
Administering a Test Session (Cont.)

The TA informs students of the test Session ID



Write or display clearly in a place all students can see
Session IDs are automatically generated upon selecting the
[Start Session] button
Additional test can be added to a session. You will receive
a confirmation message box
43
Distributing Student Logon Information

TAs need to provide students with the Session ID , the
students Statewide Student Identifier (SSID), and
confirmation code (student first name in CALPADS) for
logon


May be provided on a card or piece of paper
TAs may write the Session ID on a board where students can
see it
Note: if the session ID is not written down or known and the
TA tries to log on to the session after getting involuntarily
logged off of the system, the TA will be logged off for eight
hours before he or she is able to start a new session
44
Administering a Test Session (Cont.)

Students log on through the secure browser

To log on, students need:



45
First Name (Confirmation Code): Student’s legal first name as
spelled in CALPADS
State-SSID “CA” followed by a hyphen “-” and the student’s SSID
Session ID: TA-generated session ID
Administering a Test Session (Cont.)

Students verify their identity


46
After logging on, students will see the “Is This You” screen
Students should select [Yes] to continue
Administering a Test Session (Cont.)

Students select the test


47
All grade-level tests that the student is eligible to take are
displayed
Only the tests that the TA selected for the test session and
those that have not been completed are selectable by students
Administering a Test Session (Cont.)

Upon selecting a test, students will wait for TA approval

As students select tests, the TA will see approvals
populate in the Approvals queue of the TA Interface
48
Administering a Test Session (Cont.)

The TA selects the [Approvals (#)] button to access the
Approvals and Student Test Settings screen
49
Administering a Test Session (Cont.)

The TA verifies the student has selected the appropriate
test and reviews the students’ test settings by selecting
[See/Edit Details] button next to each student

50
Only need to check for students in session that are using
special accommodations, etc.
Administering a Test Session (Cont.)

The TA selects [Set] to confirm the current test settings
and return to the list of students awaiting approval

The TA will still have to approve the student for testing
OR

The TA selects [Set & Approve] to establish the existing
settings and approve the student for testing
Note: If a student’s test settings are incorrect, TAs should deny
the student, contact the School CAASPP Coordinator to correct
the test settings in TOMS, and test the student on another day
51
Administering a Test Session (Cont.)

Upon reviewing test settings for all students, the TA
approves students to test on the Approvals and Student
Test Setting screen


52
Select [Approve] to approve a specific student for testing
Select [Approve All Students] to approve all students
Administering a Test Session (Cont.)

After the TA has approved a student for testing, the
student will verify the test information and settings

53
The student will select [Yes, Start My test] to begin testing
or [No] if their settings are incorrect
Administering a Test Session (Cont.)

If a student has a text-to-speech accommodation or is
taking a test with ELA listening items, he or she will
verify the audio checks through headphones
Test with listening items audio
check
Text-to-speech audio check
54
Administering a Test Session (Cont.)

Finally, students will see the Test Instructions and Help page
before they begin testing

55
Students will select [Begin Test Now] to begin testing
Monitoring Student Progress

TAs can monitor student testing through the Students in
Your Test Session table on the TA Interface

56
Displays students who have logged in and been approved for
testing
Pausing Test
Pgs. 24-25 TAM
TAs may pause an individual student’s test in the Students in
Your Test Session table
− This will not affect other students’ tests

57
General Pause Rules

If there is a technical issue (i.e., power outage, network
failure), students will be logged out and the test will be
automatically paused. Student work will not be lost it will
be saved.

Students log back on with Session ID that was written on
board to pick up where they left off
If a test is paused, students will have to log back in to
resume testing
Note:
 Highlighted text and global notes will not be preserved

58
Pause Rules: Computer Adaptive Test (CAT)

If a CAT is paused for more than 20 minutes, the student is:




Presented with the test page containing the test item he or she
was working on (if the page contains at least 1 unanswered item)
OR
Presented with next test page (if all items on the previous test
page was answered)
Not permitted to review or change any test items on previous
test pages
If a Cat is paused for less than 20 minutes, the student is:


59
Presented with the test item or passage he or she was working
on when the test was paused
Permitted to answer previously shown items within a segment
Pause Rules: Performance Task

There are no pause rules for PTs


Even if a PT is paused for more than 20 minutes, the student
can return to the current section and continue
ELA PTs are divided into two parts


60
After a student completes the first part, he or she cannot
return to it
A message is given to students letting them know they cannot
return
Timeout Due to Inactivity


As a security measure, students are automatically logged
out of the test after 30 minutes of inactivity
Activity means



61
Selecting an answer
Using a navigation option in the test (i.e. selecting [Next] or
[Back], using the questions drop-down list)
NOTE: Moving the mouse or selecting an empty space on the
screen is NOT considered activity
Stopping an Entire Test Session

TAs can stop a test session ( and therefore pause tests
for all students in a session

Select the [Stop Session] on the upper-left corner of the TA
Interface

An “Important!’ box will appear, requesting verification to end
the session and log students out
Select [OK] to continue or [Cancel] to keep the test session
open

62
Reaching the End of the Test

After students answer the last item of the test, the [End]
test button will appear in the upper-left corner of the
screen
63
Reaching the End of the Test

Upon selecting the [End Test] button, an “Attention”
message will appear

64
Students will select [Yes] to proceed with reviewing answers
and submitting the test
Reaching the End of the Test

Students may review answers before submitting a test
65
Reaching the End of the Test

When a student is ready to submit the test, he or she will
receive a confirmation screen
66
Test Expiration Rules


A student’s CAT test remains active until the student
completes and submits the test or 45 calendar days after
the students have begun the CAT
A PT test remains active for 10 calendar days after a
student has begun it
Important: Student personal information is secure and
must be collected and securely stored at the end of a test
session, then securely destroyed at the end of the test
administration.
67
Classroom Activity Administration
Pgs. 28-29
Guidelines

Purpose: to provide students with important context,
concepts, and key terms prior to taking the PT



TAM
All students will have a classroom activity before they
complete a PT
A class activity may be administered in a classroom or any
other appropriate space
Class activity will take place on a separate day from the
PT

68
No more than a 3-day lapse between the class activity and the
PT administration
Classroom Activity Administration
Guidelines (Cont.)




Computers, projectors, and other technology are allowed
but not required for the class activity
Involves the participation of all students in an
instructional tasks
PT will be completed by individual students on the
testing device
Students may take notes during the class activity, but
notes must be collected prior to taking the PT
69
Classroom Activity Administration
Guidelines (Cont.)

Class activity is designed to fit into a 30-minute window

It will vary due to complexity of topic and individual student
needs

The SC should download the assigned classroom activity
and provide to the TAs at least-1-2 days prior to test

After receiving the activity from the SC, the TA or
classroom teacher should administer the classroom
activity
70
Test Security

There are three different types of test security incidents:
1.
2.
3.

Pg. 64 TAM
Impropriety
Irregularity
Breach
Test Impropriety

Unusual circumstance that has a low impact on the testing individual or
group of students

Low risk of affecting student performance, test security, or test validity

Correctable and containable at the district level

Must be reported to the District Coordinator using the Test
Administration Testing Incident or Impropriety Report Form

May be reported in the Appeals module should an Appeal be required

Example: Students talking
71
Test Security (Cont.)

Test Irregularity

Unusual circumstance that impacts the testing individual or group of
students

May affect student performance, test security, or test validity

Correctable and containable at the district level

Must be reported to the District Coordinator using the Test
Administration Testing Incident or Impropriety Report Form, who
must them report it to the CDE within 24 hours

Submitted in the Appeals module for resolution


72
Read pgs. 20-21 for appeals information
Examples: student cheating or providing answers to another,
student accessing or using electronic equipment (i.e., cellphones)
Test Security (Cont.)

Test Breach

Event that threatens test validity (i.e., release of secure
materials)

A breach incident must be reported to the CDE within 24
hours

Examples:



73
Administrator or coordinator modifying student responses or record
at any time
Administrator allowing students to take home test items, passage,
prompts, or scratch paper
Adult or student posting items or test materials on social media
Resources

CAASPP TA Training Modules


http://caaspp.org/training/caaspp/
TA Directions for Administration

http://caaspp.org/rsc/pdfs/CAASPP.ta-odfa.2015.pdf
Accessibility Supports Resources

SBAC Usability, Accessibility and Accommodations Guidelines
 http://www.smarterbalanced.org/wordpress/wpcontent/uploads/2014/08/SmarterBalanced_Guidelines.pdf

Guidelines Frequently Asked Questions
 http://www.smarterbalanced.org/wordpress/wpcontent/uploads/2013/12/SmarterBalanced_Guidelines_FAQ.pdf

TOMS online Student Test Settings User Guide

74
http://caaspp.org/rsc/pdfs/CAASPP.student-test-settings.2015.pdf
Resources (Cont.)
Accessibility Supports Resources (Cont.)
 Matrix One
 http://www.cde.ca.gov/ta/tg/ai/documents/caasppmatrix1.pdf
 Embedded Universal Tools and Online Features Training
Module
 http://www.brainshark.com/ets/vu?pi=zI9zwnui5zJ0cZz0&int
k=37891690
75
Site Support

Assessment: 307-4515 [email protected]





Questions, Policy and Procedures, Passwords, Test Security Issues
Tests not loading
TOMS and CAASPP website issues
Student login to secure browser
Technology/ IT Helpdesk: [email protected]



Broken keyboards
Student logins (to computer or tablet not secure browser)
Forbidden applications running in background
Do not call or email IT helpdesk for questions regarding TOMS or CAASPP Website issues

CALPADS: [email protected]


Enrollment
Special Education: [email protected] or Program Specialist for your school


76
Accessibility Supports
IEP questions
Questions
77