Transcript CMAS Colorado Measures of Academic Success

Colorado Measures of Academic
Success (CMAS)
Training for SAC’s
Agenda

Pearson Access
◦ Online test management

SAC Responsibilities
◦ Communication
◦ Scheduling
◦ Test Security


Proctor Training
Accommodations
Windows, grades, content areas
Assessment Window
April 14-May 2, 2014
Science grades 5 and 8
 Social Studies grades 4 and 7

Frameworks

The frameworks list
◦ Percent representation and number of score points
for each of the reporting categories/standard areas
that will appear on the summative assessments
◦ Evidence Outcomes eligible for inclusion
 Topics explicitly identified in the Social Studies standards are
legitimate content for the assessment in addition to the
standards' concept and skills

http://www.cde.state.co.us/assessment/newassesssum
Changes…..
Online Assessment vs Paper Pencil
Will be a shift for teachers, proctors,
students, parents, administratorscommunicate regularly
 Using Pearson Access to manage your
testing
 Accessibility features vs Accommodations
 Learning tech readiness for you and your
building
 More rigorous, higher DOK

Similarities
Test security is important for valid and
reliable scores
 Ethical administration and procedures and
ensuring standardized conditions
 Scheduling will have some hurdles just as
when we first scheduled TCAP
 Active Proctoring is a must

Test Structure

Each grade level and content area consists of three
sections

The assessments are composed of the following item
types:
◦
◦
◦
◦
◦
Selected Response
Constructed Response
Computer-enabled/technology-enhanced
Simulations (science)
Performance Events (social studies)
What are the Responsibilities of
School Assessment Coordinators?
SAC Responsibilities include
Communication
 Scheduling
 Test Security and Ethics

Technology
 Pearson Access
 Proctor Training
 Online Test prep

Key SAC Responsibilities
THROUGHOUT Testing
◦ Understand all CMAS procedures and
requirements
◦ Employ school level security plan
◦ Serve as liaison between DACs and Test
Administrators
 Read all communications from DAC and distribute
as appropriate
 NEW for CMAS: during testing, school level
personnel will have direct communication with
Pearson
Key SAC Responsibilities BEFORE
Testing
◦ Participate in district trainings and review all manuals
◦ Working with STC, create the testing schedule for your school,
including accommodated computer-based and paper-based testing
groups and make-up sessions
◦ Determine school specific procedures (make-up testing,
communication plan)
◦ Determine school level security plan
◦ Provide and ensure completion of all training by Test Administrators
and anyone else you have authorized or expect to come in contact
with secure materials or environments
◦ Receive, secure, and distribute all test materials
◦ Manage and return security and training compliance documents
Key SAC Responsibilities BEFORE
Testing-PearsonAccess
◦ PearsonAccess
 Set up user accounts for all necessary personnel
 Identify and maintain a list of students requiring the text-to-speech
form and accommodations (paper-based and computer-based
administrations).
 Verify that PearsonAccess accounts for students needing:
 Paper forms of the assessment, including braille and large print
 Oral scripts (paper and online)
 Request from your DAC additional secure and non-secure materials
as needed
 Assign all students to a test session
 Assign students requiring special forms for the computer-based
assessment to the appropriate form
 Register/add new students
Key SAC Responsibilities BEFORE
Testing

Establish the testing environments

PROCTOR CACHE!!!-simply choose the correct
proctor caching server from the down menu when
creating test sessions in pearsonaccess

Prepare students
◦ Meet with students to review school testing policies
and expectations, including that test results should be
reflective of individual effort.
◦ Ensure all students are familiar with TestNav 8.0—
ePats!
Key SAC Responsibilities DURING
Testing






Ensure that all students are assessed in accordance with
CMAS policies and procedures
Request additional secure and non-secure materials from
DAC as needed
MONITOR TESTING ACTIVITY
Distribute and collect paper testing materials daily
Report suspected testing irregularities (misadministrations,
security breaches, technology glitches). Work with DAC to
investigate and declare misadministrations.
Account for the assessment of all students and ensure that all
students have completed testing. Administer make-up
sections as needed.
Key SAC Responsibilities AFTER
Testing
◦ PearsonAccess:
 Account for the assessment of all students and ensure that all
students have completed testing
 Verify all testing sessions have been completed
 Verify that PearsonAccess has been updated to reflect the
accommodations used during testing, including students taking
paper-based assessments
◦ Transcribe student responses from large print and braille test books
into scannable test books.
◦ Collect, sort, secure, and return all test materials to DAC by district
determined date
◦ Manage, submit and retain compliance documents
SAC RESPONSIBILITIES:
COMMUNICATION
Communication





Lead contact for questions in their building
Always available to school proctors during
all testing sessions in case of tech or test
administration question or concern
Lead communication with building
technology staff
Be part of communication with building tech
staff and district IT
Building communication to DAC
Who to contact with questions

Test administration
◦ Proctor to SAC
◦ SAC to DAC
◦ DAC to CDE
◦ Tech questions
 Building tech to district IT-inform SAC of any tech
issues so they are aware
Assessment
Technology
CDE
SAC RESPONSIBILITIESSCHEDULING
Definitions

PearsonAccess Test Session: Virtual data grouping of
students who are grouped together within PearsonAccess
for test management purposes. Each PearsonAccess test
session consists of one grade level and one content area
(three sections). Students must be placed in a scheduled
PearsonAccess test session before they can log in to a test.

Test Section: The parts of a CMAS: Science and Social
Studies assessment. Each grade and content area of the
CMAS: Science and Social Studies assessments include three
sealed test sections.

Physical Testing Group: Students who test within a
physical testing environment at the same time.
Scheduling

Create school wide testing schedule
◦ including accommodations and make ups

Proctor scheduling: Suggest proctor and
tech savvy person both be in room so if
students are stuck with tech issues
proctor isn’t stuck with one student
CMAS Scheduling Considerations

Each grade level and content area consists of 3 sections

Test sections can only be administered to students in consecutive
order

One grade level and one content area at a time except for make-ups

To the extent possible, all students within a grade should be assessed
at the same time. Should that not prove possible, all students should
be assessed within the shortest timeframe practicable (i.e., in back-toback sections, within the same day or consecutive days if more
sections are needed than can be administered in a day, etc.)
Scheduling considerations cont…

Technology
◦
◦
◦
◦
Number of compatible devices
Bandwidth
Wireless access points
How many concurrent testing sessions can you
run?-use the site readiness tool
◦ Extra computers

Drag and drop on laptop vs ipad vs desktop
computer
◦ Fine motor considerations?
◦ External Mouse available? Vs touch pad
Section Timing

90 minutes per section
◦ Includes 10 min. for login, directions, sample
items
◦ 80 minutes of actual test time for every
student (required)
◦ 55 minutes minimum required
3 sections in Science
 3 sections in Social Studies

CMAS: Test Structure and Timing
CMAS: Science and Social Studies Section Administration
Time for Elementary and Middle School
Total
Section
Time
90 minutes
Time for Set Required
up and
Testing Time
Transitionfor All
approximate
Students
ly-may take
more or less
10 minutes
55 minutes
Required
Additional
Testing Time
for all students
25 minutes
Time for Set up and Transition
Includes….





Start up TestNav 8.0
Log in
Test transfer (proctor cache…)
Give directions
Review sample items
Best Practices-Recommendations
2 extra devices in each testing room
 Minimum: Two proctors per each testing
room
 No more than 2 test sections per day
(this year)
 Do not start first thing in the morningmake up session
 Students remain in room for at least 55
minutes, up to 80 minutes if at least one
student is still working

Recommendations

Schedule first group/first day with more of a
time cushion than other groups if you can

If you have to do multiple sessions, schedule
smallest group first

Plan for accommodations

Recommendation: Schedule students for
only one section per day (exception: makeups)

Remember: This is Colorado
School CMAS schedules
Email to Kim Terry no later than March
31st
 Also send security of testing materials
form

Make-Up Testing
Priority: minimize risk of prior exposure to
test content that could result in an
invalidation
Make-Up Testing
Student completes missed test
section during make-up testing time
in advance of re-joining original
physical testing group
Sample Make-Up Testing Schedule
Day 1
Regularly
Scheduled
Make-Up
Testing Time
(beginning of
morning)
-
Physical
Testing Group
Testing Time
Physical testing
(scheduled
group takes test
after initial
section 1
morning
make-up
testing time)
Day 2
Absent Student
Takes Section 1
Day 3
-
Absent Student Physical testing
re-joins physical group takes test
testing group for section 3
test section 2
Scheduling Discussion – Guiding
Questions

How do device and facility availability affect scheduling?

What other impacts to scheduling do you see?

How does make-up testing affect scheduling?
◦ How to schedule for students who miss mutiple days?
CMAS or CoAlt

CSAP-A decision not based on:
◦ Category of disability
◦ Time in special education
◦ Program placement

Must be documented by IEP team

Contact Barb Rice @ x21931
if you need clarification
35
SAC responsibility
Test and materials security
Test Security
Testing environment-screen security and
number of computers
 How far apart to sit students
 Can students view from front to back
the rows of screens in front of them?
 Student view side to side
 St authorization tickets
 Student pre exposure to items

Test security

No cell phones in testing room
◦ Collect (baggies with names) so students do
not have them during testing
No student tablets allowed in testing
room
 Ipads are allowed for TEST
ADMINISTRATORS only

◦ No recording
◦ No pictures of anything in test environment
ever
Security of Student Authorization
tickets-one approach
SAC print the day before the test
 Put all into an envelope labeled for that
test

◦ Maybe with a class list on the outside of it,
◦ All used tickets will be collected by the
proctor and put back in that envelope

Have another empty envelope labeled
“MAKE UP TESTING REQUIRED”
◦ Proctors put any unused authorization tickets
Chain of Custody
Count in/out and test administrator sign
for number of student authorization
tickets each time
 Paper forms
 Oral scripts
 Scratch Paper (official CMAS scratch
paper, not any other paper)
 Secure central storage location

◦ Not stored in classrooms
Maintaining Security of CMAS

All secure test materials must be secured while in
the Test Administrator’s possession

No duplication of secure CMAS materials is
permissible

No cell phones or other communication, reproduction
or recording devices are allowed in the classroom
Unauthorized Visitors and the Media

Only students, Test Administrators/Examiners, and
authorized school, district, state personnel or statesanctioned test monitors may be in testing areas during
administration

Media are not allowed to have access to the tests before,
during, or after test administration, or take pictures or
video of testing materials or testing students
Accessibility Features and
Accommodations
Overview
Increased Accessibility

TestNav 8
◦ Embedded features: available to all students to
increase accessibility
◦ Embedded accessibility features that must be
requested ahead of time: available to all students to
increase accessibility
◦ Additional embedded and non-embedded
accommodations available to students consistent with
their IEP, 504, or English learner plans
Embedded Features for All
Students
Tool Bar
 Navigation
◦ Back page
◦ Next page
◦ Review
◦ Flag
 Pointer
 Highlighter (Caution: in development)
 Ruler (cm in 5th and 8th grade, inch in all grades)
◦ Can also be used as line guide
 Calculator (4 function for 5th grade, scientific for 8th grade)
 Eliminate answer choice
Embedded Features for All
Students
Drop Down
 Magnifier
Zoom
 Dependent on device
Writing tools for constructed response
 Bold, italics, and underline
 Bullet and number
 Cut, copy, and paste
 Undo and redo
Embedded Accessibility Features and Accommodations
that Must Be Requested Ahead of Time for
Computer-based Administration


Selected by form
1. Text-to-speech with volume control
 Continue to play
 Click to hear – recommend students are taught to
default to this
2. Color contrast (accommodation)
3. Text-to-speech with color contrast (accommodation)
4. Oral script (accommodation)
Samplers for text-to-speech and text-to-speech with
color contrast will be available in the second week of
February
Accommodations

Must be based on an individual need documented in the
student’s approved IEP, 504, or English learner plan

A master list of all students and their accommodations
must be maintained by the SAC. These will need to be
recorded in PearsonAccess. Please include students
using text-to-speech.

Accommodations are available for computer-based and
paper-based administration.
◦ Paper-based form is itself intended to be an
accommodated form
Paper Pencil or Online?





Paper-based Assessments: A paper-based form
of the assessments will only be available for
students who:
Have a neurological disorder, a condition that
causes seizures or another health condition that
prevents the student from accessing the
computer
Need a braille form with tactile graphics
IEP or 504 plan requires assistive technology, such
as speech-to-text, that may not be compatible
with the test engine.
www.pearsononlinetesting.com/at
Will provide written responses in a language
other than English or Spanish
Accommodations for Computerbased Administration

Presentation
◦ English oral script used for:
 On-site translations into languages other than Spanish*
 On-site sign language presentation*
◦ Spanish oral script
◦ Color contrast, including high contrast and reverse contrast
◦ External cover overlays
◦ Word-to-word dictionaries (English/native language)
◦ Assistive technology devices – compatibility will need to be established
*Due to the number, length and complexity of sources in
the Social Studies assessment, it is recommended that this
accommodation be provided individually when practicable.
Accommodations for Computerbased Administration

Response
◦ Written responses in Spanish
◦ Scribe (must be individually administered)
 English
 Other languages
◦ Assistive technology devices – compatibility will need
to be established
Accommodations for Computerbased Administration


52
Setting
◦ Small group
◦ Individual
Timing
◦ Extended time greater than time and a half
◦ Time of day
◦ Multiple breaks (requires separate setting)
Assistive Technology

Devices with demonstrated compatibility with
TestNav8:
www.pearsononlinetesting.com/at
Accommodations for Paper-based
Administration
The paper-based form of the assessment is itself considered an
accommodation.
Use of the paper-based assessments is expected only to be
available to students who:

Have a neurological disorder, a condition that causes seizures, or another
health condition that prevents the student from accessing the computer*

Need a braille form with tactile graphics

Have a IEP or 504 plan that requires assistive technology, such as speech-totext, that may not be compatible with TestNav 8

Will provide written responses in a language other than English or Spanish
*Students with disabilities that prevent them from utilizing devices safely may
meet this criterion.
Accommodations for Paper-based
Administration

Presentation
◦
◦
◦
◦
Braille with tactile graphics
Large print (Reminder: computer-based has zoom and magnifier)
External cover overlays
English oral script (text-to-speech in computer-based)*
 On-site translations into languages other than Spanish
 On-site sign language presentation
◦ Spanish oral script (also computer-based)*
◦ Word-to-word dictionary (English/native language)
*Due to the number, length and complexity of
sources in the Social Studies assessment, it is
recommended that this accommodation be provided
individually when practicable.
55
Accommodations for Paper-based
Administration

Response
◦ Braille note-taker
◦ Scribe (also for computer-based)
 Must be administered individually
◦ Speech-to-text
◦ Assistive technology devices (word prediction)

Setting (same as computer-based)
◦ Small group
◦ Individual

Timing (same as computer-based)
◦ Extended time greater than time and a half
◦ Time of day
◦ Multiple breaks (requires separate setting)
SCHOOL
TECHNOLOGY
RESPONSIBILITIES
School tech




Java updates
shortcut or LAN school set up using login
given by IT that locks out other programs,
puts test nav screen on every computer
Site readiness tool so # of devices to run at
one time is known
http://systemcheck.co.testnav.com/systemCh
eck.jsp?acc=co_chk
Refer to district IT tech training for more
information
System check
http://systemcheck.co.testnav.com/
 If you have the specifications for which
browsers and versions work with which
operating systems, etc you may not need
this tool

Results from system check-Failed
System check-Passed
Without proctor caching- 26
students
With proctor caching
With proctor caching- 466 students
Administering the test
TEST ADMINISTRATOR
TRAINING
SAC Responsibilities
Administration Training


Read Procedures Manual in its entirety after
today’s training and prior to training your
building
A face to face proctor training is required by
you to your building proctors prior to the
start of the testing window
◦ Use the Proctor Training ppt. provided, it includes
ethics and test administration

Keep all signed confidentiality agreements
onsite for all of your trained CMAS proctors
66
CMAS Test Administrator








Individual at school level responsible for directly administering
the CMAS: Science and Social Studies assessments
Must be employed by the school/district
Be trained and have read the Test Administrator’s Manual prior
to any assessment administration
Be trained in any specific accommodations they are providing
Must sign security agreement before assessment administration
begins
Read the directions to students exactly as they appear in the
Test Administrator’s Manual
Test session management (school decision)
Administer sections actively
Active Administration
Active Administrators:






Ensure they have all necessary
materials for each session
Ensure a standardized testing
environment
Follow all scripts exactly as
written
Move throughout the room
during testing
Read directions to students when
asked
Use proximity to keep students
on task
Administrators May Not:








Provide feedback
Clarify test questions
Answer content related questions
Interfere with the students’
demonstration of skills
Interact with students in any way that
would impact student responses
Engage in other tasks during test
sections
Read sources, items, or
student responses
Use any electronics during testing
(other than to monitor in pearson
access if sac deems it necessary)
Administration Steps








Prepare the test environment
Pick up student authorization tickets/sign out from
SAC
Make sure all students from list are present or
marked absent on envelope with absent students
tickets put into the absent envelope
Start session in PearsonAccess-or SAC can doschool decision
Follow directions and read script in the Test
Administrator’s Manual
Hand out student authorization tickets and scratch
paper
Assist students in logging on to TestNav
Put Seal Code on white board or screen
Administration Steps-cont…
Complete directions to students and
practice items if there are practice items
 Actively administer the test section
 Collect student authorization tickets and
scratch paper
 Erase Seal Code from board completely
 Close session in PearsonAccess after third
section-SAC can do at end of test window
because if any student absent/not marked
complete will need to either take all sections
or have the SAC “mark complete” in
PearsonAccess

Test Administrators-helpful hints
from field test
Administrators should not hand out
authorization tickets until students are
seated in the lab
 Have students log in in waves
 Once students are logged in, proctors
should direct them to put hands off of
keyboard while directions are read
 Directions should be that students do not
touch keyboard until proctor says so

Secure Materials- Test
Administrators
Collect back student authorization tickets
immediately after students login
 Put student authorization tickets into
correct envelopes
 Return envelopes to SAC’s immediately
after testing
 Sign in the number of student
authorization tickets being returned

Test Environment
No food or drink on desks
 No content posted that would give
students an advantage
 Put up “Do Not Disturb” sign
 No music
 Same rules as TCAP
 Bathroom-don’t log students out or pause
their tests just put something (blank) in
front of screen so no one can see it

Contact Pearson directly
If a technology issue arises during an
actual live test session test administrators
should contact Pearson directly-after
informing SAC of the issue
 This is the only time test administrators
need to/should contact Pearson directly
 Tell them you are in a live testing situation
and they will bump you to the front of the
line

◦ Pearson: 1-888-687-4759
Testing Irregularities

Fill out form online if any testing
irregularity
◦
◦
◦
◦
Misadministrations
Security breaches
Technology glitches
Emergencies/unforeseen circumstances
Misadministrations

Students moving onto next section
◦ Really only able to do if in a make up section and they
have access to the seal code to the next section
Student receives help from anyone on a test item
 Students uses a cell phone, calculator, electronic
device
 Student is not given accommodations they
require
 SAC: call Kim Terry if any misadministration has
occurred

Breach of secure test materials
Discussing, reproducing, transmitting by
any means any secure test materials
 Student obtaining or sharing secure test
materials
 Secure test materials include:

◦ Student authorization tickets
◦ Official CMAS test scratch paper
◦ Seal codes
Emergencies

Student becomes ill
◦ Log that student out-directions on next
screen
◦ Record amount of time left in section
◦ Record onto the absent student list that the
student left and the amount of testing time
left
To allow ill student to finish later
Emergencies

Emergency requiring the test to be
stopped:
◦ Safety first, evaluate and respond to the
emergency
◦ If possible without risk to students, note time
left in session
◦ Students can have time left in session
When students are done
Students all remain in room until all
students are done -recommended
 Students must be allowed to have 55
minutes minimum-required

◦ If all students are done before 55 minutes, all
students must remain in room for 55 minutes
and then can all be dismissed-required
◦ All students are allowed 80 minutes, so if any
student is still working after 55 minutes, ALL
students must remain in testing room until
ALL students are done or 80 min. has passedrecommended
How students submit their tests
They should choose that they are finished
and want to submit their final answers
 This means they cannot go back into that
section
 They can practice this step in the epats
also

Technology issues
Practice Tests
E PATS
Test preparation for online
assessments
ePATs
 ePat guides for proctors are so helpful
 Familiarize students with tools, give
opportunity to use
 Practice with scrolling, drag and drop
items, simulations

ePats: Practice Tests

www.pearsonaccess.com/co
Receiving and
Returning Test
Materials
Test Materials

Online and Paper
◦
◦
◦
◦

Test Administrator’s Manual
Scratch Paper
CMAS Secure Return Envelope
School Security Checklist
Online Only
◦ Student Authorization Tickets
◦ English and Spanish Oral Scripts for Online Form
◦ Onsite Translations into Languages other than English/Spanish

Paper Only
◦
◦
◦
◦
Test Books, including Braille and Large Print
Source Book (Social Studies Only)
English and Spanish Oral Scripts for Paper Form
Onsite Translations into Languages other than English/Spanish
Receiving Test Materials

Materials will arrive via district warehouse-date tbd
◦ Purple labels on all boxes
◦ Green labels on CMAS boxes
◦ Pink labels on CoAlt boxes
Receiving Test Materials
◦ SAC to verify materials
received with the School
Packing List
Returning Test MaterialsSACs
SACs

Collect all secure materials and group as follows:
 Secure return envelopes
 Scorable test books
 Unused test books
 Accommodated materials
 Electronic materials must be removed from
any devices after testing and verification sent to the DAC

Prepare materials for packaging
 Check and deliver CMAS scorable test books
to the DAC
 Check and deliver CMAS non-scorable materials
to the DAC
 Return materials with original boxes
Questions