Transcript TELPAS Rater Manual

2013-2014 Holistic Rating
Training Requirements
Texas Education Agency
Student Assessment Division
Rater Credentials
Each teacher selected to rate an ELL must
1) have the student in class
2) be knowledgeable about the student’s ability
to use English in instructional and informal
settings
3) hold valid education credentials such as a
teacher certificate or permit
4) be appropriately trained, as required by TEA
TEA 2
TELPAS Rater Responsibilities
 A student’s TELPAS rater is the teacher
designated by the district as the official rater
of the student’s English language proficiency.
 The student’s rater must rate the student in
all domains for which the student is eligible.
A student is not permitted to have one rater
for some domains and another rater for other
domains.
TEA 3
New and Returning Rater Definitions
for K–1 and 2–12
•The training webpage informs raters that districts may sometimes require a
returning rater to complete new-rater training and to consult with their
testing coordinator if they need clarification.
TEA
4
TELPAS
Administration Procedures Training
 As part of annual spring TELPAS administration
procedures training, holistic rating training
requirements are reviewed with raters, as well as
information about how to access the online
training and calibration components.
 In addition, raters receive training on assessment
procedures such as how to assemble writing
collections, how to record students’ proficiency
ratings, etc.
TEA 5
Two Types of Training
 Online basic training course
This course is for new raters. It provides
instruction on using the PLD rubrics and gives
raters practice rating students in each language
domain. There are separate courses for K–1 and
2–12.
 Online calibration
This is for all raters. Raters use the PLDs to rate
students in each language domain. Raters have
three opportunities to calibrate on assigned
grade cluster.
TEA
6
Training and Calibration
Grade Clusters
 Raters must know their assigned grade cluster to
select the appropriate online training.
 Raters should consult with their campus testing
coordinator if they are unsure of their assigned
cluster.
Grade Clusters
Grades K–1
Grade 2
Grades 3–5
Grades 6–8
Grades 9–12
TEA
7
Preparing for Calibration Sets
 New raters must complete the online basic
training course before beginning
calibration.
 All raters have the option to review the
online basic training course (which
includes practice rating activities) before
beginning calibration.
TEA
8
Other Things to Know
About Calibration
 Raters affirm online that they will keep the contents
of the calibration sets secure and confidential.
 Calibration activities are taken from a bank and
randomized. Trainees will rate different sets of
students.
 Raters can work at their own pace, go back and
review students, and change ratings as they work.
TEA
9
Other Things to Know
About Calibration
 Raters can exit and return later to finish. They click a
“submit” button when they are finished with a set.
 After completing a calibration set, raters immediately
see results. Results show both the rating assigned by
the rater as well as the correct rating.
 Raters see annotations explaining the correct ratings.
Raters should use the annotations to go back and
review any incorrectly rated students.
TEA 10
Is calibration a test?
No, it is a training method that ensures
that raters have enough guidance,
practice, and support to assess students
consistently and accurately.
TEA 11
Will raters be able to refer to any
resources during calibration activities?
Yes, raters should use their rating rubrics (PLDs)
and refer to, as needed, information from the:
 online basic training course
 holistic rating PowerPoints produced by TEA
 TELPAS Rater Manual
 Educator Guide to TELPAS
TEA 12
How many students must be rated
successfully?
To be successful, raters need to rate
students in their assigned grade cluster
with at least 70% accuracy.
TEA 13
Supplemental Holistic Rating Training
 Raters not successful after sets 1 and 2 must
receive supplemental training.
 The rater will meet with a district-appointed
supplemental support provider.
 After the rater has received supplemental training,
he or she will be able to access the third and final
calibration set.
TEA 14
Recap of Calibration Process
 There are 3 sets of 10 students.
 Raters who calibrate on set 1 are done.
 Raters who don’t calibrate on set 1 go on to set
2. Raters who calibrate on set 2 are done.
 Raters who don’t calibrate on set 2 receive
supplemental training.
 Raters attempt third and final calibration set.
 Raters who calibrate on set 3 are done.
TEA 23
Course and Calibration Certificates
 Raters who take the basic training course get a
certificate from the online TrainingCenter after
completing the course components.
 Raters will receive a certificate of successful
calibration when they calibrate.
TEA 16
Writing Collection
Overview
2013–2014
Texas Education Agency
Student Assessment Division
Building Collections
 Strive to gather more than 5 writing samples for
each student.
 Choose at least 5 samples that meet the criteria
and do the best job of portraying the student’s
current proficiency level.
 If a student is near the border between two
proficiency levels, consider including samples
written in the latter part of the TELPAS assessment
window. The more recent the writing samples, the
more accurately they will reflect the proficiency
level of these students.
TEA 18
Building Collections
 Collections should contain some papers in which
students showcase English they know and feel
comfortable using. “Comfort zone” writing is
especially important for students at lower
proficiency levels.
 Collections should also include papers in which
students are stretched and pushed beyond their
comfort zone so the collection shows that a
student has not yet reached the next level (the
student is beginning but not yet intermediate,
intermediate but not yet advanced, advanced but
not yet advanced high).
TEA 19
Building Collections
 The papers you assemble need to help you
determine and justify your ratings. They must give
you evidence to say, “I know the student is at least
at X proficiency level because of these
characteristics in his or her writing. I know the
student is not yet at the next proficiency level
because of these other characteristics in his or her
writing.”
Reminder: The characteristics you
consider must come from the PLDs.
TEA 20
Beginning Level
 The ELPS require ELLs of all proficiency levels to
learn to write in English.
 It is not acceptable for writing collections of
students at the beginning level to include samples
written primarily in the native language. Such
samples do not provide evidence of English writing
proficiency.
 Writing tasks of these students should be adapted
to their needs. Their writing in English will likely be
formulaic or memorized, include recently practiced
vocabulary, lack detail, etc.
TEA 21
Summarizing Tips
 Students who are capable of expressing
themselves in English in a detailed,
extended way should do so. Do not include
brief responses from students who know
enough English to respond to writing tasks
in extended ways.
 In other words, consider students’ Englishlanguage proficiency levels in determining
whether papers are too brief to be
included.
TEA 22
Summarizing Tips
 Build writing collections that have a balance of
writing from language arts and other core
content areas.
 Collections should show what the student knows
and can do as well as what the student struggles
with in second language acquisition.
 Remember, build the collections to portray the
student’s overall ability to communicate in
writing in English.
TEA 23
Verification of
Collection Contents
Campuses follow procedures outlined
in the TELPAS Rater Manual to ensure
that the writing collections are
assembled correctly and include the
necessary number and types of
writing.
TEA 24
Resources with More Details
 District and Campus Coordinator Manual
 Updated annually and available in late fall from the
Assessment A–Z Directory at
http://www.tea.state.tx.us/student.assessment
 Shipped to districts in January
 TELPAS Rater Manual
 Updated annually and available in December at
http://www.tea.state.tx.us/student.assessment/ell/telpas/
 Shipped to districts in January
 Online course titled Assembling and Verifying Grades 2–12
Writing Collections
 Available online January 13 at
http://www.texasassessment.com/TexasTrainingCenter TEA
25