Transcript TELPAS-TETN8302-SpringRatingTraining

Texas English Language
Proficiency Assessment
System (TELPAS) Holistic
Rating Training
Audience: Teachers who will serve as
TELPAS raters
General Information about
TELPAS (p. 3)



Fulfills federal requirements for assessing
English Language Learners (ELLs), K-12
Aligned with the Texas English Language
Proficiency Standards (ELPS)
Results used in state and federal
accountability and performance-based
monitoring system
2
Holistically Rated Components
of TELPAS (p. 3)
Grade
Levels
Language Domains
K–1
Observational assessments in listening,
speaking, reading, and writing
2–12*
Observational assessments in listening and
speaking; writing collections for writing
*Online multiple-choice reading tests are used to assess the
reading domain for grades 2–12.
3
Georgetown ISD Calendar of Events for
TELPAS (p. 4 – Scheduling)
Events
Dates
Administration procedures training for holistic component
By February 1
New raters complete online basic course (K-1 or Gr. 2-12)
By February 4
Returning raters review TELPAS basics (see Slide 26 for
resources)
Before attempting
calibration activities
All raters complete online calibration activities
January 31 – Feb. 11
Collect samples for Gr. 2-12 writing collections
February 1 – March 25
Training for Gr. 2-12 TELPAS Online Reading Test
By March 4
TELPAS Assessment Window (holistically rating all ELLs &
completing online reading test for Gr. 2-12)
March 7 – April 1
Campus Verifiers ensure that Gr. 2-12 writing collections
meet all assembly criteria
March 25 - 31
Extended Collection Window (if needed to collect more
writing samples)
March 28 – April 4
Extended Assessment Window (to rate/test ELLs in
extenuating circumstances)
April 4 – 8
Deadline for CTCs to enter all holistic ratings in the Texas
Assessment Management System
April 8
4
Test Security & Confidentiality
Requirements (p. 5)





TELPAS – secure testing program (just like TAKS)
Only trained, certified staff and/or their trained
designees who have signed oaths have access to
test materials
Must receive training & sign oath before handling
test materials, confidential information, or
administering a test
Must administer tests in strict accordance with
instructions in manuals
No duplication of secure materials without TEA
approval
Test Security & Confidentiality
Requirements (p. 5)



No viewing, revealing, or discussing test content nor
answering questions about test content before,
during, or after a test administration unless
specifically authorized to do so by test
administration procedures
No reviewing or discussing student responses
during or after a test unless authorized to do so by
test administration procedures
No changing or amending any student response or
instructing a student to do so
Security Oaths (p. 6)



Oaths for TELPAS raters, writing collection
verifiers, and test administrators found in
Appendix A (p. 107)
Each oath must be read and completed
AFTER training and BEFORE handling or
viewing secure test materials or confidential
information.
Bottom of oath (in the box) must be signed
AFTER completion of test administration.
Testing Irregularities (p. 6)



Definition: Incidents resulting in a deviation
from documented testing procedures
Each person is responsible for reporting
immediately to the campus or district testing
coordinator any violation or suspected
violation of test security or confidentiality.
Failure to report is in violation of 19 TAC,
Chapter 101 and could results in sanctions.
Penalities for Prohibited Conduct (p. 7)




Placement of restrictions on the issuance, renewal,
or holding of a Texas educator certificate, either
indefinitely or for a set term.
Issuance of an inscribed or non-inscribed reprimand.
Suspension of a Texas educator certificate for a set
term.
Revocation of a Texas educator certificate without
opportunity for reapplication for a set term or
permanently.
Additionally, student test results may be invalidated.
Examples of Testing Irregularities
(p. 7-8)





Eligibility Error
IEP Implementation Issue
Improper Accounting for Confidential or
Secure Materials
Monitoring Error
Procedural Error
Potential Referral to the Educator
Certification and Standards Division (p. 9)






Viewing a test (unless authorized to do so)
Scoring a test
Discussing secure test content, student
responses, or student performance
Duplicating, printing, capturing, or photographing
a secure test
Assisting students with responses
Tampering with student responses or holistic
ratings
Reporting of Disciplinary Actions Against
Students for Cheating on State
Assessments (p. 9)




A district may invalidate a student’s test if the district
determines that the student cheated.
Additional disciplinary action may be taken at the local
level.
Any local actions must be submitted to TEA using a
special online reporting form.
A separate online incident report form and
documentation are necessary ONLY IF the district
determines that testing personnel contributed to, caused,
or did not detect the cheating due to inadequate
monitoring or another error.
Rater
Responsibilities
(p. 11 - 49)
Overview of the Rating &
Reporting Process (p. 13)




Districts identify ELLs in grades K–12 and designate
raters to assess students
Training on test administration procedures and holistic
rating is conducted to prepare grades K–12 raters to
assess English language proficiency consistent with
the holistic scoring rubrics, the Proficiency Level
Descriptors (PLDs)
Raters gather writing samples for ELLs in grades 2–12
to include in each student’s writing collection
Raters use PLDs to assign students a rating in each
language domain assessed
14
Rater Credentials (p. 13-14)
A TELPAS Rater is a teacher designated
to be the official rater of an ELL’s English
language proficiency.
Each TELPAS Rater must:





Have the student in class during spring semester
Be knowledgeable about the student’s ability to use
English
Hold valid teacher certificate or permit
Successfully complete holistic training & calibration
activities
Rate the student in all eligible language domains
Eligibility Requirements (p. 14)
All ELLs in Gr. K-12, including ELLs with
parent denials, are required to be
assessed in all language domains, with
the following exceptions:

ARD Decisions: only in rare cases and made on a
domain-by-domain basis as documented in the student’s
IEP

Other Rare Exceptions: such as for new enrollees
Accommodations (p. 15)

Holistic ratings are based on student classwork and
observations of student in daily instruction.

For ELLs receiving SPED services, raters take into
account the ability of students to use English to access
the general curriculum at their enrolled grade level in
accordance with accommodations called for in their IEP.

Accommodations generally should not interfere with the
holistic rating process.
Talk to your Campus Testing Coordinator if you have
any concerns in this area.

Rater Checklist (p. 16)

Will help you keep track of holistic rating
activities

We will go through each activity in more detail in
the next set of slides.
Activity 1: Complete Training on
Administration Procedures (p. 17)



Today’s training is a part of your training on TELPAS
holistic rating procedures.
Read and study the TELPAS Manual for Raters and
Test Administrators.
Sign your Oath of Test Security and Confidentiality
for TELPAS Rater on p. 109 after this training
session and return it to the Campus Testing
Coordinator (CTC). If you need more time to read
the manual before you feel comfortable signing the
oath, then return it by the date designated by your
CTC.
Activity 2: Complete Holistic Rating
Training Requirements (p. 18-23)

An online training process is used.

New raters complete an online basic training
course. (The course is optional, but highly
recommended, for returning raters.)

Both new and returning raters complete
calibration activities.
All eligible language domains are included in
the calibration activities, rather than just the
writing domain.

Activity 2: Page 19
Spring TELPAS Administration
Procedures Training
Covers holistic rating training requirements
and procedures for conducting TELPAS
New Raters
Online Basic Training
Course
Online
Calibration Activities
Sets 1 and 2:
Set 2 required if not successful
on Set 1
Returning Raters
(Basic Training Course is optional for Returning Raters.)
If not calibrated:
Supplemental Holistic
Rating Training
Final Online
Calibration Activity
Set 3
Online
Calibration Activities
Sets 1 and 2:
Set 2 required if not successful
on Set 1
Spring 2011 Online Training
Components (p. 19)

Online Basic Training Course (Required for New
Raters & Highly Recommended for Returning Raters)


One for raters of K–1 and one for raters of Gr. 2–12
(Approximate Completion Time: 4-5 hours)
Online Calibration (Required for New and Returning
Raters)

Separate calibration activities for grades K–1, 2, 3–5, 6–8, and
9–12 (Approximate Completion Time: 1-2 hours)
22
Spring 2011 Online Training
Components (p. 20)




Online courses are designed to prepare raters to
assess the English language proficiency of students
in a uniform manner statewide.
New Raters: Complete Online Basic Training Course
by February 4 (Deadline for GISD New Raters)
Training courses and calibration activities are grade
cluster specific (K-1, 2, 3-5, 6-8, 9-12), so confirm your
cluster BEFORE starting your online training.
If you will rate students from more than one grade
cluster, consult your Campus Testing Coordinator for
instructions.
23
Online Basic Training Course (p. 21)







Complete course in a distraction-free setting.
You may complete the course in more than one sitting.
The rating practice activities must be completed
independently. (online oath included)
Five practice activities are included for each language
domain.
There is no required number of students to rate correctly
since the activities are for practice.
It is a violation of state assessment procedures to share or
discuss answers to the rating practice activities with
individuals who have not yet completed the course.
You will be issued a course certificate upon completion.
24
Online Calibration Activities
(p. 21-22)



Each calibration set contains 10 students to rate, and all
holistically assessed language domains are represented
in each set.
 For K–1: listening, speaking, reading, and writing
 For 2–12: listening, speaking, and writing
A rater needs to rate at least 70 percent of the students
correctly to calibrate.
If a Returning Rater is unsuccessful on the first set, you
must complete the Online Basic Training Course (if not
already completed) and provide your CTC with your
training certificate (GISD requirement).
25
Resources for New & Returning
TELPAS Raters (p. 21)

Educator Guide to TELPAS Gr. K-12 (web-based guide to familiarize
educators with TELPAS; includes student video segments, authentic
student writing, & sample test questions)
http://ritter.tea.state.tx.us/student.assessment/ELL/telpas/K-12InfoGuide.pdf

PowerPoint presentations are available on TEA’s Student Assessment
Division website in the Rater Training for 2010-2011 section at:
http://www.tea.state.tx.us/student.assessment/ell/#telpas



Making the ELPS-TELPAS Connection Grades K12 Introduction
Introductory Training on the PLDs Gr. K-1
Introductory Training on the PLDs Gr. 2-12
(See bottom on p. 21 in TELPAS Rater Manual.)
26
Online Calibration Activities (p. 22)



Raters complete only as many sets as it takes to
calibrate.
 Many raters will be able to calibrate on the first set of
activities, and many others will be able to calibrate by
the end of the second set.
Calibration sets are available on January 31 and must be
completed by February 11 (in GISD).
Individuals who do not calibrate by the end of the second
set will be provided with supplemental training prior to
the third and final set.
27
Supplemental Support After Second
Calibration Set for Raters (p. 22)

Raters not calibrated after Set 2 to get
supplemental support and then complete Set 3
before start of testing window (before March 7)

Raters need access code to complete Set 3

3rd Calibration Set: Available February 16
28
Calibration – Key Points (p. 23)









3 opportunities to calibrate
Resources available as warm-ups (see p. 21 & slide 26)
Online oath
Self-paced
Students presented in random fashion but domain order fixed
(listening, speaking, reading (K-1 only), & writing
Score & rating annotations available right away
Although the 2nd set is available right away, DO NOT ATTEMPT
until you check in with your CTC. If you did not complete the Online
Basic Training Course, you MUST complete it before attempting the
2nd set. Review other support materials as well.
Training is complete after successful calibration on Set 1 or Set 2.
Print Calibration certificate and give it to your CTC.
Raters not successful after 2 attempts will receive supplemental
training before Set 3 can be accessed.
Accessing the TrainingCenter
Website (p. 23)





See Appendix D starting on p. 123 for specific
information about accessing the TrainingCenter website.
Site code for new TELPAS users: telpas2011
New raters must register using the directions on p. 128131
Returning raters already have an account and should
review p. 132-134.
Returning raters should review the “My Info” link and
update email, phone #, region, district, campus, etc. as
needed. Name changes must be handled through
Pearson at 800-627-0225.
30
Updates to the Texas
TrainingCenter
31
TELPAS TrainingCenter
Updates

Improvements made for easier navigation to
the correct training component
32
TELPAS TrainingCenter
Updates
33
TELPAS TrainingCenter
Updates
34
TELPAS TrainingCenter
Updates
35
TELPAS TrainingCenter
Updates
36
Activity 3: Assemble & Verify Gr. 2-12
Student Writing Collections (p. 24-25)








Writing collections must contain at least 5 samples (1 narrative
about a past event; 2 academic from math, science, or social
studies; 2 other samples).
Writing assigned on or after February 1 may be considered.
More than 5 samples should be collected.
The more recent the writing samples, the more accurate the
collection will be.
Include student name and date on all samples.
Staple the cover sheet and verification checklist found in
Appendix C (p. 121-122) to the front of the writing collection.
Photocopies are okay if clear and legible.
Collect samples from February 1 through March 25. Campuses
may set earlier deadlines or interim deadlines, so check with
your CTC.
37
Writing Examples (p. 25-27)






Type 1: Basic descriptive writing on a personal/
familiar topic
Type 2: Writing about a familiar process
Type 3: Narrative writing about a past event
(REQUIRED)
Type 4: Reflective writing
Type 5: Extended writing on a topic from language
arts
Type 6: Expository or procedural writing from
science, math, or social studies (2 samples
REQUIRED)
38
Tips for Good Writing Collections (p. 27)



Encourage students to take their time and write in
as much detail as possible.
Include writing tasks that allow students to truly
showcase the English they know while writing on
topics that are comfortable and familiar.
Also include writing tasks that stretch and push
students’ limits so that you can see the full extent of
their language abilities and any elements of second
language acquisition they still need to develop.
39
What Not to Include in a Collection (p. 28)








Papers containing language directly copied from a
textbook, lesson, or other written source
Papers with heavy reliance on a dictionary or thesaurus
Papers that show a teacher’s corrections
Papers that have been polished through editing by
peers, parents, or teachers
Papers written primarily in a student’s native language
Worksheets or question-answer assignments
Photocopies of TAKS written compositions or responses
to TAKS open-ended questions
Papers that are brief, incomplete, or obviously reflect
writing that was rushed
40
Verifying the Writing Collection Components
(p. 28-29)





TELPAS Raters and paraprofessionals are NOT
permitted to serve as a verifier.
All raters complete the verification checklist (see p. 122)
for their assigned ELL students as well as the front of
the cover sheet (see p. 121).
Campus verifiers (usually the CTC) must complete the
online training course, Assembling and Verifying Gr. 212 Writing Collections, on the Texas TrainingCenter
website by February 28 (GISD requirement).
Collections that do not meet the requirements will be
returned to the rater so that additional samples may be
collected.
The verifier will initial in each box and sign the
verification checklist at the bottom.
41
Activity 4: Prepare to Rate Your Students
(p. 30-36)
Prior to rating your students (& after all online
activities have been completed), review p. 30-36 on
the following topics:







Language Domain Definitions
Proficiency Level Descriptors and Global Definitions
Progress Through the Proficiency Levels
Social and Academic Language Proficiency
Academic Language Proficiency Versus Academic Achievement
Observing Students in Each Language Domain
Collaboration with Others (GISD requires that you collaborate
with other teachers and school personnel on your holistic
ratings.)
42
Activity 5: Rate Your Students (p. 37)
You will need the following:



TELPAS Rater Manual, which includes the PLDs
The GISD Version of the TELPAS Rater Roster (Do not
use the one in Appendix B. The GISD version has a
column to indicate the initials of the collaborator for each
student.)
The TELPAS Writing Collection Cover Sheet (p. 121122) run front/back for Gr. 2-12 students
Keep student writing collections & student rating
rosters in locked storage when not in use.
The primary rating window in GISD is March 7
through April 1.
43
Procedures for Rating Students (p. 38)











Use the PLDs on p. 39-43 to rate your students.
Always refer to the PLDs
You may copy the PLDs and make notes on individual students if helpful.
Start with students whose English proficiency levels are clearest to you.
Ratings should be based on a variety of observations.
Evaluate the most current level of English proficiency & consider at which
level the student performs most consistently.
Collaborate with others.
Complete all ratings for one student before proceeding to the next student.
Compare each student to the PLDs and do not let proficiency levels of
other ELLs influence your ratings.
Except for Gr. 2-12 writing, your determination is based on reflections of
observations, not formal language observation tasks.
Remember that academic language proficiency is not the same as
academic achievement. A student does not have to be a high achiever in
order to have an advanced high level of English language proficiency.
44
Activity 6 & 7 (p. 44-48)



You will not be required to create and manage rating
entry groups, so disregard Activity 6.
You will not be required to enter rating information
online, so disregard Activity 7.
Your Campus Testing Coordinator or the District Testing
Coordinator will complete these tasks.
45
Activity 8: Prepare Materials for Return
(p. 49)




Verify holistic ratings and rater information you supplied
on the Georgetown ISD TELPAS Student Rating
Roster are accurate and complete.
Review the accuracy of any other student data you have
been asked to verify.
Make sure you have completed, signed, and dated the
GISD TELPAS Student Rating Roster and obtained
the initials of a Collaborator for each student.
Gr. 2-12: Make sure that you have completed the
TELPAS Writing Collection Cover Sheet (front &
back) and stapled it to the front of each writing
collection.
46
Activity 8: Prepare Materials for Return
(p. 49)




Be sure that you have signed the last box of the
TELPAS Rater Oath and returned it to your CTC
Return your GISD TELPAS Student Rating Roster
Turn in your student writing collections
Return your TELPAS Manual after all TELPAS activities
have been completed.
47
TELPAS Calendar of Events in GISD: Let’s Review
Events
Dates
Administration procedures training for holistic component
By February 1
New raters complete online basic course (K-1 or Gr. 2-12)
By February 4
Returning raters review TELPAS basics (see Slide 26 for
resources)
Before attempting calibration
activities
All raters complete online calibration activities
January 31 – Feb. 11
Collect samples for Gr. 2-12 writing collections
February 1 – March 25
Training for Gr. 2-12 TELPAS Online Reading Test
By March 4
TELPAS Assessment Window (holistically rating all ELLs &
completing online reading test for Gr. 2-12)
March 7 – April 1
Campus Verifiers ensure that Gr. 2-12 writing collections meet all
assembly criteria
March 25 - 31
Extended Collection Window (if needed to collect more writing
samples)
March 28 – April 4
Extended Assessment Window (to rate/test ELLs in extenuating
circumstances)
April 4 – 8
Deadline for CTCs to enter all holistic ratings in the Texas
Assessment Management System
April 8
48