2013-2014 Holistic Rating Training Requirements
Download
Report
Transcript 2013-2014 Holistic Rating Training Requirements
Assessment, Research & Evaluation
Department
2
This training does NOT take the place of
reading the appropriate manuals.
1.
General Information
2.
Test Security and Confidentiality
Requirements
3.
General Information About
Holistically Rated TELPAS
Assessments
4.
TELPAS Additional Roles
5.
Prepare to Rate
Holistic Rater Training
January 28th, 2014
3
4
New standards were set for TELPAS reading
Standards were adjusted to meet the new definition of
grade level-appropriate that accompanies the
increased rigor of STAAR
Domain weights have shifted in composite score
calculations.
Reading is now 50%
Writing is 30%
Listening and Speaking are each weighted 10%
5
Years in US Schools Data Collection Changes
Beginning with calculations made and reported in 2014,
students must be enrolled for 60 consecutive school days in a
school year for that year to count as a year in US schools.
More specific information regarding these
calculations is available on the TEA website
6
Grades K-1
Holistically rated
observational assessments
Listening
Speaking
Reading
Writing
Grades 2-12
Multiple choice online tests in
grades 2-12
Reading
Holistically rated
Writing collections
Holistically rated observational
assessments
Listening
Speaking
7
All ELL’s in grades Kinder – 12th are required to participate
All ELL’s in grades Kinder – 12th whose parents have declined
bilingual/ESL program services
In rare circumstances, it may be necessary for the ARD
committee in conjunction with the LPAC to determine that
an ELL receiving special education services should not be
assessed in reading/writing/listening/speaking for reasons
associate with the student’s particular disability.
Participation must be considered on a domain-by-domain basis
and reasons must be supported and documented in the IEP and
the PRC/SPF
8
An ELL from another school district, state, or country
who enrolls on or after the first day of the TELPAS testing
window will not be assed by the receiving district in the
HOLISTICALLY rated domains
Newly enrolled ELL’s in grades 2-12 are required to take
the TELPAS reading test; however, TELPAS coordinators
must verify with DTC that the student hasn’t already
completed testing elsewhere
9
10
Maintaining the confidentiality of the TELPAS program
involves protecting the contents of all online assessments
and student performance documentation. This requires
compliance with, but is not limited to, the following
guidelines
Personnel meet the requirements, have been trained and have
signed the appropriate oath to have access to the assessment
materials and information
Campuses implement the controls necessary to ensure proper
storage of secure materials throughout the test administration
Campuses place confidential documentation (rating rosters) in
a limited-access locked storage
11
Testing personnel undergo training and sign the oath
affirming they understand their obligations concerning
the security and confidentiality of the TELPAS program
Testing personnel administer the tests in strict
accordance with the instructions
Testing personnel do not view, reveal or discuss the
contents of an assessment before, during, or after a test
administration
12
Any person who violates, assists in the violation of, or solicits
another to violate or assist in the violation of test security or
confidentiality, as well as any person who fails to report such
violation is subject to the following penalties:
Suspension of a Texas
Placement of restrictions
educator certificate for a set
on the issuance, renewal
or holding of a Texas
educator certificate
Issuance of an inscribed
or non-inscribed
reprimand
term
Revocation or cancellation of
a Texas educator certificate
without opportunity for
reapplication for a set term or
permanently
13
Release or disclosure of confidential test content could result in
criminal prosecution under TEC §39.0303, Section 552.352 of the
Texas Government Code, and Section 37.10 of the Texas Penal
Code. Further, 19 TAC §249.15 stipulates that the State Board for
Educator Certification may take any of the above actions based
on satisfactory evidence that an educator has failed to
cooperate with TEA in an investigation.
Additionally, irregularities resulting in a breach of test security or
confidentiality may result in the invalidation of students’
assessments.
14
All personnel who participate in state –mandated testing or
handle secure test materials must meet the eligibility
requirements and sign a security oath
Any person who has more than one testing role (e.g. a Rater who
also serves as a Test Administrator) must receive appropriate
training and sign a security oath for EACH role
Each oath for TELPAS raters, writing collection verifiers and test
administrators must be read and completed AFTER the training
and BEFORE handling or viewing any secure materials or
confidential information
Oaths must be kept on file for at least five years
15
Serious
Procedural
Sharing answers to the online
Eligibility Error
calibration activities or practice
rating activities in the online
training courses
Falsifying TELPAS holistic ratings or
writing samples
Directly or indirectly assisting
students with responses
IEP Implementation Issues
Improper Accounting for Secure
Materials
Monitoring Error
Procedural Error
Tampering with student responses
16
Procedural irregularities (examples)
Eligibility error
Eligible students were not rated in one or
more domains
Eligible students were not administered
the reading test (2-12)
Ineligible students were assessed
IEP Implementation Issue
A SPED student was provided or not
provided an accommodation
Improper Account for Secure Materials
A rater, test administrator or CTC
misplaced a writing collection
Monitoring Error
Test Administrator left a room
unattended when online tests were
open
Students were not prevented from using
cell phones or electronic devices
Procedural Error
Test Administrator who was not properly
trained was allowed to rate students
A TELPAS writing collection was not
submitted according to the assembly
criteria
A test administration failed to provide a
student with the correct student
authorization for online testing
17
Each person participating in the Texas student assessment
program is responsible for reporting any violation or suspected
violation.
Campus staff should notify their CTC or DTC if they witness an
irregularity or suspect one has occurred
The DTC must contact TEA to report any incidents as soon as the
DTC is made aware of the situation
Contact the DTC if unsure about whether an irregularity has
occurred
18
19
Each teacher selected to rate an ELL must
Have the student in class at the time of
the spring assessment window
Raters may include:
Bilingual teachers,
Be knowledgeable about the student’s
ESL teachers,
ability to use English in instructional and
informal settings
General education teachers,
Hold valid education credentials such as
a teacher certificate or permit
Special education teachers,
Gifted and talented teachers and
Teachers of enrichment subjects
Be appropriately trained, as required by
TEA
20
A student’s TELPAS rater is the teacher designated by
the campus as the official rater of the student’s English
language proficiency.
The student’s rater must rate the student in all domains
for which the student is eligible. A student is not
permitted to have one rater for some domains and
another rater for other domains.
21
• TEA requires that raters who have not rated during the
last three years complete the BASIC training.
22
Fall/Spring ELPS-TELPAS Foundation Training
For teachers who will be trained as new TELPAS raters in the spring if they lack this foundation
Spring TELPAS Administration Procedures Training
As a key part of this training, information from the TELPAS Rater Manual is reviewed to prepare
raters to proceed with online holistic rating training
New Raters
Returning Raters
Online Basic Training Course
*Set
2 required only
if not successful on
Set 1
Online Calibration
(Sets 1 and 2*)
Online Calibration
(Sets 1 and 2*)
If not calibrated: Supplemental Holistic Rating Training
Final Online Calibration (Set 3)
23
As part of annual spring TELPAS administration
procedures training, holistic rating training requirements
are reviewed with raters, as well as information about
how to access the online training and calibration
components.
In addition, raters receive training on assessment
procedures such as how to assemble writing collections,
how to record students’ proficiency ratings, etc.
24
Online BASIC TRAINING Course – required for new raters
and raters who have not completed rater training within
the last three school years. Gives raters practice rating
students in each language domains. Separated in two
courses:
(approximately 4-5 hours)
Grades Kinder-1st course – covers the four language domains
Grades 2nd – 12th course – covers listening, speaking, and
writing
Raters who have completed K–1 training but
not 2–12 training will be new raters if they
switch to 2–12 training, and vice versa.
25
Online CALIBRATION Course – required for ALL raters.
Each calibration set contains 10 students to rate and
all holistically assessed language domains are
represented in each set.
(approximately 1 hour )
Grades Kinder-1st course – rate listening, speaking,
reading and writing
Grades 2nd – 12th course – rate in listening, speaking and
writing
26
Raters must know their assigned grade cluster to select
the appropriate online training.
Raters should consult with their campus testing
coordinator if they are unsure of their assigned cluster.
A rater needs to rate at least 70 percent of the students
correctly to demonstrate sufficient calibration.
27
Basic Training
Basic Training
Calibration Clusters
Kinder - 1
Calibration Clusters
Grade 2
Grades 3 – 5
Grades 6 – 8
Grades 9 – 12
Kinder - 1
2 – 12
28
Raters complete only as many sets as it takes to calibrate
Many raters will be able to calibrate on the first set of
activities and many others will be able to calibrate by the
end of the second set.
With supplemental support, very few raters should have
difficulty calibrating by the end of the third and final set
IMPORTANT NOTE
Individuals are not authorized by TEA to serve as raters unless they complete the
state-required training and calibration activities
29
Raters are required to train and calibrate in the
cluster in which they have the most ELLs.
As a best practice, these raters should also
review the online basic training course practice
activities in the additional cluster(s).
Example: A grade 2–5 ESL teacher has most of her ELLs in grade
3. She must complete training and calibration in grades 3–5. As a
best practice, she should review the online practice activities for
grade 2 to check her readiness to apply the rubrics appropriately.
30
Because of differences in the instructional content and rating
rubrics, these raters must complete training and calibration for
K–1 and at least one other cluster in 2–12.
Raters with more than one additional cluster should train in the
cluster in which they have the most ELLs.
As a best practice, these raters should also review online basic
training course practice activities in the additional cluster(s).
Example: A grade 1–3 ESL teacher has most of his ELLs in grade 2. He
must complete training and calibration in grades K–1 and 2. As a best
practice, he should review the online practice activities for grades 3–5 to
check his readiness to apply the rubrics appropriately for his 3rd grade
students.
31
New raters must complete the online basic training
course BEFORE beginning calibration.
All raters have the option to review the online basic
training course (which includes practice rating activities) before
beginning calibration.
32
Yes, raters should use their rating rubrics (PLDs)
and refer to, as needed, information from the:
online basic training course
holistic rating PowerPoints produced by TEA
TELPAS Rater Manual
Educator Guide to TELPAS
33
Depending on campus arrangements, raters may either
complete training and calibration during school hours,
after school, or on weekends.
The online training system allows raters to access the
training from any computer that meets the minimum
system requirements.
Campuses inform raters of the details during annual
administration procedures training.
34
Raters affirm online that they will keep the contents
of the calibration sets secure and confidential.
Calibration activities are taken from a bank and
randomized. Trainees will rate different sets of
students.
Raters can work at their own pace, go back and
review students, and change ratings as they work.
35
Raters can exit and return later to finish. They click a
“submit” button when they are finished with a set.
After completing a calibration set, raters immediately
see results. Results show both the rating assigned by the
rater as well as the correct rating.
Raters see annotations explaining the correct ratings.
Raters should use the annotations to go back and
review any incorrectly rated students.
36
Raters who take the basic training course get a
certificate from the online Texas TrainingCenter
after completing the course components.
Raters will receive a certificate of successful calibration
when they calibrate.
37
• TELPAS Returning Rater Planning Roster
–
Uses records starting from the 2008–2009 school years to generate a list of previously trained
raters (Basic Training)
• TELPAS Confidential Course Completion Roster
–
Lists online basic training course completion and in-progress status and performance by user
for the current year
• TELPAS At-a-Glance Training & Calibration Report
–
Provides a comprehensive list showing basic training course and calibration completions for
all registered users for the current year (includes time spent in each calibration set)
• TELPAS Confidential Calibration Summary Report
–
Provides calibration summary information by grade cluster (K–1, 2, 3–5, 6–8, 9–12) and for
grade clusters combined
https://texas.pearson.desire2learn.com/
38
• Personnel with administrative authorization are able to access reports to
monitor TELPAS online training and calibration.
• The reports are updated nightly.
• In order for users to appear in the correct reports, their location
(region, district, campus) must be up to date in the My Info section
of the Texas Training Center.
• Users who have registered in the Training Center and have started
but not completed a calibration set are not included in the
summary reports. Only completed sets are presented in the reports.
• However, users who have started a training course will appear in the course
roster as being “in progress.”
39
Coordinators and assistants with administrative access can
modify the last name of a user
send a user password reset email
modify the email address of a user
retrieve an email address associated with a user’s
TrainingCenter account
retrieve a username associated with a user’s
TrainingCenter account
see account status (active/deactivated) for all users.
40
41
Supplemental Support Providers
Writing Collection Verifiers
Raters not successful after sets 1 and 2 must receive
supplemental training
The rater will meet with a campus-appointed
supplemental support provider (SSP)
CTC’s will notify the DTC who the SSP is by the designated
district deadline
After the rater has received supplemental training, raters
will be given an access code to proceed with the third
and final calibration set
42
A Supplemental Support Provider must have:
been a 2012-2013 fully trained rater in the applicable K-1 or 2-12
span
completed the recorded Web-based training, and
calibrated successfully on either set 1 or set 2 in 2014
43
Training is delivered through a Web-based recorded presentation that is
accessible from the coordinator resources section of the Training Center
Separate training for K–1, 2–5, and 6–12
Training will be available beginning January 27
Both new and returning supplemental support providers need to complete this
training
Campuses are encouraged to train at least one supplemental support
provider for K–1, 2–5, and 6–12 to cover the needs of the school
Reviewing the number of raters unable to calibrate by end of calibration set 2
last year may help anticipate this year’s needs
44
An online training course on writing collection assembly and verification is
available on the Texas TrainingCenter website
Each writing collection must contain at least five writing samples and must
include one narrative writing about a past event and at least two academic
writing samples from mathematics, science or social studies
Writing assigned on or after February 17, 2014 may be considered for the
writing collection (writing assigned before February 17 is not eligible for inclusion)
Writing samples may continue to be gathered until March 26th, the date raters
are required to turn in students’ rating to the campus coordinator
All writing should reflect a student’s current proficiency level
45
Assembling Grades Writing Collections
• Type 1: Basic descriptive writing on a personal/familiar topic
• Type 2: Writing about a familiar process
• Type 3: Narrative writing about a past event
• Type 4: Personal narratives and reflective pieces
• Type 5: Expository and other extended writing on a topic from
language arts
What Not to Include in a Collection (p. 23)
Verifying the Writing Collection Components (p. 24)
46
• Ensuring raters understand the importance of being
properly trained on the holistic rating process
• Ensuring new and returning raters understand which
training to take and for which grade clusters
• Monitoring that all raters complete their training
requirements and receive supplemental training
support if needed
47
48
Holistic Rating Examples
The ability to understand spoken language, comprehend and
extract information, and follow social and instructional discourse
through which information is provided
EXAMPLES:
Reacting to oral presentation
Responding to text read aloud
Following directions
Cooperative group work
49
The ability to use spoken language appropriately and effectively in
learning activities and social interactions
EXAMPLES:
Informal interactions with peers
Oral presentations
One-on-one interviews
Classroom discussions
50
The ability to comprehend and interpret written text at the grade
appropriate level
EXAMPLES:
Paired reading
Reading response journals
Sing-alongs, read-alongs
Independent reading
51
The ability to produce written text with content and format to fulfill
grade appropriate classroom assignments
EXAMPLES:
Journal writing for personal reflections
Making lists for specific purposes
Labeling pictures, objects, and items from projects
Language experience dictation
52
The ability to produce written text with content and format to fulfill
grade appropriate classroom assignments
EXAMPLES:
Descriptive writing on a familiar topic
Narrative writing about a past event
Procedural writing from science, math and social studies
Expository and other extended writing from language arts classes
53
54
As in the past, tutorials are separate from tests
Recommended for all students taking TELPAS
Several different test item formats
Since no sample items with test, practice with item
formats and online interface is useful
Tutorials available at
http://www.TexasAssessment.com/TELPAS-tutorials
55
56
TELPAS Training – Online Testing
Tuesday, February 25, 2014
8:30 – 11:30
CAB Cafeteria
*Bring final YEARS IN US SCHOOLS rosters
57