Transcript Feb DAC Meeting - Kentucky Department of Education

2014 February DAC
Meetings
1
KDE:OAA:DSR: 12/16/2013
Agenda
 Hot Topics
 Test Security/Data Quality
 Accountability
 SDRR
 K-PREP
 Text-Reader Accommodation
 Alternate K-PREP
2
 College/Career
Readiness




ACT
Compass
KYOTE
Career Readiness
 A Quick Look to the
Future
 RFI
 Fall 2014 Reporting
 CIITS
KDE:OAA:DSR: 12/16/2013
Hot Topics
1. Increased emphasis on “getting the rosters
right” to support data quality. Data returned
from vendors should match roster information.
2. At the end of data review, students remaining
on the roster, without an approved exemption
or a score from the vendor, will have a zero
score applied in reporting.
3
3. For End of Course, KDE will add two review
steps before loading data into SDRR rosters.
7/17/2015
Hot Topics
4. Administration Code and Inclusion of Special
Populations regulations are under revision
for next school year. The 2009 versions
are still effective for spring 2014.
5. Time chart for K-PREP grades 3-8 is revised
slightly. In 2014, no field testing in
mathematics has reduced testing time.
DAC/BAC manual has complete time chart.
Each TAM has appropriate grade-based
chart.
4
7/17/2015
Hot Topics
6. Text-reader is a new accommodation
option for read aloud/reader for K-PREP
(Pearson) in grades 3-8.
7. Schools will only return scorable
student response booklets to Pearson
this year. No void or blank· student
response booklets.
5
7/17/2015
Hot Topics
8. Alternate K-PREP will offer a retest
option for the high school Transition
Attainment Record (TAR) reported with
College/Career Readiness.
9. KDE is providing guidance on applying
· for
G-Codes in Infinite Campus
Alternate K-PREP students.
6
7/17/2015
Hot Topics
10. ACT will return online End-of-Course
scores from paper/pencil testing faster
this year (5 days from receipt of
answer documents at ACT).
11. ACT will provide a managed system for
EOC Computer-Based Tests to control
the number of concurrent test-takers.
7
7/17/2015
Hot Topics
12. Dates regarding K-PREP (Pearson) are required.
Encourage updates in district calendars in Infinite Campus.
Shipping Windows to Pearson
Testing Windows to KDE
Option #1: 4/15 – 5/2
 IC pull 2/17. Materials by 3/31.
 5 consecutive days in the last 14
instructional days
Option #2: 5/5 – 5/16
 IC pull 3/3. Materials by 4/14.
8
Option #3: 5/19 – 6/2
 IC pull 3/17. Materials by 4/28.
Entered in PearsonAccess.
 May vary by level (E, M, H) or be a
single window for all
 Used to verify accountability dates
(1st day of 5 day window) and site
visits
 Due to Maranda Perkins (March 1)
Test Security and Data Quality
Key assessment themes require the focus of
schools, districts and the state
9
7/17/2015
Key Assessment Themes
TEST SECURITY—checks and
processes to monitor appropriate
administration of state-required tests
DATA QUALITY—collaborative process
between OAA and school/district staff
to ensure accurate reporting of test
results
10
7/17/2015
Test Security
 Continues to be a critical issue in
Kentucky and nationally.
Loss of certification for Kentucky educators
emphasizes the seriousness of the issue.
 OAA implements and monitors the
recommendations from Kentucky’s
security audit.
11
Increased analysis of test results for irregularities and
triggers for follow up from KDE’s allegations
coordinator.
7/17/2015
Test Security: Local Requirements
Training
 Administration Code and Inclusion of Special
Populations (2009 versions)
Signatures
 Student Honor Code
 Appropriate Practices Form and Administration Code
 DAC/BAC testing certification
Documentation of Testing
 Seating charts (all required tests)
 Testing room location codes (K-PREP Pearson)
12
7/17/2015
Test Security: Training
Requires annual training on regulations
Administration Code
703 KAR 5:080
Inclusion of Special Populations
703 KAR 5:070
All persons involved with testing in
 Inclusion Regulation
any way
Anyone assisting with testing
accommodations
 Administration Code

Requires signature to confirm
individual has read, understands
and will abide by document
Documentation of training required (who, what, when, where and how)
13
KDE:OAA:DSR: 12/16/2013
Test Security: Signatures
Student Honor Code
14
Appropriate Practices Form
7/17/2015
Test Security: Signatures
Certification Form
DACs and BACs
E-mail from KDE/OAA
Available on KDE website
Completed at end of year
to reflect all required testing
in the school year
DACs collect from BACs
KDE collects from DACs
15
7/17/2015
Test Security:
Documentation of
Testing
Seating Charts are required
for
• all state-required test
administrations.
• optional tests that are
included in accountability
if the test is administered
by the school.
16
Seating Chart—
Large group
KDE:OAA:DSR: 12/16/2013
Test Security:
Documentation of
Testing
• DAC will collect and
maintain in case of
potential
investigations.
• Templates are
provided by KDE and
may be modified at
the local level.
17
Seating Chart—
Chart—Small group &
individual students
KDE:OAA:DSR: 12/16/2013
Test Security:
Documentation of
Testing
• Collection of
testing room
location codes on
each Student
Response Booklet
allows security
data analysis at
the room level.
18
KDE:OAA:DSR: 12/16/2013
Test Security: State Monitoring
 Test window visits
 Review of seating charts
 Collection of DAC certification forms
 Review/audit of randomly collected AAA Folders
for alternate assessment students
 Analysis of outlier test results by Office of
Education Accountability
 Analysis by vendors (i.e., erasure and others)
Pearson has hired Caveon to performance data analysis
19
7/17/2015
Test Security: State Monitoring
Analysis by vendor can produce multiple types of
information/reports at room, school or district levels.
 Erasure analysis (Right (R) to Wrong (W), RTR, WTR)
 Large score changes for a student
 Large mean score increases
 Probability of matched response analysis
 Previous year’s distribution around scores
 Pattern analysis
 Comparison of individual student scores by item type
20
Analysis can trigger/identify schools and districts for
follow up with Allegations Coordinator.
7/17/2015
Test Security
Reporting of Allegations
Get the facts straight.
 Record the date and time of the incident.
 List all involved (i.e., staff, volunteers, students).
 Identify content area and test parts impacted.
 Identify specific content question(s), if applicable.
 Record what you learn.
 Report the incident.
21
7/17/2015
Test Security
Reporting of Allegations
Support Documents
 Testing Allegation Reporting Form
 Notarized Statements written and signed by each
individual involved (teacher, proctor, BAC, principal
and/or others). One statement signed by multiple
individuals is not acceptable.
 Testing Schedule
 Seating Chart and Room Number
22
7/17/2015
Test Security
Reporting of Allegations
Support Documents
 Verification of Administration Code Training
 Verify/Proctor Signature Sheet
 Nondisclosure Agreement (classified/volunteers)
 Student Name & SSID
 For an allegation involving special education:
 verification of student test accommodation(s).
 verification of Inclusion of Special Populations training.
 a signed copy of the Code of Ethics, record of testing room and test
administrator for allegations regarding alternate assessment.
23
7/17/2015
Key Assessment Themes
TEST SECURITY—checks and
processes to monitor appropriate
administration of state-required tests
DATA QUALITY—collaborative process
between OAA and school/district staff
to ensure accurate reporting of test
results
24
7/17/2015
Data Quality
• The authoritative source of student data
in Kentucky is the state student data
system, Infinite Campus (IC).
• Data are downloaded from IC for
vendors and to populate internal
applications (i.e., SDRR).
• Accountability change requests made to
OAA are cross-checked with Infinite
Campus.
25
7/17/2015
Data Quality
Keys to Improved Data Matching
Correct entry of student
information in IC is critical (name,
date of birth and SSID).
Consistently use the student’s
name as it appears in IC on all
testing records.
State-testing programs are
provided SSIDs.
26
7/17/2015
Data Quality
KDE/OAA Student Data Review and Rosters
(SDRR) Application, organized by test, has
two critical data accuracy functions:
1.
Rosters
 Required by regulation
 Testing information for some tests
 Accountability record for all grades 3-12
(Grade 9 is needed for ACT Plan and grade 12 is needed
for College/Career Readiness reporting.)
2.
27
Data Review (cleanup before reporting)
7/17/2015
Data Quality
Get the Rosters Right
=
INCLUDE
on Roster
REMOVE
from Roster
28
Accurate
Accountability
Reporting
Tested students with scorable
documents returned to vendor
Not tested students that have
withdrawn from the school
Not tested students:
1) With non-participation exemption
(Extraordinary or Medical)
2) Who should have tested,
but did not
For End of Course:
Not tested students that are
still completing coursework
7/17/2015
Data Quality
Get the Rosters Right
=
Accurate
Accountability
Reporting
Students left on roster, after
data review, without an
approved exemption or test
score, will be included in
reporting as a zero (0) score.
29
7/17/2015
Data Quality
Get the Rosters Right
=
Accurate
Accountability
Reporting
For End of Course, OAA will add twophase review before loading rosters to
SDRR. (spreadsheet process)
1.
2.
30
Show enrollment counts by state course
code so districts have an opportunity to
fix their course codes in IC. (mid/late
February)
Collect which EOC courses by school to
load to SDRR for rosters. (late March)
7/17/2015
KDE State Course Codes
English II
230110-English 2
230162-Pre-IB English 2
Algebra II
270311-Algebra 2
U.S. History
450809-U.S. History Survey
450812-United States History,
Reconstruction to the Present
450814- AP American History
31
Biology
302601-Life Science/Biology 1
302602-Biology II
302611-Biochemistry
302613-Microbiology
302614-Ecology
302623-IB Environmental Systems
302643-Pre-IB Biology
302644-IB Biology 2
302645-IB Biology 3
302646-AP Biology
302680-AgriBiology
302685-Agriscience
All Course Codes
Associated with
EOC Tests
District/school
Review Process
302698-Intro Biology with Earth/Space Science
303010-Astrobiology
303091-Integrated Science I
303092-Integrated Science II
303093-Integrated Science III
302603 - Honors Biology (newly added)
302618 - Medical Science (newly added)
Course used for
required credit and
EOC tests loaded
into SDRR
Data Quality
Get the Rosters Right
Accurate
Accountability
=
Reporting
1. Show enrollment counts by state course code
32
• Includes all course codes
for 4 areas associated with
required EOC tests.
• Provides district opportunity
to fix course codes in IC.
• Is a course missing?
• Does a course incorrectly
have no students?
7/17/2015
Data Quality
Get the Rosters Right
Accurate
Accountability
=
Reporting
2. Collect course codes to load into SDRR. (late March)
New
Column in
Phase 2
Load
course
code to
SDRR
33
Yes or
Blank
• District must submit a
single spreadsheet
with new column (Yes).
• No action by district will
default to load all
courses in SDRR.
• In SDRR, add or delete
students to correct.
7/17/2015
Data Quality
Importance of Rosters
Clean Rosters in Spring
*Longer roster window means more time for work.
*Flexible routine changes include student add or delete.
*Demographic changes made directly in Infinite Campus.
*Better availability of school staff to assist.
Easier Data Review and Cleanup in August
*Data review changes require documentation and
KDE approval.
*Data review goal is small number of changes—”ones or twos”.
Accurate Data for Public Release
34
7/17/2015
Accountability Rules
How to determine whether students are in a
school’s accountability report
35
7/17/2015
Accountability Rules
Tested
Every student enrolled on the first day of the 5-day spring
summative test window must be tested for participation
rate purposes.
Accountable
Accountability determined by full academic year (any 100
instructional days of enrollment). The first day of the 5day spring summative test window is included in the 100
day count.
36
7/17/2015
Accountability Rules
The first day of K-PREP five-day spring summative window (in the
last 14 instructional days of the district calendar) is the point in
time where the 100 days of enrollment (full academic year) is
determined.
April-June
Across
Year
K-PREP
EOC
Grade
12 CCR
37
K-PREP (Pearson)
1st day of the 5-day
Summative Test
Window
March
ACT
Nov-Dec
April-May
Alternate
K-PREP
September
EXPLORE
PLAN
7/17/2015
Accountability Rules
38
7/17/2015
Accountability Terms for Schools
 “A1” means a school under administrative
control of a principal and eligible to establish
a school-based decision-making council.
 “A1” school is not an alternative program
operated by, or as a part of, another school.
 “Alternative” program means all other
education programs not classified as A1.
39
7/17/2015
Flowchart for Determining Accountability For Students in A1 Schools
Kentucky Performance Rating for Educational Progress (K-PREP)
START
Has this
student been
enrolled in
this school for 100
instructional days
this school
year?
YES
Student scores
will count
in this school’s
accountability.
40
Every student enrolled on the first day of the five-day spring summative test window is tested
for participation rate purposes. Accountability is determined by full academic year (any 100
days of enrollment) including the first day of the five-day window.
NO
NO
Has this
student been
enrolled in
another school for 100
instructional
days this school
year?
YES
NO
Has this
student not been
in any one school but
has been enrolled in
a district for 100
instructional days
this school
year?
Student scores
will count
in state’s
accountability
only.
YES
Student scores
will count
in other school’s
accountability.
Student scores
will count
in district’s
accountability.
7/17/2015
Flowchart for Determining Accountability For Students in Alternative Programs
Kentucky Performance Rating for Educational Progress (K-PREP)
START: Was the student who is in
NO
Was student placed by a
governmental
agency or the
court system? NO
YES
Accountable to the state
41
an alternative program enrolled in an A1
school in a district for 100 days this
school year ? (Note: The 100 day count
includes the first day of the five-day spring
summative test window.)
Was the
student enrolled
100 days
in the A1(or multiple A1s in a district)
and alternative
program(s) combined or
in the alternative
NO
program(s) only?
YES
YES
Accountable to A1 school
Accountable to A1 school
where the student would
have attended if not
enrolled in the alternative
program(s) or to district
(if student would have
attended multiple A1
schools).
Note: A student at Kentucky School for the Blind, Kentucky School for the Deaf
or Gatton Academy tracks back to the A1 school that he or she would have
attended if not in the alternative program.
ALTERNATIVE Program
Scenarios
Applying the Accountability Rules
42
7/17/2015
SDRR
Student Data Review and Rosters Application
43
KDE:OAA:DSR: 2/6/2014
Work in SDRR
2014 Spring Rosters
 ACCESS – (EL)
 The ACT – Grade 11
 K-PREP – Grades (3-12)
 Pearson
 QualityCore- End-of-Course
(EOC)
Training opportunities for
SDRR Rosters and Data
Reviews will be announced in a
DAC Monday E-mail.
44
2014 Data Review
ACT Explore (Feb.)
ACT Plan (Feb.)
The ACT
Alternate K-PREP
K-PREP End-of-Course
(QualityCore)
 K-PREP (3-8, 10,11)
 College/Career Ready Data
 ACT Compass
 The ACT (includes National Tests)
 KYOTE
 ACT WorkKeys
 KOSSA
TEDS
 ASVAB
 Industry Cert.





KDE:OAA:DSR: 2/6/2014
Rosters
45
KDE:OAA:DSR: 2/6/2014
Accountability Rule – 100 Days
Test Name
EXPLORE
Grade(s)
8
100 Days as of …
Spring 2013
PLAN
10
The ACT
11
Spring 2014 K-PREP Window
3-8, 10,11
3-12
High School
Spring 2014 K-PREP Window
Spring 2014 K-PREP Window
Spring 2014 K-PREP Window
K-PREP
Alternate K-PREP
END-OF-COURSE
46
Algebra II
Biology
English ll
U.S. History
Spring 2013
100 days as of the first
day of your 5-day
testing window
KDE:OAA:DSR: 2/6/2014
New Features in SDRR
 Connection of SDRR to IC
 Home Page: Information and Student Changes,
List of Tasks
 No Separate Ticket Listing (Now called Change
Requests)
 Moved From Icons To Text Features
 “Locked” Label-Submitting Accountability and
Non-Participation Changes
 Creation of a Sandbox for training
 Edit Change Request (in development)
47
KDE:OAA:DSR: 2/6/2014
Closes
Rosters Opens
in SDRR
Assessment
March 27 (New
Change
Requests)
April 3 (Final)
The ACT
Feb. 24
K-PREP Pearson
April 21 June 13 (New
Change
Requests)
June 20 (Final)
K-PREP EOC
(English ll, Algebra
ll, U.S. History,
Biology)
Alternate K-PREP
College and Career
Readiness
9th Grade (For Fall
482014 Plan)
Demographic
Changes are
made in IC
Fields/Comments
Non-participation,
Accommodations,
Accountability(done
with Spring Rosters)
Accommodations used
on test collected (IEP,
EL/LEP) with a yes/no
option,
Non-participation,
Accountability and
Annotations
Accountability
KDE:OAA:DSR: 2/6/2014
Fields to Change in…
Infinite Campus
SDRR
 Name
 Annotations
 Grade
 SSID
 Accountability
 DOB
 Non-participation
 Gender
 Race/Ethnicity
 Lunch
 IEP
 EL/LEP
49
 Accommodations
If IC change is not reflected in SDRR
within 24 hours, contact
[email protected]
KDE:OAA:DSR: 2/6/2014
Infinite Campus
Questions regarding fields coming from IC
must be submitted to the Enterprise Data
– KSIS Team at
[email protected]
OAA will be unable to assist with IC questions.
50
KDE:OAA:DSR: 2/6/2014
IEP Field (Helpful Hints)
Correcting the IEP field in Infinite Campus
1. Make certain the student has a current
Individualized Educational Program (IEP)
2. Make certain that the IEP is locked
If IC change is not reflected in SDRR within 24 hours, contact
[email protected]
51
KDE:OAA:DSR: 2/6/2014
EL/LEP Field (Helpful Hints)
Correcting the EL/LEP field in Infinite Campus:
1. Mark the student as LEP
2. Home Language other than English:0400
3. Mark at least one instructional accommodation
4. Mark at least one program services type
(cannot be declined services)
If IC change is not reflected in SDRR within 24 hours, contact
[email protected]
52
KDE:OAA:DSR: 2/6/2014
K-PREP Rosters
Some data is
pre-populated from
Infinite Campus
 Verify student enrollment in grades 3-12
 Include all students enrolled as of the first day of the testing
window
 Change requests created for:
 Non-Participation Students – First Year EL/LEP, Medical,
Extraordinary Circumstances, Foreign Exchange,
Withdrew/Expelled During the Testing Window
 Accountability
 Save an electronic version of all rosters
53
KDE:OAA:DSR: 2/6/2014
ACT Explore/Plan Data
Review Process
54
KDE:OAA:DSR: 2/6/2014
Prioritizing Your Data Review
Schools and districts may review all available student information.
OAA encourages focus on these priorities.
 Step 1
Accountability (Priority One)
 Step 2
Demographics (Priority Two)
• Confirm your 100 day students that will contribute to
your district or school accountability.
• Change any non-100 day students that should be
removed from your accountability to the correct
school/district/state accountability.
• Check free-reduced lunch and students with disability
status (IEP), since these have potential to change over
time.
55 • Confirm ethnicity and race categories.
KDE:OAA:DSR: 2/6/2014
Two Sides of Data Review
Student Listing
Data Review
Change Listing
Use to check
Use to
1. Accountability
2. Demographics
 Update information
about change
requests
 Close change requests
opened in error
56
 Check change request
status
KDE:OAA:DSR: 2/6/2014
ACT Explore/Plan Data Review
Change in SDRR












57
Name
State Student ID (SSID)
Date of Birth (DOB)
Gender
Race
Ethnicity
Non-participation
Accountability
IEP
EL/LEP
Lunch (Free/Reduced)
Accommodations
Send E-mail to KDE
 Tested Location
[email protected]
ACT Explore/Plan Data
Review opened in January
and will close in February
KDE:OAA:DSR: 2/6/2014