Transcript DAC Academy August 19, 2010 Colorado Department of Education Unit of Student Assessment.

DAC Academy
August 19, 2010
Colorado Department of Education
Unit of Student Assessment
DAC Academy 2010
Today’s Agenda
• Assessment 101
• Essential Information for 2010-2011
• Lunch
• Standard Accommodations
• Break-out Sessions
• Revised Standards and Assessment System
• School Performance Frameworks
• Non-Standard Accommodations
DAC Academy 2010
DAC Roles & Responsibilities
In regards to the administration of the state
assessments, DACs ensure:
• Quality
• Accuracy
• Security
• Documentation
• Communication
• Training
DAC Roles & Responsibilities
• What is a DAC?
–
–
–
–
District Assessment Coordinator
Appointed by the district superintendent
Responsibilities and powers may vary
Need to have a specific and specialized skill
set/level of expertise
• Educational or psychometric background
DAC Roles & Responsibilities
•
•
Training/Communication
–
Train SACs and Test Proctors
(documentation)
–
Consistent and reliable distribution of
critical information to superintendents,
teachers, and core assessment team
members
Manage Materials
–
Maintain security of all materials
–
Ensure accurate score reports
•
Pre-coded label use and student data grid
DAC Roles & Responsibilities
• Oversee Administration
– Scheduling
– Ethical practices
– Misadministrations
– Distribution, collection, packing and shipping
– Accommodations are provided and documented
– Document the chain of custody
– Keep the communication lines open
DAC Roles & Responsibilities
• Failure to implement the responsibilities of
the DAC role will probably result in some
catastrophic disaster in your district.
• Whenever you are gone, you need to have
a backup.
DAC Roles & Responsibilities
You Have a Vital Leadership Role!
• Official point of contact
- One DAC (One official district email
address)
- ONLY district staff member officially
designated to communicate with the school
district, USA/CDE, and the assessment
vendor CTB
The first question we ask those who call us:
“Are you the DAC?”
DAC EMAILS
• The DAC must forward the important
information to the appropriate people in
the district.
• When responding to a DAC email
– DO NOT click Reply
– Send an email directly to the contact person
• Forward the message to the contact person OR
click on the contact person’s email address
DAC Academy 2010
Colorado Student Assessment System
Unit of Student
Assessment
Achievement
(Colorado Model
Content Standards)
College Entrance National and
State Trends
Colorado Student
Assessment Program
Colorado ACT
National Assessment of
Educational Progress
CSAP
COACT
NAEP
English
Language Acquisition
(Colorado ELD Standards)
Colorado English Language
Acquisition Program
CELApro
CELAplace
CSAPA
LC&E
On December 18, 2006 Colorado received
FULL Approval from the USDoE for our
Standards and Assessment System
DAC Academy 2010
Unit of Student Assessment
USA
Assessment Purposes
Program Evaluation
CSAP
CSAPA
Achievement
Tests
Body of Evidence
(Program / Placement)
CELAplace & CELApro
Language
Acquisition
Test
DAC Academy 2010
Unit of Student Assessment
Management, Administration
and Data Reporting
Where the data go
Colorado
Longitudinal
Growth
Model
Accreditation
AYP
(Title I)
AMAOs
(Title III)
SchoolView
DAC Academy 2010
Accountability Contacts
Where the data go
Longitudinal Growth
Bill Bonk
Accreditation
John Condie
John Penn
AYP
AMAOs
AlyssaPearson
Donna Morganstern
Morgan Cox
Genevieve Hale
SchoolView
Bill Bonk
DAC Academy 2010
Colorado Student Assessment Program
CSAP
PURPOSE: To measure student achievement relative to the
Colorado Model Content Standards.
• The assessment has been in place since 1997 (pre-NCLB).
• 98 percent participation rate across grades 3-10
• 31 tests
• Science (Grades 5, 8, & 10)
• Reading(Grades 3-10)
• Writing (Grades 3-10)
• Mathematics (Grades 3-10)
• Lectura/Escritura (Grades 3 & 4)
DAC Academy 2010
Colorado Student Assessment Program
CSAP
2011 Early Testing
Windows
Grade 3
Reading/
Lectura
Grade 3–10 all
other tests
Feb 7 – 18
Feb 28 – Apr 1
Those approved for the early
testing window have an equal,
just earlier, testing window.
2011 Regular Testing
Windows
Grade 3
Reading/Lectura
Feb 22 –Mar 4
Grade 3–10 all
other tests
Mar 14 - Apr 15
DAC Academy 2010
Colorado Student Assessment Program Alternate
CSAPA
PURPOSE: Standards based achievement test for students with
a significant cognitive disability.
• Eligibility
 Determined through a student’s IEP team.
 Less than 1 percent of all students.
• 27 tests (all CSAP grades and content areas)
• Mandatory Training prior to
testing
2011 Testing Windows
All grades and
content areas
Feb 2 – Mar 25
DAC Academy 2010
Colorado English Language Acquisition Assessment
CELApro
PURPOSE: English Language Acquisition
• NOT to be confused with an achievement test
• Measures four domains of language development

Speaking, Listening, Reading, Writing
 Academic Language & Social, Basic Interpersonal Communication
•
Given to all NEP and LEP K-12 students
 As defined per LCE Unit definitions
•
Mandatory Training prior to
testing
2011 Testing Windows
Grades K-12
All four domains
Jan 3 – Jan 31
DAC Academy 2010
Colorado ACT
College Entrance Exam
 Accepted by U.S. Colleges, Universities, Military Academies and
NCAA
Colorado ACT (COACT)




State test date: April 27, 2011
Make-up test date: May 11, 2011
Accommodations Testing Dates: April 27 – May 11, 2011
All 11th grade students by law
11th Grade Alternate for students eligible to take CSAPA
 Managed by Exceptional Student Leadership Unit
 2011 dates TBD
 Contact Dena Coggins, ESLU 303-866-6863
[email protected]
DAC Academy 2010
Colorado ACT
COACT
Test Supervisor (TS)
DAC’s may need
to develop a
communication
process with Test
Supervisors for
Accountability*.
COMMUNICATION
DAC
*Test Supervisors must develop a
communication plan with Back-up
Test Supervisors and Test
Accommodations Coordinators.
DAC Academy 2010
CO ACT UPDATES
ONLINE Schools
• New changes online schools: Two national
tests date options to online students ONLY:
– February, 12, 2011 & April 9, 2011
• Student can choose either OPTION
• Student can pay $20 to take April 9, 2011 if
they registered and miss Feb. 12, 2011 test
option
DAC Academy 2010
National Assessment of Educational Progress
NAEP
2011 State and National sample of grades 4, 8 and 12 (about 937K
students nationwide)
-CO has shortened NAEP window: Jan 31- Feb 25, 2011
Type
Grade
Subjects
Paper / Pencil
4
8
Reading, Math
Reading, Math, Science, NAEP
TIMMS
Computer based
8
12
Writing or Math
Writing
TIMSS onlynational and
state
8
Math, Science
TIMSS/ PIRLS
4
Math, Reading, Science
DAC Academy 2009
National Assessment of Educational Progress
NAEP
Participating schools selected by National
NAEP Statisticians
– Most schools identified last May- Over 300 selected: G4 =
122, G8 = 181, G12 = 8
– Most participating NAEP districts have received initial
notification from NAEP State Coordinator
– Will receive national and state results for grade 4 in
reading and math and grade 8 in reading, math and
science (by variables and background survey responses.)
Do not receive disaggregated results for districts or
individual schools- not designed for this.
DAC Academy 2010
NAEP-TIMSS, Trends in International Mathematics
& Science Study
TIMSS window: Apr 4 - May 27, 2011
Grade 8 only in TIMSS
Two groups: national TIMSS (10 schools) & state TIMSS
(50 schools)
CO one of only 8 states selected for state TIMSS
CO will get state results compared to over 60 nations
Participating schools will get a School Report
More info to come to districts/schools in these
assessments
DAC Academy 2010
National Assessment of Educational Progress
NAEP Roles
National NAEP
Office
NAEP State
Coordinator
NSC
School &
Community
Contracted
Assessment
Team
DAC Academy 2010
National Assessment of Educational Progress
NAEP- Relationship between NAEP & School
Community
NAEP State Coordinator
District Contact
NAEP School
Coordinator
School & Community
•Assists NSC in
School
communications
•Works with NSC
and the
Supervisor to
Oversee the
process
•Confirms the
assessment date
•Provides schools
with info for
parental
notification
•Responds to
questions
•Works with
district/school
personnel to
ensure a smooth
process
DAC Academy 2010
NAEP-The Nation’s Report Card® public web site
http://nces.ed.gov/nationsreportcard/
DAC Academy 2010
NAEP-The Questions Tool
http://nces.ed.gov/nationsreportcard/itmrls/
DAC Academy 2010
NAEP-Thousands of Released Items
Sample Item: Math Grade 8 MC, Medium Difficulty
What is the intersection of rays PQ and QP in the
figure above?
A) Segment PQ
B) Line PQ
C) Point P
D) Point Q
E) The empty set
DAC Academy 2010
DATA OPERATIONS
You Live and Die by Your Data!
If you miss any element, you are
doomed!
School List
(August 31st)
Student
October
Pre-coded Labels
OR
CSAP/CSAPA, COACT
(January)
Final Label Files
CELApro, CSAP/CSAPA, COACT
Online Enrollment
(Oct/Nov)
CSAP/CSAPA
CELApro
(CTB)
CO ACT
(ACT)
NAEP
Other Tests
ALL STATE TESTING
USDOE
CDE’s Data
Warehouse
SBD
SchoolView
AYP
AMAO
CEDAR
EDEN
Schools/Parents/Students
Growth/ Accreditation
Press Releases
DAC Academy 2010
Data Operations
5 Essentials for DACs
• Online Enrollments – on CTB Navigator this fall
– CELApro
– CSAPA and CSAP special populations
• ADE Respondents
– Pre-Coded Labels
– SBD: CELApro, CSAP/CSAPA, COACT
• Logistics Training in November online
• SBD Training in February online
• New for 2010-11: Race/Ethnicity codes
DAC Academy 2010
Assessment Data Life Cycle
CSAP Example
Reporting
August/July
District Validates
their own data prior
to final reports
May/June
Student
Biographical Data
Review and
NCount
After
Oct/Nov
Nov
Feb
DAC Training
DACs Train SACs
During
Pre-coded
Labels
Mar
Apr
Admin II
Admin I
3rd Rdg
3-10th
Rdg, Wrtg, Math
5-8-10 Science
DAC Academy 2010
CTB.com and Navigator
• Secured Site – for DACs
– Online training
– Online enrollments and
special populations
– Access oral scripts,
translated oral scripts,
teacher read directions for
CSAP
– Non-standard
Accommodations Forms
DAC
– Short/Add Forms
– Archived Emails
– Data: NCounts, GRT files,
official data retrieval
CDE
CTB
DAC Academy 2010
CTB Navigator
(CSAP, CSAPA, and
CELApro)
DAC Academy 2010
Core Assessment Team
Who is it?
• District Assessment Coordinator
• ELA Director/Coordinator
• SPED Director/Coordinator
• CBLA Contact
• ADE Respondent(s)
• Title I and Title III (AYP, AMAO)
• Others…
DAC Academy 2010
Core Assessment Team
Their Role:
• Are students receiving the services they need?
 Instructional
 Programmatic
 Assessment
 Monitoring
• Collaboration and Communication are key!
 This ensures alignment between instruction and
assessment.
 Required trainings
DAC Academy 2010
Core Assessment Team
Critical functions include, but are not limited to…
• Online Enrollments/Early Test Windows
– On CTB Navigator
• Automated Data Exchange Collections
– Through CDE (Student October, Student Biographical Data, etc.)
• Accommodations in place – THINK EARLY!
• Attendance at November trainings
– CSAP, CSAPA, CELApro (face to face)
– CSAP, CELApro, CSAPA logistics (online only)
 Online administration trainings for CSAP, CSAPA, and CELApro will still be
available.
DAC Academy 2010
Core Assessment Team
District Assessment Coordinator (DAC)
• Oversees and manages all aspects of state
assessment administration
– Ensures standardized ethical test
administrations
• Training
• Manage data collection processes
DAC Academy 2010
Core Assessment Team
ELA Director
• Appropriate accommodations
 Colorado Accommodations Manual for ELs
• Accommodations monitoring
• Assessing language proficiency
• Designation and re-designations of ELs (NEP, LEP and
FEP) for assessment purposes
• Meeting the needs of ELs with disabilities (dually
identified)
• Distinguishing between language difficulty and learning
disability
Language Culture and Equity Unit
Joanna Bruno (303) 866 - 6870
DAC Academy 2010
Core Assessment Team
Special Education Director
• Meeting the needs of students with disabilities
– Colorado Accommodations Manual
• Matching a specific need by including specific
accommodations in plans
– IEP (Includes eligibility for CSAPA)
– 504
– RtI
• Instructional Accommodations monitoring
• Work with EL team to meet needs of dually identified
students
Exceptional Student Leadership Unit
Dena Coggins (303) 866 - 6863
DAC Academy 2010
Core Assessment Team
CBLA Contact
•Body of evidence from assessments
• Programmatic interventions/ILPs
• Accommodations aligned
Literacy Grants & Initiatives
John Love (303)866-6982
DAC Academy 2010
Core Assessment Team
District Trainers
• Proctors, staff, faculty and administrators
must be trained through an interactive format
on a yearly basis!
– This is true for EVERYONE…
– If you are an educational leader, you must step up.
• How do you as the DAC know they have been
trained?
– Training Schedule
– Attendance List
– Copy of Materials
(November training will be discussed in future slides.)
DAC Academy 2010
Core Assessment Team
Expansion
• Districts are unique
• Other staff may need to be included in the
work…
– ADE respondents
– Superintendents/other District Leadership
– SASID/RITS respondents
– Lead Teachers, etc.
• Communication is key
DAC Academy 2010
Core Assessment Team
Automated Data Exchange (ADE) Respondents
• Who are they?
– Responsible for submitting data for the various
data collections throughout the year.
• These include:
– Student October (November)
– Precoded Labels (January)
– CELApro Student Biographical Data (March)
– CSAP/CSAPA Student Biographical Data (June)
– COACT Student Biographical Data (July)
DAC Academy 2010
Reporting Results
• Requirements
 CRS 22-7-409-1.9
o “The results of the assessments…shall be part of the
student’s permanent academic record.”
 CSAP labels on high school transcripts
Contact CTB for these
• Parents receive reports in a timely manner
• All schools in each district receive reports in
a timely manner
DAC Academy 2010
CDE/USA – Support and Service
• We recognize that you have “other duties as
assigned.”
• How we support districts:
– Resources
• Comprehensive manuals and guides
– Trainings
• November
• Face-to-face (January)
• Web ex (CTB Navigator)
-24/7 availability
– E-mail and Phone calls
DAC Academy 2010
CDE/USA – Support and Service
• Unit of Student Assessment e-mails to DACs
– Try to limit these
– E-mails archived on CTB Navigator
• Scoop
– Weekly newsletter sent out to districts
– Subscribe by contacting the CDE Communications Unit at:
[email protected]
– Includes important deadlines, announcements and
reminders
– Compiles multi-unit info from CDE for easy access to:
• Superintendents, DACs, etc. (anyone who subscribes)
DAC Academy 2010
Assessment Resources
• 2010-2011 Procedures Manual
• 2010-2011 Accommodations Manuals
• 2011 SAC/DAC Manuals
• Test Proctor’s/Test Examiner’s Manuals
• SBD Manual
• Technical Reports
• Data Interpretation Guidelines
• Guides to Test Interpretation
• Parent Brochures (CSAP, CSAPA, Accommodations)
Questions about Items
• CSAP items are developed and then reviewed by CTB
Content Editors and Content Leads
• Items are reviewed by the CDE CSAP Consultant
• Colorado Teachers and Educators review the items at
CVAR (CSAP Item Review)
• CTB and CDE have a Post-CVAR review
• CTB does their internal review of the items
• CDE does a final review of the items in the test book
• Colorado Teachers and Educators review how constructed
response items will be scored at Anchor Review
• All test items are the property of the state of Colorado
DAC Academy 2010
Questions about Items
• Proctors should not be studying items
– Items only accessed during test administration
• If a test proctor or examiner has a question about
an item…
– Contact the SAC, who will contact the DAC
– The DAC contacts the appropriate person at CDE in the
USA by phone
• We are glad to address important questions about the items
– Include in proctor training instructions:
• Do Not copy the item
• Do Not discuss the item (with other teachers, etc.)
• Do Not email the content of the item
DAC Academy 2010
Questions about Scores
• If school/district personnel or parents have a
question about a score…
– First, the DAC should review:
• GRT file
• Past results
• District assessment results
– Irresolvable questions are forwarded to CDE by the
DAC
• USA works with CTB to examine the data
• May possibly call you in to review material with us
– Parents cannot view assessments
DAC Academy 2010
How to Handle Advice from
Sources other than USA
• When you have questions
– Contact Unit of Student Assessment
• Record the name of the staff member you speak with
– We cannot verify the accuracy of information
given by other districts or outside sources
– Know your lifelines
DAC Academy 2010
BREAK
(15 minutes)
Quotes to Inspire You
• The kids are veteran test-takers and
experts in what it means to be a
standardized assessment.
• Train early and train often.
• Have faith, but verify.
Standardized Assessment
• All students will have the same test
content, resources, directions, testing
conditions, and scoring procedures.
• One score obtained by one student in
one part of Colorado, will mean the
same as the same score obtained by
another student in a different part of
Colorado.
Media
• The media are not allowed to have access
to the tests, or take pictures or video of
the testing materials (including the front
or back covers of test booklets and
packaged boxes of testing materials for
shipping), during any part of the
assessment process. Not only is media
presence in the school disruptive, it
creates a non-standard assessment
environment and can lead to
misadministrations.
Important – Procedures Manual
• Please Note: Due to the complex nature of
a standardized assessment process, any
practice not specifically permitted should
be presumed inappropriate until and
unless specifically authorized by the
program sponsor (in this case, CDE).
What’s New for 2010-2011
• 2010-2011 Procedure Manual
– Posted in September
• Accommodations Manual
– Eligibility flowchart for CSAP vs. CSAPA
• Updates with Accommodations
– These will be explained later
• Race/Ethnicity Codes
• The Assessment System is the same as in
previous year.
– Our goal is to do the same thing this year, only
better.
DAC Academy 2010
2010 Ethnicity and Race
Reporting
DAC Academy 2010
Ethnicity and race data will be collected through two data
fields on the CELApro, CSAP, and CSAPA BIO Grids.
CELApro, CSAP, and CSAPA BIO Grid
1.Ethnicity will be indicated by choosing one bubble for:
Hispanic/Latino
Not Hispanic/Latino.
2.Race will be indicated by marking all bubbles that
apply for:
American Indian or Alaska Native
Asian
Black or African American
White
Native Hawaiian or Other Pacific Islander
DAC Academy 2010
DAC Academy 2010
DAC Academy 2010
Reminders
• Training
• Housecleaning
• Chain of Custody
• Schedules
DAC Academy 2010
Peer Review Requirements
• Training
 “Inadvertent” misadministrations do not occur
 Comprehensive, interactive, ongoing trainings
must include:
•
•
•
•
•
Test Security
Standardized Environment
Test Proctor Requirements
Teacher vs. Test Proctor
Ethics
 Required documentation
• Individual documentation (DACs keep)
• One form submitted to Unit of Student Assessment
 School administrations must be involved
DAC Academy 2010
Peer Review Requirements
• Housekeeping
 USA recommends signed statements on file in
each district
• Every principal signs ensuring that there are no
assessments or proprietary materials related to
the state assessments stored anywhere on the
school premises (e.g. test booklets from previous
years)
• Ensure that any teacher-created materials
(whether distributed or not) are free of actual
assessment items
» (Does not apply to released items) DACs need a
methodology to document that this has been taken
care of at every school.
DAC Academy 2010
Peer Review Requirements
• Chain of Custody
– Documented before, during, and after testing
– Large print, Braille, Teacher Read Directions,
Oral Scripts, Translated Oral Scripts
– Materials should not be delivered to schools
more than 1 week in advance
– Distribute only the content area being assessed
– Returned to a designated secure location
• Not stored in classrooms
Peer Review Requirements
• CSAP Testing Administration Schedules
– The DAC must ensure that all testing follow the
requirements stated in the Procedures Manual
and Proctor’s Manual.
– Allow time for the proctors to read the
directions from the Proctor’s Manual and
distribute materials before and to collect and
secure materials after the administration of
each assessment.
– The sessions for each content area must be
administered in chronological order.
Peer Review Requirements
– Multiple grades and multiple content areas
must not be tested in the same room.
– CDE strongly recommends that the SAC keep
on file the schedules of exactly when, where
and to whom each assessment is being
administered.
– CDE strongly recommends that each proctor
be given a clear schedule of when to
administer each test by content area and
session.
– Make sure that accommodation considerations
are included in the development of the
schedule.
DAC Academy 2010
Assessment Timeline 2010-2011
CSAP, CSAPA, CELApro and CELAplace
•
Training
•
Ordering and Delivery of Materials
•
Test Administration
•
Data
–
–
September: ELA, Accommodations, and Procedures Manuals
November: Required training for CSAP, CSAPA, CELApro, and logistics (i.e. SAC/DAC
manuals)
–
–
–
–
–
October: Online Enrollments for CELApro
October: Online Enrollments for CSAPA
November: Online Enrollments for CSAP special populations
November: Finalize shipping information for CTB
May: Online enrollments for CELAplace 2010-2011
–
–
–
–
October-December: Submit Nonstandard Accommodation forms for approval
November: Early testing window request completed
January: CELApro Administration window
February-April: CSAP and CSAPA Administration window
–
–
–
–
–
–
September: Student October collection opens (CELApro Precoded labels)
December: Precoded Labels collection opens (CSAP, CSAPA, and COACT
March: Grade 3 CSAP Reading Ncount and CELApro Ncount and SBD
April: Final CELApro 2011 Data Release
May: CSAP/CSAPA Ncount and SBD
July-August: Final CSAP/CSAPA 2011 Data Release
DAC Academy 2010
Assessment Timeline 2010-2011
COACT
•
Test Supervisor Checklist of Dates:
– Late September 2010:
•
–
October 2010:
•
•
–
•
Announcement of training workshop locations and dates mailed to TS, BTS, TACs, and DACs.
December 2010:
•
–
Schools are reminded to submit off-site testing requests, if appropriate.
Late November 2010:
•
–
TACs receive mailing which includes procedures for requesting accommodations, quantities of individual request
forms, and an order form for practice test.
Online schools receive a special mailing which includes National Test Date Voucher Eligibility Request and Test
arrangements.
Mid November 2010:
•
–
Receipt Deadline for principals to return School Renewal Forms.
Receipt Deadline for TS and TACs to return renewal forms to ACT
November 2010:
•
–
ACT mails initial letter from CDE and School renew form to principals with Checklist of Dates and Standard
Testing Requirements.
Receipt deadline for schools to submit completed proposals for off-site testing arrangements to ACT
Early January 2011:
•
Receipt deadline for CDE Assessment to receive National Test Date Voucher Eligibility Request for ONLINE
students
DAC Academy 2010
November Training Schedule
• CSAPA and CELA Training
• 6 Face-to-Face Trainings around CO (Nov 1-9)
• Webinar (Nov 15)
• Every DAC must attend one training
• New DACs must attend a face-to-face one
• We recommend bringing 2 other guests who are
responsible for CSAPA/CELA training in your
district.
Office of Standards and Assessments
Maile Rieple – Program Assistant
(303)866-6929
Standards
Jo O’Brien – Assistant Commissioner
Angela Norlander – Standards
Anna Huffman – Standards
Student Assessment
Beth Celva– Director
Andrina Aragon – Assessment Policy
Glen Sirakavit – CSAP
Margaret Lake – Data Operations
Mira Monroe – CSAPA
Liliana Graham – CELApro
Pam A. Sandoval – NAEP
Christine Deines – COACT
Jolene Fukuhara – Data Operations