Common Core Institute #2

Assessment Literacy

Agenda

• • • • • • • • • Welcome Announcements and Introductions Introduction to Assessment Assessment Types Assessment System – Break Technical Quality Assessment and Accountability Building an Assessment Plan – Lunch Q & A/Closing Remarks/Announcements

What is ASSESSMENT?

• • • • The systematic collection, review and use of information about educational programs to improve student learning (Palomba & Banta) A process embedded within larger systems in a school/district/state Noun: tools and strategies used to collect evidence (Heritage & Herman) Verb: process of using evidence to improve decision making (Heritage & Herman)

Purpose

• Assessments are always developed with purpose in mind: – What is to be assessed?

– Why is assessment taking place?

– How will student learning be measured?

– How will the results of the assessment be used?

Types of Assessment

• • • Summative Assessment Interim Assessment Formative Assessment

Tiers of Assessment

Wide Summative Narrow Interim (instructional, evaluative, predictive) Formative (minute-by-minute, integrated into the lesson) Less Often Frequency of Administration More Often Adapted from Perie et al.(2009) and Talbot (2011)

Summative Assessments

• • • Given once at end of a longer period of time Are typically large scale assessments, which are given state wide or nationally, but may be district wide or semester exams Generally used to evaluate performance against a defined set of content standards (ASSESSMENT OF LEARNING--Stiggins, Arter, Chauppuis and Chauppuis , 2006)

Summative Assessments

• • • Can meaningfully aggregate/disaggregate and report results Use results for accountability and policy information Least flexible

Grade Tested

K 1 2 3 4 5 6

Criterion-Referenced Tests Arkansas Assessment Overview 2011-2012 English Language Development Assessment English Language Development Assessment English Language Development Assessment Augmented Benchmark Exam: Literacy and Math Alternate Portfolios for IEP English Language Development Assessment Augmented Benchmark Exam: Literacy and Math Alternate Portfolios for IEP English Language Development Assessment Augmented Benchmark Exam: Literacy, Math, and Science Alternate Portfolios for IEP English Language Development Assessment Augmented Benchmark Exam: Literacy and Math Alternate Portfolios for IEP English Language Development Assessment NAEP Norm-Referenced and National Tests Kindergarten Screening: Qualls Early Learning Inventory (QELI) Norm-Referenced Test: Iowa Test of Basic Skills (ITBS) Reading, Language and Mathematics Norm-Referenced Test: Iowa Test of Basic Skills (ITBS) Reading, Language and Mathematics Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics Long-Term Trend (9 year olds) Reading and Math Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics Augmented Benchmark Exam: Abbreviated ITBS Reading, Language, Mathematics and Science Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics

Grade Tested

7 8 11 12 9 10

Criterion-Referenced Tests Arkansas Assessment Overview 2011-2012 NAEP Augmented Benchmark Exam: Literacy, Math, and Science Alternate Portfolios for IEP English Language Development Assessment Norm-Referenced and National Tests Augmented Benchmark Exam: Abbreviated ITBS Reading, Language, Mathematics and Science Augmented Benchmark Exam: Literacy and Math Alternate Portfolios for IEP English Language Development Assessment End-of Course Algebra I Grade 9 Math Portfolio for IEP English Language Development Assessment Long-Term Trend (13 year olds) Reading and Math Augmented Benchmark Exam: Abbreviated ITBS Reading, Language and Mathematics ACT EXPLORE Iowa Test of Educational Development (ITED) Complete Battery Reading and Mathematics End-of-Course Geometry End-of-Course Biology Grade 10 Science Portfolio for IEP English Language Development Assessment ACT PLAN/ PSAT Grade 11 Literacy Portfolio for IEP English Language Development Assessment English Language Development Assessment (17 year olds) Reading and Math ACT** SAT** **AP (Advanced Placement) ACT** SAT** **AP (Advanced Placement)

Interim Assessments

• • • • Fall between formative and summative assessments Given periodically throughout the year or course Are typically medium-scale assessments, which are given at the district or school level Generally used to evaluate performance against a defined set of content standards (ASSESSMENT OF LEARNING--Stiggins, Arter, Chauppuis and Chauppuis , 2006)

Interim Assessments

• • • Can meaningfully aggregate/disaggregate and report results Use results for program evaluation, resource allocation, and instructional information on past practice More flexible than summative

Formative Assessment

• • • Occurs moment-to-moment as part of instruction Is a process used by teachers and students frequently and is embedded in the current unit of instruction Are small scale, short cycle assessments given in the classroom to diagnose where students are in their learning (ASSESSMENT FOR LEARNING--Stiggins, Arter, Chauppuis and Chauppuis, 2006)

Formative Assessment

• • • Cannot aggregate/disaggregate and report results Use results to provide descriptive feedback or to modify instruction Most flexible

Formative Assessment

Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve students’ achievements of intended instructional outcomes (FAST SCASS, October 2006)

• • •

Five Critical Features to Guide Educators Toward Effective Use of Formative Assessment

Learning Progressions – clearly articulates the trajectory along which students are expected to progress Learning Goals and Success Criteria – immediate goals to be learned Descriptive Feedback – ideas, strategies, and tasks students can use to close the “gap” between his or her current learning level and the next level Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

• •

Five Critical Features to Guide Educators Toward Effective Use of Formative Assessment

Self and Peer Assessment – Students evaluate their own learning and their peers learning based upon a success criteria Collaboration – All participants share responsibility for learning Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Formative Assessment Cycle

Heritage, M. Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

WHAT ARE THE MAJOR DIFFERENCES AMONG THE ASSESSMENT TYPES?

Dimension

Purpose Implementation Timing Scope Audience

•

Dimensions of Assessment

Assessment for Learning

Formative

Instructional • •

Type of Assessment

Assessment of Learning

Interim

Most designed for managerial uses Some designed for instructional uses

Summative

• Managerial • Driven by moment-to-moment decisions; generated or selected by teacher; individualized • Regulated by protocols developed in or out of the classroom; teacher-generated or externally generated • • During instruction High frequency • • Narrow; one or very few learning objectives at a time Classroom (teachers and students) • • • After instruction or during a break in instruction Moderate frequency Moderate; a manageable number of objectives • Administration and/or classroom • • After instruction Low frequency • • • • Broad; comprehensive set of objectives Public Administration Classroom Adapted from Talbot (2011)

THE ASSESSMENT SYSTEM

Comprehensive Assessment System

• • Consists of three types of assessments – Summative – Interim – Formative Assessment of learning and assessment for learning

Assessment System

Formative Assessment Summative Assessment Assessment System Interim Assessment

Assessments in the Assessment System

Learning Goals/Progressions

Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).

The Three C’s

• • • Comprehensiveness Coherence Continuity Pellegrino, J., Chudowsky, N. & Glaser, R. Knowing What Students Know. National Research Council.

Building Complementary Assessment System

• • • Role of Classroom Assessment Role of District Assessment Role of State Assessment

Role of Classroom Assessment

• • Monitor ongoing progress Make ongoing adjustments – Adapting and responding to learning needs – Use different teaching methods – Use different teaching materials Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Role of District Assessment

• • • •

For Districts

Monitor progress Identify needed changes in: – policies/procedures – – funding school organization Guide curriculum changes Guide professional development • • • •

For Schools

Set school improvement goals Determine staffing needs Guide instructional changes Guide professional development Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Role of State Assessment

• • • • •

For States

Monitor standards Monitor equity of opportunity Guide changes in: – laws and regulations – – policies and funding organizational structures Set improvement goals Guide professional development • • • •

For Districts

Monitor standards Anchor district standards and assessments Guide changes in: – board rules – policies and funding – organizational structures Monitor school improvement efforts Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

BREAK: WE WILL RETURN IN 15 MINUTES

TECHNICAL QUALITY

• • •

Why does technical quality matter?

Test construction Test administration Usefulness of results – Credibility – Accuracy of score interpretation – Sound basis for adjusting instruction Adapted from Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Assessment Design and Format

• • • • Items, tasks, strategies, rubrics must be purposely designed to serve purpose Various assessment formats and contexts: –

Paper-pencil, performance

–

Selected, constructed, extended response

–

Formal “tests,” analysis of work, homework

Format does NOT = quality Good assessments DO well serve their purpose(s) Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).

Credibility • Valid and reliable for purpose intended • Ensuring fairness and eliminating bias

Validity and Reliability

• • Validity is measuring what is intended to be measured and using data to make accurate inferences.

Reliability is consistency of test results from one test to another, from one part to another, from one time to another, and from one score to another.

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Bias and Fairness

• • Bias is giving or not giving an advantage to one group or another.

Fairness refers to the purposes and uses of tests in ways that are clearly communicated, ethical, and valid for all students to demonstrate their knowledge and skills.

Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

• •

Accommodations and Validity

Individuals with Disabilities Education Act (IDEA) Section 300.160(b)--Accommodation guidelines – – (1) A State (or, in the case of a district-wide assessment, an LEA) must develop guidelines for the provision of appropriate accommodations.

(2) The State's (or, in the case of a district-wide assessment, the LEA's) guidelines must: • (i) Identify only those accommodations for each assessment that do not invalidate the score; and • (ii) Instruct IEP Teams to select, for each assessment, only those accommodations that do not invalidate the score.

Elementary and Secondary Education Act (ESEA), Title I Section 200.2(b)(2) requires that a State’s assessment system be designed to be valid and accessible for use with the widest possible range of students, including students with disabilities and students with limited English proficiency.

Dos and Don’ts When Selecting Accommodations

Do…make accommodation decisions based on Don’t…make accommodations decisions based on individualized needs.

whatever is easiest to do (e.g., preferential seating).

Do…select accommodations that reduce the effect of the disability to access instruction and demonstrate learning.

Don’t…select accommodations unrelated to documented student learning needs or are intended to give students an unfair advantage.

Do…be certain to document instructional and assessment accommodation(s) on the IEP or 504 plans.

Don’t…use an accommodation that has not been documented on the IEP or 504 plans.

Do…be familiar with the types of accommodations that can be used as both instructional and assessment accommodations.

Don’t…assume that all instructional accommodations are appropriate for use on assessments.

Do…be specific about the “Where, When, Who, and How” of providing accommodations.

Don’t…simply indicate an accommodation will be provided “as appropriate” or “as necessary.” Thompson, S., Morse, A., Sharpe, M. & Hall, S. Accommodations Manual: How to Select, Administer, and Evaluate Use of Accommodations for Instruction and Assessment of Students with Disabilities. Council of Chief State School Officers, Assessing Special Education Students State Collaborative.

Dos and Don’ts When Selecting Accommodations

Do…refer to state accommodations policies and understand implications of selections.

Don’t…check every accommodation possible on a checklist simply to be “safe.” Do…evaluate accommodations used by the student.

Do…get input about accommodations from teachers, parents, and students, and use it to make decisions at IEP team or 504 planning committee meetings.

Don’t…assume the same accommodations remain appropriate year after year.

Don’t…make decisions about instructional and assessment accommodations alone.

Do…provide accommodations for assessments routinely used for classroom instruction.

Don’t…provide an assessment accommodation for the first time on the day of a test.

Do…select accommodations based on specific individual needs in each content area.

Don’t…assume certain accommodations, such as extra time, are appropriate for every student in every content area.

Thompson, S., Morse, A., Sharpe, M. & Hall, S. Accommodations Manual: How to Select, Administer, and Evaluate Use of Accommodations for Instruction and Assessment of Students with Disabilities. Council of Chief State School Officers, Assessing Special Education Students State Collaborative.

Accuracy of Score Interpretation

• • • Use results based on the design and intended use of assessment Avoid overinterpretation/misuse of results Factors affecting trend lines: change in test blueprint; change in content standards; change in scale; change in cut scores; change in student population Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

WHAT IS THE RELATIONSHIP BETWEEN THE ASSESSMENT SYSTEM AND ACCOUNTABILITY?

Accountability System

• • • Data system for reporting selected pieces of data about student learning and factors affecting learning to determine success of the educational system Used as means of holding schools and their supports (district, state, legislation, policies, procedures, practices, funding) accountable for how well students are learning Assessment is one part of the accountability system Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

Accountability System

Assessment System Data Graduation Rates Highly Qualified Teacher Rates Accountability System Attendance Rates Other Indicators Discipline Data

State

How well the laws and funding passed by legislature have positively impacted student achievement of the standards

Tiers of Accountability

District

How well students have achieved standards

Schools

How well students have achieved the standards

Students

How well students have achieved the standards How well the State Department’s policies and procedures have resulted in student achievement of the standards and high functioning schools How well funds are used to carry out laws, policies, and procedures in supporting student achievement How effectively the school has maximized students’ opportunity to learn by keeping them in school How diligent students have been in using the opportunities to learn provided by the school How well qualified teachers are to teach the standards How effectively the school has created a safe and positive learning environment How well schools have used given resources to maximize students’ opportunity to learn Adapted from Sheinker, J. & Redfield, D. Handbook for Professional Development in Assessment Literacy. Council of Chief State School Officers, Comprehensive Assessment Systems State Collaborative.

BUILDING AN ASSESSMENT PLAN

Sample Assessment Plan

Central Bucks School District, Doylestown, PA

Sample Assessment Plan

Central Bucks School District, Doylestown, PA

Sample Assessment Plan

Sample Assessment Plan

Sample Assessment Plan

Assessment Cycle

Assessment Plan Template

Assessment type(s) Assessment tools Expectations for student use of data Primary Expectations for teacher use of data Expectations for school and/or district level use of data Emphasis area for professional development Continuous or Daily Weekly or Monthly Unit Quarterly Annual

Ultimate goal of a coherent assessment approach

: Create an aligned system of assessment so that all assessments are measuring agreed upon learning goals and discrete measurement takes place along the learning progression embedded within each goal.

Courtesy of Syracuse School District in Syracuse, NY, and the New York Comprehensive Center. http://www.nycomprehensivecenter.org/docs/form_assess/SyracuseAssessmentFramework.pdf

Ultimate goal of a coherent assessment approach: Create an aligned system of assessment so that all assessments are measuring agreed upon learning goals and discrete measurement takes place along the learning progression embedded within each goal.

Assessments in the Assessment System

Learning Goals/Progressions

Herman, J. & Heritage, M. Assessment Systems That Work. Joint Presentation sponsored by the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) and Assessment and Accountability Comprehensive Center (AACC).