Making the Conditions Right for Common, Formative Assessments Jim Lloyd, Ed.D. Response to Intervention Summit August 4, 2010 http://bulldogcia.com.
Download ReportTranscript Making the Conditions Right for Common, Formative Assessments Jim Lloyd, Ed.D. Response to Intervention Summit August 4, 2010 http://bulldogcia.com.
Making the Conditions Right for Common, Formative Assessments Jim Lloyd, Ed.D. Response to Intervention Summit August 4, 2010 http://bulldogcia.com Resources & Contact Information http://bulldogCIA.com [email protected] 427-6531 “If we are serious about improving student achievement, we have to focus relentlessly on changing those things that teachers do that are the most important to change: those things that, when we focus on them, and change them, improve student learning.” Wiliam, D. (2007). Content then process a chapter from Ahead of the Curve (2007) Common Assessments: Theory to Practice There is no single right way to do this…however there are some things that make it easier Don’t be fooled by people wanting to sell you programs It is important to balance process with product This tends to be a shift in the way C.I.A. development has taken place so it takes time There is more than 1 way to do this…this is a work in progress…as you do this you will continue to grow and get better This is collaborative work This is about changing teaching practice Learning Intentions for Today Understand that it is critical that we get Tier 1 correct http://bulldogcia.com/Documents/Articles/Marolt_Goss _Lloyd_2008.pdf Relay my experiences with formative assessment and how it has worked Provide you with an overview of how you can begin to consider doing this or; refining this or; expanding this Essential Questions 1.What do we mean by common, formative assessments? 2.What does the research indicate about assessment for learning? 3.What predicates the success of common formative assessments? 4.How can you do this work? First a few graphic organizers Organizes Our District/Building Improvement Efforts—CFA needs to be included in Stage 2 Organizes our professional development and CIA efforts for teachers Where do these assessments fall on the assessment continuum? Balanced Assessment Summative Formative Formal and informal processes teachers and students use to gather evidence to directly improve the learning of students assessed Provides evidence achievement to certify student competence or program effectiveness Formative uses of summative data Assessment for learning Assessment for learning Use assessments to help students assess and adjust their own learning Use classroom assessments to inform teacher’s decisions Use of summative evidence to inform what comes next for individuals or groups of students **Excerpted from ATI Author: Rick Stiggins Assessment Continuum • • • Classroom Assessments Short-cycle Assessments Common Assessments External Assessments Most Formative Less Formative More Summative Most Summative Ongoing between student and teacher Risk free nonevaluative feedback Is a communication between teacher and learner as to where they are and what they need to work on. Minute to Minute Hour to Hour Day to Day • Embedded within instructional units. • If you grade them they are not formative from a purest sense • If you provide quality feedback regarding how students can improve they approach formative Weekly, Unit or Monthly • • • • Collaboratively developed and standards-based. Come from unconnected texts. Novel assessment experience Report to students and others at the end of learning • Standardized Achievement Tests • OAT, OGT • NAEP, SAT, ACT Monthly to Quarterly *Framework based on Many & Jakicic (2006) National Staff Dev. Council Semester to Annual Essential Question #1: What do we mean by Common, Formative Assessments? Formative assessments… Not a program…these are concepts Not something you can simply purchase Is likely an instructional practice that your teachers currently incorporate Has a great deal of research to support its effectiveness when it is done as a matter of practice in the classroom Increases its power when developed collaboratively and administered commonly Reconstitutes traditional curriculum, instruction and assessment experiences for students AND teachers Common, Formative Assessments? Formative assessments Have certain characteristics Not graded Arrive in time to help students Informs the assessment users Common Formative Assessments…A, B or all of the above? Common formative techniques that teachers collaborative on and use (e.g. stars and steps; find it, fix it; status, target, plan) B. Common formative assessments that teachers collaboratively develop and administer to students A. We will talk about both types as both serve a specific purpose and are effective A Common, Formative Assessment Technique A collaboratively development common formative assessment Essential Question #2: What does the research say? What works & How do we know? We explain “what works” through effect size (d) Most things do have an impact on students The “hinge point” on whether or not something is a difference maker is an effect of .40 or greater The typical teacher gets an effect between .20 to .40 General Rank Order of Effect Size: By Category Contribution Student Home School Teacher Curricula Teaching Effect d=.40 d=.31 d=.23 d=.49 d=.45 d=.42 Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge: London. What does the research find to be the “difference maker”? The teacher… …but the most important variable is what teachers do, rather than what they know. All things being equal…pedagogy matters more than content knowledge. This, coupled with student involvement in the process, leads to high levels of learning. IF TEACHERS ARE THE DIFFERENCE MAKERS THEN WHAT WORKS? We know the things in this book do, but can they be universally applied with high frequency? Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge: London. Contributions from the Teacher Area Goal setting Adv. Org. Concept Map Teacher Clarity Feedback Formative assesmnt Questioning Inquiry-based inst Cooperative learn. Study Skills Effect d=.56 d=.41 d=.57 d=.75 d=.73 d=.90 d=.43 d=.31 d=.41 d=.59 Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge: London. What do “difference makers” do? Engage in impactful instructional practices Macro v. Micro Strategies What is a micro strategy? What is a macro strategy? Micro strategies—are effective, but not as robust (e.g.—Identifying Similarities and Differences, Generating and Testing Hypotheses) Macro strategies—are equally effective and robust These Macro-Strategies Macro-strategies— can be used to facilitate a common instructional language for educators Can be used in just about all instances When the classroom environment is engineered with these in mind and they are implemented as a matter of routine, they are a powerful educational innovation. Macro-Strategies Teacher Clarity d=0.75 (27%ile) Formative eval of programs d=0.90 (31%ile) Providing feedback d=0.73 (26%ile) Authors commonly associated with these: Arter, Chappuis, Brookhart, Stiggins, Hattie Bibliography located at: http://bulldogcia.com/Articles.htm Teacher Clarity Clarity (d=0.75) Clear and understandable visions of the learning goals Making distinctions between learning goals and learning activities Not just rubrics, but sharing models of strong and weak work Read scenarios 1 and 2 from learning module Clarity—Where am I going? A classroom litmus test for learning: Ask students what did you learn today. If… The answer(s) focus on what students DID as opposed to what the targets were, clarity hasn’t been achieved. Research Shows: students perform better when learning goals (those that describe learning intentions) are used as opposed to performance goals Clarity—Where am I going? Key Point: differentiate between learning targets and learning activities How might you do this? Take 5 and talk to your neighbor. Clarity—make it clear…don’t make it clear…which one is it? “I’m afraid of dumbing down” the target…how can engineer clarity without doing this? Point the way to success without giving away the store Make sure that you define the expectations of the verb e.g. Understand means: identify, define, explain? Clarity—Sharing target v. not sharing “Just make sure students can describe the intended learning before you ask them to engage in sustained independent practice and before the summative assessment.” Chappuis (2009). Seven Strategies of Assessment for Learning p. 41 Clarity—Defining quality work Students must first be able to: Understand concepts that define quality before being able to self-assess them HOW? One way to do this is to use rubrics—refer to page 30 (Chappuis, 2009) On page 33 there is a protocol for introducing criteria to younger students Strong and weak work Use anonymous samples After defining the criteria… have students look at 2 samples and determine which ones do/do not meet the criteria and why See p. 44 (Chappuis, 2009) for a protocol Clarity—Using rubrics It acts as a diagnostic tool so.. Each feature of quality needs described—from beginning to proficient. “This is beginning because…” It aligns with your teaching the elements of quality Allows you to provide feedback related to elements of quality Feedback Feedback (d=0.73) Teachers to students Students to teacher What students understand, where they make errors, when they have misconceptions… Quality Feedback—what is it? It’s descriptive—more than just “good job” It directs attention to intended learning Occurs during learning Addresses partial understandings Doesn’t do the thinking for the student Limits correctives to what students can act on Can come from: Teacher, peer or student Chappuis, J. (2009). Seven Strategies of Assessment for Learning. Feedback Purpose—fill the gap between what is understood and what needs to be understood (Chappuis & Chappuis, 2008) Helps students begin to monitor where they are in relationship to where they need to be Indistinguishable from high-quality instruction (Stiggins, Arter, Chappuis & Chappuis, 2006) Formative Evaluation of Programs Formative Evaluation of Programs (d=0.90) Teachers create structures to determine whether or not instruction is working in real-time This allows them to make changes in time to help Essential Question #3: What predicates successful formative assessment implementation? Things that “grease the skid” What is… Why do we need… Coherence Establishing Coherence Professional Development Products Considerable curriculum mapping Identification of most important learning targets Identification of instructional anchors through Big Ideas & Essential Questions Professional Development Application Building level commitments to make learning clarity a reality (learning targets v. teacher activities) A district commitment to formative assessment as a matter of instructional practice Coherence Clear Standards-Based Learning Targets High Quality Instruction Balanced Assessment (formative & summative) If you focus here, you can establish a common instructional language which will greatly help Example of the process our staff went through… Coherence Power Standards/Indicators Identifying the most important learning targets Endurance Leverage Foundation/Readiness Located at http://bulldogCIA.com Student Friendly Language Power indicators can then be transferred into statements that students are more able to understand. They are incorporated into the classroom environment and deliberately pointed out during instruction This helps to make the learning objectives explicit Coherence Unwrapping the Standards Deconstructing the standards to have them make more sense for teachers…and…students Examples of Essential Questions What does the research say about instruction and assessment? Why curriculum coherence? How does one engineer and facilitate high-quality instructional practices and environments? What does balanced assessment practices mean and why are they important? Formative Evaluation of Programs Video Erica Caso—OFCS 5th grade teacher Two Purposes for Assessment SUMMATIVE Assessments OF Learning How much have students learned as of a particular point in time? FORMATIVE Assessments FOR Learning How can we use assessment information to help students learn more? Summative Assessments Have a different purpose than formative Provide evidence of student competence or program effectiveness Report out achievement Formative Evaluation Teacher Where are they going? Where are they now? What’s next? Student Where am I going? Where am I now? What’s next? These are on-going What is formative assessment? The “instrument” doesn’t make the assessment formative…it is the USE that does Use: Immediately used Info fed back into the instructional system Multiple users The use in assessment for learning is to gather evidence to adjust teaching and learning 5 Keys to Quality Classroom Assessment 1. Why assess? (PURPOSE) 2. Assess what? (TARGETS) 3. Assess how? (DESIGN) 4. Communicate how? (COMMUNICATION) 5. Involve students how? (STUDENT INVOLVEMENT) 7 Strategies of Assessment for Learning Based on the underlying belief that students are the most influential decision makers in the classroom Addresses 3 questions: Where am I now? Where am I going? How do I close the gap? Chappuis, J. (2009). Seven strategies of assessment for learning. Portland, OR: ETS. The Seven Strategies of Assessment for Learning Where am I going? 1. Provide a clear statement of the learning target 2. Use examples and models Where am I now? 3. Offer regular descriptive feedback 4. Teach students to self-assess and set goals How can I close the gap? 5. Design focused lessons 6. Teach students focused revision 7. Engage students in self-reflection; let them keep track of and share their learning Chappuis, J. (2009). Seven strategies of assessment for learning. Portland, OR: ETS. Why differentiate between the two? In order to reap the effective size benefits of formative assessment practices, certain conditions must be met Formative assessments: Directly aligned to standards Match what has been taught Provides detailed information to pinpoint student misunderstandings Arrive in time to help Users take action based on results What gives formative assessment its power? The closer to everyday instruction, the stronger it is Uses instructional opportunities to improve learning and correct misunderstandings Focuses on developing both self and peer assessment skills Where am I now? 3. Provide regular descriptive feedback Fills the gap between what is understood and what needs to be understood (Chappuis & Chappuis, 2008) Helps students begin to monitor where they are in relationship to where they need to be Indistinguishable from high-quality instruction (Stiggins, Arter, Chappuis & Chappuis, 2006) 4. Teach students to self-assess and set goals Students need to know where they are in order to set goals They need to be taught how to self-assess How Can I Close the Gap? 5. Design lessons to focus on one target or quality aspect at a time Makes monitoring the learning more specific 6. Teach students focused revision Task analyze the target into small chunks Practice and revise 7. Engage students in self-reflection; have them track and share This impacts the retention of things learned and increases motivation Essential Question #4: How can you do this work? Bringing the work together Using a common unit planning approach for a common educational experience Unit Design Framework Served as our C.I. & A. planning tool Brought cohesiveness and additional collaboration to: PD learning outcomes and products It focuses on implementation Questions? Thanks!