A New Vision of Assessment

Association of American Publishers Annual Meeting Laura Slover, PARCC

June 4, 2013 1

PARCC States

PARCC Priorities

1. Determine whether students are college and

career ready or on track

2. Connect to the Common Core State Standards 3. Measure the full range of student performance, including that of high- and low-achieving students 4. Provide educators data throughout the year to inform instruction 5. Create innovative 21st century, technology-based

assessments

6. Be affordable and sustainable 3

In the Last Year…

June 2012 August 2012 September 2012 October 2012 November 2012 December 2012 January 2013 March 2013 April 2013

                 Minimum Technology Specifications, Version 1.0, Released Item Development Began Item and Task Prototypes Released Adopted Principles for Comparability with SBAC College- and Career-Ready Determination Policy Adopted Item Tryout and Field Testing RFP Released Retest Policy Adopted Minimum Technology Specifications, Version 2.0, Released Formed Executive Committee for Streamlined Decision-Making Assessment Administration Capacity Planning Tool Released Estimated Time on Task Released Item Development Research with 2,300 Students in 6 states PARCC Non-Profit Formed Added Project Management & Procurement Capacity Draft Performance Level Descriptors Released for Comment Draft Accommodations Manual Released for Comment Test Blueprints & Evidence Statements Released 4

Looking Ahead: 2013

June 2013 July 2013 August 2013 Fall 2013 Winter 2013-2014

• • • • • • • • • • • • • • • • • • • • Item Tryout Studies with 4,800+ Students in 4 States Guidance on Participation in Field Test and Practice Tests Final Subject- and Grade-Level Performance Level Descriptors Adopted First Edition of PARCC Accommodations Manual Estimates for Summative Assessment Costs Released PARCC Technology Components RFP Released Diagnostic & Formative Assessment RFPs Released Phase I Item Development Complete Additional Sample Items Released Schools/Districts Notified of Selection for Field Testing Partnership Resource Center RFP Released Additional Information about Testing Windows Released Sample Items Re-Released on Technology Platform Design of Student Score Reports Released Minimum Technology Specifications, version 3.0

Training Materials for IEP Writing Teams on Accommodations Manual Timeline / Plan for Student Registration for Operational Testing Specifications for Online Testing Portal Released Final Information about Field Testing Final Information about Timing of Data Return 5

Looking Ahead: 2014 and Beyond

Spring 2014 Summer 2014 Fall 2014 Winter 2014-2015 Spring 2015 Summer 2015

• • • • • • • • • • • Performance-Based Assessment Field Test Administration (Feb-March) End-of-Year Field Test Administration (April-May) Practice Test Available Standard-Setting Methodology Released Final Information about Methodology for Calculating Student Scores Final Test Security Policies Released Final Registration Timeline and Process for Operational Testing Released Final Technology Specifications Released Phase II Item Development Complete Operational Assessment Administration Manual Forms Construction for Operational Administration Complete • 1 st Operational Assessment – Fall 2014 Block Schedule Administration • 1 st Operational Assessment – Spring 2014 Administration • Student Performance Levels and Associated Cut Scores for all Grades and for College- and Career-Ready Determination (Standard-Setting) 6

Assessments

ELA/Literacy and Mathematics, Grades 3–11

Beginning of School Year Flexible administration

Diagnostic Assessment Mid-Year Assessment Performance Based Assessment End-of-Year Assessment

End of School Year

Speaking and Listening Assessment Key: Optional Required

7

Evidence-Centered Design (ECD) for the PARCC Assessments

Model Content Frameworks Evidence Statements Tasks

To make claims about what students know, we must operationalize the standards Based on analysis, evidence drive task development Tasks are designed to elicit specific evidence from students 8

What is Different About PARCC

’

s Development Process?

• • • PARCC states first developed the Model Content Frameworks to provide guidance on key elements of excellent instruction aligned with the Standards.

Then, those Frameworks informed the assessment blueprint design.

Aligned evidence statements and task models followed.

• •

So…

PARCC is designing the assessments around exactly the same content shifts the standards expect of teachers and students.

PARCC is communicating in the same voice to teachers as it is to assessment developers 9

Advances in the PARCC ELA/Literacy Assessment

10

What Are the Shifts at the Heart of PARCC Design (and the Standards) for ELA/Literacy?

1.

2.

3.

Complexity: Regular practice with complex text and its academic language Evidence: Reading and writing grounded in evidence from text, literary and informational Knowledge: Building knowledge through content rich nonfiction 11

PARCC’s Four Core Commitments to ELA/Literacy Assessment Quality

1.

2.

3.

4.

Texts Worth Reading: The assessments will use authentic texts worthy of study instead of artificially produced or commissioned passages (commissioned passages used only if necessary to “ round out ” a research task).

Questions Worth Answering: Sequences of questions that draw students into deeper encounters with texts will be the norm (as in an excellent classroom), rather than sets of random questions of varying quality.

Better Standards Demand Better Questions: Instead of reusing existing items, PARCC will develop custom items to the Standards.

Fidelity to the Standards (now in teachers’ hands): PARCC evidences are rooted in the language of the Standards so that expectations remain the same in both instructional and assessment settings.

12

Claims Driving Design: ELA/Literacy

Students are on-track or ready for college and careers

Students read and comprehend a range of sufficiently complex texts independently Reading Literature Reading Informational Text Vocabulary Interpretation and Use Students write effectively when using and/or analyzing sources. Written Expression Conventions and Knowledge of Language Students build and present knowledge through research and the integration, comparison, and synthesis of ideas.

13

Evidence-Based Questions

Not Text-Dependen t

In “ Casey at the Bat, something.

” Casey strikes out. Describe a time when you failed at In “ Letter from a Birmingham Jail, ” Dr. King discusses nonviolent protest. Discuss, in writing, a time when you wanted to fight against something that you felt was unfair.

From “The Adventures of Tom Sawyer,” have students identify the different methods of removing warts that Tom and Huck talk about. Ask students to devise their own charm to remove warts. Are there cultural ideas or artifacts from the current time that could be used in the charm?

Text-Dependent

What makes Casey’s experiences at bat humorous?

What can you infer from King’s letter about the letter that he received?

Why does Tom hesitate to allow Ben to paint the fence? How does Twain construct his sentences to reflect that hesitation? What effect do Tom ’ s hesitations have on Ben?

14

Which Prompt Exhibits Writing to Sources?

In “Letter from Birmingham Jail,” Martin Luther King: 1.

Gives several reasons to justify his presence in the city at that time. Write an essay in which you relate a similar situation in your own life. Tell about an experience in which you had to justify your reasons for being in a particular place at a particular time. 2.

3.

Describes a process that he and his followers have recently undertaken. Write an essay in which you describe this process and tell how the letter shows that this process is important to the civil rights movement.

Is specifically responding to criticism about the goals of the civil rights movement. Write an essay in which you relate these goals to aspects of the modern-day civil rights movement. 15

Which Questions Require Rigorous Analysis of Complex Texts?

1.

2.

3.

4.

5.

When the author states that “Lee was acting like an angry bear,” is he using simile, metaphor, hyperbole, or onomatopoeia? What is the setting of the story about Lee’s adventures?

What is the relationship between the setting of the story and the main event of the plot? How did Lee’s decision to stay affect the outcome of the story?

a.

b.

c.

d.

Which of the following words describes Lee: brave, determined, careful, or hopeful?

Questions #1 and #2 Questions #2 and #3 Questions #3 and #4 Questions #4 and #5 16

Sample Items Illustrating the Advances in ELA/Literacy

17

Understanding the Research Simulation Task (Grade 7 Example)

• • • Students begin by reading an anchor text that introduces the topic. EBSR and TECR items ask students to gather key details about the passage to support their understanding.

Students read two additional sources (may include a multimedia text) and answer a few questions about each text to learn more about the topic so they are ready to write the final essay and to show their reading comprehension. Finally, students mirror the research process by synthesizing their understandings into an analytic essay using textual evidence from several of the sources.

18

Texts Worth Reading?

• Range: Example of assessing reading across the disciplines and helping to satisfy the 55%-45% split of informational text to literature at the 6-8 grade band. • Quality: The texts on Amelia Earhart represent content-rich nonfiction on a topic that is historically significant.

• Complexity: Quantitatively and qualitatively, the passages have been validated and deemed suitable for use at grade 7.

19

ELA/Literacy: Grade 7 Sample Item

Claims

Earhart and Noonan lived as castaways on Nikumaroro Island.

Earhart and Noonan’s plane crashed into the Pacific Ocean People don’t really know where Earhart and Noonan died. Part A: Highlight the claim that is supported by the most relevant and sufficient facts within “ Earhart ’ s Final Resting Place Believed Found.

” Part B: Click on two facts within the article that best provide evidence to support the claim selected in Part A. 20

Questions Worth Answering?

Grade 7 Analytical Prose Constructed-Response Item #1

Based on the information in the text “ Biography of Amelia Earhart, ” write an essay that summarizes and explains the challenges Earhart faced throughout her life. Remember to use textual evidence to support your ideas.

21

Questions Worth Answering?

Final Grade 7 Prose Constructed-Response Item #2

You have read three texts describing Amelia Earhart. All three include the claim that Earhart was a brave, courageous person. The three texts are: • • • “ Biography of Amelia Earhart ” “ Earhart's Final Resting Place Believed Found ” “ Amelia Earhart ’ s Life and Disappearance ” Consider the argument each author uses to demonstrate Earhart ’ s bravery.

Write an essay that analyzes the strength of the arguments about Earhart ’ s bravery in at least two of the texts. Remember to use textual evidence to support your ideas.

22

Advances in the PARCC Mathematics Assessment

23

What Are the Shifts in the Math Standards at the Heart of PARCC Design?

1.

Focus strongly where the Standards focus

2.

Coherence: Think across grades and link to major topics within grades

3.

Rigor: In major topics, pursue conceptual understanding, procedural skill and fluency, and application. 24

PARCC’s Core Commitments to Mathematics Assessment Quality

    Focus: Instead of randomly sampling a mile-wide array of topics, PARCC assessments will focus strongly. Teachers will be rewarded for teaching crucial material in depth, not penalized for failing to “cover topics.” Problems worth doing: Multi-step problems, conceptual questions, applications, and substantial procedures will be common, as in an excellent classroom.

Better Standards Demand Better Questions: Instead of reusing existing items, PARCC will develop custom items to the Standards.

Fidelity to the Standards (now in Teacher’s hands): PARCC evidences are rooted in the language of the Standards so that expectations remain the same in both instructional and assessment settings.

25

Claims Driving Design: Mathematics

Students are on-track or ready for college and careers Students solve problems

involving the major

content for their grade level with connections to practices Students solve problems

involving the additional and supporting content

for their grade level with connections to practices Students express

mathematical reasoning

by constructing mathematical arguments and critiques Students solve real world problems engaging particularly in the modeling practice Student demonstrate fluency in areas set forth in the Standards for Content in grades 3-6 26

The Special Role of Technology in Advancing Assessment and the Shifts

• • • • • Technology enhancements supporting

accessibility

(e.g., the ability to hover over a word to see and/or hear its definition, etc.)

Transformative formats

making possible what couldn’t be done at all with paper (e.g., running a simulation to improve a model, game-like environments, drawing/constructing diagrams or visual models, etc.) Getting

beyond the bubble

and avoiding drawbacks of traditional selected response such as guessing or choice elimination Capturing

complex student responses

through a device interface (e.g., using drawing tools, symbol palettes, etc.).

Machine scorable multi-step tasks are

more efficient

to administer and score. 27

Sample Items Illustrating the Advances in Mathematics

28

Overview of PARCC Mathematics Task Types

Task Type I. Tasks assessing

concepts, skills and procedures

II. Tasks assessing

expressing mathematical reasoning

• • • • • • • • • •

Description of Task Type

Balance of conceptual understanding, fluency, and application Can involve any or all mathematical practice standards Machine scorable including innovative, computer-based formats Will appear on the End of Year and Performance Based Assessment components Sub-claims A, B and E Each task calls for written arguments / justifications, critique of reasoning, or precision in mathematical statements (MP.3, 6). Can involve other mathematical practice standards May include a mix of machine scored and hand scored responses Included on the Performance Based Assessment component Sub-claim C

III. Tasks assessing

modeling / applications

• • • • • Each task calls for modeling/application in a real-world context or scenario (MP.4) Can involve other mathematical practice standards May include a mix of machine scored and hand scored responses Included on the Performance Based Assessment component Sub-claim D For more information see PARCC Task Development ITN Appendix D. 29

Math Grade 3 Illustrative Sample Item: Type II Task

• • • This a fairly traditional fraction task in a computer-based setting. Unlike traditional multiple choice, it is difficult to guess the correct answer or use a choice elimination strategy and there is more than one correct solution. Unlike paper and pencil tests, students can create a visual representation even though the task is scored automatically. 30

Math Grade 3 Illustrative Sample Item: Type II Task

• Second part of multi-step problem, and, unlike traditional multiple choice, it is difficult to guess the correct answer or use a choice elimination strategy. 31

Math Grade 7 Illustrative Sample Item: Type I Task

32

Aligns to the Standards and Reflects Good Practice

• • • •

Grade 7 Sample Illustrative Item Key Features and Assessment Advances

The PARCC assessment will seek to preserve the focus of the Standards by thoroughly exploring the major work of the grade. In this case, a multi-point problem is devoted to a single standard about proportional relationships, which are a major focus in grades 6 and 7. Unlike traditional multiple choice, it is difficult to guess the correct answer or use a choice elimination strategy. Variants of the task could probe understanding of unit rates and representations of proportional relationships by showing different scales on the two graphs, and/or by presenting the data in tables C and D with the ordered pairs not equally spaced in time.

33

Additional Sample Items in ELA/Literacy

34

Item Types That Showcase Students

’

Command of Evidence with Complex Texts

• • • Evidence-Based Selected Response (EBSR)—Combines a traditional selected response question with a second selected-response question that asks students to show evidence from the text that supports the answer they provided to the first question. Underscores the importance of Reading Anchor Standard 1 for implementation of the CCSS.

Technology-Enhanced Constructed Response (TECR)—Uses technology to capture student comprehension of texts in authentic ways that have been difficult to score by machine for large scale assessments (e.g., drag and drop, cut and paste, shade text, move items to show relationships). Range of Prose Constructed Responses (PCR)—Elicits evidence that students have understood a text or texts they have read and can communicate that understanding well both in terms of written expression and knowledge of language and conventions. There are four of these items of varying types on each annual performance-based assessment.

35

Evidence-Based Selected Response Example Part A: In paragraph 4 of the text Who Was Marco Polo? , the author states that an exaggeration became known as a “marco polo.” What is the meaning of the word exaggeration?

A. a mistake or misunderstanding B. a long journey C. an exciting individual D. an untruth or overstatement*

Part B: Which words from paragraph 3 and 4 best help the reader determine the meaning of exaggeration?

A. didn’t know much E. another phrase B. strange lands F. wild stories* C. too incredible* G. make the crowds laugh D. the nickname 36

Technology-Enhanced Item Example: Highlighting Evidence

•

From the The Great Fire: The passage provides

several reasons that Chicago was a city “ready to burn.” The passage also gives reasons that Chicago burned down when it did, rather than at an earlier or later time. In the passage, highlight two sentences that provide reasons that the city burned in October 1871 rather than earlier or later. 37

Technology-Enhanced Item Example: Drag and Drop

The passage gives reasons that explain why Marco Polo may have been truthful in his book and also gives reasons that explain why he may have made up his stories about China. The headings in the chart below list these two different ideas from the text. Complete each row of the chart by dragging and dropping facts and details from the text to support each idea. The first row has been done for you.

Evidence from the text that Marco Polo may have told the truth in his book Evidence from the text that Marco Polo may not have told the truth in his book

But a list of his belongings around the time of death suggests that he did leave behind one of Kublai Khan’s gold tablets. He said the Chinese city of Hangchow had twelve thousand bridges, but it had far fewer. 38

Understanding the Narrative Writing Task (Grade 6 Example)

• Students read one or two brief texts and answer a few questions to help clarify their understanding of the text(s).

• Students then write either a narrative story or a narrative description (e.g., writing a historical account of important figures; detailing a scientific process; describing an account of events, scenes, or objects). 39

Texts Worth Reading?

• Range: Example of assessing literature and helping to satisfy the 55%-45% split of informational text to literature at the 6-8 grade band. • Quality: Julie of the Wolves was a winner of the Newbery Medal in 1973. This text about a young Eskimo girl surviving on her own in the tundra by communicating with wolves offers a story rich with characterization and imagery that will appeal to a diverse student population.

• Complexity: Quantitatively and qualitatively, the passages have been validated and deemed suitable for use at grade 6.

40

ELA/Literacy: Grade 6 Sample Item

41

ELA/Literacy: Grade 6 Sample Item

42

Questions Worth Answering?

Grade 6 Prose Constructed Response from Narrative Writing Task

In the passage, the author developed a strong character named Miyax. Think about Miyax and the details the author used to create that character. The passage ends with Miyax waiting for the black wolf to look at her.

Write an original story to continue where the passage ended. In your story, be sure to use what you have learned about the character Miyax as you tell what happens to her next.

43

Additional Sample Items in mathematics

44

Math High School Illustrative Sample Item: Type I Task

   Item has two possible solutions Students have to recognize the nature of the equation to know how to solve Technology prevents guessing and working backward 45

Aligns to the Standards and Reflects Good Practice

High School Illustrative Item Key Features and Assessment Advances

The given equation is quadratic equation with two solutions. The task does not clue the student that the equation is quadratic or that it has two solutions; students must recognize the nature of the equation from its structure. Notice that the terms 6x – 4 and 3x – 2 differ only by an overall factor of two. So the given equation has the structure 𝑄 2 = 2𝑄 where Q is 3x – 2. The equation Q 2 - 2Q is easily solved by factoring as Q(Q-2) = 0, hence Q = 0 or Q = 2. Remembering that Q is 3x – 2, we have 3𝑥 − 2 = 0 or 3𝑥 − 2 = 2 .

These two equations yield the solutions 2 3 and 4 3 .

Unlike traditional multiple-choice tests, the technology in this task prevents guessing and working backwards. The format somewhat resembles the Japanese University Entrance Examinations format (see innovations in ITN Appendix F). A further enhancement is that the item format does not immediately indicate the number of solutions.

46

Math High School Illustrative Sample Item: Type III Task

47

http://www.ccsstoolbox.com/parcc/PARCCPrototype_main.html

MP 1, 2, 3; F-BF.1; F-LE.2,3

Math High School Illustrative Sample Item: Type III Task

48

http://www.ccsstoolbox.com/parcc/PARCCPrototype_main.html

MP 1, 2, 3; F-BF.1; F-LE.2,3

Math High School Illustrative Sample Item: Type III Task

49

http://www.ccsstoolbox.com/parcc/PARCCPrototype_main.html

MP 1, 2, 3; F-BF.1; F-LE.2,3

www.PARCConline.org

50