Georgia Alternate Assessment

2013 GAA Mid-Fall Webinar Training

Recording:

https://sas.elluminate.com/mr.jnlp?suid=M.FBDDF1C2CEACC

C9EB70053458371E8&sid=2012003 1

Topics for Today

• This slide presentation is designed to provide GAA teachers and portfolio reviewers information regarding the 2013-2014 administration. – Key Dates reminder – Brief walk-through of 2013-2014 Examiner’s Manual – Review of nonscorable codes and examples of the most common nonscorables – Strategies for ensuring Scorable Entries 2

Key Dates for the GAA

• Administration Window: September 3 rd , 2013 - March 28 th , 2014 • Systems return Fall Retest materials to Questar for Scoring: October 25 th – November 8 th , 2013 • Systems receive GAA Fall Retest Score Reports: January 13 th – 16 th , 2014 • Materials - Pre-ID Labels, Main Administration SDIF’s (Purple K, 3-8 and Orange HS), and Return Kits arrive: February 3 rd - 4 th , 2014 • Systems return materials to Questar for Scoring: March 14 th - 28 th , 2014 • Systems receive GAA Score Reports: June 2 nd – 10 th , 2014 3

GAA Examiner’s Manual

• • • • • • • • • • Quick-Glance Index (p. iii) Key dates (p. 1) High School Retest (pp. 3-4) Participation Guidelines (pp. 8-9) Portfolio Components (p. 10) Alignment (pp. 11, 13,66,166) Blueprint (pp. 185-194) Standards (pp. 197-328) Sample Entries (pp. 69-164

)

Tips and Tools (pp. 43-53) 4

GAA Resources

The following materials are available from the GAA web page: http://www.doe.k12.ga.us/Curriculum-Instruction-and Assessment/Assessment/Pages/GAA.aspx

―Examiner’s Manual ―School and System Test Coordinator’s Manual ―Score Interpretation Guide ―Forms ―Blueprint ―Participation Criteria http://www.gadoe.org/Curriculum-Instruction-and Assessment/Assessment/Pages/GAA-Presentations.aspx

— PowerPoints on previously presented topics 5

Frequency of Nonscorables by Number and Percent

2012-2013 NS Code Frequency Percent of NS Percent of all entries 0.37% ME 235 3.03% ES NA IE IT OG IS Total 126 6,113 1,101 77 27 65 1.63% 78.94% 14.22% 0.99% 0.35% 0.84% 7,744 100.0% 7,744 of 64,102 entries received NS codes 0.20% 9.54% 1.72% 0.12% 0.04% 0.10% 12.08%

6

Most Common Nonscorables

•

The vast majority of nonscorables are assigned in the Not Aligned (NA) category

– ≈79% of all nonscorables were the result of alignment issues.

– The most common NA subcodes show that one to three tasks do not align to standard and element.

• Usually only one of the 4 tasks does not align.

7

Most Common Nonscorables

•

The Insufficient Evidence (IE) category had the next highest frequency of nonscorables

– ≈14% of all nonscorables were the result of insufficient evidence.

– Most common subcode–evidence submitted for only one collection period.

• Possibly attributable to transfer students. – Next highest frequency demonstrates problems with redocumentation and/or lack of documentation regarding correctness of student response.

8

Understanding Nonscorable Conditions

• • Across all grades and content areas, the vast majority of students met or exceeded expectations as demonstrated by their Performance Level Indicator. A better understanding of the conditions that lead to nonscorable entries and strategies for portfolio review can ensure that this trend continues.

9

Nonscorable Categories

• • • • • • •

Missing Entry (ME) Off Grade (OG) Ineligible Standard (IS) Entry Sheet Errors (ES) Insufficient Time (IT) Not Aligned (NA) Insufficient Evidence (IE)

10

Nonscorable Codes and Conditions

•

Missing Entry (ME)

Entry is missing from the portfolio – The entire entry is missing from the binder.

•

Off Grade (OG)

Standard is not at student’s grade level – The Entry Sheet reflects a standard that is not at the student’s grade level.

– The student work submitted addresses a standard for a grade other than that in which the student is enrolled (FTE).

11

Nonscorable Codes and Conditions

• •

Ineligible Standard (IS)

Standard addressed is not eligible for assessment – The standard assessed for the entry is not on the GAA Blueprint of required and eligible standards.

Required standard has not been addressed (applies to ELA and Mathematics and all content areas for High School portfolios) – Of the two entries for ELA and Math, Entry 1 must address a specified required standard – for example, ELA Entry 1 is always Language or a Reading domain.

– If this required standard was not assessed for the content area entries, one of the entries receives this code.

12

Nonscorable Codes and Conditions

•

Ineligible Standard (IS)

Same standard is addressed in previous entry – For content areas requiring more than one entry, each entry must address a unique, eligible standard from the GAA Blueprint.

• • Grades K, 3-8–ELA and Math require 2 entries.

For HS–all four content areas require 2 entries.

– Should both entries address the same standard or another standard from within the same strand/domain, at least one entry will be nonscorable. * HS math allows for the same strand to be selected for both math entries, as long as there is a correct course combination. e.g., Math I Algebra and Math II Algebra (the standards and elements will differ) 13

Nonscorable Codes and Conditions

•

Entry Sheet Errors (ES)

Entry Sheet is missing from portfolio – Without an Entry Sheet to provide information regarding the standard and element being assessed, the entry cannot be scored.

– The Entry Sheet is an important required form that serves as a “Table of Contents” for the entry.

14

Nonscorable Codes and Conditions

• •

Entry Sheet Errors (ES)

Standard number and description are missing or incorrect – It is not possible to determine the standard being assessed. – If either the number or the description is present and correct, the entry can be scored.

Element letter and description are missing or incorrect – It is not possible to determine the element being assessed. – If either the letter or description is present and correct, the entry can be scored.

15

Nonscorable Codes and Conditions

• •

Insufficient Time (IT)

Dates on evidence do not reflect two distinct collection periods – Evidence for Collection Period 2 is dated before the last date in Collection Period 1. – One collection period must be completed before the next begins.

There are fewer than 14 calendar days from date on Primary Evidence for CP1 to date on Primary Evidence for CP2 – There are two distinct collection periods, but there are not at least 14 days from primary to primary (required) 16

Nonscorable Codes and Conditions

•

Insufficient Time (IT)

Date on evidence indicates that student work was collected before the testing window opened – For the 2013-2014 administration, a piece of evidence dated before 9/3/2013 or after 3/28/14 is not eligible for inclusion in the portfolio.

17

Nonscorable Codes and Conditions

•

Insufficient Evidence (IE)

Evidence has been submitted for only ONE Collection Period – The student was enrolled in a Georgia school after January 1, 2014 and there was only time to work on evidence for the first collection period.

• This should apply only to students new to Georgia schools, not to students transferring from one GA school/system to another.

18

Nonscorable Codes and Conditions

•

Insufficient Evidence (IE)

Primary Evidence does not meet evidence requirements or is missing from the portfolio – Primary evidence submitted does not meet the evidence requirements for Primary Evidence specified in the manual and in training.

• Examples: Only one captioned photo of student participating in task; Insufficient documentation, etc. – There are two pieces of secondary type evidence (Observation/ Interview/ Data Form) but no primary type.

– There is only one piece of evidence submitted for the collection period.

19

Nonscorable Codes and Conditions

•

Insufficient Evidence (IE)

Secondary Evidence does not meet evidence requirements or is missing from portfolio – Secondary Evidence submitted does not meet the evidence requirements for Secondary Evidence specified in the manual and in training.

• Observation or Interview Sheet cannot be scored because of the absence of proper documentation.

• A data sheet is submitted with fewer than 3 dates on which evidence of student performance was collected.

20

Nonscorable Codes and Conditions

•

Insufficient Evidence (IE)

Secondary Evidence is a redocumentation of Primary Evidence – Secondary Evidence must represent a separate distinct event, different materials used, and/or a different activity.

• A task that takes multiple days to complete is still considered a single task and will not be scored as a separate piece of evidence.

• A work sample and an observation of the student completing that work sample would be considered a redocumentation.

21

Nonscorable Codes and Conditions

•

Insufficient Evidence (IE)

Cannot determine the correctness of student response on one or more pieces of evidence – There is no score or answer key and it is not possible for a reader to determine if the student responses are correct.

• Example: The student is measuring objects in the classroom, but the work is not graded and no answer key was provided. • Example: The annotation states that the student completed the task, but the work is not graded and there is no statement regarding the correctness.

22

•

Nonscorable Codes and Conditions

•

Not Aligned (NA)

Tasks are not aligned to standard and/or indicator – There must be a connection between the task and both the standard and indicator.

One or both tasks for either collection period do not align to the standard and element – Should any one piece of evidence not align to the standard and/or element being assessed, leaving fewer than 4 pieces of scorable evidence, the entry is nonscorable.

– A task that documents instruction, rather than assessment, is considered not aligned.

23

Nonscorable Codes and Conditions

•

Not Aligned (NA)

No evidence of the Characteristic of Science indicated – The tasks are aligned to the standard and element, but there is no evidence of the co-requisite Characteristic of Science. • • Example: If the student “uses safety techniques” during a science investigation, it must be clearly documented within the evidence.

Characteristic of Science must be evident/documented on 1 of the 4 assessment tasks submitted for the entry.

– The Characteristic of Science was not documented on the Entry Sheet.

24

Common Concerns Regarding Nonscorable Entries

•

Two recurring concerns have been voiced regarding assignment of nonscorables:

1. Scoring is not consistent because one student received a nonscorable code when he/she was assessed via the same standard as another student, whose entry was scored, 2. Tasks were considered aligned and scorable in the past that were not scored this year.

25

Common Concerns Regarding Nonscorable Entries

1. How is it possible for a teacher to submit the same activity for 2 (or more) students for which one student receives a score and the other a nonscorable code? In this scenario, two of the main issues are: a) an alignment problem: the connection between the skill being assessed and the standard/indicator is tenuous and unclear.

• If an explanation is needed to determine how or if a task is related to the standard/element, then it is probably questionable and should not be used, or b) documentation is not the same; what is incomplete or unclear in one portfolio may be stated clearly and completely in another.

26

Common Concerns Regarding Nonscorable Entries

2. How is it possible that the same task that was scored last year received a nonscorable code this year? In this scenario, the issues are: a) This was a transition year in which the ELA standards for all • grades as well as the Math standards in grades K and 3-8 changed from GPS to CCGPS. Because some of the standards changed and became more specific in their intent, some of the previous assessment tasks that may have once aligned to a GPS standard may no longer have aligned to a CCGPS standard.

• Retrofitting tasks was not recommended and was cautioned against in the 2012-2013 Fall Training.

27

Common Concerns Regarding Nonscorable Entries

b) Documentation was not included to connect the task to the intent of the standard and indicator.

– e.g., baking brownies only aligns to a standard/element that assesses knowledge of chemical reactions if it is connected back to chemical reactions through a question or response; baking brownies, in and of itself, does not constitute a science investigation c) A task or materials used for a standard at one grade level does not work for another.

d) The task was scored in error; the reader made assumptions/connections in an effort to make a nonscorable task or piece of evidence scorable.

28

• • • •

Common Concerns Regarding Nonscorable Entries

If the evidence is not clear, either in the way it is presented or because the annotation is missing, incomplete, or too generic, then scoring may not be possible.

Readers make every effort to score all evidence in the interest of the students.

HOWEVER

If a reader finds an issue with an entry, it is referred to a Team Leader, Scoring Director, or GaDOE representative to determine whether it can be scored. Only a Team Leader or Scoring Director can assign a nonscorable code.

29

Reworking the Nonscorable Task

• • If an entry is deemed to be Not Aligned, even if it has been scored for another student, it should be considered nonscorable as is. • • Before submitting any of the tasks for that entry again, they should all be carefully reviewed and the necessary changes to the task or additions to the documentation should be made.

Foremost consideration must be given to the intent of the standard.

Each assessment task must align to the standard and element (when applicable) independently of the other three tasks.

30

Understanding Alignment

Examples of Nonscorable Entries

31

Alignment to State-Mandated Content Standards

• • Alignment is to the grade level content standard.

– Assessment tasks may be at a more simplified level but must still connect to the grade-level standard.

Alignment of all 4 assessment tasks must be to the “Big Idea” (intent/essence) of the standard. – The standards-based skill being addressed by the assessment task must still connect back to the intent of the standard and element/indicator and be taught in the context of the standard.

32

Alignment–Identifying the Skills

• • • Some of the current state standards are broader and encompass more skills within a standard, while others are very specific.

– There can be more than one “Big Idea” and a number of standards-based skills within the same standard.

It is appropriate for many of our students to choose one skill around which to design the assessment tasks.

It is critical that all 4 assessment tasks submitted for that standard demonstrate a connection to the same standards based skill.

– – The same skill(s) must be demonstrated across both collection periods.

Additional skills can be added in the second collection period.

33

Examples of Nonscorable Entries

• • • The following slides include examples of portfolio entries determined to be nonscorable by a Rangefinding Committee of Georgia educators.

They are provided to illustrate some of the more common issues that result in nonscorable entries.

Understanding the nonscorable codes and conditions serves to provide teachers with more information as to the situations resulting in nonscorable entries and how to avoid them.

34

NA-A– Not Aligned: None of the four assessment tasks submitted align to the standard and element.

MM1D1 a. Apply the addition and multiplication principles of counting 35

It is important to remember that although Collection Period 1 evidence represents the student’s baseline performance of knowledge and skills related to the standard and element, it does not mean that it should be the student’s first and only exposure to the vocabulary, concepts, or materials that provide access to the standard and element.

This task exposes the student to vocabulary that may be useful as he learns more about the DAP strand–as such, this is an instructional task that should precede the assessment.

This is a generic DAP activity. It does not address the specific standard and element chosen for assessment.

This task does not align.

36

Although these tasks appear to relate to the mathematics strand of Data Analysis and Probability, there is nothing that connects them to the intent of either the standard (event outcomes) or the element .

These tasks do not align.

37

NA– Not Aligned: One assessment task in each collection period does not align to the standard.

MCC.6.G.4 Represent 3-D figures using nets made of rectangles and triangles, and use the nets to find the surface area of these figures.

38

39

NA-Not Aligned: One or both pieces of evidence submitted for Collection Period 2 do not align.

ELACC.4.W.2 b. Develop the topic with facts, definitions, details, quotations, to other examples related to the topic.

40

In this evidence submitted for Collection Period 2, the student is answering multiple choice comprehension questions about community trips the class went on. Circling a response is not considered “writing” as is necessary to produce a written product.

41

NA-Not Aligned: One or both pieces of evidence submitted for Collection Period 1 do not align.

S3L1 a. Differentiate between habitats of Georgia (mountains, marsh/swamp, coast, Piedmont, Atlantic Ocean) and the organisms that live there. 42

What is the student’s task as it aligns to the standard and element?

The student is following prompts, but there is no task for which she can be assessed on.

What is the question being asked?

The student does not have to discern between two or more choices, but rather is responding to the directions she is given.

This is an excellent exercise for instruction on the subject matter and has potential for alignment, but without the appropriate documentation aligning the task to the intent of the standard,

This task does not align.

43

The student must be given the opportunity to show what they know and have learned from instruction, but the activity/process of instruction should not be submitted as evidence for the GAA.

The evidence found in Collection Period 2 was aligned using very similar materials, because the documentation within the evidence explained the student’s task as it relates to the standard and element, including questions such as: “What animals live in the Georgia Mountains habitat?” 44

NA-Not Aligned: One or both pieces of evidence submitted for Collection Period 1 do not align.

S3E1 a. Explain the difference between a rock and a mineral.

S3E1a. Explain the difference between a rock and a mineral.

45

What is the student’s task as it aligns to the standard and element?

The student is assisted in making a peanut butter and jelly sandwich as an example of sedimentary rock

.

What is the criteria on which the student will be evaluated?

She is not graded on how well she followed the steps to create the “rock formation.” She is not asked any question(s) nor is she asked to make any kind of differentiated response (e.g., which layer represents the oldest part of the “rock”–bottom or top?). Had the student labeled the layers after making the sandwich or answered a question that connects the sandwich back to the rock formation, the task could have aligned.

This task does not align.

46

What is the student’s task as it aligns to the standard and element?

The student is investigating different kinds of rocks and soil.

What is the criteria on which the student will be evaluated?

In the pictures, the student is shown “playing with” various rocks (not shown, described, or identified), soil, and sand. She received a grade of “100% for participation,” but she is not presented with a task that demonstrates her knowledge about rocks.

This task exposes the student to vocabulary, materials, and concepts, but it is not an assessment task as it does not require the student to answer any questions, make any choices, follow any steps, or demonstrate any kind of differentiated response.

This task does not align.

47

NA-Not Aligned: One or both pieces of evidence submitted for CP 2 do not align.

In order to be scorable, all four pieces of evidence must align to the standard and element.

SS8G1a: Locate Georgia in relation to region, continent, and hemisphere.

48

A Prerequisite Skill is one that is essential to the acquisition of the standard and element. – allows the student to be exposed to and assessed at a level that is appropriate – must still focus on the intent of the standard and element being assessed Knowing the state bird or state crop of Georgia does NOT demonstrate anything about the student’s ability to locate Georgia in relation to the region, nation, continent, or hemisphere.

This task does not align.

49

Strategies for Ensuring Scorable Entries

Alignment Checks Evidence Requirements Documentation Portfolio Review

50

Alignment Checks

• • Reading – The student MUST read, be read to, or listen to some form of text (story, book, poem, article, advertisement, etc.).

– Text can be adapted to be accessible to the student; may include words, pictures, symbols, or any combination thereof.

Speaking/Listening – When the task evaluates “listening” specifically, measurable criteria related to the standard & indicator must be assessed.

– When the task evaluates “speaking,” the intent of the indicator must be assessed and documented.

51

•

Alignment Checks

Writing – The student must produce some form of “written product” that connects the task to the intent of the specific standard and element. Writing should be the “output” of student knowledge.

– A written product can: • be dictated and scribed, • use pictures or symbols that are chosen and then glued, • • use prepared lines of text that are ordered, fill in the blank to complete a sentence, etc. – Circling or bubbling in multiple choice answers on a worksheet or tracing letters or words that were not student generated does not meet these requirements.

52

•

Alignment Checks

Prerequisite Skills – A prerequisite skill is one that is essential to the acquisition of the standard and the indicator.

– It must address the intent indicator being assessed.

• of the standard and The skill is a prerequisite if working on it could eventually lead to the skill targeted by the intent of the standard or indicator.

• The skill is not a prerequisite if it’s purpose is to expose the student to generic concepts, vocabulary, or background information or skills not addressed by the standard or indicator.

53

Evidence Requirements

• • • Each type of evidence, whether Primary or Secondary, has specific requirements that must be met in order to be scorable.

Explanations and samples of each type of evidence are included in the 2013-2014 GAA Examiner’s Manual on pages

15-31

.

Be certain to review the samples and evidence requirements with both new and returning examiners.

54

Evidence Requirements

• • Recommendations as to when and how to use each type of evidence can be found on pages

16 and 17

of the manual. Use the GAA Evidence Checklist in order to familiarize test examiners and portfolio reviewers with the conditions that must be met for each type (found on pages

53-56

of the manual).

55

Documentation

• Complete and thorough documentation of evidence is critical! – Incomplete, ineffective, or generic documentation can result in lower scores or in the entry being nonscorable.

– Fall 2012 Training Presentations “GAA Documentation and a Review of Scoring August 21, 2012” provides specific guidance and examples of both effective and ineffective documentation: http://www.gadoe.org/Curriculum-Instruction-and Assessment/Assessment/Pages/GAA-Presentations.aspx

Nonscorable entries due to insufficient evidence can be avoided by following the evidence and documentation requirements.

56

The Importance of Portfolio Review

• Use the Checklist for Teachers and Portfolio Reviewers to ensure that all evidence and documentation requirements have been met (page

57

of the GAA Manual).

Each of the 4 tasks submitted for the entry must be reviewed for alignment.

– Tasks must be academic – All 4 tasks must demonstrate connection to the standard and element – Instruction precedes assessment; task must provide evidence of student’s assessed performance after instruction.

57

The Importance of Portfolio Review

When reviewing evidence documentation, the teacher and portfolio reviewer must ask him or herself the following questions: – – – – – What, specifically, was the student asked to do as it aligns to the standard and indicator?

What were the actual questions/actions asked of the student?

What were the student’s answers? How did he/she respond?

Were the answers/responses correct? Has evaluation of student performance by the teacher been clearly documented?

What was the type and frequency of prompting required for the student to successfully complete the task?

58

The Importance of Portfolio Review

• • • • Task descriptions, though brief, must be clear and written with sufficient information to tie the task to the intent of the standard and indicator.

Observation and Interview Forms must be written to describe the specific responses and level of performance for each individual student.

If documenting a group activity, be specific about what was asked of the individual student, how and in what manner he/she responded, and whether his/her response was correct.

Annotation that is too generic or non-specific could result in the entry being nonscorable.

59

Contact Information

Questions About Test Administration



Call:

GaDOE Assessment Administration Division Toll free (800) 634-4106  Contact: Deborah Houston, Assessment Specialist (404) 657-0251 

Email:

[email protected]

60

Contact Information

For information about access to the state-mandated content standards for students with significant cognitive disabilities  Contact: Kayse Harshaw Division for Special Education Services 

Call:



E-Mail:

(404) 463-5281 [email protected]

61

Contact Information



Call:

Questions About Materials, Distribution, or Collection

Questar’s GAA Customer Service Toll free (866) 997-0698 

Email:

Questar’s GAA Customer Service [email protected]

62