Transcript Ideas and Solutions SLO Data Issues

Everything an Administrator
Needs to Know About
Student Learning Objectives
REVIEWING AND EVALUATING
STUDENT LEARNING OBJECTIVES
CURRICULUM, INSTRUCTION,
TECHNOLOGY
OCTOBER 12, 2012
Created by Jane Boyd, Program Coordinator, Nassau BOCES Student Support Services
Components of a SLO: 5 District Decisions
1.
2.
3.
4.
5.
Assess and identify priorities and academic
needs.
Identify who will have State-provided growth
measures and who must have SLOs as
“comparable growth measures.”
Determine District rules for how specific SLOs
will get set.
Establish expectations for scoring SLOs and for
determining teacher ratings for the growth
component.
Determine District-wide processes for setting,
reviewing, and assessing SLOs in schools.
From NY DOE engageny.org Introduction to SLO Deck
Translating SLO’s to the HEDI Scale
This table illustrates the relationship between the Growth or Comparable
Measure component to the Overall Composite Score.
Components of a SLO: NYSED SLO Framework
All SLOs MUST include the following eight basic components:
Student Population
Which students are being addressed?
Learning Content
What is being taught? CCSS/National/State standards? Will this
goal apply to all standards applicable to a course or just to specific
priority standards?
Interval of
Instructional Time
What is the instructional period covered (if not a year, rationale for
semester/quarter/etc)?
Evidence
What assessment(s) or student work product(s) will be used
to measure this goal?
Baseline
What is the starting level of learning for students covered by this
SLO?
Target(s)
What is the expected outcome (target) by the end of the
instructional period?
HEDI Criteria
How will evaluators determine what range of student performance
“meets” the goal (effective) versus “well-below” (ineffective) , “below”
(developing), and “well-above” (highly effective)?
Rationale
Why choose this learning content, evidence and target?
From NY DOE engageny.org Introduction to SLO Deck
4
Student Learning Objective: SLO







Definition:
Academic Goal
Set at the start of the course
Represents the most important learning
Specific and measurable
Based on prior student learning data
Aligned to standards
Aligned to school or district priorities
Guidance on NYS Dist. Goal-Setting Process: SLO. March 2012
Student Learning Objective: SLO
 Today!
 Where you are and where you are going
 What are the criteria for each aspect of the SLO
 How can you manage this process
 What are your concerns
District rules for
how specific SLOs will get set
 Will the District require the use of existing, common




District-wide assessments for a specific grade/subject?
Are there grades/subjects where the District wants to
prioritize building or buying additional District-wide
assessments?
Are there groups of teachers where group or team
results based on state assessments are appropriate?
Are there grades or subjects where the District can identify
priority learning standards or other District-wide
guidance for schools and teachers?
What will the District require for any remaining teachers
not covered by the above?
District Expectations for Scoring
 For each group of grade/subject teachers with
similar growth goal approaches, Districts must
determine and communicate the District’s
expectations for student learning growth relative to
baselines and specify how teachers will be awarded
HEDI ratings and earn from 0-20 points based on
the results obtained, consistent with State
Regulations and guidance.
District-wide Processes
 What tools will principals use to assess the rigor of teacher-
determined targets?
 Will Districts review all goals or spot check goals and targets
set by schools and teachers in any way to ensure rigor and
comparability?
 How will procedures to monitor progress of students on
SLO targets and final results reviews be handled?
 Districts may want to consider other teacher evaluation
procedures like evaluator/teacher conferences, and procedures
around classroom observations as well as District processes for
data-driven inquiry meetings to be sure that sufficient time and
coordination is provided.
Room for Decision Making
 D23, p.40. In some cases, district and BOCES leaders may develop
entire SLOs (or just targets and/or HEDI expectations) for all
teachers or certain subject area teachers, and in other
instances, district and BOCES leaders may leave the
development of the SLO (or just targets and/or HEDI
expectations) to principals and/or principals and teachers.
 In some cases, the district/BOCES may decide that all SLOs must use
the same target expectations based on growth progress norms
while in other instances the district/BOCES may allow principals
and/or teachers to propose a target based on the starting
level of the students covered within the SLO. Decisions as to
whether all teachers of the same subject will use a common SLO (or
whether principals may even dictate this) are allowable; however, these
are decisions first to be made by district/BOCES leaders.
Student Learning Objective: SLO
 When setting SLO’s by course or grade:
 Learning content, Interval of Instructional Time, Evidence,
Targets, HEDI Criteria are all the same.
 Population, Baseline data, Differentiation of targets based
upon baseline data, and rationale will be unique to the
roster.
Student Learning Objective: SLO
 Different district approaches for setting SLO’s:
1.
Director’s and Chairpersons create the SLO’s and
assessments with Central Office guidance and
disseminate them to faculty
2. Director’s and Chairpersons train and over see the
creation of SLO’s and assessments within departments or
across grade levels with Central Office guidance
3. Teacher’s create SLO’s using district decisions and SLO’s
are evaluated by Supervisors – principals, chairpersons,
etc.
 In all situations, there comes a point where a
review needs to take place.
Student Learning Objective: SLO
 Director’s and Chairpersons create the SLO’s and
assessments and disseminate them to faculty


May select Learning Content – standards
Make decisions regarding evidence:
 Create
baseline assessments or make decisions regarding data
to be used as the baseline
 Create the summative assessments

Apply district decisions for target setting and HEDI scores
 Teachers can:
 Teacher’s
describe their population
 Analyze the data and record that analysis in the “Baseline”
 Add any information to “Rationale”
Student Learning Objective: SLO
 Director’s and Chairpersons train and oversee the
creation of SLO’s and assessments within
departments or across grade levels




Highly collaborative within the department or grade level
Director’s/Chairpersons facilitate as teachers contribute to
conversations regarding learning content for
courses/grades
Director’s/Chairpersons create a format for the creation of
assessments or use of baseline data
Data is analyzed by teachers and discussed with
Director’s/Chairpersons to create appropriate targets.
Student Learning Objective: SLO
 Teacher’s create SLO’s using district decisions and
SLO’s are evaluated by Supervisors – principals,
chairpersons, etc.



Teachers receive training and create SLO’s for their courses
using district decisions regarding any learning content,
targets and the HEDI scoring.
Appropriate assessments are a collaborative effort and
finalized by a supervisor
Data is gathered and the SLO is written with the idea of
recommending appropriate targets, etc. with the given
district guidelines.
Student Learning Objective: SLO
 Who needs an SLO?
 Right now, it is anyone that is not receiving a state-provided
growth measure which is 4-8 ELA and Math. See “Purple
Memo” and “Assessment Options for SLO’s”
 More than one SLO?
 Maybe
 How do you know what you need?
 50% or more of your students must be covered under your
SLO’s. That might mean you need two or maybe even three.
 Begin with the courses taught that have the largest number of
students, combining sections with common assessments.*
*p10 & 12 SLO Guidance March
Minimum Group Size
 Who needs an SLO? Another situation  When the minimum number of students is not met.
 Other situations?
 Part-time teachers? §80-1.1 – Does not discuss part time
classroom teaching service.
 Long term subs?
 Librarians – Speech?
 Second or third SLO just in case my population changes
Who Needs an SLO?
 D43. How are SLOs for Library/Media Specialists established
if these teachers do not have regular classes scheduled and
only schedule on-demand/teacher-requested basis for
specific topics and projects?
 Districts/BOCES will need to determine their specific rules around
which courses must have SLOs when contact time varies following
the State’s rules and the general principle of including the courses with
the most students first and making practical judgments about how
to consider different course meeting schedules like those in this
example.
Classroom Teaching Service
Teacher of Record
New Guidance - B12. When must school librarians be evaluated
under Education Law § 3012-c?
 "Classroom teacher" is defined as a teacher in the classroom teaching
service who is a teacher of record.
 Librarians who are certified as a library media specialist or school
media specialist (library) are teachers in the classroom teaching
service. For the 2012-2013 school year, teacher of record is
defined as a teacher who is primarily and directly responsible for a
student’s learning activities that are aligned to the performance
measures of a course, consistent with guidance.
 Therefore, a certified librarian who is not a teacher of record is
not a "classroom teacher“ and therefore would not need to be
evaluated under Education Law § 3012-c. However, if a certified
librarian is a teacher of record, he/she would be considered a
"classroom teacher" and therefore must be evaluated under Education
Law § 3012-c.
Is a Librarian a Classroom Teacher?
 Certified Library Media
Specialist or School
Media Specialist
 Teacher of Record
 Teachers in Classroom
Teaching Service
 Not a Teacher of Record
A teacher who is primarily and directly responsible for a
student’s learning activities that are aligned to the
performance measures of a course consistent with guidance
M2. What kinds of data will districts and BOCES need to collect in order
to determine who is the teacher of record for evaluation purposes?
Districts and BOCES will need to collect additional data elements to support
teacher of record determinations. (Teacher-Student Data Linkage)
 APPR and SLO’s
 Other evaluation
processes
Speech Teacher – Addendum August APPR
 B10. If a speech teacher (i.e.,
certified teacher of the speech
and hearing handicapped or
certified teacher of the speech
and language disabilities) is just
providing related services, is
he/she required to be evaluated
under Education Law §3012-c?
 No. If a certified speech and
language therapist is providing
related services only, as defined in
Education Law §4401 and section
200.1(qq) of the Commissioner's
Regulations, he/she would not be
considered a classroom teacher and
would therefore not be required to be
evaluated under Education Law
§3012-c.
 B11. If I am certified as a teacher
of the speech and hearing
handicapped and provide
instructional services, am I
required to be evaluated under
Education Law §3012-c?
 If a teacher who is certified as a
teacher of the speech and hearing
handicapped is assigned to provide
instructional services and
he/she is a teacher of record,
then he/she is considered a
classroom teacher and therefore
must be evaluated under Education
Law §3012-c.
Student Learning Objective: SLO
 Student Population: Students included in
the SLO






There is no minimum number of students
The population is set on BEDS day
Even if the class enrollment changes it is not
abandoned. D27
District set date for adding students
Addresses the growth of “each student” (A ROSTER
IS REQUIRED) but then it is the aggregate growth
of all the students that determines whether or not
the target is met.
Other student information to describe the
population may be added (SLO Development Guide)
P37, Section D. APPR Guidance April 2012
Student Learning Objective: SLO
Population*
These are the students assigned to the course section(s) in this SLO - all students who are
assigned to the course section(s) must be included in the SLO. (Full class rosters of all
students must be provided for all included course sections.)
 Population examples





Health and Occupations Technology 11th Grade; 24 students,
including 2 with learning disabilities on attached roster
18 Kindergarten Students listed on the attached roster
23 8th grade Math AIS students from 3 sections of the course.
See roster
All ___ students enrolled in 2 sections of AP Calculus. See
attached roster
All ___ students in 10th Grade English. See attached rosters
which includes course sections, student names, and ID numbers

11th and 12th grade students enrolled in Fitness for Life, see
attached roster which includes students with disabilities
Population
Student Learning Objective: SLO
 Learning Content: Content to be taught in the SLO
 Identify the source for the standards: Common Core, State,
National, College Board, Industry, AASL Standards, etc.
 Include any relevant Common Core Literacy/Math standards
with NYS standards (Literacy in History/SS or Science & Technical
subjects) – might include district decisions

Name the exact standards and performance indicators


As appropriate, choose the “most important learning” or a “subset
of power standards.” What do you traditionally assess and does it
give you a picture of the year?
Align to district or school goals and vertically align
Learning
Content*
What is being taught over the instructional period covered? Standards?
Will this goal apply to all standards applicable to a course or just to
specific priority standards?
SLO Development Checklist OCMBOCES
*SLO Template
Student Learning Objective: SLO
 Where can I find my standards?
 http://engageny.org/resource/common-core-state



standards/ for Common Core
http://www.p12.nysed.gov/ciai/cores.html for Arts,
MST, Social Studies, Science, Health, PE, Family &
Consumer Science, etc.
http://www.p12.nysed.gov/cte/cdlearn for Career
Development and Occupational Studies
http://www.nbea.org/newsite/curriculum/standards
/marketing.html for Business and Marketing
Where else?
Learning Content
Student Learning Objective: SLO
 Interval of Instructional Time: Timeframe within
which the learning content is being taught



One academic year, a semester? a quarter?
Every other day? - Describe
Start and end date
SLO Development Checklist OCMBOCES, SLO Guidance March 2012, APPR Guidance April 2012
Interval of Instructional Time
Student Learning Objective: SLO
 Evidence: Assessments used to determine students’
level of learning

List the specific baseline assessments (pre-assessment) and
summative assessment(s) that will be used to provide baseline
and summative data for the SLO.
1.
2.
3.

If the course ends in a NYS assessment or a Regents exam, it
must be used as the summative assessment
List of 3rd party assessments and Regents equivalents - FLACS
District, Regional, or BOCES developed. Districts or BOCES
must verify comparability and rigor.
What could the other option have been?

(School-or BOCES-wide, group, or team results based
on State assessments)
New York SLO Development Guide
SLO Development Checklist OCMBOCES
Student Learning Objective: SLO
 What if the third option was chosen?
District,
Regional, or BOCES developed.
Align tightly to learning content
Include components of good assessment
practices
Scoring procedures – “vested interest”
BASELINES
VESTED INTEREST
• G4: Does vested interest rule apply to pre-tests
given to establish a baseline for a SLO?
• Answer: To the extent practicable, districts or
BOCES should ensure that any assessments or
measures, including those used for performancebased or performance task assessments that are
used to establish a baseline for student growth are
not disseminated to students before administration
and that teachers and principals do not have a
vested interest in the outcome of the assessments
they score.
CIT, Lupinskie Center, One Merrick Ave.
Westbury, NY 11590
BASELINES
VESTED INTEREST
• G4: Does vested interest rule apply to pre-tests given
to establish a baseline for a SLO?
• Answer continued: If it is impracticable to comply with
this requirement for pre-tests, such as in certain cases
when using a performance-based or performance task
assessment, the district or BOCES must have adequate
procedures in place to ensure that the security of such
assessments is not compromised (i.e., as with all SLOs the
principal and/or supervisor must ensure the rigor and
fairness of the targets and set the goals based on the
assessment that is used as the baseline and ensure that
such goals are adequately met based on summative
data).
CIT, Lupinskie Center, One Merrick Ave.
Westbury, NY 11590
Performance Tasks?
Performance Tasks
 How do SLO’s meet the requirement for
comparability? “Specify priority learning
standards in a grade or subject around which
assessments or performance tasks for
students will be constructed by District
Teams” (SLO Guidance March, Page 14)
 Includes a “standards-based” rubric (SLO Guidance
March, Page 30)
Student Learning Objective: SLO
OR – other
baseline options!
 Algebra 2/Trig for Physics
 Grade 8 NYS English Assessment for 9th grade
English
 Second Grade Math Summative for Third grade
BASELINES
D28: Question
• The SLO documents
state that SLOs “must
be specific and
measurable, based on
available prior student
learning data" – what is
meant by prior?
APPR Guidance June 2012, p37-38
Response
• Teachers are encouraged
to use a variety of student
historical academic data,
if available. This historical
academic data might
include prior course
results, or it might only
include the current
baseline data from the
course taken at the
beginning of the interval
of instruction.
CIT, Lupinskie Center, One Merrick Ave.
Westbury, NY 11590
BASELINES
D29: Question
• What should teachers
use as a pre-assessment
for establishing a
baseline when students
enter a sequence for
the very first time and
have little to no
background information
of a subject?
•
APPR Guidance June 2012, p38
Response
• . . .it will be important for teachers
to see what other courses they
can draw on to provide other
historical data. Any information
about what entering students
understand from other subject
areas based on academic history
in other courses, can give a
teacher a wealth of information
as to where the students may
struggle or thrive.
• The teacher may want to also
collect other baseline information
(e.g., collect a writing sample) in
order to more broadly assess the
student's skills that may impact
instruction for the year.
CIT, Lupinskie Center, One Merrick Ave.
Westbury, NY 11590
BASELINES
D47: Question
• Can SLOs for students
with disabilities have a
different target for
growth?
APPR Guidance June 2012, p43
Response
• The target for students in
any SLO may be
differentiated because of
the baseline (starting point
of learning) and historical
academic data.
• It is important to keep in
mind that targets for all
students, regardless of any
special education
classification, should be
differentiated because of
baseline data and not
because of any special
education classification.
CIT, Lupinskie Center, One Merrick Ave.
Westbury, NY 11590
SUMMATIVE – VESTED INTEREST
• This can be an issue.
• Teachers may not score the summative assessment.
• Talk of scoring guidelines for Regents exams to
become more rigorous
• Performance task issues
• Questions regarding teachers’ involvement in the
creation of summative assessments
CIT, Lupinskie Center, One Merrick Ave.
Westbury, NY 11590
Evidence
Student Learning Objective: SLO
 Baseline: Level of students’ knowledge and skill in
the targeted learning content at the beginning of
the interval of instructional time.
 Describes how students performed on the preassessment or other previous summative assessment
 Include any other data sources being considered –
that may impact your “target”
 Roster

Actual baseline scores for each
student are required
Baselines
On the Roster
Student Learning Objective: SLO
 Target:
Level of growth and skill that
students
are expected to achieve at the end
point of the interval of instructional time.
 Numeric
 Roster – growth goal and scores for each student
 Written in the form of a statement on the SLO

Different formats consider the individual students growth,
might connect baseline to summative, might be a “ n percent of
students will ____________.”
 Aligned to school/district expectations
 Includes all populations: “including special populations”
Important to Consider:
The Number of Students
 Look at Co-taught Living Environment SLO example
 How many students are in this class?
 Describe what you see in the baseline component.
What is the target statement?
 What HEDI rating is aligned to the target?
Important to Consider:
The Number of Students
 90% of all students = 90% of 19, which is aligned to




14 points on the HEDI scale
Do the math. 18/19 would be 95% or 18 points
17/19 students would be 89% or 13 points
16/19 students would be 84% or 8 HEDI points
What do you observe?
Differentiated Targets
 Take a second look at the Co-taught Regents Living
Environment SLO. Examine the Baseline and Target
section again. (Regionally developed pre-assessment)
 District decisions regarding Targets can be used while
differentiating for clusters of student results.



Raw score of 60-75 – achieve raw score of 80 or
higher
Raw score of 30-59 – achieve a raw score of 66 or
higher (mastery)
Raw score of 29 or below – achieve a raw score of 41
or higher (passing)
 Target statement: 90% of all students will meet or
exceed their goal.
Differentiated Targets
 What is the likelihood that students will hit
their targets when they are created in this
way?
 How is this more realistic?
Raw score of 60-75 – achieve raw score of 80 or
higher
 Raw score of 30-59 – achieve a raw score of 66 or
higher (mastery)
 Raw score of 29 or below – achieve a raw score of
41 or higher (passing)
 Target statement: 90% of all students will meet
or exceed their goal.

Differentiated Targets – Another Example
Student Population, Baseline, and Target(s)
Training SLO Target Approach 1: Set a common growth
target.
90% of students, including special populations, will grow
by 60 percentage points or more on their summative
assessment compared to their pre-test for the standards.
(e.g., Student E’s target is 60 more than 30, or 90.)
Student
Pre-Test Score Summative Target
Student A
10
70
Student B
20
80
Student C
5
65
Student D
0
60
Student E
30
90
Student F
10
70
www.engageNY.org
50
Student Population, Baseline, and Target(s)
Training SLO Target Approach 2: Set a growth to
mastery target.
85% of students, including special populations, will
grow to score 75% or higher on the summative
assessment for the selected standards.
Student
Student A
Pre-Test Score
10
Summative Target
75
Student B
Student C
Student D
20
5
0
75
75
75
Student E
Student F
30
10
75
75
www.engageNY.org
51
Student Population, Baseline, and Target(s)
Training SLO Target Approach 3: Set differentiated
growth targets by student.
85% of students, including special populations, will
meet or exceed their individualized target.
Student
Student A
Student B
Pre-Test Score
10
20
Summative Target
80
80
Student C
Student D
Student E
5
0
30
75
70
85
Student F
10
80
www.engageNY.org
52
Target
Student Learning Objective: SLO
 HEDI Criteria: Translates different levels of
student growth to the four categories, Highly
Effective, Effective, Developing, Ineffective.
 Allocates points clearly and objectively
 It must be mathematically possible for a given
teacher to obtain any point value in the scale
 Description




Highly effective – exceeds district/BOCES expectations
Effective – meets district/BOCES expectations
Developing – below district/BOCES expectations
Ineffective – well-below district/BOCES expectations
Guidance on NYS Dist. Goal-Setting Process: SLO. March 2012
More than one SLO?
HEDI Criteria
Student Learning Objective: SLO
 Rationale – Include reasoning behind decisions.
Particularly targets and HEDI alignment. That
rationale could include




Reference to the population and content
Baseline data recorded on the roster
Other data considered – historical, cohort
National
 Any information that will help the reader grasp the
rigor of the SLO
 Rate your SLO! – using the Annotated SLO Rubric
Form
Rationale
Resources
 APPR Guidance Document, June, 2012
 SLO Guidance Document, March, 2012
 From NY DOE engageny.org Introduction to SLO
 Assessment Options for SLO’s on engageny.org
 Nys-eval-plans-guidance-Purple memo.pdf
 Ira Schwartz Update on Accountability DATAG July
2012
 http://www.ocmboces.org/teacherpage.cfm?teacher
=1518
 SLO Connections to NYS Teaching Standards.pdf
 Annotated SLO Rubric Form
Special Ed, ESL, and More
 http://engageny.org/resource/student-learning-
objectives-webinar-series-ii-fall-2012/