Transcript Preconference Session MSTC 2014 Assessment Literacy

PRECONFERENCE SESSION
ASSESSMENT LITERACY
STANDARDS
February 18, 2014 MSTC Pre-conference Session
Session Purpose & Outcomes





Build the case Assessment Literacy is needed and
lacking in our field
Familiarize with NEW MI Assessment Literacy
Standards
Explore implications of assessment literacy for
variety of stakeholder groups
Highlight existing resources/tools
Gather your feedback regarding
desired/necessary future PD, resources, tools.
Session Overview





Set the Stage: Our context
Introduce Assessment Literacy Standards
Quick Primer: Quality Classroom Assessments
Explore Standards and Implications for Stakeholder
Groups
Current Resources and Future Possibilities
Setting the Stage…
OUR CONTEXT
Remember society’s
demands of its school:
ALL students lifelong learners
 Narrow achievement gap among students
 Universal graduation for students
 All students ready for colleges or
workplace training
 Raise achievement levels students

excerpted Rick Stiggins MDE- DAS 2013 Fall Conference presentation
So, how are we doing so far?




NAEP scores have flat-lined for decades
Drop out rates remain stable and high; some are
astronomical
USA’s place in international rankings stable
Excellent teachers and new teachers are leaving
the profession in unprecedented numbers
excerpted Rick Stiggins MDE- DAS 2013 Fall Conference presentation
Our Context: assessment in all of
this…





Public accountability for test scores supposed to
improve schools (local, state, national, international)
Linking federal funding to test scores supposed to
improve schools
Writing tougher standards & tests—raising the bar
supposed to improve schools
Competing for federal $ -- RtT, NCLB, -- supposed
to improve schools…
Evaluating teachers based on annual test scores is
supposed to improve schools
Adapted from Rick Stiggins MDE- DAS 2013 Fall Conference presentation
How is this possible?







No assessment training for teachers or admins
Lack of assessment literacy among policy makers at
local, state, and federal levels
Little awareness throughout of how to link
assessment to teaching and learning
No norm for quality assessment in higher ed.
Aggressive selling of test services to unqualified
users
Standards of quality ignore 99% of assessments
Technical apps exacerbate quality problems
excerpted Rick Stiggins MDE- DAS 2013 Fall Conference presentation
What is “Assessment Literacy”?

When thinking about a
person being assessment
literate, what might
he/she believe, know,
and be able to do?
Pause and Self-Assess
Overview…
ASSESSMENT LITERACY
STANDARDS
ALS Development

The MAC has developed assessment literacy
standards

These standards will describe the dispositions,
knowledge and skills needed by all parties
regarding student assessment

The goal is to provide a common basis for
work to help all become more assessment
literate
ALS Development

After internal review and revision, several external
reviewers were asked to comment on the standards.
These included:
 Susan
Brookhart
 Carol Commodore
 Margaret Heritage
 Ken O’Connor
 Jim Popham
 Rick Stiggins
 MASSP, MEMSPA and MASCD
Assessment Literacy Standards
Local and State
Policymakers
District
Administrators
Building
Administrators
Teachers
Students
and Parents
Pre-service
teachers
Administrator
Certification
Assessment Literacy Standards
I.
II.
III.
Dispositions
Knowledge
Performance
Standards
are lettered
and
numbered for
easy
reference.
Assessment Literacy…
Quick Primer
Quality Classroom Assessments
Think…Pair…Share…
What elements are necessary to ensure quality
classroom assessments?
 List
these qualities
 Discuss why these are important
Thoughts From an Expert
What does assessment involve?



Making expectations explicit and public
Setting appropriate criteria and high expectations
for learning quality
Systematically gathering, analyzing, and
interpreting evidence to determine how well
performance matches those expectations and
standards….and
What does assessment involve?

Using the resulting information to document, explain,
modify, and improve performance.
Essential Questions to Dialogue
Around…

As you develop quality assessments, keep the
following questions in mind…
 What
is the purpose of the assessment?
 Who will use the information?
 How will it be used?
 Is the use formative or summative?
A Formative View

As you develop quality assessments, keep the
following questions in mind…
 What
is the purpose of the assessment?
 To
provide teachers immediate information on student
learning
 Who
will use the information?
 Teachers
 How
 To
 Is
and students in the classroom
will it be used?
inform next steps in the learning process
the use formative or summative?
 Formative
A Summative View

As you develop quality assessments, keep the
following questions in mind…
 What
is the purpose of the assessment?
 Educator
 Who
will use the information?
 Teachers
 How
 To
 Is
Evaluation / Accountability
and Administrators
will it be used?
certify the learning process
the use formative or summative?
 Summative
Quality Assessments…

are Reliable and yield Valid data.

In order for these two requirements to be met assessment
developers must pay special attention to the following:
Standard/Item Alignment
 Balance of Representation
 Target-Method Match
 Quality Items


The best way to ensure your assessment is reliable and valid
is to create a test blueprint and follow the blueprint while
developing the assessment.
Implications for the Classroom



Assessments will evolve to be more rigorous
and real world relevant, must match our
teaching to this same standard
Assessment data must be used in the moment
to inform “next steps” in the learning
process
Schools must have a balanced assessment
system in place within their classrooms
Implications for the Classroom

All stakeholders must be assessment literate
 Students
(Parents and Public)
 Teachers
 Administrators
 Policymakers
Assessment Literacy for…
STUDENTS
Student Assessment Literacy
The documentation of student learning and
progress now plays a primary role in how our
schools and educational programs are
evaluated. Assessment in all its forms (e.g.,
formative, summative, self-assessment) has
become one of the biggest discussion points in
education today. Educational accountability, must
now be demonstrated in the classroom through
the documented collection of student learning
evidence.
-- Dr. Raymond Witte
Student Assessment Literacy
Review the Standards
 Reflection Activity
See Handout

In the words of Popham…
“…assessment illiteracy is professional suicide…”
 James
Popham
Consider this…


Research suggests that teachers spend from onequarter to one-third of their professional time on
assessment related activities.
Almost all do so without the benefit of having
learned the principles of sound assessment.
 Rick
Stiggins, 2007
Assessment Literacy for…
TEACHERS
Teacher Assessment Literacy

Educator Evaluation
 Danielson Observation Tools
Teacher Assessment Literacy
Review the Standards
 Reflection Activity
 See Handout

Article Read

Read the short article by James Popham
√ = Makes sense; affirms my thinking
! = “aha”; new insight.
? = Raises a question, challenges my thinking.
Final Reflection

“A solid understanding of assessment issues should
be part of every teachers’ knowledge base, and
teachers should be encouraged to equip themselves
with this knowledge as part of their ongoing
professional development.”
 Dr.
Sara Cushing Weigle, Georgia State University
Final Reflections

“We owe it to ourselves and our students to devote
at least as much energy to ensuring that our
assessment practices are worthwhile as we do to
ensuring that we teach well”
 Dr.
David Boud, University of Technology, Sydney
Questions
Assessment Literacy for…
PRE-SERVICE TEACHERS
Pre-service Training & Accreditation of
Programs
NCATE and TEAC consolidated on July 1, 2013
Council for Accreditation of Educator Preparation (CAEP) advances
excellence in educator preparation through evidence-based
accreditation that assures quality and supports continuous improvement
to strengthen P-12 student learning.
http://caepnet.org
CAEP’s vision is to create a model unified accreditation system and
elevate teacher education preparation to the level of excellence that the
public & policymakers have come to expect.
In 2013, the CAEP Commission on Standards & Performance Reporting
has been charged to transform the preparation of teachers by creating
a rigorous system of accreditation that demands excellence and
produces teachers who raise student achievement.
CAEP Accreditation Standards for
Teacher Prep Programs
Standard 1: Content and Pedagogical Knowledge
Postsecondary institutions ensure that graduates use research and
evidence to develop an understanding of the teaching profession and
use both to measure their P-12 students’ progress and their own
professional practice.
1.3 Graduates apply content and pedagogical knowledge as reflected in
outcome assessments in response to standards of Specialized Professional
Associations (SPA), the National Board for Professional Teaching
Standards (NBPTS), states, or other accrediting bodies(e.g., National
Association of Schools of Music – NASM).
1.4 Graduates demonstrate skills and commitment that afford all P-12
students access to rigorous college- and career-ready standards (e.g.,
Next Generation Science Standards, National Career Readiness
Certificate, Common Core State Standards).
CAEP Accreditation Standards for
Teacher Prep Programs
Standard 4 of 5: Program Impact
Postsecondary institutions demonstrate the impact of its graduates on P-12
student learning and development, classroom instruction, and schools, and the
satisfaction of its graduates with the relevance and effectiveness of their
preparation.
Impact on P-12 Student Learning and Development
4.1 Postsecondary institution providers, using multiple measures, that graduates
can contribute to an expected level of student-learning growth. Multiple
measures shall include all available growth measures (including value-added
measures, student-growth percentiles, and student learning and development
objectives) required by the state for its teachers and available to educator
preparation providers, other state-supported P-12 impact measures, and any
other measures employed by the provider.
Assessment Literacy for…
BUILDING and
DISTRICT ADMINISTRATORS
PRINCIPAL Evaluation Instrument
Ineffective
Domain 3 – Programs
High Quality/Fidelity/Reliability Instructional Program Factors
Assessment Characteristics
Minimally Effective
Effective
Has a working knowledge of different
kinds of assessments, their purposes,
and the types of information they
yield to inform teaching and learning.
● Formative/summative
● Achievement
● Aptitude/ability
● Attitude/perception
Works with staff to develop and
consistently utilize assessments to
monitor and report on student
learning
Works with teachers to clearly
communicate assessment results to
students and parents
Understands and follows ethical, legal,
and technical guidelines for
assessment practices and the handling
of student assessment data
Highly Effective
And has a working knowledge of the
construction of different types of
assessments and the appropriate uses of
the data from those assessments
And works with staff to increase their
knowledge and improve their assessment
practices
And works with staff to choose, develop,
administer, analyze, and interpret the
results of both externally produced and
teacher-produced assessments
And provides training for staff in
assessment literacy and practices
And works with staff to increase their
knowledge and improve their ability to
interpret and use assessment data to
achieve better student results
And develops staff leaders in assessment
literacy and practices
And works with staff to ensure that
common assessments are administered
and analyzed with sufficient frequency
and consistency to inform instruction
And works with staff to use assessment
results when making decisions about
individual students and conferencing with
students and parents
And ensures staff understands and
follows ethical, legal, and technical
guidelines for assessment practices and
the handling of student assessment data
And develops team processes for teachers to
work together to analyze and interpret
assessment results and plan instruction
based on those results
And works with staff to use assessment
results to help students track their own
learning progress and set their own learning
goals
And assists in developing district ethical and
legal standards and technical guidelines for
assessment practices and the handling of
student assessment data
And works with staff to develop and refine
building and district policies and practices for
student assessment and the handling of
student assessment data
School ADvance™ Principal Evaluation Instrument©, Re ves, P. & McNeill, P. 2011
11
for Building-Level Administrators

There are different purposes for student assessment.

The definitions of and uses for different types of assessment

The differences between the types of assessment tools.

Promoting assessment literacy for self and staff
for Building-Level Administrators




Providing time and support for staff to implement a
balanced assessment system by providing opportunities to
develop skills in: Scoring/Analyzing results
Leading dialogues with staff in interpreting results
Clearly explaining how to analyze and use assessment
results
Clearly communicating results to various constituents
DISTRICT LEADER Evaluation Instrument
Domain 3 – Systems Alignment
High Quality, Fidelity, and Reliability Instructional Program Factors
Standards Based Assessment and Feedback Characteristics
Ineffective
Minimally Effective
Effective
Highly Effective
Has a working knowledge of the
And works with district leaders and
And works with district leaders and staff
construction of different type of
staff to increase their knowledge and
to develop a comprehensive assessment
assessments and the appropriate
improve their ability to employ
system
uses of the data from those
effective assessment practices.
assessments.
Works with district leaders to
monitor the use of district
assessments
Has a working knowledge of
analysis and interpretation of
assessment data
Works with district leaders and
staff to ensure and timely
communication of assessment
results to students and parents
Understands and follows ethical,
legal and technical guidelines for
assessment practices and the
handling of student assessment data
And works with district leaders and
staff to insure that common
assessments are administered and
analyzed with sufficient frequency and
consistency to inform instruction and
school improvement
And works with district leaders and
staff to improve analysis and
interpretation of assessment data to
achieve better student results
And works with district leaders and
staff to develop a reliable system for
providing timely feedback to students
and parents based on assessment
results
And works with district leaders and staff
to develop team processes for analyzing
and interpreting assessment results and
planning instruction based on those
results
And ensures district leaders and staff
understand and follow ethical, legal,
and technical guidelines for
assessment practices and the handling
And assists in developing district ethical
and legal standards and technical
guidelines for assessment practices and
the handling of student assessment data
And develops administrative and staff
leaders in assessment, analysis, and
interpretation practices
And works with district leaders staff
develop a reliable system for students to
use assessment results to track their own
learning progress and set their own
learning goals
School ADvance™ District Leader/Superintendent Evaluation Instrument©, Reeves, P. & McNeill, P. 2012
14
for District-Level Administrators

There are different purposes for student assessment.

The definitions of and uses for different types of assessments

The different types of assessment methods and when educators
should use each

How to develop or select high quality assessments

Promoting assessment literacy with staff
for District-Level Administrators
Assuring that each and every staff member is:


A confident, competent master themselves of the targets that they are
responsible for teaching
Sufficiently assessment literate to assess their assigned targets
Assisting teachers to collaboratively analyze and use data
Leading dialogues with staff in interpreting results and creating
goals for improvement
Clearly communicating results to various constituents
and the
School Improvement Framework
STRAND 1: TEACHING FOR LEARNING
Standard 1: Curriculum
A.
Alignment
B.
Coherence
Standard 2: Instruction
C. Instructional Design
D. Effective Instructional Practices
E. Learning Environment
F. Reflection
Standard 3: Assessment
and the
School Improvement Framework
STRAND II: LEADERSHIP FOR LEARNING
Standard 4: Instructional Leadership
K. A Vision for Learning
L. Guidance and Support for Teaching and Learning
M. Results Focused
Standard 5: A Climate for Learning
N. Safe and Supportive Environment
O. Shared Leadership for Learning
STANDARD 6: ORGANIZATIONAL MANAGEMENT
P. Communication Systems
Q. Intentional Practices
and the
School Improvement Framework
STRAND III: PROFESSIONAL LEARNING
STANDARD 7: PROFESSIONAL LEARNING CULTURE
S. Collaborative Teams
T. Collective Responsibility
STANDARD 8: PROFESSIONAL LEARNING SYSTEM
U. Purposeful Planning
V. Impact of Professional Learning
and the
School Improvement Framework
STRAND IV: SCHOOL, FAMILY AND COMMUNITY RELATIONS
STANDARD 9: COMMUNICATION
W. Approaches and Tools
X. Cultural Responsiveness
STANDARD 10: ENGAGEMENT
Y. Learning Opportunities
Z. Partnerships
and the
School Improvement Framework
Standard 3:Assessment
G. Assessment System
• Implements a balanced
assessment system and ensures
that summative and on-going
formative assessments are
aligned to curriculum and
instruction
• Classroom assessments are
designed to be developmentally
appropriate and are aligned to
the depth of knowledge required
to demonstrate proficiency with
standards
Assessment Literacy Standards:
Teachers
Knowledge
• The definitions of and uses for
different types of assessments
• How to unpack standards into
clear learning targets and that
are written in student-friendly
language and that are used as
the basis for the everyday
curriculum.
and the
School Improvement Framework
H. Shared Understanding
• All educators can communicate
the appropriate purposes and
uses of assessment.
• Reports of student data are
communicated to students and
parents in a manner that they
can understand.
Dispositions
• An effective assessment system
must balance different
purposes for different users
and use varied methods of
assessment and communication.
Performance
• Communicate effectively with
students, parents, other
teachers and administrators
about student learning.
and the
School Improvement Framework
I.
Data Analysis and
Decision-Making
• Educators use a combination of
student achievement,
demographic, process and
perception data over time to
make informed instructional
decisions to meet individual
student needs
• Assessment data are used to
place students, monitor progress
and drive timely interventions.
Knowledge
• There are different purposes for
student assessment: Student
improvement; Instructional
program improvement; Student,
teacher or system accountability:
Program evaluation; Prediction for
future performance or
achievement
• Different users have different
assessment purposes
• Different assessment purposes
may require different assessment
methods
and the
School Improvement Framework
J. Student Involvement in
the Assessment Process
• Students understand the criteria and
expectations for demonstrating their
learning.
• Students receive descriptive feedback
based on student performance, as well
as guidance on how to improve.
• Students are taught how to self-assess
and plan for improvement.
• Students learn to track and use their
own achievement data and related
feedback to monitor, evaluate, and
reflect on how to improve their own
performance.
Knowledge
• How to engage students in using
their own assessment results for
reflection and goal setting
• How to provide effective
feedback from assessments
suitable for different audiences:
descriptive vs. evaluative
Performance
• Support student use of assessment
feedback to improve attitudes,
aspirations, mindsets and
achievement.
Case Study…
POLICYMAKERS
for Policy Makers
Balance is needed in an assessment system because:
1. Different users have different assessment purposes
2. Different assessment purposes may require different
assessment methods
There are different purposes for student assessment:
1. Student improvement
2. Instructional program improvement
3. Student, teacher or system accountability
4. Program evaluation
5. Prediction of future performance/achievement
for Policy Makers
The differences between the types of assessments in a balanced
system of assessment:
1. Summative Assessments
2. Interim Benchmark Assessments
3. Formative Assessment
There are different ways to measure student achievement; each
has advantages and challenges.
for Policy Makers
There are different ways to report results:
1. Norm-referenced interpretations
2. Criterion-referenced interpretations
3. When each type of interpretation may be more useful
There are several essential technical standards for high quality
assessments:
1. Reliability – Do the assessments produce replicable
scores?
2. Validity – Is there evidence that supports the intended
uses of the assessment?
Assessments can be purchased or developed locally; each
approach has advantages and challenges.
for Policy Makers
There are a number of steps in the assessment development
process to produce high quality assessments.
Quality assessment of any type requires adequate funding and
time.
Users of the assessment require time to learn to administer and
use the assessment results appropriately, and resources may be
needed to carry out these activities.
Should know which student measures are appropriate for teacher
and administrator evaluation.
Assessment Literacy…
RESOURCES AND
NEXT STEPS
Individual Reflection on the Standards
Silently read and record on the standards
document:

Feedback on the standards:
Clarifications/suggested revisions
 Deletions
 Additions
 Strengths/positives regarding the standards

What are the Implications?

What are your thoughts about implications of the
Assessment Literacy Standards …
For you?
For your building?
For your district?
For higher education?
Silently record your thoughts on an index card.
A Method to Review Multiple Standard
Sets
If you want to have your group look at more than one
set of standards, you could do a JIGSAW.
Jigsaw

Groups of 3 or 4 receive a specific set of standards. The
group jigsaws reading and reports the assigned reading to
their group.
 Once each group has read and reviewed their entire
assigned set of standards, they report out to the whole
group what their set of standards included.
 WHILE READING, the group members record their feedback
on the standards (clarifications, additions, etc.).
 They discuss this and report it to the whole group.

Time to Process

Now, think about two questions:
 If
Michigan educators had achieved these standards,
what would be different in Michigan’s students?
 If this is a preferred future, what would need to be
done to push this vision forward?

You may wish to turn and talk about this for a few
minutes
Putting the Standards to Use

Professional development for K-12 teachers,
administrators, etc.


Higher Education


Workshops, full-length courses, tools and resources, etc.
Pre-Service Teachers, Administrator Certification
Future collaboration with other organizations (PD,
development of public policy, etc.)
Thank You!
For copies of draft Assessment Literacy Standards, as well as an
opportunity to respond to an electronic survey about the
standards – go to the MAC website, in a column on left hand side
of home page – click on hyperlink for Assessment Literacy
Standards