Transcript Document

Measure What You
Treasure
Camille Catlett
FPG Child Development Institute (NC)
Susan P. Maude
IOWA STATE UNIVERSITY (IA)
Please be
generous
Sharing your examples and
experiences, as experienced
professional development
providers, will benefit us all.
Setting the Stage:
Three Big Ideas
1. All → Each & Every
2. Knowledge acquisition + knowledge
application
3. Facts →Evidence-based practices
All
Each &
Every
The Real Early Learning Challenge:
Meeting the Needs of Each & Every Child
An effective teacher can have a stronger
influence on student achievement than
poverty, language background, class size, or
minority status
NPDCI Definition of
Professional Development
“Professional development is facilitated
teaching and learning experiences that
are transactional and designed to
support the acquisition of professional
knowledge, skills, and dispositions as
well as the application of this knowledge
in practice…..
Definition (continued)
The key components of professional
development include:
a) characteristics and contexts of the
learners (i.e., the “who” );
b) content (i.e., the “what” of professional
development); and
c) organization and facilitation of
learning experiences (i.e., the “how”).”
Parts of a whole
Culturally
Appropriate
Individually
Appropriate
Developmentally
Appropriate
You’ll need a handout that looks like this . . .
. . . and one
that looks
like this
With a partner . . .
First
Review the
course
assignments
AND the key
considerations
for
assignments in
the sample
syllabus
Second
Discuss
whether the
assignments
provide
opportunities
to measure
achievement
of each
learning
outcome.
Third Fourth
Discuss
whether the
assignments
incorporate
opportunities
for knowledge
acquisition and
knowledge
application
AND whether
they presume
prior
knowledge or
experience.
Revise
Assignment 1 to
reflect key
considerations.
Then, discuss a
new possible
assignment for
this course that
reflects key
considerations.
Clear instructions, great expectations
Roger Graves
Sources used include: Stevens &
Levi (2005); Zhang & Fiore (2011)
Rubrics and
Student Instructions
Why use rubrics
HAS A STUDENT EVER SAID TO YOU REGARDING
AN ASSIGNMENT, "BUT, I DIDN'T KNOW WHAT YOU
WANTED!" OR "WHY DID HER PAPER GET AN 'A'
AND MINE A 'C?'"
Instructions for assignments
see pdf by Graves, R. (nd)
• Orient your readers
• Define terms
• Brief overview of the entire procedure
• Provide list of tools or concepts needed to be successful
• Break instructions into steps
• Keeps steps discrete
• Use imperative sentences (Verb + object)
• Nice-to-know vs. Need-to-know****
•
•
•
•
Guidelines for writing instructions
How students read assignments
Provide exemplars
Your tips?
What is a rubric?
Rubrics are developed to provide the instructor
with a tool for consistently scoring work based
on explicitly defined performance levels for
the specific criteria that are being evaluated.
Students can use the provided rubric to guide
their efforts and fine tune their submission so
as to meet the activity requirements.
Purpose of a rubric
• For formative (midterm) or summative (end of
semester/course) evaluation:
• Create a common framework and language for
evaluation.
• Provide clear expectations about what will be assessed
and standards that should be met.
• Increase the consistency and objectivity of evaluation,
especially scoring or rating of performance and
products, by providing operational definitions for the
standards.
• Provide clear feedback and identify areas for
improvement.
Use
 Rubrics can be used to grade any assignment or task:
research papers, book reviews, oral presentations,
and more.
• Discussion, teamwork, and cooperative learning
(online discussion boards; class participation)
• Link to rubric about online courses
(exemplar/resources section)
• Wiki rubric, blog rubric, twitter rubric
• Portfolio’s and E-portfolio’s
• Field Experience:
• Assessing teaching practices (classroom or field
site observation)
• Evaluate implementation fidelity
Situations where rubrics can be applied for student
evaluation: Comparison of types of student
assessment (Zhang & Fiore, 2011).
Traditional Testing
(e.g., standardized, multiple choice)
•
•
•
•
•
•
•
•
•
Given annually, one shot
Based on a single setting
One correct response
Norm-referenced
Test/teacher-driven
"Teacher proof"
Paper/pencil
Narrow measure of skill
Separate from curriculum and
instruction
• Comparisons to others
• Produces anxiety
Rubric-Based Performance Assessment
(e.g., performance assessment, portfolio)
•
•
•
•
•
•
•
•
Ongoing, cumulative
Based on a variety of settings
Open-ended, multiple possibilities
Criterion-referenced
Student-driven
Teacher-mediated
Performance
Real-world, integrated application
that measures capacity for
constructing and using knowledge
• Integral to curriculum and instruction
• Comparisons to self and goals
• Produces ability to self-assess and
self-correct
Types of rubrics
• Holistic rubric: Assessor generates a single global
score for the entire work, sometimes based on a
set of descriptors.
• Analytic rubric: Assessor judges the work by
examining specific aspects of the work that will
be evaluated independently. These aspects may
be reported separately or they may be combined
to create a global score (in some cases, the score
is weighted based on the importance of the
dimension/indicator).
(Holistic Rubric Example)
Oral Report
5
Excellent: The student clearly describes the question studied and provides
strong reasons for its importance. Specific information is given to support the conclusions
that are drawn and described. The delivery is engaging and sentence structure is consistently
correct. Eye contact is made and sustained throughout the presentation. There is strong
evidence of preparation, organization, and enthusiasm for the topic. The visual aid is used to
make the presentation more effective. Questions from the audience are clearly answered with
specific and appropriate information.
4
Very Good: The student described the question studied and provides reasons for
its importance. An adequate amount of information is given to support the conclusions that
are drawn and described. The delivery and sentence structure are generally correct. There is
evidence of preparation, organization, and enthusiasm for the topic. The visual aid is
mentioned and used. Questions from the audience are answered clearly.
3
Good: The student describes the question studied and conclusions are stated, but
supporting information is not as strong as a 4 or 5. The delivery and sentence structure are
generally correct. There is some indication of preparation and organization. The visual aid is
mentioned. Questions from the audience are answered.
2
Limited: The student states the question studied, but fails to fully describe it.
No conclusions are given to answer the question. The delivery and sentence structure is
understandable, but with some errors. Evidence of preparation and organization is lacking.
The visual aid may or may not be mentioned. Questions from the audience are answered with
only the most basic response.
1
Poor: The student makes a presentation without stating the question or its
importance. The topic is unclear and no adequate conclusions are stated. The delivery is
difficult to follow. There is no indication of preparation or organization. Questions from the
audience receive only the most basic, or no, response.
0
No oral presentation is attempted.
(Analytic Rubric Example)
Rubric for Research Papers (45 points)
Performance Element
Distinguished
__5__ Points
Establishes and maintains clear focus; evidence of
distinctive voice and/or appropriate tone
Proficient
_4___ Points
Focused on a purpose; evidence voice and/or suitable
tone
Purpose and Focus
Development of Ideas
Cross Cultural
Perspectives
Depth and complexity of ideas supported by rich, engaging Depth of idea development supported by elaborated,
and/or pertinent details; evidence analysis, reflection and relevant details
insight
Use of references indicate ample research (at least 15
references used, they are from the last 5 years unless
seminal; uses mostly research journals and secondary
resources). Good use of APA, MLA, SLA formatting
References
5
Unelaborated idea
development;
unelaborated and/or
repetitious details
5
Minimal idea
development, limited
and/or unrelated details
Some references (at
Few references (less
least 10 references used, than 10 used, older than
they are from the last 5 the last 5 years and
years or older - unless mostly uses secondary
seminal; uses some
resources). Poor use of
research journals and
APA, MLA, or SLA
mostly secondary
formatting.
resources). Some use of
APA, MLA, or SLA
formatting.
10
5
Logical organization
Lapses in focus and/or
coherence
Random or weak
organization
5
Variety of sentence structure and length
Controlled and varied sentence structure
Simplistic and/or
awkward sentence
structure
Incorrect or lack of topic
and/or ineffective
wording and/or
sentence structure
5
Simplistic and/or
imprecise language
Incorrect and/or
ineffective wording
and/or sentence
structure
Precise and/or rich language
Acceptable, effective language
Language
Control of surface features
Total Points
Possible
An attempt to establish Limited awareness of
and maintain purpose
audience and/or
and communicate with purpose
the audience
Careful and/or suitable organization
Sentence Structure
Grammar and
Formatting
Novice
__2-1__ Points
We will generate these indicators as a group this semester given our readings from
Barrera & Kramer as well as other readings and discussion.
Use of references indicate substantial research (at least 15
references used, they are from the last 3-5 years unless
seminal; uses mostly research journals and some
secondary resources) Strong use of APA, MLA, SLA
formatting
Organization
Intermediate
__3__ Points
Few errors in grammar or format relative to length and
complexity
Some errors in grammar
and/or format that do
not interfere with
communication
Errors in grammar and
format (e.g., spelling,
punctuation,
capitalization, headings)
5
5
45
Score
Types of rubrics: Comparison
Type
Holistic rubric
Analytic rubric
Purpose
Gives the “big
picture”
Advantages
Disadvantages
Efficient and holistic Lacks specificity and
does not provide
extensive feedback
Identifies needs,
strengths, and
weaknesses
Sometimes total
does not equal to
sum of the parts.
Online Course Systems - Rubrics
• If you use an online system (Blackboard, WEBCT, etc.), see if
they provide a template for creating and using rubrics.
• Two handouts for steps in Blackboard (how to) in creating and
using an online rubric will be posted.
Four (4) Part-by-Part
development of a rubric
• Part 1:
• Part 2:
• Part 3:
• Part 4:
Task Description
Scale
Dimensions (Domain or attributes)
Description of the Dimensions
(indicators)
Part 1: Task description
• Framed by the instructor and involves a
“performance” of some sort by the students.
• Place the task description at the top of the grading
rubric (lift from your syllabus).
• Most include a descriptive title and a task
description.
Part 2: Scale
• Scale is the metric used to assign points to global
categories (holistic) or specific elements (analytic) to
assess how well any given task in the activity has been
accomplished.
• Terms should be tactful but clear.
• Some prefer nonjudgmental, non competitive language
such as “high” “middle” and/or “beginning”.
• How many scales should I use? First time using a rubric
you may want to start with three and then build to 4 or
5.
• Example: Excellent, Competent, Needs Work (3 scale)
Other Scaling Examples
• Exemplary, proficient, marginal, unacceptable
• Below expectations, meets expectations,
exceeds expectations
• Inadequate, barely adequate, good, exemplary
• Ineffective, moderately effective, effective, highly
effective
• Below basic, basic, proficient, advanced
• Novice, developing, proficient, expert
• Rarely, sometimes, often, almost always
Part 3: Dimensions
• Lay out the parts of the task simply and completely.
• Clarifies components and which are most important.
• “Kind of” a task analysis.
• Elements needed for the final product.
Part 4: Description of the dimensions
(indicators)
• All-encompassing categories.
• Should contain the highest level of performance per
dimension.
• Specific indicators needed to meet that component or
element.
Four key stages in constructing a
rubric
• Stage 1: Reflecting. What do we want the students to do? Why
did we create/use this assignment? What happened last time
we gave it?
• Stage 2: Listing. Focus on the particular details of the
assignment and what specific learning objectives we hope to
see in the completed assignment.
• Stage 3: Grouping and labeling. Organize given #1 and #2,
group expectations together in what will become the rubric
dimensions.
• Stage 4: Application. Apply to a format/grid.
Stevens & Levi (2005).
Practice
Using A rubric for rubrics (Mullinex, B. 2003) and
working in teams (or individually):
1. Start developing a rubric for the previous
assignment.
2. Critique example provided.
3. Critique your own rubric.
Four (4) Part-by-Part
development of a rubric
• Part 1:
• Part 2:
• Part 3:
• Part 4:
Task Description
Scale
Dimensions (Domain or attributes)
Description of the Dimensions
(indicators)
Rubric Resources
• Graves, R. (nd). Clear instructions, great expectations: Creating
good writing assignments. Retrieved from
http://www.ualberta.ca/~graves1/assignments.pdf
• Mertler, C. (2001). Designing scoring rubrics for your
classroom. Practical Assessment, Research, & Evaluation,
7(25). Retrieved from
http://pareonline.net/getvn.asp?v=7&n=25
• Moskal, B. (2003). Recommendations for developing
classroom performance assessments and scoring rubrics.
Practical Assessment, Research, & Evaluation, 8(14). Retrieved
from http://pareonline.net/getvn.asp?v=8&n=14
• Reynolds-Keefer, L. (2010). Rubric-referenced assessment in
teacher preparation: An opportunity to learn by using.
Practical Assessment, Research, & Evaluation, 15(8). Retrieved
from http://pareonline.net/pdf/v15n8.pdf
Highly referenced text
• Stevens, D., & Levia, A. (2005). Introduction to rubrics: An
assessment tool to save grading time, convey effective
feedback and promote student learning. Sterling, VA: Stylus
Publishing.
• Also ppt available @
http://www.introductiontorubrics.com/index.html
Links to exemplars
• CSU. (2009). Rubric for online instruction. http://sites.newpaltz.edu/tlc/wpcontent/uploads/sites/4/2013/02/Chico_rubric_online_teaching.pdf
• Creative Commons – Share alike site: Teach Philosophy 101 – Tests, papers, assignments.
• http://www.teachphilosophy101.org/Default.aspx?tabid=64
• Spielberg, L. (2011). FINE-Flamboyan Foundation Classroom Family Engagement Rubric (HFRP, March 2011
issue). Retrieved at http://www.hfrp.org/publications-resources/browse-our-publications/successful-familyengagement-in-the-classroom-what-teachers-need-to-know-and-be-able-to-do-to-engage-families-in-raisingstudent-achievement
• Stevens & Levi text has a web site with samples:
http://www.introductiontorubrics.com/samples.html
• University of Wisconsin-Stout has some great resources on various ways to develop and use rubrics. Also
provides many exemplars:
• http://www.uwstout.edu/soe/profdev/assess.cfm
• http://www.uwstout.edu/soe/profdev/rubrics.cfm
• Zhang, X., & Fiore, T. (2011). Measuring project performance with rubrics: Rules for development,
use, and analysis. Workshop presented at the 2011 OSEP Project Director’s Conference, Washington,
DC. Retrieved from https://www.osep-meeting.org/2011conf/presentations/Large_Group_Panels/Tue_AMMeasureProjPerformanceWRubric/Zhang_7-19_AM%281%29.pdf
Sites to help in rubric makers:
• RubiStar – free tool to help teachers create quality rubrics. http://rubistar.4teachers.org/
• Rcampus (personal) – Over 30,000 rubrics shared and build one electronically https://www.rcampus.com/
• teAchnology – rubric makers http://www.teach-nology.com/web_tools/rubrics/
Thank you
for your generous
participation and
your dedication to
supporting each
and every child and
their unique family.
“Even if you’re on
the right track,
you’ll get run over
if you just sit there”
- Will Rogers