Transcript The Seven Principles for Good Practice

Writing Outcomes
& Measures
Adapted from Susan Hatfield
Presented to DC faculty
InterCampus Day
Spring 2012
Designing academic
programs
1. Identify
desired
outcomes.
2. Determine
acceptable
evidence.
3. Plan learning
experiences &
instruction.
Doing assessment
Nitsáhákees: establish
learning goals
Siihasin:
assess student
learning and
use the results
Nahat’a: plan
learning
opportunities
Íiná: teach
The most important thing
Start with good
student learning
outcomes.
Avoid…
•
•
Students should be able to
comprehend, interpret, analyze and
critically evaluate material in a variety of
written and visual formats.
Students will demonstrate creative and
evaluative thinking in the analysis of
theoretical and practical issues in the
areas of politics, the economy, and the
environment.
Models of
understanding
Bloom’s
taxonomy
Understanding
by Design
Knowledge
Explanation
Comprehension
Interpretation
Application
Application
Analysis
Perspective
Synthesis
Empathy
Evaluation
Self-knowledge
Linda
Suskie
Familiarity
Application
Habit of mind (or
disposition)
SLO formatting
FORMAT:
Students should be able to
<action verb> + <something>
RULE OF THUMB:
If you have more than one action verb,
keep the one that represents the
highest order of thinking.
COMPREHENSION
EVALUATION
APPLICATION ANALYSIS SYNTHESIS
KNOWLEDGE
Cite
Count
Define
Draw
Identify
List
Name
Point
Quote
Read
Recite
Record
Repeat
Select
State
Tabulate
Tell
Trace
Underline
Associate
Classify
Compare
Compute
Contrast
Differentiate
Discuss
Distinguish
Estimate
Explain
Express
Extrapolate
Interpolate
Locate
Predict
Report
Restate
Review
Tell
Translate
Apply
Calculate
Classify
Demonstrate
Determine
Dramatize
Employ
Examine
Illustrate
Interpret
Locate
Operate
Order
Practice
Report
Restructure
Schedule
Sketch
Solve
Translate
Use
Write
Analyze
Appraise
Calculate
Categorize
Classify
Compare
Debate
Diagram
Differentiate
Distinguish
Examine
Experiment
Inspect
Inventory
Question
Separate
Su rize
Test
Arrange
Assemble
Collect
Compose
Construct
Create
Design
Formulate
Integrate
Manage
Organize
Plan
Prepare
Prescribe
Produce
Propose
Specify
Synthesize
Write
Appraise
Assess
Choose
Compare
Criticize
Determine
Estimate
Evaluate
Grade
Judge
Measure
Rank
Rate
Recommend
Revise
Score
Select
Standardize
Test
Validate
Lower-level
outcomes
COMPREHENSION
EVALUATION
APPLICATION ANALYSIS SYNTHESIS
KNOWLEDGE
Cite
Count
Define
Draw
Identify
List
Name
Point
Quote
Read
Recite
Record
Repeat
Select
State
Tabulate
Tell
Trace
Underline
Associate
Classify
Compare
Compute
Contrast
Differentiate
Discuss
Distinguish
Estimate
Explain
Express
Extrapolate
Interpolate
Locate
Predict
Report
Restate
Review
Tell
Translate
Apply
Calculate
Classify
Demonstrate
Determine
Dramatize
Employ
Examine
Illustrate
Interpret
Locate
Operate
Order
Practice
Report
Restructure
Schedule
Sketch
Solve
Translate
Use
Write
Upper-level
outcomes
Analyze
Appraise
Calculate
Categorize
Classify
Compare
Debate
Diagram
Differentiate
Distinguish
Examine
Experiment
Inspect
Inventory
Question
Separate
Summarize
Test
Arrange
Assemble
Collect
Compose
Construct
Create
Design
Formulate
Integrate
Manage
Organize
Plan
Prepare
Prescribe
Produce
Propose
Specify
Synthesize
Write
Appraise
Assess
Choose
Compare
Criticize
Determine
Estimate
Evaluate
Grade
Judge
Measure
Rank
Rate
Recommend
Revise
Score
Select
Standardize
Test
Validate
Good SLO’s
•
•
•
•
•
•
•
•
•
•
•
Locate online resources on a particular issue or topic.
Apply scientific principles to everyday life.
Analyze errors.
Compare and contrast perspectives and values.
Justify potential careers.
Use an appropriate mathematic procedure to solve a problem.
Conduct a research study.
Design a community service project.
Demonstrate intellectually curiosity.
Appreciate the perspective of people from backgrounds other
than one’s own.
Choose ethical courses of action.
Direct measures
of learning
Familiarity
Locally developed
tests
• Essay questions
scored blindly by
faculty
• Standardized tests
• National certification or professional
exams
•
Application
Performancebased assessments
• Juried review of
projects
• Externally reviewed exhibitions and
performances
• Evaluation of
internships based
upon program
learning outcomes
• Capstone
experience
•
Disposition
Observations of
student performance
• Student reflections
• Student selfassessment
•
More performance
measures
Abstract or executive summary
Advertisement of commercial
Annotated bibliography
Autobiography from an historical period
Briefing paper
Brochure or pamphlet
Campaign speech
Case study
Client report
Collaborative group activity
Database
Debate or discussion
Debriefing interview preparation
Dramatization of an event or scenario
Revising a poorly written paper
Evaluation of opposing points of view
Experiment or other lab experience
Field notes
Game invention
Graph, chart, or diagram
Graphic organizer or taxonomy
Handbook or instructional manual
Journal or log
Letter to an editor or business
Model, simulation, or illustration
Narrative
Newspaper story or news report
Oral history recording of an event
Plan for researching or solving a problem
Portfolio
Poster, display, or exhibit
Presentation or demonstration
Process description
Proposal for solution to a problem
Reflection on what was learned
Review and critique of one’s own work
Selected portions of an essay
Survey with analysis of results
Video or audio recording
Web site
Kinds of rubrics
• Checklist rubrics
• Rating scale rubrics
• Descriptive rubrics
• Holistic scoring guides
• Structured observation guides
Designing rubrics
Create the rating scale
•
•
•
•
List what you are looking for (refer back to your student
learning outcomes).
Create at least 3 levels.
Label each level with names, not just numbers, e.g.,
“complete evidence, partial evidence, minimal evidence, no
evidence” or “exceeds standard, meets standard,
approaching standard, or below standard.”
If developing a descriptive rubric, describe in the boxes the
different levels of performance.
Try out the rubric! Adjust as needed.
Assessment measures –
what the HLC expects
1990
2000
2010
INDIRECT
MEASURES
DIRECT
MEASURES
Indirect performance
measures
•
•
•
•
•
•
Student, employer, and alumni satisfaction survey
Exit interviews and focus group interviews of graduates
Retention and transfer studies
Length of time to degree
Graduation and transfer rates
Job placement rates