Transcript IDEA--Student Ratings of Instruction

What do you know about
IDEA?
Student Learning Model
Welcome
The IDEA System
Overview and Coaching Tool
Agenda and Purpose
Introductions
IDEA as a Diagnostic to Guide
Improvement
Benefits:
A Tool to Improve
Student Learning and
Teaching Effectiveness
Purpose of Presentation
• To provide a practical model for colleges
• Get us on the “same page”
• Describe The IDEA Center’s underlying assumptions
•
about the use of student ratings
Review IDEA - Student Ratings of Instruction system
–
–
–
–
–
–
FIF
Selecting objectives
Diagnostic Survey
Diagnostic Form Reports
Administration
Resources
• To share Best Practices
7/21/2015
4
Improvement of Student
Learning
• Student Ratings can have a positive impact if...
– The instrument
• Is “learning focused”
• Provides a diagnostic
– The emphasis for “summative” faculty evaluation is
appropriate
• 30%-50% of the overall evaluation of teaching
• Results are not over-interpreted
• Faculty trust the process
• 6-8 classes-more if small <10
7/21/2015
5
What does IDEA stand for?
Individual Development and Educational
Assessment
7/21/2015
6
Improvement of Student
Learning
• Campus and college culture has great
impact
– Is teaching excellence a high priority?
– Are student ratings given an appropriate
weight in the overall evaluation of teaching?
7/21/2015
7
Improvement of Student
Learning
– Does the institution provide resources to aid
those seeking to improve their instructional
skills?
• Teaching and Learning Centre
• Department Chair/Faculty Development Director
• Mentor Faculty
• Brown bag sessions
• IDEA Resources
– Is credible support for improvement available?
7/21/2015
8
Underlying Assumptions
• Students are not qualified to assess:
– Faculty expertise
– Appropriateness of goals, content, and
organization of course
– Materials used in delivery
– How student work is evaluated, including
grading practices
7/21/2015
9
Underlying Assumptions
• Nor are they qualified to assess “indirect”
contributions to instruction
– Support for departmental efforts
– Assistance to colleagues
– Contributing to a positive atmosphere
7/21/2015
10
Other Sources of Evidence for
Evaluation
• Self-evaluation/reflective statement
• Course materials
• Graded appraisal tools
•
•
•
•
•
– tests, essays, papers, etc.
Feedback from mentors/colleagues
Peer observations
Classroom assessment/research efforts
Teaching portfolio
Faculty Development Plan
7/21/2015
11
IDEA Student Ratings of
Instruction
The Student Learning Model
Student Learning Model
• Types of learning must reflect instructor’s
purpose
• Effectiveness is determined by student
progress on objectives stressed by
instructor
• Specific teaching behaviors influence
certain types of student progress under
certain circumstances.
7/21/2015
13
IDEA Student Ratings of
Instruction Overview
•Faculty Information Form
•Student Survey
–Diagnostic Form
–Short Form
IDEA: FIF
Faculty Information Form
Handout #1
Faculty Information Form FIF
7/21/2015
16
12 Learning Objectives
FIF: Selecting Objectives
• 3-5 as “Essential” or “Important”
– Is it a significant part of the course?
– Do you do something specific to help students
accomplish the objective?
– Does the student’s progress on the objective
influence his or her grade?
7/21/2015
18
Relevant Objectives
• Basic Cognitive Background
– Items 1, 2
• Applications of Learning
– Items 3, 4
• Expressiveness
– Items 6, 8
7/21/2015
19
Relevant Objectives
• Intellectual Development
– 7, 10, 11
• Lifelong Learning
– 9, 12
• Team Skills
–5
7/21/2015
20
Course Description Items
• Optional
• Best answered toward end of term
• Do NOT influence your results
Bottom of Page 1
Top of page 2
Student Survey
Diagnostic Form
Handout # 2
Diagnostic Survey Form
Diagnostic Student Survey:
• Specific teaching behaviors…
• Teaching Methods: Items 1-20
• ...influence certain types of student
progress…
• Learning Objectives: Items 21-32
• …under certain circumstances.
• Student and Course
– Student Characteristics: Items 36-39, 43
– Course Management/Content: Items 33-35
• Others
• Summary Measures: Items 40-42
• Experimental Questions: Items 44-47
• 20 Additional Questions
Student Survey
• Using extra questions
• Comments
– Constructive
– How to become a better teacher
– Administration’s role
7/21/2015
25
False Assumptions
• Effective instructors effectively employ all
20 teaching methods.
• The 20 teaching methods items are used
to make an overall judgment about
teaching effectiveness.
• Students should make significant progress
on all 12 learning objectives
7/21/2015
26
Best Practices
Faculty
• Know the objectives before the course starts
• Day 1
– Discuss meaning of objectives with students
– Inform them they will be asked to rate their own
progress on objectives
– Reflect on their understanding of course purpose
– Discuss differences in perception of objectives’
meaning
– Throughout the term - reinforce
7/21/2015
27
Best Practices
• Department
– Multi section courses
– Coaching, feedback and follow-up
– Department
• Institution
– The scorecard
• Excellence of teaching
• Progress on learning
• Recognition
7/21/2015
28
Best Practices for Online
• Know the objectives prior to the start of
the class
• Incorporate into your announcements,
discussions, email, chat
• Follow-up in the discussion
thread and use “Kim’s” post
to illustrate the objective
7/21/2015
29
Let’s Review
• How many learning objectives should you
select?
• What is one way to communicate your
expectations for learning with students?
• Were there any new key points in this
section?
Report Overview-Handout #3
7/21/2015
31
Interpreting the IDEA Report
Administrative Decisions
The IDEA Report
• Diagnostic Form Report
– Administrative Decisions
• What were students’ perceptions of the course and
their learning?
• Pages 1 & 2
– Professional Improvement
• What might I do to improve my teaching?
• Page 3
Page 1
Page 1: Global Assessment
• Weighted average of
Progress on instructor
selected (“relevant”)
objectives
• Overall, I rate this
instructor an
excellent teacher
• Overall, I rate this
course as excellent
7/21/2015
35
Summary Evaluation of Teaching
Effectiveness
Adjusted Scores
• Control for factors beyond instructor’s
control
• Regression equations
7/21/2015
37
Adjusted Scores
• Diagnostic Form
– Student Work Habits (#43)
– Student Motivation (#39)
– Class Size (Enrollment, FIF)
– Student Effort (multiple items)
– Course Difficulty (multiple items)
• Short Form
– Adjusted for 3 factors
Summary Evaluation of Teaching
Effectiveness: Adjusted Scores
Understanding Adjusted Scores
Page 2: Student Progress
• Report of progress on
•
•
“Essential” or
“Important”
objectives
Student perceptions
of course demands
Student selfassessment of
motivation, academic
effort, and work
habits
Page 2: Progress on Specific
Objectives
4.5+4.5
4.4+4.4
4.2+4.2
+3.2
7
Page 2: Description of Course
and Students
Page 3: Diagnostic – Typically
Used Only Formatively
• The diagnostic
•
•
component
Teaching methods to
consider employing
more frequently to
support learning
Not intended for
making summative
judgments
Page 3: Improving Teaching
Page 4: Statistical Detail
• Statistical Detail
• Additional Questions
Using IDEA
Formative Use
The Big Picture Page 1
Progress on Learning Page 2
Teaching Opportunities Page 3
Back to Page 2: Additional
Reflection
Additional Reflection - Statistical
Detail Page 4
Page 4: Additional Reflection Additional Questions
Always Something New with IDEA
• Premark Software
• Online capability to administer the surveys
• Online Dean and Academic Director
surveys
• Launch of new web site
• Senior Research Officer
• Ongoing support
7/21/2015
54
7/21/2015
55
IDEA Resources
7/21/2015
57
Resources: Administering IDEA
• www.theideacenter.org
All resources
on our website.
– Support for Student Ratings
• Best practices
• Directions to Faculty
• Using Additional Questions
• Some Thoughts on Selecting IDEA Objectives
• Disciplinary Selection of Learning Objectives
• Guide to Administering IDEA
• Team Teaching
Relationship of Learning Objectives
to Teaching Methods
Handout #5
7/21/2015
59
Resources: Administering IDEA
• www.theideacenter.org
All resources
on our website.
– Client Resources
• IDEA Resources
– Best practices
– Directions to Faculty
– Using Additional Questions
– Some Thoughts on Selecting IDEA Objectives
– Disciplinary Selection of Learning Objectives
– Guide to Administering IDEA
– Team Teaching
A Model for Learning CenteredClosing the Loop
Collect
Feedback
Improve
•
•
•
•
IDEA
Student Performance
Peers/Mentors
Other Sources
Interpret
Results
Reflect &
Discuss
Read &
Learn
The System Approach-Putting it All
Together
•
•
•
•
•
•
•
•
New Faculty
Experienced Faculty Member
Department
Institution
Metrics
Institutional Effectiveness
Faculty Development Plan
Year to Year Continuous Improvement
7/21/2015
62
Reflection and Summary
Write down:
–What is one thing that is squared away
or clear from this session?
–What is one thing that is still circling
around or unclear?
–What are you willing to try as a result of
this session?
For more best practices
• Visit our session on Friday
• Thank you all very much for your
participation!
7/21/2015
64