Transcript Faculty Evaluation System: Student Ratings

Faculty Evaluation Systems:
Student Evaluations of Faculty
What are we measuring?
“The evaluation of teachers
is a mark of a good college.”
Ernest Boyer, 1987, p.155
Introduction:Community
College Faculty Evaluation
• Community College –Teaching Mission
– Diverse mission of a comprehensive
community college
– As a Community College Teaching is
at the Center of our Purpose.
– RSCC a Teaching Institution – First &
Foremost
– Can we Measure (Evaluate) Teaching
Excellence?
Defining Teaching Excellence:
Three Major Emphases
• 1) Input
– Student, teacher and course characteristics
• 2) Process
– Classroom atmosphere, teacher behavior,
student learning activities, course
organization, & evaluation procedures
• 3) Product
– End-of-course learning, affective/cognitive
change, skills acquisition
– Long-term learning, affective/cognitive
change, skills acquisition
Presentation Outline
• Part 1: Faculty Evaluation Systems
(Purpose)
• Part 2: Faculty Evaluation System:
Student Evaluations of Faculty
(Student Ratings - One Piece)
Part 1: Faculty EvaluationWhat’s the purpose?
Overall institutional purpose is key to
proper evaluation.
• Formative & Summative
– Improve Pedagogical Methods &
Student Learning
– Faculty, Course & Curriculum
Development
– Excellence in Teaching
– Tenure & Promotion
Faculty Evaluation
Purpose: Accountability
Renewed attention to undergraduate
instruction…
– State Legislature, Coordinating and
Governing Boards
– Accrediting Bodies
– SACS
• "The institution regularly evaluates the
effectiveness of each faculty member in accord
with published criteria, regardless of contractual or
tenured status."
• "Standards for all educational programs include
all on-campus, off-campus, and distance learning
programs."
– Source: The new SACS Comprehensive Standards
governing Educational Programs.
Evaluation of College
Faculty
• First student ratings took place in the
late 1800s (Sioux City).
• Last 2 decades, evaluation of college
faculty has increasingly become a way of
life for those in higher education.
• Today, virtually all American colleges
assess faculty performance using
student evaluations along with other
instruments.
Faculty Evaluation Systems
• Comprised of multiple measures
– Student Evaluations of Faculty
– Peer Evaluations
– Self Evaluation
– Supervisor Evaluation
– Exit Interviews & Alumni Surveys
– Teaching Portfolios/Dossier
– Classroom observation
Part 2: Student Evaluation
of Faculty (Ratings)
• Most widely used structured method of
evaluating faculty in higher education
• Integral to most systems is the
“instrument” by means of which student
opinions & observations are collected.
• Student appraisal of instruction seems
appropriate because, students are the
only ones who observe teaching for an
entire course.
Student Evaluations of
Faculty (Ratings)
• Do these ratings accurately portray
teaching effectiveness?
Student Evaluations of Faculty
Quality of Ratings?
Legitimacy?
• Higher education tends to question
the quality and legitimacy of the
info. collected through student
ratings of faculty ….
Concerns about
Evaluations (student ratings)
No surprise a significant # of studies focused
on objectivity and validity of such evaluations
–2,000 + published studies
• Concerns include:
–
–
–
–
Low validity & reliability
Appropriateness of items (instrument)
Correlation between grades & evaluation
Measure “How good an entertainer rather
than how good his/her teaching skills are”
– Popularity contests that reward classroom
entertainers & easy graders
– Students don’t take these seriously
Concerns Continued…
• Student opinion can be affected by
variables beyond the faculty
member’s control..
– Class size
– Course content
Student Evaluations of
Faculty – Grades & Ratings
• Research findings support the
notion that instructors CANNOT
purchase FAVORABLE student
ratings through easy grading.
• Research also indicates the
relationship between grades &
ratings is a function of the better
achieving students’ greater interest
and motivation.
Despite reservations…
• More than half of the higher
education institutions in the nation
continue to use student appraisals
of faculty within their faculty
evaluation systems.
• Student perceptions of end-ofcourse evaluations indicate that do
take these seriously, but they need
to understand how the results are
used…
The Quality of Evaluative
Information
• Reliability – Free from error (Computing
& Measurement- accurate and
consistent results)
• Validity – Meaning (Measuring what you
think you are?)
• Generalizability – Representativeness
(Whose opinions are reflected by data?)
(Does sample of info. portray the totality
of the person’s teaching?)
• Utility (What purposes can the data
serve?)
The Instrument:
Asking the right questions
• Definition of teaching quality (input,
process & product)
• Purpose of evaluation
• Usefulness of evaluative info.:
Information should be detailed,
diagnostic, and focused on specific
teaching behaviors and course
characteristics to derive maximum
benefit of information.
The Instrument:
Evaluative Questions
• Two types of evaluative questions found
in most student evaluation instruments
– 1) Student reaction to instructor traits or
behaviors; characteristics of course material;
and social and physical environment
– 2) Student outcomes (Progress toward
general or specific educational goals)
– 3) You might find a third type: Demographics
The Instrument:
Format of Questions
• Whether outcomes or trait
questions…
– Multiple Choice/Scaled Responses
• (Best # of points for a particular scale –
empirical question)
• Some better than others
– Open-ended
• Augment multiple choice items
• Helpful & constructive recommendations
for improvement
Areas of Measurement:
Grouping the Questions
• Global: require high inference (making judgments)
– Instructor
– Course
– General (instructional climate)
• Specific Items: (descriptive & diagnostic)
–
–
–
–
–
–
Course management
Instructor characteristics & style
Reading material/assignments
Exams
Student behavior (effort, involvement, etc.)
Student outcomes of instruction
• Open-ended:
High Inference Questions
vs. Specific Questions
• Specific multiple choice/open-ended
probing questions
• Diagnostic information
• Most useful for course/instructor improvement
• Very general or high inference rating
scale items
• How would you rate this instructor’s overall
teaching ability?
– Little helpful diagnostic information
– Most useful for administrative decisions,
advisement of students…
RSCC Faculty Evaluation
Instrument - completed by students
• Examples:
• Question #9: The Instructor’s examinations
and/or assignments reflected the content and
emphasis of the course.
• Question #10: The instructor returned
examinations and/or assignments within a
reasonable time and with appropriate
explanations.
• Interpretation?
– Self or course improvement?
– Exam or assignment improvement?
• Question #9 reworded: The instructor’s
examinations reflected the course content.
Flexibility of Instrument
• Many institutions tried a single
institutional instrument
– Criticism from faculty and students as
irrelevant to the specific needs of particular
programs and/or courses
– Solutions:
• Leave room on the questionnaire for the instructor
to add his/her own items
• Module approach – different instruments for
different programs/courses (labs, distance
learning courses, clinicals, small or large lecture
etc.)
Flexibility of Instrument
• Does one size fit all?
Different instruments for different types of
courses?
–
–
–
–
–
–
–
Lab
Clinical
On-line
IDEA Room
Small lecture or large lecture
Seminar
Other
Utility of Assessment
Information depends on…
• Quality of instrument
• Reporting
• Relies on faculty perceptions of
evaluation process –quality issue
• Student perceptions of the
evaluation process
Summary
• In sum, then, provided that data are
gathered carefully, reported
appropriately, and interpreted
judiciously, student evaluations
appear to make a useful
contribution to course improvement,
faculty/self development, personnel
decisions, and, possibly to student
learning.
Remember
• Research tells us that Students’
early evaluations are quite stable
and instructors should be aware
that the first few days of class may
be very important in determining the
eventual image students evaluate
at the end of the course.
Questions?
Can we measure teaching
excellence?
Future direction?
(Planning/Implementation Time)