Transcript Document

The Evaluation and Enhancement of Teaching
James Wisdom
Visiting Professor in Educational Development,
Middlesex University
[email protected]
Possible confusions:
I will be using UK examples
“Module, unit, course, programme”
• module, unit, course – parts of a programme
• course, programme – the whole degree experience.
“Evaluation” (review of our teaching)
“Enhancement” (improvement of our teaching)
“Assessment” (our judgements on our students’ work)
The enhancement of teaching?
The enhancement of student learning?
These two are not the same.
Importance of research into the enhancement of
student learning
This lecture will be in four parts:
1. How we usually evaluate teaching now
2. Other processes we could use to evaluate
teaching
3. How HE institutions and others are using
evaluation processes
4. All through, I will be asking how good these
processes are at enhancing teaching
Typical or traditional methods
a) End of unit questionnaire (created by the teacher
or department, or for the whole institution))
b) Appointing a student representative to sit on the
Course Committee
c) Comparative analysis of data about each unit, e.g.
i. Proportions of students leaving the unit
ii. Assessment results
iii. Comments by external examiner
iv. If web-based - records of student activity
Other traditional but less usual methods
e) Peer (colleague) observation
f) RateMyProfessors.com
g) Applications for promotion to Teaching Fellowships
h) Discussion forum if web-based
How might any of these methods help to enhance
teaching?
a) End of unit questionnaire:
Widespread, but ineffective
Amateur design, un-researched questions
Defensive reaction to institutional quality processes
Timing – students uncommitted
Poor educational model – focussed on satisfaction
with teaching delivery
But – can lead to discussion and thought
An element in the annual unit (module) report and
future plan.
b) Student representatives on course
committees:
Need to give students training and time to do
this job well
c) Analysis of data:
Yes, if there is an annual reporting and forward
planning process for each module
d) Peer observation
If voluntary - very effective
For quality of teaching – do not use subject
specialists as observers
e) RateMyProfessors.com
USA and UK – claims 1.5m professor ratings
Scoring: Helpfulness, Clarity, Easiness, Interest level
before, Textbook use, Appearance.
And Reviews
Consumer response to teacher performance
g) Applications for promotion to Teaching Fellowships
Self-evaluation valuable
Real impact is from the whole scheme
h) Web-based discussion forum
Suitable for “blended learning” (part face to face, part
web-based)
Too many traditional methods adapted for the web
We are slow to devise new mechanisms for evaluation
of web based courses
Conclusion from the first section:
Where the focus is on teacher performance, teaching
delivery and institutional quality assurance:
The traditional methods are poor for enhancement
But can be improved by discussion (with experts, peers
and students) and module reporting and planning.
Section 2 – Other processes
The importance of focussing on student learning,
not teacher performance, and of using wellresearched instruments.
a) A student consultation process (qualitative)
b) The Course Experience Questionnaire (CEQ)
c) The Approaches and Study Skills Inventory
(ASSIST)
d) Student Evaluation of Educational Quality
(SEEQ)
e) National Survey of Student Engagement (NSSE)
a) A student consultation process
Whole programme, in year groups
External facilitator
The “pyramid” discussion process
We are interested in anything which is affecting the
way you are learning this course
First - on your own:
Positive points, which we should keep in future
Negative points, with positive recommendations for
change
Then in groups of four
Then groups report ideas to the full assembly, for
debate and recording
Produces high-quality information, easy to use for
enhancement
Picks up relationships between units or modules
Notices significance of course organisation
Student-led data – a different agenda to
lecturers’.
Students understand their learning processes as
a result.
Topics raised at a Final Year Fine Art Consultation
The emphasis of the
course
Tutoring and feedback
The organisation of the
modules
Overseas students
Art in the modular
system
The private view
The faculty building
Induction sessions
Assessment and learning
contracts
Employment or life after
college
Individual modules
Miscellaneous points
Relations with the staff
Same question can be used for a unit or module,
during its progress (not at the end)
Or as a 5 minute, two-question paper:
Three good things which we should keep
Three things which we should change, with
suggestions
b) The Course Experience Questionnaire
(Paul Ramsden)
Course = Programme or major area of study.
36 Questions on:
Good Teaching
Clear goals and standards
Generic skills
Appropriate assessment
Appropriate workload
(Emphasis on independence)
Course Experience Questionnaire
Description of student’s perceptions of key aspects
of the whole learning experience
Perceptions can be positively changed by intelligent
course design (constructive alignment) leading to
increased likelihood of positive outcomes
Can be used just for units
Extensive use since the early 1980s. Australian
national quality assurance process
See the Institute for Teaching and Learning,
University of Sydney
c) The Approaches and Study Skills Inventory for
Students
(Noel Entwistle - Publications pages of “Enhancing
teaching-learning environments” Edinburgh University
www.etl.tla.ed.ac.uk/)
Questions on:
What students understand as “learning”
Reasons for entering higher education
Preferences for different types of course and teaching
Their “approaches to studying”
• Taking a deep approach
• Taking a surface approach
The effectiveness of their strategies for studying
Not a student’s self-description of permanent
characteristics.
Approaches to studying will vary according to the
design of the unit or programme –
We can stimulate more to take a deeper approach
by forms of course design and teaching.
Students’ understanding of the nature of learning
can be influenced, with positive effects
Extensively used since late 1970s.
d) Student Evaluation of Educational Quality
(Herb Marsh)
• Learning/academic
value
• Instructor enthusiasm
• Individual rapport
• Examinations/grading
• Overall rating
•
•
•
•
•
Organisation/clarity
Breadth of coverage
Group interaction
Assignments/readings
Workload/difficulty
SEEQ
Devised in the early 1980s.
Extensively used in the USA.
Focussed on classroom performance of teaching,
more than analysis of contextual factors influencing
student learning
Best used with follow up conversation with
educational advisor.
e) National Survey of Student Engagement
Since 1998.
Based on factors known to relate to outcomes.
The way teaching and programmes are organised
can increase the number and quality of educationally
purposeful activities
Questions relating to:
• Learning activities – such as contributing to class
discussion, doing work which requires multiple
sources, discussing ideas outside class.
• Intellectual activities
• Amounts of work, reading, writing, private study.
• Activities outside and around college
• How your college supports you
• Quality of relationships at college
• Personal development
• And many more
NSSE
400,000 students completed it in 2010
Reports at institutional level
Comparisons with national and selected colleges.
Recognising that enhancement work is institutional,
not just by the tutors alone.
Conclusion from the second section:
Research into the student learning experience is
valuable and productive for enhancement.
It reveals the importance of much more than teacher
performance, and reveals the significance of the
design of learning processes and the role of
assessment.
Report all findings back to students and engage in
discussion about them
Student engagement is an interesting approach to
synthesising the two positions.
Final section:
How HE institutions and others are using
evaluation processes
Support for the development of the scholarship of
teaching and learning.
Promotion to “Teaching Fellow” etc
Evidence of a scholarly approach to enhancing
student learning
UK: National Student Satisfaction Survey
60 – 70% of final year undergraduates reply
25 question “hybrid” of Course Experience
Questionnaire and other questions.
Data published on “Unistats”
To use information to create a competitive
market for student fees, and so to enhance
teaching.