Transcript Slide 1

Ivan Moore
Higher Education Consultant
Learning and Teaching
Evaluating the student
learning experience
Ivan Moore
HE consultant
[email protected]
Centre for Excellence in Enquiry Based Learning
Ivan Moore
Higher Education Consultant
Learning and Teaching
Objectives
By the end of the session, you should:
1. be able to describe a range of approaches to
designing an evaluation form, based on
internationally recognised methodologies;
2. have experienced an activity through which you
will have begun to develop a customised
evaluation methodology;
3. have experienced an approach to evaluating
the student learning experience;
Centre for Excellence in Enquiry Based Learning
What is evaluation?
• It’s something we talk about in Educational
Development. It is important for Learning
and Teaching but we often don’t do it very
well……or in time to do much about it.
Somehow, we often seem to not be able to
find the time to evaluate what we are
doing or the effects of our interventions. It
seems just a little too difficult to do, or we
have other priorities.
The purposes of evaluation
• Evaluation for accountability
(measuring results or
efficiency)
• Evaluation for development
(providing information to help
to improve practice)
• Evaluation for knowledge
(to obtain a deeper
understanding of some
particular area of practice such
as student learning or change
management)
• Chelimsky (1997)
• Summative
• Formative
• Research
Why should WE evaluate?
• Because it is part of our professional
practice
Principles of evaluation
• An integral part of our teaching practice
• An ongoing process, so that we learn from
systematic reflection
• Should be participatory
• Should enable us to make appropriate
modifications along the way
• Should enable us to make judgments on
specific sessions, but also to draw out wider
implications
Purposes of evaluation
• Mike Prosser
• Quality Assurance
– student satisfaction
– the mean score is important as a measure of
quality
• Quality Enhancement
– Student conceptions/how they experience the
course
– The deviation is important: more focused view
Purposes of evaluation
• Mike Prosser
• Is the learning environment/teaching
approach having any influence on student
conceptions/approaches?
• A student experience survey is more
important than a student satisfaction
survey
Approaches
• Off the peg questionnaires
– Learning Styles (Honey and Mumford)
– Motivated Strategies for Learning
– Learning and Study Strategies Inventory
(LASSI)
– Approaches to Study Inventory
– Course Experience Questionnaire
(Ramsden)
The Course experience questionnaire
(P. Ramsden)
•
•
•
•
•
Designed as a performance indicator
24 statements relating to 5 aspects
1 overall satisfaction statement
Research-based
Drawn from statements made by students in
interviews
• Students with positive responses take a deep
approach
The five sub-scales
•
•
•
•
•
Good teaching
Clear goals and standards
Appropriate assessment
Appropriate workload
Generic skills
Enquiry-Based Learning
• Learning is driven by a process of enquiry
– Students determine the lines of enquiry
– They identify what they need to know, how
they will learn it, how they will know that they
have learned it
– They will share this learning with others
– They have a range of support resources:
• Online; paper-based; equipment; room; other
students; the tutor
Enquiry-Based Learning
• Students
– Academic development:- information skills,
analysis, synthesis, evaluation, hypothesis
– Professional development:- teamworking,
leadership, shared goals, communication
– Personal development:- taking responsibility,
planning, problem solving, confidence
• Change of approach
– Locus of control, motivation
From Ramsden CEQ
• EBL has helped me develop my ability to work as part
of a group
• I have usually had a clear idea of where I was going
and what was expected of me
• I have found EBL motivating
• The EBL tutors motivated me to do my best work
• EBL has helped sharpen my analytical skills
• There has been more assessment of what I have
memorised than of what I have understood
• As a result of EBL, I believe that it is more important to
have found new ways of thinking than to have gained
specific knowledge about the subject areas
Designing an evaluation
questionnaire
• In groups – 30 minutes
Outcomes based design
• Agree up to 4 goals/objectives for your group
• For each, design 4 statements that will help to
determine if the goal is being achieved
• Record the goals and statements on a flip chart
Approaches
• Scriven
– Goal – free evaluation
• EBL has helped me develop my ability to work
as part of a group
– Presupposes and leads the student
• What parts of the module motivated you most?
– (ok so it presupposes motivation, but not EBL)
• Why?
– Illuminative evaluation
Goal-free evaluation
• In pairs
• Choose another pair on which to focus
• Devise up to 5 questions that you might ask
students to answer that might provide
information on what they are experiencing
• Record your questions
• Compare them with the other pair
• Discuss what information you might receive from
these questions
• Draw up your conclusions for feedback
Context-free evaluation
• Since the beginning of the year:
• What skills, if any, have you developed?
• What helped you to develop these skills?
• How have you changed the way you
study?
• What prompted you to make these
changes?
Other methods
• Focus groups
• Structured interviews
• Continuous feedback
– What did you find most difficult/confusing
today?
– What things did you find helped you learn
last week?
– What should I:
• Start/stop/continue?
The three minute paper
• What was the most useful of meaningful thing
you learned during this session?
• What question(s) remain uppermost in your
mind as we end this session?
• What was the ‘muddiest’ point in this session?
• What would you like me to stop doing?
• What would you like me to start doing?
• What would you like me to continue doing?
Other instruments
•
•
•
•
•
Kolb’s Learning Style Inventory (LSI)
Honey and Mumford’s LSQ
Sternberg’s thinking styles
Felder’s Index of Learning Styles
Weinstein’s Learning and Study Strategies
Inventory (LASSI)
• Entwistle’s Approaches to Study Inventory
Ivan Moore
Higher Education Consultant
Learning and Teaching
Evaluating the student
learning experience
Ivan Moore
HE consultant
[email protected]
Centre for Excellence in Enquiry Based Learning