Transcript Peer assessment of group work – management and support via

Peer assessment of group
work using WebPA
Symposium on the Benefits of eLearning Technologies
University of Manchester, in conjunction with the Higher
Education Academy Subject Centres in Engineering, Physical
Sciences, and Mathematics, Statistics and Operational Research
Wednesday 13 April 2011
Neil Gordon
Introduction – why include group work?
• It is perceived as a key skill
• QAA expect it
• Employers expect it
• Accrediting bodies require ““working with others”
• It is a useful skill for our students – one which they will need
after their studies
Practical Issues
• a key problem with group work is assessment.
• Students may consider the process unfair if individual levels of
contribution are not recognised.
• But the tutor rarely has full knowledge of the team interactions to
allow realistic allocation of marks.
• Gathering data to do this from students is not viable with large
cohorts.
• WebPA provides a tool to allow this approach – providing a web
interface to gather data, and implement algorithms to allow the
allocation of individual marks based on a team’s overall mark.
Technology and Group Work
• Use of technology to support team activities is standard in
industry
• In the context of teaching, use of these technologies in
education helps develop “real world” skills, alongside its
educational value
• Many technologies now in common use (especially with Web
2.0) : VLEs, GroupWare and Intranet, Wikis, etc.
Peer (and self) Assessment
• Many potential benefits – especially in ensuring that
students become aware of marking criteria
• It can reduce the marking burden
• Different approaches e.g.
– group agrees a weighting for each group member
– Each student assesses each other student
– Groups of students assess other groups
• Can be problems with the volume of data gathered, as well as
mediation issues
Transparency in marking criteria
• By supplying clear marking criteria to students they can
understand the assessment process
• Particularly important when marking each other
• Explaining this helps make students aware of assessment
processes;
Example Peer marking criteria
• Students asked to assess each
other in terms of:
• Contribute to meetings/online
discussion
• Offer constructive input to
discussions
• Contribution to production of
report
• Overall contribution to
assignment
• Marked in the range 0 (no
input) to 10 (excellent input)
WebPA: a tool for peer assessment
• WebPA provides a tool to automate much of the above with
regards to peer (and self) assessment
• Allows groups to be set up, and for students to assess their
other team members
• Uses a simple algorithm to allocate a weighted mark
• Provides for students to enter marks in a faceless
environment
Issues in peer and self assessment
• Tools such as WebPA allow students to mark each other in a safe
environment - through a web interface which protects the
anonymity of individual students’ marks of their peers from other
students;
• WebPA also applies a suitable algorithm to apportion contribution
based on the student marks;
• One major issue when considering the use of peer assessment is
the volume of data and its management
• Arranging to collect and then use the data becomes a problem
with large classes (e.g. >100): WebPA can manage this process in
an automated way.
Case Study: Computer Science @ Hull
• Regularly use team/group projects
• I have used peer assessment across several modules,
throughout the range of teaching (level 3 to level 6 and from
cohort sizes of 6 up to 180). Nearly 1000 students
• In small modules, originally did by paper, where it was fairly
easy to manage – but was impractical for large modules
Student Views of group work
• Some students would not contribute, and those who were active would
end up doing all the work;
• A desire to be able to choose their own groups, so they were confident
that people would put in the effort;
• Concerns about random groups, where members do not contribute fully
and the workload is unbalanced;
• Concerns about having to rely on others;
• Acknowledgment that group work is essential in the software industry;
• Concerns that criticising the work of a fellow student may mean that
student gives them a low mark in the peer assessment;
• Perceptions that it is more fun than working alone, but more complex
Student views of peer assessment prior to it’s use
• Asked “do you like the idea of this form of assessment” - This
question generated more negative responses, many from
those who has not experienced it in practice
• concerns were about biased marking
• About other students not taking it seriously.
• Several students felt that as long as the teacher was involved
in the marking process, then it would be acceptable.
Solution: WebPA
Adding Data
Student’s views after module
• Majority (70%) prefer to work alone (strongly agree or agree ). BUT
#
• 65% would# use peer and self assessment in other modules
given the choice and
• 67% like# a mix of group and individual work
• 80% comfortable# assessing own performance
• 69% comfortable# assessing peers
Positive views of WebPA
More views on WebPA
When asked “What changes (if any) to WebPA would you like
to see? “:
• 80% of students said no change
• 9% asked for “Provide textual feedback to explain the scores
you gave yourself“
• 1% for a “general feedback to add any other information”
My comments on using WebPA
• Issues where a student had dropped out, but the marks had
been applied before I updated that – so having to reverse
engineer the mark to stop inappropriate marks of 100%
• Would be helpful to be able to import groups (I usually
manage these in Excel)
• But generally –
excellent
Conclusions
• WebPA can allow the scaling up of group work with peer and
self assessment
• It assists the teacher/lecturer by reducing admin associated
with the activity
• It benefits students as it provides a safe environment in
which to carry out the assessment
• It can encourage self reflection and a focus on assessment
criteria
• It can be integrated fairly well into a VLE