Enhancing feedback - Enhancement Themes

Transcript Enhancing feedback - Enhancement Themes

Assessment and Feedback Developing Graduate Attributes

Professor David Nicol Facilitator: QAA Scotland: Assessment and Feedback

Deputy Director Centre for Academic Practice and Learning Enhancement (CAPLE) University of Strathclyde [email protected]

University of Strathclyde,28 th Queen Margaret University, 29 th September 2010 September 2010

Complex Appraisal and Graduate Attributes

Graduates for the 21 st century Implementation in the disciplines – creating the right ‘decision space’ Institutional barriers and opportunities

Graduates for the 21

century

Graduate attributes descriptors The Australian influence (Sydney: Curtin: Melbourne) Scottish HE developments

Questions

What underpins attribute development? 2.

How would you monitor and evaluate attribute development?

How does all this relate to Royce Sadler’s ideas?

The Sydney attributes

The Sydney Attributes

1. Research and Inquiry

graduates will be able to create new knowledge and understanding through the process of research and inquiry

2. Information Literacy

graduates will be able to use information effectively in a range of contexts

3. Personal and Intellectual Autonomy

graduates will be able to work independently and sustainably, in a way that is informed by openness, curiosity and a desire to meet new challenges.

4. Ethical, Social and Professional Understanding

graduates hold personal values and beliefs consistent with their roles as members of local, national and international and professional communities.

5. Communication

graduates will use and value communication as a tool for negotiation and creating new understanding, interacting with others, and furthering their own learning.

The argument

‘…..the core process underpinning all the… [Sydney].. attributes is the students’ ability to think for themselves, to be self-reliant and able to evaluate and make judgements about their own thinking and actions: that is, all the attributes require that students are self regulating learners’ [Nicol, 2010, p4].

Indeed, these ideas are already contained under the attributes descriptors at Sydney.

Reference: Nicol, D (2010) The foundation for graduate attributes: developing self regulation through self and peer processes, Published by QAA for HE.

http://www.enhancementthemes.ac.uk/documents/G21C/Assessment_150910.pdf

The Sydney Attribute Descriptors Research and Inquiry

Be able to identify, define and analyse problems and create processes to solve them Be able to exercise critical judgment and critical thinking in creating new understanding.

Be creative and imaginative thinkers Have an informed respect for the principles, methods, standards, values and boundaries of their discipline and the capacity to question these Be able to critically evaluate existing understandings and recognise the limitations of their own knowledge

The Sydney Attribute Descriptors Information literacy

Recognise the extent of information needed Locate needed information efficiently and effectively Evaluate information and its sources Use information in critical thinking and problem solving contexts to construct knowledge Understand economic, legal, social and cultural issues in the use of information Use contemporary media and technology to access and manage information

The Sydney Attribute Descriptors Personal and Intellectual Autonomy

Be intellectually curious and able to sustain intellectual interest Be capable of rigorous and independent thinking Be open to new ideas, methods and ways of thinking Be able to respond effectively to unfamiliar problems in unfamiliar contexts Be able to identify processes and strategies to learn and meet new challenges Be independent learners who take responsibility for their own learning, and are committed to continuous reflection, self-evaluation and self-improvement Have a personal vision and goals and be able to work towards these in a sustainable way

The Sydney Attribute Descriptors Ethical, Social and Professional Understanding

Strive for truth, honesty, integrity, open-mindedness, fairness and generosity Acknowledge their personal responsibility for their own value judgments and behaviour Understand and accept social, cultural, global and environmental responsibilities Be committed to social justice and principles of sustainability Have an appreciation of and respect for diversity Hold a perspective that acknowledges local, national and international concerns Work with, manage, and lead others in ways that value their diversity and equality and that facilitate their contribution to the organisation and the wider community

The Sydney Attribute Descriptors Communication

Use oral, written, and visual communication to further their own learning Make effective use of oral, written and visual means to critique, negotiate, create and communicate understanding Use communication as a tool for interacting and relating to others

Developing graduate attributes

Requires that students have regular opportunities to:

critically evaluate the quality and impact of their own work, both during and after its production (e.g. academic texts, problem solutions, designs) critically evaluate the quality and impact of the work of others (e.g peers). This is an enabling condition for 1 and is a valued skill sought by employers.

Feedback still has a role to play

Generating own feedback and giving feedback to others is more productive than receiving it It is cognitively more demanding: cannot do this passively Puts students in role of teacher Students actively exercise criteria from many viewpoints Peers see a wide sample of work (alternative approaches, remedies) That recognise that quality is not a fixed attribute – a range of possibilities

Implementing complex appraisal in HE

Make complex appraisal (self/peer review) explicit Consider a whole-institution approach Make implementation easy – many ways to do it, many pitfalls and its management can be time consuming Bring the students on-board – clarify their role as feedback producers Address the NSS at the sector level Collaborate and share expertise across Scottish HE

1. Make core processes explicit

Course learning outcomes rarely state: At the end of this module: …you will be able to evaluate critically the quality and/or impact of your own work …you will be able to evaluate critically the quality and impact of the work produced by others

Benefits of this approach to attributes development

it identifies a core focus for curriculum design and an efficient solution to attributes implementation addresses co-curricular activities where students must evaluate their own performance without the help of the teacher supports transfer of attributes and skills to new contexts: this depends on self-monitoring and self-assessment it aligns attributes with most university policies which emphasise autonomy and independence makes mapping of attributes easier – specific attributes brought into play by fine fine-tuning of task parameters/learning activities (e.g. ethical awareness, internationalisation)

2. Develop an institutional approach

Texas A&M University (TAMU) see peer review as not only developing higher order thinking but also student writing skills At TAMU, over the last 3 years, over 15,300 students have engaged in peer review in 340 courses led by 368 instructors TAMU use Calibrated Peer Review software developed by UCLA (this is widely used across US) TAMU provide a dedicated support service to help staff use the software and introduce it to students

MAJORS USING CPR at TAMU

TM Accounting Ag. Economics Animal Science Anthropology Archeology Biochemistry Biology Botany Bus. Admin.

Ctr. Acad. Enh.

Chemistry English Ed. Curriculum Ed. Psychology Engineering Film French German Kinesiology Learn. Comm.

Math Microbiology Nutrition Physics Poultry Science Psychology Reading Secondary education Vet Integr. Bio Sci.

Wildlife and Fish Sci.

Zoology This slide taken from a presentation by Dr. Wendy Keeney-Kennicutt, Calibrated Peer Review Administrator, TAMU

3. Make self/peer review easy to implement

Many ways to implement peer review – disciplinary focus and educational process (e.g. rubrics, calibration, responsiveness, self-review etc.) Raises many issues (grading and development: collaboration versus collusion) Time consuming to implement when numbers are large

Peer Review in Education Evaluation [PEER]

PEER project is funded by JISC (July 2010) and led by University of Strathclyde. The aims are to: review evidence base for peer review develop educational designs for peer review (and self-review) identify software support for peer review pilot implementations of large-scale peer review produce guidelines for the HE/FE – why do it, how to do it, pitfalls, and software possibilities See http://www.jisc.ac.uk/whatwedo/programmes/elearning/ltig/pe er.aspx

4. Bring the students on-board as feedback producers

4. Bring the students on-board

5. Address the NSS

A distorting influence….!

Investigate what the NSS is measuring Consider developing and piloting a Scottish National Student Survey alongside the NSS

Share ideas across the

Share ideas across the Scottish HE sector

Scottish HE sector This can begin today

….

Some of my Publications

Nicol, D (2010) From monologue to dialogue written feedback in mass higher education. Assessment and Evaluation in Higher Education. 35(5), 501 -517 Nicol, D and Draper, S (2010), A blueprint for transformational organisational change in HE: REAP as a case study (see reap.ac.uk website) Nicol, D (2009), Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, Scotland ( http://www.enhancementthemes.ac.uk/documents/firstyear/FirstYear_Transform ingAssess.pdf

Nicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher

Education, 34(3), 335-352

Nicol, D (2007), Laying the foundation for lifelong learning: cases studies of technology supported assessment processes in large first year classes, British Journal of Educational Technology, 38(4), 668-678 Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1), 53-64.

Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment, Routledge.

Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218.