Transcript PowerPoint Guidelines

Written feedback on language assignments:
What students need, what they get,
and what they make of it
María Fernández-Toro
The Open University
Languages in Higher Education 2010
Why is assignment feedback worth
investigating?
 Teaching through assignment feedback
 Students acknowledge its importance
(Nicolson & Gallastegi, 2006; Furnborough & Truman, 2009)
– In distance learning: prevents sense of isolation
– In any context: opportunity for targeted individual feedback
 Impact of assignment feedback on motivation (Dörnyei, 2001)
 Impact on retention
What is feedback?
 ‘Feedback is information about the gap between the actual level
and the reference level of a system parameter which is used to
alter the gap in some way’ (Ramaprasad, 1983).
Awareness of a gap between present and desired level of
knowledge, understanding or skill, and the action taken to close
it (Black & Wiliam, 1998; Ramaprasad, 1983)
 Bridging the gap between actual and desired performance
levels (Hunt, 2001)
 Retrospective vs. future gap-altering feedback (Walker, 2009)
Aim of the study
To replicate a previous study of written feedback in Technology
assignments conducted by Walker (2008, 2009), looking at Spanish
Language assignments instead.
Research questions:
1.
To what extent can the methodology developed for studying written
feedback in Technology be applied to the study of written feedback in
Languages?
2.
To what extent are tutors’ approaches to written feedback in
Technology and Languages comparable?
3.
To what extent are students’ responses to written feedback in
Technology and Languages comparable?
4.
What can be learned from this comparison in order to improve the quality
of the written feedback given to students on their assignments?.
Method

Focus on two Spanish courses offered by the Open University :
L204: Viento en Popa (Level 2)
L314: A Buen Puerto (Level 3)

Sample:
L204: 34 scripts and feedback forms from TMA 4 (12 tutors)
L314: 38 scripts and feedback forms from TMA 5 (13 tutors)

Tutor feedback data:
72 scripts & related feedback forms  4000 comments/annotations
Each comment/annotation coded for category and depth (Brown and
Glover 2006)

Student response data:
Telephone interviews with 20 of the students whose commented work
had been analysed, discussing specific annotations/comments made on
their assignment (Student response study funded by COLMSCT)
Student interview results
Initial response to assignment feedback


Once the eTMA system had notified you that your marked
assignment was waiting how long was it before you collected it?
–
Immediately: 13
–
Not immediately: 7
What did you do with the comments your tutor had made on this
assignment when you first received the assignment back?
–
Look at mark first; read through (x 8)
–
Look at feedback summary first; check errors; disagree/do not understand
feedback (x 7)
–
Contact tutor (x5); Correct errors (x3)
–
Take notes; skim through; relate feedback summary to annotations on script;
print feedback; store on PC (x1)
Tutor feedback - Coding conventions used
Based on Brown and Glover (2006)
Category






comments relating to the content of the answer:
inaccuracies, omissions, irrelevancies, clarifications;
comments relating to skills development:
use of Spanish (accuracy, range, style & register),
structuring an essay;
motivating comments: praise, encouragement;
demotivating comments;
comments making reference to resources – e.g. course materials;
comments making reference to future work.
Tutor feedback - Results
80.0
70.0
60.0
50.0
40.0
30.0
20.0
10.0
0.0
Languages
Category
Ref. to
materials
Ref. to future
study
Demotivating
Motivational
Skills
Technology
Content
Percentage
Category usage compared
Student interview results
Perceived usefulness of feedback on language / content
No data
Feedback on language
is more useful
Feedback on content
is more useful
4
8
6
2
students
students
students
students
Tutor feedback - Results
Average number of comments per assignment
Spanish:
N=72
Technology:
N=106
Spanish: 56.5
Technology: 29.2
On form
On script
On form
On script
 Content
1.2
1.5
1.4
10.6
 Skills
development
6.2
36.2
0.9
5.2
 Motivating
8.1
1.1
3.2
6.2
17.2
39.3
6.1
23.1
 Overall
Tutor feedback - Results
Comments
on summary form
Annotations
on student’s script
Spanish
3% 4%
8%
52%
content
40%
93%
skills development
motivating
Technology
25%
28%
48%
58%
17%
24%
Student interview results
Perceived usefulness of feedback in feedback summary
and on assignment script
None of them
is useful
Annotations on script
are useful
Feedback summary
is useful
1
6
9
4
student
students
students
students
Tutor feedback - Coding conventions used
Depth
(Adapted from Brown and Glover 2006)
Comment
category 
Content /
Skills development
Motivational
 Depth 1
Indicates a weakness
Indicates a strength
 Depth 2
Provides a correction
Describes a strength
 Depth 3
Provides an explanation
Provides an explanation
Tutor feedback - Results
Depth usage compared
80.0
70.0
Percentage
60.0
50.0
Languages
40.0
Technology
30.0
20.0
10.0
0.0
1 (Indicated)
2 (corrected)
Depth
3 (explained)
Interview results
Perceived function & usefulness of Depth 1 feedback

For certain errors, your tutor did not provide a full Spanish
correction. Why do you think that is?
–
accents, spelling etc. require no explanation, students can
correct those themselves
–
–
reflecting on your own mistakes helps you learn
at this level students should be expected to be able to correct
such mistakes
even if the correction is not immediately obvious it can be found
in reference material
tutors do not want to put off students by correcting every single
mistake <affect & motivation>
–
–
Interview results
Individual responses to Depth 1 feedback
No data
Need corrections
Satisfied with amount
of corrections given
4
5
7
4
students
students
students
students
Interview results
Feedback and motivation
 To what extent did tackling this assignment, and the feedback you received
on your assignment, encourage you to want to study the rest of the
course?
–
A lot (x11); mixed feelings (x3); unaffected (x4); put me off (x2)
 Students may engage with praise from their tutor in two possible
ways (Vallerand & Reid 1984):
–
metacognitively (as a reference for evaluating their own performance
and in order to plan future strategy),
–
or affectively (as a means of encouragement through the comforting
feeling that “somebody is there for you”)
Interview results
Use of feedback in subsequent assignments & exam
Not used
again
Used for subsequent
assignment
Used for exam
preparation
7
3
6
4
students
students
students
students
Interview results
Student’s requests for helpful feedback
1/2

Positive feedback = a recurrent theme across the sample

An overwhelming number of requests for more specific feedback
on their language i.e. more corrections, explanations and examples

Comments on content appreciated, but only by a minority

Feedback summary seen to play an important role in the students’
learning:
–
in relation to the balance between encouragement and criticism:
–
as a synthesis of areas to focus upon (helps you make sense of feedback on
script)
Interview results
Student’s requests for helpful feedback


2/2
Meeting their needs as distance learners:
–
provide average mark of the class as a reference
–
important to know that ‘there is somebody there working for you’.
–
encouragement is important when you are learning at home
–
oral feedback also mentioned, even though not relevant to this assignment
Other requests:
–
Use of codes to categorise errors (one request, but mostly detractors)
–
Comments that do not take long to read (even a feedback summary is too
much for some!)
–
Feedback in English preferred by one student (could be an important minority)
Conclusions
Feedback is generally welcome. The variety of responses
indicates that a range of feedback strategies should be integrated
in all formative assessment



Comments on content not perceived as the most relevant to
language students  Two possible approaches:
–
Match criteria to students’ expectations
–
Help learners appreciate the relevance of content criteria
Comments on language need to be more specific
–
Provide corrections (i.e. use Depth 2) wherever possible
–
Delay presentation of correction to allow self-correction where possible
–
Depths 1 and 3 also useful, but only if they are in tune with the learners’
competence (learning dialogue is essential)
Motivational comments are very important for most students
–
As well as being ‘encouraging’ they need to be informative (i.e. Depths 2 or 3)
References
1/2

Brown, E. & Glover, C. (2006). Evaluating written feedback. In C. Bryan &
K. Clegg (Eds.), Innovative assessment in higher education (pp. 81-91).
Abingdon, UK: Routledge.

Black, P. & Wiliam, D. (1998). Assessment and Classroom Learning.
Assessment in Education, 5(1), 7-74.

Fernández-Toro, M.; Truman, M. (2009). Improved learning through
improved feedback on Languages TMAs. COLMSCT Final Report.
Available from
http://www.open.ac.uk/colmsct/activities/details/detail.php?itemId=492fcd7
66c828&themeId=48a94349b1870

Furnborough, C. & Truman, M. (2009). Adult beginner distance language
learner perceptions and use of assignment feedback. Distance Education,
30( 3), 399 – 418.
References
2/2

Hunt, M. (2001). Checking on progress: developing approaches to
giving formal and informal feedback. In L. Arthur & S. Hurd (Eds.),
Supporting lifelong language learning: theoretical and practical
approaches (pp. 165-176). London: CILT / The Open University.

Nicolson, M. & Gallastegi, L. (2006). 2005 Research on TMA Feedback in
beginner languages. Edinburgh: The Open University in Scotland.

Ramaprasad, A. (1983). On the definition of feedback. Behavioural
Science, 28, 4-13.

Vallerand, R.J. & Reid, G. (1984). On the Causal Effects of Perceived
Competence

on Intrinsic Motivation: A Test of Cognitive Evaluation Theory. Journal of
Sport Psychology, 6, 94-102.

Walker, M. (2009). An investigation into written comments on
assignments: do students find them usable? Assessment & Evaluation in
Higher Education, 34(1) 67-78.
A full electronic copy of this presentation (including speaker´s notes)
can be downloaded from:
http://kn.open.ac.uk/document.cfm?docid=13113
For further queries, contact:
[email protected]