Transcript Writing ePlatform: Developing an innovative feedback tool for

Writing ePlatform:
Developing an innovative feedback tool for
students’ writing to expand assessment for,
of, as learning.
Outline
(1) Introduction: What is the Writing ePlatform
(2) Theory behind design:
 Assessment of, for, and as Learning
 Technology Enhanced Learning
(3) The eLab & eTutor: What are they?
(4) Planning design, and Pilot.
(5) Unintentional Outcome: Learning Analytics
& Assessment for Learning.
(6) What’s Next & Conclusion
Writing ePlatform
The Writing ePlatform is a set of tools
that assist KS3 students with their
writing, allowing them to take a
discovery-based approach to accurate
and fluent English.
Writing ePlatform
Take advantage of online resources to help
KS3 students and teachers shift from a
machine- or teacher-centered pedagogy to one
that puts the KS3 students at the center of the
writing process (Milton, 2006, 2011).
Guided by three key principles based on
education and educational technology
researchers such as Anderson, Chickering and
Ehrmann, Siemens, and the TBLT Group (2004,
1996, 2009, and 2012).
Writing ePlatform
The platform should:
(1) assist with constructing the lexicogrammatical and discoursal/rhetorical
knowledge of the target language and the
skills required to access and apply that
language;
Writing ePlatform
(2) encourage reflection and
metacognition, where students are
encouraged in independent learning and
self-confidence; and
Writing ePlatform
(3) develop ‘cognitive apprenticeship’,
where coaching and modeling occur, and
where scaffolding is provided to support
language learning.
Assessment of, for,
and as Learning
Using Classroom Assessment
for Differentiating Learning
Assessment As Learning (Earl, 2003)
Assessment As Learning (Earl, 2003)
Features of Assessment
of, for, and as Learning
Assessment As Learning (Earl, 2003)
Assessment of Learning
What
roleitwould
ePlatform
in to
How will
certifythe
learning
and play
report
summative
assessment?
students and
teachers students’ progress
in writing?
Assessment as learning
How
canways
the ePlatform
encourage
In
what
can the ePlatform
students toindependent
become active,
engaged, and
encourage
learning?
critical assessors of their writing?
Assessment for learning
How will the ePlatform enable/enhance
formative assessment?
How can the ePlatform be interactive,
providing assistance as part of/during
the assessment?
In what ways could the ePlatform assist
teachers with collecting data so that they
can modify the learning work for their
students?
Technology
Enhance Learning
Content, Pedagogy, and Technology Knowledge
A teacher’s knowledge
of the content to be
taught
C
A teacher’s knowledge
of pedagogy
P
A teacher’s knowledge
of technology
T
C
P
T
C
P
T
Technological pedagogical
content knowledge
C
P
T
Mirsha, P. & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge.
Teachers College Record, 108(6).
E-learning vision
ICT integration and curriculum design
ICT integration and curriculum design
Pedagogical Content
Knowledge
Technological
Content Knowledge
Technological
Pedagogical Content
Knowledge
Technological
Pedagogical
Knowledge
Assessment Roles and Goals
with Writing ePlatform
eLab &
eTutor
eLab
The eLab is a webbased platform where
students can submit
their writing to receive
feedback and
suggestions on how to
improve their written
English
eLab
and the writing process
Student writes a draft
Student submits draft to eLab
eLab analyses student’s text
eLab identifies problems in writing
Student receives feedback
Student rewrites/adjusts draft
Student sample
Nowadays, English is more and more important. It is very difficult to learn.
How can I speaking English well?
Nowadays, English is more and more important. It is very difficult to learn.
How can I speaking English well?
Using the wrong ending is a very common error. Be careful with words
ending in –ing.
A common mistake may look like this:
How can I speaking English well? 
The correct way is:
How can I speak English well? 
eLab
and the writing process
eTutor
The eTutor is a webbased tool that offers
comprehensive advice
on writing accurate and
fluent English.
Planning, Design,
& Pilot
Planning, Design,
& Pilot
Compiled rules based on previous research at UST + rules
based on research data provided by EDB.
Example: Although + but in a sentence
Although the food was not great but I felt very happy
that I was able to cook the meal by myself.
Although the food was not great, I felt very happy
that I was able to cook the meal by myself.
Planning, Design,
& Pilot
Rule:
!{well|such|known}{as|since|although}<{(CC)|(IN)}> *<0-6> (CC)<{but}>
Planning, Design,
& Pilot
Planning, Design,
& Pilot
Planning, Design,
& Pilot
Planning, Design,
& Pilot
1. Piloted program in 7 schools
Student survey
Teacher observation report
2. Analyzed a learner corpus
4 Topics
400 papers from each topic
Planning, Design,
& Pilot
Student Survey (Anonymous)
336 responses
6 schools (1 school could not complete
pilot due to time constraints).
Planning, Design,
& Pilot
Responses to the question ‘What could be done to improve the Writing eLab?’
can be categorized in the following ways:
FALSE POSITIVES
ERROR IDENTIFICATION
INSTANT PROMPTS
INSTRUCTIONS
VOCABULARY
PERFORMANCE
Analysis of Learner
Corpus
Although + but in a sentence (start with a capital, end in a full stop)
Hits: 22
Issues: None.
Other error type?: None.
False positives: None.
Correct Identification (samples):
{Although they were so old but }they were still very strong.
{Although it looked like a high-class hotel but }I paid fifthty-five dollars only.
{Although it is quite expensive, but }it is a fun way to learn English as we can
know more about the culture of the place at the same time.
{Although we can't said Japan's language, but }we can said english with they.
Analysis of Learner
Corpus
Error Rule 1: A sentence / clause has more than 2 verbs
Hits: 23874
Issues: This rule is very problematic – too many false positives. Detection issues.
Other error type?: None.
False positives:
During the past summer holiday, my family and I {went to a village in the
Mainland China to visit }my grandparents.
When we arrived, grandpa and grandma {were in excitement since we had}n't
visit them for a long period of time.
They {took out a watermelon from the fridge to greet }us in ecstasy.
eLab
A simple
interface
Choice for
different
end-users
Access to rich
feedback
eLab
and feedback
eLab
and feedback
eLab
and feedback
eLab
and feedback
eLab
and Word Tag
eLab
and vocabulary profile
eLab
and vocabulary profile
Unintentional Outcome:
Learning Analytics
&
Assessment for Learning
Learning Analytics
What is Learning Analytics?
Learning analytics is the measurement,
collection, analysis and reporting of data
about learners and their contexts, for
purposes of understanding and
optimising learning and the
environments in which it occurs.
-- Wikipedia
Back to Assessment
Back to Assessment
62
600
662
2
15
17
18
19
23
27
31
33
36
49
50
58
80
84
85
49
97
58
5
6
2
8
1
9
76
5
2
6
0
52
6
382
82
66
19
5
312
11
36
1
13
4
23
11
20
11
30
1
645
43
5
54
11
13
1
10
15
0
8
7
6
2
0
10
0
185
13
14
18
0
5
0
3
0
0
29
0
0
3
0
6
1
92
10
18
32
41
56
58
70
91
100
101
135
136
137
138
155
158
2
3
7
5
0
4
3
80
103
47
20
4
6
7
10
18
319
0
1
5
0
0
1
1
16
58
23
3
0
0
0
0
25
133
0
0
0
0
3
0
0
0
5
11
2
0
0
0
1
15
37
0
2
1
0
0
1
0
2
15
6
14
0
2
5
1
7
56
Total
392
2006
2398
190
1799
1989
582
3805
4387
62
111
76
5
11
2
11
1
9
105
5
2
9
0
58
7
474
125
71
73
16
325
12
46
16
13
12
30
17
22
11
40
1
830
187
182
149
21
336
14
57
17
22
117
35
19
31
11
98
8
1304
2
5
8
5
0
5
3
82
118
53
34
4
8
12
11
25
375
0
1
5
0
3
1
1
16
63
34
5
0
0
0
1
40
170
2
6
13
5
3
6
4
98
181
87
39
4
8
12
12
65
545
Total
Topics categorized into two group types
Describing = 1 + 4
Giving Advice = 2 + 3
Old Rules
Example
49
482
531
Notes
141
1317
1458
Giving advice
Describing
4. My Three-day Camping Trip
3. Give Advice About a Teenager's Misbehaviour
2. Fun Ways to Improve English
1. An Enjoyable Trip
330
1406
1736
New Rule Errors Identified
Old Rule Errors Identified
Total Errors Identified
promblematic: error in classification
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
I {think my mum do} had a great meeting with them.
I took {many photo }because I love Hello Kitty so much.
I'm so glad that I {can had }that happy trip and I think it might be my most unforgetable memory.
The beach is clean and beautiful, it {can makes }people relax.
Finally, we {can learning }English and playing game togather.
After you joined it, you must feel your speaking level a much {more improve}.
Also, we can watch movies or tv are talking {about improve }english.
Also, If {he have }problem in homework, you should tech him, help he to get the good result.
{It have }many ways, I will give you some idea.
We went there {to bought }some souvenirs and dolls.
It's just like having the writing {everyday}.
You can write blog {everyday or }every week.
Thirdly, you can improve {you English language }skills by writing more.
You just need to pratice {more and more }in you daily life!
I've already heard that Macau have so {many delicious food}.
{It located }in countryside.
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
accurate
Also, that trip was about sightseeing, it could {let me learn }more about an other country things !
{Although they were so old but }they were still very strong.
I am looking forward to going to {another places }tomorrow.
I love Japan {very very }much.
David’s poor result {may due to }the lack of study.
{She just like }a friend I've made for many years.
and I will go to {the another }country to take photos.
{There had a lot }of exciting games, my family and I had so much fun in there.
I took {many photo }because I love Hello Kitty so much.
There are {many big museum }and old buildings there.
I am {very enjoy }this trip.
When we had a spa, I {felt very relax }too.
I {am very enjoy }this trip.
It {was very enjoy}.
I can even eat the whole chicken soup as it was {very delicious}.
There were some funny mask {and also }some delicious snack which sold on the street.
New Rules
eLab
and Assessment for Learning
There is a lot of data.
76
4
8
29
105
12
117 accurate
Example
Notes
Total
Giving advice
Describing
Topics categorized into two group types
Describing = 1 + 4
Giving Advice = 2 + 3
4. My Three-day Camping Trip
To + 'ed':
3. Give Advice About a Teenager's Misbehaviour
2. Fun Ways to Improve English
1. An Enjoyable Trip
eLab
and Assessment for Learning
to{(VBD)|(VBN)}
We went there {to bought }some souvenirs and dolls.
School #1
School #2
School #3
School #4
School #5
School #6
School #7
Total
11
8
19
0
1
0
1
0
1
2
12
6
1
8
0
1
27
49
12
6
0
0
19
8
1
46
Total
Giving advice
Describing
Topics categorized into two group types
Describing = 1 + 4
Giving Advice = 2 + 3
My Three-day Camping Trip
Give Advice About a Teenager's Misbehaviour
To + 'ed':
Fun Ways to Improve English
An Enjoyable Trip
eLab
and Assessment for Learning
to{(VBD)|(VBN)}
1
0
0
1
0
1
0
3
13
6
0
1
19
9
1
49
What’s Next?
What’s Next?
• Analyse additional sample writings
from KS3 students to develop a better
understanding of what type (and scope)
of feedback is required for students
when using the program;
• Provide more accurate and meaningful
feedback prompts (and vocabulary
profiles) for both students and teachers
to use;
What’s Next?
Expand the project to create a userfriendly interface to generate reports for
learning analytics.
Thank You.
Sean McMinn
[email protected]