Transcript Supporting Pre-service Teachers' Video Project Work

Assessing critical thinking through
scaffolded asynchronous online
discussion
Yiong Hwee, TEO
National Institute of Education
Nanyang Technological University, Singapore
[email protected]
Evaluations & Assessment Conference
Adelaide
November 24-25, 2003
Outline
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
1.
2.
3.
4.
Introduction
Collaborative critique framework
Socratic questioning in Online Discussion
Using Knowledge Community to scaffold &
assess thinking
5. Conclusion
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
2
Singapore Education
Initiatives
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• Thinking Programme & Project Work (PW) in
schools
• Masterplan for IT in Education (1997-2002)
• In-service teacher training at NIE
– Hypermedia design module (Jul 03; 8 weeks)
– Design a hypermedia project
– Engage in Computer-supported Collaborative
Learning (CSCL) using Knowledge Community
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
3
Collaborative Critique
Framework
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• Value of critique of design projects
– Use examples of professional work to establish
standards
– Use examples of students work to define high
and low quality work
(Thomas & Mergendoller, 2000)
• Why use asynchronous online critique?
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
4
Research Question
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• What types of thinking skills and levels of
information processing are exhibited by inservice teachers in a scaffolded online group
discussion?
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
5
Collaborative Critique
Framework
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
In-service teachers critique past year student
hypermedia project in online discussion:
5 – step critique process
1. Identify project process, audience & expertise
2. Justify strengths and suggest improvements
3. Justify weaknesses and suggest improvements
4. Weigh alternatives
5. Transfer to own project
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
6
Thinking Skills
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
•
•
Dimensions of Thinking (Marzano, 1998)
Map of Thinking Domain Framework (Swartz &
Parks, 1994)
• Socratic Questioning Method (Socrates; Richard
Paul, 1995)
All reasoning has a purpose, is based on assumptions,
done from some point of view, and based on
information & evidence; All reasoning leads
somewhere, has implications and consequences
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
7
Socratic Thinking
Prompts
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
A. Probe
Reasons
B. Probe
Viewpoints
C. Probe
Assumptions
D. Probe
Implications
&
Consequences
E. Probe
Clarification
F. Agree/Do
Not Agree
My Opinion is
An unusual
idea is
The indicators/
facts supporting
my opinion are
The evidence
is
What evidence
An improvement to the
suggestion
I am looking
from the point of
view of
Reasons behind
this comment are
This idea is from
Your viewpoint
is
A different
viewpoint is
The assumptions
are
The assumptions
on the availability
of resources are
The assumptions
on possible
constraints are
You are
assuming
A different
assumption could
be
The justification
of this assumption
is
This suggestion
will have
consequences of
If this is the case,
then
Do you imply
that
The
consequences
would lead to
Priority should
be given to
My question is
Please clarify:
Please elaborate
I need to find out
Do you mean
Is the main point
Please give an
example on
To summarize
Is there
Can this
Should
I agree that
I do not agree
that
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
8
Using Knowledge Community to
Scaffold Thinking
Y H Teo
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Assessing critical thinking through scaffolded asynchronous online discussion
9
Using Knowledge Community to
Scaffold Thinking
Y H Teo
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Assessing critical thinking through scaffolded asynchronous online discussion
10
Using Knowledge Community to
Assess Participation &Thinking
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• Evaluation of online participation
– Statistics on reading & writing behaviour
Students’
Names
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
11
Evaluation of types of
Socratic thinking
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• Message units according to Socratic Thinking Types
Thinking Types
Y H Teo
Total Message Units (%)
Probe Reasons
49 (35.3%)
Probe Viewpoints
20 (14.4%)
Probe Assumptions
6 (4.3%)
Probe Implications & Consequences
8 (5.8%)
Probe Clarification
28 (20.1%)
Agree/Disagree
28 (20.1%)
Grand Total
139 (100%)
Assessing critical thinking through scaffolded asynchronous online discussion
12
Evaluation of types of
Socratic thinking
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• Sorting by Scaffold Prompts
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
13
Evaluation of level of
information processing
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Henri (1992) method of coding
• Critical thinking – surface level
– “I do not agree that the quit button is redundant”
• Critical thinking – in depth level
– “A different viewpoint is that the choice of content is excellent.
Pupils can relate to the content chosen since it depicts pupil’s own
daily experience i.e. doing various activities in a day.”
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
14
Conclusion
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Preliminary Analysis
• Promising potential of ‘scaffolding prompts’ of KC
reveals that
• More students take a stand
• Comments were focused
• Comments were longer, detailed, with justifications
• Software-scaffolding enable students to use all
thinking types BUT they are weak in ‘Probing
Assumptions’ (4.3%) & ‘Probing Implications &
Consequences’ (5.8%)
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
15
Conclusion
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• Educators can help students to improve their
thinking in certain areas where they are weak
and further study can help us design better
thinking scaffolds in online discussion forums
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
16
Future Research
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
• How does scaffolding prompts help preservice teachers in online discussion?
• What have pre-service teachers learned and
benefited from producing a group
educational video?
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
17
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
THANK YOU!
[email protected]
(Knowledge Community: www.globalkc.net)
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
18
Critiquing Video Cases
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Research Question 1: How does critiquing
professional productions and past
student video projects help pre-service
teachers’ learning?
Method:
• Content Analysis of discussion log of both
groups
• Online survey questionnaire at end of course:
MCQ & essay type questions
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
19
Critiquing Video Cases
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Q1: My knowledge of what makes a good
educational video was
90
80
70
60
50
Before
Video Cases
40
30
After Video
Cases
20
10
0
V Good
Y H Teo
Good
Average
Poor
V Poor
Assessing critical thinking through scaffolded asynchronous online discussion
20
Critiquing Video Cases
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Q: Seeing & discussing past student projects is a good
way to define what low and high quality work look like
70
60
50
40
After Video
Cases
30
20
10
0
S Agree
Y H Teo
Agree
Not Sure
Disagree
S Disagree
Assessing critical thinking through scaffolded asynchronous online discussion
21
Critiquing Video Cases :
Transfer of Learning
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Q: My group has incorporated some of the insights
and knowledge gained from the online video critique
discussion in our own video production.
80
70
60
50
40
30
20
10
0
Y H Teo
S Agree
Agree
Not Sure
Disagree
S Disagree
Assessing critical thinking through scaffolded asynchronous online discussion
22
Critiquing Video Cases :
Qualitative Results
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Research Question 1: How does critiquing professional
productions and past student video projects help pre-service
teachers’ learning?
•
Free response in survey questionnaire & Content Analysis
• “…have a better idea of how the standards differ and allow me
to gauge where my project stands”
• “…see clearly what are the certain do’s and don’t’s when
producing a video”
• “…give direction and pre-empt mistakes in my own production”
• “learn from more people’s comments rather than just within my
group members”
• “Engaging in video critique by looking at people’s comments
and critiquing is a fun activity”
Y H Teo
Assessing critical thinking through scaffolded asynchronous online discussion
23
Asynchronous Online
Discussion
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Q: The use of asynchronous online discussion has more
advantages over face-to-face discussion.
60
50
40
30
20
10
0
S Agree
Y H Teo
Agree
Not Sure
Disagree
S Disagree
Assessing critical thinking through scaffolded asynchronous online discussion
24
Asynchronous Online
Discussion
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Q: After reading my peers’ comments, some of my
beliefs and ideas changed.
90
80
70
60
50
40
30
20
10
0
S Agree
Y H Teo
Agree
Not Sure
Disagree
S Disagree
Assessing critical thinking through scaffolded asynchronous online discussion
25
Critiquing Video Cases :
Qualitative Results
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Research Question 2 : What benefits do pre-service
teachers report that asynchronous online discussion
has on their learning?
•
Free response in survey questionnaire & Content Analysis
•
•
Y H Teo
Reading peers’ comments broadens point of view and
obtaining a different perspective
Confirmed advantages of asynchronous discussion:
– tend to contribute more as not afraid of ‘reprisals’ faceto-face
– Permanent record encourages reflective thinking
– Overcome limitations of distance and time
Assessing critical thinking through scaffolded asynchronous online discussion
26
Role of Scaffolding Stems
Introduction
Critique Framework
Socratic Questioning
KC for scaffolding thinking
Conclusion
Q: The use of scaffolding prompts before posting a
note helps me focus my thoughts
80
70
60
50
40
30
20
10
0
S Agree
Y H Teo
Agree
Not Sure
Disagree
S Disagree
Assessing critical thinking through scaffolded asynchronous online discussion
27