Transcript New tools for assessment in distance education

Supporting group learning and
assessment through Internet
Liane Tarouco
Antonio R. de Vit
Marlise Geller
Luciano Hack
Graduate Program Informatic on Education
Universidade Federal do RS - Brazil
CMDL

Computer Mediated Distance Learning
(classroom is routinely a far more
scrutable educational environment
than the physical classroom)
 email & news groups
 chat
 videoconference
CSCL

Computer Supported collaborative
Learning
 group learning is more effective

Learning through reflexive discussion
CSCL based on Internet
services

Computer Supported collaborative
Learning

CSCL based on Internet
 asynchronous
 email, news groups,
electronic board
 synchronous
 chat, videoconference
Collaborative learning

Problems
 heterogeneous contributions
 lack of capability for monitoring all
group work
Learning assessment
 Learning
assessment became a
challenge to those who work with
distance education
Learning assessment
 Evaluation
process should include a
great variety of evidences that go
beyond the traditional final examination
based on paper and pencil.
Face-to-face context
Professors use more than
just formal mechanisms to
evaluate students.
 body language
 participation
Quality of questions
proposed by students are
good indicators of learning
Distance Education
 What
normally occurs it is the use of
formal mechanisms only
Why ?

It occurs not only because of the lack
of accessible mechanisms that can
assist instructors on this task, but
also, because many professors prefer
to work with more traditional methods
of evaluation.

Authentication

Legal aspects
Internet evolution
 Evolution
of networks and computers
and especially the Internet
 New
mechanisms and services started
to show up
Keeping track
 Tools
to keep track of students
activities and interactions
 with learning environment
 with the group of students
 with the professor
Internet for distance
education

Reduction of the distribution costs;

Internet facilitates collaborative
writing;

The corrections and updates are
simpler to handle;

Several techniques for evaluation are
possible using multimedia for the
interaction between professor and
students and between students.
Distance education
assessment

Assessment based on contributions for
group discussions

Tests (automatically handled by
computer program)

Term papers (analyzed by professor or
assistants)

Oral or written tests conduced in the
presence of the instructor (some times
through videoconference) or with a
remote assistant.
Time spent with assessment
Types of assessment
 case
studies
 exams
 papers
 projects
All of them requires extensive time to
grade and are very difficult to get
automated help to do the evaluation.
Distance education
technologies
Ancient forms
Internet based
environment
Printed text books
HTML pages
Exercises
Forms and CGI
TV broadcast
MBONE
VHS tapes
Real Video, MPEG
Audio tapes
Real audio
Assesment
Forms and CGI
Topics
Kirkpatrick's
model for evaluation
- set of tools to provide capability
to asses according this model

consensus

tracking

voting

self-evaluation
Kirkpatrick’s model
Results
Behaviour
Learning
Reaction
Formative evaluation
 Formative
evaluation is important to
make easy for the students not only
inform their opinions and feelings
about how course is going on but see
this contributions being handled and
considered.
Using quiz system
 Teachers
them
 CGI
may built artifacts and link
allows dynamically building and
editing
Basic functions included
 Feedback
for each
alternative
 Indication
answer
 Nothing
of correct
more, just
counting incidence of
selections on each
alternative
Complementary tools
 Consensus
 Tracking
 Voting
 Self-evaluation
Tool of consensus

It allows on time creation of enquete
proposing instigating questions, as
well as the collection and tabulation of
responses;

The tool has the capability to keep
track of each student's level of
contribution to the discussion as well
as enquete responses.
Tracking tool
 The
register of learning activities
(accessed page with dates and time)
is important because it allows the
professor to know the progress of
students trough the course;
 The
collected information will show
that type of access the student has
made (visited units, tools used, etc),
as well as the time spent on it.
Voting tool
 Aimed
to provide conditions for a fast
feedback for the instructor on specific
subjects.
Self-evaluation tool
 Allows
the dynamic building of
enquetes for self-evaluation by
professor and by students.
Group activity handling
Group activity may give opportunity
for student to present contributions
that some times are relegated to
forgetfulness.
Consolidation

It is also needed a voting system to
support group decision about the
consolidated results from discussion
 ordering
 reordering
 selecting
 discarding
What was needed

Development of applications to
handle participants contributions

Provide support for group decision

Utilization of contributions that came
up during group activity
Handling group activity results
A set of tools was designed
and developed to support
distance education
collaborative work.
Group activity results

Group activity results on a log file
containing interchanged texts
(synchronous or asynchronously)
during a group debate.
Example: Using an
environment like Palace
for chat
Other services used for
group discussion

WWW based chat

Videoconferencing complemented with
chat
Developed solution

Application for handle results of group
discussion (brainstorming)
 consolidation of contributions
 summaries with more relevant ideas
 voting process
Previous efforts and results
Issue analyzer
Eurekha
ESCOOP
Issue Analyzer

Tool supporting brainstorming

Used natural language recognition

Grouping of similar ideas

Organize them according with
definitions of the group

Client server application

Supported only synchronous
communication
Eurekha

Tool initially developed for information
retrieval

Uses methods for grouping textual
objects

Objects are organized automatically in
similar objects groups

Directed to natural language

Designed to work with files (each text
object must be presented as a
different file)
ESCOOP


Handle log files derived from group
activity
Clean and cut

remove stop words

spell checking

remove names of senders

Uses Eurekha for clustering

Handle voting process
Conclusions
 Formative
evaluation plays an
important role to get distance
students motivated because they feel
a sense of not being lost in space
 Just
the opposite, they feel that being
closely monitored and having their
opinions being asked often is more
like having a personal tutor that pay
attention on way they do and what
they say.
Conclusions
 This
kind of attention provided by the
system helps to minimize solitude that
sometimes is referred as main
desisting factor from students in
distance education programs
Conclusions

Cognitive development improved
 development of better selfexpression
 help student to learn about how to
work collaboratively
 stimulate problem solving critical
thinking and collaborative analysis
 learning through reflexive
discussions
Results

colaboration valued/considered

better follow up from teacher

contributions more homogeneous

secundary results
 writing skills improved
 more reflexive thinking about what
is being presented
More details / Contacts

Marlise Geller
[email protected]

Liane Tarouco
[email protected]