Transcript Cecil - London Knowledge Lab

Computer Supported Learning.
The Design and Application of A
Large-Scale, Web-based Learning
and Assessment Systems to
Support Flexible Education
Dr. Lesley A. Gardner
Department of ISOM, University of Auckland
The Story of Cecil
• We are located in a small nation (New Zealand)
– 3.7 million people, 14 sheep per person
– Few resources
• Difficult to gain expertise
• Rapid adopters of new and innovative technologies
– Many students (large volume classes)
• 27,000 within the university
• Students with different needs
• What could we do to ease this pressure
Cecil History
• Designed in 1995. Launched in 1996.
– 10 million logins since then
• MSIS Department (now ISOM)
– Don Sheridan & David White
• Originally on-line assessments & gradebook
– Peoplesoft R8 gradebook informed by Cecil v.1.0
Cecil Now
•
•
•
•
Database size: 130Gb
Approx. 40-50% of all courses use Cecil
84% of all enrolled students
External ‘users or customers’
–
–
–
–
Using an ASP business model
IPLS (Institute of Professional Legal Studies)
Secondary Schools
Other Universities (?)
The Statistics
• Students course registrations:
• Students active in Cecil
• Individual logins:
– Average no. of weekday users
– Average no. of weekend users
28516
24196
108,000
University wide
8500
4300
per day
per day
• Tests taken by students in 2004: 179232
(84%)
per week
(Sem 1 only)
Cecil Features
•
•
•
•
•
Topics (KM)
Resources
Assessment
Announcements
Discussions & Chat
www.cecil.edu
The world’s first
LMS - 1995
Citrix-based
Instructor’s interface
Semester
(Ski)
Break!
Cecil is a locally designed enterprise-wide system with ~20,000
sessions per day. Over 10 million logins since 1997.
Cecildom
130 Gb of
data
Web farm
7 servers
ASP.Net
The
Internet
Application Servers
5 servers
Delphi
Database
SQL Server
2000
Cecildom
•
•
•
•
•
•
3 Developers
2 Database Admin / Production specialists
1 Project Manager
1 Chief Customer Advocate
1 Tester
5 part-timers
Information Model of Question
Perceived Benefits
Perceived Benefit
• Greater individual flexibility
– Students have access to the materials and on-line tests 24 hrs
• They work when they like (some at 4.30 in the morning)
• Students who have additional commitments can fit in their
studies
– Student gain feedback instantly from online tests
• Improved deep learning
– So we think
Student Acceptance
• The students have seen Cecil’s potential embraced
the system
• Our surveys have indicated that
–
–
–
–
They find it useful
It assists their studies
Helps them to plan and practice
Enables deeper learning
• They love it!
Responses to 2003 Survey
Response content.
Access from home sometimes
Response
91%
85%
First access to Cecil found to be V. easy to
OK
90%
Students would use Cecil as a study aid
for no credit.
85%
Students would use Cecil to look at their
marks in other papers
89%
Students would like to take Cecil tests for
their other papers
71%
Cecil Student Components
• Cecil Student Web
• Cecil Student Discussions
• Cecil Booking System
Student support
• Learning
– M/C Tests can be sat online, at home, at their convenience
– Tests can be either assessed or non-assessed
– Results and helpful feedback can be published immediately
Communication
• Announcements to inform the whole class of relevant
information.
• A Discussion Board for communicating ideas with the tutor
and shared with the rest of the class.
• Online Chat provides students with the ability to
communicate with their instructors and with each other in
real time.
• Information Gathering - Students can retrieve all information
that has been made available to them by their tutor via Cecil
• Organisation - Cecil provides a calendar for each course.
– due dates for assignments.
– Cecil has a computer booking function
Discussion group for stage 1 course
Chat to Assist Socialising
But
If they are cheating
how can I tell?
Data Sources
• Each student’s response + time stamp
– Includes skipping & changing answer
• Question difficulty rating
– Statistic for current class(es), year(s)
• Question answer latency
– Statistic for current class(es), year(s)
• Question context
– Sequence / position in test
– In relation to other questions in pool
– Feedback option (nil, immediate, delayed)
Real-Time Detection
• Load laboratory configurations in Cecil’s booking
system
• Allocate students to laboratory seats randomly
• Associate “odd” behaviors with person and location
Catching Cheaters
•
•
•
•
Error Analysis
Temporal Analysis
Latency analysis
Item Analysis
But
How do I know if my
questions are good
enough?
A Cecil question analysed using TestGraf
 In this example, a five option multi-choice question performing
well – note the wrong answers drifting downward as the better
students select option #1
Research Teams
• Central repositories for information
• Discussion boards for posting ideas
– Stop your email getting clogged up
• Distributed environment allowing multiple platform
access.
• Secure
– Only named users can access repositories
Knowledge Repository
•
•
•
Storing all file types
Sharable among classes, papers, colleagues
Industry standard database
Courses
Learning
Units
Learning
Units
Learning
Units
Learning
Units
Goals
Objectives
BoK
Elements
IS
Body of Knowledge
Research Projects Discussion Group
Central Repositories for Information
Central Repositories for Information
Keeping the Group Informed
But
What’s in it for me?
Cecil Explorer!
Products for Instructors
• Cecil Explorer
• Course Descriptions Editor
• Presentation Creator (Web-based PowerPoint
conversion)
• Cecil Supervisor Web
• Cecil Supervisor Discussions
Instructor Support
• Cecil has been designed to closely mimic the
processes of teaching common to all instructors.
• Cecil is based around the concept of structuring
information to create activities like lectures,
handouts, tests and exams.
• Instructors who use Cecil to its full potential will find
that it saves time, aids instruction and clarifies
information.
Instructor Support
• Cecil has concentrated on the similarities that unify
instructors based on feedback and discussion with
users over the last 8 years.
• Enables multiple perspectives and learning
philosophies to be accommodated within the one
flexible learning system.
Enticing the Academic End User
• Persuading academics to take up the new tool has
had some success
• 987 classes currently use Cecil
• We know that:
–
–
–
–
All academics are busy
Technology failures are a great fear to many
Cecil has some initial set-up and operations costs
We must demonstrate the usefulness Cecil
Perceived Usefulness
• Time savings up to 40%
• Efficiency
• Course co-ordination and control grades posting
electronic
• Safe storage (backed up and reliable 99.9%)
• Privacy of information you BOK is yours unless you
decide to share it
• Security of data
Costs to the Individual
• Set-up time
– Individuals may be trained
– Support facilities are available through help desk and online help
files
• Support is the most necessary item
– New entrants
– Existing staff
But
I don’t need all
those features!
Cecil Web Interface Modifications
• The Cecil development team are developing
modified and alternative interfaces for other
products.
– Moodle
– Specialist courses
Professionals Online – Modified Cecil Web Interface
Student Support
• Are such systems actually good for students?
• Should students be made to struggle?
• Is the use of all electronic media in this form the
same as photocopying?
• Do systems of this type reduce social interaction and
reduce the richness of the tertiary education field?
Research Aspects
• Student deep learning.
• Can we use and mine the data from a tool like Cecil
to attempt to find indicators to support deep learning.
• Cecil has the ability with its underlying database to
be mined for data.
• Can we as a group of researchers use Cecil to
extend our teaching practices.
Research Aspects
• Case study which requires in-depth learning
• Tested using multiple choice questions
• Each multiple choice question has feedback
associated with each choice.
• The feedback therefore should assist the student in
reading the material in more depth and therefore
creating understanding.
– Can we test for this?
Research Aspects
• How can we monitor the modifications that we are
making to student and staff behaviour.
– Students can study and be examined 24 hrs a day
– Staff can return work and grades 24hrs a day.
• Is this really a satisfactory thing to do.
Conclusions
•
•
•
•
Amazing teaching, learning and administrative tool
Streamlined all areas of academic operations
Students love it and it works for them
How can we learn from the system
– The depth of learning
– The dynamics of student behaviour
– How the social behaviour of students is modified by these systems