Transcript Dia 1 - London Knowledge Lab

E-XCELLENCE
Benchmarking quality in
European HE e-learning
QA/QE in e-learning SIG
Workshop
Feb 27th 2009
Project Origins
• QA in HE is receiving much attention at institutional,
national and European level.
• Still missing: QA specific to e-learning
-criteria based on ease of access
-new forms of interaction (students and staff)
-flexibility, personalisation and other pedagogical
aspects that are now deliverable by e-learning
These express the added value of e-learning in HE.
EADTU established a members task force and secured EU
funding for 2 year project through 2005-2006
E-xcellence:
Benchmarking Quality in e-learning
Main Contributors:
EADTU
OULU-University (Finland)
OUNL (Netherlands)
OUUK (United Kingdom)
with support from:
CNED (France)
UNED (Spain)
UOC (Spain)
EITSA (Estonia)
NETTUNO (Italy)
APERTUS (Hungary)
+
NVAO (Netherlands/Belgium)
European University Association (EUA)
General objective
QA in e-learning that:
Optimises the learning process and offering assurance to
stakeholders that e-learning provision is of high quality.
Is complementary to existing national quality assurance systems
related more to content, staff and infrastructure.
Our main aim was to establish:
• a framework of quality criteria for the development,
operation and evaluation of e-learning programmes
• an appropriate set of performance indicators,
parameters and guidelines by which the quality of elearning programmes can be measured by assessment
• a definition of excellence
APPROACH
•Analysis of existing quality processes of partners and/or
requirements of national quality organisations
•Development of a suite of criteria relevant to e-learning
(using EADTU members’ experience)
•Stakeholder consultation procedure
•Translation of these parameters into a flexible quality assessment
tool based on benchmarks
•Testing in 3 pilots the benchmarks, guidance manual and
approach:
• Univ Oulu
Finland
• EITSA
Estonia
• UNINETTUNO
Italy
Options Explored
• Structured CMM style models vs flexible informal
systems
• EADTU accreditation vs Integration with national
systems
Diversity of European systems favoured flexibility
in design and implementation.
Outputs
Set of (33) European benchmarks
These cover institutional, pedagogical, technical,
ethical and management aspects of e-learning
organised into the three main categories of :
 Management (institutional and programme levels)
 Products (curriculum, course design and delivery)
 Services (student and staff support)
An important aspect of E-xcellence is that it offers a European-wide
set of benchmarks, independent of particular institutional or national
systems, and with guidance for educational improvement
Structure of Resources
• Benchmark Statements
– Define desired features
• Indicators
– Source of evidence
• Indicators of excellence
– Exemplars of evidence
• Assessors Notes
– Provide background guidance for assessors
The E-xcellence manual
• Mindmap QUALITY manual as backbone of web-tool
Web: http://www.eadtu.nl/e-xcellenceQS/
Modes of use
• Informal Self Assessment via Quickscan
• Identify hot spots (and cold spots)
• Internal Quality Assurance/Improvement
• embed selected benchmarks in internal process
• adopt and formalise indicators
• External Quality Assurance - informal
• embed use of benchmarks in Self Assessment
• Adapt to meet QA agency requirements
• External Quality Assurance - formal
• external QA agencies embed benchmarks within their
processes
Quickscan
• Simplified version of the full e-xcellence assessment tool
• Quick self-assessment of your institution’s e-learning
performance
• Identifies areas in your e-learning programme/course that need
more attention
• Score yourself on the most relevant aspects
Results will show after completing quick-scan. These help you:
• To identify elements to be improved
• To guide the internal discussion
• To learn if a full quality assessment procedure is useful
http://www.eadtu.nl/e-xcellenceQS/
Full e-learning assessment
Evaluate the institution/program/course using the E-xcellence
method and tool.
Manual; Benchmark statements, in relation to criteria and
indicators.
Questionnaire: comments on the specific issue: in the box
you can refer to documents or other references which can be
used as evidence on that specific aspect of e-learning.
The on-line questionnaire needs to be filled out by different
disciplines in your organisation coming from management,
course designers, tutors and students
On-site visit of e-learning expert
Full assessment
REVIEW TOOL
AGAINST
INSTITUTIONAL
STRUCTURE
+ PROCESS
PREPARE
SELF
ASSESSMENT
USING THE
TOOL
SITE VISIT
- SELF
ASSESSMENT
- ANALYSIS
- INTERVIEWS
- CONCLUSIONS
ACTION
PLANNING
Support: manual, proposed agenda, assessors’ notes, glossary.
Activity Breakout
• Quickscan provides overview of
benchmarks to consider:
– Relevance
– Confidence
– Evidence
Pilot Experience
3 Internal Quality Assurance reviews with
E-xcellence team members participating
Some difficulties with language and
interpretation of benchmarks
Suggestions for additional factors and
regrouping
More needed on the process of the
exercise
More on best practice
Pilot Experience
Conclusions
•
•
•
•
•
•
an intensive and productive exercise
relevant information for the project: materials,
process, procedure
relevant for the institutions
relevant for the piloting teams
work in progress
base document/material for different products
Review Extracts
• The E-xcellence tool has the potential for becoming a
useful instrument in supporting European universities
towards developing the quality of e-learning applications
(EUA)
• E-xcellence has evolved into an attempt to work within
existing QA frameworks rather than re-inventing them. In
our view, rightly so. We are convinced that the approach
and products of E-xcellence are useful for institutions
that aspire to having the e-learning component of their
Bachelor’s and Master’s degree courses assessed within
the existing NVAO accreditation framework (NVAO)
E-xcellence +
• EU funding for dissemination and
implementation
• Strengthened links with ENQA
• Additional project partners
E-xcellence + Partners
Core Contributors:
EADTU
OULU-University (Finland)
OUNL (Netherlands)
OUUK (United Kingdom)
with support from:
CADUV + Uni Hradec Kralove ( CZ Rep)
UNED (Spain)
EITSA (Estonia)
NETTUNO (Italy)
Kath Univ Leuven ( Belgium)
APERTUS (Hungary)
NVAO (Netherlands/Belgium)
NHSU + Univ Lund ( Sweden)
Slovak Univ Technology ( Slovakia)
MESI ( Russia)
IFEL ( Switzerland)
E-xcellence + Activities
• European Seminar May 2008 Madrid
– Jointly organised with ENQA
– ~ 50 participants
• National Seminars
– One in each partner country
– Participation of institutions and Accreditation
agencies/ministries
– Trialling of Quickscan and Full Assessment
E-xcellence + Activities
• Quickscan trials
– Use of Quickscan by each participating
institution
• Full Assessment trial
– Two online PG programmes at Univ Lund in
association with NHSU
– OUNL in association with NVAO
E-xcellence + Activities
• Current outcomes:
– Quickscan analysis at Univ Hradec Kralove
– Internal staff training programme in use of
Web CT
– Identified discrepancies between departments
in implementation of central policy
– Interesting exemplars of learner support
provision and staff reward structures
E-xcellence + Sustainability
• E-xcellence Associates label
– Commitment to continuous improvement in e-learning
activity
– Use of E-xcellence tools to identify and review
improvement
• Virtual Benchmarking community
– Peer review by community members
– Input to update and revision of E-xcellence tools
Web: http://www.eadtu.nl/e-xcellenceQS/
• Thank you for your attention