Transcript AUN-QA Assessments, Challenges and Lesson Learnt


AUN-QA Assessments, Challenges
and Lesson Learnt
Dr. TAN Kay Chuan
AUN-QA Chief Quality Officer and Lead Assessor
Director, Office of Quality Management
National University of Singapore
Outline
 Background
 Initiation
 Trial implementation & challenges
 Tools development
 Implementation
 Actual quality assessments
 Challenges
 Special projects
 Improvement
 Lesson learnt and Challenges
Background
Bangkok
Accord
2000
AUN-QA
Guidelines
2004
AUN-QA
Manual
2006
AUN-QA
Assessments
2007
AUN-QA
Revised
Manual
AUN-QA
Operating
Guidelines
Present
Initiation
Implementation
Improvement
Bangkok Accord

Appoint Chief Quality Officer (CQOs)

Formulate common AUN-QA policies, criteria and benchmarking
procedures

Identify and encourage the implementation of QA good practices

Enhance mutual collaboration and information exchange

Invite and facilitate auditing, assessment and review by other
member universities as well as by external bodies

Be responsible for the implementation of this Accord

Seek further and deeper engagement regarding QA in higher
education with ASEAN Dialogue Partners
AUN-QA Guideline
 Guidelines are benchmark
standards towards quality and
quality assurance
 Strengthen the position of higher
education in ASEAN
 Promote the harmonisation of
quality assurance in higher
education within and outside ASEAN
 Endorsed by AUN Board of Trustees
as well as Presidents, Rectors and
Vice-Chancellors of AUN member
universities
AUN-QA Guideline

AUN-QA Criteria

Criteria 1: QA System

Criteria 2: Teaching and Learning







Course Curriculum
Academic Staff
Student Assessment
Learning Process
Environmental Health and Safety Standards
Learning Resources
Criteria 3: Research


Funding and Facilities
Research Output

Criteria 4: Services

Criteria 5: Ethics

Criteria 6: Human Resource Development (HRD)
AUN-QA Manual (2006)
 Facilitate implementation of AUN-
QA Guidelines
 Improve clarity and interpretation
of quality criteria and standards
 Manual for Self-assessment and
Auditing
 Endorsed by AUN Board of Trustees
as well as Presidents, Rectors and
Vice-Chancellors of AUN member
universities
AUN-QA Manual (2006)
Strategic
(QA at
Institutio
nal
Level)
Systemic
(Internal
QA
System)
Tactical
(QA at Programme
Level)
AUN-QA Revised Manual (2011)
 First revision of the AUN-QA
Manual
 Criteria and assessment process
of AUN Actual Quality
Assessment at Programme Level
 Associated resources (templates
and samples)
 Effective August 2011
AUN-QA Standards – Institutional QA
Stakeholder Satisfaction
Mission
Policy
Plan
A
C
H
I
E
V
E
M
E
N
T
S
Educational
Activities
Management
Research
Goals
Human
Resources
Aims
Funding
Community
Service
Quality Assurance and (Inter)national Benchmarking
Source: AUN QA Network
AUN-QA Manual – IQA System
Internal Quality Assurance
Monitoring
Instruments
Student
Progress
Pass Rate
Drop-out Rate
Feedback from the
Labour Market
and Alumni
Research
Performance
Evaluation
Instruments
Student
Evaluation
Course and
Curriculum
Evaluation
Research
Evaluation
Service
Evaluation
Special QA
Processes
Assurance
Student
Assessments
Assurance
Quality
Staff
Quality
Assurance
Facilities
Quality
Assurance
Student Support
Specific QA
Instruments
SWOT
Analysis
Inter-Collegial
Audits
Information
System
Quality
Handbook
Follow up
AUN-QA Standards – Programme QA
Stakeholders Satisfaction
Programme
Specification
Expected
Learning
Outcomes
Programme
Content
Programme
Organisation
Support Staff
Quality
Student
Quality
Quality
Assurance
Teaching/
Learning
Student
Evaluation
Curriculum
Design
Graduate
Profile
Pass Rates
Drop Out
Rates
Academic Staff
Quality
Didactic
Concept
Student
Advice &
Support
Student
Assessment
Facilities &
Infrastructure
Staff
Stakeholders
Development
Feedback
Activities
Graduation
Time
Employability
Quality Assurance and (Inter)national benchmarking
A
c
h
i
e
v
e
m
e
n
t
s
Initiation

Trial Run
 Burapha University, May 2007
 To put into practice the “Manual for the Implementing of the
Guidelines” of AUN-QA models for IQA system, institutional
level and programme level
 Challenges
 Clarification of assessment criteria and self-assessment report
(SAR)
 Key documents and records in local language
 Lack of a structured assessment process
 Lack of assessment and reporting tools
Tool Development
 Formalisation of assessment process (PDCA)
Tools Development
Desktop Assessment Template
 Standardisation of assessment and reporting tools
Example
Desktop assessment
Tool Development
Assessment Report Template
Example
Assessment Report
Tool Development
Assessment Feedback
Tool Development
SAR Diagnosing
Requirement of Self-Assessment Report

Part 1: Introduction





Part 2: AUN-QA Criteria Requirements


Write-up on how the university, faculty or department addresses the requirements of the
AUN-QA criteria (use checklist as a reference)
Part 3: Strengths and Weaknesses Analysis





Content page
Executive summary
Organisation of the self-assessment
Brief description of the university, faculty and department
Summary of strengths
Summary of Weaknesses
Completed checklist
Improvement plan
Part 4: Appendices


Glossary
Supporting documents and evidences
Implementation

Actual Assessment (2007 Onwards)
4
Years
7
Universities
13
Assessments
32
Programmes
Actual Assessment
No.
Period
University
Programme
1st
Dec
2007
University of Malaya
• Biomedical Engineering
• Computer-Aided Design/Manufacture
Engineering
2nd
Aug
2008
De La Salle University
• Chemical Engineering
• Applied Economics
3rd
Dec
2008
Universitas Indonesia
• Civil Engineering
• Mechanical Engineering
4th
Dec
2008
Institut Teknologi Bandung
• Physics
• Pharmacy
5th
Oct 2009
Universitas Gadjah Mada
• Pharmaceutical Sciences
• Chemistry
• Medical Education
6th
Dec
2009
Vietnam National University • Information Technology
- Hanoi
Actual Assessment
No.
Period
University
Programme
7th
Dec 2009
Vietnam National University – Ho Chi
Minh City
• Information Technology
• Computer Science & Engineering
• Electronics & Telecommunications
8th
Oct 2010
Universitas Indonesia
• Architecture
• Electrical Engineering
• Chemistry
• Metallurgy & Material Engineering
9th
Nov 2010
De La Salle University
• Chemistry
• Psychology
• Literature
10th
Dec 2010
Vietnam National University - Hanoi
• Economics
11th
Jun 2011
Universitas Gadjah Mada
• Biology
• Geology Engineering
• Civil & Environmental Engineering
12th
Nov 2011
De La Salle University
• Physics
• Applied corporate Management
• Software Technology
13th
Dec 2011
Vietnam National University – Ho Chi
Minh City
• Biotechnology
• Manufacturing Engineering
• Vietnamese Studies
Results Overview
Mean
Min
Max
1. Expected Learning Outcomes
15. Stakeholders Satisfaction
7.0
6.0
14. Output
5.0
2. Programme Specification
3. Programme Structure and
Content
4.0
13. Stakeholders Feedback
3.0
4. Teaching and Learning Strategy
2.0
1.0
12. Staff Development Activities
5. Student Assessment
11. QA of Teaching and Learning
Process
10. Facilities and Infrastructure
9. Student Advice and Support
6. Academic Staff Quality
7. Support Staff Quality
8. Student Quality
Challenges in Actual Assessment
 Shortage of qualified assessors
 Different stages of QA development and maturity within
ASEAN
 Improving assessment efficiency and effectiveness
 Recognition of AUN QA Assessment
 Harmonisation of AUN QA framework within and outside
ASEAN
 Sharing the knowledge gained and best practices
Special Projects
 Initiative for ASEAN Integration (IAI-CLMV) Project
 ADB-CLM Project (Supported by Asia Development
Bank)
 ASEAN-QA Project (Collaboration with DAAD, Germany)
 Training for Non-member Universities
ASEAN-QA: Joint QA Capacity Development in the
ASEAN region
Intro IQA
Theory
SAR
Writing
Finalizing
SAR
Intro EQA
Theory
Peer
Training
Preparing
Site Visits
EQA: Staff of QA Agencies
Site Visits
Sensitization: Delegation
of Vice-Presidents
IQA: University QA Officers
Training for Non-member Universities
 Sep 2011, 1st Workshop
 Apr 2012, 2nd Workshop
 Dec 2012, 3rd Workshop
More than 50 participants
from the ASEAN Region
CLM Project
 Enhance and strengthen the knowledge on QA system
implementation and management particularly based on
AUN-QA system,
 Build up qualified University QA personnel team in CLM
countries,
 Establish QA system to leverage the quality of education
management in CLM countries, and
 Provide countries with options for harmonizing
university QA system to ensure applicability across
Southeast Asia.
Improve

Training of New Assessors
 1st AUN New Assessors Training Workshop (2008)
 7 new assessors
 2nd AUN New Assessors Training Workshop (2009)
 12 new assessors
 Growing of qualified assessors from 6 to 25
 Development path: junior to senior assessors
Revision of AUN-QA Manual for Programme Level
Original AUN-QA Model for Programme Level
18 criteria, 72 statements
Revised AUN-QA Model for Programme Level
15 criteria, 67 statements
Guidelines for AUN Quality Assessment and
Assessors
Assessment
Certification
Guidelines for AUN Quality Assessment and
Assessors
 The establishment of an AUN-QA Council which wil
govern and formulate policies for the operation of the
AUN-QA Network
 Define roles, responsibilities and requirements of the
applying university, assessors, observers and staff of
AUN Secretariat
 Provide a common frame of reference and accountability
to the AUN quality assessment process and stakeholders
for the purpose of certification
Sharing the knowledge gained and best practices
via proposed IT platform
AUN
Secretariat
Applicant
QA Agency
Assessor
AUN Member
Public User
38
Lessons Learnt

Lessons Learnt
 Trial implementation – validation of QA manual
 Formulate tools and assessment process before actual
implementation
 Start small to gain experience
 Training and development of assessors
 Review for continuous improvement
 Assist CLMV partners within ASEAN to catch-up
 Gain credibility before considering QA certification
 Reach out to QA organisations within and outside ASEAN
 Use of IT to hasten harmonisation and sharing best practices
Challenges
 Globalisation of education
 Rising demand for better quality university education
 ASEAN Charter - promote greater mobility of students,
faculty members, talent and labour
Harmonisation of AUN-QA and national/local
quality assurance framework for higher
education
Quality Assurance in ASEAN
 11 Countries: Different culture, language, political
system, economy and HEI system
Quality Assurance in ASEAN
As of
end
2011
QAA + NQF
QAA without
National
Qualification
Framework (NQF)
Either
No Quality
Assurance Agency
(QAA)
or just being established
•
•
•
•
Indonesia
Malaysia
Philippines
Thailand
•
•
•
Brunei-Darussalam
Singapore
Cambodia
•
•
•
•
Myanmar
Lao PDR
Vietnam
Timor Leste
SEAMEO RIHED: Quality Assurance Landscape in Southeast Asia
Comparison of Assessment/Accreditation
Criteria
AUN-QA
1. Expected Learning Outcomes
Timor
Leste
√
Thailand
Philippine
Malaysia
Indonesia
Vietnam
√
√
√
√
√
2. Programme Specification
√
√
√
√
√
√
3. Programme Structure and Content
√
√
√
√
√
√
√
√
√
√
√
√
√
√
4. Teaching and Learning Strategy
5. Student Assessment
6. Academic Staff Quality
√
√
√
√
√
7. Support Staff Quality
√
√
√
√
√
8. Student Quality
√
√
√
√
√
9. Student Advice and Support
√
√
√
10. Facilities and Infrastructure
√
√
√
√
√
√
√
√
11. Quality Assurance of Teaching and
Learning Process
√
12. Staff Development Activities
√
13. Stakeholders Feedback
√
14. Output
√
15. Stakeholders Satisfaction
√
√
√
√
Harmonisation of AUN QA framework within and
outside ASEAN
nus.edu.sg