Internal Quality Assurance from a HEI Perspective – Uppsala University

2 nd Conference on Internal quality assurance at higher education institutions, 1 Dec. 2006, Bern 1 Background of the quality system at UU and the national demands 2 Main features of the quality system at UU 3 Some examples, outcomes and benefits of the system at UU 4 Some conclusions [email protected]

Main University Building

Uppsala University – some facts

 Oldest university in the Nordic countries - founded in 1477   6 000 employees - including 4, 000 teachers/researchers Education and research across nine faculties  Three disciplinary domains: Arts and Social Sciences Science and Technology Medicine and Pharmacy  Tradition-drenched student activities  Interdisciplinary campus areas

Education

40,000 undergraduate students More than 40 programs of study 1,800 single-subject courses 2,500 graduate students Student exchange programs with 400 universities in 40 countries

Organisation

1995 1993-98 1996 1999-2000 1999-2001

Background of the actual quality system at UU

National Agency for Higher Education established 1 st central quality group at UU 1 st quality audit at UU 2 nd central quality group & 2 nd quality audit at UU SAUNA – an

internal

quality evaluation 2001-2006 From 2001 From 2002 The National Agency policy changes from quality audit to quality assessment of all subjects and programs A new quality and evaluation unit 3 d central quality group The actual quality enhancement policy

Some demands from the National Agency

1.

Quality audits

at all institutions between 1995 and 2002

2.

Quality assessments

of all subjects & programs between 2001 and 2006 – ”The six year plan” 3.

Evaluations with focus on specific aspects, e.g.

- students ´ influence, internationalization

4.

Both quality audits and quality assessments

between 2007 and 2013 A new ”six year plan” will be decided later this year Perhaps including the possibility to apply for ”honour for an excellent educational environment”

Our view of the National Agency ´s six year plan between 2001 and 2006

Positive effects and conclusions

:  UU has been successful  The NA has focused on important national problems, e.g. the situation for small subjects, lack of resources

Negative effects and conclusions:

 Sometimes (often?) a heavy burden for departments  Lack of clarity of the aims and focus

The usefulness for departments and programs?

 the self evaluation  the site visit

To discuss………..

What are your own experiences of quality audits and/or quality assessment concerning the effects on quality, quality assurance and quality enhancement?

Are your experiences the same as the experiences at Uppsala university?

Or are your experiences quite different?

If YES: In what ways?

The concept of quality

”You know it when you see it”

Quality is a concept with different potential meanings… 1.

2.

3.

4.

The standard of something when it is compared to other things like it; how good or bad something is High standard

a thing that is a part om a person ´s character, especially something good a feature of something, especially one that makes it different from something else Oxford Advanced Learner ´s Dictionary

Quality …

depends ”on the eye of the beholder” develops and changes over time cannot be described in a subtle way using quantitative data can focus different aspects: prerequisites, processes, results and effects is a relative concept - fitness for purpose - value for money will have different concrete meaning depending upon what you focus, e.g. students ´view of a certain course or the development of a new vocational program

The actual quality assessment system at Uppsala university

* The nine faculties are responsible for their quality and quality assurance in education and research * Decisions in the University Board concerning infrastructural matters and resources * A long term policy for quality enhancement in education and research * An action plan for central quality enhancement has been decided each year * Central unit for quality and evaluation and a quality committée * 2006: - evaluation of research at UU - work on a new quality enhancement policy - quality assurance will be more integrated with the operational planning & annual reporting

Some values and conditions of quality assessment at UU

 Quality is created and shaped in faculties and departments … by teachers and students  The central strategies and actions must reinforce a culture of quality enhancement  The Vice Chancellor ´s engagement is essential for a well-functioning quality system  An active student union enhances work with quality issues  Some external demands are good for enhancing the internal quality work

Internal resources and quality work

Central level

A quality committée Units for quality & evaluation, for teaching & interactive learning, for management training etc.

Policies, e.g. Quality Enhancement Program Educational policy Projects, e.g. Teaching portfolios Training, e.g. teacher training ACTION ORIENTED EVALUATIONS

Faculty level

Different activities on quality issues, e.g. analyses of resource allocations, quality enhancement etc.

Departmental

Different projects

level

Program- & subject- course evaluations

Aims and objectives for the unit for quality and evaluation



Policy work

together with the quality committé, e.g. this year work on Bologna issues 

Implementation

of the actual action plan; among others

action oriented evaluations

, e.g. this year reporting on an PhD-alumni evaluation 

Support and consultative work

to faculties, departments and programs, e.g. this year - the faculty of social science – webbased course evaluations - the faculty of pharmacy – a webbased evaluation of the students views of each semester in programs - together with the Student Union: how to prevent dropouts  Covering the

national and international discussion

The National Agency ´s Quality Assurance of Subjects and Programmes every year 2001 –2006 Audit of UU 2001 2002 2003 2004 2005 2006

Unit for Quality & Evaluation

- Alumni evaluations of all subjects & programs evaluation of PhD-studies PhD-alumni evaluation - Consultations & ACTION ORIENTED evaluations

The National Agency ´s Quality Audit & Quality Assessments 2007-2013 Audit of UU 2007 2008 2009 2010 2011 2012 2013

Unit for Quality & Evaluation

- Continued Alumni evaluations and probably a second set of PhD-evaluations Action oriented evaluations

Faculty level

Responsible for the quality Demands for quality enhancement in the operational plan for all faculties a certain year Reporting on quality enhancement in the annual report

Undergraduate alumni evaluations

1.

2.

3.

4.

Selection of approx. 6000 students who have graduated at least two years earlier in subjects or programmes assessed by the National agency a certain year, e.g. 2006: nursing, pharmacy, law, history of art am.others

Questionnaires are sent by mail Separat – more than 80 – reports to all departments of the specific subject/program A final report on the results of the different faculties

Undergraduate alumni evaluations: the Questionnaire

 Background questions - Sex, age, degree  Position in the labour market - How long time until employment? Working position, tasks, salary etc.

 Evaluation of the education Attainment of goals set for higher education in the the Higher Education Act and goals set by the Faculty Board Attainment of skills , e.g. communication skills - What the students were especially satisfied and dissatisfied with  The demands of the present employment To what extent do the respondents use certain skills?

- The usefulness of the education

Forms of teaching, learning and study work

Independent work Written individual PMs Lectures Group work Oral presentations Seminars Study visits etc.

Groupdiscussions Laboratory work Problembased learning 0% 20% 40% 60% 80% 100% More of Satisfactory extent Less of No experience

Total

Part of working time in the relevant educational field

All the time Nursing/medicine Pharmacy Law Theology Engineer Civil engineer Science & technology Social sciences Arts Languages 0% 20% 40% 60% 80% 100% More than half the time Less than half the time Not at all

Undergraduate alumni evaluations cont.

Some more examples of the results:

 nine out of of ten students would have chosen UU again  many students satisfied with teaching & learning in problemsolving & written work  about six out of ten students dissatisfied with the training in oral communication

Some outcomes and benefits:

 faculties, departments, programs and teachers get more interested and find evaluations useful  quality enhancement in different areas

The value of the alumni evaluations?

an

interview study

with responsible professors/lecturers at twelve different departments and programs the

most interesting parts of the reports

, e.g. the most interesting results, are - the employment situation for the alumni - how the alumni value the outcomes and their teaching and learning experiences - the appendixes in which the answers from every alumni on all open ended questions are described, e.g. current employment especially valuable in the education what was missing in the education

The value of the alumni evaluations? – cont.

The usefulness of the report:

 very useful when - doing the

self assessment

- working with the

Bologna process

 the report actualizes both new and old questions on the agenda for

further discussion

 gives arguments for

changes

 gives suggestions for

concrete action



confirmation

that the quality enhancement and/or course/program planning is

working well

 gives much information to use in

study counselling

To discuss…

The alumni evaluations we have implemented are ACTION ORIENTED: Which are your own experiences of the value of alumni evaluations?

Can you see clear advantages and disadvantages with these centrally implemented alumni evaluations?

Evaluation of PhD-studies

1. Questionnaires to all PhD-students * importance of and satisfaction with working & research conditions * importance of and satisfaction with supervision & dissertation work * courses * pedagogical experience & training 2. Separate reports of the results to each department

Evaluation of PhD-studies cont.

Support of the supervisor (percent)

Very important resp. very satisfied (5) 4 3 2 Unimportant resp. very dissatisfied (1) Importance Satisfaction 0 20 40 60 80 3. Departments: self-assessment 4. Analysis of results and summary report for all faculties at institutional level 5. Faculties: analysis and suggestions for enhancement in each faculty

Evaluation of PhD-studies: some outcomes and benefits

Some results:

  big differences between departments most PhD-students satisfied with physical aspects of their working environment  some students dissatisfied with some aspects of supervision & support of their own lecturing

Some outcomes and benefits:

 Department management and teachers are interested and find the report useful  Quality enhancement in departments and faculties

Evaluation of PhD-alumni

1. Questionnaires to all PhD-alumni who graduated between 1997 to 2001 * Current relation to Uppsala university * Labour market placement * Main working assignments * To which extent is research included in the assignment? * Usefulness of the PhD-education * Satisfaction and demands with the PhD-education in different aspects, e.g. critical scientific thinking, working in a project group (a total of 18 aspects) 2. Separate reports of the results to each faculty

Evaluation of PhD-alumni cont.

Demands and satisfaction: generic skills (Faculty of Medicine)

Satisfied Demands

5 4 3 2 1 1 2 3 4 5

Some conclusions

 Necessary with some external demands, especially in the initial work of building an internal quality system  Important that as many teachers and students as possible are involved in some elements of the quality system (incl. pedagogical training)

and find it useful

 The University management & University Board must be involved both in discussions and decisions

and find this work useful

 It is important to make the quality work

visible

 Quality enhancement should be

”evidence based” (Action oriented evaluations)

 The quality system must be well known and it is very important to have quality in the quality work

to strengthen the legitimacy of the system

Thank you for your attention

www.uadm.uu.se/kvalitet [email protected]