Transcript PowerPoint Template Suggested Usages

Conducting programme
evaluations/reviews with a view to
sustainable quality promotion
Jan Botha
University of Stellenbosch
South Africa
Keynote Address
Oman Quality Network Conference
Muscat, 28 October 2008
Key questions
• Can we apply the 80/20 principle to
programme evaluations?
• What are universities learning from
the evaluation of (formative)
programmes?
2
Outline of the paper
1. Suggestions for the academic leader
2. QA in higher education context
3. Misconceptions about QA in higher
education context
4. Programme evaluations and the 80/20
principle
5. What are we learning from programme
evaluations
3
Suggestions for the
academic leader
The predicament of the academic leader
• My research …
•
•
•
•

I have become “one of them”
In the middle of the sandwich
From critic to custodian
I have to “manage”… “quality”
Academic Leadership Capabilities
• “Learning Leaders in times of change”
• Geoff Scott, Hamish Coates, Michelle Anderson
(May 2008)
Four suggestions for the Academic Leader
1. Use quality management as instrument to
achieve your vision and goals for your
School / Department / Programme
2. Quality management can be a powerful
change mechanism
3. Synchronise and integrate evaluation
activities
4. Use an academic and research-informed
approach to quality management
QA as an Academic Project
• QA to be approached as a form of research
•
•
•
•
an intellectual enterprise
driven by curiosity, creativity and pragmatics
underpinned by social-scientific inquiry
accompanied by experimentation and
reflection
• informed by theorising
• resulting in application and publication
A Principle of Design
Specifications are continually moving upwards
Opportunity for incremental changes
Specification
s
Calculate this cost
Poor design
Good
design
Poor design
Quality Assurance in
Higher Education Context
“Living with ideology in the university”
• Ronald Barnett*
The “idea of the university” as “a site of reason”
• Universities are beset by many ideologies
Foucault: ideology is “that which is opposed to truth”
• Competition and Entrepreneurialism
• Quality
• Managerialism
• Research
• Access, inclusivity, multicultiralism
2. Turn the power of ideology onto itself, move from
ideologies to ideaologies – recall the idea of the
university
* Beyond all reason. Living with ideology in the university. 2003. Open University Press.
The idea of the university?
“Athens”
Knowledge for the sake of knowledge;
educated citizen; wisdom (e.g. Philosophy
Department)
Pectora Roborant Cultus Recti
“Berlin”
All-round humanist education; transmission of
culture; research in basic disciplines (e.g. Physics
Department)
“New York” Market, entrepreneurialism; professions
(Business School)
“Culcutta”
12
Focus on problems and needs of the
community; applied research (Nursing School,
Engineering, Agriculture)
What are universities for?
• Universities until recently believed their existence and value
was self-evident
• Pressure of governments: universities should address the
needs of society (and become “skills factories”?)
• Universities state their vision, mission and objectives:
•
•
•
•
•
A place of quality, a place to grow
Creating futures
Your Knowledge Partner
Innovation Generation
To advance Learning and perpetuate it to Posterity
• Sultan Qaboos University aspires to be an academic institution of
excellence recognized both nationally and internationally, a center
for learning where individuals can develop their capabilities in an
environment that promotes academic achievement and research
excellence; and a community where the highest moral and ethical
values prevail for the purpose of both self-advancement and service
to the community.
International snapshot
• International Network for Quality Assurance
Agencies in Higher Education (INQAAHE)… a
world-wide association of some 200 organisations
• Members include
•
•
•
•
•
•
Oman Accreditation Board
Commission for Academic Accreditation, UAE
Netherlands Quality Agency (NQA)
Malaysian Qualifications Agency (MQA)
Higher Education Commission, Pakistan
CONEAU - Comision Nacional de Evaluacion y Acreditacion
Universitaria, Argentina
• National Accreditation Agency (NAA), Russian Federation
India
• The National Assessment and
Accreditation Council (NAAC)
• accredits institutions of higher education
• size of Indian system: assessing hundreds of
general education programs in 12 000
institutions of higher education is not
feasible
• institutional accreditation for general
education institutions has been the strategy
United States of America
• Many decades of institutional and programme
accreditation in the USA
• Council on Higher Accreditation Accreditation
• Six regional accreditation organizations (MSA, SACS,
NEASC, NWCCU, NCA, WASC)
• Specialised accreditation agencies, e.g. ABET
• 2,800 programs at more than 600 universities,
over 1,500 dedicated volunteers participate
annually in ABET activities
• As many as 142 calendar days devoted in one year to
visiting panels
South Africa
• Council on Higher Education (CHE)
• “just hang on, it will go away, as has happened
with the other acronymous bodies…”
•
•
•
•
Institutional Audits
Programme Accreditation
National Reviews of Programmes
Quality Promotion and Capacity Building
• Three steering instruments in Higher
Education
GOVERNMENT STEERING OF THE PUBLIC HE SYSTEM
QUALITY
Institutional
quality audits
Accreditation of
academic
programmes
PLANNING
FUNDING
Approval of institutional
missions, operational and
equity plans, academic
programme profiles
HEMIS data
analyses
New funding
framework
Institutional inputs
& outputs for
funding
Student enrolment
plans
(Slide: Ian Bunting)
Government funds
allocated to HE
institutions
Misconceptions about
Quality Assurance
in Higher Education
Challenging the “management” of quality
• Is “managerialism” not contradictory to the nature of
a university ?
• “Management” of quality used to cover for poor
quality?
• More management of quality results in less quality…
• CHE publication
• Academic Freedom, Institutional Autonomy and Public
Accountability in South African Higher Education
(August 2008) (www.che.ac.za)
Misunderstandings (1)
• Point of departure: lecturers are doing poor
quality work,
• we need QA to ‘catch the lazy buggers out”
• I am satisfied with 95% of the work of the people
in my university/faculty/institution
• therefore QA is not necessary
• Just make sure that you meet the minumim
standards and “keep them off your back”
• Aim for the minimum
• Comply…
Misunderstandings (2)
• If good academics are appointed at universities,
we do not need QA systems. If the wrong people
are appointed, no document (however thick it
is), will ensure quality.
• If nobody steals, we do not need laws and police.
If there are thieves, no law (however thick it is),
will ensure that no theft takes place.
• If nobody is ill, we do not need doctors. Since we
know beforehand that everyone is in any case
going to die some day, the whole medical
profession is senseless.
The case for Quality Management
• Quality management is a
necessary condition for quality,
however,
• quality management is not
sufficient for quality
Programme evaluations
and the 80/20 principle
Acknowledgement
• Co-researchers
• Prof Carools Reinecke
• Emeritus Vice-Chancellor, Potchefstroom University
(South Africa)
• Mr Mr Kamal Bhagwandas Jogibhai
• University of the Witwatersrand, Johannesburg, South
Africa
• Research Assistant
• Mr André Müller, University of Stellenbosch
25
The Pareto Principle
• Vilfred Pareto (1848-1923)
• Wealth distribution in 19th Century England is predictably
unbalanced
• Most income and wealth go to a minority of people
• Consistent mathematical relationship between proportion of
people and amount of income and wealth
• Principle of least effort
• Principle of Imbalance
• Zipf’s Principle of Least Effort
• Resources tend to arrange themselves to minimize work
• 20-30 % of resources accounted for 70-80% activity
• Jurian’s Rule of the Vital Few or the 80/20 Principle
26
The 80/20 Principle illustrated
Effort
Results
R Koch 2007. The 80/20 principle. The secret of achieving more with less.
London: N Breadly Publishing
27
Purposes and types of programme evaluations
• Mouton & Babbie (The practice of social research. Oxford, 2001):
• judgement-orientated evaluations
• improvement-orientated evaluations
• knowledge-orientated evaluations
• Trow (Academic reviews and the culture of excellence. Stockholm1994):
• internal supportive
• internal evaluative
• external supportive
• external evaluative
• Evaluation outcomes used by different role players
for different purposes
28
What is a programme?
• A social intervention
• A set of planned, structured learning experiences
leading to a qualification
• One qualification with various programmes
leading to that qualification
• BSc in Biodiversity and Ecology
• BSc in Mathematical Sciences
• One programme in a “field” with various
qualifications
• The “programme in mathematical sciences” consists of
a Certificate, Diploma, Bachelor, Masters, and PhD
29
Unit of evaluation/review
• An institution as a whole
• An academic organisational units within an
institution
• department, centre, institute, bureau, school, faculty
• Specific activities or processes within an
institution
•
•
•
•
30
learning and teaching programmes
research programmes
community engagement programmes
capital campaigns
Types of programme evaluations
• Evaluation of professional programmes as part of the
periodic programme accreditation process
• conducted by professional councils
• National reviews of programmes
• conducted by the a national QA body
• the same programme evaluated and (re)accredited simulatenously
at all institutions
• Internal programme evaluations/reviews (for different
purposes: improvement/closure/consolidation etc.)
• conducted by institutions
• professional and formative programmes
• undergraduate and postgraduate programmes
31
Challenges in programme evaluation
• Coherence within broader internal and external QA
system
• Formative programmes as unit of evaluation
• BA, BSc, BComm, B SocSc
• Spread over many departments in Schools and Faculties
• Conflating “evaluation” with “reporting the results of
an evaluation process”
• Too much effort and resources for outcomes? 80/20
• Administrative burden
• Evidence
33
34
Expectations of Programme Evaluations
• Sustainable quality promotion
• An instrument for change
management
• Integrated with other forms of evaluation
• Outcomes should justify the effort
• for academics and evaluators and HE managers
• Standard methodology used
• self-evaluation based on criteria/standards, report,
panel visit, report, improvement action
• Applicable to formative programmes
35
A “Pareto Approach” to Programme Evaluation
1. Consider the ideal situation
• refer to criteria / standards
• clustered in 11 themes
• simplified (and “translated”)
2. Formulate a key question (or questions)
• focus, contextualise (for us)
3. Express the evaluation in a symbol (5 point scale)
discipline to make a judgement
4. Motivate the evaluation symbol (brief narrative, with
substantiating evidence)
5. Formulate strategies (in bullet points)
36
Template for the Self-evaluation Report
1. Programme rationale
Criteria
The programme is consonant with the faculty’s mission, planning and resource allocation.
The design maintains an appropriate balance of theoretical, practical and experiential
knowledge and skills. It has sufficient disciplinary content and theoretical depth at the
appropriate level.The programme offers opportunities for community interaction.The
design offers learning and career pathways to students with opportunities for articulation
with other programmes within and across institutions, where possible. The 2003
formulation of the strategic importance of the programme was re-evaluated.
1.2 Key Question
1.3 Evaluation
1.4 Motivation
1.5 Strategy
Reports limited (8,9 pages). Strategies formulated as actions
37
Criteria/standards clustered in themes
1.
2.
3.
4.
5.
6.
7.
8.
9.
Programme rationale
Academic integrity
Student recruitment, admission and selection
Staff
Learning facilitation
Assessment
Infrastructure and academic information sources
Programme co-ordination
Student success and academic support for student
success
10. Service Learning and work-based learning
11. Programme Evaluation and Development
12. Research basis (postgraduate programmes)
38
Case study
• Faculty of Agri-Sciences, Stellenbosch University
• BScAgric in Animal Production Systems; Agricultural
Economics; Wine Production Systems; Crop
Production Systems; Forestry; Food Science
• (BAgric at Elsenburg College)
• Faculty of Sciences, Stellenbosch University
• BSc in Physics; Chemistry, Mathematical Sciences,
Earth Science, Biodiversity, Molecular Biology, Human
Life Sciences, Sport Science, Education
• Faculty of Humanities, University of the
Witwatersrand (Johannesburg)
39
Rationale
Academic integrity
Student recruitment,
admission, selection
Staff
Facilitation of learning
Assessment
Infrastructure and access to
academic information
Programme coordination
Student success and student
support
Service learning and workbased learning
Programme evaluation and
development
Agri-Sciences
Programme 1
4
5
4
5
5
4
5
5
4
3
5
4.45
Programme 2
4
3
4
4
4
5
3
5
5
5
5
4.27
Programme 3
3
4
2
3
4
3
3
2
4
3
3
3.09
Programme 4
3
4
4
3
5
4
3
2
4
2
3
3.36
Programme 5
3
4
2
3
4
5
4
4
4
4
3
3.64
Programme 6
3
3
2
4
3
4
2
4
3
3
4
3.18
3.33
3.83
3.33
3.67
4.00
3.33
3.83
40
3.00 3.67 4.17 4.17
Rationale
Academic integrity
Student recruitment, admission,
selection
Staff
Facilitation of learning
Assessment
Infrastructure and access to
academic information
Programme coordination
Student success and student
support
Service learning and workbased learning
Programme evaluation and
development
Agri-Sciences
Programme 1
4
5
4
5
5
4
5
5
4
3
5
4.45
Programme 2
4
3
4
4
4
5
3
5
5
5
5
4.27
Programme 3
3
4
2
3
4
3
3
2
4
3
3
3.09
Programme 4
3
4
4
3
5
4
3
2
4
2
3
3.36
Programme 5
3
4
2
3
4
5
4
4
4
4
3
3.64
Programme 6
3
3
2
4
3
4
2
4
3
3
4
3.18
3.33
3.83
3.00
3.33
3.67
4.00
3.33
3.83
41
3.67 4.17 4.17
Rationale
Academic
integrity
Student
recruitment,
admission,
selection
Staff
Facilitation of
learning
Assessment
Infrastructure
and access to
academic
information
Programme
coordination
Student success
and student
support
Service learning
and work-based
learning
Programme
evaluation and
development
Sciences
Programme 1
3
3
4
3
3
3
4
3
3
-
-
2.64
Programme 2
3
4
4
4
4
4
5
5
4
4
3
4.00
Programme 3
5
4
3
3
4
4
5
4
4
2
4
3.82
Programme 4
3
3
3
4
4
4
4
4
4
2
5
3.64
Programme 5
4
4
3
4
4
4
4
4
4
4
4
3.91
Programme 6
3
4
2
4
3
5
4
2
4
-
3
3.09
Programme 7
5
5
4
5
3
4
5
4
4
-
5
4.00
Programme 8
4
4
4
4
4
4
2
3
3
2
3
3.36
Programme 9
4
3
3
4
4
4
5
4
3
-
4
3.45
Programme 10
3
5
3
4
4
4
3
4
3
2
3
3.45
Programme 11
4
3
3
3
3
3
2
2
4
3
3
3.00
Programme 12
5
4
4
5
3
4
5
4
4
-
5
3.91
3.83
3.83
3.33
3.92
3.58
3.92
4.00
3.58
3.67
1.58
3.50
42
Rationale
Academic
integrity
Student
recruitment,
admission,
selection
Staff
Facilitation of
learning
Assessment
Infrastructure and
access to
academic
information
Programme
coordination
Student success
and student
support
Service learning
and work-based
learning
Programme
evaluation and
development
Sciences
Programme 1
3
3
4
3
3
3
4
3
3
-
-
2.64
Programme 2
3
4
4
4
4
4
5
5
4
4
3
4.00
Programme 3
5
4
3
3
4
4
5
4
4
2
4
3.82
Programme 4
3
3
3
4
4
4
4
4
4
2
5
3.64
Programme 5
4
4
3
4
4
4
4
4
4
4
4
3.91
Programme 6
3
4
2
4
3
5
4
2
4
-
3
3.09
Programme 7
5
5
4
5
3
4
5
4
4
-
5
4.00
Programme 8
4
4
4
4
4
4
2
3
3
2
3
3.36
Programme 9
4
3
3
4
4
4
5
4
3
-
4
3.45
Programme 10
3
5
3
4
4
4
3
4
3
2
3
3.45
Programme 11
4
3
3
3
3
3
2
2
4
3
3
3.00
Programme 12
5
4
4
5
3
4
5
4
4
-
5
3.91
3.83
3.83
3.33
3.92
3.58
3.92
4.00
3.58
3.67
1.58
3.50
43
Academic
integrity
Student
recruitment,
admission,
selection
Staff
Facilitation
of learning
Assessment
Infrastructur
e and access
to academic
information
Programme
coordination
4
5
4
5
5
4
5
5
4
3
5
Programme 2
4
3
4
4
4
5
3
5
5
5
5
Programme 3
3
4
2
3
4
3
3
2
4
3
3
Programme 4
3
4
4
3
5
4
3
2
4
2
3
Programme 5
3
4
2
3
4
5
4
4
4
4
3
Programme 6
3
3
2
4
3
4
2
4
3
3
4
3.33
3.83
3.00
3.67
4.17
4.17
3.33
3.67
4.00
3.33
3.83
Student
success and
student
support
Service
learning and
work-based
learning
Programme
evaluation
and
development
Rationale
Programme 1
Student
recruitment,
admission,
selection
Staff
Facilitation of
learning
Assessment
Infrastructure
and access to
academic
information
Programme
coordination
3
3
4
3
3
3
4
3
3
-
-
2.64
Programme 2
3
4
4
4
4
4
5
5
4
4
3
4.00
Programme 3
5
4
3
3
4
4
5
4
4
2
4
3.82
Programme 4
3
3
3
4
4
4
4
4
4
2
5
3.64
Programme 5
4
4
3
4
4
4
4
4
4
4
4
3.91
Programme 6
3
4
2
4
3
5
4
2
4
-
3
3.09
Programme 7
5
5
4
5
3
4
5
4
4
-
5
4.00
Programme 8
4
4
4
4
4
4
2
3
3
2
3
3.36
Programme 9
4
3
3
4
4
4
5
4
3
-
4
3.45
Programme 10
3
5
3
4
4
4
3
4
3
2
3
3.45
Programme 11
4
3
3
3
3
3
2
2
4
3
3
3.00
Programme 12
5
4
4
5
3
4
5
4
4
-
5
3.91
3.83
3.83
3.33
3.92
3.58
3.92
4.00
3.58
3.67
1.58
3.50
Programme
evaluation and
development
Academic
integrity
Programme 1
Student
success and
student
support
Service
learning and
work-based
learning
Rationale
4.45
4.27
3.09
3.36
3.64
3.18
What are we learning
from programme
evaluations?
1. Programme rationale (69)
• To enhance interaction with stakeholders (subject
specific societies, industry, extraordinary lecturers, alumni) to broaden academic and
industry specific networks (through a voluntary advisory committee)
To review/restructure the content/focus of the
•
programme to ensure that module outcomes strongly reflect the programme specific
and generic outcomes, reviewing programme structures annually and implementing
student feedback and industry input, and developing a formal system to gather
information from graduates on their appraisal of the course, and suggestions for
improvement.
To adapt and better integrate courses, share
•
expertise/reduce duplication and use resources more cost-effectively, reducing
lecturing load by constructing generic courses across departments and including and
designing modules to fill theoretical gaps and deepen theoretical knowledge or to add
attractive career paths
To define service/experiential/work based learning
•
within the programme and incorporate electives/optional subjects to better prepare
students for the work environment, extending practical training/internship periods and
awarding credits for lab or field work experience; To offer more opportunities for
community interaction, especially for graduate students.
46
1. Programme rationale (continued)
• To outcomes communicate clearly to
contextualize lectures, and information about administrative and support
services to students, staff and stakeholders (advertising student assistantships
effectively)
•
•
To identify and support students at risk
in
small groups and give academic and non-academic help and to communicate
the importance of class attendance; also addressing low/increased student
numbers
To sharpen recruitment and marketing
efforts, improving outreaches where scholars, prospective and current
students can be informed about the programme and its strategic value. (Open
days, information sessions...)
47
2. Academic integrity (65)
• To increase reseach and benchmarking opportunities for staff with
international scholars, to ensure the programme remains at forefront of new developments, and
encouraging research/teaching interaction;
To review the undergraduate programmes annually, continuously and
•
coherently planning modules in terms of content, level of difficulty and credit value; constantly reviewing
the curriculum, learning materials, learning methods and programme outcomes; moderating it and
ensuring that the feedback from external moderators is taken into account
To increase collaboration between lecturers to improve programme
•
cohesion, filling in theoretical gaps through new and adapted modules, and cutting out duplication; also
working with timetabling administrators to free up module combinations; allowing greater articulation
with cognate programmes, structuring the programme to expose students to the core themes, balancing
practice and theory to ensure the relevance of prescribed modules that are presented by other
departments from both within, and external to, the faculty
To identify and remove obstacles inhibiting the use of
experential learning and increase laboratory time, re-evaluating the module composition
•
and structuring of the programme in order to make place for a longer period of internship; to evaluate
the efficacy of the practical portions of the modules and investigate coherent year-long practical
modules at second and third year levels;
To communicate the reason for the approach followed during the first year
•
of study, and maintain a challenging learning environment for students despite low/increased numbers
48
2. Academic integrity (continued)
• To develop writing, information and computer
skills and improve the teaching of practical skills through the use of continuous
assessment for all modules from second year upwards, and to identify and adapt
modules fairly early in the academic programme; To focus on improving student
mathematic skills; To complete computer literacy and scientific communication skills
modules in the first year
• To manage or establish new research
institutes/units/centres such as an Institue for Food Technology (IFT)
or Institute of Food Science and Technology (IFST)
• To recruit students from historically
disadvantaged backgrounds
• To recruit more international students
49
3. Student recruitment, admission and selection (93)
• To involve industry stakeholders in recruitment and funding for
undergraduate scholarships; To acquire additional bursaries from all sector role players; To expand facilities
and staff; To increase support staff, especially in terms of popular media and marketing; To involve subject
societies in recruitment actions; To increase interaction with bursary providers about the criteria for
successful students and the problems that are experienced by current bursary holders; To investigate the
roles professional bodies play in promoting the image of certain careers
•
To review admission requirements for mathematics and science related
to forest science programme; To evaluate the tempo at which graduates are employed; To investigate
existing recruitment actions and material in terms of success and relevance; To monitor the extended degree
programme to ascertain whether it is increasing student diversity; To monitor student numbers at the
beginning of the year and consider the adjustment of entrance criteria if necessary; To review policy on
ranking on application form so as to admit students who desire to follow; To review academic support
available to struggling students in all years
•
To adapt intake in terms of quantity, quality and equity of
students. To align recruitment efforts of the university, faculty, programme-committee and
departments; To develop and implement mechanisms, including assessment methods, to widen access, (e.g.
summer school, bridging programme); To increase the diversity of the student body in terms of South African
cultural groups and international students; To create a structured marketing and recruitment plan as top
priority, buying in expertise if needed, consulting with SU marketing divisions, utilizing existing marketing
mechanisms, negotiating for undergraduate bursaries and addressing the student diversity; To improve the
administrative implementation of admission criteria; To reconsider admission requirements at first year level
to curb the high failure rate, as well as at honours level (possibly); To re-assess the modules which have
become “service courses”;
50
3. Recruitment, admission and selection (continued)
• To make the bridging programme compulsory for students with a
matric mark between 50%-56%; To visit underprivileged schools in our immediate vicinity; To sponsor
prizes (e.g. book prizes) for the best Biology (now Life Sciences?) student in Grade 12 at a few selected
schools; To consider an orientation period including visits to departments as well as the experimental farm
before registration of second-year students to allow informed choices on major subjects; To supply
information on programmes at the Expo for Young Scientists and Olympiad candidates, as well as high
school science teachers; To encourage third years to attend final years’ product development
presentations; To focus on the recruitment of coloured students due to the demand from Agri businesses;
To initiate discussions with the govrnment on fragmentation of higher forestry
•
education in South Africa; To broaden the communication base with students at first- and second year
level; To develop a culture of uniqueness and excellence amongst staff members and students; To raise
awareness of language issues in the academic programme
To implement an extended degree programme (and first year academy) to
•
benefit students that have to overcome academic inadequacies due to historical barriers, and help students
to overcome cultural and language difficultiesTo prevent over-subscription to the course; Cap student
numbers (no more laboratory space available);.; To identify the optimum number of students that can be
accommodated in the programme (estimated at between 40 and 50 students), keeping economic factors in
mind; To market a BSc(Hons) as a career route – both commercially and in parastatal, particularly
academic spheres; To pay careful attention to entrance requirements in the next few years as the new
school leaving certificate is implemented. Aim to raise entrance requirements; To critically review failures
and examine the reasons for these
51
3. Recruitment, admission and selection (continued)
• To make better use of recruitment opportunities (e.g. Open Day,
visits to schools); To recruit black students via industry contacts; To send promotional material
(pamphlets) to all Secondary schools in the Western Cape; To represent the department at open
days organized by IPS; To write an article promoting Conservation Ecology and Entomology in
general and our department in particular for the popular science magazine Quest; To ensure the
web-site “inspires” students; To continue with current promotions activities and transparent
admission policies; To have greater flexibility within the language policy; To provide admission
requirements into the program (especially as of 2009) on departmental and faculty web pages and
brochures.; To advertise the programme campus wide on notice board, emphasizing the financial
support through industry bursaries, especially to students from disadvantaged backgrounds.; To
promote the need for a Faculty Open Day with a smaller, but more specific learner group (eg. top
10 learners within a grade with mathematics as school subject or learners from strong feeder
schools) so that departments can participate more effectively. Our perception is that the University
Open Day fails to recruit quality motivated students for the program. To improve the distribution
of marketing material; To recruit talented learners with bursaries; To contact CPS to disseminate
the information on courses in the programme, job opportunities and available bursaries; To give
more students access to bridging programmes; To try to attract the best Afrikaans and English
students; To attract black students with the Fundza Lushaka bursary scheme
52
4. Staffing (62)
• To enhance continuous professional development; To make sure all
newly appointed lecturers attend the University’s induction and training courses; To ensure on-going
professional development, letting staff attend various courses (e.g Web-CT training) as the need arises.
To deliver graduates who are in demand in the food and cognate industries; To develop C2 and C3
staff new skills and knowledge; To enable personnel to develop academically in their fields of
specialization; To encourage research activity by giving all staff members an “off semester” with no
undergraduate teaching commitments and additional training in assessment methods for large classes;
To maintain good research profiles and to expose students and lecturers to current research and
teaching issues and trends; To quantify time usage and check against workload document to enforce
contract hours; To manage time more effectively by setting fixed consultation times
To beter utilize the annual performance evaluation process, Permanent
•
staff members to be required to compile a personal work agreement and performance contract. This
assists in quality assurance of teaching and learning, research and service; To have all teaching staff,
with one exception, have a PhD; To review teaching load of staff through programme evaluation; To
have positive departmental evaluations; To ensure an equitable distribution of the teaching load by a
yearly review; To review and evaluate the deployment of technical staff for optimum technical support
To ensure more realistic work loads for academic staff; To align
•
specialisation and research focus of staff needs with programme. To ensure greater cooperation and
less duplication
53
4. Staffing (continued)
• To utilize personnel seconded from industry to the departments can assist
in maintaining research capacity; To investigate mechanisms by which specialised technical / practical
knowledge can be imparted to students. This may be linked to a specific academic staff member, specific
tasks during the internship, and/or exposure during practicals; To ensure that student assistants are
well-trained
To better communicate the benefits of an excellent diversity
profile and the spin-offs for the student body, who are exposed to a new experiences, approaches
•
and ideas. To inform the faculty through the departmental evaluations that a critical mass of academic
staff and sufficient administrative support are not present within all departments in the program to
effectively perform and excel in all 3 areas expected namely teaching, research and community service;
To actively encourage students and lecturing staff to interact about the expectations of web presence of
modules, particularly in smaller senior modules
To maintain quality despite the increase in student numbers; To
•
manage part-time staff carefully; To have adequate facilities and personnel to allow students to do
practical work and experiments
To address the perceived insufficient levels of remuneration
•
to
attract and retain young academics, especially for young black staff. The current personnel evaluation
has shortcomings and financial incentives linked to excellence are insufficient to motivate academic staff.
54
5. Learning facilitation (70)
• To get more clarity on the meaning of “student centered teaching”
and its implications; To encourage participation by academic staff in the activities of Centre for Teaching and
Learning; To utilize WebCT to effectively communicate with large groups, but not replace lecturer-student
interaction or class notes; To evaluate and compare learning material with international universities; To
develop a policy on student centered teaching so that independent, enthusiastic and spontaneous learning
consistently takes place; To incorporate fundamental knowledge much more overtly
•
To act on students’ feedback and evaluation; To give a copy of each module
evaluation to the programme coordinator (improve efficiency of the feedback); To request lecturers to
complete module frameworks, literature bibliographies, goals, outcomes in accordance with the module
framework requirements of Senate; To review the relevance of all units; To revisit the PBL process in terms
of life long learning, critical thinking and professional reasoning; To review what is lectured, the links between
problems and lectures and workshops and feedback; To review problems and evaluate their incremental
demands over the four years; To review decision to make curriculum books only available to students in eformat
•
To align all study guides with the University’s policy requirements
for study guides (specifically the inclusion of module-level outcomes in study guides); Investigate ways to make
class experience more stimulating; To make more use of text books and journal publications, and less class
notes; Powerpoint slides always to be provided to students; To employ a variety of assessment opportunities
to evaluate the facilitation of learning; To use text books and scientific publications to a greater extent and in
preference to class notes; To review the modules to ensure that they contain learning opportunities for the
development of these skills, without unnecessary duplication; To relook at the orientation course in first year
(preparation for PBL curriculum) and add a test to ensure the content is mastered and internalized
55
5. Learning facilitation (continued)
• To investigate the option of providing credits for lab or field
work experience
•
To adequately communicate the module outcomes to the
students yearly by the chairperson and via the website; To advertise student assistantships &
opportunities effectively; To expose second year students to the layout and cohesion of the
programme; To involve students in more personal feedback (similar to this evaluation exercise) at
a module and programme level; To highlight the fact that ability to work in a team is part of the
supposed programme outcomes and establish where this is in fact done;
•
To assign tutors dedicated to the support of disadvantaged
students; To inform departments of the need to apply computer literacy skills and gradually
introduce sourcing and use of scientific literature earlier in the program (3 rd-year level). Basically,
do not leave all tasks, seminars and research projects for the 4th year; To discuss with computer
literacy convenors options to allow mathematical science students to do fewer but more relevant
modules within computer literacy; To investigate the possibility of introducing opportunities for
students to improve and perfect their written and verbal communication skills at early stages in
their studies; To develop oral presentation skills for senior students
56
6. Assessment (58)
• To ensure that all staff members are aware of departmental
policies and requirements for assessment and the publication and storage of
marks; To use peer-reviewing within student/study groups as an effective supplementary method of
assessment; To encourage continues assessment training of staff; To change fieldwork rubrics to be more
user-friendly and precise (with student input); To review the number of assessment activities that contribute
to the marks and activities;To review time needed to answer questions and consider the advisability of seen
vs unseen questions; To analyze all exam questions according to Bloom’s taxonomy;
To align assessment practices all course frameworks, study guides
and assessment opportunities with the university’s requirements;
•
To ensure that all tests and exams are aligned with the principles of the University’s Assessment Policy; To
investigate “Turn-It-In” for electronic submission of assignments; To investigate ways to reduce the
significant increase in administrative load associated with internal and external moderation; To have rigorous
internal moderation, and external moderation for third year and honours modules; To establish, manage and
oversee an accurate and reliable administrative system to record marks onto the SIS.
To enhance strategies to eradicate plagiarism; To make assessment
•
challenging, in particular to test problem-solving abilities; To give more smaller tests rather than only a few
major tests and an exam; To assess individuals fairly in a group assignment and CTL should be approached
for assistance in this regard; To pursue a more systematic manner for collecting student feedback on each
module; To improve module frameworks to include assessments details (dates, type of assessment as well as
expected timeframe for feedback); To use a range of assessment methods such as a seminar, laboratory,
written and oral examinations;
57
6. Assessment (continued)
• To clearly communicate the means by which problem-solving
abilities will be assessed, i.e. the quality of the questions to be expected, the amount of
insight that will be required; To update the assessment dates and weights on the website; To keep
lecturing staff (and yearbooks) updated with regulations regarding assessment and moderation (internal
and external) at departmental level; To communicate the different assessment methods of different
modules clearly to the students; To provide reasons or motivations for giving a particular mark,
especially for essay-type projects and similar essay-type exam questions
To conduct individual interviews with students scoring >30% in
•
semester test, determine reasons, plan for support; To devise an early warning system for students
who are struggling (more difficult with larger classes); To give attention to entrance requirements and
streaming of students to preserve standards; To handle question papers with care to avoid corruption
of the assessment process;
To monitor individual student progress in terms of the First Year Academy’s
•
mechanisms.
58
7. Infrastructure and academic information sources (55)
• To keep teaching facilities to up to date; To improve, maintain and replace
laboratory equipment; To challenge the insufficient budgets to support the running cost of
undergraduate practical training in some courses (especially those with high student numbers); To
expand laboratory space, personnel; To develop IT support services to personnel
To utilize excellent library services optimally, faculty librarian
•
specialists and proper student training and regular purchases of new DVDs and material; To
investigate the possibility of using NARGA for training and tests
To re-allocate facilities to departments so that lecture theatres,
laboratories and the cellar can be in the same building; To arrange for
•
continued access to academic & scientific information (library) once graduates are working in rural
areas; To maintain the effective academic support offered by the Library, SunMedia and Information
Technology. To monitor access to and choice of electronic journals To identify one or more
modules to be enriched in teaching of writing skills and use of library facilities and undergraduate
students should be required to submit a literature review at some point in the academic
programme to ensure that the desire to continue reading and learning beyond graduation is
fostered.
59
7. Infrastructure and academic information sources (cont)
• To improve the equipment for undergraduate practicals so that
each student can complete practical assignments individually; To update the computer facilities and
associated software; To optimize laboratory equipment for teaching purposes (more student-proof
research equipment)
•
To optimally utilize the class space, modern equipment,
laboratory facilities, and computer and library services
•
To maintain laboratory equipment; To monitor the lecture hall size : amount of students
•
To attend to the insufficient field laboratories; To prevent
overshooting capacity of a maximum of 40 students in the programme; To challenge the tariffs
quoted by the General Maintenance Section of the University for services are perceived to be
inflated due to a sub-contracting system and is not seen to be consistently offering value and high
quality service.
60
8. Programme coordination (61)
• To integrate programme inputs from different stakeholders:
into curriculum development; undergraduate, post-graduate, extraordinary
appointments, alumni and industry;
• To align teaching methods with international best practrice;
To state programme outcomes (generic and specific) for all lecturers and students involved in the
programme through updated module frameworks
To monitor/review service levels regularly on all levels for
relevancy within the programme; To evaluate and implement final year
•
feedback; To monitor throughput of modules and support modules; To monitor student feedback
earlier in the modules to report back; To consider student representation in the Programme
Committee
To institute ongoing actions to improve and evaluate the
efficiency and quality of the teaching programme; To align individual
•
courses and course materials towards the main focus of the programme, especially for new
members of staff; To improve communication between departments to avoid unnecessary overlap
between courses and course material;
61
8. Programme coordination (contunued)
• To ensure better communication between lecturers in different
departments about the contents of programme; To introduce programme coordinator to 1st year
students & provide contact details; To ensure regular communication between teaching
administration at the University, and teaching staff and students; To introduce the programme coordinator to first and second year students, and provide contact details in class and on the internet;
To ensure that module frameworks exist for all modules, also on the website; To hold more
frequent committee meetings and to create more regular opportunities for student input and
participation; To explain lines of communication and authority to students
To identify and support at risk students by tracking their
records and analyzing it to help the department; To explore how the information in the
•
Student Oracle system can be utilized; To ensure that deadlines are met;
• To gain better understanding of the reasons why certain
students do not study successfully
62
9. Student success and academic support (60)
•
To provide more opportunities for individual face to face
contact between lecturers and students; To develop presentation skills
for lecturers and technical staff; To engages with CHSE to ensure academic staff have adequate
educational background and facilitation and assessment skills; To give both new and experienced
staff the opportunity to attend CTLD assessment and teaching courses; To adhere to University
Teaching Assessment policy; To maintain classes on/below 40 students per class
To improve the monitoring system for 1st and 2nd year
students, To monitor student success more systematically; To review mentorship strategy
•
with a view of developing a formal approach, close contact student / lecturer; To monitor
continuously that the pressure to improve the pass rate of students, especially at first year level, is
not carried over on to the second- and third year level to finally compromise the integrity of the
whole program; To monitor the success rate in every module in the department (and faculty)
where it resides.
To adapt evaluation techniques to maintain current
standards; To make the existing mentoring program more accessible to students in the
•
programme. To evaluate and adjust the mentoring system in order to make it more effective and to
ensure continuity from 1st year onwards. To look afresh at entrance requirements and options for
students who struggle To investigate the possibility of continuing the mentor system for support of
students at second and third year levels, in addition to the current first year support.
63
9. Student success and academic support (continued)
• Improve support to part time/sessional staff in their teaching
role; To teach students to improve skills rather than just to supervise and critique them
•
To assist students to overcome the challenge posed by language.
T-options for courses could be made more readily available; To remind lecturing staff that all assessments,
including small tutorial tests, should be bilingual; To encourage language use accessible to all students
To alert students to the available student support services; To
•
scrutinise students’study records in all their modules; To improve language, numerical and cognitive skills
within the programme; To let students participate in the first year Learning Academy; To assist students
with the challenges of the first-year of the program. The Extended degree program (also First Year
Academy) is seen as an option to improve student success; To make better use of tutor demonstrators.
To extend the successful tutor system to other modules if financial support can be obtained from the
Faculty; To identify at risk students earlier and offer compulsory extra tutorial time; To identify good role
models for students
•
To provide much broader access to an “extended degree”,
especially within the mathematical sciences. Come to terms with the fact that
a 4-year degree is the norm and use this to plan curricula. Consider a universal 4-year programme with
some students emerging with an honours; others with a BSc.
64
10. Service learning and work-based learning (43)
• To cooperate with stakeholders to create opportunities for
WBL interaction with industry for all students; To update lists of industry knowledge partners;
To explore collaboration efforts with MoUs to clarify roles; To ensure that the commercial entity is
suitable for receiving undergraduate students and that there is no family connection with the
student
To review and improve the WBL coordination responsibility
guidelines and monitor the success and/or need for WBL through departmental evaluations;
•
To follow up past-graduates to see how feasible their recommendations are
To adapt programmes to integrate WBL modules; To use
•
experimental farms optimally; To lengthen period of (vacation) internships; To extend mentoring
system to alumni in the industry
To ensure the commercial entity is fully aware of its WBL
obligations to the student before placement (e.g. internship); To require literature
•
searches during internships; To formalize community interaction projects with SUSPI; To consider
the possibility of site visits and voluntary internships
•
65
10. Service learning and work-based learning (cont)
• To communicate to all parties the outcomes of the WBL
and what the student should be achieving, To ask all module
coordinators to discuss work opportunities in the context of their modules;
•
To monitor WBL risks continuously e.g. to provide mentoring, support
and networking options for supervisors (employers); To overcome logistic difficulties of WBL in
certain industries, e.g. the mining and petroleum industries in the Western Cape;
To encourage students to take initiative for finding WBL
placement
• To address any WBL misconceptions in the industry regarding
•
the abilities of graduates and the balance of theoretical, practical and experiential teaching and
training during the academic programme. To challenge lecturing staff to guard against modules
being too theoretical given the context in which they are taught. To do an impact study to
measure the effectiveness of the programme in preaparing community servers for practice
66
11. Programme evalution and coordination (44)
• To maintain regular interaction with sector stakeholders and survey
employers on the impact of the programme and levels of satisfaction; To
improve liaison with industry / employers; To improve industry’s understanding of the purposes and
outcomes of the programme; To review the mechanism used to give stakeholders information about
curriculum changes based on their input
To implement coordinated strategies to review programme
content and outcomes annually; To obtain data on student recruitment generated during
•
departmental evaluations; To ensure that all departments obtain feedback from industries; To send evaluation
forms to alumni one year after graduation; To evaluate all quality assurance practices used in the department
To adapt courses to eliminate redundancy and streamline the learning
process; To evaluate the relevance and efficacy of all modules, including the practical portions thereof
To initiate a process of communication and interaction with
departments at this university and at other institutions to ensure that we are offering modules of the
•
•
highest standards and relevance; also consider the employability of exit level students;
To encourage final year students to continue with post-graduate
studies to better their qualifications and improve their employability and develop a society for past
•
graduates
67
Reflections and conclusions
• Selectivity justified due to 80/20 approach?
• “The 20% doctor”: all students must achieve all
outcomes
• Each programme must meet all the
criteria/standards
• Sampling and focus in evaluations
• How to select evidence?
• Just right and just enough
• Superficial?
68
Reflections and conclusions
• Strategies at different levels (Programme,
Department, Faculty, Institution, System, and
beyond)
• Encourage creativity
• Encourage debate, discussion across silo’s
• Closing the loop remains one of the
biggest challenges
• Good systems, procedures
• Good people (at all levels)
69