Transcript (slides 13/5/15)

Assessment and Examinations
John Kirby and Alison Tyson-Capper
Graduate School
Faculty of Medical Sciences
University of Newcastle
19th May, 2014
Assessment
• Why?
– Monitor student progress
– Identify problems with
• Project
• Supervisor(s)-student relationship
– Completion rate
• ‘exit’ strategy for struggling students
• Student re-registration
• Increased focus on the timeline
Assessment
• When?
http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Resstudenthandbook2013-14_final.pdf
Assessment
• When?
http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Resstudenthandbook2013-14_final.pdf
Assessment
• What?
– First assessment
Assessment
– Intermediate progress reviews
Assessment
– Final progress review
All PgR students in FMS now include an oral
(PowerPoint) at the start of the meeting
• IT facilties – their
rooms/lecture theatres
responsibility
• Remember - students can be nervous!
(meeting
Before the interview
•
Read all the documents!
– Read the student’s research report
– Read the report from the student for the progress panel
– Read the report from the supervisor
Your report
Empty boxes!! ‘No comments’ or
even ‘n/a’
• Provide constructive comments
• Flag up ideas/ concerns/ suggestions – no
surprises!
• Do commend a student if:
• Include some interesting results
• Mastered a complicated technique
• Selected to present at a conference
• Published a paper
• Progress HAS been made !
The new look –e-portfolio: the panels form
Exit strategy
• If you feel a student is struggling do not
give him or her the benefit of the doubt!
• Students who fail to complete often show
weak 1st reviews
– But in some cases a prod in the first year can
be very beneficial!
• If a student “exits” within 12 months, they
don’t appear on our overall PhD
completion statistics.
– And some (not all) grant bodies regenerate the
funds for a replacement student
The Viva Voce
Purpose of an examination
• For the University
– To assess and maintain quality
– To mark ‘completion’ of the degree programme
• For the Student
– Potentially leads to award of a degree
– Is an important and memorable life event
• can be a real emotional roller coaster (for everyone involved)
Who needs an oral examination?
• All PhD and MD candidates
– Students need one internal and one external examiner
– All staff candidates require two external examiners and an
internal ‘moderator’
• Not all MPhil candidates
– Same criteria for examiners as above
– Oral examination held at examiners request
• Not only for weak students
• With a good candidate can be fun for everyone!
Appointment of examiners
• See nomination forms on Graduate School
http://medical.faculty.ncl.ac.uk/postgraduate/internal/staff.html
– Completed by supervisor and Head of school/Institute
• Examiners must:
–
–
–
–
be cognisant of standards
have subject knowledge (need CV)
not have played a role in the research
(if external) not have been a member of Newcastle
staff for at least 3 years
– be able and willing to examine
– “command authority”
Internal examiners
• Do NOT organise the exam, food,
accommodation, travel, etc
– This is the supervisors responsibility.
• Ensure the examination complies with Newcastle
University protocol
• Ensure appropriate report forms are completed
and submitted in a timely manner
• Provide balance, fairness and ensure good
examination conduct
• Potentially play a role during any appeals process
External examiners
• Often is the scientific specialist
• Is an experienced examiner
– Chicken and egg….
• Often thought to have the ‘casting vote’
– But there are procedures for disagreement
• Maintains inter-university quality
– Report provides important feedback to
Graduate School (and QA etc)
• May not be completely ‘up to speed’ with
local regulations!
What if 2 external examiners?
• Need to appoint an internal moderator
• This person need not read or understand
the thesis
• Role is to provide advice on Newcastle
examination process
– May be required to present records of the
exam if the result is questioned (appeal
process).
Criteria – all theses
• Should be:
– Authentic
– Scholarly
– Professional
– Well-structured, written and presented
MPhil candidates
• Should
– Demonstrate advanced knowledge
– Have good knowledge of literature
• Theses need not
– Demonstrate consistent originality
– Be worthy of publication
PhD/MD candidates
• Should
– provide evidence of adequate industry
– demonstrate ability for originality
– understand relationship with wider field
– thesis should contain material worthy of
publication
Types of thesis
• ‘Standard’
– Divided into chapters with results and interpretations
• By publication
– A review and a series of ~5 related papers
– Can be difficult to examine as papers have already satisfied
external referees!
– Staff candidates only
Useful WWW sites:
•
Examiners handbook
http://www.ncl.ac.uk/students/progress/staff-resources/pg-research/examiners.htm
•
ncl.ac.uk/calendar/volume1/…
– (conventions, regulations etc)
Reading a Thesis
• Are you a proof reader or a scientist?
• You will need to provide a list of corrections if you
require them
• For a good thesis, I (and most colleagues) usually
stick “post-it” notes to the margin to localise my
questions within the thesis
– Be sensitive though; hundreds of these can look very
scary!
• If the thesis is poor, it might be better to have a
more detailed critique with lists of specific
questions and problems.
Preliminary Report
• Regulations vary between institutions -read
them!
• Many (but not all) institutions require examiners
to independently produce reports before the
examination
– Some need these to be submitted (well) before the
examination to flag up potential problems
– Some don’t require submission of these reports until
after the examination!
• But they should be exchanged with the other examiner’s
report on the day.
Newcastle
Imperial college
Liverpool University
It is almost unheard of to
suggest examiners speak
to each other before an
exam!
Viva Seminars: all FMS students
Three formats;
1.Open seminar
1.Selected audience (e.g. research group and
assessors)
2.Solely to the examiners (and internal chair if
required)
* Questions : Students decision
**Examiners do not ask questions**
How to conduct the examination - 1
• Arrange the room
– Often good to have pencils and paper to draw
on
• At the start candidates can be very
nervous!
– Put them at their ease if possible with a soft
start (but don’t anticipate the result!)
– Remember to arrange refreshment breaks
– Consider the candidate’s bladder
How to conduct the examination - 2
• Agree a plan with your co-examiner
• Remember:
– Oral examination of a good candidate can/should be one
of the most pleasurable academic experiences for all
involved
– Examination of a poor thesis/candidate can be truly awful!
• You need to devise different strategies for both
situations
Questioning
• The soft start
• “what result in your thesis are you most proud of?”
• “what led you to choose this study”
• “what are you doing now?”
How long should the exam last
• No fixed duration but >3 hours is
exhausting for everyone.
• Often examination of good students will
last longer!
• Use your judgement
At the end (if all has gone well)
• Ask the candidate to withdraw for a few minutes
• REMEMBER you do not award the degree!!
• Work out what you wish to say, then invite the
candidate back
• Tell the candidate what recommendation you will be
making to the higher degrees committee (or other
appropriate authority).
• Smile and shake hands
Examination Results
Total submissions
Doctoral Prize Nominations
Assessment Irregularities
On Time
Out of Time
C
B
Aiii
Aii
Ai
0
20
40
60
80
100
120
140
Can you do more to recognise
excellence?
• In many countries, 1st rate PhD students can be
defined.
– the French system allows PhDs to be awarded as "honourable
(not very good)", "very honourable" (average) and "very
honourable with felicitations" (top 5%).
– there is no similar recognition in the UK.
• In Newcastle we will soon ask examiners (on a separate
report sheet) to indicate whether they consider the
thesis to be in the top 10% of theses they have
examined.
– A committee will consider this recommendation and prizes
will be awarded.
• We hope this will be good for the student’s CV.
At the end (if it has NOT gone well) - 1
• Make absolutely sure you know what your options
are (read the regulations)
• You are not obliged to tell the candidate anything
(although you will feel some pressure to do so)
– the candidate will receive written confirmation of the
outcome in due course
• You may wish to speak to the supervisor
• You might need clarification of the regulations
(Graduate School)
At the end (if it has NOT gone well) - 2
• You will need to fill in the report form with very
comprehensive details of any changes you require
– This outcome results in much more effort in the future
for the examiners!
• All report forms look different.
– Make sure you know the precise significance of a tick in
every optional box!
– For example, ticking box 3 (pass) at Newcastle can
produce a very different outcome from box 3 at
Imperial College (fail)!
The appeal process
www.ncl.ac.uk/spo/AppealsForm.pdf
Invitation to examine
• This is not necessarily an honour!
– Not even a valued career move?
• Why you and not someone else?
– Are you a ‘soft touch’?
– Are you the supervisor’s best friend?
• You will/should see the abstract of the
thesis at the time of invitation
• Think about the thesis
– You have the right to REFUSE!