Presentation - University of Exeter
Download
Report
Transcript Presentation - University of Exeter
2001: published results of first job analysis,
funded by Clearing House. Authors: Phillips,
Gray and Hatton from Lancaster Programme
2005: Lancaster begin using the competencies
established using written task, interview and
presentation to collect evidence for
competencies
2006: Lancaster begins to use a written task to
screen candidates instead of application forms
2007: Clearing House funded research into use
of application forms and written test as first sift
of applicants. Report in 2008. Published 2010
(Simpson and Hemmings)
2005/6 Leicester Programme fund an
occupational psychologist to develop
competencies and design an assessment
centre approach to selection based on
these
2007 Leicester begin screening using a
written task
2009 Surrey commence using written test for
screening, Salomons join forces for 2011
2011 Shrops/Staffs use written test for
screening
2006: Working party founded by Roth, Wang, &
Reynolds.
› Fast Stream civil service email in tray task demonstrated at
GTiCP
› http://faststream.civilservice.gov.uk/How-do-Iapply/Example-e-Tray-Excercise/
› Prohibitive costs to develop this for our use
2009: Results of Clearing House survey of all UK
programmes to look at whether there is a will to
move to national screening: 54% express interest
2009: Selection sub group of GTiCP formed and in
2010 a bid is made for funding to develop a more up
to date set of trainee competencies, for use at the
point of selection
Helen Baron, Independent Consultant and
selection specialist engaged to undertake the
work
Steering group for the project:
Mary O’Reilly, Leicester
Cathy Amor, Lancaster
Laura Simonds, Surrey
Matthew Jones -Chesters, UEL
Linda Hammond, Salomons
Shrops/Staffs Rep (Helena Priest)invited to join
us in 2011
Interviews
Survey
Leading academics
x
Academic course staff
x
x
Clinical Tutors
x
x
Placement
Supervisors
x
x
Trainees
x
x
Service Users
Total
x
31
647
Role
Course Director
N
25
% of total
4%
Course Lecturer or Tutor
136
21%
Clinical Supervisor
231
35%
Trainee
287
44%
71
11%
7
1%
Service User
Other
Total
647/656 100%/ 116%
Intellectual Ability
Communication Skills
Self Aware and Open to Learning
Personal Maturity
Warmth and Empathy
Resilience
Organised
Autonomy and Initiative
Motivation and Application
Contextual Awareness
Course staff
Clinical Supervisors
Trainees
Communication Skills
Communication Skills
Warmth and Empathy
Intellectual Ability and
Academic Rigour
Warmth and Empathy
Communication Skills
Warmth and Empathy
Self Awareness and
Openness to Learning
Self Awareness and
Openness to Learning
Self Awareness and
Openness to Learning
Intellectual Ability and
Academic Rigour
Resilience
Resilience
Personal Maturity
Intellectual Ability and
Academic Rigour
Personal Maturity
Resilience
Personal Maturity
Organisation
Motivation and
Application
Motivation and
Application
Autonomy and Initiative
Autonomy and Initiative
Organisation
Motivation and
Application
Organisation
Autonomy and Initiative
Contextual Awareness
Contextual Awareness
Contextual Awareness
Great 8
Current Model
Analysing and interpreting
Intellectual Ability
Creating and conceptualising
Self Aware and Open to Learning
Interacting and presenting
Communication Skills
Supporting and cooperating
Warmth and Empathy
Organizing and executing
Organised
Adapting and coping
Resilience
Enterprising and performing
Motivation and Application
Contextual Awareness
Leading and deciding
Autonomy and Initiative
Personal Maturity
*Bartram et al, 2000
Sub-sample from Survey engaged in
selection
86 respondents
› 23 course directors
› 45 other course staff
› 13 clinical supervisors
› 5 trainees
Definitely no
Probably no
Definitely
yes
Probably
yes
Possibly –
depends
on
measure
In principle
would you be
interested in
the
development
of a national
screening test
as an aid to
shortlisting for
your course?
Test of Research Comprehension
Situational Judgement Test
Test of writing skills
Verbal Reasoning Measure
Test of English Language and
Grammar
Cognitive Ability Tests
Personality Questionnaire
% appropriate or somewhat appropriate
94%
85%
82%
82%
82%
70%
54%
Most courses interested in national
shortlisting
Over half interested in pilot
Many willing to share costs with students
Variety of assessment options seen as
appropriate
› Critical Reasoning and other cognitive measures
› Situational Judgement
› English Language skills
Three options presented
› No change
› Limited pilot
› Wholesale adoption of screening
Five groups looked at differing issues
›
›
›
›
›
What competencies to measure
Development process
Evaluation process
Logistics and liaising with clearing house
Mandate and accountability
In 2011 there were 2000 of 3500 applicants not
shortlisted by any course
› think of the opportunity costs on staff time (HEI and
NHS) of this duplication.
Local decision-making needs to be retained
Current off-the-shelf-measures the preferred
way forward
› an up-front fee per use that can be known, vs
resources needed to develop and scrutinise bespoke
measures being the decider
There needs to be proper transparency and
accountability (at local programme level and
national GTiCP/DCP level)
It is crucial to work closely with the Clearing
House on any forward planning
Time savings within courses especially
with high applicant numbers
Potential to reduce duplication of effort
across courses
Addresses double tick applicants
Mapped to competence model
Based on sound selection practice
Capable of empirical study to check
validity etc.
Many courses involve clinicians and
service users at the ‘screening by form
information’ stage
Feeling that if it isn’t broke don’t fix it
Concern about costs of development
Lancs:
› Daiches and Amor (2006)
› Simpson et al (2011)
Leics: GTiCP conference 2007
Surrey/Salomons: GTiCP conference 2009
To date, 5 programmes are using a
screening task
This is a written task
Two programmes have collaborated to
use the same task and process (and are
reported as one)
•
Completed
– At the University: all programmes
– On computer: 3 programmes
– By hand: 1 programme
•
Marked
– By computer: 1 programme
– By hand: 3 programmes
•
Past papers
– 2 programmes make these available on-line,
2 do not
•
Length of exercise
– 30 mins to 1 hour (Mean 49 mins)
•
Number of questions
– 7-14 (mean = 9)
•
•
2 programmes allow flexibility in order of
answering questions
2 programmes ask only about quantitative
research
2 programmes give equal emphasis to
quantitative and qualitative methods
No psychological knowledge beyond
undergraduate is assumed
Cut off score
– 1 programme has a cut off score (50%), 3 rank order
Intellectual ability and academic rigour
• Written communication
Resilience
Organisation
Motivation and application
•
Analyses and integrates
complex/contradictory material
› 2/4
Synthesises ideas from a variety of sources
› 2/4
Solves problems creatively
› 3/4
Critically evaluates information
› 4/4
Shows good academic and research skills
› 4/4
All other indicators in this area endorsed by
one programme or other
All competences under written
communication
Precise, well expressed and clear
› 4/4
Well organised and grammatical
› 4/4
Adapted to purpose and reader
› 2/4
Coherently argued
› 4/4
Copes well with pressure
› 4/4
Tolerates anxiety and uncertainty
› 4/4
Shows flexibility when required
› 2/4
Shows confidence in own ability to deal
with multiple demands
› 2/4
Is punctual and reliable
› 3/4
Prioritises tasks and uses time effectively
› 3/4
Is committed to completing tasks as well
as possible
Invests additional effort willingly
Is dedicated to career as clinical
psychologist
This competency area mapped by just
one course, however, arguably applies
to all test takers!
Multi-stage selection process can manage
all double tick candidates
All those who meet essential criteria are
invited to sit the screening test
Need to set a cut off score below which no
candidate can be progressed to next stage
› Eg minimum of 60%
› Ranking of candidates not accepted under
double tick
All double tick applicants above cut score
then invited to next stage
Directors have said ‘go on’, and want to
take more ownership of the project
Steering group made up of directors
across regional areas
Working group
Terms of reference to be drawn up by
Malcolm Adams
Proposal to Pilot use of a common measure
for 2013 intake
Requesting further funding for development
of a common test
› Perhaps one course centre to take lead under
the steering group
Initial ideas: One day (Saturday)
› One test (form and content)
› Computer administration
› Scoring options available
machine scored for all but many centres will hand
mark
Advance notice in Clearing House
course entries
One common test vs one common form
and different content
One testing day vs several
Regional/area testing centres
All applicants to the ‘opted in’
programmes attend one centre for test
No charge to candidates during pilot
Competencies
Screening
National
Screening
?