Transcript Slide 1

Student’s perception of
competence to practice
Dr J D Cortis
Assessment and Learning in Practice Settings
Centre for Excellence in Teaching and Learning
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
The work of ALPS
• One of 74 CETLs funded by HEFCE, spans 16
health and social care professions and 5
universities
• Aims to ensure that:
“Students graduating from courses in health and
social care are fully equipped to perform
confidently and competently at the start
of their professional careers”
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
The BORG (no, not that one!)
Baseline and Outcomes Research Group
• Wanted a “before” and “after” ALPS view of
competence and confidence – with usual caveats
• 2007 - Reviewed current work on measures of
competence and competence
• Did not find a measure that would work across
all 16 ALPS professions – and did not find many
actual ‘measures’ at all
• Developed Competency in Practice Assessment
(CIPA) tool
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Measuring Competence?
Context (1)
In spite of overwhelmingly positive outcomes and
reports of recent QAA and NHS Major Reviews and
generally positive reports from ongoing
professional and regulatory body programme
approval, monitoring and review processes, there
remain sporadic outbursts of anecdotal evidence of
employer dissatisfaction with the competency of
new graduate health and social care professionals.
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Measuring Competence?
Context (2)
The ability to demonstrate that new graduate
professionals are competent and confident at the
point of qualification and initial professional
registration therefore remains a significant
challenge
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
The CIPA Tool
Development
• Usable by 16 different health and social care
professions (in the first instance) – hopefully
more as time progressed – ‘generic’ but
meaningful language
• Once written, consulted with an academic
representative from each of the 16 ALPS
professions – some negotiation over language,
then final version agreed
• Online to ensure widest coverage
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Pilot Phase 1
• Involved three pre-registration cohorts at Leeds
Metropolitan University who were mid-way through
their final year of study.
• 33 students completed the pilot
• This was done in order to calibrate the data
collection tool (using Rasch analysis) as we needed
to be able to identify students who did not feel
‘confident’ or ‘competent’ to practice as well as those
who did.
• In theory, students who were not qualified would not
score highly across the tool.
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Approach
Perceptions of Confidence and Competence
• Rather than a ‘snapshot’ of perceptions
of confidence and competence, a
longitudinal view was preferred
• Led to decision to use CIPA Tool:
- At point of graduation
- 6 months post graduation
- 12 months post graduation
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Pilot Phase 2
• The second phase of the pilot involved attempting
to implement the tool with all graduating cohorts
across the institutions and courses involved. Whilst
we were obviously keen to collect data in this pilot,
we also wanted to explore what issues would arise
when attempting to implement such a large scale,
cross-institutional study.
• 258 students from 4 universities completed the
CIPA
• This data was also used to feed in to the first Rasch
analysis to look at the reliability of the data collection
tool
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
e
C
ct
iv
ba
l
A
Ve
r
us
se
om
se
m
un
ic
e
an
at
L
d
is
io
Id
ob
te
...
en
ni
se
tif
ng
rv
y
ba
e
Sk
D
no
ea
rr
ill
ie
lw
n
s
r
v
s
ith
er
to
ba
B
B
c
a
...
om
e
em rr ie
m
rs
pa
un
D
ea
to
th
...
lw
co
et
ic
ith
m
w
M
co
he m..
ai
.
nf
nt
n
lic
ai
de
n
ta
al
P
i..
nd
ro
f
es
D
ne
ea
s
go
io
B
l
w
t..
e
na
ith
op
.
lb
en
C
ou
E
ul
to
st
...
tu
ab
fe
ra
ed
lis
lI
ss
h
b
C
ac
a
ue
om
k
go
s
fr
m
od
o
un
m
w
ic
...
o
at
rk
e
in
ef
g
fe
re
ct
...
iv
el
y
w
i..
U
U
respondents = 258
Phase 1
160
140
120
100
80
60
40
20
0
questions
not at all prepared
Adequately prepared
Extremely well prepared
somewhat prepared
Well Prepared
se
V
er
b
al
C
om
us
m
U
un
e
se
A
ic
ct
at
an
iv
io
e
d
n
Id
L
o
S
en
is
bs
ki
t
e
e
tif
lls
n
r
y
ve
i
n
ba
g
D
no
Sk
rri
ea
n
er
ills
lw
v
s
er
ith
to
ba
B
c
B
lf
om
e
...
em arri
m
e
un
pa
rs
D
ic
t
t
h
o
ea
at
co
l w etic
io
m
n
w
ith
m
h
M
un
en
co
ai
i..
n
nt
de
fli
.
ai
ct
al
n
in
an
P
g
ro
d
w
fe
ne
i..
ss
g
ot
io
D
B
n
ia
e
a
e
te
a
l
op
lw
bo
...
en
un
ith
E
da
to
C
st
u
rie
f
ab
ee
ltu
s
lis
ra
db
h
l
ac
C
Is
a
om
su
k
go
fro
es
m
o
d
un
m
w
ot
ic
or
at
he
k
e
i
rs
n
ef
g
fe
re
ct
la
iv
t..
el
.
y
w
ith
w
r..
.
U
respondents = 37
Phase 2
25
20
15
10
5
0
questions
not at all prepared
Adequately prepared
Extremely well prepared
somewhat prepared
Well Prepared
se
Ve
rb
al
Co
m
U
us
m
se
un
e
an
Ac
ic
at
d
t
i
io
ve
ob
n
Id
se
Li
Sk
en
st
rv
ills
en
tif
e
y
n
i
n
D
on
ba
g
ea
Sk
r
v
r
lw
er
ie
ills
rs
ba
ith
Be
lf
Ba to c
or
em
om
m
rri
...
er
pa
m
s
u
th
D
to
ni
et
ea
ca
c
i
c
om
lw
tio
wh
ith
n
m
e
u
n
co
ni
M
de
ca
ai
nf
nt
al
lic
...
in
ai
t
g
an
n
w
Pr
d
ith
of
ne
...
es
go
si
tia
on
D
Be
te
ea
al
d.
op
lw
bo
..
en
un
ith
Es
da
to
ta
C
bl
u
rie
f
ee
ltu
is
s
h
ra
db
C
a
l
om
ac
Is
go
su
k
m
o
fro
un
d
es
wo
m
ica
ot
rk
te
he
in
ef
g
rs
fe
re
ct
la
iv
tio
el
...
y
wi
th
w
rit
...
U
resondents= 34
Phase 3
25
20
15
10
5
0
not at all prepared
Adequately prepared
Extremely well prepared
questions
somewhat prepared
Well Prepared
Percent
Deal with cultural issues
45
40
35
30
25
20
15
10
5
0
Phase 1
Phase 2
Phase 3
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Be open to feedback from
others
60
Percent
50
40
30
20
Phase 1
Phase 2
Phase 3
10
0
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Establish a good relationship
60
Percent
50
40
30
20
Phase 1
Phase 2
Phase 3
10
0
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Student comments;
“Overall I thought the course was somewhat difficult at times,
however I found that I have learnt more than I thought I actually
did as now I am in the position where I am teaching other
students”
“I was thoroughly prepared for both the hard work and every
single aspect of the (named course) profession…I now feel
completely confident and competent in my abilities”
“When I started my course one of the lecturers said that we
would be different people after 3 years. She is absolutely
correct, I am a different person. I am now doing my dream job
as a nurse and I’m earning a great wage also”
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk
Summary
•The CIPA tool has highlighted “big” issues for the
ALPS partners e.g. importance of practice
•It has also highlighted the “small” issues which can
inform future curriculum development e.g. dealing
with cultural issues
•It is an effective tool, allowing us to pool powerful
data
•We can see trends over time – more work required
to collect data following qualification.
Assessment and Learning in Practice Settings (ALPS) ©
http://www.alps-cetl.ac.uk