Transcript International Diabetes Federation (IDF) East Mediterranean and Middle East Region (EMME) Workshop on Professional Educational Methodology in Diabetes 11-12 August 2008 Bibliotheca Alexandrina, Egypt.

International Diabetes Federation
(IDF)
East Mediterranean and Middle East
Region (EMME)
Workshop
on
Professional Educational
Methodology
in Diabetes
11-12 August 2008
Bibliotheca Alexandrina, Egypt
Professional Technology
Standards in Educational
Development
by
Prof. Morsi Arab
2
Evaluation
Written Exams :The Long Essay Questions
Disadvantages:
1- Students do not know what is exactly required (vague)
2- Lack reliability ( different marks (scores) by different
examiners).
3- Usually not valid (in their future job they are not going to
write an article, but expected to do some task).
4- Take long time to mark ( if well done ?).
5- Will not give any feedback to the students.
6- Covers a limited part of the curriculum.
To improve reliability of L. Essay questions:
1- Give a well defined , structured question.
2- Prepare model answer and define in advance:
- the main points.
- the marks allocated for each point ( scoring ).
3- Explain to students this plan of scoring.
4- More than one examiner.
5- All students subjected to the same examiner for each
part of the exam (e.g. Q 4 ).
Essay Questions :
Minimal advantages :
1- Very easy to prepare ( for the examiner).
2- Tests ability of student to construct an article.
Conclusion:
- Better not to use it.
- If necessary try to improve .
Multiple Short Essay Questions
Advantages:
1- covers a large part of the curriculum
2- less unreliable
3- answered in shorter time ( by students).
4- also marked in shorter time ( by examiner).
Disadvantages:
Still tests more knowledge and no skills or application of the
knowledge in solving problems ( taking decisions).
The MCQ
Type 1: Short stem + a number of choices
Example: During a hypoglycemic episode :
(a) there may be excessive sweating √
(b) palpitations
√
(c ) there is hypotension
√
(d) very dry mouth
√
(e)……………………………………….. √
N.B. Here all choices should be taken
×
×
×
×
×
Multiple Choice Questions (MCQ)
Type 2:one stem +4-6 (usually 5) choices
- Example : After describinag some clinica features
…………..what is the most probable diagnosis ?
(NB To answer select only one .
( a )…………………………
Answer: e.g. (c)
( b )…………………………
( c )………………………….
( d )………………………….
( e )……………………………
Advantages of MCQ Test
- scored in a short time and precisely.
- if numerous , covers widely the syllabus.
- can be used for self assessment.
- high reliability.
Disadvantages :
- difficult to prepare good MCQs.
- takes a long time to prepare,
- usually test only knowledge
(rarely skills e.g. in decision making ).
Scoring of MCQ
- Time : give 2 minutes for each (√ ×) Q with 5 choices .
So, for 30 Qs give 30 min. to answer.
- Penalty: give +1 for each” correct” answer , 0 for “no
answer” & -1 for incorrect answer ( penalty is needed in
the type with + 5 Yes/No choices).
(no need for penalty in the type of stem + select one
choice).
- The score to pass: the pass mark should be high e.g.
80% or 90%.
- it is better to use easy Qs with high pass % (80-90%
than difficult Qs with low pass % (50-60%).
- For quick scoring punched paper may be applied.
Problem Solving Questions
(To be used best for evaluation of competence in decision
making ).
* Example : give a clinical situation followed by an MCQ on
the best decision to be taken
1-……………………….
2-………………………
3-……………………….
Then add more information followed by more MCQ decision
taking questions :
1-…………
2-………………………
3-………………………
( cont.
)
Problem Solving Q ( cont. )
Advantages :
1- High validity (simulates a clinical situation)
2- high reliability with a good preplan for scoring **
3- short time to mark (score).
**Preplan for scoring
write down all expected student responses (both correct and
incorrect ) and decide marks to be given for each response.
Oral Examination
- By one or more examiners :
- Forms : -what do you know about?
- what would you do if ……. ( take decision ).
- Advantages (few): vivid, can probe increasing depth of
knowledge (escalate).
- Disadvantages:
- induces anxiety to students (stress not existing in
practical life).
- consumes long time.
- unreliable
- can not test skills (v. rarely).
- does not give good FB to the education process.
Conclusion : better avoided
Evaluation by Check Lists
- very useful in clinical and practical examinations (Testing skills)
- Example: during a particular performance, the student’s
actions are observed by the examiner to score for
each appropriately performed step.
1-……………………………………………… √
2-……………………………………………… √
3-……………………………………………… √
4-……………………………………………… √
5-……………………………………………… √
6-……………………………………………… √
7-……………………………………………… √
8-……………………………………………… √
9-……………………………………………… √ etc.
- High reliability
- Gives good feed-back to teachers . (e.g. , if all the students
miss a particular step or do it with bad performance
reconsider the training course.
Check List / Rating scale for measuring attitudes
(behaviours) of a person during his training or working
(job)
0
1
2
3
4
5
1- shows interest
2- accepts instructions
3- cares for patients
4- shows desire to learn
5- keeps on time
×
×
×
×
×
0: strongly lacking…………5: strongly agree/ evident
Evaluation of behaviours during group training
(by Check List/Rating Scale)
- to report attitudes by observing specific behaviours of
different candidates. ( have good relations with patients,
accurate , punctual, organized. keeps good records,etc)
- it can be used to guide the trainees along the learning
process.
- it is used for final accreditation.
- it guides the educators (FB) the next training course
The Continuous evaluation
( during a training course ,e.g. every week, month or term.)
Advantages:
- lowers the risk from one faulty assessment.
- lowers apprehension of a one final test.
- encourages students to learn (study) all through the year.
- guides the student to improve his study.
Disadvantages:
- takes a long time and effort in preparation of frequent
exams.
What form
could be different : written test , achievement of skills,
observation of attitudes…etc.
Evaluation for self assessment.
Record of Satisfactory Performance
during a training course for a group
* The Record contains a number of tasks enlisted
(1,2,3 …).
* Each trainee (A,B,C,…) should fulfill the
performance of all tasks.
* Satisfactory achievement of each task
( separately ) is monitored.
* Unsatisfactory performance indicates repeat
training for a specific individual or for the
whole group
Tasks
Trainee
A
(B)
1
X
2
X
3
X
4
X
5
X
(6)
X
X
C
D
X
X
X