Transcript Document

The DCST’s, research and
health service evaluation
Fundamental questions to be
addressed
-Should the DCSTs be doing research?
-Why should they be doing research?
-What research should they be doing?
-How do they ensure that research
findings are used
Purpose of research for DCST
 Evaluation of health programmes
 Advocating for change
 To Make better and informed decisions on operations of
the health care system
 Provide evidence for the effectiveness of interventions
 Scholarship and academic growth
DCST can do research at almost any
stage of the planning cycle
We can do research at almost
any stage
 Situation analysis
• Health needs
• Health care utilisation
• Perceptions about the health care services
 Priority setting / option appraisal
• Economic analysis
• Cost benefit analysis
 Evaluation
• Determine impact of interventions
What kind of research should the DCST’s
be doing?
• Priority problems (national, provincial, local)
• Action orientated – aimed at developing solutions
• Broad – multi-disciplinary approach
• Timely – results should be available in time for key
decision making
• Simple research designs
• Presented in format that useful for decision makers and it
should be used
Focus on Action orientated
research
 Translational research
• Integrating basic research, patient-oriented research, and populationbased research so that scientific discoveries made in the laboratory,
clinical setting, or population setting result in clinical applications
such as interventions to prevent, diagnose, or treat disease.
 Need to think explicitly about implication for practice
What is research?
Research is defined as a
SYSTEMATIC, SCHOLARLY
SCIENTIFIC INVESTIGATION or
INQUIRY into an issue, materials
and sources in order to establish
facts, generate new knowledge
and reach new conclusions
Purposes of research
The primary purposes of research are:
 Documentation
 Discovery
 Interpretation
 Development of new knowledge
 Verification of existing facts
 Advancement of knowledge
General research principles
Research
 is based on an open system of thought that allows
exploration
 It involves systematic gathering of data
 It involves critical analysis of data
 It involves the drawing of conclusions that may be
generalised or that specify limits of generalisation
Approaches in research
Depending on the purpose of the study, method of collection,
analysis, and interpretation of data, two main approaches
are applicable:
 Quantitative research
 Qualitative research
(None of these is deemed superior to the other)
 Others include Operational research, action research, health
services research
Quantitative research
Quantitative Research
 Research that examines phenomena
through the numerical representation of
observations and statistical analysis
 Quantitative research - measures variables;
compares results; tests a hypothesis;
generalises its findings to the target
population
 Quantitative research can either be
descriptive or analytical
QUANTITATIVE STUDY DESIGNS
Analytical
Descriptive
Examples:
Experimental
Observational
Researcher compares
Outcomes through
intervention
Researcher compares
outcomes
through observation
Examples:
Examples:
• Clinical trials
• Case-control (Retrospective)
Case-report
Case of 1 scorpion sting
Case series
Treatment of 30 scorpion
stings
Comparison of effectiveness of 2
anti-convulsants
• Educational intervention
assessment using OSCE
Compared to “long case”
Smoking and lung ca
• Cohorts (Follow-up)
Alcohol use and liver cirrhosis
• Cross-sectional
Obesity among teenagers
Descriptive studies
Descriptive studies use numerical depiction of the
characteristics of a single occurrence/ episode OR
several similar occurrences.
Descriptive studies include the following designs:
 Cross sectional studies
 Case report
 Case series
Analytical studies
 Analytical studies: look for risk factors, associations
(between exposure and outcome) and causation.
 Analytical studies are either Observational or
Experimental
Observational studies
 cross-sectional: snapshot of characteristics of a
population at a specific point in time.
Associations are examined; exposure and
effect/outcome are measured at the same time
 Case control:
- outcome is known, exposures are looked for retrospectively,
associations are examined
 Cohort:
- exposure is known, individuals are followed up until outcome
of interest
Experimental studies
 Involve interventions directly controlled by the
researcher
 Randomised Controlled Trials:
- randomised clinical trials, gold standard of
experimental design, every participant has an equal
chance of being assigned to either group
 Field trials: healthy at-risk participants, eg vaccine
trials
 Community trials: whole communities participate
Procedures in quantitative
research
 Study population identification
 Sampling with the aim of representation hence random or
systematic sampling
 Data collection techniques/tools: Questionnaires, data
collection sheet/table, surveys
 Data analysis: Usually numerical and uses statistical
methods
Questionnaires
What is a questionnaire
• List of questions, answered by the respondent
• Indirect measure of the variables you are
interested in.
• Questions can be:
-Self administered – respondent fill the
questionnaire in themselves
-Interview – interviewer asks the questions
General aspects of a
questionnaire
Consult the respondent

Able to express opinion freely, opinions valued
Keep it short

Ask only pertinent questions
Achieve precise measurement

Collect data in its rawest form (income in rand, precise
occupation, age to the year, not age category)

highest level of measurement possible
Steps in developing a
questionnaire
Step 1.
• List all the variables to be measured
• These are based on the objectives
Steps in developing a
questionnaire
Step 2a)
• Decide what type of questions you will ask
Step 2b)
• Formulate the questions you will ask
Phrase your questions

Use simple words

Be concise and clear

Vary wording to produce variation in replies

Avoid complexity – make it simple

Use existing wording if comparative study

Take the edge off sensitive questions

Be precise, highly specific when choosing wordings
Step 3. Sequence of questions
• Start with an introduction – who is doing the study
and why – legitimate.
• Often put demographic information at beginning –
non-threatening
• Do not put most sensitive questions at the beginning
• Do not put the most important questions last –
participant may have given up by then / tired
.
Organise questions
 Have consistent formatting so respondents can get used
to answering
 Group questions by objectives
 Anticipate computer data entry

Pre-code questions before data collection
 Clearly indicate branching

Clear respondents away from irrelevant questions
 Make it easy to fill out; will increase response rate
Step 4. Plan the layout and the
design
 Have a distinctive look
 E.g., coloured paper, graphics for questions is
appealing
 Well laid out questionnaire more likely to be
filled in
 Don’t squeeze too much onto one page
 Crowding questions on a page can make the
questionnaire appear too long
Pre-testing
Pre-test the questionnaire



Start by filling out the survey yourself, then ask
individuals to do so
Ask for feedback on the questionnaire
Is it too long? Any problem with wording?
Pilot study


Send questionnaire to a small sample of
respondents
Use data to determine which items will be used
for indexes, modify unclear questions, create
pre-coded, single-choice questions based on
responses to open-ended questions
Disadvantages of questionnaires
• Provides only limited insight into problem
– Limited response allowed by questions
– Maybe not the right questions are asked
• Varying response
– Misunderstanding/misinterpretation
• Need to get it right first time
– Hard to chase after missing data
End product of Quantitative
research
 Reports usually with numerical and statistical indices of
comparison e.g. prevalence, incidence, Relative Risk, Odds
Ratio, sensitivity, specificity, P-value, confidence intervals
 Comparison, association, causation
 Tested hypothesis
 Generalizable information
Qualitative Research
Qualitative Research
 This is any a type of research that produces
findings not arrived at by means of numbers and
statistical procedures or other means of
quantification
 Qualitative research focuses on the way human
beings live and interact in their natural setting.
 Has an interpretive character, aimed at
discovering the meaning events have for the
individuals who experience them
When to use qualitative methods
 Exploration: they can reveal the nature of certain
situations, setting, processes, relationships, systems
or people
 Interpretation: enable the researcher to;
a) gain new insights about a particular phenomenon
b) discover problems that exist within the
phenomenon
c) Elicit feelings, thoughts, ideas, perceptions
 Verification: allow the researcher to test the validity
of certain assumptions, claims, theories or
generalisations within real-world contexts
Qualitative designs
 Case study
 Ethnography
 Phenomenological study
 Narrative study
 Participatory Action Research
Case study
 Case study:
- a particular individual, program or event is studied in depth
for a defined period of time
- suitable for learning more about a little known or poorly
understood situation
- useful for generating or providing preliminary hypothesis
Ethnography
 Ethnography:
- looks in depth at an entire group that shares a common culture
- the group is in its natural setting for a lengthy period of time
- intention is to identify cultural norms, beliefs, social structures
and cultural patterns
Phenomenology
 Attempts to understand peoples’ perceptions, ideas,
perspectives and understanding of a particular situation
or phenomenon

Phenomenological studies are appropriate in providing
insight into human experiences e.g. what is it like to have
a terminal illness?
Narrative research
 This design uses stories of experiences by an individual
or group of individuals in establishing an understanding
of an event or way of life.
 Stories are a powerful way of presenting qualitative
information and data is collected over long period of time
from one or more participants
Participatory Action Research
 A research method that involves researcher(s) and
participants collaborating to create
change/improve a particular situation
 In this design the researcher gains deep insights by
being a participant, living together and observing the
other study participants during the life event under study
 It allows the researcher an unobstructed view of the
participants lives/experiences
 Useful in studying intense social circumstance e.g. prison
life, refugees, times of war
Procedures in Qualitative studies
 The researcher acts as a “human instrument” of data
collection
 Sampling usually aims at richness of the information
hence Purposive sampling
 Data is in the form of speech, language, stories,
expressions, observations etc
 Analysis include: Content, language, discourse,
ethnographic (culture patterns)
Data collection techniques
1. Interviews
 Individual standardized interview
 Semi-structured interview
 Unstructured interview
2. Focus group interviews
3. Observation
 External observer
 Internal observer
 Participant observer
End product of qualitative
research
 Informative Reports that are descriptive, incorporating
expressive language and the “presence of voice of
participants in the text”
 Ideas, perceptions, experiences, feelings
 Hypothesis generation
Other priority research approaches
for Department of Health
Action research
• Quality of care
• Strengthening the system –
drugs, pharmacy, laboratories
”Essentially action research
is concerned with
generating knowledge
about a social system,
while, at the same time,
attempting to change it”
(Meyer, 2001:173).
Health service evaluation
• Health service evaluation is always used
for decision making
• The research questions are derived from
goals
• Always takes place in real world setting
• Results pertain ONLY to programme /
service being evaluated.
• NOT generalisable
Health services research
• Health services research is the scientific investigation of the
use, costs, delivery and effects of health care treatments or
services for individuals and populations.
• It is not about collecting information for information’s sake or
transforming facts from one place to another.
• Health services research involves systematically seeking
knowledge which will lead to improvements in the delivery of
health care.
Use of Research findings
• Sometimes No use
• Have it as an interesting fact
• Use it to make decisions
• Use it to persuade / influence others
• Use it to change behaviour
• Use it to evaluate services, improve patient
care, improve management etc
Barriers to utilisation of research
findings
• Time taken to do research
• Way it is presented – complicated and use of specialised
terminology
• Specific recommendations absent
• Recommendations impractical
• Results not communicated at all
• Decision often based on more than research findings – cost,
politics, equity etc.
Facilitating implementation of
research findings
• Involve stakeholders throughout the research process
• Give them regular feedback
• Develop a systematic dissemination and communication
strategy for reaching potential users including publishing
• Identify and present research to all stakeholders and
obtain feedback on findings and recommendations
• Actively lobby and mobilise action for and support the
use of results
How to ensure research is used
 Communication between researcher and consumer of
research.
 Decision makers should be involved in development
of research so that it is relevant and meets their
needs
 Identify all potential users of research at the
beginning – if they are only involved at the end the
results will never be used
 Manage conflict of expectations – policy makers
want results quickly and research takes time
Conclusion
If a man will begin with certainties, he
will end in doubts; but if he will be
content to begin with doubts, he will
end in certainties.
Francis Bacon (15611626),_Advancement of Learning_