Transcript SRM Res process and quality

Social Research
Methods
The Research Process and the
Quality of Research
1
Shipman’s 4 key questions
about the quality of research
(Shipman (1988) The Limitations of Social Research)
1. If the investigation had been carried out again by
different researchers using the same methods,
would the same results have been obtained?
= RELIABILITY



subject error (different results on different days)
subject bias (try to please researcher)
Observer error and bias
2
Question 2
Does the evidence reflect the reality under
investigation? Has the researcher found out
what he/she thinks or claims it’s about?
= VALIDITY (internal validity)

Eg. Cook and Campbell list of threats to internal
validity
3
Question 3
3.What relevance do the results have beyond
the situation investigated?
= GENERALISABILITY (external validity)
 Threats to external validity




Selection (specific to group)
Setting (specific to setting)
History (particular past experience)
Construct effects (only this group has these
constructs)
4
Question 4
4.Is there sufficient detail on the way the
evidence was produced for the credibility of
the research to be assessed?
= CREDIBILITY
5
These associated with quantitative research (&
especially experiments or quasi-experiments)
Qualitative equivalents
from Marshall, C & Rossman, G (1995)
Designing Qualitative Research
6
Criteria of soundness
Qualitative
Quantitative
Truth value/credibility
Internal validity
Applicability/ transferability External validity
Consistency/
replicability/
dependability
Neutrality/not
biased/Confirmability
Reliability
Objectivity
7
Stages in carrying out
research
Bell, C and Newby, H (eds) (1984) Doing
Sociological Research
Suggest a distinction between


Normative accounts of research - How it
should be done
Descriptive accounts - how it is actually
done
8
Normative
Conceptualising the issues, getting a focus, developing a
research question. Use literature review, search journals,
use statistical abstracts
 Choose a research strategy
 Select methods
 Organise the practicalities - access, sampling, instruments etc
 Collect data - fieldwork, transcribe, code etc.
 Analyse data
 Report
and possibly
 Act on findings

9
Descriptive
Much more messy.
 Do in a different order - e.g. Collect data before
getting a research question





(esp. if using emergent methods - e.g. grounded theory)
Re-do parts if they didn’t work
Start again from the beginning
Key = not following to the letter, but to the spirit
Careful, and self-critical development
10
Developing a Research
Question

Research area - very broad, the set of ideas,
theories, social worlds etc.
Research topic - part of the above

Classic approach (normative) says do this first



But others take varying view




In experiment, need hypothesis
E.g. Lincoln, Y S and Guba, E G (1985) Naturalistic Inquiry
and Strauss, A & Corbin, J (1990) Basics of Qualitative
Research.
The grounded theory approach.
Still no harm in trying to develop research question.
11
Developing a research
question 2






Need to narrow
Need to know background literature
(literature search)
Group decisions (keep all ‘on board’)
Negotiate with sponsors
Have a personal stake - not in results, but in
area. Keeps you going in bad times
Be realistic about what you can achieve with
given resources.
12