Transcript SOFTWARE ENGINEERING Introduction to Case Study Research Raymund Sison, PhD College of Computer Studies De La Salle University [email protected].

SOFTWARE ENGINEERING
Introduction to Case Study
Research
Raymund Sison, PhD
College of Computer Studies
De La Salle University
[email protected]
Definition
A case study is an empirical inquiry that
 investigates a contemporary
phenomenon within its real-life context,
especially when
 the boundaries between phenomenon
and context are not clearly evident.
.3
Definition
Case study inquiry
 copes with the technically distinctive
situation in which there will be many
more variables of interest than data
points, and as one result
 relies on multiple sources of evidence,
with data needing to converge in a
triangulating fashion, and as another
result
 benefits from the prior development of
theoretical propositions to guide data
collection and analysis.
Relevant Situations for Different
Strategies
Strategy
Experiment
Form of
Research
Question
Requires
Control of
Behavioral
Events?
Focus on
Contemporary
Events?
How, why?
Yes
Yes
Survey
Who, what,
where, how many,
how much
No
Yes
Archival
analysis
Who, what,
where, how many,
how much?
No
Yes/No
History
How, why?
No
No
Case study
How, why?
No
Yes
Phases of Case Study Research
1.
2.
3.
4.
5.
Design
Preparation for data collection
Data collection
Analysis of evidence
Reporting
Phase 1: Research Design Components of Research Design
1.
2.
3.
4.
5.
Research questions
Theoretical propositions, if any
Unit(s) of analysis
Logic linking the data to the propositions
Criteria for interpreting the findings
Phase 1: Research Design –
Role of Theory in Design Work
Example of a theory:
• The case study will show why
implementation only succeeded when the
organization was able to re-structure, and
not just overlay the new MIS on the old
organizational structure.
Phase 1: Research Design –
Role of Theory in Design Work
A rival theory:
• The case study will also show why the
simple replacement of key persons was not
sufficient for successful implementation.
Phase 1: Research Design Case Study Designs
Single-case designs
Holistic
(Single-unit of
analysis)
Multiple-case designs
Context
Case
Context
Context
Context
Case
Case
Case
Context
Embedded
(multiple units
of analysis)
Case
Embedded Unit
of Analysis 1
Embedded Unit
of Analysis 2
Context
Context
Context
Case
Case
Case
Embedded Unit
of Analysis 1
Embedded Unit
of Analysis 1
Embedded Unit
of Analysis 1
Embedded Unit
of Analysis 2
Embedded Unit
of Analysis 2
Embedded Unit
of Analysis 2
Phase 1: Research Design –
Tests of Quality of Designs
 Construct validity
• Establishing correct operational measures
•
for the concepts being studied (and not
using “subjective” judgments to collect data)
Tactics
 Use multiples sources of evidence
 Establish chain of evidence
 Have key informants review draft case study
report
Phase 1: Research Design –
Tests of Quality of Designs
 Internal validity
• Establishing a causal relationship, whereby
•
certain conditions are shown to lead to other
conditions
Tactics




Do pattern-matching
Do explanation-building
Address rival explanations
Use logic models
Phase 1: Research Design –
Tests of Quality of Designs
 External validity
• Establishing the domain to which a study’s
•
findings can be generalized
Tactics
 Use theory in single-case studies
 Use replication logic in multiple-case studies
Phase 1: Research Design –
Tests of Quality of Designs
 Reliability
• Demonstrating that the operations of a study
•
– such as the data collection procedures –
can be repeated, with the same results
Tactics
 Use case study protocol
 Develop case study database
Phase 2: Preparation for Data
Collection – Case Study Protocol
 An overview of the case study project (objectives,
issues, topics being investigated)
 Field procedures (credentials and access to
sites, sources of information)
 Case study questions (specific questions that
the investigator must keep in mind during data
collection)
 A guide for case study report (outline, format for
the narrative)
Phase 2: Data Collection –
Skills of a Good Case Study Investigator
 Able to ask good questions – and interpret
the answers
 Good listener
 Adaptive and flexible
 Has firm grasp of the issues being studied
 Unbiased by preconceived notions
Phase 3: Data Collection –
Sources of Evidence
 Documents
 Archival records
 Interviews
 Direct observation
 Participant-observation
 Physical artifacts
Documentation
Documents could be letters, memoranda,
agendas, administrative documents,
newspaper articles, or any document that is
relevant to the investigation.
Archival Records
Archival records can be service records,
organizational records, lists of names,
survey data, and other such records.
Interviews
Interviews are one of the most important
sources of case study information. There are
several forms of interviews that are possible:
Open-ended, Focused, and Structured or
survey.
Other Sources of Evidence
Direct observation occurs when a field visit
is conducted during the case study.
Participant-observation makes the
researcher into an active participant in the
events being studied.
Physical artifacts can be tools, instruments,
or some other physical evidence that may
be collected during the study as part of a
field visit.
Phase 3: Data Collection –
Principles of Data Collection
1. Use multiple sources of evidence
2. Create a case study database
3. Maintain a chain of evidence
Convergence of Evidence
Archival
Records
Documents
Open-ended
Interviews
FACT
Observations
(direct and
participant)
Structured
Interviews and
surveys
Focus
Interviews
Four Types of Triangulation
1.
2.
3.
4.
Data triangulation
Investigator triangulation
Theory triangulation
Methodological triangulation
Components of a Case Study
Database




Notes
Documents
Tabular materials
Narratives
Maintaining a Chain of Evidence
Case Study Report
Case Study Database
Citations to Specific Evidentiary
Sources in the Case Study Database
Case Study Protocol
(linking questions to protocol topics)
Case Study Questions
Phase 4: Analysis of Evidence
Data analysis
• consists of examining, categorizing,
•
tabulating, testing, or otherwise recombining
both quantitative and qualitative evidence to
address the initial propositions of a study
is one of the least developed and most
difficult aspects of doing case studies
Phase 4: Analysis of Evidence
Three General Strategies
 Defending theoretical propositions
• most preferred
 Thinking about rival explanations
• you should have pursued your data
collection about them vigorously
 Developing a case description
• least preferred because you might
meander
Phase 4: Analysis of Evidence
Specific Analytic Techniques
 Pattern matching
• Nonequivalent dependent variables as a
pattern
• Rival explanations as patterns
• Simpler patterns
 Explanation building
Phase 4: Analysis of Evidence
Specific Analytic Techniques
 Time series analysis
• Simple time series
• Complex time series
 Logic models
 Cross-case synthesis
Phase 4: Analysis of Evidence –
Four Principles of Analysis
1. Attend to all the evidence.
2. Address, if possible, all major rival
interpretations.
3. Address the most significant aspect of
your case study.
4. Use your own prior, expert knowledge in
your case study.
Phase 5: Reporting Case Studies
As a general rule, the compositional phase
puts the greatest demands on a case study
investigator.
Phase 5: Reporting Case Studies
– Targeting Case Study Reports
For academic colleagues…
• the connections among the case study, its
findings, and previous theory or research are
likely to be most important.
For nonspecialists…
• the descriptive elements in portraying some
real-life situation, as well as the implications
for action, are likely to be more important.
Phase 5: Reporting Case Studies
– Targeting Case Study Reports
For a thesis committee…
• mastery of the methodology and theoretical
issues, along with an indication of the care
with which the research was conducted, is
important.
For research funders…
• the significance of the case study findings,
whether cast in academic or practical terms,
is probably as important as the rigor with
which the case study was conducted.
Phase 5: Reporting Case Studies
– Formats for CaseStudy Reports
 Classic single-case study
 Multiple case version of the classic case
• Multiple narratives, usually presented as
separate chapters, followed by a chapter
on cross-case analysis and results
 Series of questions (research questions)
and answers
 Entire report as cross-case analysis
Phase 5: Reporting Case Studies
– Procedures
1. Start composing early in the analytic
process.
• Draft the RRL, methodology, and
descriptive (quantitative and qualitative)
data about the case(s).
2. Determine whether you can disclose the
identities of both the case and the
individuals.
• Disclosing both identities is desirable
because it facilitates integration (with
other research) and review.
Phase 5: Reporting Case Studies
– Procedures
• However, if anonymity is needed, see if
•
•
the anonymity of the individuals might be
sufficient.
If not, you can name the individuals but
avoid attributing any particular point of
view or comment to a single individual.
Or, in multiple-case studies, compose only
a cross-case analysis in which individual
responses are not disclosed.
Some Occasions When
Anonymity is Necessary
 When the case study has been on a
controversial topic
 When the issuance of the case study
report may affect the subsequent actions
of those that were studied
 When the purpose of the case study may
be to portray an “ideal” type
Phase 5: Reporting Case Studies
– Procedures
3. Have the draft report reviewed, not just by
peers, but also by the participants and
informants in the case.
• The informants and participants may
disagree with an investigator’s
conclusions and interpretations, but they
should not disagree over the actual facts
of the case.
What Makes an Exemplary
Case Study?
 The case study must be significant.
• The case(s) are unusual and of general
•
public interest
The underlying issues are nationally
important, either in theoretical terms or in
policy of practical terms
 The case study must be “complete”.
• Logical argument or the presentation of
evidence shows that as the analytic
periphery is reached, the information is of
decreasing relevance to the case study.
What Makes an Exemplary
Case Study?
 The case study must consider alternative
perspectives.
• The case study anticipates alternatives,
even advocates their positions as forcefully
as possible, and shows – empirically – the
basis on which such alternatives can be
rejected.
 The case study must display sufficient
evidence.
• The case study judiciously and effectively
presents (only) the most relevant evidence,
so that the reader can make an
What Makes an Exemplary
Case Study?
•
independent judgment.
The evidence should convince the reader
that the investigator has indeed been in
field, made penetrating inquiries while
there, and has become steeped in the
issues about the case.
 The case study must be composed in an
engaging manner.
• The good investigator might think that the
case study contains earth-shattering
conclusions.