Transcript AOM Process Workshop - Process Research Methods

Process Research Workshop: A Spectrum
of Methods, AOM PDW, Saturday, August 6
 102 SPDW: (RM, BPS, OB, OMT, TIM) Process Research
Workshop I & II A Spectrum of Methods
8:30am - 11:30am; 1 – 4 pm Hawaii Convention Center: Room 311
 Pre-registration required at
https://spears.okstate.edu/rmdpdwregister. There is a $15 fee for
non members of the Research Methods Division.
 Presenters: Ann Langley, HEC Montreal; Kevin Dooley, Arizona
State U.; Marshall S. Poole, Texas A&M U.; Andrew H. Van de
Ven, U. of Minnesota
4/8/2015
Process Research Workshop: A Spectrum of
Methods
Agenda
8:30
8:45
9:15
9:45
10:15
10:30
11:00
Welcome & Introductions
Process Research Epistemology – Scott Poole
Small Group Exercise in studying a problem as a process
Discussion and Break
Designing Process Research Studies – Andy Van de Ven
Small Group Exercise in designing a process study
Discussion
11:30
Conclusion – Lunch on your own
1:00
1:30
2:00
2:30
3:00
3:30
4:00
Qualitative Methods for analyzing process data – Ann Langley
Exercise in writing a process study
Discussion and Break
Quantitative Methods for analyzing process data – Kevin Dooley
Small group discussions
Concluding Discussion
End
4/8/2015
Participants’ Questions for Morning Session
1.
2.
3.
4.
Discuss representation in process research. (Clive Smallman, Lincoln U.
New Zealand). Discuss the relevance of personal experience in field
research in light of Bourdieu’s (2003) “Participant Objectivation” (Francois
Collet, Oxford)
Who offers courses dedicated to process research methods? What are
platforms for process scholars to exchange ideas and methods? (Matthias
Brauer, U. of St. Gallen).
How do you combine different theories? I plan to use the Alternate
Templates Strategy to analyze data on inter-organizational relationships and
write a narrative using three theoretical lenses: the transaction costs
economy -theory, resource based view and evolutionary theory. (Paivi
Karjalainen, Teliasonera.com)
Process Research has been criticised for its use second hand retrospective
reports given by senior executives, the absence of consideration for
managerial agency issues, its lack of practical relevance, the absence of
consideration for content (diversification, internalization context) and the
difficulty to generalise from some in-depth empirical studies. (Johnson, Melin
et al. 2003). Are these critics justified ? (Francois Collet, Oxford U.)
4/8/2015
Participants’ Questions for Afternoon
Session
1.
2.
3.
4.
What are the most appropriate statistical packages to handle process
analyses? (Matthias Brauer, U of St. Gallen,Switzerland)
When do you use parametric vs. non-parametric tests in event-based
process analysis? or put differently: when do you view your events as a
sample or an entire population? (Matthias Brauer)
How do you apply multiple sensemaking strategies in one paper given the
page limitations of ordinary management journals? (Matthias Brauer)
How should authors report/frame a "QUAL-quant" process study so that
reviewers and editors steeped in variance-theoretic methods won't place
inordinate weight on the supporting quantitative methods, killing papers that
are mostly qualitative in nature? (Todd Chiles, U.of Missouri-Columbia)
4/8/2015
Sourcebook for Process Research
Methods
Marshall S. Poole, Andrew Van de Ven, Kevin Dooley, and Michael
Holmes,
New York: Oxford University Press, 2000.
1. Perspectives on Change and Development
2. Process Theories and Narrative Explanations
3. Process Theories of Organizational Change
4. Overview: Methods for Process Research
5. The Design of Process Research Studies
6. Stochastic Modeling
7. Phasic Analysis
8. Event Time Series Regression Analysis
9. Nonlinear Dynamical Analysis
10.Conclusions
4/8/2015
Epistemology of Process Research
•
•
•
•
•
Marshall Scott Poole
Texas A&M University
4/8/2015
The Meaning of Process
Ontological views of Process
Process and Variance Methods
Cognitive Transitions
Exercise: Study a problem as a
process
Variance and Process Epistemologies
VARIANCE APPROACH
Fixed entities with varying attributes
Entities participate in events and may
change over time
Explanations based on necessary and
sufficient causality
Explanations based on necessary
causality
Explanations based on efficient
causality
Explanations based on final, formal, and
efficient causality
Generality depends on uniformity
across contexts
Generality depends on versatility across
cases
Time ordering among independent
variables is immaterial
Time ordering of independent events is
critical
Emphasis on immediate causation
Explanations are layered and
incorporate both immediate and distal
causation
Attributes have a single meaning over
time
Entities, attributes, events may change
in meaning over time
Lawrence Mohr
4/8/2015
PROCESS APPROACH
Langley’s Picture of Variance and Process Theories
Variance Theory
Attributes of:
• Environment (x1)
• Technology (x2)
• Decision
Process (x3)
• Resources (x4)
Process Theory
Organization
Outcomes
(Y)
Y = f(x1, x2, x3, x4)
4/8/2015
State
B
State
A
• events
• activities
•choices
T0
Ann Langley
T1
Bruner’s Two Modes of Thought
Logico-Scientific Mode
Narrative Mode
Purpose
Develop and test a theory that
explains the causes or
consequences of a general
phenomenon in its context.
Develop a plausible story
that interprets meaning to a
particular experience or
sequence of events
Method
Logical “if-then” propositions
that derive testable hypotheses
among variables in specified
context
Plot linking intentional
actions of characters in in
events and settings.
Discourse:
Triggers assumptions,
Is reflexive, and
open to multiple views
Evaluation
Criteria
Valid argument
Empirical truth
Boundary conditions
Verisimilitude
A good story
Reflexive
Open to multiple views
4/8/2015
Jerome Bruner
(1915 - )
EVOLUTION (Competitive Change) DIALECTIC (Conflictual Change)
Variation
Selection
Retention
Multiple
Entities
Thesis
Conflict
Synthesis
Antithesis
Population Scarcity
Environmental Selection
Competition
Unit of
Change
Pluralism (Diversity)
Confrontation
Conflict
LIFE CYCLE (Regulated Change) TELEOLOGY (Planned Change)
Dissatisfaction
4 (Terminate)
Single
Entity
Stage 3
(Harvest)
Stage 1
(Startup)
Implement
Goals
Stage 2
(Grow)
Set/Envision
Goals
Immanent Program
Regulation
Compliant adaptation
Prescribed
Search/
Interact
Purposeful enactment
Social construction
Consensus
Mode of Change
Constructive
Process Models of Organization Change
Note: Arrows on lines represent likely sequences among events, not causation between events.
Source: Van de Ven & Poole, Explaining Development and Change in Organizations, AMR, 1995.
4/8/2015
Approach I
Variance
Methods
Approach IV
Variance study of change in organizations
Variance study of process patterns
Causal analysis of independent variables
explaining change (dependent variable)
Quantitative analysis of event time series:
Markov, time series, event history, &
nonlinear complex adaptive systems
Newtonian view of time
Time is a variable of change process
Epistemology
Approach III
Approach II
Process study narrating emergent
organizing activities
Process study narrating sequence of
change events in organization
Process
Methods
Progressions of change (stages, cycles, etc) Qualitative narrative interpretation of
complexity metaphor
In the development of org. entity
Transaction or event-based view of time
Social construction view of time
a verb, a process
A noun, a thing
Organizational
Ontology
Alternative Approaches for Studying Organization Change
4/8/2015
Designing Process Research Studies
•
•
•
•
Andy Van de Ven
U. Of Minnesota
4/8/2015
Basics of Process Research
Designing Field Studies
Analyzing Process Data
Exercise: Design a process
study
Basics of Process Research
• Define the meaning of process:
• A logic that explains a causal relationship
• A category of concepts or variables
• A narrative of how things change over time
• Clarify theory of process (vs. variance theory)
• process vs. variance theories
• life cycle, teleology, dialectic, & evolution process theories
• Adopt a process vocabulary
• simple, multiple, cumulative, conjunctive & iterative
progressions
• Design research to observe and analyze process
4/8/2015
4/8/2015
Steps & Decisions in Designing Process Study
Key Step
1. The topic
Key Decision(s)
Suggestions
What is the research question or How/why an organization changes?
problem?
How a change process unfolds?
2. The research question Variance or process research?
Variance for causal questions
Process is geared to how questions
3. Frame of reference
Who’s viewpoint is featured?
Observe change process from a
What is the researcher’s role?
specific participant’s viewpoint
4. Mode of inquiry
Sound general argument or
General explanations – causal theories
good particular story?
Particular understanding - narratives
5. Conceptual model
One or more models/stories?
Compare plausible alternative models
Which ones?
6. Observational method Real-time or historical
Observe before outcomes are known
observations?
7. Field research design How design the field research? Develop parallel, synchronic, and
Diachronic research design
8. Sample diversity
Homogeneous or heterogeneous? Compare the broadest range possible.
Compare different viewpoints.
9. Sample size
Number of events and cases?
Focus on number of temporal
intervals and granularity of events
4/8/2015
A Critical Realist Call for Intellectual Pluralism
• There is a real world out there, but our understanding of it is limited
• All facts, observations & data are theory laden
• Social science has no absolute, universal, error-free truths or laws
• No form of inquiry can be value free & impartial; each is value full
• Knowing a complex reality demands use of multiple perspectives
• Robust knowledge is invariant (in common) across multiple models
• Models that better fit the problems they are intended to solve are
selected, producing an evolutionary growth of knowledge.
4/8/2015
Steps & Decisions for Analyzing Process Data
Key Step
Key Decision(s)
1. Developing process What concepts or issues will you
concepts
look at?
2. Defining incidents & What activities or incidents are
events
indicators of what events?
3. Specifying an incidentWhat is the qualitative datum?
Suggestions
Begin with sensitizing concepts
and revise with field observations
Incidents are observations, events
are unobserved constructs
Develop decision rules to bracket
or code observations
4. Measuring an incidentWhat is a valid incident?
Ask informants to verify incidents
5. Identifying events What strategies are available to Apply a mix of qualitative and
tabulate and organize field data? quantitative data analysis methods
6. Developing process How move from surface
Identify characteristics of
theory
observations
narrative theory
to a process theory?
4/8/2015
Barley’s Field Research Design
Barley, S (1990) “Images of Imagining: Notes on Doing Longitudinal Field Work,” Organization Science, 1, 226.
4/8/2015
4/8/2015
Qualitative Methods for Analyzing
Process Data
•
•
•
•
•
•
•
Narrative Strategy
Template Matching
Grounded Theorizing
Visual Mapping
Temporal Bracketing
Synthetic Strategy
Quantitative Strategy
Ann Langley
HEC, Montreal
4/8/2015
Quantitative Methods for Analyzing
Process Data
Kevin Dooley
Arizona State University
4/8/2015
• Analyzing Event Sequence
Data
• Structures of Event Time
Series
• Models for examining
different structures of time
series
• Orderly data
• Chaotic data
• Random data
Quantitative & qualitative
 Some sources will naturally be quantitative
 But many will be qualitative
 Symbolic time series
• Event type sequences
 Numerical time series
• Number of events per fixed time period
• Quantification of qualitative content (manifest/computerized
content analysis)
 When change qualitative to quantitative?
• Large volumes of qualitative data
• Modeling skills present on research team
4/8/2015
Temporal analysis
4/8/2015
Descriptive analysis
example
 RQ: Are there temporal patterns of new venture
activities (start-up events) which are predictive of
venture emergence?
 Does the type of event matter?
 Method





PSED sample
Case = nascent entrepreneur
A priori event list
Respondent indicates month of event completion
Time series formed by number of events each month
4/8/2015
Activity
Which entrepreneurial
process is more likely to
be successful?
Month
Lower concentration, average timing
SAME RATE
Higher concentration, late (high) timing
Lichtenstein, Dooley, Carter, & Gartner, 2005
4/8/2015
Month
Sequence analysis example
• RQ: Are there temporal patterns of activities in
large-scale, group-based development
activities?
– Does process depend on task?
• Method
–
–
–
–
Development of ebXML standards
Case = Task force
Almost all of process on-line (20k emails)
Text analysis to identify dominant narrative theme
(activity) in each month
4/8/2015
ebXML event sequences
Code
R—Requirements
S—Search
M—Model
D—Design
I—Internal review
E—External review
What story
can you
see?
Business process standard:
R—S—S—R—M—M—M—R—R—R—M--I –I—I—I—E—R—I—I—E—
R—E—E—E
Technical standard:
R—R—R—R—R—D—I—I—I—I—D—D—I—I—D—D—E—E—D—
D—E—E—E—E
Choi, Raghu, Vinze, & Dooley 2005
4/8/2015
Change point analysis
example
 RQ: What happens during organizational emergence?
 Method
 Biweekly interviews and empirical data from
entrepreneur
 Change point analysis of multiple time series
 Temporal analyst blind to case details
4/8/2015
Where is/are the change point(s) in each series?
What story is told?
Lichtenstein, Dooley, &
Lumpkin JBV 2005
4/8/2015
DCP 18 also
corresponded to
• most “central”
Interview
• time of
incorporation
Variable
Data
Estimated CP Degree of Change
EXPENSES
Mean
DCP 18 *
Decreased by 58%
Variance
DCP 18 **
Decreased by 55%
Mean
DCP 18 **
Decreased by 55%
Variance
DCP 18 ***
Decreased by 82%
DCP 18 *
Decreased by 34%
Variance
none
–
Mean
none
–
DCP 18 *
Decreased by 27%
B/W
TOTAL
JOB
4/8/2015
Mean
Variance
Dynamical analysis example
 RQ: What are the generative mechanisms
behind media attention to 9-11 players?
 Methods
 All Reuters articles related to 9-11 over 66 days
(approx. 100 pages/text per day)
 Text analysis to identify influence of name in media
texts
 Time series
• ARMA models
• Spectral analysis
• Chaos detection
4/8/2015
NAME INFLUENCE (STACKED GRAPHS)
Dooley & Corman, 2004
DAYS
POST 9-11
4/8/2015
NAME INFLUENCE (STACKED GRAPHS)
ARMA(2,1) with (stochastic) four day cycle
MA(1), correlated with binLaden
at one day lag
Shift; white noise
Episodic
Sustained episode
DAYS
POST 9-11
4/8/2015
Dynamical Systems:
Patterns over Time
4/8/2015
Induction
Dooley & Van de Ven, 2000
4/8/2015
Correlative analysis
example
 RQ: What are the semiotic processes occurring in
business news genre?
 Method
 Media articles across multiple cases
 Text analysis to identify theme influence, tone, and
intensity
 Change point analysis to identify epochs
 Correlate themes with tone, intensity within epochs
4/8/2015
Table shows whether theme was positively or negatively influenced
with tone or intensity during a particular epoch
• Tone: Ratio of positive to negative words
• Intensity: Ratio of emotive words to non-emotive words
HOW ARE TONE AND INTENSITY USED BY THE MEDIA?
4/8/2015
Why does temporal analysis work?
1. Only time tells stories
2. The dynamics of the parts embed the dynamics of the
whole
Dynamics of (e.g.) g (w, x, y, z) =
Dyn. of g {w(t), w(t-K), w(t-2K), w(t-3K)}
4/8/2015
Pragmatics
 Plot temporal data!
 Software
 Sequence analysis—Social network software based on transition
matrix (e.g. UCINet)
 Time series analysis—Most advanced stats programs (e.g.
Statistica, SPSS)
 Nonlinear dynamics—Chaos Data Analyzer
 Change point analysis—quality control charts; Change Point
Analyzer
 Challenges
 Skills in exploratory statistical modeling
 Communicating to readers
 Can only examine dynamics, change points, and correlation in a
hierarchical manner
4/8/2015
Process as generative
 Dominant closed path is
“normative” (R-M-I-E).
 Requirements-centric
 Lots of transitivity
AGILE
4/8/2015
• No closed path
• Design-centric (hub)
WATERFALL
Additional Slides
For display on questions or issues discussed
4/8/2015
4/8/2015
A Sample Event Data Entry Form
Data Entry Forms
Event #: ______
Date:__________
Event:_____________________________________
__________________________________________
__________________________________________
Observation: _______________________________
__________________________________________
Source: ____________________________________
2. Data Entry Forms
Keywords: __________________________________
4/8/2015
A Sample Event Report
CIP Event Printout as of 02/25/94
Number:
38 Date: 02/01/77
Event: University of Melbourne approaches 3M on a joint venture to
develop and manufacture CI. News of the development of a
"bionic ear" triggers interest of executives at 3M.
Observ: The relationship was not established, and 3M decides to
pursue the "bionic ear" idea separately.
Leader: I S SD
Number:
41 Date: 12/15/77
Event: 3M evaluates U. of Melbourne, Australia proposal for the
"bionic ear." A report to 3M executives states the project is
a promising business opportunity. However, exclusive rights
and patent protection is reported as unclear.
Observ: On the surface the project is very promising -- the US market
potential using $ 1000 device (conservative) is $ 1000 mm. The device
is an emerging technology, I am not aware of any published
on-going research in this type area. (As with heart pacers, the first
company in the market can dominate). There is a good fit with existing
3M technology. On the minus side, I have some doubts about the patent
protection. The Australian proposal does not indicate a strong position.
There is also the problem with the distance involved and the proposal
is rather vague about exclusivity after investments by 3M.
Leader: S C SD
4/8/2015
4/8/2015
Example of Visual Mapping Strategy in CIP Case
Source: Van de Ven, Polley, Garud & Venkataraman, The Innovation Journey, NY: Oxford, 1999.
4/8/2015
Example of Temporal Bracketing Strategy in CIP Case
Source: R. Garud & A. Van de Ven, “An Empirical Evaluation of the Internal Corporate Venturing Process,”
Strategic Management Journal, 13 (1992): 93-109.
4/8/2015
4/8/2015
Example of 3D
Graphing of
Event Sequences
4/8/2015
Langley, A. (1999) “Strategies for Theorizing From Process Data,” AMR, 24, 1, p. 696.
4/8/2015
Narrative Features of Process Theory
• In narrative theory, the the story includes more
than just event sequence. A process theory
should include:
• Sequence in Time
• Focal Actor(s)
• Narrative Voice
• Evaluative Frame of Reference
• Indicators of Content and Context
Brian Pentland, “Building Process Theory with Narrative: From
description to explanation,” Academy of Management Review, 24,
4 (1999): 711-724.
4/8/2015
Xt = kXt-1(1-Xt-1)
k = 1.8
k = 3.2
k = 3.7
4/8/2015
Bifurcation Structure of the Limit Set of
Logistic Map Xt = kXt-1 (1 – Xt-1) for varying
values of k
4/8/2015
Dooley, K. and Van de Ven, A. (1999) “Explaining Complex Organzational Dynamics,”
Organization Science, 10, 3: p. 367.
4/8/2015
Dimension - Physics

1000
100
10
I
II
100 % Deterministic
III
Mathematics
IV
100 % Stochastic
I. Solvable dynamic system, e.g. gear trains, physical pendulum
II. Amenable to perturbation theory, e.g. satellite orbits
III. Chaotic dynamic systems, e.g. climatology, Lorenz equations
IV. Turbulent/stochastic systems, e.g. quantum mechanics, turb. flow
(Source: Morrison, 1991)
4/8/2015
4/8/2015
4/8/2015
4/8/2015