Transcript Design Research in Information Systems Research

Proactive Research Approaches:
Design and Action Research
Professor Matti Rossi (Helsinki School of Economics)
& Professor Maung Sein (Agder University College,
Norway)
This workshop is based on an ongoing collaborative effort with Dr.
Sandeep Purao, Penn State University, USA, Dr. Ola Henfridsson and
Dr. Rikard Lindgren both of Viktoria Institute, Sweden
Partnership-seminar -
Agenda
• To present the “proactive” research paradigms in
IS research:
– Design Research
– Action Research
• To map the similarities between the two methods
and discuss how each can learn from the other
– Action Design – an integrated approach
• To illustrate the concepts through an example
Partnership-seminar -
Program
•
•
•
•
•
09:00
09:20
10:00
10:30
10:40
-
09:20
10:00
10:30
10:40
11:30
Proactive research approaches
Design Research (DR)
Action Research (AR)
Break
Mapping AR & DR
– Action Design - integrated approach to designing in
action
• 11:30 - 12:30 lunch
• 12:30 - 13:30 discussion of possible research
projects and wrap-up
Partnership-seminar -
Matti’s background
• Acting professor of information systems at Helsinki
School of Economics
• Held visiting assistant professorships both at
Georgia State University and Erasmus University
Rotterdam
• All studies at University of Jyväskylä
• Thesis on advanced CASE tools 1998
• Minority owner and former board member of
MetaCase Consulting (www.metacase.com) a spin
off of the thesis project
Partnership-seminar -
Maung’s background
• Personal background
– Ethnically Arakanese, Born in Pakistan/Bangladesh
– Moved to USA in 1982, to Norway in 1995 … and back in 1998
• Educational background
– Undergraduate (Electronics Engg.), Masters (Finance and IS), PhD.
(MIS)
• Work experience
– Industry: Hardware, software, systems analyst, consultant
– Academic: Indiana, Florida International, Georgia State Universities
(USA), University of Bergen (Norway)
• Main research areas:
– End-user training and learning (Conceptual frameworks, best
practices)
– IS development (Methods, projects, conceptual modelling)
– Theoretical/conceptual issues (Relevance of research, Research
methods)
– Societal issues of IT (ICT and national development, e-Government)
Partnership-seminar -
Proactive Research Approaches:
Design and Action Research
Partnership-seminar -
Research perspectives
• Natural sciences typically observe reality
• Social sciences interpret organizational and social
phenomena
• Computer science assumes natural science as the
way of doing research
• Information systems take a more multiparadigmatic view
Partnership-seminar -
The Complex world that we operate
in…
Letters
Social Sciences
Natural Sciences
Management
Engineering
Information Systems
Information Systems Practice
Partnership-seminar -
Reactive and Proactive paradigms
• “Reactive” approaches take the world as a stable
environment governed by laws that need to be
discovered by scientists (i.e. are descriptive in
nature)
• “Proactive” approaches aim at developing ways to
achieve human goals (i.e. are prescriptive or
constructive)
• The distinction between the two:
– natural vs. artificial phenomena
– the intent of the research.
Partnership-seminar -
Reactive and Proactive paradigms
• Goals of research in Reactive paradigms
– Explanation research: Truth Seeking and/or
Understanding
– Knowledge for its own sake
• Goals of research in Proactive paradigms
– Design and Action Research: Improving Practice,
solving problems
– Utilitarian perspective
Partnership-seminar -
Link between Reactive and Proactive
paradigms
• Proactive (Design) creates artifacts, giving the
phenomena that Reactive (Explanation research)
can study
• Proactive (Design) may depend on knowledge
created by Reactive in creating new artifacts
• Proactive (Action) may depend on knowledge
created by Reactive as a basis for intervention
Partnership-seminar -
Proactive Research Approaches:
Design Research
Partnership-seminar -
Why use Design Research approach?
• Things that do not exist cannot be observed
• "... without research efforts directed toward
developing new solutions and systems, there would
be little opportunity for evaluative research"
Nunamaker et al., 1991
Partnership-seminar -
Remarks...
• “Design … is the core of all professional training; it
is the principal mark that distinguishes the
professions from the sciences.”
• “ … business schools have become schools of finite
mathematics.”
Herbert A. Simon, The Sciences of the Artificial. The
MIT Press, 1981.
Partnership-seminar -
Design Research
• Reference disciplines
– Psychology, sociology, ethnography, computer
science, economics, management
• Level of analysis
– Society, profession, inter-org, org, project, group,
individual, concept, system, component
Partnership-seminar -
Design research premises
• Ontology:
– Realist (real world exists but we are not seeking it)
• Epistemology:
– We can intervene in the world to improve it
• Methodology:
– Development/Design of systems, models
– Qualitative and exploratory way of thinking, but
could lead to quantitative confirmations
• Axiology:
– Relevance is stressed
Partnership-seminar -
When to use Design Research?
• New areas
• There are theories, but they cannot be tested
• There are clear deficiencies in former systems
Partnership-seminar -
Products of Design Research
Purao 2002
Partnership-seminar -
Steps in Design Research
• Identify a need
Problem solving
• Build
Model, Instantiate
• Evaluate
Verify, Validate
• Learn
Current, Emergent
• Theorize
Conceptualize and generalize findings
Partnership-seminar -
Identify a need
• Find a deficiency in current systems
• Do field studies of problems in the field
• After a problem is found perform a thorough
search of previous research on the topic
• If previous research does not address the problem
and it is interesting
– > go to next step
Partnership-seminar -
Build
• Design the system
• Use good software engineering principles
• Get the best tools and reuse everything that You
can
• Define the measures of success
– > Just do it!
Partnership-seminar -
Evaluation of Design Research
•
•
•
•
•
Analysis of the built systems
Trials in laboratory
Field trials
Commercial success
Measure of success should be defined before the
implementation
• Systems should be evaluated against the defined
measures
Partnership-seminar -
Evaluation: (Hevner et al., 2004)
Partnership-seminar -
Evaluation: Chen et al.
• The purpose is to study an important phenomenon
in areas of information systems through system
building
• The results make a significant contribution to the
domain
• The system is testable against all the stated
objectives and requirements
• The new system can provide better solutions to IS
problems than the existing systems and design
expertise gained from building the system can be
generalized for future use.
Partnership-seminar -
Evaluation: Sein, Purao, Rossi - 1
• Internal criteria:
– Match between the artifact and the “abstract idea”.
How well does the artifact embody the abstract idea
that is being researched?
– Match with generally accepted principles of designed
artifacts
– Is the artifact a “good system” as defined by the
field (good interfaces, easy to use etc.)
Partnership-seminar -
Evaluation: Sein, Purao, Rossi - 2
• External:
– Advancement of design theory:
• Is the abstracted idea generalisable to other contexts or
at least advance our understanding of other design
contexts?
• Are the ideas, if not the elements of the artifact,
reusable?
– Advancement of information systems discipline:
• Does the artifact behave in / influences/improves the
environment/context in which it is intended to be used?
Partnership-seminar -
Examples of measures
• How well the proposed algorithm performs in real
life situations
• The speed of systems development using the
constructed system
• The market share won
– Lukka & Kasanen’s weak & strong market test
Partnership-seminar -
Learn and theorize
• Reflect on the process and product
• Try to generalize findings
• Try to confirm or reject the original assumptions
– > Start a new cycle, which analyzes the system in
use
– < Start from the beginning...
Partnership-seminar -
Design Research Guidelines
Partnership-seminar -
Reporting a building project
• Introduction
– explain the problem
– Identify the related research
• Research method
– what type of approach used?
– Nunamakerian, Marchian or Hevnerian?
• Identify the system properly
– What problem it solves?
– What is the “new idea” behind the system and is indeed
“new”
• State the approach
– explain logical design,
– some idea about the physical/platform aspects,
– explain the implementation project
• The product evaluation
– Measure against the "success" criteria
• In Discussion Specify lessons learnt and theorize
Partnership-seminar -
Further reading
• March, S., Hevner, A. and Ram, S. (2000). "Research
Commentary: An Agenda for Information Technology
Research in Heterogeneous and Distributed
Environments." Information Systems Research 11(4):
327-341.
• Hevner, A., March, S., Park, J. and Ram, S. (2004).
"Design Science in Information Systems Research." MIS
Quarterly 28(1): 75-105.
• Purao, S. (2002). “Design Research in the Technology of
Information Systems: Truth or Dare.” GSU Department
of CIS Working Paper. Atlanta.
• Brooks, F. (1996). "The Computer Scientist as Toolsmith
II." Communications of the ACM 39(3): 61-68.
• Link:
http://www.isworld.org/Researchdesign/drisISworld.htm
Partnership-seminar -
Partnership-seminar -
Proactive Research Approaches:
Action Research
Partnership-seminar -
Action Research: Definition
• ”Action research simultaneously assists in practical
problem-solving and expands scientific knowledge,
as well as enhances the competencies of the
respective actors, being performed collaboratively
in an immediate situation using data feedback in a
cyclical process aiming at an increased
understanding of change processes in social
systems and undertaken within a mutually
acceptable ethical framework.”
Hult & Lennung, 1980
Partnership-seminar -
Action research premises
• Ontology:
– Information systems are Social systems with
technical implications or Technical systems with
social implications
• Epistemology:
– Knowledge for action
– Knowledge for critical reflection
– Reflective science or Philosophy
• Methodology:
– Active intervention in organizational contexts
– Qualitative and exploratory way of thinking
• Axiology:
– Relevance is vital: prime goal is problem solving
Partnership-seminar -
Action research basics
• Assumptions:
– Social settings cannot be reduced for study
– Action (i.e. intervention) brings understanding
– Action research is performed collaboratively;
Researchers and practitioners are partners;
• Action research is building/testing theory within
context of solving an immediate practical problem
in real setting
• Thus it combines theory and practice, researchers
and practitioners, and intervention and reflection
• Action research is not consulting: it is action, but
still research
Partnership-seminar -
Action Research process
Partnership-seminar -
Susman & Evered, 1978
Action Research process
• Diagnosing a problem
– develop a theoretical premise
• Action planning
– guided by theoretical framework
• Action taking
– intervention, introducing change
• Evaluating, reflecting
– effects of change, theoretical premises
• Specifying learning
– “double loop”
– feed next iteration
– theorise
Partnership-seminar -
”Action Case”
• Focus on method development and evaluation
– ”Action” - from action research
– ”Understanding of context” from case studies
– ”experimental approach” - from field experiment
• Features:
–
–
–
–
–
projects with short duration
intervention in real time
emphasis on ”quasi-experiments”
reducing complexity - one issue at a time
focus on changes in small scale
Partnership-seminar -
Canonical Action Research Criteria
(Adapted from Davison, R. M., Martinsons, M. G., and Kock, N. “Principles of Canonical Action Research,” Information Systems Journal (14:1),
2004, pp. 65-86.)
Criterion
Description
1. Principle of Researcher-Client
Agreement (RCA)
The RCA provides the basis for mutual commitment and role expectations.
2. Principle of Cyclical Process Model
(CMP)
The CPM consists of the stages diagnosing, action planning, action taking, evaluating,
and specifying learning.
3. The Principle of Theory
Theory must play a central role in action research.
4. The Principle of Change Through
Action
Action and change are indivisible research elements related through intervention
focused on producing change.
5. The Principle of Learning Through
Reflection
Considered reflection and learning allow a researcher to make both a practical and
theoretical contribution.
Partnership-seminar -
Further reading
• Baskerville, R. "Investigating information systems
with action research", (paper) (A tutorial on how to
do action research)
• Braa, K. "Priority workshops: Springboard for user
participation in redesign activities", Proceedings of
the Conference on Organisational Computing,
ACMSIGOIS, California, 1995, 90-112.
• Hult, M. and Lennung, S-A. (1980). Towards a
definition of action research: A note and
bibliography, Journal of Management Studies, May,
pp. 241-250.
Partnership-seminar -
Break!
Partnership-seminar -
Proactive Research Approaches:
Mapping AR & DR
Partnership-seminar -
Commonalities between AR and DR
• Ontology: the phenomenon of interest does not
remain static through the research process.
• Epistemology: knowledge is created through
intervening to effect change, and reflecting on this
intervention.
• Axiology: both value the relevance of the research
problem, and emphasise practical utility and
theoretical knowledge
Partnership-seminar -
A Common Paradigm: Pragmatism
• Applying the tenets of pragmatism (that
characterize AR) to DR:
– Consequences defining concepts: In DR, there is a
need to establish the purpose of the resultant
artefact.
– Practical outcome embodying truth: The focus of DR
is practical action, which ensures that the notion of
truth lies in the utility of the produced artefact.
– Logic of controlled inquiry: The essence of DR is
that designing must inform theory in that the
produced artefact should embody a theoretical
premise or a “new idea,” which can be evaluated by
evaluating the artefact.
– Social context of action: In DR, the act of designing
is socially and organizationally situated, specifically
in our conceptualization.
Partnership-seminar -
AR-DR Cross-Fertilization
• Adding Reflection to Augment Learning from DR
– Interjecting an AR cycle at the last stage of the DR
process
– A DR project may be framed as an AR project if an
organizational problem needs to be solved, and the
action involves building a system
• Concretizing Learning from AR by Adding Build
– Frame the output of AR as a DR artefact, such as
prototypes, frameworks, or models
– Enhancing the AR action taking phase by including
the building of a design artefact.
• Envisioning an AR-DR Integrated Research Process
– Action Design
Partnership-seminar -
AR-DR Cross-Fertilization
Diagnosing
a problem
Action
planning
Start a DR process ->
Action taking
Build
Evaluating, reflecting
Specifying learning
Partnership-seminar -
Mapping AR and DR processes
Action Research
Design Research
•
•
•
•
•
DR1
DR2
DR3
DR4
DR5
-
Identifying a need
Building
Evaluating
Learning
Theorizing
•
•
•
•
•
AR1
AR2
AR3
AR4
AR5
-
Mapping
Map 1 - DR1 -> AR1
Map 2 - DR2 -> AR2 + AR3
Map 3 - DR3 -> AR4
Map 4 - DR4 + DR5 -> AR5
Partnership-seminar -
Diagnosing a problem
Action planning
Action taking
Evaluating, reflecting
Specifying learning
Action Design
DR1 - Identifying a need
DR2 - Building
Problem
Definition
AR1 - Diagnosing a problem
Building and Intervention
AR2 - Action planning
AR3 - Action taking
DR3 - Evaluating
DR4 - Learning
DR5 - Theorizing
Evaluation
AR4 - Evaluating, reflecting
Reflection and Learning
AR5 - Specifying learning
Partnership-seminar -
Principles of Action Design - 1
• Principle of utility:
– “truth” in knowledge is determined by the utility of
the developed artifact (based on pragmatism,
Baskerville and Myers 2004)
• Principle of artifact:
– The product of the AR-DR synthesized approach is an
artifact (Hevner et al., Järvinen, DR core)
• Principle of knowledge through building,
intervention and reflection
– The epistemology of our research subscribes to
Partnership-seminar -
Principles of Action Design - 2
• Principle of evaluation in an organizational context
(March & Smith, AR core)
• Principle of mutual learning and informing:
– Theory-to-practice and practice-to-theory (CAR core)
• Principle of purposeful action (CAR core)
Partnership-seminar -
Map 1 (Problem definition)
• DR1 = AR1
• Both start with diagnosing the problem, but
• Question is the level of abstraction of problem
articulation: abstract at the beginning of the
research process or at the end?
– in DR, abstraction a priori is an important concern
– in AR, it is debatable
• ideal to define it at a higher level of abstraction
• often it is defined in a contextual manner
Partnership-seminar -
Map 2 (Building and Intervention)
• DR2 = AR2 + AR3
• Design and action are both intervening into reality
to improve or support existing organizational
activities/processes, but
– In DR the idea of intervention is not clearly
“planned” i.e. it does not involve a clear set of steps
– In AR, planning and acting are distinct steps
Partnership-seminar -
Map 3 (Evaluation)
• DR3 = AR4
• Both approaches stress problem solving
• For DR, evaluation involves additionally:
– Internal criteria
• Match between the artifact and the “abstract idea”
• Match with generally accepted principles of designed
artifacts
– External criteria
• Advancement of design theory
• Advancement of information systems discipline:
Partnership-seminar -
Map 4 (Reflection and Learning)
• DR4 + DR5 = AR5
• Both depend on reflection and generalization to
theoretical concepts and other contexts
• In AR, what the practitioner members of the
research team learn is vital
Partnership-seminar -
DR-AR Mapping: Some Issues
• Role of theory
– AR community is divided on whether a priori theory is
necessary
– In DR, a theoretical stance is not a prerequisite to starting
the research process; theoretical stance often emerges
during design.
• Role of the user
– In AR, there is always a user (practitioners)
– In DR, a user is either present (systems designed for
specific organizational context), or assumed
• Iteration
– In DR, iterations are more frequent than in AR
• Continual modification – element of play
– Design research involves play – in DR, the idea of
intervention is true though it is not clearly “planned” i.e. it
does not involve a clear set of steps
Partnership-seminar -
Further reading
• Cole, R., Purao, R., Rossi, M. and Sein. M.K.
(2005). ”Being Proactive: Where Action Research
meets Design Research”, Proceedings of ICIS
2005, Las Vegas, USA, Dec 2005
• Järvinen, P. (2005). Action Research as an
Approach in Design Science”, presented in THE
EURAM (European Academy of Management)
Conference, Munich, May 4-7, 2005
Partnership-seminar -
Partnership-seminar -
Contact Information
Matti Rossi
Helsinki School of Economics
P.O. Box 1210
FIN-00101 Helsinki
Finland
Email: [email protected]
Phone: +358-9-43138996
Fax : +358-9-43138777
• http://www.hkkk.fi/~mrossi
Partnership-seminar -