Transcript Factors Affecting DOD FM Usage of a GDSS

Factors Affecting DOD FM
Usage of a GDSS
Jeff Bohler & Dianne Hall, PhD
Auburn University
Pre-ICIS 2006 SIGDSS
Research Workshop
10 December, 2006
The opinions and observations expressed in this presentation are mine alone and
do not represent the opinions of the DOD, the USAF, or the USN.
Overview







Research Motivation & Relevance
Decision Support Method
GRASP
Research Model & Hypotheses
Methodology & Analysis
Results
Limitations & Future Research
10 December, 2006
2
Research Motivation & Relevance
 Changes in the DOD workforce
 Personnel reductions
 Retirement of “Baby Boom” generation
 Shortage of analytical skills in FM
 Financial Manager & Comptrollers School
 Decision Support concept introduced
 What makes this study interesting?
 Decision Support training effectiveness
 Professional population
 Many IT, GSS, DSS changes underway in DOD
10 December, 2006
3
GRASP - Background
Group Resource Allocation Simulation Program
 Course includes “Strategic Economics”
 Macro view of political, military, and economic
relationships
 GRASP Usage
 Capstone exercise, used in both curricula
 Decision Task




National level budget decisions
Group oriented (president & cabinet)
Face to face, synchronous, no “right” answer
Multiple objectives to optimize
10 December, 2006
4
GRASP – As a GDSS
Group Resource Allocation Simulation Program
 Contains a body of knowledge (DIKW)
 Data + Rules + Relationships + Context =
Knowledge
 Record keeping ability
 Displays data history (17 variables / 4 turns)
 Provides standardized reports to the decision
maker
 Allows decision maker choice in KM activities
 Enables decision support from “cabinet”
 Provides estimates of likely outcomes from
decisions => new knowledge
10 December, 2006
5
Relevant Literature
 Carroll & Johnson (1990)

Temporal stages of decision making
 Nunamaker, Dennis, et al. (1991)

GSS Research Framework
 Zopounidis et al. (1997)


Survey of DSS in financial management
Proposed a “Knowledge Based DSS”
 Hall & Paradice (2003)

Adapted Simon’s IDC model, incorporating KM
10 December, 2006
6
GRASP Decision Support Method





Identify Issue
Analyze Issue
Develop Alternatives
Evaluate Alternatives
Make a recommendation
 DS provides a framework for FM&C course


Addresses need for DS training in DOD FM
Congressional call for improved FM practice
10 December, 2006
7
Research Hypotheses
H1: Groups that only use FtF communication methods
will achieve greater decision quality than groups that
use hybrid proximity (FtF & paperless).
H2: Groups with higher complexity tasks (wars) will
achieve lower decision quality than groups that have
lower complexity tasks.
H3: Groups that display greater adaptation of GRASP
capabilities (intelligence gathering) will achieve
greater decision quality than groups with a lower rate
of adoption.
H4: Groups that used DS methods will achieve greater
decision quality than groups that did not.
10 December, 2006
8
Analysis Model
Communication
H1
Complexity
H2
Outcome
(DQ)
H3
Adaptation
H4
DS Method Used
10 December, 2006
Goal
Accomplishment
9
Methodology
 Subjects (~850) randomly assigned
Seminars balanced for military/civilian/job
 Each seminar was split into two teams:
 President is self selected
 President picks cabinet (4 – 6 members)
 Resulted in n=172 teams
 Competition
 A faculty facilitated exercise = lab experiment
 Four years of exercise data
 Used for debriefing, student feedback
10 December, 2006
10
Analysis
 Linear Regression Model: AR2 = .291
 Decision Quality = Communication + Complexity +
Adaptation + DS Model Use
 Operationalizing Decision Quality…
 Analysis of variance (ANOVA):
 Unequal cell sizes
 Brown-Forsythe test (constancy of error
variance)
10 December, 2006
11
Results (at α= 0.05)
 H1: Communication – Not supported
 Beta = -0.097, p = 0.228
 H2: Complexity – Supported
 Beta = -0.532, p = 0.000
 H3: Adaptation – Not supported
 Beta = 0.540, p = 0.410
 H4: DS Model – Supported
 Beta = 0.170, p = 0.034
10 December, 2006
12
Limitations & Future Research
 Observational study
 Hard to know cause & effect relationship
 Other possible explanatory factors
 More detailed analysis required
 Content analysis of user reports will allow a
better understanding of the decisions made,
priorities, & tradeoffs.
 Gather more data & use PLS
 Future research
 The decision making environment of DOD
financial managers
10 December, 2006
13
Summary
 Simulations useful, but limited:
 Allows isolation of decision behaviors (individual,
group, organization)
 Limited generalizability to other groups
 Need more research on decision support
 Role of financial managers in decision making
 Effectively decision support training methods
 Questions?
10 December, 2006
14
GRASP Exercise Model
Class Factors:
- Class Timing
- Class Size
- Class Character
- Year Attended
Group Factors:
- Group Size
- Group Character
Task Factors:
- Complexity
+Wars
+Communications
- Time Available
- Resources Available
10 December, 2006
Instruction Factors:
- Instructor
- Course type
- Decision Support Concept
- Decision Support Methodology
Decision
Support
Group
Decision
Support
Personal Factors:
- Computer Literacy
- Education Level
- Military / Civilian
- Rank
- Gender
Feedback
- Internal
- External
Decision
Maker
Decision
Quality
Score
4th year
Score
Bonus
Points
Environmental Factors:
- Competition Stress
- External Distractions
- Algorithm Anomalies
16
GSS Research Framework
(Nunamaker, Dennis et al., 1991) adapted
H1:Communication
n ~ 850
Randomly
Assigned
H4:DS Methods Used
Group
H2:Complexity
Controlled:
•Instructor
•Course
Task
Context
DSS
H3:Adaptation
Intervening
&
Adaptation
Process
Outcome
DV:
Decision Group
Environment
Decision Quality
Feedback
•Phase
GRASP
10 December, 2006
Organizational
Decision-Making
Environment
Next Phase / PLS
17