Transcript No Slide Title

Interactive Computer Support in
Decision Conferencing
Two Cases on Off-Site Nuclear Emergency
Management
Jyri Mustajoki
Raimo P. Hämäläinen
Kari Sinkko
Systems Analysis Laboratory
Helsinki University of Technology
www.sal.hut.fi
S ystems
Analysis Laboratory
Helsinki University of Technology
1
New Methods for Group Problem Solving
• Portable web-based group decision support
system
• Interactive multiattribute modeling
• Hands-on use of the system
• Group models
• Multiattribute modeling in radiation emergency
management is a new approach
• Health risk is not the only issue affecting policy
decisions
S ystems
Analysis Laboratory
Helsinki University of Technology
2
Cases: Nuclear Emergency
• Simulated nuclear accident
• Milk case: Planning of countermeasures for the
milk pathway in a nuclear accident
• Urban case: Planning of clean-up actions in
inhabitated areas
• Part of the EU’s EVATECH project: similar
workshops arranged in seven European countries
• A day-long decision conference exercise held to
consider the problem from different perspectives
S ystems
Analysis Laboratory
Helsinki University of Technology
3
Portable Group Decision Support System
Server
Projector
Internet
Stakeholders
S ystems
Analysis Laboratory
Helsinki University of Technology
4
Software used
• Web-HIPRE (www.hipre.hut.fi)
• Multiattribute value tree approach
• Individual models aggregated into a group model
• Opinions-Online (www.opinions.hut.fi)
• Approval voting on the alternatives
• WINPRE (www.sal.hut.fi/ Downloadables)
• Tested in the Milk case
• Global sensitivity analysis with intervals
• All available at www.decisionarium.hut.fi
S ystems
Analysis Laboratory
Helsinki University of Technology
5
Participants
• All stakeholders from the institutions and
organizations that would be involved in the
decision making process also in a real
accident
• Used the systems by themselves
• Most of them had no prior experience about
decision analysis or DSS software
 Strong requirements to the usability of the
methods and the software
S ystems
Analysis Laboratory
Helsinki University of Technology
6
Progress of the Conferences
1. Introduction to the decision conference and
the case
2. Identification of the problem and structuring
of the model
3. Evaluation of the model
• Hands-on use of the supporting software
4. Analysis of the results and conclusions
All in one day
 High intensity
S ystems
Analysis Laboratory
Helsinki University of Technology
7
Support Stuff
• Independent facilitator
• Experts in radiation
• Three technical assistants for helping the use
of the software
• Decision analyst
S ystems
Analysis Laboratory
Helsinki University of Technology
8
Structuring the Model
• First set of attributes in preparatory meetings
representing early administrative preparation
• Experiences of the earlier decision
conferences used
• Attributes discussed and refined in the
conferences
• Value tree created simultaneously with an
analyst operating with the software
S ystems
Analysis Laboratory
Helsinki University of Technology
9
Generation of Decision Alternatives
• Feasible countermeasures in preparatory
meetings:
• Milk case, e.g.
• provision of uncontaminated fodder
• banning of the milk
• Urban case, e.g.
• continuation of evacuation
• sweeping streets
• Actual strategy alternatives combinations of
these countermeasures
S ystems
Analysis Laboratory
Helsinki University of Technology
10
Value Tree (Milk Case)
S ystems
Analysis Laboratory
Helsinki University of Technology
11
Evaluation of the Model
• SWING weighting
• Six groups of two to three people
• Each group evaluated independently the
jointly structured model
• As a result, six instances of the same value
tree
• Reflect the preferences of each group
S ystems
Analysis Laboratory
Helsinki University of Technology
12
Analysis of the Results
• Individual results
shown for discussion
• Understanding of
the other
participants'
preferences
• Sensitivity analyses
S ystems
Analysis Laboratory
Helsinki University of Technology
13
Overall Group Results
• Individual results aggregated with the
Weighted Arithmetic Mean Method
S ystems
Analysis Laboratory
Helsinki University of Technology
14
Global Sensitivity Analysis by Intervals
Two analyses:
• Uncertainties given
by participants with
interval SMART/
SWING
• Uncertainties
estimated from the
Opinions-Online
survey (with PAIRS)
S ystems
Analysis Laboratory
Helsinki University of Technology
15
Final Strategy Recommendations
• Milk Case: Approval voting of the alternatives
with Opinions-Online
• Two alternatives approved by all but one
participants
• Either of these could be recommended as a final
strategy
• Urban Case: One clearly the best alternative
• The participants were able to still improve this
with some minor modifications on actions
S ystems
Analysis Laboratory
Helsinki University of Technology
16
Experiences from the Conferences
• Very positive feedback
• One-day time frame applicable
• Careful planning in advance needed
• Not much room for improvisation
• Experiences emphasize the use of simple
models
• A comprehensive view of the problem can still
be provided
S ystems
Analysis Laboratory
Helsinki University of Technology
17
Feedback from the Participants
Milk Case
–– –
?
Urban Case
+ + +– – –
?
+ ++
4 8
The decision conferencing approach is suitable for providing
a comprehensive view of the situation in training and
exercises.
1 1 1 10 -
-
-
-
The decision conferencing approach is suitable for providing
a comprehensive view in the case of a real emergency.
2 3 1 7
-
-
-
1 7 5
The decision conferencing approach is suitable for finding a
strategy in training and exercises.
-
2
8 3
-
-
-
The decision conferencing approach is suitable for finding a
strategy in the case of a real emergency.
-
4 1 7 1
-
-
4 5 4
The ranking achieved with Web-HIPRE corresponds to my
intuitive expectations.
-
-
1 12 -
-
1 1 8 3
It was easy to grasp and follow the method used in WebHIPRE to elicit the trade-offs between attributes.
-
1 1 10 1
-
2
It was easy to grasp and follow the method used in WINPRE
to elicit the trade-offs between attributes.
1 2
-
The decision conference exercise was useful in general.
-
- 12 1
-
-
9 1
-
4 9
7 4
WINPRE not
used
-
-
-
1 12
‘– –‘ = strongly disagree, ‘–‘ = disagree, ‘?’ = no opinion, ‘+’ = agree, ‘+ +’ = strongly agree
S ystems
Analysis Laboratory
Helsinki University of Technology
18
Feedback Much Better in the Urban Case
Major differences between the conferences that
are likely to have affected the feedback:
• Three different software used in the Milk case
• Much time spent for the explaining the
software
• Framing of the alternatives
• Urban case took a broader view to protect the
whole inhabited environment
• In the Urban case most participants were
local inhabitants  Increased commitment
S ystems
Analysis Laboratory
Helsinki University of Technology
19
Suitability of the Approach in Nuclear
Emergency Management
• The approach seen more suitable in planning
of protective actions in advance and in
excercises than in real cases
• The majority of participants saw the approach
applicable also in real cases
• Some issues caused conversation
• Including alternative do nothing
• Difference between attributes reassurance
and anxiety
S ystems
Analysis Laboratory
Helsinki University of Technology
20
Independent Use of the Software
• In general, the participants were able to
independently use the systems
• Some misunderstandings in the use of the
interval method in WINPRE
• More time would be needed to describe the
use of the software
 The use of the software should be kept
simple
S ystems
Analysis Laboratory
Helsinki University of Technology
21
Arranging the Conference on the Web?
• Technically possible
• Impacts and other information available on the
web
• Web-based multiattribute software available
• Role of the facilitator?
• Behavioral issues need to be kept in mind
• How to assure the proper use of the MCDA
methods?
S ystems
Analysis Laboratory
Helsinki University of Technology
22
Conclusions
• The participants
• considered the conferences useful
• reached a consensus on countermeasure policies
• were able to use the systems by themselves
 Strong support to the MCDA based decision
conference with interactive software
• With the ’right’ participation and intensive
workshop much better choices can be found
• Approach applicable also in other environmental
problems
S ystems
Analysis Laboratory
Helsinki University of Technology
23
Conclusions
Main prerequisites:
• Simple models and software
• Transparency and accountability of the
process
• All the key players’ values incorporated into
decisions
• The participants have a feel of control
• Competent neutral facilitator and analyst
S ystems
Analysis Laboratory
Helsinki University of Technology
24
References
Ammann, M., Sinkko, K., Kostiainen, E., Salo, A., Liskola, K., Hämäläinen, R.P.,
Mustajoki J. (2001): Decision analysis of countermeasures for the milk pathway
after an accidental release of radionuclides, Radiation and Nuclear Safety Authority,
STUK-A186, (Downloadable at www.sal.hut.fi/Publications/pdf-files/ramm01.pdf).
Hämäläinen, R.P. (2003), Decisionarium - Aiding Decisions, Negotiating and Collecting
Opinions on the Web, Journal of Multi-Criteria Decision Analysis, 12(2-3), 101-110.
Hämäläinen, R.P., Kettunen, E., Marttunen, M., Ehtamo, H. (2001), Evaluating a
framework for multi-stakeholder decision support in water resources management,
Group Decision and Negotiation, 10, 331-353.
Hämäläinen, R.P., Leikola, O. (1996), Spontaneous decision conferencing with top-level
politicians, OR Insight, 9(1), 24-28.
Hämäläinen, R.P., Lindstedt, M., Sinkko, K. (2000a), Multiattribute Risk Analysis in
Nuclear Emergency Management, Risk Analysis, 20(4), 455-467.
Hämäläinen, R.P., Sinkko, K., Lindstedt, M., Ammann, M., Salo, A. (1998), RODOS and
decision conferencing on early phase protective actions in Finland, Radiation and
Nuclear Safety Authority, STUK-A159. (Downloadable at
www.sal.hut.fi/Publications/pdf-files/rham98.pdf)
S ystems
Analysis Laboratory
Helsinki University of Technology
25
References
Hämäläinen, R.P., Sinkko, K., Lindstedt, M., Ammann, M., Salo, A. (2000b), Decision
analysis interviews on protective actions in Finland supported by the RODOS
system, Radiation and Nuclear Safety Authority, STUK-A173. (Downloadable at
www.sal.hut.fi/Publications/pdf-files/rham00.pdf)
Mustajoki, J., Hämäläinen, R.P. (2000), Web-HIPRE: Global Decision Support by Value
Tree and AHP Analysis, INFOR, 38(3), 208-220.
Mustajoki, J., Hämäläinen, R.P., Lindstedt, M.R.K. (2005): Using intervals for Global
Sensitivity and Worst Case Analyses in Multiattribute Value Trees, European
Journal of Operational Research. (to appear)
Mustajoki, J., Hämäläinen, R.P., Sinkko, K. (2005), Interactive Computer Support in
Decision Conferencing: Two Cases on Off-site Nuclear Emergency Management,
Manuscript. (Downloadable at www.sal.hut.fi/Publications/pdf-files/mmus05.pdf)
Salo, A., Hämäläinen, R.P. (1992), Preference assessment by imprecise ratio
statements, Operations Research, 40(6), 1053-1061.
S ystems
Analysis Laboratory
Helsinki University of Technology
26