Transcript Slide 1

Measuring the immeasurable?
The Effects-Based Approach in
Comprehensive Peace Operations
04 juni, 2013
Bas Rietjens
Current effect measurement methodologies
1.
Many different systems to measure effects, often
disconnected, using different methodologies and
terminology
2.
Examples include:
- World Bank’s Low-Income Countries Under Stress
(LICUS) initiative
- International study on effective transitions of DFI
International
- Measuring Progress in Conflic Environments (MPICE)
of USAID and others
3.
Within military effect measurement is included in Effectsbased approach to operations (EBAO)
Effects-based approach to operations:
A brief history
•
Thinking in effects dates back to Sun Tzu and Von
Clausewitz
•
Flourishing in the form of “target value analysis” (prior
to operations)
•
Traditional methods of establishing effects include
“battle damage assessments” and “after action
reviews”
•
The effects of current military operations are broader
which led to EBAO
Effects-based approach to operations:
Definition & scope
EBAO: A coherent and comprehensive application of the various
instruments of the Allied Command and cooperation with all
involved non-NATO actors, in order to create the effects’.
(NATO definition)
Effect: intended change in the situation in the area of operations
Domain: any conceivable domain, including physical, economic,
managerial and psychological (e.g., the sense of security
among the population).
May result both from purely military and not so purely military
activities, such as civil-military cooperation, psychological
operations or economic development.
Effects-based approach to operations:
Characteristics
•
Harmonizing and synchronizing military actions with the
other instruments of power
•
Appreciating strategic and operational effects – outcomes
seperated in space and time from their causative actions –
whether they are desired or undesired
•
Assessing system behaviors and capabilities: an emphasis
on effects attainment rather than task accomplishment
•
Collaborating more extensively with superiors,
subordinates, the interagency community and multinational
partners
MoP versus MoE
???
MoP’s and MoE’s
Measures of Performance
The MOP enables a more rigorous execution analysis, intending to answer
“Are the actions being executed as planned?”
Measures of Effectiveness
The MOE will help answer the question “Was the intended new system
state—the desired effect—created?” This may require multiple MOEs
per Effect to fully capture the changes in system states.
NATO Engagement Space Handbook,
HQ SACT 10 august 2007. (Version 1.0)
EBAO under discussion
Mattis states that EBAO:
•
“assumes a level of unachievable predictability
•
cannot correctly anticipate reactions of complex systems
•
discounts the human dimension
•
fails to deliver clear and timely direction to subordinates
•
and uses confusing terminology”.
Mattis concludes as follows:
•
“Effective immediately, USJFCOM will no longer use,
sponsor, or export the terms and concepts related to” EBAO
“in our training, doctrine development, and support of joint
professional military education”.
Ruby acknowledges that:
•
EBAO is no panacea performing miracles.
•
EBAO can be misused by commanders and planners to
carry out wrong actions.
•
the concept of thinking in terms of effects has various
users, each of whom has his own interpretations and
convictions.
Otherwise, he condemns the reasoning and selective
examples of Mattis:
•
Replacement of thinking in terms of effects by the
commander’s intuition a huge step backwards.
•
It is much easier to define what activities the military
organization employs (e.g. the amount of dollars spent,
of weapons collected).
EBAO and Result Based Management (1)
• Armed forces are an executive instrument of the national
government  national audit bureaus (rekenkamer) concern
themselves with the military and comment on its spending.
• For that same reason the military has had to deal with a shift
from input and asset-oriented thinking to process- or activityoriented thinking over the last 20 years, and from there to
thinking in terms of results, in management science referred
to as New Public Management (NPM).
• In plain terms, NPM endeavors to introduce a more business
oriented approach and more value-for-money thinking in
public governance.
EBAO and Result Based Management (2)
• One of the central tenants of NPM is an increased focus on
public-sector performance, often termed RBM.
• The intentions of RBM are twofold:
- internally, to direct and motivate people and to enable them
to learn from their own experiences;
- externally, to be accountable to the various parties
concerned. “measuring is knowing”.
EBAO and Result Based Management (3)
Phase
Results-Based Management
Effects-Based Approach to Operations
1
Formulate objectives (results)
Define Effects, Objectives and End States
2
Select indicators to measure progress towards
each objective
3
Set explicit targets for each indicator to judge
performance
Regularly collect data on results to monitor
performance
Review, analyze, and report actual results vis-àvis the targets
Define Measures of Effectiveness (MoE) for
each Effect, Objective, and
Define Measures of Performance (MoP) for each
Action.
Establish thresholds for each MoE and MoP
4
5
6
Integrate evaluations to provide complementary
performance information
7
Use performance information for accountability,
learning, and decision making.
Regularly collect data for MoE and MoP
Conduct assessment of Effect and Objective
status; conduct assessment of Action status to
determine progress toward the target ( )
Conduct assessment of Effect versus Action
status; Objective versus Effect status to find
insight as to why results were or were not
achieved.
Use the results of this assessment to inform the
Commander and staff; identify required changes.