Measuring the immeasurable?The Effects-Based Approach in

Comprehensive Peace Operations

04 juni, 2013

Bas Rietjens

Current effect measurement methodologies

1. Many different systems to measure effects, often disconnected, using different methodologies and terminology

2. Examples include: - World Bank’s Low-Income Countries Under Stress (LICUS) initiative- International study on effective transitions of DFI International- Measuring Progress in Conflic Environments (MPICE) of USAID and others

3. Within military effect measurement is included in Effects-based approach to operations (EBAO)

Effects-based approach to operations: A brief history

• Thinking in effects dates back to Sun Tzu and Von Clausewitz

• Flourishing in the form of “target value analysis” (prior to operations)

• Traditional methods of establishing effects include “battle damage assessments” and “after action reviews”

• The effects of current military operations are broader which led to EBAO

Effects-based approach to operations: Definition & scope

EBAO: A coherent and comprehensive application of the various instruments of the Allied Command and cooperation with all involved non-NATO actors, in order to create the effects’. (NATO definition)

Effect: intended change in the situation in the area of operations

Domain: any conceivable domain, including physical, economic, managerial and psychological (e.g., the sense of security among the population).

May result both from purely military and not so purely military activities, such as civil-military cooperation, psychological operations or economic development.

Effects-based approach to operations: Characteristics

• Harmonizing and synchronizing military actions with the other instruments of power

• Appreciating strategic and operational effects – outcomes seperated in space and time from their causative actions – whether they are desired or undesired

• Assessing system behaviors and capabilities: an emphasis on effects attainment rather than task accomplishment

• Collaborating more extensively with superiors, subordinates, the interagency community and multinational partners

MoP versus MoE

???

MoP’s and MoE’s

Measures of EffectivenessThe MOE will help answer the question “Was the intended new system state—the desired effect—created?” This may require multiple MOEs

per Effect to fully capture the changes in system states.

Measures of PerformanceThe MOP enables a more rigorous execution analysis, intending to answer

“Are the actions being executed as planned?”

NATO Engagement Space Handbook,

HQ SACT 10 august 2007. (Version 1.0)

EBAO under discussion

Mattis states that EBAO: • “assumes a level of unachievable predictability • cannot correctly anticipate reactions of complex systems • discounts the human dimension • fails to deliver clear and timely direction to subordinates • and uses confusing terminology”.

Mattis concludes as follows: • “Effective immediately, USJFCOM will no longer use,

sponsor, or export the terms and concepts related to” EBAO “in our training, doctrine development, and support of joint professional military education”.

Ruby acknowledges that:• EBAO is no panacea performing miracles. • EBAO can be misused by commanders and planners to carry out

wrong actions. • the concept of thinking in terms of effects has various users,

each of whom has his own interpretations and convictions.

Otherwise, he condemns the reasoning and selective examples of Mattis:

• Replacement of thinking in terms of effects by the commander’s intuition a huge step backwards.

• It is much easier to define what activities the military organization employs (e.g. the amount of dollars spent, of weapons collected).

EBAO and Result Based Management (1)

• Armed forces are an executive instrument of the national government national audit bureaus (rekenkamer) concern themselves with the military and comment on its spending.

• For that same reason the military has had to deal with a shift from input and asset-oriented thinking to process- or activity-oriented thinking over the last 20 years, and from there to thinking in terms of results, in management science referred to as New Public Management (NPM).

• In plain terms, NPM endeavors to introduce a more business oriented approach and more value-for-money thinking in public governance.

EBAO and Result Based Management (2)

• One of the central tenants of NPM is an increased focus on public-sector performance, often termed RBM.

• The intentions of RBM are twofold: - internally, to direct and motivate people and to enable them to learn from their own experiences;- externally, to be accountable to the various parties concerned. “measuring is knowing”.

EBAO and Result Based Management (3)

Phase Results-Based Management Effects-Based Approach to Operations

1 Formulate objectives (results) Define Effects, Objectives and End States

2 Select indicators to measure progress towards each objective

Define Measures of Effectiveness (MoE) for each Effect, Objective, andDefine Measures of Performance (MoP) for each Action.

3 Set explicit targets for each indicator to judge performance

Establish thresholds for each MoE and MoP

4 Regularly collect data on results to monitor performance

Regularly collect data for MoE and MoP

5 Review, analyze, and report actual results vis-à-vis the targets

Conduct assessment of Effect and Objective status; conduct assessment of Action status to determine progress toward the target ( )

6 Integrate evaluations to provide complementary performance information

Conduct assessment of Effect versus Action status; Objective versus Effect status to find insight as to why results were or were not achieved.

7 Use performance information for accountability, learning, and decision making.

Use the results of this assessment to inform the Commander and staff; identify required changes.