KRIs – a failed experiment or a misunderstood tool?

Scottish Chapter Annual Conference 1 November 2013

Disclaimer

• • Your speaker is a Fellow of the Institute of Directors, a Director of Council and Vice-Chairperson of the IOR, as well as the Chief Executive of RiskBusiness International Limited.

The views expressed in this presentation are the sole responsibility of the presenter and do not and can not be construed as representing the view of either the Institute of Operational Risk or RiskBusiness International Limited.

What is a KRI?

• Basically, whatever you want it to be!

– It’s a metric, a piece of information – It only has value to those who can use it – There is no difference between KRIs, KPIs and KCIs – depends on source and use of the information

Why is Op Risk “Hard” to do?

• It’s fundamentally different to other types of risk: – No direct correlation to volumes, market volatility, economic cycles or other easily quantifiable factors – Its embedded into each of the other types of risk – how and when do we separate them?

– It has a direct link to the “human factor” – The business intuitively accept it as part of “business as usual” and have difficulty in understanding the “regulatory” rationale behind elevating it to a distinct risk class – The issues of “face”, deniability, ignorance, the “Titanic” effect and historical audit processes – Multiple independent programs

Why are KRIs left to last?

• • • How do we correlate them to…… – Risk?

– Exposure?

– Loss Data?

– Capital Requirements?

– Senior Management Requirements and General Business Management Needs?

Where do we start?

Can they ever be predictive? If not, why bother?

Predictive metrics

• • • By themselves, individual indicators will never be truly predictive Avoid thinking that trends are predictive KRIs can be used to “predict” events before they manifest themselves –

if

: – There is a common structure against which risks and losses are classified and to which indicators are linked – Indicators are monitored on a real-time basis by business units carrying the exposure – Management reacts to stimuli as and when warning patterns emerge – We have had sufficient history to test and prove the correlations

The role of KRIs in time

History Internal Event Data External Loss Data

OpRisk Framework: Time Perspective

Short-term Future Now KRIs RCSA Risk Profiling Scenario Analysis Medium- to Long-term Future

Who owns KRIs?

• • • • • Who decides which KRIs to monitor?

Who decides how the KRI values are calculated?

Who decides who receives KRI reports and when they receive them?

Who decides when to escalate a KRI submission?

Who decides to discontinue using a specific KRI?

Providers

Roles in the KRI Process

Producers Risk Management Consumers Observers

Operational Risk as a “Bow Tie”

Org’l Hierarchy

Glob Serv Inv Mgmt Sec Lend

Processes

Market and Sell Products and Services Trading and/or Investment Management Trade Settlement

Events Causes Impacts Products

Card - Retail FX Swap Lockbox

The Role of KRIs

Internal causes

Org’l Hierarchy

Glob Serv Inv Mgmt Sec Lend Defined Business Process Defined Business Process Preventive Controls EVENT

Processes

Market and Sell Products and Services Trading and/or Investment Management Trade Settlement

Products

Card - Retail FX Swap Lockbox External causes Risk Category Detective Controls Oversight Controls

KRIs

Expected Result

However....

• The Bow-Tie Model suggests that causal factors drive everything else, therefore the normal business model needs to in corporate cause as a major component

Clerk A S’visor A

Causal chains

Manager Clerk A Clerk A Clerk B Market volatility Trade Record Short staffed Complex trade System failure Review and check details Staff personal problems Staff € problems Lack of training Correct?

No Return to F/O S’visor B Clerk B Yes M’ment offsite Annual holidays Clerk B Send to Confirm

Evolution of risk classification

Org’l Hierarchy

Glob Serv Inv Mgmt Sec Lend

Business Process

Preventive Controls Internal causes

Processes

Market and Sell Products and Services Trading and/or Investment Management Trade Settlement

Products

Card - Retail FX Swap Lockbox Process Risks Preventive Controls Process Risk Causes

Risk

External causes Detective Controls Detective Controls Risk Category

Data Management

• • • Quality management is one of the most critical aspects of the KRI programme: – Use standard specifications covering what, when and how data will be measured, along with who and why – Investigate discrepancies and challenges – Utilise other LoD to assist in QM Collect data at the earliest opportunity, disseminate it as quickly as you can Consumers often already have the data, add value by augmenting the data with context – #/$ of suspense account items – Broken down by time-buckets – Analysed by party/system/entity

Top-Down v Bottom-Up

• • • The concept is to obtain reports on the same metrics from across the firm Can we identify any metrics which can actually be compared across the entire firm?

Are these metrics in any way correlated to risk exposure?

Derived indicators

• • • “Manufactured” metrics whose value is derived from a series of underlying metrics Examples of underlying metrics: – Staff availability – Average error count – Average cost per error – Transactional volumes – Economic cycles – Cycle volatility The resultant metric seeks to measure the impact of staff taking leave at specific points in time, assessing the increased potential cost

Thresholds

• • • Thresholds are essentially pre-defined limits which, when the value of a KRI reaches that level, generates warnings or alerts Different types: – Touch – Repetitive touch – Percentile breach – Trend threshold Thresholds should never be absolute; they need to address cycles and correlations.

Threshold issues

• • • • Never start with a “big bang” – management will be confused if they suddenly start receiving numerous “alerts” and “warnings” Start with high-level thresholds and fine tune them over time – what is the correct level for any metric?

Use layered structures so that increasing levels of seniority receive warnings when appropriate – and the “worker bees” do not get a surprise Revise thresholds from time to time

In summary……

• • • • • • There is no such thing as a global set of “top 10” indicators which everyone should monitor Excluding composite or index-based metrics, indicators are not in of themselves predictive A good indicator programme will involve a large number of players Your risk profile is constantly changing, causal drivers continuously morph into different impact chains – the indicator set being monitored should thus not be set in stone Don’t expect instant gratification – the benefits of the indicator programme will take time to manifest themselves

Read the IOR SPG on KRIs issued November 2010!

Questions and Comments

• •

Contact Details:

Mike Finlay, Chief Executive, RiskBusiness – Telephone : +44 7721 969 224 – E-mail : [email protected]

or – [email protected]

URL : www.riskbusiness.com

and www.ior-institute.org