Best practice in data & scoring

Dr Paul Russell Director Analytical Solutions © Experian Limited 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited. Other product and company names mentioned herein may be the trademarks of their respective owners. No part of this copyrighted work may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian Limited.

Confidential and proprietary.

Agenda

 Some themes  Analytics and the customer life cycle  The role of scoring  Building a scorecard  Using scoring systems  Risk management infrastructure © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

2

Themes

 Best practice is often discussed but almost never seen  Do the simple things well  Risk management is more than just a scorecard  The same principles apply across the credit lifecycle © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

3

13 ways to grow bad debt

Credit process step Target population

Customer acquisition Customer management Collections New customers Existing, non delinquent customers Existing, delinquent customers

Description

1. Identifying potential customers; 2. Selling credit products to new customers; 3. Identifying the credit risk of the customer and the proposed transaction; 4. Identifying the risk of fraudulent application 5. Deciding whether to accept or decline the transaction; 6. Deciding, for accepted transactions, on the terms, e.g., credit amount, pricing.

7. Reviewing the customers facilities (e.g., credit limits, price, etc.); 8. Cross-selling new products to the customers; 9. Ensuring good customers are retained; 10. Identify fraudulent transactions.

11. Identifying self-cure customers; 12. Rehabilitation of potentially good customers; 13. Work-out customers where relationship is broken.

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

4

Why is credit risk management important?

European consumer finance business, Profit Before Tax and Impairment Charges ($m)

Get it right and it can support phenomenal value creation 2,196 676 288 802

Profits

836 478 1,094 740 1,230 804 1,382 924 1,520 1,522 1,374 340 764 2,986

Impairment charges

1998 1999 2000 2001 2002 2003 © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

2004 2005 2006

Source: Annual Reports

5

5 core components

Component Data Statistical Models Description

Application data (for new customers) Account behaviour data (for existing customers) External data (e.g., credit bureaux) Risk models (PD, LGD), fraud models (application and transaction fraud) and revenue models

Credit strategies Implementation tools

Business rules that translate the outcome of statistical models in credit decisions (accept/decline, price, credit limits, etc.) that maximise profit Software tools to automate the calculation of the above scores and credit strategies on-line on high volumes, with a high degree of flexibility to change credit strategies “on the fly”

Evaluation tools

Software tools to evaluate the performance of statistical models and credit strategies, and accuracy of implementation © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

6

Agenda

 Some basic themes  Analytics and the customer life cycle  The role of scoring  Building a scorecard  Using scoring systems  Risk management infrastructure © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

7

Analytics and the customer life cycle

Solicitation Application Customer management Collections Debt recovery

Population Information

Analytics touches every part of the customer lifecycle

  Analytics touches every part of the customer life cycle Amount of information about the customer grows as the relationship advances through the customer life cycle © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

8

Analytics and the customer life cycle

Solicitation • Channel preference • Contact history • Demographics • Location • Bureau data • Action outcomes • Costs Application • Channel • Product holdings • Demographics • Bureau data • Previous relationships • Account performance • Costs Customer management • Product holdings • Usage • Delinquency • Customer contacts • Preferences • Bureau data • Actions taken • Action outcomes • Costs Collections • Action history • Promises to pay • Promises fulfilled • Action outcomes • Bureau data • Costs Debt recovery • Action history • Promises to pay • Bureau data • Agents used • Promises fulfilled • Litigation outcomes • Costs © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

9

Analytics and the customer life cycle

Define Goals Agree objectives Understand results Track progress against expectations Plan Assess Monitor Strategy Review Review Design Implement Ensure operational deployment Assess current challenger Build new strategy

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

10

Agenda

 Some basic themes  Analytics and the customer life cycle  The role of scoring  Building a scorecard  Using scoring systems  Risk management infrastructure © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

11

The role of scoring

   Credit scoring is a technique for predicting the future This prediction can be anything of importance to the business        Arrears Fraud Profit Response Account closure Company failure Etc.

All scoring is based on one key assumption:  The past predicts the future © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

12

The role of scoring

How does scoring work?

Example Scorecard

• • • •

Baseline Constant

Scorecards add and subtract points to a baseline constant according to individual’s or account’s data

Applicant Age in Years

< 22 22 - 25 26 – 40 41 – 55 > 55 Scorecards are easy to apply and simple to understand The resulting

score

gives a prediction of future behaviour

Worst Status L6M (on all Accounts)

0 1 - 2 3+

Joint Applicant Present

Y Scores are used to actions

rank

individuals to assign the best N

Etc.

… © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

800

-50 -20 0 +30 0 0 -45 -100 +20 0

Etc.

… 13

The role of scoring – application scorecard

• • Consider a scorecard built to predict whether a new applicant for a credit product will default in the next 12 months This scorecard is used when a new customer applies…

Application Form Data Score card Score External Data (Bureau etc.)

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

Take most appropriate action for each individual

14

The role of scoring - scores can drive actions

Application Score Low Score / High Risk Extremely High Risk Reject High Risk Reject or price to cover the high expected loss Standard Risk Accept on standard terms

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

High Score / Low Risk Extremely Low Risk Consider for cross sell of other products

15

The role of scoring - benefits

 Best use of data  Objective  Consistent  Automation  Control  Reduced losses © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

16

Agenda

 Some basic themes  Analytics and the customer life cycle  The role of scoring  Building a scorecard  Using scoring systems  Risk management infrastructure © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

17

Building a scorecard – 3 requirements



Development sample

– the historical data on which the scorecard will be built 

Outcome

– what we are trying to predict 

Modelling methodology

– the statistical tool that will help us form our scoring model

The recent past Development Sample Some time later Statistical Model

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

Outcome

Score card 18

Is my sample any good?

 Representative  Products   Business cycle The future  Robust  Volumes  Mature Is the outcome reliable?

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

The recent past Development Sample

19

Building a scorecard – the development sample

• • This data can come from a number of sources All relevant data should go into the development sample

The recent past Development Sample Application Form Credit Bureau Data Historical Account Behaviour Other Account Information Information collected from the applicant at the application point Information on the individual’s other credit commitments Information on the historical behaviour on the account Information on the historical behaviour on other accounts with the same lender

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

20

Building a scorecard – the outcome

• • This is the behaviour that we are trying to predict Can be a continuous variable (profit, revenue, loss given default, etc.) More commonly it is dichotomous - yes/no     Will this applicant default?

Is this transaction fraudulent?

Will this company fail?

Etc.

Outcome

Observation - Now THE FUTURE Good Bad Outcome - Prediction © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

21

What are we trying to predict?

Consumer Limited business Non-limited business Bad Good 3 payments in arrears Failed Not 3 payments in arrears Still going Bankruptcy, court judgements or defaults No bankruptcy, court judgements or defaults

Outcome

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

22

Building a scorecard – the statistical model

  Many statistical tools available Data is the most important factor

Statistical Model

Observation Data Outcome

Statistical Model

Score card   Statistical tool needs to be:  Powerful – to get the best prediction from the data  Flexible – can handle varying data types and outcomes Interpretable Transparent – easy to understand and to overlay business intelligence – should be non-’black box’ for regulatory reasons and to ensure understanding © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

23

Building a scorecard - the statistical model

Statistical Model

 Linear regression  Logistic regression  Artificial neural networks  Etc  Other things being equal the choice of algorithm has relatively little impact on the ultimate power of the model © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

x x x x x x x x x x x x x x x x x x x x x

Prediction

24

Building a scorecard – assessing the model

Does the model solve the business problem?

Statistical Model

  Discrimination bad – the power to polarise individuals between good and

Gini statistic & Kolmogorov-Smirnov statistic

Accuracy the model – how much of the variability of the outcome is explained by   Validation – ensures that over-modelling has not occurred or that an anomalous sample has not been used Improvement – the new model should outperforms the existing model © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

25

Agenda

 Some basic themes  Analytics and the customer life cycle  The role of scoring  Building a scorecard  Using scoring systems  Risk management infrastructure © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

26

Using scoring systems

SCORE CARD DECISIONS STRATEGY DATA    The data feeds the scoring system, which is used to aid the decisioning The decisions a company makes determine its strategy It is the aims and strategy of the business that must be considered when deciding how to use a scoring system, e.g.

 Growing the market share  Reducing bad debt  Increasing automation  Maximising response for given marketing cost  Combating fraud © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

27

The role of scoring - scores can drive actions

Application Score Low Score / High Risk Extremely High Risk Reject High Risk Reject or price to cover the high expected loss Standard Risk Accept on standard terms

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

High Score / Low Risk Extremely Low Risk Consider for cross sell of other products

28

Using scoring systems - the score distribution

Score Band ≤ 400 401 – 550 551 – 650 651 – 700 701 – 750 751 - 800 801 – 850 851 – 900 901 – 950 ≥ 951 TOTAL # Goods 500 700 815 1008 976 950 1000 1050 960 1000 8959 # Bads 500 350 163 84 61 38 25 21 16 10 1268 GB Odds 1 2 5 12 16 25 40 50 60 100 7.1

% Applicants 9.8

10.3

9.6

10.7

10.1

9.7

10.0

10.5

9.5

9.9

100

• • Score distribution is obtained by applying the score to the development sample Gives us a prediction for new applicants falling into a given score range © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

29

Building a scorecard - the score distribution

Score Band ≤ 400 401 – 550 551 – 650 651 – 700 701 – 750 751 - 800 801 – 850 851 – 900 901 – 950 ≥ 951 TOTAL # Goods 500 700 815 1008 976 950 1000 1050 960 1000 8959 # Bads 500 350 163 84 61 38 25 21 16 10 1268 GB Odds 1 2 5 12 16 25 40 50 60 100 7.1

Score + Policy Rules + Terms of Business = Strategy % Applicants 9.8

10.3

9.6

10.7

10.1

9.7

10.0

10.5

9.5

9.9

100 REJECT REFER ACCEPT ACCEPT WITH X SELL

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

30

Agenda

 Some basic themes  Analytics and the customer life cycle  The role of scoring  Building a scorecard  Using scoring systems  Risk management infrastructure © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

31

Implementation – the Business Rules Engine

Rules Definition

(Strategy Design Studio)

Data Rules execution

(Decision Agent)

Results

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

Deployed in:  Origination  Application processing  Portfolio Management  Customer level decisioning  Collections  Authorisations  Intelligent Messaging  Event Management  Basel II Stress testing  …..

32

The unsecured lending origination process

Gather & validate application data A full range of client options and interfaces for channel independence and data accuracy Gather existing customer information Online links to gather data about existing relationships and customer behaviour Detect application fraud Get policy decision & enrichment strategy Application screening and data matching Business-driven scoring and decision-making Invoke enrichment strategy Credit bureau links Get decision & terms of business Business-driven scoring & decision making Handle referrals and manual procedures Comprehensive workflow capabilities and provision of relevant data for users Implement final decision Automated account set-up. Provision of hand-off files. Letter and e-mail production © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

33

Implements Business logic, Segmentation, Scorecards, Strategies and Champion Challenger Defines Business logic, Segmentation, Scorecards, Strategies and Champion Challenger

H O S T

Active History

Extract

e.g. Account Management System, Authorisation System etc

Feedback

Variables Decision Engine Strategy Implementation

Data Manager © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

Operational environment

Results

Analytical Data Mart Evaluation Optimisation Reporting

Rule Definition Strategic business environment 34

Beyond scoring - strategy optimisation

• • • There are disadvantages to traditional champion/challenger testing… The time frame for observing results can be long It can be hard to design the next step The result can become a “semi-random walk”...

Value

We want to get there with the first challenger !

The challenger strategy proven in one time period, may no longer be appropriate for another time period – things change Champion Decision strategy “deploy-learn deploy” process Challenger 1 Challenger 2 Challenger n Using performance data enables better decisions, but is also more complex to combine all the decision influences to maximise value Challenger 3 Challenger 4

Time

Influence due to: • Macro-economics?

• Use of intuition?

• Misunderstanding?

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

35 #35

Developments in analytics - strategy optimisation

Incremental benefit ROI Some organisations are still here The next step… Most are here

Elaborate Strategies Manual • • X X X X X X X X X X X X X X X X X X X X X X Experience and intuition Trial and error Scoring X X X X X X X X X X X X X X X X • • single predictive model e.g. credit risk score “Heuristic” cut-offs assigned using good:bad odds • • Segmentation based on predictive model dimensions: e.g. risk and revenue “Subjective” judgment used to manage trade-offs Optimised Strategies • • Allocates optimal action for each customer within constraints Objective, mathematical goal maximisation

Underlying decision complexity

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

36 #36

Stage 1: build the infrastructure



Centralisation of credit decisioning



Set-up of IT tools required to automate credit risk and market management processes and the interaction between front line and back office



Development of decision support tools



Development of credit / marketing databases

Automate the processes

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

37

Stage 2: fine-tune for performance

 Fine customer segmentation based on customer profile, product holding and behavior data  Advanced credit and marketing databases drive increased sophistication in statistical models development  Customer interactions for risk and marketing are proactively initiated at all key points  Strategies are designed at customer-level

Automate the decisions

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

38

Stage 3: optimize for excellence

  Risk and marketing strategies are centrally designed based on advanced statistical techniques and drive customer profitability  Infrastructure enables total proactive control of the business – decision analytics becomes a way of life Decision analytics is well structured and integrated across business functions including risk, marketing, sales, operations, finance

Optimize the decisions

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

39

The Road Map

Build the infrastructure Fine-tune for performance Optimize for excellence Strategy Credit policy in place Profit driven credit strategy in place and reviewed regularly Monthly review of credit strategies. Champion/challenger a way of life

“a world-class consumer finance company ”

How are credit policies and strategies defined, reviewed and improved?

Processes How well defined and the processes, and what is the degree of automation?

Knowledge Processes well How well do staff defined and automated understand all profit drivers? What is the degree of expertise in credit scoring and decision science?

Processes regularly reviewed and refined. Little manual intervention Processes fully support profit-driven strategy, and are integrated across functions Education on strategy review process, fully understanding the use of MIS Include all available data into the process. Focus underwriter on “key” review, not second scorecard Tools Scorecards in place for all critical segments, decision engine used to control terms of business. Generate key KPI’s What credit management tools are used? How flexible are they? How easy is it for business user to change processes and strategies?

Ongoing knowledge improvement Full suite of scorecards, ability to optimize credit Ability to review and modify credit strategies ‘on the fly’ Organisation strategies Profit-driven organisation across functions Create strategy review cross-functional team Ensure clear assignment of responsibilities for risk management functions How are credit risk, marketing and finance working together? How are operational and strategic decisions taken?

© Experian Limited 2007. All rights reserved.

Confidential and proprietary.

40

Conclusions

 It all starts with data  Scorecards are important  Strategy is more important  Implementing the strategy properly is vital  If you don’t monitor you’re wasting you time  Risk management is a never-ending journey © Experian Limited 2007. All rights reserved.

Confidential and proprietary.

41

Best practice in data & scoring

Dr Paul Russell Director Analytical Solutions © Experian Limited 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited. Other product and company names mentioned herein may be the trademarks of their respective owners. No part of this copyrighted work may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian Limited.

Confidential and proprietary.