Transcript Title of Presentation

Fact From Fiction:
Demystifying Credit Score Features
Why the FICO® Score Looks the Way It Does
Frederic Huynh
Senior Principal Scientist, Analytic Development—Scores
FICO
© 2014 Fair Isaac Corporation. Confidential.
This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent.
Much care and thought is put into the design of
the FICO® Score. With each subsequent
development we challenge our prior design
conventions and meticulously evaluate the merits
of any challenging approach.
2
© 2014 Fair Isaac Corporation. Confidential.
Agenda
►Scoring
More Consumers Is Easy…
►Engineering
Robustness in the
FICO® Score
►Why
You Want Three Unique Score
Algorithms
3
© 2014 Fair Isaac Corporation. Confidential.
Scoring More Consumers Is Easy…
► …But
4
Scoring More Consumers Responsibly Is Hard
© 2014 Fair Isaac Corporation. Confidential.
The Challenge to Score More Consumers
► As
lenders focus on growing their portfolios, the credit underserved
population is becoming of interest again
► Segments
that could be evaluated for potential inclusion in the
scoreable universe for the FICO® Score:
► Files
with no trade line history
►
Collection and public record only files
► Inquiry only files
► Collections, public records, inquiries
► “Stale”
files (no trade line updates within the last 6 months)
► Very young files (less than 6 months trade line history)
5
© 2014 Fair Isaac Corporation. Confidential.
Key Considerations in Evaluating Unscoreable Segments
Is the credit data
available sufficiently
predictive of future
repayment risk?
Is there sufficient
credit repayment
history from which
to develop models?
Will the score
continue to effectively
rank-order risk?
?
6
© 2014 Fair Isaac Corporation. Confidential.
Is the Available Data Sufficiently Predictive?
Categories of Predictive Variables in the FICO® Score
10%
10%
35%
15%
30%
► Inquiry
Payment History
Amounts Owed
Length of Credit History
New Credit
Types of Credit Used
Only Files: Only New Credit predictors are available
► Collection/Public
Record files: Only Payment History and/or Amounts Owed
predictors are available
7
© 2014 Fair Isaac Corporation. Confidential.
Example
Tepid Prediction Available for Very Thin Credit Files
Gini Coefficient
Test Segment = Collections and Public Records Only
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Scoreable Population
► Risk
Test Segment
discrimination attainable from the extremely sparse data is much weaker
than what lenders currently expect from the FICO® Score
8
© 2014 Fair Isaac Corporation. Confidential.
Sufficient Performance History Is Needed for
Model Development
Bureau Snapshot A
Scoring date
Bureau Snapshot B
2 years
Performance date
Only consumers who had measurable credit repayment history between Snapshots A and B
(i.e., “classifiable performance”—can be included in the model development sample)
9
© 2014 Fair Isaac Corporation. Confidential.
Challenge #1: Truncation
► Current
FICO® Scoreable Universe
of current scoreable universe have “classifiable performance” in the following 24
month window
► ~85%
► Test
segment: “Stale” files (last update 24+ months ago)
► Only
4.6% of test segment have “classifiable performance” in the following
24 month window
► Risk
patterns observed on 4.6% of sample with ‘classifiable performance’ must be
assumed to hold for the other 95.4% of the segment
► Like
10
trying to draw the whole puppy based on its tail
© 2014 Fair Isaac Corporation. Confidential.
Challenge #2: Bias
► Test
segment: Collection and Public Record only files
► Only
6.5% of test segment have “classifiable performance” in the following
24 month window
► The
6.5% of sample with ‘classifiable performance’ likely to be “cherry-picked” by
lender—based on non-CB qualifications such as high income, high assets, VIP
status, etc.
► Repayment
rates for this small, favored, non-representative slice of the segment
likely to be much better than would be observed for the segment as a whole
► The
make-up of data available for model development is impacted by previous
lender decisions
► The
bias in these segments can over-estimate the credit quality of the
“new scoreable” segment
11
© 2014 Fair Isaac Corporation. Confidential.
Is There Sufficient Credit Repayment History from Which
to Develop Models?
% of Potential Scorable
% with Perf
Young Exclusions
5.4%
56.2%
Kind of Stale Exclusions
19.9%
8.3%
Really Stale Exclusions
40.0%
4.3%
No Trade Files
34.8%
7.0%
► If
you are building a model to predict good and bad payment behavior, you need
enough information to determine if a consumer is a good or bad payer
► Young
Exclusions: <6 Months Time on File
► Kind of Stale: Months Since Most Recent Report Date between 7–23
► Really Stale: Months Since Most Recent Report Date >= 24
12
© 2014 Fair Isaac Corporation. Confidential.
Challenge #3: Robustness in Rank Ordering
Test Segment: “Stale” Files
Slope of Odds-to-Score Line
► Rank
Months Since
Reported
Slope
PDO
0–6
0.0202
35
7–11
0.0159
44
12–23
0.0150
46
24–47
0.0155
45
48+
0.0115
60
ordering weakens as staleness increases
► Degradation
in slope as staleness increases is observed across a wide variety of
industry performance variables (bankcard, auto, mortgage, etc.)
13
© 2014 Fair Isaac Corporation. Confidential.
A Misleading Score May be Worse than No Score
► Building models on populations with low classifiable performance rates is likely to:
► Over-estimate the credit quality of the “new scoreable” segment
► Be much less robust—across industries, across applications (originations vs. account management),
and over time
► A credit
score returned from a sparse data file may be unreliable or inaccurate.
This may yield:
►
Credit lines/loans higher than safe, or lower than necessary
► Interest rates higher than necessary
► Declination where there should have been acceptance, or vice versa
► No
one benefits when faulty credit risk assessments yield credit offerings
(or declinations) inconsistent with the consumers’ true repayment ability
14
© 2014 Fair Isaac Corporation. Confidential.
Alternative Data and More Robust Modeling Techniques
Are the Solution
► Limited
predictive signal you can extract from a traditional credit bureau report
► Solutions
incorporating alternative credit data are essential to growing portfolios
responsibly when traditional credit data is unavailable or sparse
► Selection
of the best combination of alternative credit sources is critical
► Coverage
► Regulatory
compliance
► Predictive power
► Modeling
► Stay
15
techniques need to be incorporated to mitigate truncation and bias
tuned for our latest research in alternative data solutions
© 2014 Fair Isaac Corporation. Confidential.
Engineering Robustness in the FICO® Score
► Designing
16
Scores to be Effective Across the Economic Cycle
© 2014 Fair Isaac Corporation. Confidential.
Engineering Robustness in the FICO® Score
► A key
value of the FICO® Score—designed to be a robust rank-ordering tool
over 25 years FICO® Scores have been proven to hold up well across wide variety of
economic conditions
► For
► The
FICO® Score is developed using a single pair of snapshots
► Blending
multiple pairs of snapshots together to build credit scores is sometimes
used for model development
FICO® Score 9, we revisited our development convention to assess the merit
of developing using blended samples
► For
17
© 2014 Fair Isaac Corporation. Confidential.
FICO’s Blended Snapshot Research Overview
► Conducted
study into relative predictive strength and through-the-cycle
robustness of three approaches:
► FICO®
►
Score status quo
Based on 2010–2012 sample
► “Light”
blend: Dual timeframe sample, with samples closely related in time and economic
characteristics
►
2009–2011 and 2010–2012
► “Premium”
blend: Six-timeframe sample, covering a seven year period of economic boom,
bust, and initial recovery
►
2005–2007, 2006–2008, 2007–2009, 2008–2010, 2009–2011, and 2010–2012
► Validated
► Both
18
these approaches on out-of-time samples from 2010, 2011, and 2012
12 and 24-month performance windows
© 2014 Fair Isaac Corporation. Confidential.
FICO’s Blended Snapshot Research Findings
Limited Support for
Light Blend Score
►
Light blend score consistently less
predictive (up to 1 KS point) than
“status quo” score built on single
snapshot
► Light blend score shows slightly less
stability in odds to score alignment
across segments compared to “status
quo” score
19
© 2014 Fair Isaac Corporation. Confidential.
Premium Blend Score
►
Performs better than light blend score
► In some cases performs better than
“status quo” score
Additional Consequences of Blended Samples
10/2003–
10/2005
10/2005–
10/2007
10/2007–
10/2009
10/2009–
10/2011
Fields to identify
medical collections
were introduced in
September 2011 for
one CRA
The CARD Act of 2009
impacted the underwriting
of credit cards
10/2011–
10/2013
Diluting your development sample with older data can
prevent innovation from leveraging newly reported
fields and deemphasize current market realities
20
© 2014 Fair Isaac Corporation. Confidential.
How Well Does FICO® Score 9 Perform in a
More Turbulent Time Period?
► FICO®
Score 9 was developed on October 2011–October 2013 data, a time
period where many vintages were performing exceptionally well
► To
address concerns about the model’s ability to hold up during a more stressed
economic period, the model’s predictiveness was assessed on April 2006–April
2008 data
► This
represents one of the FICO® Score 8 development databases
all key industries and applications, FICO® Score 9 is highly competitive
with FICO® Score 8 on FICO® Score 8 development data despite the fact that
FICO® Score 8 has “home court advantage”
► Across
► Strong
out of time performance results is testament to the robustness of the score
and advancements in predictive modeling incorporated in FICO® Score 9
21
© 2014 Fair Isaac Corporation. Confidential.
Despite the Radical Change in the Mortgage Industry
FICO® Score 9 Performs Extremely Well
Trade-off Curve Comparison
FICO® Score 9 vs. FICO® Score 8 on 2006–2008 Data
Mortgage—Originations
FICO Score 9
FICO Score 8
% 90+/Any Derog Accounts
100
90
80
70
60
50
40
30
20
10
0
0
22
10
© 2014 Fair Isaac Corporation. Confidential.
20
30
40
50
% Total Accounts
60
70
80
90
100
Conclusions
► There
are pros and cons with any analytic approach
► With
a blended sample approach, much thought needs to be put into the selection
of the actual timeframes used for development
► Selecting
two samples very close to each other:
►
Defeats the whole purpose of building more variation into the development data and does meet the
objective of covering a broader spectrum of the economic cycle
► Delays/waters down introducing enhancements if necessary data is available in current time periods
but not in late time periods
stress testing the model, FICO® Score 9 was assessed on FICO® Score 8
development data and proven to be extremely competitive in a drastically different
environment
► In
FICO® Score 9 out of time performance analysis underscores the value of
innovation and advancements in predictive modeling
► The
23
© 2014 Fair Isaac Corporation. Confidential.
Why You Want Three
Unique Score Algorithms
► Balancing
24
Score Consistency with Leveraging Unique Data Elements
© 2014 Fair Isaac Corporation. Confidential.
Score Variation Across Bureaus
► Key
driver of material score differences: material differences in the data
► FICO®
Scores employ a unique algorithm tailored to each CRA
► This
is done to leverage the unique aspects of the data captured by each bureau
► Voice-of-customer feedback: “leverage the unique data available at each bureau”
► Identical
FICO® Score design blueprint applied across the bureaus
► Same
segmentation
► Same performance window (24 months)
► Same performance definition (90+ days past due)
► Same candidate set of 500+ predictors
facilitate greater consistency with FICO® Score 9 each model was developed
using data from 10/2011–10/2013
► To
25
© 2014 Fair Isaac Corporation. Confidential.
Data Formats/Storage Can Differ Across Bureaus
► One
bureau does not report negative information associated with authorized user
accounts
► One
bureau reports the date of first delinquency associated with a collection
agency account as opposed to reporting only when the collection agency account
was assigned
► The
minimum score criteria is the same at all three bureaus—yet the score
exclusion rate varies from 15% to 25% depending on the bureau
► Some
► One
bureaus provide more granular information on fields than others
byte industry “kind of business” code
► Each
bureau highlights unique sources of information. For example, one bureau
highlights their inclusion of rental data from a national network of property
management companies
26
© 2014 Fair Isaac Corporation. Confidential.
Conclusions
► The
data available at each CRA can be different
► FICO®
Scores are built with a consistent design blueprint across CRAs and has
maintained alignment between CRAs and prior versions
Scores maintains the competitive advantage of each CRA’s unique data to
provide the best predictive value
► FICO®
27
© 2014 Fair Isaac Corporation. Confidential.
28
Thank You!
Frederic Huynh
[email protected]
(415) 492-5323
© 2014 Fair Isaac Corporation. Confidential.
This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent.
Learn More at FICO World
Related Sessions
►The FICO® Score: 25 Years of Democratizing Access to Credit
►Leveraging Alternative Credit Data to Make Better Risk Decisions
Products in Solution Center
►Unveiling
FICO® Score 9—Your Credit Score for the New Normal
Experts at FICO World
►Frederic Huynh
►Tommy Lee
►Julie Wooding
White Papers Online
►To Score or Not to Score? #70
Blogs
►www.fico.com/blog
29
© 2014 Fair Isaac Corporation. Confidential.
Please rate this session online!
Frederic Huynh
[email protected]
30
© 2014 Fair Isaac Corporation. Confidential.