Transcript Title of Presentation

New Reasoning on Applying Sequential
Decisions for Customer Management
Dr. Gerald Fahner
Senior Director Analytic Science
FICO
© 2014 Fair Isaac Corporation. Confidential.
This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent.
You Can Increase Customer Value by
Optimizing Sequential Decisions
What are the Analytic Foundations for
Moving Beyond Single-Shot Decisions?
2
© 2014 Fair Isaac Corporation. Confidential.
Single-Shot Perspective on Decision Optimization
A1(t = 1)
A2(t = 1)
X(t = 1)
Customer
state
Business
decision
A3(t = 1)
Rationale: Optimize
Potential
actions
expected outcomes
through t = 2
3
© 2014 Fair Isaac Corporation. Confidential.
Y(t = 2)
Potential outcomes
(response, revenue, loss, profit)
Customer Lifetime Value (CLV)
Kotler and Armstrong (1996)—A profitable customer is a person whose
“revenues over time exceed, by an acceptable amount, the company costs of
attracting, selling and servicing that customer.”
CLV is the excess, defined as
“The present value of the future cash flows attributed to the customer during
his/her entire relationship with the company.”
Wikipedia: http://en.wikipedia.org/wiki/Customer_lifetime_value
4
© 2014 Fair Isaac Corporation. Confidential.
Sequential View of Customer Relationship
Decision sequence has cumulative effect on all future outcomes and on CLV
Y(t=2)
X(t=2)
X(t=1)
A(t=1)
Y(t=3)
X(t=3)
A(t=2)
Y(t=4)
X(t=4)
A(t=3)
New rationale:
Optimize decision sequence to maximize CLV
5
© 2014 Fair Isaac Corporation. Confidential.
Sequential Problems Are Ubiquitous, Impact Bottom Line
► Customer
acquisition
► Targeting
and offer design
► Customer/account
management
► Changing
card limits and pricing
► Influencing migration to profitable states
► Managing inactive customers
► Orchestrating customer dialogue
► Collections
► Optimizing
6
treatment sequences/scenarios
© 2014 Fair Isaac Corporation. Confidential.
When to End Marketing to Inactive Customers?
7
© 2014 Fair Isaac Corporation. Confidential.
Inactive Customer Problem
New customer
makes first
purchase
$$
… makes a few more
purchases
…may have lost interest
$$ $$
$
$
$
$
$
$
Q1
Q2
Q3
Q4
Q5
Q6
Periodic marketing
8
© 2014 Fair Isaac Corporation. Confidential.
End marketing,
or try a
different offer
Parameters to Consider When to End Marketing
1. From business economics:
► Profit
contributions from purchases, costs for marketing, operational costs
2. From customer behavior:
► Probabilities
Predictors of Purchase Probabilities
Possible Measures
Periods since last purchase
Recency (R)
Depth of relationship
Cumulative Frequency of purchases (CF)
Other customer characteristics
Time as customer, monetary, demographics, ...
► How
to compute optimal marketing policy based on these parameters?
► Develop
9
of customers making future purchases
and solve a Markov Decision Process (MDP) model
© 2014 Fair Isaac Corporation. Confidential.
1. Define a “State Space” Representing Dynamic
Customer Relationship
► Guidelines:
► Current
state should suffice to predict next state (Markov property)
► States should inform actions (Marketing $ / No Marketing
) and associated rewards
► We
chose State = (Recency, Cumulative Frequency) = (R, CF)
► New
customer: State = (1, 1)
► Customer who made 3 purchases, then did not buy for 4 periods: State = (4, 3)
► We
add a special terminal State = (END) for customers no longer marketed to
► Who
10
goes there, stays there
© 2014 Fair Isaac Corporation. Confidential.
2. Estimate State Transition Probabilities
► Customers
experience state transitions between discrete time periods
► Empirical purchase probabilities that customers will buy in next period: Pbuy(R,CF)
customer makes purchase:
(R, CF)  (1, CF+1), with probability Pbuy
► Marketed customer makes no purchase: (R, CF)  (R+1, CF), with probability 1-Pbuy
► Customer is no longer marketed to:
(R, CF)  (END), (END)  (END) with probability 1
► Marketed
Structure of transition graph
(R, CF)
(END)
1
1
(R, CF)
Pbuy
$
1-Pbuy
11
© 2014 Fair Isaac Corporation. Confidential.
(1,CF+1)
(R+1, CF)
3. Specify Reward Structure
► Business
collects (or loses) reward points during each time period
► Profit Contribution PC, Marketing (operational) Expense ME are reward parameters
customer makes purchase:
PC – ME, with probability Pbuy
► Marketed customer makes no purchase:
– ME, with probability 1-Pbuy
► Customer is no longer marketed to:
0, with probability 1
► Marketed
Structure of transition graph
(R, CF)
(END)
1,0
1,0
(R, CF)
Pbuy, PC-ME
$
1-Pbuy, -ME
12
© 2014 Fair Isaac Corporation. Confidential.
(1,CF+1)
(R+1, CF)
Estimating and Optimizing CLV
13
© 2014 Fair Isaac Corporation. Confidential.
How to Estimate CLV From a Finite Data Window?
► CLV
►
is a concept based on an indefinite time horizon:
“The present value of the future cash flows attributed to the
customer during his/her entire relationship with the company.”
► Markov
►
State transition probabilities and reward structure can be learned
from a finite data window and/or augmented by judgment
► Model
►
14
model allows CLV estimation from finite data window
entails assumptions
Transition probabilities, rewards depend only on state variables, are
independent of time
© 2014 Fair Isaac Corporation. Confidential.
George E. P. Box (1919–2013)
“Essentially, all models are
wrong, but some are useful”
Hypothetical Portfolio
Parameters and Assumptions
► New
customers’ data over 3 years
►
Discretize activities into quarterly intervals
► Use Recency, Cumulative Frequency to define states
► Historic
marketing policy:
►
All customers marketed until Recency = 10
► Pbuy = 0 after marketing ends
► Few
customers make more than 5 purchases
► Discretize
►
R and CF into 51 states:
R = {1, …, 10}  CF = {1, …, 5+}, (END)
► Estimate
Pbuy for each state
► Reward
parameters: PC = $100, ME = $10
► Discount Rate: 12% p.a.
15
© 2014 Fair Isaac Corporation. Confidential.
Empirical purchase
probabilities for 50
nonterminal states
CLV Estimation Via Random Walks
Example: CLV for New Customers Subject to Historic Policy. PC = $100, MC = -10
New
Simulate all future marketing, purchases, rewards, until END state is reached
customer
NoBuy
NoBuy
NoBuy
Several Periods NoBuy NoMarketing/NoBuy
Buy
Buy
-10
90
-10
0
-10
-10
-10
100
…
(END) 0
(11, 2)
(1, 2)
(3, 1)
(2, 1)
(2, 2)
(1, 1)
$
$
$
$
$
$
 10
 10
90
 10
 10
 10





...

0
1
2
3
4
5
13
1  d  1  d  1  d  1  d  1  d 
1  d 
is the associated discounted cumulative reward ( d : Discount Rate)
100 
Perform 10,000 random walks. CLV(1,1) is the average discounted cumulative reward.
(Similarly, can estimate CLV for all other states, by starting random walks from each state)
16
© 2014 Fair Isaac Corporation. Confidential.
Easier: Calculating CLV Using Matrix Algebra
Purchase probabilities
Matrix algebra[1]
Dynamic model
1
Pbuy, PC-ME
(R, CF)
(1,CF+1)
$
(R+1, CF)
1-Pbuy, -ME
(R, CF)
(END)
1,0
Reward parameters
17
© 2014 Fair Isaac Corporation. Confidential.
1,0
P 

CLV   Ι 
 R
1

d


P : Transition Matrix
I : Identity Matrix
R : Reward Vector
d : Per - period discount rate
Historic Policy and Associated CLV
All customers marketed ( $ ) until Recency = 10
18
© 2014 Fair Isaac Corporation. Confidential.
Computing Optimal Policy
► CLV
is negative for 17 out of 50 states
► Might
►
improve CLV by not marketing to customers in those 17 states
But it’s not as simple...
Policy Iteration Algorithm[2]
► Initialize
with an arbitrary policy (e.g. historic policy)
► Alternate between two steps:
►
Policy evaluation:
Given current policy, calculate CLV for all states
► Policy improvement: For each state, find action that maximizes one-step look-ahead
estimate of CLV. If any action changes, replace current by new policy
► Stop
19
when CLV no longer improves
© 2014 Fair Isaac Corporation. Confidential.
Historic Policy and Associated CLV
All customers marketed ( $ ) until Recency = 10
20
© 2014 Fair Isaac Corporation. Confidential.
Improvements From Historic Policy After First Policy Iteration
$
21
© 2014 Fair Isaac Corporation. Confidential.
: Market
: Don’t market
Marketing to (R = 4, CF = 1) (
$
22
© 2014 Fair Isaac Corporation. Confidential.
: Market
)Yields Further Improvements
: Don’t market
Optimal Policy and Associated CLV After Convergence
$
23
© 2014 Fair Isaac Corporation. Confidential.
: Market
: Don’t market
Value of Optimizing Sequential Marketing Decisions
for a New Customer
Policy
Historic
CLV(1,1)
$131.03
$134.92
Profit Contribution from initial purchase
$100.00
$100.00
$31.03
$34.92
Expected present value from future cash flows
% Improvement of future cash flow
24
Optimal
© 2014 Fair Isaac Corporation. Confidential.
+12.54%
Summary of CLV Optimization
► CLV
is not a passive metric—it can and should be managed
► Markov
Decision Processes offer framework to optimize sequential decisions, CLV
► Model
formulation (states, transition probabilities, rewards)
► Data acquisition and parameter estimation
► Software for matrix algebra and optimization
25
© 2014 Fair Isaac Corporation. Confidential.
Extensions and Challenges
26
© 2014 Fair Isaac Corporation. Confidential.
Extensions
► Marketing
example could be extended in many directions
► Inclusion
of additional customer attributes (monetary, product holdings,
time as customer, demographics, ...)
► Customer
► Varying
dialogue—asking for customer preferences ahead of offers
profit contributions per customer per period
► Alternative
► Drop
marketing offers and their costs
assumption that customers don’t buy unless marketed to
► Transition
probabilities could depend on macro-economic parameters
(such as consumer confidence)
27
© 2014 Fair Isaac Corporation. Confidential.
Challenges and How to Address Them
► “Curse
► As
of dimensionality”:
more variables are added to define the state
►
State space will explode
► Data will be too sparse to estimate transition probabilities directly
 Be parsimonious, leverage dimension reduction/machine learning techniques
► “Curse
► For
of consistency”:
a business that does not experiment with alternative actions
►
Only a tiny sliver of state-action combinations will be observed
► Data will give little guidance on alternative policies
 Implement a learning strategy that tests reasonable actions for each state
28
© 2014 Fair Isaac Corporation. Confidential.
Discussion
► Sequential
► Markov
► “Art
► Need
decisions can substantially increase customer value
Decision Models provide analytic framework
and science” to develop model
customer transactions, business treatments, cash flow information
► What,
when, how much?
► Advanced
analytic methodologies address curses of dimensionality, consistency
Let us know if you’re interested in a Proof of Concept!
29
© 2014 Fair Isaac Corporation. Confidential.
References
[1] Modeling customer relationships as Markov chains. Phillip E. Pfeifer and Robert L.
Carraway, John Wiley & Sons, Inc. and Direct Marketing Educational Foundation, Inc. Journal
of Interactive Marketing Volume 14, Issue 2 (2000), pp. 43–55.
[2] Dynamic Programming and Markov Processes. Ronald A. Howard, The M.I.T. Press, 1960.
30
© 2014 Fair Isaac Corporation. Confidential.
31
Thank You!
Dr. Gerald Fahner
[email protected]
++1 512 698 0609
© 2014 Fair Isaac Corporation. Confidential.
This presentation is provided for the recipient only and cannot be reproduced or shared without Fair Isaac Corporation’s express consent.
Learn More at FICO World
Related Sessions
►Addressing Attrition: Ultra-Dynamic Multi-Dimensional Attrition Analytics with Tree Ensemble Models
Experts at FICO World
►Larry Rosenberger
►Oliver Bastert
White Papers Online
►Closing the Loop and Eliminating Costly Delays in Analytic Development and Deployment
►Customer Centricity: Four Bank Success Stories
Blogs
►http://www.fico.com/en/blogs/category/analytics-optimization/
32
© 2014 Fair Isaac Corporation. Confidential.
Please rate this session online!
Dr. Gerald Fahner
[email protected]
++1 512 698 0609
33
© 2014 Fair Isaac Corporation. Confidential.