Measuring Success:

Be careful what you measure… …You just might get it!

Lizanne Kaiser, Ph.D.

Customer Experience Designer Genesys [email protected]

Dog Training 101:

“Be careful what you train for …You just might get it!” “Fido, Sit! Sit, Fido, Sit! Come on, Fido, Sit! I said Sit!

Sit, please??” “I guess I don’t have to sit till she says ‘Sit’ at least 6 times.”

2

Measuring Success in Speech Projects

3

Be careful what you measure… …you just might get it!

Envision Success

Before After

4  What does success look like to you?

• •

Envision success

before

beginning project Will this future vision yield your desired business results ?

How will Success be Measured?

 Clearly define & prioritize specific Metrics  Decide how these will be gathered & analyzed • •

Method used can impact data reliability Avoid:

“Garbage in; garbage out!”

 Include in project plan: • • • •

(don’t treat as “afterthought”) Application logging & reporting IVR and Call Center metrics Staff training

 Align Department/Channel Goals, MBOs, Rewards & Remuneration  Consider what else can impact success •

Business conditions and policies

•

Changes over time

5

Does this sound familiar?

At the Start of a Speech Project…

Consultant: Business Manager: 1.

“How will you measure success for this speech project?”

2.

“Our main goal is we must increase our Automation Rate .”

3.

“What does ‘Automation Rate’ mean to you?”

6.

“How do you measure ‘Customer Satisfaction’ today?”

8.

Hmm… Are your customers really satisfied? And how do you keep them satisfied if you don’t want to talk to them??

4.

“The Containment Rate of our current touchtone IVR is 85%. So, the new speech system should allow no more than 15% of calls through to our Agents.”

5.

“Oh, and by the way, the new system shouldn’t negatively impact Customer Satisfaction either.”

7.

“Our Agents periodically do end-of-call surveys. We typically get around a 4 on a scale of 1-to 5.”

6

Containment Rate

a.k.a. “Automation Rate”  • • Encourages systems that lock caller into “IVR Jail”

For example:

By disabling or not advertising commands for “Agent” or pressing 0

Bank example:

Callers who use Agent for task which could have been done in automation are charged a Service Fee

Ouch !

  • Impacts Customer Satisfaction & First Call Resolution

Companies that made it harder to reach Agent saw:

61% decrease in Customer Satisfaction & 49% in First Call Resolution More repeat calls; More frustrated callers. Data from 60 studies by Vocal Laboratories • Doesn’t give “partial credit” for Partial-Call Automation

Insurance example:

Collecting HIPAA authentication info in speech system before transfer to Agent saved 2 minutes in Average Talk Time 7

Containment Rate

a.k.a. “Automation Rate”  8 • Even in well-designed systems that customers want to use, there’s a risk to over-emphasizing Containment …

Bank example:

Current touchtone IVR Call Containment = 85+% of 20+M calls/year Longer-term Goals: 1.

Automate 99% of self-service functionality cross-channels 2.

Product Penetration: Aggressive organic growth through Cross/Up-Selling: Increase average from 1-2 products/customer to 6 (industry average = 3)

How do you grow your relationship with your customers when you’re automating so many customer interactions?

Success Metrics

9  Measure a blend of both Efficiency and Effectiveness…

Efficiency

• • • • Containment Rate Average Speed of Answer Average Talk Time per Call/Agent Average Calls Handled per Agent

Effectiveness

• • • • • • • Task Rates Attached Data Rates Total Talk Time across Calls/Agents First Call Resolution Staff Engagement Customer Satisfaction/Engagement Customer Lifetime Value  Traditional Efficiency Metrics don’t always shed light on the value of partial-call automation

Task Rates & Attached Data Rates

Medical Insurance example 10  Identify Tasks , Task Type , Start & End Points • and Attached Data

Enable application logging and reporting to track these

 Tasks:

Authentication, Claims, Benefits, ID Card Requests, etc.

 Task Types: •

Transactional Tasks:

Caller uses automated system to perform some transaction • Clear Start Point (when process begins) & End Point (when transaction is completed)

Informational Tasks:

Caller uses automated system to give/get some information Clear Start Point ; No clear End Point (caller may stop at any point, without process being incomplete)  Task Initiation Rate: •

% of times Start Point

•

of a particular Task is reached Useful for both Transactional and Informational Tasks

 Task Completion Rate: •

% of times End Point of a particular Task is reached

•

Useful for Transactional Tasks ; inconclusive for Informational (no clear End Point)

 Attached Data Rate: •

For transferred calls, track % of Attached Data collected during a particular Task

Authentication Task

Medical Insurance example  Task Type: Transactional  Start point: •

Get ID Number

If caller reaches Get ID Number, log as Task Initiation

 End Point: • •

Retrieve Profile

If reaches Retrieve Profile, log as Task Completion If doesn’t reach this point (hangs up or transfers), Task Completion = “unsuccessful”

 Authentication-related Attached Data: •

Task transferring from: Authentication

• • •

ID Number Relationship to policy holder Date of Birth Agent:

“Hello, am I speaking with Mrs. Smith?... I already have your ID Number and Relationship you gave our automated system. I just need to get your Date of Birth.”

Claims Summary Task

Medical Insurance Summary   Task Type: Informational Start Point: •

Get Date of Service

If caller reaches Get Date of Service, log as Task Initiation

 No clear End Point: •

If reaches Claims End Menu, log as Task Completion

•

If doesn’t reach this point (hangs up or transfers), Task Completion = “inconclusive”

 Claims-related Attached Data: •

Task transferring from: Claims

• • •

Date of Service requested Specific claim(s) that were played Claim(s) that Explanation of Benefits was requested for Agent:

“Hi Mr. Jones. I see you’re calling to check on your claim from June 10 th for $125. How can I help you with that?”

Average Talk Time vs. Total Talk Time

Telco example: Unintended consequences of traditional measurement 13 My phone isn’t working

Before

(If I don’t resolve this in another 20 seconds I’ll get a negative review) My phone isn’t working

After

(I’m not going to finish this call until we find the problem) We’ll send an engineer tomorrow to find the problem OK, we’ll work on this together to get this resolved Results: • Engineer dispatched unnecessarily • Delays in resolving customer problems • Huge costs for unnecessary onsite visits “Wrong” Measurement: Average Talk Time per Call/Agent Results: • Tickets closed faster • Customer satisfaction increases • Overall operations costs reduced Key Measurements: First Call Resolution & Total Talk Time across all Calls/Agents

Customer Satisfaction

Avoid:

“Garbage In



Garbage Out!”

14  Customer Surveys seem deceptively simple: •

“Just ask some questions & collect responses. That’s all there is to it, right?”

 But data from a survey is only as good as the design going into it and the analysis coming out of it  Is Speech System replacing Touchtone-only System or Live Agent?



“On a scale of 1-to-5, 5 being the best, how satisfied…?”

•

Yields very little “spread” in the satisfaction ratings (get a lot of 4s)



Many things

impact Customer Satisfaction •

Difficult/Impossible to isolate impact of speech automation

 Callers don’t differentiate Self vs. Assisted- Service •

It’s all just one type of service … Customer Service

Customer Satisfaction vs. Customer Engagement

Customer Satisfaction is

not

Satisfactory  15

Satisfied

customers • aren’t necessarily loyal ; those who are

engaged

are loyal

Making “loyalists” out of just 5% more customers increases profit per customer by 25-100%

Frederick Reichhold,

The Loyalty Effect

 The Gallup Organization:

Customer Engagement

•

Emotional connection: Customer’s Loyalty , Confidence , Integrity , Pride , and Passion in Brand

 Emotional connections pay off: (based on Gallup Study) • •

“Extremely Satisfied” customers represented no added value Regardless of Satisfaction, “Emotionally Engaged” customers spent 46% more

 So, rather than asking customers if they agree with “vanilla” statements like… •

“I am satisfied with the customer service I received.”

 Use more emotionally compelling statements like… •

“[Brand] is the perfect company for people like me.”

•

“I can't imagine a world without [brand].”

 Not all opinions are created equal •

What do your most valuable & loyal customers think?

Agent Engagement

“Build relationships one customer and one employee at a time.”

16

Brand Differentiation is best achieved by establishing a Relationship with your Customer, not based on products and price

Key Takeaways

17  How you define and measure Success consequences has broad-reaching throughout the entire speech project and to your business  Don’t over-emphasize Call Containment  Instead, leverage speech to build a strategy around Partial-Call Automation which yields real cost savings

and

allows you to grow Customer Relationships through human-to-human interactions

Any Questions?

Thank you!

References

19 Applebaum, A. (2001) “The Constant Customer.”

Gallup Management Journal

, Jun 17, 2001. BAI (2006) “Converting to a Relationship: Banks Need to Tap Young Service Seekers to Add More Loyalist Customers, Build Profits.” PR Newswire Association LLC, Nov 16, 2006. Coffman, C. & G. Gonzalez-Molina (2002)

Follow this Path: How the World's Greatest Organizations Drive Growth by Unleashing Human Potential.

New York, NY: Warner Books, Inc. Conley, D., D. James, L. Kaiser & J. Pilkin (2006) “Highmark’s Integrated Solution for Speech Automating Medical Member Calls.”

Blue Cross Blue Shield Association Call Center Conference

. Salt Lake City, UT. James, D. & L. Kaiser (2006) “Don’t Block the Entrance. Don’t Block the Exit.”

Nuance Conversations

. Orlando, FL.

Larson, J. A., et al. (2005) “Ten Criteria for Measuring Effective Voice User Interfaces.”

Speech Technology Magazine,

Vol. 10, Num. 6, Nov/Dec 2005: 31-35. Leppik, P. (2005) “Does forcing callers to use self-service work?” http://www.vocalabs.com/resources/newsletter/newsletter22.html McEwen, W. J. (2005)

Married to the Brand: Why Consumers Bond with Some Brands for Life.

Princeton, NJ: Gallup Press.

McEwen, W. J. (2004) “Why Satisfaction Isn’t Satisfying.”

Gallup Management Journal

, Nov 11, 2004. McEwen, W. J. & J. H. Fleming (2003) “Customer Satisfaction Doesn’t Count.”

Gallup Management Journal

, Mar 13, 2003. Reichheld, F. F. & T. Teal (2001)

The Loyalty Effect: The Hidden Force Behind Growth, Profits, and Lasting Value.

Cambridge, MA: Harvard Business School Press.