Transcript Measurement & Evaluation - Fisher College of Business

Measuring the Effects
of a Campaign
MKT 846
Professor West
Agenda
 Measuring
the Impact of IMC
 PACT Principles
 Testing Methods
 IMC Audit
General Thoughts
 Accountability
is a must in today’s business
environment!

Be prepared to demonstrate your worth
 Gap
analysis
Is there a difference between what customers expect from
a brand (based on brand messages) and what they
actually experience?
 Is your marketing communications sending the right
message?

General Thoughts
 If
you’ve done your homework then measuring
effectiveness is easy
Begins with objective setting
 Evaluation ultimately entails testing whether specified
objectives have been met
 Concept testing and concurrent testing allow you make
adjustments along the way and minimize the risk

 Predicting
what communications efforts will succeed
or fail is a lot like picking stocks.
General Thoughts
 Why
do you evaluate a campaign?
To determine if the strategy worked. If it didn’t, what went
wrong?
 To quantify and justify the return on investment.
 To make changes in the future.

 When

is it done?
Risk & Reward
Measuring the Impact of IMC
 What
do we know about the effectiveness of IMP
programs relative to the effectiveness of program
elements?
 Recent
studies have begun to shed light on the
interactive effect of communications methods.
Measuring the Impact of IMC
 Tests
of recall and memory of online and television
advertising alone or in combination

Design: 4 groups,


Results:


Ad exposure while visiting a web page on ESPN.com, Ad exposure
during an episode of Who Wants to Be a Millionaire, Ad exposure
while watching TV and visiting the web page, No exposure to the
ad (control group)
There was a synergistic effect when TV and online were used in
combination leading to a 9 percent gain in recall, a 48 percent
increase in recognition of the online ads.
Conclusion:

Online supports TV commercials and vice versa
(Online Publishers Association & IntelliQuest)
Measuring the Impact of IMC
 Tests
of recall and memory of online and television
advertising alone or in combination

Design:


Online exposure, TV exposure, Print exposure, a combination of
exposures
Conclusion:
Online supports TV and print ads
(Consortium: Microsoft, Marketing Evolution, ARF, and Unilever)

Measuring the Impact of IMC
 Nextel
When sales began to slide it sought to identify which MC
functions were producing the best returns
 Analysis revealed that web advertising had been the most
cost-effective in producing sales leads
 Budget allocation was revised and 100 percent of the
budget was allocated online (full-screen pop-ups, special
offers…)
 Increased the number of calling plans sold while reducing
the cost per sales lead from $35 to $11

PACT Principles:
 Provide
measurements are relevant to the
objectives of the advertising.
 Require agreement about how the results will be
used in advance of each specific test
 Provide multiple measurements and look for
convergence
 Consider how people respond to communication
Filter
Marketing Communication Model
Medium/Chael
M
M
Seder
A
Noise
E
Feed
bac
Basic Communication Model
 Six
key elements:
– the source of the message
 Message – the information being communicated either
verbally or nonverbally
 Receiver – the recipient of the message
 Common experiences (represented by the overlapping
fields of experience) – allows communication to occur.
 Feedback loop – the recipient’s response to the
message
 Noise – anything preventing the sender from getting
the message to the receiver or vice versa
 Sender
Information Processing
 McGuire
Circulat
ion,
Reach
Model Testing Ad Effectiveness
Listener
Recogni
tion
Recall,
Checkli
sts
Exposu
Attentio
Comprehe
re
n
nsion
Attitude,
Purchase
Intent
Delayed
Recall
Scanne
r Data
Accepta
nce/
Yielding
Retenti
on
Purcha
se
Hierarchy of Effects Models
Models:
Stages:
AIDA
DAGMAR
Innovation/
Adoption
Think
Attention
Awareness
Awareness
Exposure
Attention
Comprehensi
on
(Cognitive)
Knowledge
Feel
Interest
(Affective)
Desire
Liking
Preference
Conviction
Interest
Yielding
Evaluation
Retention
Purchase
Trial
Adoption
Behavior
Do
(Behavioral)
Action
Information
Processing
Persuasive Communication
Temporary
attitude shift
Motivated to
Process?
•Involvement
•Relevance, etc.
Yes
Yes
No
No
Peripheral cue present?
•Source characteristics
•Visual imagery
•Executional elements
Ability to Process?
•Issue familiarity
•Arousal, etc.
Yes
No
Nature of Cognitive Processing
Mostly
Favorable
Thoughts
Mostly
Mostly
Unfavorable Neutral
Thoughts Thoughts
Cognitive Structure
Change
•New cognitions adopted
•Different responses
Yes
Yes
Retain
Initial attitude
No
Elaboratio
Likelihood
Boomerang:
Persuasion:
enduring positive
attitude change
enduring negative
attitude change
The FCB Planning Grid
Thinking
High
Involvement
1. Informative (thinker)
Investments, House,
Appliances
Think – Feel - Do
Implications
Test: Recall, Diagnostics
Media: Long copy,
reflective vehicles
Execution: Information,
Demonstration
Low
Involvement
3. Habit Formation (doer)
Food, Household Items,
Cleaners
Do – Think – Feel
Implications
Feeling
2. Affective (feeling)
Jewelry, Cosmetics,
Apparel, Sports Car
Feel– Do – Think
Implications
Test: Attitude Change,
Arousal
Media: Large space, Image
specials
Execution: Impact
4. Self-Satisfaction
(impulse)
Cigarettes, Liquor, Candy,
Magazines
Do – Feel – Think
PACT Principles:
 Consider
whether the advertising stimulus should be
exposed more than once

Do you remember how many exposures it takes to have
an impact?
 Consider
the fact that finished copy can be more
soundly evaluated than rough copy, and require that
alternative executions be tested in the same degree
of finish

Apples versus oranges problem
PACT Principles:
 Provide
controls to avoid biasing effects of exposure
context

Primacy and recency effects
 Take
into account basic considerations of sample
definition

Target audience, Geographic representation, Random
sampling,
 Demonstrate

reliability and validity
Use experimental procedures
The Value of Pretesting (Concept testing)

McCollum Speilman Worldwide studied 4,637 on-air
commercials used to build normative measures.
40
35
30
25
Percetage
Not pretested
Pretested
20
15
10
5
0
Failure
Success
Market Testing of Ads (Concurrent Testing)
 Inquiry
tests
Measure ad effectiveness based on the number of
inquiries generated – web visits, coupons returned, phone
calls, etc…
 Split-run tests are used to compare different version of the
ad or running the same ad in different media.

Market Testing of Ads (Concurrent Testing)
 Recognition

Starch Ad Readership Report lets advertisers assess the
impact of an ad in a single issue of a magazine over time,
or across different issues.


Tests
Measures 25,000 ads in more than 400 issues across more than
100 magazines and newspapers per year.
The value of norms and measures of reliability make this a
widely used means of testing.
Market Testing of Ads (Concurrent Testing)
 Recall

Gallup & Robinson MIRS (Magazine Impact Research
Service) and Burke Day-After-Recall



tests
Can be used to assess an ad’s impact on memory
Once again, the normative value is very important
Suggestive brand names (Keller, Heckler, and Houston)



Tend to facilitate recall of the brand benefits (initial brand
positioning)
However, they inhibit recall of subsequent claims (more difficult to
introduce a new attribute at a later time).
Jack-in-the-Box, Old Spice, Kentucky Fried Chicken, Oldsmobile
How are recognition and recall related?
 NIPO
Study (Netherland’s Institute of Public Opinion)
The average correlation between recognition and recall of
print ads is very high (r = .95)
 Recall stems from recognition
 Likeable and interesting ads doubled recall scores
 Creative ads were more effective at enhancing preference
and recall than larger ads

Market Testing of Ads (Concurrent Testing)
 Tracking
Studies
One of the most useful and adaptable forms of posttesting
 Involves taking measurements at regular intervals
 A standard set of questions is used



Brand Metrics: awareness, recognition, recall, brand perceptions,
attitudes toward the ad and/or brand, purchase intention
Relationship Metrics: lifetime customer value, recency of purchase,
frequency of purchase, monetary value, referral index, share of
wallet
Sample sizes range from 250 – 500 cases per period
 Can be used to measure the differential impact of budget
size and scheduling as well as copy execution and media
selection

Performance Measures (Evaluative Testing)
 An
effective campaign is one that meets or exceeds
its objectives.
 The more closely the objectives are aliened with
sales and profit the better they are.
 Remember, MC is only one of many elements of the
mix that can impact sales. Others include:
Product performance
 Pricing
 Distribution
 Competition

Performance Measures (Evaluative Testing)
 Measuring
performance requires baseline measures
 Tracking studies are very useful in establishing
baseline and performance measures
IMC Audit
 Organization
Infrastructure
Is managing brand reputation and stakeholder
relationships a cross-functional responsibility?
 Are the strengths and weaknesses of all MC tools
considered?
 Is internal marketing utilized consistently and effectively?
 Do our agencies communicate with one another?

IMC Audit
 Interactivity
Does our media plan have a balance between mass and
interactive media?
 Is there facilitation of customer inquiries and complaints?
 Are customer databases being used to capture information
to facilitate relationship building?
 Are customer databases easily accessible and userfriendly?

IMC Audit
 Mission
Marketing
Is the organization’s mission evident in its MC plans?
 Does our mission provide a reason for customers to
believe our message and support the company?
 Are philanthropic efforts concentrated in one specific area?

IMC Audit
 Strategic
Consistency
Are company-created brand messages strategically
consistent?
 Do we periodically review all brand messages for
consistency?
 Do we consider what brand message is being sent by our
pricing, distribution, product performance, and customerservice

IMC Audit
 Planning
and Evaluating
Do we use SWOT analysis in MC planning?
 Are we using a zero-based approach?
 Are we fully leveraging intrinsic brand-contacts before
investing in new brand-contact points?
 Do we have a tracking study to monitor relationships with
customers and other stakeholders?
 Are we maximizing the unique strengths of various MC
functions?
 Is the overall objective of our MC program to create and
nourish profitable relationships with customers and
stakeholders?
