Evaluation - Did it really work?

Effective public relations measurement and evaluation

Presented by Dr Tom Watson, Associate Professor in Communication, Charles Sturt University

Flying on one instrument?

Agenda

 The communications process  What is evaluation?

 Models of evaluation  Brief case studies

How PR is practiced

 Grunig’s four models  One-way • • Press agentry Informational  Two-way • • Asymmetrical Symmetrical

Communications process

Presentation  Attention  Comprehension  Acceptance  Retention  Action - Outcome McGuire (1984 )

Communication and evaluation

Output (L1)

Presentation Attention Comprehension Acceptance Retention Action

Out-growth (L2) Outcome (L3)

Lindenmann 1993

Types of evaluation

 Summative – final impact  Formative – “in progress” assessment  Should be integrated into all stages of programs

What can be measured?

 Success or failure of strategy  Message distribution and reception  Understanding, favourability and acceptance  Trends

What else?

 Internal  Staff attitudes; value of newsletters, video, team briefings  SWOT  External  Sponsorship impact  Issues management and prediction  Stakeholder attitudes

Case studies

 PricewaterhouseCoopers (PwC) accountants and biz advisers  DIMIA – Harmony Day  Polestar – major printing group  Volvo - XC90 launch

PwC

 Why aren’t we getting coverage? – asked the bosses  Merger of PW with C&L not seen as equal – a “takeover”  Change in approach

PwC lessons

 Charts and graphics improve story acceptance and favourable write-up  CEO in interviews increases favourability – “voice of the organisation”  Benchmark coverage to find “who’s writing about what”

DIMIA

 Monitored media coverage of events  Moved from celebrity to community basis  More outside metros  Increased participation

Polestar

 Europe’s biggest print group  Going through major changes  Monitored media coverage  Able to see strategy and tactics working

Polestar – mid-2001

6 -2 -4 -6 4 2 0 Negatively scoring items Positively scoring items -8 -10 Investment Innovation Employer Forward thinking Financial security CTP

Message

Demographics Customer service Value Environment

Polestar – early 2003

9 6 5 8 7 Negatively scoring items Positively scoring items 2 1 4 3 0 Investment Innovation Employer Forward thinking Financial security CTP

Message

Demographics Customer service Value Environment

Volvo – new car launch

UK Launch program

 Competing against BMW and Mercedes  Strategy was to have continuing campaign instead of launch “blip”  Closely targeted – 25 to 44, mid to high income, degree educated, adventurous  Total of 609,000 people (1% of pop.)

PR-led campaign

 Media relations drove enquiries to internet and dealers  More interest than cars to sell; All ordered before sale date announced  Advertising ran for one month and cancelled – saving of c.$6m  “PR helped sell the required number of cars”

PR works!

¦ ¦ ¦ ¦ ¦ ¦

Evaluation

 Three methods of evaluation used:  Responses to internet and then dealers  Reach of coverage to target audience 96% of target group saw it once; 90% twice  Sales

Evaluation – your future

 Forget the “Holy Grail”, it’s about relationships (Hon & Grunig 1999)  Most PR campaigns don’t influence behaviour. (McCoy & Hargie 2003)  Avoid dead-ends like management language for evaluation, e.g. ROI (Watson 2005)

References and Reading

        Hon LC & Grunig JE (1999). Guidelines for measuring relationships in public relations. www.instituteforpr.com

Lindenmann, W.K. (1993) An “effectiveness yardstick” to measure public relations success. Public Relations Quarterly , 38 (1) McCoy M & Hargie (2003) Implications of mass communication theory for asymmetric public relations evaluation. Journal of Communication Management

7

(4) McGuire, WJ 1984 Attitudes and Attitude Change in Lindzey, G. & Aronson, E. Ed. The Handbook of Social Psychology Vol.2 3 rd Edn, Random House Walker, G (1997) Public relations practitioners’ use of research, measurement and evaluation, Australian Journal of Communication

24

Watson T (2005) ROI or evidence-based PR: the language of public relations evaluation. PRism

3

, http://praxis.massey.ac.nz/issue_3.html

(forthcoming) (2) Watson T & Noble P (2005) Evaluating PR, Kogan Page, London Watson T & Simmons P (2004) Public Relations Evaluation – Survey of Australian Practitioners, ANZCA04, July 2004

Web sources

 www.instituteforpr.com

 www.carma.com.au

Up and away!

Key thinking

“PR is concerned with ill-defined problems. It cannot be evaluated by a single method or metric because it does not deal with simple issues …” “Likewise, business in general recognises that a simple, single financial measure does not give an accurate reflection of a company’s true worth.” Gregory & Watson 2006: 1

Policy and resources

 Research in the UK sponsored by CIPR  Produced by Henley Management College  Looked at practitioner and academic literature.

CIPR policy 1

 PR is part of the management task and is subject to the same disciplines, such as the need to set direction, allocate and manage resources, and monitor progress.

CIPR policy 2

 Measurement and evaluation are problematic in all areas of management. Complexity is a key factor and in … business, it is difficult to separate out the effect of one area of management such as public relations.

CIPR policy 3

 However, the situation can be eased by better planning and objective setting where precise, measurable objectives allow for enlightened judgements of progress in campaigns and their management.

CIPR policy 4

 PR can be measured and evaluated in terms of:  Contribution to social and economic development  Contribution to management, leadership and organisational performance by aiding better decision making and avoiding mistakes  Being a process and part of programme development and implementation  The contribution of individual practitioners

CIPR policy 5

 Available methods, research based, provide information that is good enough for decision making for PR programmes  Existing methods … are adequate for measuring the contribution of PR.

 Each method has limitations, so a raft of measures appropriate to particular situations needs to be employed.

The CIPR resources

 Go to www.cipr.co.uk/research

Most Admired Companies

 A range of evaluation metrics were used – between four and eight.

 Most frequent were informal and/or qualitative such as journalist feedback and discussions with stakeholders.

 “ ‘Just checking’ everything is on track”.

Gregory, Morgan & Kelly 2005