Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC Professor of Public Communication

Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC

University of Technology Sydney

Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC

Professor of Public CommunicationUniversity of Technology Sydney

Where are we? Where to next?



This discussionThis discussion A quick update on international standards Continuing gaps and inconsistencies Some further barriers to overcome



Barcelona Declaration 2010 Barcelona Declaration 2010 Goal setting and measurement are fundamental aspects of any

PR program Measuring outcomes is preferred to measuring outputs The effect on business [organisation] results should be

measured where possible Media measurement requires quantity and quality Advertising value equivalents (AVEs) are not the value of PR Social media can and should be measured Transparency and replicability are paramount



International standards pushInternational standards push Coalition for Public Relations Research Standards

• Established 2011 by AMEC, IPR and CPRF Social Media Standards Conclave (#SMMStandards)

• Established 2012 by Coalition + 8 other PR/communication organizations– Global Alliance, PRSA, CIPR, IABC, SNCR, FIBEP, WOMMA, DAA

• Working with eight companies– Dell, Ford, General Motors, McDonalds, Proctor & Gamble, Thompson

Reuters, South-West Airlines, SAS

• Also consulting with – Media Ratings Council, AAAA, ANA, Web Analytics Association

Measurement Summit discussions 2011–2013 Academic input?





M&E outputs and guidesM&E outputs and guides Proposed Interim Standards for Metrics in Traditional Media

Analysis Social Media Standard Definitions for Reach and Impressions Sources and Methods Transparency Table

Institute for Public Relations (IPR)

Association for Measurement and Evaluation of Communication (AMEC)



• Message recall/ retention• Consideration• Responses• Feedback• Engagement

Evaluated against objectivesCommunication

Objectives

PR Activity Undertaken

PR Outputs Analysed

PR Outcomes Analysed

For example:

• Media relations• Publications sent• Events held• Sponsorships undertaken• Communication engaged in

Organisation/Business Results

Organisation/Business Objectives

Did activities generate outputs that reached publics

effectively and efficiently?

• Audience reach• Tone/sentiment of communication• Key messages• Share of voice• Competitor bench- marking

What outtakes and outcomes were achieved among key publics and to what extent?

• Awareness• Understanding• Consideration • Attitudes/perceptions• Action/behaviour• Relationships• Reputation

• Brand positioning• Sales/profits• Market share• Share price• Donations/grants• Customer/staff retention

PRIA Evaluation Model

MonitoringMonitoring MeasurementMeasurement ReportingReporting

Formative ResearchFormative Research

Did organisation/business results shift as a consequence?

Evaluative ResearchEvaluative Research

Q:Q:

PR Outtakes Analysed




How far we’ve come …How far we’ve come …



“… REACH represents the total number of unique people who had an opportunity to see an item”

#SMMStandards, 2012, para. 15.

IMPRESSIONS– the number of people having the opportunity for exposure to a media story; also known as ‘opportunity to see’ (OTS) … usually refers to audited circulation”

Proposed Interim Standards for Metrics in Traditional Media Analysis (Eisenmann et al., 2012, p. 3)

Impressions – “the number of people who might have had the opportunity to be exposed to a story”

Dictionary of Public Relations Measurement and Research (Stacks & Bowen, 2013, p. 14)

“Impressions represent the number of times an item was displayed”

“Impressions represent the gross number of items that could have been seen by all people, including repeats”

Social Media Standards Definitions: Reach and Impressions (Digital Analytics Association, 2013)

Impressions represents “the number of possible exposures of a media item to a defined set of stakeholders” …

Dictionary of Public Relations Measurement and Research (Stacks & Bowen, 2013, p. 14)

Definitions – e.g., ‘reach’ and ‘impressions’Definitions – e.g., ‘reach’ and ‘impressions’



Return on Investment (ROI) PR ROI Return on Impressions (ROI) Return on Media Investment (ROMI) Return on Target Influence (ROTI) Return on Earned Media (ROEM) Return on Expectations (ROE) Return on Communication Investment (ROCI) Return on Brand Investment (ROBI) Social media ROI (SROI)



MetricsMetricsBasic outputs Outputs Outtakes Outtakes Outcomes

Counting press clippings Unique visitors Engagement

Audience Views Influence

Reach Likes Impact

Target audience reach Followers Awareness

Impressions Fans Attitudes

Opportunities to see (OTS) Clickthroughs Trust

Share of voice Downloads Loyalty

Cost per thousand (CPM) Comments Reputation

Hits Tone Relationships

Visits Sentiment ROI?



• Message recall/ retention• Consideration• Responses• Feedback• Engagement

Evaluated against objectivesCommunication

Objectives

PR Activity Undertaken

PR Outputs Analysed

PR Outcomes Analysed

For example:

• Media relations• Publications sent• Events held• Sponsorships undertaken• Communication engaged in

Organisation/Business Results

Organisation/Business Objectives

Did activities generate outputs that reached publics

effectively and efficiently?

• Audience reach• Tone/sentiment of communication• Key messages• Share of voice• Competitor bench- marking

What outtakes and outcomes were achieved among key publics and to what extent?

• Awareness• Understanding• Consideration • Attitudes/perceptions• Action/behaviour• Relationships• Reputation

• Brand positioning• Sales/profits• Market share• Share price• Donations/grants• Customer/staff retention

PRIA Evaluation Model

MonitoringMonitoring MeasurementMeasurement ReportingReporting

Formative ResearchFormative Research

Did organisation/business results shift as a consequence?


Q:Q:

PR Outtakes Analysed




The so-called barriersThe so-called barriers Cost Time Too hard Management don’t want it



Major barriers to addressMajor barriers to address Conflation of measurement and evaluation

Measurement is taking measures – metrics Evaluation is determining VALUE Value = a perception from a perspective

Measurement Evaluation

Early findings used to fine-tune campaign

Data collection Data analysis

Findings primarily used for reporting



Major barriers to addressMajor barriers to address Most measurement and evaluation look backwards

• “Tells us what we know intuitively”• “What’s done is done … can’t change the past”• “Post-rationalisation and self-justification”

Look forwards• What insights can improve PR/communication/relationships in future?• What insights can contribute to the organisation’s strategy and

outcomes?– Gaps– Trends– Opportunities– Competitor strategy– Emergent issues



Obsession with numbersObsession with numbers



ConclusionsConclusions Recognise measurement and evaluation as different and

separate• One is about metrics and analytics, the other is about VALUE

Collect qualitative research data as well as quantitative• Noting that many outtakes, outcomes and outflows are perceptions and

attitudes (e.g., satisfaction, trust, reputation) and relationships Look forward, not simply reporting retrospectively

• Contribute to organisational strategy and outcomes





References/resourcesReferences/resources Interim Standards for Metrics in Traditional Media Analysis

http://www.instituteforpr.org/topics/proposed-interim-standards-for-metrics-in-traditional-media-analysis

Social Media Standards ‘Sources and Methods Transparency Table’http://www.smmstandards.com/category/content-sourcing-methods/ ‘Social Media Standard Definitions for Reach and Impressions’,

http://www.smmstandards.com/2013/03/proposed-social-media-standard-definitions-for-reach-and-impressions-from-the-digital-analytics-association/. Also at http://www.smmstandards.com/wp-content/uploads/2013/03/SMM-Standard-Definitions_DAA_v4.pdf

Jim Macnamara, (2014), ‘The development of international standards for measurement and evaluation of public relations and corporate communication: A review’, available at http://www.uts.edu.au/research-and-teaching/our-research/australian-centre-public-communication/research/research-reports

Tom Watson & Anzgar Zerfass, (2011), ‘Return on investment in public relations: A critique of concepts used by practitioners from communication and management sciences perspectives’, PRism, vol. 8, no. 1, available at http://www.prismjournal.org/vol8_1.html

Documents

Professor Jim Macnamara PhD, FAMI, CPM, FPRIA, FAMEC University of Technology Sydney Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC Professor of Public Communication