WPAS Are We Engaged Yet

People talk. We listen.

Are we engaged yet? A framework for measuring engagementA presentation to WPAACrystal City, VAMay 2009 Katie Delahaye PaineCEOKDPaine & Partners, LLCDurham & Berlin, NHkdpaine.blogs.comwww.measuresofsuccess.com


Eyeball

counting

HITS

Engagement

MSM

Online

Social

Media

A measurement timeline

Page 2


The Social Media Decision TreeDefined Goal

Sales:Memberships

Donations Applications

Traffic/Users

Influence

Web analyt

ics

LeadsLoyalty

Frequency

of visits

LeadsTime on site

Comments

LinksTime on site

Messaging

Visibility

Tonality

Loyalty/

renewal

SalesClickthru


The Engagement Decision TreeAwareness

Consideration

Preference

Trial

Purchase

FindObserve/

LurkParticipate Engagemen

t

Purchase/Act/Link/

WOM

Page 4


A Proposed Engagement Index

ClickthruDonations/

ordersSignups

Time on siteRepeat visits

Forwards/links

/comments

RelationshipsTone/content of

conversationMembership

One Number?

Output Outtake Outcome

+ +

Page 5


Engagement Indices

Forrester: Involvement

Website visitsTime spentPage views

InteractionCommentsReviews

IntimacySentimentPositioning

InfluenceLikelihood to recommendBrand affinityForwardsLinks

Others: Stowe Boyd:

Posts/commentsNielsen – time on

siteQuantcast –

Frequency of return

Comscore – visits per visitor

Eric Petersen

Page 6


What do you need to measure?

Outputs?Did you get the coverage you wanted?

Did you produce the promised materials on time and on budget?

Outtakes?Did your target audience see the messages?

Did they believe the messages?

Outcomes?Did audience behavior change?

Did the right people show up?

Did your relationship change?

Did sales increase?

7


Goals, Actions and Metrics Goal Action Output

Metric Outtake Metric

Outcome Metric

Recruitment

Twitter campaign

Number of tweets

% inclined to support% improvement in perception of organization

% increase in online requests for information % increase traffic to recruitment site

Increase in support

Flickr photo contest

Number of entrants

% likely to support the cause

Improvement in relationship scores

Message consistency

Blogger/Speaker training

% postings/articles containing one or more key messages

Share of positioning on key issues

% hearing message% believing message

% consistency in messaging between external and internal communicationsMarket shareRatio of on-message to off-message quotes

8


Metrics that show engagement with your own site

% increase or decrease in unique visits Change in page rank - i.e a list of the top ten most popular areas and how it has changed in the last week How many sessions on our blog or web site represent more than 5 page views In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email address

Courtesy of Eric Peterson

Page 9


Metrics that show engagement with other sites

Conversation IndexLinksForwardsDigg, StumbleuponFacebook, Twitter, Pownce, Utterz, SeesmicCommentsRatingsRankingTime between postsTone of discussion

Page 10


Share of conversation vs share of engagement

Page 11

2

2

1

2

1

6

5

3

1

1

1

1

1

1

1

1

1

1

2

2

2

1

2

1

2

1

4

2

1

4

2

1

1

4

1

6

7

6

2

2

2

2

1

3

2

3

1

0 2 4 6 8 10 12 14 16 18 20

Faculty

Students

Research, Physical Sciences

Courses

Research, Earth Sciences

Projects, Non - Research

Financials

Alumni Topics

Research, Life Sciences

Staff

Admissions

Legal News

Other

Research, Agriculture

Policies

Institution, Overall

Campus Life

Research, Social Sciences

Share of Subject

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4

15.3%

68.7%

100.0%

4.4%

33.3%

96.8%

28.6%

34.9%

12.5%

43.3%

28.6%

13.0%

38.3%

100.0%

23.6%

66.7%

6.3%

28.6%

20.8%

2.3%

95.6%

33.2%

5.8%

28.6%

100.0%

86.8%

13.0%

31.0%

22.1%

3.2%

71.4%

43.5%

18.8%

94.2%

56.7%

14.2%

13.2%

53.2%

28.4%

21.1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Admissions

Alumni Topics

Campus Life

Community Relations

Courses

Events

Faculty

Financials

Institution, Overall

Inventions

Legal News

Other

Partnerships

Policies

Projects, Non - Research

Research, Agriculture

Research, Earth Sciences

Research, Life Sciences

Research, Other

Research, Physical Sciences

Research, Social Sciences

Staff

Students

Share of Engagement by Subject - ,External Blogs

Peer 1

Michigan State

Peer 2

Peer 3

Peer 4


The vast majority of discussion in external blogs is neutral.

Page 12

23

29

12

14

20

5

8

4

1

4

0

5

10

15

20

25

30

University of Michigan Purdue University Penn State Michigan State Arizona State

Share of Tone

Negative

Neutral

Positive

71%

3%

29%

94%

83%

42%

58%

6%

14%

58%

42%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Arizona State Michigan State Penn State Purdue University University of Michigan

Share of Engagement by Tone - External Blogs

Negative

Neutral

Positive


For all institutions, most postings were simply making an observation or distributing media.

Page 13

3

6

1

1

7

36

1

29

5

15

14

2

16

1

2

12

7

2

6

2

24

787

3

2

203

12

12

46

11

1

3

2

1

4

1

4

3

6

2

1

13

2

2

1

13

2

6

18

4

1

1

5

35

3

17

2

8

9

1

1

1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing criticism

Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion

Making an observation

Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Conversation Types

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

44.2%

6.5%

30.9%

49.5%

100.0%

100.0%

100.0%

1.6%

53.9%

100.0%

26.9%

23.1%

10.8%

38.7%

72.7%

10.9%

15.5%

46.1%

66.6%

27.3%

35.1%

39.7%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Acknowledging receipt of information

Advertising Something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media


Expressing support

Expressing surprise

Giving a heads-up

Giving a shout-out

Making a suggestion


Offering an opinion

Playing a game

Rallying support

Recruiting people

Showing dismay

Share of Engagement by Conversation Type - Institutional Blogs

Arizona State

Michigan State

Penn State

Purdue University

University of Michigan

cx


Aspects of relationships

Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship

14


Components of a Relationship Index

Control mutuality In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.

TrustThis organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it will do.

SatisfactionGenerally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.

CommitmentThere is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization more

Exchange relationshipEven though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.

Communal relationshipThis organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other people. (Reversed)

15


How to implement relationship metrics

Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed


Data mining the numbers you have

Look for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, last yearFigure out what worked and what didn’t work


Best Practices:

Engagement leads to bottom-line impact

ASPCA: Donations Memberships

War Child: 38% increase in

donations 300 requests to be

voluntersNWA:

65000 new wildlife spotted

Using SMM engagement for planning

Define the market/topic you want to focus onDefine the bloggers that influence the marketplace

Benchmarking against your peers

Looking at what the best doSet goals accordinglyUse data to persuade the recalcitrant

Social Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, image or reputationCommunicate directly to people who need to hear you


Correlation exists between traffic to the ASPCA web site and the organization’s

overall media exposure

-

100,000

200,000

300,000

400,000

500,000

600,000

700,000

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Web

Sit

e Vi

sito

rs

Expo

sure

Overall Exposure

Web Traffic


Tying activity to development/marketing goals

0

50,000,000

100,000,000

150,000,000

200,000,000

250,000,000

300,000,000

350,000,000

Exposure

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

$1,400,000

$1,600,000

$1,800,000

Donations

Overall exposure

Online donations

20


Case study USO


Anatomy of a perfect measurement program

Start with message map Create measurement map Start measuring traditional media Establish relationship and awareness benchmark Start measuring social media Conduct social media program Measure relationships again

People talk. We listen. 23

42

15 7

45

12

67

6

88

17

5

30 33 45

34

116

33

22

12

43

9

7

12

-

20

40

60

80

100

120

140

Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar

2008 2009

Opp

ortu

nitie

s to

See

in M

illio

nsKey Message Communication Over Time

Connect troops with home Delivers comforts of home Enable meaningful involvement

Good business practices Home away from home


10 15 21 15 7

35 21

12 12

12 20

27 13 14

26 47

8

13 8

24 12

13

20

15 53

4

4

47 51

15

12

42 32

44

126

16 23

112

25

18

21

28 13

15

62 112

47

-

20

40

60

80

100

120

140

160

180

Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar

2008 2009

Opp

ortu

nitie

s to

See

in M

illio

nsExposure Over Time by Subject

Centers Corporate Partners Development Donations

Historical Operations Organizational Programs

Thought Leadership Tours Volunteers


Key Indicators Over Time


Conversation Type January February March

Making an observation 143 152 6

Expressing support 20 40 42

Offering an opinion 18 1 0

Advertising something 10 0 0

Rallying support 4 4 1

Distributing media 4 2 0

Making a suggestion 2 3 0

Calling for action 1 2 0

Giving a shout-out 1 2 0

Asking a question 1 2 0

Disclosing personal information 1 1 0

Recruiting people 1 0 0

Putting out a wanted ad 0 1 0

Expressing criticism 0 1 1

Expressing agreement 0 1 0

Answering a question 1 0 0

Expressing surprise 0 1 0

Grand Total 207 213 50 0 10 20 30 40 50


Rallying support


Expressing support

1

37

Mentions

Conversation Type by ToneMarch 2009

Positive

Neutral

Negative

0 10 20 30 40 50


Rallying support


Expressing support

4

1

1

2

42

Mentions

Conversation Type by Message SaturationMarch 2009

Contains no message

Contains 1+ Messages


Tonality of social media towards USO over time

27

63 6138

143 151

11

0

50

100

150

200

250

Jan Feb Mar

2009

Me

nti

on

s

Tonality Over Time

Positive Neutral Negative


130 137

43

1215 12

42 55

7

0

50

100

150

200

250

Jan Feb Mar

2009

Men

tion

sMentions by Subject Over Time

Entertainment Fund raising Local USO Military Support Organizational

What people are talking about


Social Media OverviewMarch 2009

29

131

47

4

76

166

46

0

50

100

150

200

250

Jan Feb Mar

2009

Men

tion

s

Key Message Saturation Over Time

Contains no message Contains one or more full messages

5

67

149

45

413

0

20

40

60

80

100

120

140

160

180

Jan Feb Mar

2009

Men

tion

s

Key Message Communication Over Time

Connect troops with home Delivers comforts of home Enable meaningful involvement

Good business practices Home away from home


For more information on measurement, check out our blog: kdpaine.blogs.com orSubscribe for free to The Measurement Standard, www.themeasurementstandard.comTo start developing your own dashboard or for a copy of this presentation go to: http://www.kdpaine.comOr call me at 1-603-868-1550Or give us your business card and we'll be happy to send it to you

Thank you

Page 30

http://www.themeasurementstandard.com/

http://www.kdpaine.com/

Business

WPAS Are We Engaged Yet