Network Analysis Methods for Assessment & Measurement

January 14, 2012

Patti Anklam

With June Holley and Claire Reinelt

Webinar Goals

Share current thinking about how network analysis is used in designing and evaluating nonprofit programs

Provide examples of network analysis used in assessment and measurement contexts

Stimulate thinking about correlating network analysis with measurement and evaluation outputs and outcomes

2

Network Thinking & Non-Profits

The Evolution of Network Thinking

4

What is Network Analysis?

• Social network analysis (SNA) is a collection of techniques, tools, and methods to map and measure the relationships among people and organizations

• Organizational network analysis (ONA) often refers to the use of SNA methods in the context of organization dynamics and development

• In practice, we use these tools to map connections among people and ideas, issues, and other entities as well as the social and organizational connections

5

Network Analysis: The Method in a Nutshell

6

Step Activities/Tools

Design Identify boundaries Clarify and design questions

Collect Data Surveys Interviews Facebook, LinkedIn Email logs

Analyze data to generate maps and metrics

(Netdraw/UCINET, NodeXL, Gephi … many others)

Review data Validate; look for questions

Prepare evaluation Match network results with context and stories

Move into action Weaving & other interventions

Survey Example

7

Survey Example – Demographic Component

8

Survey Example – Affiliation Component

9

Survey Example – Network Questions

10

Network Questions Probe Relationships

11

Analyses Outputs: Map Patterns

Multi-Hub Hub and Spoke

Stove-piped (Siloed) Core/Periphery

12

Quick View: What an Analysis Can Tell

13

• Overall very well connected

• One region distinctly clustered with few connects to other regions

• Staff are highly central

• Identification of key connectors

Reasons for a Network Analysis: Examples

1. Assessment, Planning, & Weaving

2. Measure changes over time

3. Sense-making & story-finding

4. Positioning and working with individuals in the network

14

Assessment, Planning, & Weaving

• Assess the network’s capacity for collaboration, information transfer, innovation

• Identify key individuals

• Establish goals for enhancing connectivity

• Create an action plan

Strategic Purpose

15

Assessment: Capacity for Collaboration

Source: Transcending Boundaries: Strengthening Impact. The Full Potential of a Justice Network (Research & Network-Building Project Report, April 2011, Criminal Justice Funders Network). Courtesy of June Holley.

When funders indicate with whom they would like to work in the near future, the network becomes more robust. Funders are saying they want to work more together.

Current Funder Interaction Network Future Funder Interaction Network

16

Assessment: Affiliation Network

• Identify potential relationships among people based on shared events, meetings, ideas, or areas of expertise

• Nonprofits use this to see which organizations “attach” to different ideas

• Forms the basis for network weaving

Strategic Purpose

17

Drill Down Into Affiliation Network

18

• Identify people with common interest – basis for building communities of practice

• See which people share interest in multiple issues or topics

• A way for the network to reveal itself and have rich conversations

Measuring Changes Over Time

1

2

3

4

5

67

8

9

10

1112

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

Very Well

Well

Somewhat

2005

Maps copyright © 2012 New Directions Collaborative Source: Boston Green & Healthy Building Network, Beth Tener and Al Nierenberg, January 2008

19

1

2

3

4

5

67

8

9

10

1112

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

2007

Boston Green & Healthy Building Network

• Look at the whole network and its components:

– Overall cohesion

– Degrees of separation

• Good for comparing groups within networks or for comparing changes in a network over time

Analyses Outputs: Metrics

• Look at positions of individuals in the network:

– # of connections

– Favorability of position

• Good for identifying people who are well positioned to influence the network or to move information around

Individual position metrics Overall network metrics

20

How the Metrics Enhance the Maps

2010

2011 Year # Density Avg #

ties

2009 55 2.2% 1.2

2010 90 2.7% 2.4

2011 85 5.3% 4.5

2012 82 8% 6.88

2009

2012

21

Sense-Making & Emergence

• Barr Foundation Fellows Program

– See changes over time, but really to see how the network has supported emergence

– Work to shift Barr staff from the center

Pat Brandes

Source: Networking a City, Marianne Hughes & Didi Goldenhar, Stanford Social Innovation Review, Summer 2012

22

Sense-Making: New School Development in Boston

• An intentional network may have no other purpose than to enable emergence

• Maps that show the evolving relationships within a network help to identify powerful network stories

23

“This person has helped me accomplish work-related tasks.”

Source: Networking a City, Stanford Social Innovation Review, Summer 2012

Positioning: The Individual View

24

Node Betweenness Indegree OutDegree

62 792.67 26 30

80 660.48 17 32

64 530.61 20 33

23 333.36 20 14

71 321.42 21 20

56 316.42 20 18

• Centrality metrics identify people with the most ties (in-degree and out-degree)

• Those positioned to move information around in the network or be in the know (betweenness)

• Can identify people to lead task teams, to provide resources to, or to train as weavers

Tracking Individuals’ Changes

25

I learned something from this person that made me a better leader. – 2009

2005

2007

2008

2009

2010

Tracking Individuals’ Changes

26

I learned something from this person that made me a better leader. – 2011

Network Analysis & Measuring Outcomes

Summary – What We Know

• Measure the cohesion of the network overall:

– High-level structure (stove-piped, core/periphery, highly clustered)

– Average degree of separation

– Average number of connections each person has

• Identify individuals by their centrality to the network:

– Core or periphery? How do you bring people in from the outside?

– Broker? Connector? Facilitator? Bottleneck?

– Number and diversity of connections

• See changes over time

28

What We Can Measure and Show in an Analysis:

Things We Can Do With What We Know

Ways to change patterns in

networks

Practices from the KM/OD Repertoire

Weaving. Create intentional connections

Convene. Make introductions through meetings and webinars, face-to-face events

Increase the flow of knowledge Establish collaborative workspaces, install instant messaging systems, make existing knowledge bases more accessible and usable; implement social software or social network software

Create awareness Provide expertise directories

Connect disconnected clusters Weave: establish knowledge brokering roles; expand communication channels

Create more trusted relationships Assign people to work on projects together

Alter the behavior of individual nodes Create awareness of the impact of an individual’s place in a network; foster network literacy

Increase diversity Add nodes; connect and create networks; encourage people to bring knowledge in from their networks in the world

29

Measurement Challenges

• Maps area snapshot in time

• Targets and thresholds

– How much cohesion is “enough?” Is there a point at which increasing the number of ties makes the network less efficient?

– Is it reasonable to set a target for the cohesion metric?

• Tying Network Metrics to Outcomes

– We have to think of the metrics as indicators and as correlates of other survey questions

30

Source: Dave Snowden, Cynefin Advanced Practitioner’s Course December 2012

Questions?

31

Question

• patti@pattianklam.com http://www.pattianklam.com

•claire@leadershiplearning.org http://www.leadershiplearning.org/

•June@networkweaving.com http://www.networkweaving.com

Thank you.