A CROSS-CASE

ANALYSIS OF 44

RESEARCH BROKERING

ORGANIZATIONS

ACROSS CANADA

Dr. Amanda Cooper

April 12, 2013

CONTEXT

Multiple Definitions by Sector

GETTING THE RIGHT

INFORMATION TO THE RIGHT PEOPLE IN THE RIGHT FORMAT AT THE RIGHT TIME SO AS TO

INFLUENCE DECISION-

MAKING (ONF)

MOVING KNOWLEDGE INTO ACTIVE SERVICE FOR

THE BROADEST POSSIBLE COMMON

GOOD (SSHRC)

THE PROCESS FROM THE

CREATION OF EVIDENCE TO ITS ULTIMATE

IMPACT

(Knowledge Translation,

CIHR)

COLLABORATIVE PROBLEM-SOLVING

BETWEEN RESEARCHERS AND DECISION MAKERS THAT

HAPPENS THROUGH

LINKAGE AND EXCHANGE (Knowledge

Exchange, CHSRF)

We prefer the term ‘knowledge mobilization’,

because it best embodies the idea that the use of knowledge is a social process, not just an intellectual

task, and as such is multidirectional, not just a matter

of moving information from those that know to those that do not.

At the same time, ‘mobilization’ implies effort and direction, not just random interaction. There are

multiple, iterative phrases including the generation of

new research knowledge when needed, the communication and application of established

research knowledge, and the contextualization of

research to suit particular environments (Cooper et al., 2009, p.166-167)

TIME

SOCIAL CONTEXT

RESEARCH USE

RESEARCH PRODUCTION

Knowledge Mobilization

INTERMEDIARIES/

Research Brokering Organizations (RBOs)

KMb

Rising number of intermediary organizations Significant role: interpret, package, disseminate Underexplored role, virtually no empirical work

RBOs: THE MISSING LINK?

SIG

NIF

ICA

NC

E

RESEARCH QUESTIONS

What is the nature and impact of the work of Canadian RBOs in knowledge mobilization in education?

1. What types of intermediary RBOs exist in education across Canada?

2. What are the organizational features of RBOs?

3. What knowledge mobilization processes are RBOs involved in, and what dissemination mechanisms do they use?

Intermediaries (RBOs)

CONCEPTUAL FRAMEWORK

TIME

SOCIAL CONTEXT

RESEARCH USE

RESEARCH PRODUCTION

1. TYPE OF ORGANIZATION Gov’t Non-Profit For Profit Membership

3. KMb PROCESSES

Message Strategies Functions

Dissemination Mechanism

2. ORGANIZATIONAL FEATURES

Mission Scope Target Audience

Size Resources Membership Composition

Methodology

Challenge of identifying RBOs

Inclusion Criteria

1. Target audience: Connects Producers AND Users

2. Mission Statement: Explicitly KMb Related

Sampling Strategies

Ki-es-Ki Handbook, 4000+ educational contacts, Canadian

Education Association

Systematic Web Search (record search strings)

RSPE program

Sampling

541 Potential Organizations

479 Excluded

18 Meet

Criteria #1

44 RBOs Meet Both Inclusion

Criteria

-24 excluded because

no website

-67 excluded because

they were French

-388 do not meet

criteria 1 and 2

A Matrix To Measure KMb Efforts

STRATEGIES

INDICATORS

Products Events Networks Extra

Strategies

INDICATOR

TOTAL

Different types 3 6 6 5 /20 Ease of use 2 4 2 2 4 /14 Accessibility 3 6 4 3 /16 Audience Focus 4 2 4 /10 Extra Indicators 4 8 /12 STRATEGY TOTAL /12 /20 /20 /20 /72

• Interrater Reliability Testing of Tool: Intraclass Correlation= .799

• 3 raters per organization for 9 orgs (20% of RBOs Sample)

0

10

20

30

40

50

60

70

80

90

Types

Ease o

f U

se

Accessib

ility

Audie

nce F

ocus

Oth

er

Indic

ato

rs

Pro

ducts

Events

Netw

ork

s

Oth

er

Str

ate

gie

s

TO

TA

L

INDICATORS STRATEGIES

Mea

n S

co

re (

%)

Matrix Categories

KMb Efforts by Organization (N=105)

RBOs

Faculties of Education (N=21)

School Districts (N=14)

Ministries of Education (N=26)

Kruskal-Wallis Test was significant:

χ2 (3, N=105) = 42.31, p= .00

- There are significant differences between these groups

Pairwise Comparison reveals:

- RBOs & Faculties of Education perform similarly

- School Districts & Ministries perform similarly

0 5 10 15 20 25

0 to 10

10 to 20

20 to 30

30 to 40

40 to 50

50 to 60

60 to 70

70 to 80

80 to 90

90 to 100

Frequency (Number of Organizations)

Sco

re o

n K

Mb

Mat

rix

(%)

Histogram: Total Scores by Type of Organization (N=105)

RBOs

Faculties of Education (N=21)

School Districts (N=14)

Ministries (N= 26)

Types of Canadian RBOs

1. What types of intermediary RBOs exist in education in Canada?

Intermediary RBOs

Governmental

Ministry Research Branches

District Level Research Services

Standard & Evaluation

Funding Agencies

Not for Profit

University Research Centres

Advocacy

Issue

Based

Think

Tanks

For Profit

Textbook publishers, Instructional

program vendors

Research Consulting Companies

Media

Membership

Professional

Network

44 RBOs ACROSS CANADA

1980

2000

•LDAC

•LDAS

•LDAO

•CMEC

4

19

6

10

3

2 Pre- 1960s

•CEA

•CD Howe

•AERO

•FRASER

•HANEN

•LitBC

•CODE

•TLP

•AIMS

•CPRN

•P4E

•SAEE

•EQAO

•Galileo

•MCLE

•CCPA

•CCBR

•CRRU

•LCBN

•LDANS

•SK Lit

•LEARN

•CEED

•HELP

•ORION

•MERN

•SQE

•Harris

•E-Best

•CCKM

•CSC

•CCL

•RSPE

•E-Best

•EYEON

•PREVNet

•SKE-EDU

•ERESB •The Centre

•Research Impact

1990

1970

1960 43% of the sample has

arisen in the past decade

9% of the sample

closed, lasting

between 6-15 years

Rise of RBOs

Organizational Features

2. What are the organizational features of RBOs?

Organizational Features

Target Audience & Membership Composition

Heterogeneous, homogeneous, no members

Scope

Local (n=2), Provincial (n=29), National (n=13)

Most RBOs are small (63%)

RBO Size (FTE) n Min Max Mean Mode

Small (1-10 FTE) 26 1 10 5 3

Medium (11-19 FTE) 4 11 14 12 11

Large (20+ FTE) 13 20 77 41 30

Rank RBO Size (FTE) Budget Score % 1 1.2.1 RI Small (3) $250 000 81

1.2.4 Fraser Large (60) $12,808,690 81 1.4.2 CEA Small (9) $2,044,892 81

2 1.2.4 AIMS Small (5) $872 234 78 3 1.2.0 CCL Large (77) $20,583,490 76

1.2.3 The Centre Large (25) $5,685,000 76 4 1.2.0 TLP Large (74) $5,293,039 75

1.2.1 HC Med (11) --- 75 5 1.2.0 CCBR Med (12) --- 74 6 1.1.2 E-BEST Small (6.5) --- 72

Top 10 RBOs

Resources do not necessarily imply stronger

KMb efforts

3. What knowledge mobilization processes are RBOs involved in, and what dissemination mechanisms do they use?

KMb PROCESSES

RESEARCH

PRODUCTS

CAPACITY

BUILDING

NON

RESEARCH

EVENTS NETWORK MEDIA

Reports Glossaries Strategic Plan Panel/ Talk E-Bulletins Press

Release Summaries Research FAQs Editorials Conference Network

Push

Lit Reviews Toolkits Promotional

Materials

Annual

Meetings

Directories of

Contacts

Newspaper

Systematic Online Tutorials Advocacy Workshop Social Media Radio

Conceptual Research

Support Services

Annual

Report

Awards

Ceremonies

Online Forum TV

Reference Lists Blog

Policy Briefs

Fact Sheets

Success Stories

Multimedia

KMb Strategies

Comparing High, Med, Low

RBOs vary widely in levels & kinds of KMb

activities(product strategies vs. events, networks, and

media)

Think tanks use far more media strategies

0

100

200

300

400

500

CEA CCL SAEE SQE LDAO

Media

Networks

Events

Capacity Building

Research Products

HIGH KMb EFFORTS LOW

Think tanks usually use

MORE media strategies

0 5 10 15 20 25 30 35 40 45

Linked In

Flickr

Delicious

YouTube Channel

Blog

Online Forum

RSS Feed

Share button

Twitter

Facebook

Website

Frequency (Number of RBOs)

On

lin

e S

tra

teg

y

Blogging, Microblogging (One/Two-way communication)

Social Networking (Two-way communication)

Social Bookmarking (Collaborative tool)

Multimedia (Depends on active user)

Passive (Depends on active user)

Push (One way communication)

Use of Online Strategies

Twitter: Small networks, Low use

30% of RBOs use social media

RBO T

ota

l T

wee

ts

F

oll

ow

ing

F

oll

ow

ers

T

ota

l T

wee

ts

(Sep

t-D

ec 2

010)

Aver

age

Tw

eets

per

mon

th

S

D T

wee

ts p

er

mon

th

Mean 416 344 905.5 149.29 34 9.719

Max 2250 2326 4913 594 149 26.3

Min 7 1 11 0 0 0

SD 612 589 1309 167.26 41 7.748

0 100 200 300 400 500 600

PREVNet

CEA

LCNB

CCPA

AIMS

HC

SQE

LDAC

P4E

LiteracyBC

TLP

E-Best

SSHRC

Fraser

RI

Total Tweets (Sept 2010 – Dec 2010)

RB

O

Frequency of Twitter Usage

Social Media: Nature of Posts

Opinion:

“Are parents really partners in education? Should they be?”

Promotion:

“Last chance to register for our conference”

Information:

“Premier’s arts awards tonight”

Update:

“Just watched waiting for Superman”.

Research Based:

“StatsCan: Canadian drop out rate declining. Drop out

highest in AB, MB, QC, lowest in NL, BC, ON

http://ning.it/a2auTU”

Opinion 21%

Promotion 14%

Information 27%

Update 4%

Research Based 24%

Discussion: RBOs

Diverse RBOs exist at various levels of

education

RBOs vary widely in size, levels of KMb efforts

and dedicated resources

Dedicated resources do not necessarily imply

stronger KMb efforts

Few organizations have comprehensive KMb

strategies or systematic approaches

Social media emerging but not focused use

Future Research

KMb Metric provides a preliminary approach,

but should be coupled with in-depth case

studies

Extend to RBOs in other countries & sectors

More work (& larger samples) needed to

differentiate efforts of diverse types of RBOs

Further exploration of relationships between

organizational features and KMb efforts

Comparing effectiveness of different strategies

is needed to assess impact of intermediary work

Thank-you!

Email: amanda.cooper@queensu.ca

Twitter: @ACooperKMb

Website: www.amandacooper.ca

Former Work-

Research Supporting Practice in

Education Website:

www.oise.utoronto.ca/rspe