Concept Mapping · model development •State level planning •Evaluation, using repeated measures. State Level Planning Example Planning for new programs, expansions, and budgets

Concept MappingConcept MappingConcept MappingA Next Generation Multi-

Use Strategy

The Research & Training Centerfor Children’s Mental Health

22nd Annual Research ConferenceMarch 4, 2009

PresentersPresentersPresenters

Lenore B. Behar, Director Child & Family Program Strategies

www.lenorebehar.com

William M. Hydaker, Director Hydaker Community Consulting

PresentersPresenters

Robert Paulson, (ret.) Professor Dep’t of Child and Family Studies

Louis de la Parte Florida MentalHealth Institute

University of South Florida

Intensive Workshop OverviewIntensive Workshop OverviewIntensive Workshop Overview

• Concept Mapping Overview• Examples from the Field• Break (10 minutes)• More Examples from the Field• Readiness Study and Community

Readiness Assessment• Discussion & Wrap UP

Concept Systems, Inc.Concept Systems, IncConcept Systems, Inc.

All projects discussed today use

• the methods and software developed byConcept Systems, Inc., Ithaca, NY.

• the Concept System© software: Copyright2004-2007; all rights reserved. ConceptSystems Inc.

Why Concept Systems?Why Concept Systems?Why Concept Systems? This system offers a next generation

approach to data collection andmanagement

• Moved concept mapping beyond focusgroup model

• Added web-based method to collect data• Added statistical analyses, formerly

subjective interpretation• Added graphic presentations of finding• Is flexible, applicable to many situations

Overview of Concept Mapping

from Concept Systems, Inc.

Overview of Concept MappingOverview of Concept Mapping

from Concept Systems, Inc.from Concept Systems, Inc.

• Why Concept Mapping?• Multiple Uses• Multiple Methods• Process and Products

Why Concept Mapping ?Why Concept Mapping ?

DefinitionDefinitionDefinition• A structured process used to

organize the ideas of a group thatwishes to develop framework forplanning and evaluation.

• The process is used to developpictorial representations of theideas generated by the group andthe relationships of these ideas toeach other.

Definition--MoreDefinition--MoreDefinition--More• The participants express both

their individual ideas andinteract (face-to-face orvirtually) with the entire group.

• The input is analyzed statisticallyand the findings are notsubjective.

AdvantagesAdvantagesAdvantages• Time efficient

• Easy to understand; no jargon

• Effective engagement strategy

• All participants have equal voice

• Supports ownership; empowersaction

Advantages--MoreAdvantages--More

• Allows for assessing multipledimensions, such as importanceand feasibility

• Grounded by statistical analyses,not subjective interpretation

Uses for Concept MappingUses for Concept MappingUses for Concept Mapping• Community planning for change: to

set goals, for implementationand/or sustainability, for logicmodel development

• State level planning

• Evaluation, using repeated measures

State Level Planning ExampleState Level Planning ExampleState Level Planning Example

Planning for new programs,expansions, and budgets

Quote from a deputy commissioner

It got to the heart of my frustrations over allthe Plans and task forces we used to have inthe Division that took forever and were littlemore than the nominal group method. Ourplanning never took "feasibility" intoconsideration and thus most plans ended up aspipe dreams to be revisited over and over andover again. Concept Mapping provides astructured, valid and much more timeefficient method of identifying actions thatboth the community and the State canembrace, irrespective of resources. Ourplans all just resulted in requests for millionsof $ that had no prayer of passing. It is neatto see this kind of advance in the "science" ofconsensus building.

In Order to Avoid ThisIn Order to Avoid ThisIn Order to Avoid This

And Get Organized Inputfrom

Stakeholders

And Get Organized InputAnd Get Organized Inputfromfrom

Stakeholders Stakeholders

Two Methods for theConcept Mapping ProcessTwo Methods for theTwo Methods for the

Concept Mapping ProcessConcept Mapping Process

• Can be done face-to-face with agroup

• Can be done using a web-basedprogram; participants respondonline

Process – how does it work?Process Process –– how does it work? how does it work?

• Convening a group• Brainstorming• Sorting the responses• Rating the responses• Feedback session & products

Convening a GroupConvening a GroupConvening a Group

Organizers determine theparticipants.

They may include administrators,staff, board members, familymembers, youth, communityleaders and others.

Develop the Focus PromptDevelop the Focus PromptDevelop the Focus Prompt

The Focus Prompt is the triggerfor Brainstorming. It sets theframework for how the groupthinks about the task.

Examples of a Focus PromptExamples of a Focus PromptExamples of a Focus Prompt

• “To develop a system of care inour community, we must havethe following characteristicsand functions …..”

Or

• “To have a system of care in ourcommunity, we must…..”

BrainstormingBrainstorming

• Generate ideas in response to aprompt

Sorting• Sort each idea into groups that

are related to each other

• Label each group

• Generate ideas in response to aprompt

SortingSorting• Sort each idea into groups that

are related to each other

• Label each group

What is Brainstorming?What is Brainstorming?What is Brainstorming?

• Participants generate ideas about the topic of interest

• There are no limits on thenumber of ideas, thoughexcessive numbers may becomecumbersome.

The Brainstorming ProcessThe Brainstorming ProcessThe Brainstorming Process• As a statement is made, it is typed

into the computer and projected ontothe screen.

• Brainstorming is complete when thegroup cannot generate any morestatements.

• Or the process stops at about 100statements.

• The prompt statement sets the task.

SortingSortingSorting• The group returns the next day

• Each statement is printed on anindex card; a full set of cards isprovided to each person.

• Each person puts items intopiles that go together andlabels the piles

RatingRatingRating

• Utilizes a Likert scale• Each respondent rates each

item on two separate dimensions• Each item is rated on a 1 to 5

response scale with 1 equalingstrongly disagree and 5 equalingstrongly agree

ProductsProductsProducts• Point Map/Concept Map from

sorting and labeling.

• Ladder graph from ratings ofimportance and feasibility.

• Go-zone map from ratings ofimportance and feasibility.

How the Data are AnalyzedHow the Data are AnalyzedHow the Data are Analyzed• Data from the brainstorming,

sorting and rating are statisticallyanalyzed, using item analyses,hierarchical cluster analyses andmultidimensional scaling

• Findings are not based on opinionsor subjective interpretations.

Multidimensional ScalingMultidimensional ScalingMultidimensional Scaling• Concept Systems software program

calculates spatial relationships of paireddata from the total square similaritymatrix much as one would position citieson a blank map if the distance betweenthe cities was known.

• Mathematical scaling is done using amultivariate analysis to represent thelocation of all of the statements on atwo dimensional point map.

Multidimensional Scaling, cont’dMultidimensional Scaling, contMultidimensional Scaling, cont’’dd• Points on the map represent individual

statements and their positions relativeto each other.

• Points close to each other areconceptually similar; points that are farapart are conceptually different.

• The point map forms the basis for theconstruction of the cluster map.

Point MapPoint MapPoint Map

1 2

3

4 5

6

7 8

9

10

11

12

13

14

15

16

17

18 19

20 21

22

23

24 25

26

27

28

29

30

31 32

33

34

35

36

37

38

39

40

41 42

43

44

45

46

47

48 49

50

51

52 53

54

55 56

57

58 59

60

61

62 63

64

65

66

67

68

69

70

71

72

73 74

75

76 77

78

79

80

81

82

83

84 85

86

Hierarchical Cluster AnalysisHierarchical Cluster AnalysisHierarchical Cluster Analysis• Individual statements are mapped

showing their relationship to oneanother, forming a point map.

• The point map is grouped into clustersof statements that in theory reflectsimilar concepts, forming a cluster map.

• There can be as many clusters asstatements, and the final number isdetermined by the analyst, with advicefrom the participants.

Hierarchical Cluster Analysis,cont’d

Hierarchical Cluster Analysis,Hierarchical Cluster Analysis,contcont’’dd

• With 100 statements, 20 down to 3clusters are examined.

• As the analysis moves from onecluster level to the next, (e.g., from10 to 9 clusters) the grouping isexamined to determine if it makessense to the overall framework.

Participant InputParticipant InputParticipant Input

Discussion with participantsfocuses on

• Number of cluster solutions

• Names of clusters; programgenerates several options basedon the labels participants gave

Communication/Collaboration

QualityServices

CommunityDevelopment

Training/Evaluation

Five Cluster SolutionFive Cluster SolutionFive Cluster SolutionYouth & FamilyFocus

Quality Services

Training/Evaluation Communication/Collaboration


CulturalIssues

Youth &Family Focus

Six Cluster SolutionSix Cluster SolutionSix Cluster Solution


CulturalIssues

Training/Evaluation

Youth FocusFamilyFocus

Seven Cluster SolutionSeven Cluster SolutionSeven Cluster Solution

Quality Services


The Ladder GraphThe Ladder GraphThe Ladder Graph

• Shows the results of comparisons betweenparticipant responses on importance andfeasibility, by cluster

• Depicts the correlation between these twodimensions

• A correlation ranges from -1.00 to +1.00

• In the next example, the correlationcoefficient (r value) for the level ofagreement between the two dimensions issignificant at 0.93

Cluster Ratingson Importance and Feasibility

Cluster RatingsCluster Ratingson Importance and Feasibilityon Importance and Feasibility

r = 0.93

Importance Feasibility

4.1

2.9

4.1

Funding

Funding

Service Delivery


Advocacy

Service Delivery

Evaluation

Advocacy


Family Issues

Community Education

Evaluation Youth Focused Initiatives

Community Education Family Issues

Youth Focused Initiatives

2.9

Go-Zone PlotGo-Zone PlotGo-Zone Plot• Facilitates planning. By focusing

on those items rated as themost important and mostfeasible, action plans can thenbe created.

• Facilitates development of alogic model for systems change,by defining areas of importance.

Go-ZonesGo-ZonesGo-Zones

4.223.38 4.96

3

4.71

Importance

Fea

sib

ility 3.91

1

2

3 4

5

6

78

9

10

11

1213

14

15

16

17

1819

20

21

22

23

24

25

26

2728

29

30

3132

33

34

35

36

37

38

39

40

41

4243

44

45

4647

48

49

50

51

52

53

54

55

56 57

58

59

60

61 6263

64

65

66

67

6869

7071

72

73

74

757677

78

79

80

81

82

8384

85

86

Highest StatementsHighest StatementsHighest Statements

• Listen to youth• Ask youth what they want• Deal with confidentiality• Develop a plan for

communications• Train direct service staff to

serve children and families

Benefits• The concept mapping framework

provides an easy, organized way tohelp participants with planning or anevaluation study.

• The methodology helps participantsdescribe their ideas in a familiarlanguage rather than that of theevaluator or planner, free ofjargon.

BenefitsBenefits•• The concept mapping frameworkThe concept mapping framework

provides an easy, organized way toprovides an easy, organized way tohelp participants with planning or anhelp participants with planning or anevaluation study.evaluation study.

•• The methodology helps participantsThe methodology helps participantsdescribe their ideas in a familiardescribe their ideas in a familiarlanguage rather than that of thelanguage rather than that of theevaluator or planner, free ofevaluator or planner, free ofjargon.jargon.

• The results are presentedgraphically, showing the ideasand relationships, to theparticipant group as well as toother interested groups.

• The process increases groupownership of the actions basedon the results.

Using the ResultsUsing the ResultsUsing the Results

Examples from the FieldExamples from the Field

Wm. Martin Hydaker, DirectorHydaker Community Consulting

How Others HaveUsed Concept Mapping

How Others HaveHow Others HaveUsed Concept MappingUsed Concept Mapping

Concept Systems, Inc.Example of Clients

Concept Systems, Inc.Concept Systems, Inc.Example of ClientsExample of Clients


Wm. Martin Hydaker, DirectorHydaker Community Consulting

How Others HaveUsed Concept Mapping

How Others HaveHow Others HaveUsed Concept MappingUsed Concept Mapping

Concept Systems, Inc.Example of Clients

Concept Systems, Inc.Concept Systems, Inc.Example of ClientsExample of Clients

Examples from the FieldExamples from the FieldExamples from the Field

• National Traumatic StressInitiative

• State of Mississippi• Maternal Child Health Bureau• Comprehensive Community

Mental Health Services Programfor Children and Their Families

CommonalitiesCommonalitiesCommonalities

• Engagement activity• Planning for change

Site Specific AdaptationsSite Specific AdaptationsSite Specific Adaptations• Enhancing community voice• Questionnaire and survey development• Evaluation & continuous quality

improvement• Group comparisons• Multiple implementation sites• Increase family & youth “voice”• Logic model/strategic plan development

Trauma Recovery for YouthNetwork (TRY)

Trauma Recovery for YouthTrauma Recovery for YouthNetwork (TRY)Network (TRY)

A partnership working todevelop a multi-agency, childand family centered trauma-informed community networkserving Hinds, Madison andRankin Counties.

TRY Major ActivitiesTRY Major ActivitiesTRY Major Activities

• Build upon existinginfrastructure to develop acommunity practice center;

• Provide services to, and collectdata on trauma affectedchildren;

Major Activities cont.Major Activities cont.Major Activities cont.

• Bring expertise to bear oneffective practices, servicefinancing, and other issues;

• Provide leadership and trainingon child trauma mitigation forproviders and staff in the area.

TRY Concept MappingTRY Concept MappingTRY Concept Mapping

• To develop an approach guided bythe community.

• Identification of highly valuedactivities.

• Use prioritized activities to buildthe local network.

• Establish a baseline for evaluation

Examples of Important andFeasible Activities

Examples of Important andExamples of Important andFeasible ActivitiesFeasible Activities

• Increase awareness of servicesavailable through the network

• Provide trauma training to directservice staff

• Identification of new partners• Link first responders & service

providers• Increase understanding of the values

and cultural norms of the populationserved.

ConclusionsConclusionsConclusions

• A shift in implementation focus• Prioritized activities became a

focus for evaluation andcontinuous quality improvement

• Information for interviews anda brief rating scale derivedfrom concept mapping

Mississippi’s Department ofMental Health

&Mississippi Families as Allies

MississippiMississippi’’s Department ofs Department ofMental HealthMental Health

&&Mississippi Families as AlliesMississippi Families as Allies

Planning for statewideimplementation of communitysystems of care for children andyouth with seriousemotional/behavioral disorders andtheir families.

Planning for StatewideImplementation

Planning for StatewidePlanning for StatewideImplementationImplementation

• Identification of critical stepsnecessary for statewideimplementation of SOC.

• Comparison of response from twogroups; community stakeholders andkey state level personnel

Focus StatementFocus StatementFocus Statement

Generate a list of things thatneed to be done if the systemof care is to be developed andsustained successfullythroughout the state.

Results, using combined groupsResults, using combined groupsResults, using combined groups

• A low level of agreement betweenimportance and feasibility, r = .31

• Relatively little agreement betweenthe two groups on importance,r = -.40

• Far more agreement between thetwo groups on the dimension offeasibility, r = +.95

Results – CombinedResults Results –– Combined Combined

Importance ComparisonImportance Comparison

Feasibility ComparisonFeasibility ComparisonFeasibility Comparison

Results – Community GroupResults Results –– Community Group Community Group

• High level of agreement on whatwas important and feasible,r = +.73

Results – Community GroupResults Results –– Community Group Community Group

Results – State level groupResults Results –– State level group State level group

• Moderate to high level ofdisagreement between whatthey considered to be importantand what they viewed to befeasible, r = -.30

Results – State level groupResults Results –– State level group State level group


• The clusters for both groups wereconsistent with the essentialelements of SOC.

• The two groups placed differentimportance on next steps.

• Remarkable agreement on what thetwo groups saw as feasible.


• The community group is moreoptimistic and has differentpriorities.

Integrating Families,Communities, and Providers

(IFCAP)

Integrating Families,Integrating Families,Communities, and ProvidersCommunities, and Providers

(IFCAP)(IFCAP)• Grantee - Florida Institute for

Family Involvement• Funding – Federal Maternal and

Child Health Services Divisionfor Children with Special HealthCare Needs

IFCAP Vision IFCAP Vision IFCAP Vision

Build and sustain a familycentered, community based,culturally competent system ofcare for all children and youthwith special health care(CYSHCN) needs and theirfamilies in Florida throughcommunity partnerships.

IFCAP ApproachIFCAP ApproachIFCAP Approach

• Enhance family capacity to befull partners

• Enhance provider capacity toimplement medical homeapproach and to partner withfamilies

IFCAP ApproachIFCAP ApproachIFCAP Approach

• Enhance system capacity tofacilitate, fund and sustainquality care and partnershipsfor CYSHCN

• Support existing communitypartnerships to build localsystems

IFCAP Concept MappingIFCAP Concept MappingIFCAP Concept Mapping

• To understand more clearly theenvironment surroundingCYSHCN

• Focus work efforts during theremainder of the grant period

• Individualized approach to eachcommunity

IFCAP Concept MappingIFCAP Concept MappingIFCAP Concept Mapping

• Monitor progress and increaseaccountability - follow upinvolved key informantinterviews and completednessscales

• Integrated report


• Improved understanding of theenvironment in each community

• Invigorated interest andplanning at the community level

• Improved accountability

Integrated FindingsIntegrated FindingsIntegrated Findings

• A consensus that improvement inservices could not occur withoutincreases in funding

• Increased belief that a focus onthe family is key improving the livesof CYSHCN

• Education and training areimportant and feasible

System of CareSystem of CareSystem of Care

commUNITYcarescommUNITYcares(uniting neighborhoods (uniting neighborhoods –– integrating through youth) integrating through youth)

andand

Circle of HopeCircle of Hope(Hope, Opportunities, Parents and Professionals(Hope, Opportunities, Parents and Professionals

Empowerment)Empowerment)

System of CareSystem of CareSystem of Care

• Funding – FederalComprehensive CommunityMental Health Service Programfor Children and Their Families

commUNITYcare & HOPEFirst Year Planning

commUNITYcare & HOPEcommUNITYcare & HOPEFirst Year PlanningFirst Year Planning

• Governance• Vision, mission, & goal

development• Population of focus• Strategic and sustainability

planning

commUNITYcare & HOPEConcept Mapping

commUNITYcare & HOPEcommUNITYcare & HOPEConcept MappingConcept Mapping

• Broad based community input• Increased family and youth

“voice”• Incorporated into logic model• Evaluation and continuous

quality improvement


• Enhanced community, family andyouth involvement

• Logic model development• Completedness measures

Concept Mapping & LogicModel Development

Concept Mapping & LogicConcept Mapping & LogicModel DevelopmentModel Development

A theory of change for a localsystem of care is “theory” inthe sense that it representsstakeholders’ best ideas aboutthe action they need to take.(Hernandez & Hodges 2003)

commUNITYcareFocus StatementcommUNITYcarecommUNITYcareFocus StatementFocus Statement

What specific actions/stepsneed to be taken for the systemof care to be successful?

HOPEFocus Statement

HOPEHOPEFocus StatementFocus Statement

Generate a list of things thatneed to done if the system ofcare is to be developed andsustained successfully.

Concept Mapping & Logic ModelDevelopment References

Concept Mapping & Logic ModelConcept Mapping & Logic ModelDevelopment ReferencesDevelopment References

Anderson LA, Gwaltney MK, Sundra DL, Brownson RC,Kane M, Cross AW, et al. Using concept mapping todevelop a logic model for the Prevention ResearchCenters Program. Prev Chronic Dis [serial online]2006 Jan [date cited]. Available from:URL:http://www.cdc.gov/pcd/issues/2006/jan/05_0153.htm.

SVETLANA YAMPOLSKAYA, TERESA M. NESMAN,MARIO HERNANDEZ, AND DIANE KOCH, UsingConcept Mapping to Develop and Logic Model andArticulate a Program Theory: A Case Example.American Journal of Evaluation 2004


Robert Paulson, (ret.) Professor Dep’t of Child and Family Studies

Louis de la Parte Florida MentalHealth Institute

University of South Florida

Three Pragmatic Examples ofthe Use of Concept MappingThree Pragmatic Examples ofThree Pragmatic Examples ofthe Use of Concept Mappingthe Use of Concept Mapping

• Getting a quick consensus for anewly formed heterogeneousgroups-The Florida SuicideImplementation Project

• Catalyzing a Non-system intoAction-The Collier CountyChildren’s Mental Health NeedsAssessment Project

Three Pragmatic Examples ofThree Pragmatic Examples ofthe Use of Concept Mappingthe Use of Concept Mapping

• Identifying the critical (active)ingredients in an effective modelfor the purposes of replication-TheIngham County, MI Home BasedCare Project

Context for The Florida SuicideContext for The Florida SuicideImplementation ProjectImplementation Project

• A suicide prevention coordinating councilwas created by the legislature andappointed by the governor to advise anewly created Office of SuicidePrevention in the Governor’s Office ofDrug Control

• The council consisted of over thirtymembers and only meets four times a year

• The new council consisted of state andlocal agency representatives, advocatesand survivors some of whom had workedtogether others whom were new to thefield of suicide prevention and each other

Challenges for the ProjectChallenges for the Project• How to build on prior work of a state wide

coalition and state wide task force whichhad created a Florida Suicide PreventionStrategy but give the new organization achance to put its “own stamp” on suicideprevention activities and develop acollective ownership

• How to develop consensus quickly so workcan begin with so few meetings and littleopportunity to have a more natural organicprocess of developing trust and rapport

Challenges for the ProjectChallenges for the Project

• No opportunity to do planning withentire group and develop ownership ofthe process although the decision to useconcept mapping was made jointly withthe Director of the Office of SuicidePrevention and Drug Control

• Introduction to Concept Mapping andbrainstorming had to occur within a onehour and fifteen minute time period inthe initial four hour meeting of thecoordinating council

Need for Global VersionNeed for Global Version• Since there would not be another meeting

for three months there was no choice butto use the global version

• Because not all of the council membersattended and there was a desire to get asmuch public input as possible, the webbased version allowed for more people toparticipate in each phase and patternmatching would detect any major sub-group differences (e.g., council members vs.others)

98

BrainstormingBrainstorming• Brainstorming participants generated

statements in response to the prompt“In order to move the Florida SuicidePrevention Strategy into ongoingcommunity programs and/or initiatives,a specific activity/task thatshould/must occur is…

• The brainstorming process generated149 statements which were consolidatedto the system limit of 125 statements.

99

Sorting and RatingSorting and Rating• Following brainstorming, participants were invited to go

on the website and sort the 125 statements intocategories in a “way that makes sense” to them and toname each pile of statements.

• Participants were also asked to rate the statementsfrom 1 to 5 in terms of:Importance (1 being not important and 5 being very

important)Ease of implementation (1 being not easy and 5

being extremely easy)Impact on Reducing the Suicide Rate (1 being little

to no reduction and 5 being an extreme reduction)

100

• Participants:• 51 brainstorming• 20 sorting,• 22 rated importance• 20 rated the ease of implementation• 21 rated impact on the suicide rate

• Response rate is within the acceptablelimits reported in the literature.

Response RateResponse Rate

Response Rate and Use ofResponse Rate and Use ofGlobal VersionGlobal Version

• The biggest reason for not doing therating and sorting was the amount of timeit took to complete the process

• Secondly, even though one of the projectteam had been involved with suicideprevention for many years and was on thecouncil we were an outside group

• Many council members representedagencies where suicide prevention was notpart of their primary mission so there wasless incentive to take the time to completethe task

Cluster AnalysisCluster Analysis• The very heterogeneous nature of the

council resulted in very different sortingpatterns so the clusters were not asclosely connected as they frequently are insituations where there is some unifyingcharacteristic (agency, profession, sharedissue)

• However, Cluster Analysis still proved tobe useful in organizing the statements intogroupings which could be useful forforming sub-committees or ad hoc taskgroups of interested parties to continuework in between meetings

103

Six Cluster Solution

1. Planning, Capacity Building and Strategic Engagement

2. Social Marketing, Legislation and Advocacy

3. Communication of Resources

4. Initiatives for Specific Populations

5. Programs

6. Training

1

2

3

4

5

6

7

8

9 10

11

12 13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28 29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62 63

64

65

66

67

68

69

70 71

72

73

74

75 76

77

78

79

80

81

82

83

84 85

86

87

88

89

90

91 92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

104

Spanning AnalysisSpanning Analysis

• Because of the heterogeneity of thegroup, many statements were sortedinto different piles by so many peoplethat the computer can’t assign it to aparticular cluster so it is placed in thecluster which geographically is theaverage of the sorting.

• Spanning analysis was useful in helpingpeople understand why a very differentkind of statement was found in aparticular cluster

105

1. Planning and Capacity Building

2. Social Marketing

3. Communication

4. Youth Initiatives

5. Programming and Training Targeted for Specific Age Groups

6. Suicide Prevention Training

1

2

3

4

5

6

7

8

9 10

11

12 13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28 29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62 63

64

65

66

67

68

69

70 71

72

73

74

75 76

77

78

79

80

81

82

83

84 85

86

87

88

89

90

91 92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

Spanning Analysis for Statement 24

106

• As might be expected there was ahigher correlation (.6) between thestatements which were viewed asimportant and those which would havean impact on suicide reduction.

• As is frequently the case there was amuch higher correlation in ratings onfeasibility since everyone shared thesame environmental realities andconstraints (e.g., legislation, politics,resource constraints)

Pattern MatchingPattern Matching

Pattern MatchingPattern Matching• Because the majority of positions on the

council were legislatively mandated therewas a natural concern that it might notrepresent the views of the general public

• Pattern matching was very helpful inresolving these concerns because theresults from a comparison of responsesbetween Council Members and Non-Councilmembers had a very high correlationindicating that the council was highlyrepresentative of the views of the largergroup (i.e., the general public)

108

Go Zone PlotGo Zone Plot• Go Zone Plots proved to be very useful for

planning the next steps.• Action plans can be created by focusing on

those statements that are perceived to bethe most important and easiest toimplement while making strategic decisionson whether to devote resources to difficultbut important activities or to chose some“easy” but somewhat less important tasksto create quick victories.

• While the actual plots were interesting forpeople to see the relationships it wasessential to undertake the time consumingtask of listing out the statements in eachzone.

7 Engage the media on advancing state suicide prevention efforts. Florida is welcome to utilize the SPAN USA Media Action Center to reach local Florida media to assist in this effort and to utilize SPAN USA's Engaging the Media Guide.12 All schools need to have "Where To Turn" books listing agencies to help all youth and their families.15 Identify and support programs targeted to at-risk youth that are found to have a lower suicide rate than the Florida average for that demographic group.20 Use the Strategic Prevention Framework's 5-step process to continue to strengthen state planning so that resources are targeted where there is the greatest need and in a manner that tracks progress and measures outcomes.24 Address issues (e.g., medical problems, loss of spouse & friends, financial problems) that cause depression in the elderly.37 Develop a comprehensive list of all types (areas of focus), locations and contact information of all suicide prevention gatekeeper programs throughout the state to be disseminated throughout the state to encourage the establishment of a network of gatekeepers.43 Partner with state agencies to educate and train not only the personnel but the populations they serve.46 Identify an assortment of effective public awareness and educational materials that can be used to carry the suicide prevention message into the state agencies and various organizations.47 Identify a network of trainers around the state who can train various populations about suicide prevention in a variety of settings.48 Offer counseling and prevention services in schools.49 Make school administrators aware of the suicide prevention tools they can implement in their districts.51 Support legislation to educate public school employees about youth suicide.53 Use federal and national public awareness resources (e.g., SPRC and 1-800-273 TALK) and identify state and local avenues through which to disseminate them.56 Integrate suicide prevention activities with other prevention initiatives that are currently ongoing around the state so we can utilize their resources and ensure that we are a part of the broader prevention network.58 Ensure teachers receive training (and refresher training) on identification of childhood depression and suicidality, as well as how to respond appropriately.63 Engagement in a regular (e.g. annual) data-driven strategic planning process.64 Creation of a resource directory that identifies resources and programs around the state.65 Use PSA's to disseminate messages, e.g., "Suicide prevention is important because each new suicide puts others at risk, promote life as a solution, Stopping one suicide saves more than one life.“69 Develop a One Stop comprehensive web site.

Continued on next slide.

Zone Charts With StatementsZone Charts With Statements

Zone 1: Most Important and Easiest to ImplementZone 1: Most Important and Easiest to Implement1

110

Results of Concept Mapping:Results of Concept Mapping:Comparison to FSPS ObjectivesComparison to FSPS Objectives

• Comparison of the statements generated by theconcept mapping process to the ten objectives in theFlorida Suicide Prevention Strategy revealed thefollowing:The newly formed coordinating council confirmed

the work which had gone on before and iscurrently reflected in the OSP Action Items plan.

New areas of action that arose during conceptmapping can be incorporated into the plan as itevolves.

The clustering provided a useful starting point tohelp organize all of the activities into manageablegroups

• Concept mapping proved to be very useful in jumpstarting the work of the OSP and the Council

Project Spin-offProject Spin-off• Based on the success of the initial

concept mapping a local suicideprevention coalition which had“stagnated” asked us to use the processto reinvigorate the coalition by invitingpersons and groups who hadparticipated in the past but were nolonger active, who had expressed someinterest, or who the group thoughtwould be helpful to the cause (e.g. faithbased groups) to a planning meetingusing concept mapping

Project SuccessProject Success

• The use of a novel planning processhelped attract a much larger group ofparticipants

• The results identified a number ofoptions for the coalition to expand itsactivities and membership

• Over time the new energy anddirections have been sustained

Collier County Children’s Mental HealthNeeds Assessment

Concept Mapping Project

Collier County Children’s Mental HealthNeeds Assessment

Concept Mapping Project

Planning

Outcomes

Team process

Four-cluster solution (effectiveness)

Evaluation/assessment

Project ContextProject Context

• The project was funded by the NaplesChildren’s Education Foundation toconduct a needs assessment ofChildren’s Mental Health

• Project consisted of 34 Key Informantinterviews, concept mapping, networkanalysis, community forums, a financingsub-study, and an analysis of availableCollier County data

Project ContextProject Context• A project advisory committee and key

informants emphasized they understoodwhat the problems were and were muchmore interested in how they could improvethe system particularly incorporatinginnovative programs and policies whichwere effective in other sites

• The major agencies and sectors knew eachother personally and served on manycommittees together but admitted theyhad little understanding of what eachother actually did

Project ContextProject Context

• A major issue was that there was nocoordinating group or authority responsiblefor children's mental health or children’sservices in general

• Concept mapping appeared to be a goodway of getting a consensus on what wouldbe the best ways to improve the systemand provide the foundation withbackground information they could use incrafting a focused children’s mental healthinitiative

Concept MappingConcept Mapping• In this case a small group of available members

of the advisory committee convened for theinitial brainstorming process

• A follow-up e-mail was sent to all key informantsand other interested parties with the statementsand a request to add any additional statements tothe focus prompt

• A packet for hand sorting and rating along with apre-paid returned envelope was sent to the entirelist

• The inability to conduct a meeting for sorting andrating led to an initial small response rate butencouragement by the foundation boosted theresponse rate considerably

Focus Statement:

“The most important things toimprove systems and services forchildren with mental healthchallenges in Collier County so thatthey and their families couldachieve better outcomes are:”

Planning

Outcomes

Team process

Four-cluster solution (effectiveness)

Evaluation/assessment

Cluster AnalysisCluster Analysis• As can be seen in the following cluster

diagram the clusters are logical forfocusing on system improvement.

• The center cluster included many of thestatements regarding the exchange ofinformation around the services of theagencies and a centralized information andreferral resource

• The cluster served as a useful mechanismfor integrating the findings across all ofthe different data sources

Modified Seven Cluster Solution

1. Infrastructure/Workforce

2. Prevention

3. Funding

4. Access to Services 5. Identifying Needs/Marketing & Outreach

6. Training/Education

7. Provider Collaboration

1

2

3

4

5

6 7

8

9

10

11

12

13

14

15

16

17

18 19

20

21

22

23

24 25

26

27

28

29

30

31

32

33

34

35

36

37

38

39 40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72 73

74

75

76 77

78

79

80

81

82 83

84

85

86

87

88

89

Pattern MatchingPattern Matching• Since there was no children’s service system

per se there was a concern that there mightbe major differences on perceptions of needsand solutions

• Pattern matching showed that there was verylittle disagreement on the importance ratingsbetween mental health respondents and othersectors and unanimity on ease ofimplementation

• As in most projects items were rated ashigher on importance than in ease ofimplementation showing again they operatedunder similar constraints

Go-ZonesGo-Zones• While the Go-Zone charts are useful we

experimented with a number of ways topresent the data in the most useful format

• In addition to listing out the statements by goZone, we found that the best way, illustratedin the next slide, for action planning purposeswas to take the top quarter or third of theimportance rankings (depending on wherethere were cut-offs because of many ties inscores) and listing these statements bycluster with the top importance ratingsfollowed by the ease of implementationratings

Go-ZonesGo-Zones

• This kept the organization value of theclusters with the top priorities in eacharea

• It also showed which clusters had themost high priority statements and evensome clusters which had no statementsin the top tier of importance

1. Identifying needs 4.50 3.09

58. using data to inform practice 4.20 2.70

Thirty Most Important Statements By Cluster With Importance & Ease Of Implementation Average

Cluster 1: Infrastructure/Workforce Importance EOI

4. identifying the children who need services 4.80 2.82

18. more early identification and prevention services 4.70 2.4519. making prevention services a high priority 4.67 3.1846. early intervention 0 to 3 4.50 3.0074. abuse prevention 4.40 3.0984. enhance in-home services 4.20 2.45

Cluster 2:Prevention

Cluster 3: Funding

25. maximize funding resources available 4.80 2.36

24. seek funding resources in the private sector 4.50 2.2782. advocate for insurance parity 4.40 1.7034. united front with legislature around budget issues 4.30 2.1883. eliminate lack of coverage for pre-existing conditions 4.20 1.09

Cluster 4: Access to Services

9. use of best practices model 4.80 3.36

62. transportation 4.30 1.73

Usefulness of Concept MappingUsefulness of Concept Mapping• Concept mapping was helpful as a means of

triangulating and integrating the data fromthe various sources

• It also provided a vehicle for the variousactors and sectors to think in an organizedway about what they could do to improvethe system

• The degree of consensus proved to be amotivator for further action

• A recently formed children's mentalhealth coalition has emerged as a possiblecoordinating body and whether or notthere is foundation funding will use theconcept mapping (and report in general) asthe basis for its agenda

126

Concept Mapping for theConcept Mapping for the Home Based Services Home Based Services

ProgramProgram

Project ContextProject Context• The Home Based Services Program

(HBS) of the Family Guidance Servicesof Ingham County MI had been shownthrough a state-wide comparison ofCAFAS scores to be one of the mosteffective state programs

• There was interest on the part of theState and the Ingham County System ofCare in providing additional support forHBS as an evidence based practice andto develop a method for replicating theprogram in several pilot sites inMichigan

Project ContextProject Context• The program had been established about 15

years ago by the current program directorout of frustration with the ineffectiveness ofthe prior system

• Many of the staff had been with the programover ten years and there was very littleturnover

• Staff were generally hired from MSU internsafter they had the opportunity to be trainedand screened through the internship process

Project ContextProject Context• Consequently there was a high degree of

consistency among all staff in how theyviewed the program and very high staffmorale

• The program was a mixed methods studyusing key informant interviews, teamobservations, shadowing of home visits, sixcase studies where four respondents(parent, child, HBS worker, and othersector worker) were interviewed aboutHBS and what was successful orunsuccessful in serving that particularchild

Project Context• All key informants were incredibly consistent

in identifying HBS as a value based programwith a consistent clinical philosophy whichmirrors system of care values (strengthbased, family driven, data based decisionmaking) and an agency practice andphilosophy encouraging supportiveadministration, a whatever it takes approach,an encouragement on trying new things, highquality clinical and peer supervision

• Staff were treated the way they wereexpected to treat families-with trust andrespect

Concept Mapping

• We used concept mapping to identify the“active or critical” ingredients of theintervention

• It in fact is a quicker and more scientificversion of the nominal group expertprocess used by many programs such asACT in developing there fidelity scales

132

BrainstormingBrainstorming• The concept mapping group consisted of the

program director, senior supervisors, homebased clinicians, workers from the schools andcourts working with HBS and 4 HBS familymembers

• Participants generated statements inresponse to the prompt “The most importantingredients contributing to the effectivenessof the Home Based Services Program are:”

• Brainstorming generated 136 Statements• Sorting and rating was completed the

following day

Cluster Analysis ResultsCluster Analysis Results• The cluster analysis perfectly mirrored

what we had learned from observations,key informant interviews and case studies

• The management beliefs and practices andthe central value of respect for the childand family were the center bridgingclusters

• Program characteristics are reflected inthe two clusters on the left while clinicalphilosophy (including the importance ofcollaboration) and therapistcharacteristics form the region on theright

134

7 Cluster Solution

1. Home Based/family Driven

2. Qualities of effective home-based services

3. Respect for Child and Pamily

4. Management beliefs/practices

5. Atrributes of Therapists

6. Community Collaboration

7. Clinical Philosophy and Approach

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18 19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42 43 44

45

46

47

48 49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75 76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97 98

99

100

101

102

103

104

105 106

107

108

109

110

111

112

113 114

115 116

117

118

119

120

121

122

123

124 125

126

127

128

129

130 131

132

133

134

135

136


• Once we had established that there wasremarkable consistency between all ofthe sub-groups we used the analysis ofpattern matching to highlight potentialissues which need to be addressed inreplication

• Some examples are provided in the nextfew slides


• Both direct service workers (HBS andother sectors) and consumers ratedmanagement beliefs and practices last

• Consumers rated management beliefsand practices as easiest while providerssaw it as most difficult

• The attributes of therapists on theother hand were seen as quite difficultby consumers but much easier byproviders

136

Pattern MatchingPattern Matching• Interestingly, in comparing HBS with other

sectors the one cluster that there wascomplete agreement on was the ease ofimplementation of the home based/familydriven cluster

• There was a big difference however in theratings of the ease in implementingrespect for child and family.

• Since this is a core principle which isalready implemented in HBS it was seen aseasy compared to the other sectors

137

138

• These ladder graphs emphasize that as wewould expect different roles (administrator,provider, consumer) and sectors (MH andother) influence the perceptions ofimportance and ease of implementation.

• A replication strategy should take thesedifferences into account when presentingthe model and be careful to tailor themessage and strategies to take thesedifferences into account

Pattern Matching


• The biggest differences for the groupsin their perception of ease ofimplementation was between themanagement beliefs/practices and theattributes of therapist clusters

139

Lessons LearnedLessons Learned

• Concept mapping can be used in a numberof different ways and in differentcontexts other than the more traditionaluses for evaluation and research

• It should be noted that in each of theseexamples follow up concept mapping can beconducted as one form of evaluation usinga focus prompt focusing on the progress inachieving the next steps identified in theinitial process

Lessons LearnedLessons Learned• The process was valuable in getting feed back

from participants at a presentation of theresults and giving the funders confidencethat “we had gotten it right”

• The visuals produced by concept mapping werehelpful to many participants

• All of these projects had short time framesand limited resources and concept mappingproved invaluable in helping to get quickagreement on the issues and next steps

DEFINING COMMUNITYREADINESS

for theIMPLEMENTATION OF A

SYSTEM OF CARE

DEFINING COMMUNITYDEFINING COMMUNITYREADINESSREADINESS

for thefor theIMPLEMENTATION OF AIMPLEMENTATION OF A

SYSTEM OF CARESYSTEM OF CARE

Lenore B. Behar, Director Child & Family Program Strategies

www.lenorebehar.com

CreditsCreditsCredits This study was developed under Contract 280-

03-4200, Task Order Number 280-03-4200,funded by the Child, Adolescent and FamilyBranch, Center for Mental Health Services,Substance Abuse and Mental Health ServicesAdministration, United States Department ofHealth and Human Services. The contents ofthis presentation do not necessarily reflect theviews or policies of the funding agency andshould not be regarded as such.

The Comprehensive CommunityMental Health Services Programfor Children and Their Families

The Comprehensive CommunityThe Comprehensive CommunityMental Health Services ProgramMental Health Services Programfor Children and Their Familiesfor Children and Their Families

Provides funds to• States• Communities• Territories• Indian tribes & tribal organizations

Government InvestmentGovernment InvestmentGovernment Investment

• Since 1993, 144 grants• $4-5 million per site, over 6 years• Technical assistance, training• Evaluation• Currently, 59 funded sites, 83

graduated• 2008-09, 18 new sites funded

Program PrinciplesProgram PrinciplesProgram Principles

• Services should be driven by theneeds and preferences of the childand family

• Service planning should bestrengths based

• Management of services shouldoccur in a multi-agencycollaborative environment

•

Program Principles-MoreProgram Principles-MoreProgram Principles-More

• Programs should be responsive tothe cultural context of thepopulation served

• Families should be the leadpartners in planning andimplementing the system of care.

Easy to believe inEasy to believe inEasy to believe in

Hard to do

Where to start?

Hard to doHard to do

Where to start?Where to start?

Community ReadinessA Missing Piece

Community ReadinessCommunity ReadinessA Missing PieceA Missing Piece

• Start by determining the areas ofreadiness and areas needingstrengthening

• But first, how to define readiness

Readiness Study DesignReadiness Study DesignReadiness Study Design

Concept mapping fit the requirements togather and organize information

• Needed to build consensus amongdiverse populations

• Participants were scattered across thecountry

• Wanted a transparent and notsubjective process

Study Design--MoreStudy Design--More

• Used Concept Systems “Global”software to gather information

• Through e-mail invited participants,gave instructions, and link towebsite

Study SampleStudy SampleStudy Sample• Selected participants experienced in

implementing systems of care

• Invited two groups totally 223 people– Group 1: 151 representatives from 27

5th and 6th year sites– Group 2: 72 experts in systems of

care (trainers, researchers,evaluators, consultants)

Phase 1: BrainstormingPhase 1: BrainstormingPhase 1: Brainstorming

Participants responded by enteringstatements online to:

“To be ready to develop a systemof care, the following specificcharacteristics and functions areessential to be in place before anapplication for funding can becompleted.”

Respondents to BrainstormingRespondents to BrainstormingRespondents to Brainstorming

• Responses from 115 people (52%)

• Broad representation across targetgroups, age, race, gender, ethnicity– 28% administrators (PIs, PDs)– 13% outside experts– 11% TA coordinators– 8% parent coordinators– 4% parents

Responses to BrainstormingResponses to BrainstormingResponses to Brainstorming

• 336 statements generated

• 109 unduplicated ideas

• “collaboration” was the mostfrequently misspelled word!

Phase 2: Rating the ItemsPhase 2: Rating the ItemsPhase 2: Rating the Items

Invitations to participate were issued

Group 1 (27 sites) members were asked torate the 109 items on a scale of 1-5

Ratings were for the Importance of theitem and the Ease of Implementation

65 of 155 members responded (42%)

Phase 2: Sorting the ItemsPhase 2: Sorting the Items

Invitations to participate were issued

Group 2 (72 experts) members wereasked to sort the 109 items into groupsthat went together

Group 2 members were asked to label thegroups

36 of the 72 members responded (50%)

Note about Response RateNote about Response RateNote about Response Rate

The response rate was from 42% - 52%;25 of 27 sites responded

Although this appears low, it met thepurposes of the study:• To offer participation to many of people

• To obtain responses from enough peoplefor a robust sample, according toConcept Systems criteria (n=14-20)

ResultsResultsResults

Seven Cluster SolutionSeven Cluster SolutionFamilies and Youth as Partners

Plan to Expand Services

Evaluation

AccountabilityLeadership

Shared Goals

Inclusive Planning Process

Moving From Seven to EightMoving From Seven to EightClustersClusters

Eight Cluster SolutionEight Cluster SolutionFamilies and Youth as Partners


Evaluation

AccountabilityLeadership

Shared Goals Collaboration

Network of Local Partners

Moving From Eight to NineMoving From Eight to NineClustersClusters

Nine Cluster SolutionNine Cluster SolutionFamilies and Youth as Partners


Evaluation

Accountability

Shared Goals


Collaboration

Governance

Leadership

Cluster RatingsCluster Ratingson Importance and Ease of Implementationon Importance and Ease of Implementation

Importance Ease of Implementation

4.35 4.35

3.10 3.10


LeadershipCollaboration

Families & Youth as Partners

Accountability Plan to Expand Services

Shared GoalsEvaluation

Plan to Expand Services Evaluation Accountability

Shared Goals Collaboration

Families & Youth as Partners


Leadership

r = .75

Plot of Focus ZonesPlot of Focus Zones

3.312.43 3.97

3.25

4.55

4.14

12

34

5

6

7

8

910

1112

13

14

1516

17

18

19

20

21

22

23

24

25

262728

29

30

31

32

333435

3637

38

39

40

41

42

43

44

45

46

474849

50 51

52

53

54

55

5657

5859 60

61

62

63

64

65

66

67

68

69

7071

72

7374

75

76 77

78 79

80

8182

83

84

85

868788

8990

919293

94

95

9697

98

99

100

101

102

103

104

105106

107

108

109

Ease of Implementation

Impo

rtan

ce

Five Most Important ItemsFive Most Important ItemsFive Most Important Items

• There should be input from youthand families to determine the needsin the community.

• It must be understood thatsustainability of services should bepart of discussions beginning in the1st year not waiting until the end.

Most Important Items--MoreMost Important Items--More

• It is important to have a realcommitment to the effort fromkey community stakeholders -people with the ability toinfluence attitudes and actionsof others such as electedofficials, community champions,respected individuals, etc

Most Important Items--MoreMost Important Items--More

• The concept of permanent systemchange needs to be understood andaccepted as the end goal.

• There must be a commitment frompolicy makers, community leaders,partners, and staff to the systemof care values and principles.

Next StepsNext StepsNext Steps

• Use the list of items to create theCommunity Readiness AssessmentScale (CRAS)

• Work with new sites to assessreadiness; report within 30 days

• Re-assess in 12 months to measureprogress

Documents

Concept Mapping · model development •State level planning •Evaluation, using repeated measures. State Level Planning Example Planning for new programs, expansions, and budgets