1. Presented at the One Voice Board MeetingJune 20, 2014Gerald R. Goodman, DrPHProfessor and Program DirectorHealth Care AdministrationTexas Woman's University1

2. The purpose of the One Voice Collaborative Surveywas to evaluate the degree of success of thecollaboration process, determine member satisfaction,examine the benefits and drawbacks of participation asperceived by the collaborative members, and gainmember feedback. The survey examined member demographics,members perceptions of components of successfulcollaboration including environment, membershipcharacteristics, process and structure, communication,purpose, and resources, perceived benefits anddrawbacks of collaborative, member satisfaction, andmember feedback.2 3. Comparison Report: 6 years and 3 surveyslater how are we doing?1. Review of the Survey Structure2. Who were the respondents?3. Findings Comparing 3 SuccessiveSurveys 2008, 2011, 20144. Conclusions and Next Steps3 4. The survey statements were adapted from the WilderCollaboration Factors Inventory developed by the Amherst H.Wilder Foundation. The survey (inventory) was developed and tested over aperiod of eight years. The inventory is designed to helpcollaboratives and community collaboratives assess theirstrengths and weaknesses relative to 20 success factors. Research suggests that these factors can apply tocollaborative efforts of nonprofit, governmental and otherorganizations. Respondents are asked to rate their level of agreement with40 different statements ** about the collaborative.4 5. Category (6)Category Factory (20)Category Statement (40) The survey respondents answered statementquestions. The statement questions are groupedinto 6 categories for analysis.5 6. 6Example of Rank OrderCategory Factor StatementEnvironment History of collaboration in the community 5. Agencies in our community have a history ofworking together.6. Trying to solve problems through collaboration hasbeen common in this community. It's been done a lotbefore.Collaborative group seen as a legitimate leaderin the community7. Leaders in this community who are not part ofCare for Elders seem hopeful about what we canaccomplish.8. Others (in this community) who are not part ofCare for Elders would generally agree that theorganizations involved in this partnership are the"right" organizations to make this work.Favorable political and social climate 9. The political and social climate seems to be "right"for a partnership like this one.14. The time is right for this partnership. 7. To simplify analysis, the authors grouped the40 individual statements into six globalcategories, which were in turn grouped into20 success factors: Categories:1. Environment2. Membership Characteristics3. Process and Structure4. Communication5. Purpose6. Resources7 8. So we have Categories (6), Factors (20), and individualStatements (40). Analysis can then be done on a macro level,Factors or Categories , or a micro level, Statements. Respondents were asked to indicate how much they agreed ordisagreed with each statement. The rating scale used was asfollows: Strongly agree (5 points) Agree (4 points) Neutral (neither agree or disagree) (3 points) Disagree (2 points) Strongly disagree (1 point) For the purpose of this survey, dont know was added as anoption to allow respondents to indicate that they had noknowledge on a particular subject rather than a lack of opinion.Since the survey required an answer to each question, thisfacilitated the completion of the survey.8 9. The survey and our report were dividedinto six sections as follows:1. Respondent Information2. The Collaborative Factors Inventory3. Benefits and Drawbacks of Participation4. Satisfaction with Participation5. Feedback from Partners9 10. The survey was emailed to collaborative members in February2014. Members were asked to respond to the survey online. Survey response rates were as follows: 2008 - 37% response rate 2011- 46% response rate 2014 21% response rate Comment: The mailing list in 2014 has grown to 322 namesfrom perhaps 100 in the previous 2 surveys. How many of the322 are really active participants in One Voice? For any survey, however, as response rate of 21% is verygood.10 11. 11 12. 12Table 1A: Summary of Respondent DemographicsType ofParticipation(Q1: only 1answer possible2008 2011 2014Part of a non-profitorganization80% 74% 69%Part of agovernmentalagency17% 11% 18%Other 2% 5% 13% 13. 13Table 1A: Summary of Respondent DemographicsSector or Service (Q2:multiple choices werepossible)2008 2011 2014Mental Health 14% 13% 19%Healthcare 10% 13% 15%Housing 6% 6% 10%Persons with Disabilities 6% 6% 9%Substance Abuse 6% 5% 6%Seniors 9% 7% 6%Transportation 4% 2% 6%Utilities 5% 5% 6%Food 4% 2% 6%Public Education 6% 8% 5%Early ChildhoodEducation6% 8% 4%Other 10% 7% 10% 14. 14Sector or Service-Other 2014Child Welfare (deleted as a category in 2014).Health Assessment, Education & Outreach.We are a non-profit funder.Multi discipline including seniors, health care, mental health,utilities.Veterans.Adult Education.Community based organization - healthcare, seniors, trans,housing, utilities and food. 15. 15Table 1B: ParticipationLevel of Participation(Q12: one choice waspossible)% 2008 % 2011 % 2014Less than 2 meetings 19% 13% 22%2 to 6 meetings 37% 44% 38%7 to 12 meetings 22% 32% 27%13 or more meetings 22% 12% 13%Two comments about participation:(1) There was a statistically significant relationship found between ahigher level of participation and the answer to the statement Myorganization benefits from being involved.(2) The change over the past several years in participation in 13 ormore meetings may be explained by the changing frequency forworkgroup and collaborative-wide meetings. 16. 16Issue2011WeightedRank2014WeightedRankAvailability of appropriate services 292 253Sustainable funding 311 249Availability of comprehensive services286 233Capacity to deliver needed services 288 231Accessibility to needed services 286 226Affordability of services 278 176 17. 17Other Issues NotedThe ACA and Ryan White Care Act. What is the impact?We are not a direct service provider so our interests aren't really any of the above. Weare more focused on transforming the homeless response system and making sure thesupportive laws, funding and other resources are there to integrate the changes intolong term practices.Public awareness about our program.Fostering partnership and support from other organizations.Quality of HC.The need for local organizations to continue to collaborate and provide shared servicesto increase efficiency and comprehensive programming.The mentally ill have special problems as well as special needs that need to beaddressed. Co dependency being one.Medication and crisis services before people are danger to self or others.Stopping the cycle of juveniles and adults through the juvenile and criminal justicesystems, particularly those with mental health and substance abuse issues.Specialty Health Care Services for low income and Women's Health Care. 18. Looking at Statements and Categories18 19. In the 2008 and 2011 surveys, the factors with overallmeans greater than 4.0 were related to the uniquepurpose of the collaborative as demonstrated in themission statement: working together to ensure thatthe health and human service needs of all Texans areaddressed in public policy, legislative and regulatoryinitiatives. The previous 2 surveys rated the categoriesEnvironment and Purpose as 4.0, excellent. Thecategory Resources had 3 questions, 2 of whichconsistently had 30% of respondents answer DontKnow. All 3 of these categories were not included inthe 2014 survey.19 20. The 2014 survey demonstrated continuedimprovement in Process and Structure, andeffectiveness of Communication of thecollaborative. This was suggested by Statement 33, People inthis collaborative have a clear sense of their rolesand responsibilities. In 2008, no one strongly agreed with this statement. In 2011, 11% of respondents strongly agreed with thisstatement. In 2014, 16% of respondents strongly agreed with thestatement.20 21. 21The bottom line to the survey was thecumulative score for Satisfaction With OneVoice.StatementNumber ofresponses Percent of responsesCompletely satisfied 13 24%Mostly satisfied 31 56%Somewhat satisfied 9 16%A little satisfied 1 2%Not at all satisfied 0 0%Not applicable 1 2% 22. Collaborative Factors Results As a general rule, the instrument developers recommend: Scores of 4.0 or higher show a strength and probably dontneed special attention Scores from 3.0 to 3.9 are borderline and should bediscussed by the group to see if they deserve attention. Scores of 2.9 or lower reveal a concern and should beaddressed. The mean scores on the statements ranged from: 2008: a range of 2.3 to 4.2, average of 3.7 2011: a range of 2.4 to 4.5, average of 3.8 2014: a range of 2.9 to 4.4, average of 3.8 The differences are slight but interesting. In the 2014survey, the results reflect a more positive opinion overall.In 2014, 8 items were ranked below the 2008 level. Thosedifferences were typically one-tenth of a point drop.22 23. One item in the 2014 survey dropped by three-tenthsof a point; Category of Communication,Statement 43, asked: I personally have informal conversations aboutOne Voice with others who are involved in thecollaborative. Only 5% of respondents strongly agreed with thestatement. The mean of the item fell from 4.2 to3.8, which is still a strong showing. Interpretation: Unknown23 24. For the 3 surveys, the same 10 statementsranked highest, 3 related to MembershipCharacteristics (3 of 6, or 50%), 3 to Processand Structure (3 of 13, or 23%), and 4 relatedto Communication (4 of 5, or 80%). The highest ranking statements remainconsistent through the 3 surveys as shown inthe following table. However, rather than afocus on the individual statements, I believethe strengths shown in the Category list mostimportant.24 25. 25Category Highest Ranking StatementsMean Score2008Mean Score2011Mean Score2014MembershipCharacteristics24. My organization will benefit from being involved inOne Voice. 4.2 4.5 4.4MembershipCharacteristics20. I have a lot of respect for the other people involved inthis collaborative. 4.3 4.5 4.6Process and Structure26. Everyone who is a member of One Voice wants thisproject to succeed. 4.2 4.3 4.3Communication43. I personally have informal conversations about OneVoice with others who are involved in the collaborative. 3.9 4.2 3.9Communication41. The people who lead this collaborative communicatewell with the partners. 4.0 4.2 4.2Communication42. Communication among the people in thiscollaborative happens both at formal meetings and ininformal ways. 4.1 4.2 4.2Process and Structure35. This collaborative is able to adapt to changingconditions, such as fewer funds than expected, changingpolitical climate or change in leadership. 3.9 4.1 4.0MembershipCharacteristics21. The people involved in One Voice represent a crosssection of those who have a stake in what we are trying toaccomplish. 4.1 4.1 4.2Communication40. I am informed as often as I should be about what goeson in the collaborative. 3.9 4.0 4.0Process and Structure36. This group has the ability to survive even if it had tomake major changes in its plans or add some newmembers in order to reach its goals. 4.0 4.0 3.8 26. 26Category Highest Ranking StatementsMeanScore2008MeanScore2011MeanScore2014MembershipCharacteristics24. My organization will benefit frombeing involved in One Voice. 4.2 4.54.4MembershipCharacteristics20. I have a lot of respect for the otherpeople involved in this collaborative. 4.3 4.54.6Process andStructure26. Everyone who is a member ofOne Voice wants this project tosucceed. 4.2 4.34.3Communication43. I personally have informalconversations about One Voice withothers who are involved in thecollaborative. 3.9 4.23.9Communication41. The people who lead thiscollaborative communicate well withthe partners. 4.0 4.24.2Top 5 27. 27Communication42. Communication among the people inthis collaborative happens both at formalmeetings and in informal ways. 4.1 4.24.2Process andStructure35. This collaborative is able to adapt tochanging conditions, such as fewer fundsthan expected, changing political climate orchange in leadership. 3.9 4.14.0MembershipCharacteristics21. The people involved in One Voicerepresent a cross section of those who havea stake in what we are trying toaccomplish. 4.1 4.14.2Communication40. I am informed as often as I should beabout what goes on in the collaborative. 3.9 4.04.0Process andStructure36. This group has the ability to surviveeven if it had to make major changes in itsplans or add some new members in order toreach its goals. 4.0 4.03.8Next 5 28. 28CategoryLowest Ranking StatementsAverage 3surveyscoreAverage% DontKnowProcess andStructure29. Each of the people who participate in decisionsin One Voice can speak for the entire organizationthey represent, not just a part.3.2 29%Process andStructure38. We are currently able to keep up with the worknecessary to coordinate all the people,organizations, and activities related to One Voice.3.5 22%MembershipCharacteristics22. All the organizations that we need to bemembers of this collaborative have becomemembers of the group.2.9 21%Process andStructure33. People in this collaborative have a clear senseof their roles and responsibilities.3.6 18%Lowest Ranking CategoriesStatement 22, All the organizations that we need to be members of thiscollaborative have become members of the group, increased positivelymore than any other statement, an increase of 0.5 to a mean of 2.9. 29. 29Table 7: Benefits of Participation ranked according to frequencyselected (respondents could select more than one option) (ranked high tolow)Benefits 2014 % of Partners ExperiencingBenefitAcquired useful knowledge about services,programs, or people86%Enhanced ability to address an important issue 62%Developed valuable relationships 66%Provided ability to have a greater impact than I/we53%could have had on my/our ownEnhanced ability to affect public policy 55%Enhanced ability to make contribution to the51%communityHeightened public profile 40%Increased utilization of our expertise or services 29%Enhanced ability to meet the needs of our27%constituents or clientsDeveloped new skills 26%Acquired additional financial support 9%None of the above 4% 30. 30Table 8: Drawbacks of Collaborative: (could select more than one option)(ranked low to high by 2014 ranking)Choice #/Drawbacks 2008% ofPartnersExperiencinga Drawback2011% ofPartnersExperiencing aDrawback2014% ofPartnersExperiencinga DrawbackChoice 3. Viewed negatively due toassociation with other partners or thecollaborative17% (n=6) 0% (n=0) 2%(n=1)Choice 5. Received insufficient credit forcontributing to the accomplishments of thecollaborative3% (n=1) 0% (n=0) 4% (n=2)Choice 6. Created conflict between my joband the collaboratives work6% (n=2) 4% (n=1) 2% (n=1)Choice 2. Wielded insufficient influence incollaborative activities6% (n=2) 13% (n=3) 7% (n=4)Choice 4. Resulted in frustration oraggravation11% (n=4) 30% (n=7) 9% (n=5)Choice 1. Diverted time and resourcesaway from other priorities57% (n=20) 52% (n=12) 16% (n=9) 31. 31 32. 32StatementTable 9: Satisfaction Means by StatementHowsatisfied areyou with theway thepeople andorganizationsin One Voiceworktogether?(1 is excellent, 5 is poor)How satisfiedare you withyourinfluence inthecollaborative?Howsatisfied areyou withyour role inthecollaborative?How satisfiedare you withthecollaborativesplans forachieving itsgoals?How satisfiedare you withthe way thecollaborativeisimplementingits plans?2008StatementMean 2.2 2.2 2.3 2.2 2.12011StatementMean 2.1 2.2 2.2 2.0 2.02014StatementMean 2.0 2.5 2.5 Not included 2.2 33. A recommendation following the 2008 surveywas that there should be a well-plannedresponse to the feedback provided by theresponding partners. This was seen ashaving the potential to help the collaborativemove to a higher level of collaboration. Wesuggested that the collaborative administerthe collaborative inventory on an annualbasis, or as needed, to provide on-goingfeedback on the status of the organizationsfunctioning.33 34. The leadership of One Voice has doneexactly as recommended. A surveyspecific to the functioning of the workgroups was conducted in 2009. Thefollow-up collaborative survey was initiatedin 2011. The 2011 survey providedoutcome measures of the effectiveness ofthe organizational and executive-levelchanges implemented in 2008 and 2009.34 35. As noted in the previous surveys, the scores onthe Collaboration Factors Inventory are not anabsolute reflection of the collaboratives ability towork effectively. There is no critical value on eachfactor to ensure success. As suggested earlier,scores of 4.0 or higher probably indicate strengthon a factor (factor as opposed to a category).Scores from 3.0 to 3.9 ought to prompt somediscussion to determine if the collaborative needsto devote attention to them. Based on the category results, with a mean of 4.0being excellent, we find the following:35 36. 36Category FindingMembershipCharacteristicsAppropriate cross-sectionof membersMutual respect,understanding and trust3 survey mean of 3.9; median of 4.2. Only 1 of the responses fell below 3.0. Statement 22had a mean of 2.9 All of the organizations that weneed to be members of this collaborative have becomemembers of the group. And, as noted, this statementincreased positively more than any other statement, anincrease of 0.5 to a mean of 2.9, with 21% answeringDont Know. Without Statement 22, the mean is 4.15. The median is the more accurate rating for thiscategory. Possible focus area, Statement 19 People involvedin our collaborative always trust one another. Fourresponded as strongly disagree, 10 responded asdisagree with the statement. The outcome improvedfrom earlier surveys with a 3 survey average of 3.5.However, only 10% responded as Dont Know overthe 3 surveys. 37. 37Category FindingProcess andStructureMultiple layers ofdecision makingMembers share a stakein both process andoutcomes3 survey mean of 3.7, median of 3.9. The category is large, 16 statements. 1 statementwas below 3.5, Statement 29 Each of thepeople who participate in decisions in One Voicecan speak for the entire organization theyrepresent, not just a part. Over 3 surveys, 29%answered Dont Know. The statement answersreflect a normal lack of information aboutmember organizations in a large collaborativesuch as One Voice. Statement 31 The organizations that belong toOne Voice invest the right amount of time in ourcollaborative efforts, is similar. 3 survey meanof 3.4, but 23% of respondents answered DontKnow. The Median is the more accurate score. No specific intervention is indicated. 38. 38Category FindingCommunicationOpen and frequentcommunicationEstablished informalrelationships andcommunication links3 survey mean of 3.8, median of 4.0. For the 3 survey average, 4 of 5 statements ranked 4.0 orhigher. One statement ranked below 4.0, Statement 39 - Peoplein this collaborative communicate openly with oneanother. The score on this response over 3 surveys hasaveraged 3.8. Average answering Dont Know for the 3surveys is 12%. Without this statement, the overall meanscore is 4.1. The median is a more accurate measure than the mean. An area to watch is reflected by the one item in the 2014survey that dropped by three-tenths of a point, Statement43 - I personally have informal conversations about OneVoice with others who are involved in the collaborative.Only 5% of respondents strongly agreed with thestatement. The mean of the item fell from 4.2 to 3.8.The 3 survey average was 4.0, with 2% answering DontKnow. However, Statement 39, above, with only 12% answeringDont Know, may also be an area for additional study. 39. 39Table 11: Drawbacks to Participation (reference Table 8)Comment 2008Survey2011 Survey 2014SurveyChoice 2. Wielded insufficientinfluence in collaborative activities6% (n=2) 13% (n=3) 7% (n=4)Choice 4. Resulted in frustration oraggravation11% (n=4) 30% (n=7) 9% (n=5)Choice 1. Diverted time andresources away from otherpriorities57% (n=20) 52% (n=12) 16% (n=9)The drop in negative comments on Choice 1, divertedtime, is important and reflects the change in meetingformat and frequency. Also possibly the betterorganization of the workgroups. 40. I am very dismayed that all issuespertaining to seniors in our community arebeing ignored while it seems that all effortsand all committees and workgroups arebeing focused on the issues of children.While the issues of children are important, Iam very disappointed at this ongoingdevelopment.40 41. It is perhaps worth the time to track andreport by service sector (Table 1A) theprimary activities of the collaborative,noting how well the collaborative activitiesmatch (and dont match) members servicesectors. If growth is important, do we need to be moreinclusive?Bottom line, we have the data to make thatdetermination.41 42. 42Table 1A: Summary of Respondent DemographicsSector or Service (Q2:multiple choices werepossible)2008 2011 2014Mental Health 14% 13% 19%Healthcare 10% 13% 15%Housing 6% 6% 10%Persons with Disabilities 6% 6% 9%Substance Abuse 6% 5% 6%Seniors 9% 7% 6%Transportation 4% 2% 6%Utilities 5% 5% 6%Food 4% 2% 6%Public Education 6% 8% 5%Early ChildhoodEducation6% 8% 4%Other 10% 7% 10% 43. Plan to resurvey the workgroups based onthe legislative agenda submitted in 2016. Consider, as a research agenda, whysome issues succeed as regards a placeon the legislative agenda, and some do notsucceed?43