1. t t m en o i n age g P an r nin e M Tu c o f rman es eri Perfo as i n s on i r d ce T h sour re 3 Survey on Performance Management Practices in States Results of a Baseline Assessment of State Health Agencies Prepared by Public Health Foundation for the Performance Management National Excellence Collaborative TurningPoint Collaborating for a New Century in Public Health

2. The Turning Point Performance Management Collaborative Availability Online and in Print Survey on Performance Management Practices in States was produced by the Turning Point National Program Office at the To read, search, or download this report in Adobe Acrobat University of Washington. This report was researched and Reader on the Internet, visit: written by the Public Health Foundation, under contract to the http://turningpointprogram.org/Pages/pmc_state_survey.pdf Turning Point Performance Management Collaborative, funded by The Robert Wood Johnson Foundation through Turning Please direct requests for print copies of this report to: Point: Collaborating for a New Century in Public Health. Laura B. Landrum, Lead State Coordinator Public Health Futures Illinois 100 West Randolph, Ste. 6-600 Suggested citation: Chicago, IL 60601 Public Health Foundation. Turning Point Performance E-mail: llandrum@idph.state.il.us Management Collaborative Survey on Performance Management Practices in States. Seattle, WA: Turning Point National Program Office at the University of Washington, February 2002. Turning Point Survey on Performance Management Practices in States February 2002

3. Acknowledgments We would like to thank the staff in each participating state Members health agency who devoted valuable time and effort to providing quality information for this Survey and helped us Illinois: Laura B. Landrum, Coordinator; Michael C. Jones achieve such a high response rate. We are also grateful for the insights and assistance from the Turning Point Alaska: Alice Rarig, Kristin Ryan, Delisa Culpepper Performance Management Collaborative (PMC) members who contributed to the development of the Survey and this report. Missouri: Judy Alexiou, Larry D. Jones State PMC members include Alaska, Illinois, Missouri, Montana: Melanie Reynolds, Drew Dawson, Stephanie Montana, New Hampshire, New York, and West Virginia. Nelson, Jane Smilie National partners include the Association of State and Territorial Health Officials, National Association of County and New Hampshire: William Kassler, MD; Charles C. Smith City Health Officials, Centers for Disease Control and Prevention, Health Resources and Services Administration, New York: Marie D. Miller and Association of State and Territorial Local Health Liaison West Virginia: Amy Atkins, Kay Shamblin Officials. Association of State and Territorial Health Officials: We welcome your comments and questions about this report. Leslie M. Beitsch, MD, JD; Joan Brewster Please contact PHF (202-898-5600, info@phf.org) or the Turning Point National Program Office (206-616-8410, National Association of City and County Health Officials: turnpt@u.washingon.edu). Patrick Libbey The Turning Point Performance Management Collaborative Centers for Disease Control and Prevention: Paul K. Halverson, Dr.PH; Michael T. Hatcher, Dr.PH Health Resources and Services Administration: Michael L. Millman, Ph.D. Turning Point National Program Office: Jack Thompson Public Health Foundation: Stacy Baker; Ron Bialek; Kristen Hildreth; Yoku Shaw-Taylor, Ph.D.; Bernard J. Turnock, MD Turning Point Survey on Performance Management Practices in States February 2002

4. Contents Page Page Foreword............................................................................... iii Figure 10. Percentage of SHAs that use specified Introduction ............................................................................v agencies/offices for the coordination of Methods ............................................................................... vii performance management efforts ............... 13 Key Findings ..........................................................................x Figure 11. Percentage of SHAs that incorporate specified models or frameworks ................. 14 Figures and Abbreviated Titles Figure 12. Most prevalent method of collecting data .... 15 I. Performance Management Characteristics of All SHAs Figure 13. Percentage of SHAs with a system that Figure 1. Agencies/programs to which SHAs apply integrates and uses performance data ........ 16 performance management efforts .................. 2 Figure 14. Percentage of SHAs that use performance Figure 2. Extent to which SHAs apply performance reports to guide public health practice......... 17 management efforts to local public health Figure 15. Percentage of SHAs that have specified agencies ........................................................ 3 components of performance management.. 18 Figure 3. Other agencies to which SHAs apply Figure 16. Proportion of local public health budgets performance management efforts .................. 4 provided by SHAs by agencies to which Figure 4. Percentage of SHAs that conducted specified performance management is applied .......... 19 public health processes ................................. 5 Figure 5. Types of aid identified as most useful to III. Performance Management Characteristics of SHAs with improve SHA performance management ....... 6 Any Performance Management Efforts Figure 6. Percentage of SHAs that are taking action to Figure 17. Reasons for initiating and continuing improve performance management ............... 7 performance management efforts ............... 22 Figure 18. Percentage of SHAs that use incentives or II. Performance Management Characteristics of SHAs with disincentives to improve performance ......... 23 Agency Wide or Locally Applied Performance Figure 19. Percentage of SHAs that report their efforts Management Efforts resulted in improved performance............... 24 Figure 7. Percentage of SHAs that have performance management components in place .............. 10 IV. State Infrastructure Characteristics Figure 8. Areas for which SHAs are most/least likely to Figure 20. Organization of local public health services . 26 have performance mangement components. 11 Figure 21. States that have state boards of health ....... 27 Figure 9. Percentage of SHAs that provide dedicated Figure 22. Proportion of most local public health agency resources for performance management ..... 12 budgets provided or administered by SHA .. 28 Turning Point Survey on Performance Management Practices in States February 2002 i

5. Contents Page Page State Public Health Performance Management Profiles New Jersey ......................................................................... 57 Key ..................................................................................... 30 New Mexico ......................................................................... 58 Alabama ............................................................................. 31 New York ............................................................................. 59 Alaska ................................................................................ 32 North Carolina ..................................................................... 60 Arizona................................................................................ 33 North Dakota ....................................................................... 61 Arkansas ............................................................................ 34 Ohio .................................................................................... 62 Colorado ............................................................................ 35 Oklahoma ............................................................................ 63 Connecticut ........................................................................ 36 Oregon ................................................................................ 64 Delaware ............................................................................ 37 Pennsylvania ....................................................................... 65 Florida ................................................................................ 38 Rhode Island ....................................................................... 66 Georgia .............................................................................. 39 South Carolina .................................................................... 67 Hawaii ................................................................................ 40 South Dakota ...................................................................... 68 Idaho .................................................................................. 41 Tennessee .......................................................................... 69 Illinois ................................................................................. 42 Texas .................................................................................. 70 Indiana ............................................................................... 43 Utah .................................................................................... 71 Iowa ................................................................................... 44 Vermont .............................................................................. 72 Kansas ............................................................................... 45 Virginia ................................................................................ 73 Kentucky ............................................................................ 46 Washington ......................................................................... 74 Louisiana ........................................................................... 47 West Virginia ....................................................................... 75 Maryland ............................................................................ 48 Wisconsin ............................................................................ 76 Massachusetts ................................................................... 49 Wyoming .............................................................................. 77 Michigan ............................................................................ 50 Minnesota .......................................................................... 51 Appendices Mississippi ......................................................................... 52 Missouri ............................................................................. 53 A. Index of Selected State Characteristics............................ 79 Montana ............................................................................. 54 B. Glossary of Terms............................................................ 81 Nebraska ........................................................................... 55 New Hampshire ................................................................. 56 C. State Survey Instrument .................................................. 82 Turning Point Survey on Performance Management Practices in States February 2002 ii

6. Foreword On behalf of the Turning Point Performance Management management systems in public health. In 2001, the PMC Collaborative (PMC), the Turning Point National Program produced with PHF a review of the performance management Office, and the Public Health Foundation (PHF), we are literature, highlighting references from the public health, pleased to present the results of the 2001 Survey of business, government, health care, education, and non-profit Performance Management Practices in States (the Survey). sectors, available at The Survey was sponsored by the PMCa group of seven www.turningpointprogram.org/Pages/pmc_lit_synthesis.pdf. states and five national partner organizations working to study and promote systems to manage public health performance. The Survey builds on this literature review and will be a The four-year collaborative project is funded by The Robert catalyst for the next phase of the PMCs work to identify and Wood Johnson Foundation through Turning Point: develop model practices. Collaborating for a New Century in Public Health. The Survey design, administration, and analysis were conducted by PHF Implications and Uses with assistance from the Association of State and Territorial The outstanding response (94 percent) to this Survey has Health Officials (ASTHO). resulted in an unprecedented set of baseline data on state public health performance management practices. These data Purposes of the Survey expose the striking lack of information available to decision- The purpose of this Survey was to characterize state health makers trying to choose an effective approach to performance agency (SHA) efforts to measure and manage public health management in their jurisdictions. The findings underscore performance. The PMC desired this information to (1) develop SHAs desires for more information about models and best a baseline of what is happening in SHA performance practices in this arena, as well as the challenges in fulfilling management across the nation; (2) develop state performance their needs. No single performance management approach is management profiles that will help SHAs identify and learn used by most SHAs, and there are insufficient data to know from states with similar systems; and (3) set the stage for the which among the variety of SHA approaches are good models next phase of PMC workthe development of performance or ones to avoid. management resources for states based on the findings. The members of the PMC appreciate the challenge before PMC Performance Management Series them in ensuring that SHAs receive the assistance they need according to this Survey. The development of performance This report is part of a series of PMC activities to promote and management models that are feasible for implementation by advance the use of accountable performance states will test the limits of our knowledge, vision, and Turning Point Survey on Performance Management Practices in States February 2002 iii

7. creativity. The diversity of state public health systems and the practices and outcomes within the areas of performance evolution of their management systems is always a key that SHAs most and least often address (health status and consideration in evaluating and promoting state public health human resource development, respectively). practices. But the need for improved accountability and the capability of monitoring and articulating the value of public Bobbie Berkowitz, PhD health is of clear concern to states. Director Turning Point National Program Office For the many public health leaders that need current information about performance management in public health, Laura B. Landrum the state performance management profiles and other Illinois Lead State Coordinator information contained in the report are important tools. The Turning Point Performance Management Collaborative PMC is committed to continuing to gather and share information about SHA practices, as well as provide models, Ron Bialek options, and guidance for SHAs grounded in available President research. Public Health Foundation We call on our partners in academic, philanthropic, federal, and practice settings to help formulate and fund a practice- oriented research agenda to assess the effectiveness of models of performance management in public health. This report provides excellent leads for developing such an agenda and carrying out research at both the state and local levels. As examples, this report invites further exploration of the following: components of performance management (e.g., performance targets, processes for change and quality improvement), their relationship to improving performance, and ways to operationalize these components within various SHA structures; factors that may account for differences in outcomes among performance management efforts; and Turning Point Survey on Performance Management Practices in States February 2002 iv

8. Introduction Origins of Performance Measurement in Public Health 1970s, the Urban Institute worked with several state and local agencies to refine procedures and activities for monitoring Performance measurement has increasingly become a performance and tracking outcomes. The Government powerful tool for assessing program outcomes and program Performance and Results Act of 1993 provided needed activities. Indeed, the concepts of managing and measuring impetus not just to federal agencies, but to state and local performance are not entirely new and are embedded in the agencies to emphasize performance monitoring and to assess principles of evaluation, or the systematic application of outcomes. In addition to the principles of cost-effectiveness, scientific procedures for assessing the utility of programs practitioners in the public health arena rely on health-specific (Rossi and Freeman, 1989). In the public health arena, the frameworks to conduct performance monitoring. principles of performance measurement have been applied in one form or another for about 80 years now, according to Current Performance Measurement Frameworks & Tools Turnock (1997). In 1914, a survey of state health agencies documented service delivery and the development of local In the Guidebook for Performance Measurement (1999), health departments. In 1921, the American Public Health written for the Turning Point National Program Office, Lichiello Association (APHA) constituted a Committee on Municipal summarizes eight frameworks used in public health to assess Health Department Practice that developed a survey to gather performance; the list culminates in the Ten Essential Public information on service delivery of health departments in 80 Health Services established in 1994 by the Public Health cities. The reconstituted committee developed what was Functions Steering Committee. Currently, the broad called an Appraisal Form, which was, in fact, a self- objectives of the Healthy People 2010 document provide assessment tool used by local health officers to assess public guidance for action and performance monitoring. Additionally, health practice (Turnock, 1997:170-173). Subsequent the National Public Health Performance Standards Program iterations of these assessment efforts involved the (NPHPSP) developed by the Centers for Disease Control and development of an Evaluation Schedule, the Emerson Prevention (CDC) and its partners provides a uniform template Report, and the adoption of policies by the APHA to examine for effective evaluation and measurement of public health the practice of public health, especially at the local level. performance at the state and local levels. These frameworks illustrate the efforts of the performance measurement According to Hatry (1999), the current principles of results- movement in public health to assess activities and link them based performance measurement are derived from the to health outcomes through protocols, activity guidance, and principles of cost-effectiveness and program budgeting that self-assessment instruments or tools. were initiated by the RAND Corporation for the Defense Department. These principles were modified to make them Other tools or instruments for performance measurement in relevant for the non-defense sector. During the 1960s and public health include Mobilizing for Action Through Planning Turning Point Survey on Performance Management Practices in States February 2002 v

9. and Partnerships (MAPP) developed by the National characterize many aspects of organizational and system Association of County and City Health Officials (NACCHO), performance examined by state public health agencies. which incorporates the local NPHPSP instrument; Health Plan Employer Data and Information Set (HEDIS) developed by the Through its work and the Survey, the PMC has identified National Committee on Quality Assurance (NCQA); and several processes related to performance management, such Community Health Accreditation Program (CHAP) developed as performance measurement, program evaluation, by the National League of Nursing. Many of these tools have assessment and planning, and cost analysis. Although such evolved to include a significant emphasis on managing processes are usually present in state public health agencies, performance. their scope differs dramatically. The PMC has perceived that agencies often do not organize these management processes Moving from Performance Measurement to Management: in a coherent or effective way. Survey on Performance Management Practices in States In light of the array of performance management or The formation of the Turning Point Performance Management measurement information and tools, the Survey is meant to Collaborative (PMC) marked a growing desire to move the provide comprehensive baseline data on public health field of public health from simply measuring performance to performance management at the state level, and inform actively managing it across agencies and systems. The PMC practitioners and researchers about how states are currently has perceived performance management to be a fragmented measuring and managing their performance in the public state public health function that is neither well understood nor health arena. practiced in a comprehensive fashion in most states. References The PMCs definitions of performance management set forth in the Survey represent important steps to clarify what is Hatry, Harry P. 1999. Performance Measurement. Washington, D.C.: meant by performance management in public health. A The Urban Institute Press. greater emphasis on the analysis and use of performance data Lichiello, Patricia. 1999. Guidebook for Performance Measurement. is seen in the PMCs four defined components of performance A Report Submitted to the Turning Point National Program, Funded management: setting performance targets, using performance by The Robert Wood Johnson Foundation. University of Washington. measures or standards, reporting progress, and having a Rossi, Peter H. and Howard E. Freeman. 1989. Evaluation: A process for quality improvement or making changes based on th Systematic Approach, 4 edition. Newbury Park, California: Sage the performance data. By defining several areas in which Publications. public health agencies may manage performancesuch as Turnock, Bernard J. 1997. Public Health: What It Is and How It financial systems, human resource development, and health Works. Gaithersburg, Maryland: Aspen Publishers. statusthe PMC has broadened the dimensions of performance in public health. The Survey is the first to Turning Point Survey on Performance Management Practices in States February 2002 vi

10. Methods Survey Design and Input ASTHO hosted the web-based Survey and assisted PHF with the development and manipulation of the database of The Survey was developed by PHF, with the Turning Point responses. PMC providing extensive input into the design, content, and protocol of the survey by participating in conference calls and reviewing draft versions. Study Population The Survey study population was defined as the SHAs of 49 The Survey, available in both web-based and paper versions, states and the District of Columbia. For purposes of this consists of five sections: A) State Characteristics, B) Survey, state refers to states as well as the District of Components of Performance Management, C) Characteristics Columbia. One state, Nevada, was not included in the Survey of State Health Agency (SHA) Performance Management to honor that SHAs request not to be included in surveys from Efforts, D) Questions for States with Any Performance ASTHO, which was hosting the web-based Survey. On each Management Efforts, and E) Questions for All SHAs. (A chart or table presented in this report, the N represents the glossary of key terms used in the Survey is provided in number of states included in the analysis presented. Because Appendix B. See Appendix C for the Survey instrument.) of the skip patterns, not all SHAs were asked to answer all questions. The Survey was designed with skip patterns, enabling SHAs to answer only questions that pertain to their level of Survey Testing implementation of performance management efforts. Respondents to the web-based version were automatically Six individuals assisted with testing the Survey before the final taken or skipped to the next appropriate question based on deployment. All testers were current or former SHA responses. All states answered sections A and B. Those employees who had familiarity with state performance SHAs that indicated they apply performance management management systems, but who would not be any SHAs efforts SHA wide, to SHA and local public health agencies, or designated respondent. Four testers were recommended by to local public health agencies only, completed sections C, D, the Collaborative; one was identified through the Association and E. Those SHAs that indicated efforts were applied to of State and Territorial Local Health Liaison Officials; and one categorical programs only completed sections D and E. Those tester, a former employee of PHF with extensive survey SHAs that indicated efforts were applied to no agencies or experience, volunteered. The testers completed the Survey programs were skipped to section E. online and answered a short comment form about the Survey. All testers were called and asked to describe their SHAs performance management efforts. This was done to confirm Turning Point Survey on Performance Management Practices in States February 2002 vii

11. that their verbal descriptions matched their answers to Phase II: Deployment of the Survey (August 2001) question B1 about the agencies to which the SHA applies performance management efforts. Careful testing of this The Survey was sent via e-mail to the SHA in 49 states question was important because it determined which sections and the District of Columbia with the web URL for the of the Survey respondents would be asked to complete, and Survey embedded in the e-mail message, requesting triggered automatic skip patterns to appropriate sections on completion within three weeks. A paper version of the the web-based Survey. Survey was offered by request. None of the testers had any technical problems with Phase III: Extensive follow-up (August 2001 February 2002) completing the Survey online, and all completed appropriate Two days before the stated deadline, an e-mail reminder sections. The most common concern was the desire of testers was sent to the 36 SHAs who had not completed the to explain why they answered as they did. This concern was Survey, again with the URL embedded in the text. remedied by providing a comment box with question B1 and a After five weeks, 12 SHAs still had not completed the general comment box at the end of the Survey. Survey. Four SHA respondents were contacted by PMC or other Turning Point representatives. PHF contacted the Minor changes were made to the Survey based on tester eight remaining SHAs by phone and sent another e-mail comments with the Collaborative's input. with the Survey URL embedded in the message and a Word version of the Survey attached. Respondents were Survey Administration and Follow-up asked to complete the Survey within 10 days. Phase I: Identification of designated respondents (July 2001) To the six non-responding SHAs remaining after seven weeks, a final e-mail was sent to both the designated A letter was sent to senior SHA deputy directors requesting respondents and the individuals who designated them, a designated respondent for the Survey. requesting completion within approximately one week. Sixteen SHAs that did not return the Survey Respondent A total of 47 SHAs submitted Surveys (a 94 percent Form within three weeks were again contacted either by response rate). SHAs in California, the District of phone or e-mail. Columbia, and Maine did not respond. Survey data were Overall, 27 senior deputies designated alternative staff accepted between August 1, 2001, and February 1, 2002. within their division/department to complete the Survey, 17 designated themselves as Survey respondents, and 6 Survey Deployment SHAs did not return the form, so the Survey was sent to the senior deputy. Overall, the process of using a web-based survey was successful in increasing the response rate. All but three SHAs completed the Survey online. However, there were some Turning Point Survey on Performance Management Practices in States February 2002 viii

12. technical and procedural problems. These included: (1) some charts are provided on state infrastructure characteristics. SHAs delayed completing the Survey due to rerouting the Where frequencies are provided, the number of states is also Survey e-mail within their departments to find the most indicated in parentheses. Finally, state profiles for each appropriate person; (2) two SHAs asked to make changes to participating state are included, highlighting the state structure, the Survey after submission; (3) technical problems existed SHA characteristics, and components of their performance with two SHAs indicating that they had submitted the Survey, management efforts. although no record existed, and they had to resubmit their Surveys; (4) a virus disabled the server for a few days so no The Collaborative was presented with a preliminary draft Surveys could be submitted during that time; (5) nine SHAs report. Members of the Collaborative provided input on the submitted inconsistent or incomplete responses to questions layout, data presentation, and headlines that accompany each that triggered the automatic skip patterns, requiring the chart. research team to contact them to ensure they completed appropriate sections. Limitations of Survey Study limitations include the following: (1) many multiple Data Analysis Methods and Notes choice questions forced answers that may not have fully All responses submitted online were stored in the web-based captured the complexity of SHA structures, their performance survey application, Inquisite, hosted by ASTHO. These data management efforts, and varying stages of implementation of were then transferred to an Access database for analysis. performance management systems; (2) several questions Data from the three surveys submitted by fax were entered asked for estimation on the part of the respondent; (3) while manually into the Access database. the process of obtaining a designated respondent for each SHA helped to ensure that the most appropriate person Using SPSS, univariate frequencies were run for all variables. answered the Survey, the degree to which respondents were Based on Survey objectives, variables were chosen for familiar with their SHAs performance management efforts multivariate analysis and the examination of significant cannot be verified; and (4) no follow-up was made to SHAs to correlations between variables. verify reported or missing information unless there was an inconsistent response to the same question, missing data Report Design and Input on Data Presentation suggestive of a skip pattern error, or missing data to D7 (a critical question for analysis). Charts and tables are provided for performance management data according to the objectives of the Survey. Additionally, Turning Point Survey on Performance Management Practices in States February 2002 ix

13. Key Findings Characteristics of State Health Agency (SHA) Performance Over three-quarters of reporting SHAs5 incorporate one of Management Efforts three frameworks into their statewide performance Almost every reporting SHA (45) has some type of management efforts: Healthy People, Core Public Health performance management process in place. About half of Functions, or Essential Public Health Services. SHAs (25) apply performance management efforts Most reporting SHAs6 with statewide performance statewide and across programs, while 20 apply perform- management efforts dedicate staff or financial resources to ance management to only categorical programs such as the task. maternal and child health, STD/HIV, or nutrition. (N=47) Reporting SHAs1 most frequently measure, report, and use Desired Aids to Improve Performance Management Efforts performance data related to health status or their data and Funding is the number one aid SHAs report needing to information systems, often ignoring other organizational or improve state performance management efforts in public system performance measuresparticularly human health. Other top choices, in rank order, are detailed resource development. In addition, few SHAs have examples or models from other states, technical components of performance management for financial assistance, how to guides, and a set of voluntary national systems, public health capacity, or customer focus and performance standards for public health systems (tied with satisfaction. how to guides). (N=47) Fourteen SHAs2 reported having all four components3 of performance management for the SHA or local public Achieving Positive Outcomes from Performance Management health agencies (LHAs). Over three-quarters of reporting SHAs (76 percent) with Few SHAs reported having a process to conduct quality any performance management efforts say that these improvement or to carry out changes based on efforts have resulted in improved performance. Most performance data for the SHA (15) or LHAs (8).4 reports of improved performance relate to structures and 1, 2, 4, 5, 6 processes (e.g., contracting, reporting systems, policies, Based on data from SHAs with statewide, cross-cutting funding, priority setting, staff development, service performance management efforts applied to state or local public health delivery), with several states reporting improvements in agencies (N=25). 3 Components include (1) performance targets, (2) performance standards health and health-related outcomes (e.g., immunization and measures, (3) reporting of progress, and (4) a process to conduct rates, cancer death rates, cancer screening rates, coronary quality improvement or to carry out changes based on performance data. bypass surgery survival rates). (N=41) Refer to Appendix C for definitions. Turning Point Survey on Performance Management Practices in States February 2002 x

14. I. Performance Management Characteristics of All SHAs Turning Point Survey on Performance Management Practices in States February 2002 1

15. Nearly All SHAs Have Some Performance Management Efforts However, only about half apply performance management efforts statewide beyond categorical programs Figure 1. Agencies or programs to which SHAs apply performance management efforts (N=47) None 4% (2) SHA wide 32% (15) Categorical programs only 43% (20) SHA wide and local public health Local public agencies health agencies 17% (8) only Turning Point Survey on Performance Management Practices in States February 2002 2

16. Nearly All SHAs with Performance Management Efforts Aimed at Local Public Health Agencies Include All Local Public Health Agencies Figure 2. Extent to which SHAs apply performance management efforts to local public health agencies, of those states that indicated they apply performance management efforts SHA wide and to local public health agencies, or to local public health agencies only (N=10) The majority of local public health agencies 10% (1) All local public health agencies 90% (9) Turning Point Survey on Performance Management Practices in States February 2002 3

17. Performance Management Is More Often Applied to Public Health System Partners if Under SHA Contract Figure 3. Other agencies in the public health system to which SHAs apply performance management efforts (N=47) 100 90 Percentage of States (N=47) 80 68 (32) 70 60 50 40 30 (14) 30 21 (10) 20 6 (3) 4 (2) 10 0 Other State Gov't Other State Gov't Non-Gov't Agencies Non-Gov't Agencies None Agencies Under SHA Agencies Not Under Under SHA Contract Not Under SHA Contract SHA Contract Contract Note: Respondents could choose more than one response, so total does not equal 100 Turning Point Survey on Performance Management Practices in States February 2002 4

18. Most SHAs Recently Conducted a Public Health Process Related to Performance Management Figure 4. Percentage of SHAs that conducted specified public health processes related to performance management in the last 12 months (N=47) 96 (45) 100 90 81 (38) Percentage of States (N=47) 80 66 (31) 66 (31) 70 57 (27) 60 50 40 30 20 10 0 Establishing Health Public Health Cost Analysis Management Health Status Assessment Assessment Assessment Priorities & Plans Capacity Internal Turning Point Survey on Performance Management Practices in States February 2002 5

19. Funding for Performance Management Chosen as Number One Way to Improve SHAs Efforts Figure 5. Types of aid identified as most useful to SHAs to improve SHA performance management efforts, in rank order (N=47) Number of SHAs that ranked each answer 1-3 1st 2nd 3rd 1. Funding sources/support 18 7 3 2. Detailed examples/a set of models from other states 9 10 4 performance management systems 3. Consultation/technical assistance 3 5 7 4. How to guide/toolkit (tie) 4 5 3 4. A set of voluntary national performance standards for 6 1 5 public health systems (tie) Turning Point Survey on Performance Management Practices in States February 2002 6

20. Nearly Every SHA Is Taking Action to Improve the Way It Manages Performance Figure 6. Percentage of SHAs that are currently taking any actions to improve the way they manage performance (N=39) Reported SHA Actions to Improve No Performance Management: Themes 3% (1) MAJOR THEMES Learning about performance management gathering information, training staff Improving performance data systems and measures Strategic planning Conducting regular reviewsof plans, budgets, performance Developing or expanding state performance management systems MINOR THEMES Healthy People 2010 planning Using the National Public Health Performance Standards Program instrument Participating in Turning Point Assessing public health capacity Assessing clinical performance Yes Looking at personnel performance Working with policy makers or advisory groups 97% (38) Turning Point Survey on Performance Management Practices in States February 2002 7

21. Turning Point Survey on Performance Management Practices in States February 2002 8

22. II. Performance Management Characteristics of SHAs with Agency Wide or Locally Applied Performance Management Efforts SHA wide (includes local agencies operated by the state) SHA wide and local public health agencies Local public health agencies only Turning Point Survey on Performance Management Practices in States February 2002 9

23. More SHAs Have Components of Performance Management for Their Own Agency Than for Locals 14 of 25 states (56%) have all components of performance management for SHA wide or locally applied efforts Figure 7. Percentage of SHAs that have performance management components in place [targets, measures or standards, reports, and process for quality improvement (QI)/change] for SHA and for local public health agencies, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25) 100 90 Percentage of States (N=25) 76 (19) 80 (20) 80 68 (17) 70 56 (14) 60 (15) 60 44 (11) 50 40 (10) 40 32 (8) 30 20 10 0 Performance Targets Performance Measures or Performance Reports Process for QI/Change Standards SHA Local Public Health Agencies Note: For definitions of performance management components as used in the survey, see Glossary of Terms, Appendix B. Turning Point Survey on Performance Management Practices in States February 2002 10

24. SHAs Most Likely to Have Components of Performance Management for Health Status; Least Likely for Human Resource Development Figure 8. Areas most and least likely to have performance targets, measures or standards, reports, and processes for quality improvement (QI)/change, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25) Most Likely Least Likely Performance Targets Health Status Human Resource Development Data & Information Systems Public Health Capacity Performance Measures or Health Status Human Resource Development Standards Data & Information Systems Customer Focus and Satisfaction Performance Reports Health Status Human Resource Development Data & Information Systems Public Health Capacity Management Practices Process for QI/Change Health Status Human Resource Development Customer Focus and Satisfaction Public Health Capacity Management Practices Note: For definitions of performance management components as used in the survey, see Glossary of Terms, Appendix B. Turning Point Survey on Performance Management Practices in States February 2002 11

25. Most SHAs with Agency Wide or Locally Applied Performance Management Efforts Provide Dedicated Staff or Financial Resources for the Task Figure 9. Percentage of SHAs that provide dedicated resources for performance management efforts, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25) 100 90 84 (21) 80 Percentage of States (N=25) 70 60 56 (14) 50 40 30 20 10 0 Dedicated Personnel Dedicated Financial Resources Note: Dedicated personnel was defined as at least one person who spends 50 percent of his/her time on performance management efforts. Turning Point Survey on Performance Management Practices in States February 2002 12

26. Top Management Teams Top List of Agency or Office in Charge of SHA Performance Management Efforts Figure 10. Percentage of SHAs that use specified agencies or offices to coordinate and direct performance management efforts, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=24) 100 90 Percentage of States (N=24) 75 (18) 80 70 60 50 (12) 50 40 25 (6) 30 21 (5) 17 (4) 20 4 (1) 4 (1) 4 (1) 10 0 Coordination/Management Decision-Making/Strategic Direction SHA staff SHA Top Management Team Other State Agency Other SHA Staff = SHA staff within a single Bureau/Division SHA Top Management Team = interdisciplinary team from multiple Bureaus/Divisions Turning Point Survey on Performance Management Practices in States February 2002 13

27. Healthy People Objectives, Core Public Health Functions, and Ten Essential Public Health Services Top List of Models/Frameworks Explicitly Incorporated by SHAs into Their Performance Management A variety of models/frameworks, in a variety of combinations, are being used by state Figure 11. Percentage of SHAs that indicated specified models or frameworks are explicitly incorporated into their performance management efforts, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25) Healthy People 2000/2010 Objectives 80 (20) Core Public Health Functions (Assessment, Policy Development, Assurance) 76 (19) Ten Essential Public Health Services 76 (19) State-specific performance frameworks 68 (17) Community Assessment & Planning Frameworks like APEXPH, MAPP, & PATCH 52 (13) Healthy People Leading Health Indicators 48 (12) National Public Health Performance Standards Program 44 (11) HEDIS or other clinical performance measurement systems 36 (9) Federal performance frameworks, such as GPRA 24 (6) Healthy Cities/Healthy Communities 16 (4) Baldrige Award Criteria 16 (4) Other 16 (4) Balanced Scorecard 12 (3) None 4 (1) 0 10 20 30 40 50 60 70 80 90 100 Note: Respondents could choose more than one response, so total does not equal 100 Percentage of States (N=25) Turning Point Survey on Performance Management Practices in States February 2002 14

28. Paper Submission Is Still Used by Nearly One-Third of SHAs for Collecting Agency Wide or Local Performance Management Data Figure 12. Most prevalent methods of collecting data for SHA performance management efforts, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25) Other 20% (5) Paper 32% (8) Onsite visits/audits 8% (2) Electronic 40% (10) Turning Point Survey on Performance Management Practices in States February 2002 15

29. SHAs with Integrated Performance Information Systems Use Custom Software to Gather Data Figure 13A. Percentage of SHAs that have a system that integrates and uses performance data from programs, agencies, divisions, or management areas, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=24) Figure 13B. Of SHAs with such integrated performance information systems, the percentage that uses specified methods of data synthesis (N=11) A. Information System that Integrates and Uses B. How Data Are Synthesized from Multiple Programs Performance Data Across Programs Manually 18% (2) Yes 46% (11) No 54% (13) Software customed- designed 82% (9) Turning Point Survey on Performance Management Practices in States February 2002 16

30. Performance Reports Are Used in Public Health Practice Figure 14. Percentage of SHAs that use performance management reports to guide specified public health practices, of SHAs that produce performance reports and have performance management efforts targeted SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=20) 100 95 (19) 90 (18) 90 (18) 90 (18) 90 80 Percentage of States (N=20) 75 (15) 75 (15) 70 60 50 40 30 20 10 0 Drafting Developing Developing Establishing Allocating Funds Administering Legislation Admin. Agency Policy Health Priorities Programs Regulations and Plans Turning Point Survey on Performance Management Practices in States February 2002 17

31. Most SHAs Have Performance Measures, Targets, and Reports, While Fewer States Have Process for Quality Improvement or Change* Figure 15. Percentage of SHAs that have specified components of performance management for public health capacity (N=25) 100 Percentage of States (N=25) 90 80 70 60 (15) 60 44 (11) 50 40 (10) 36 (9) 40 30 20 10 0 Performance Targets Performance Measures Performance Reports Process for QI/Change or Standards *Correlation analysis revealed that there is a comparatively weak relationship between having performance targets, performance measures, or performance reports and process for quality improvement (QI)/change. That is, in general, fewer states indicated that they did have a process for change, even though they indicated having performance targets, performance measures, or performance reports. This was the case for all areas of performance management studied (Human Resource Development, Data & Information Systems, Customer Focus and Satisfaction, Financial Systems, Management Practices, Public Health Capacity, and Health Status). Figure 15 illustrates this finding. Note: For definitions of performance management components as used in the survey, see Glossary of Terms, Appendix B. Turning Point Survey on Performance Management Practices in States February 2002 18

32. Most of the Agencies to Which SHA Applies Performance Management Derive More Than Half of Their Funding from the SHA Figure 16. Estimated proportion of most local public health agency budgets provided or administered by the SHA by agencies to which performance management is applied (N=23) Estimated proportion of most local public health agency budgets in the state that are provided or administered by the SHA Agencies to which 0-25% 26-50% 51-75% 76-100% Don't Not performance management Know Applicable is applied SHA Wide 3 1 2 6 1 1 21% 7% 14% 43% 7% 7% SHA Wide and Local 3 3 1 Public Health Agencies 43% 43% 14% Local Public Health Agencies 1 1 Only 50% 50% Total 7 2 5 6 2 1 23 Percent of Total 30% 9% 22% 26% 9% 4% Note: Due to rounding, percentages may not add up to 100 Turning Point Survey on Performance Management Practices in States February 2002 19

33. Turning Point Survey on Performance Management Practices in States February 2002 20

34. III. Performance Management Characteristics of SHAs with Any Performance Management Efforts SHA wide (includes local agencies operated by the state) SHA wide and local public health agencies Local public health agencies only Categorical programs only (e.g., MCH, STD/HIV, nutrition) Turning Point Survey on Performance Management Practices in States February 2002 21

35. Quality Improvement and Health Status Improvement Are the Primary Reasons for Most SHAs to Initiate and Continue Performance Management Efforts Figure 17. SHA reasons for initiating and continuing performance management efforts, in rank order (N=42) Number of SHAs that ranked each answer 1-3 1st 2nd 3rd 1. Improve quality and/or performance Initiating 10 7 9 Continuing 13 5 10 2. Improve community health status Initiating 11 4 7 Continuing 15 3 6 3. Ensure accountability to legislature and policy Initiating 7 9 7 Continuing 5 10 4 makers or as a requirement of legislation Turning Point Survey on Performance Management Practices in States February 2002 22

36. Most SHAs Use Neither Incentives nor Disincentives to Improve Performance Figure 18. Percentage of SHA performance efforts that include incentives or disincentives to improve performance (N=40) 100 90 80 Percentage of States (N=40) 70 63 (25) 60 50 40 30 (12) 30 20 (8) 20 13 (5) 8 (3) 10 0 Incentives for Incentives for Staff Disincentives for Disincentives for Staff None Agencies, Programs, Agencies, Programs, Divisions Divisions Note: Respondents could choose more than one response, so total does not equal 100 Turning Point Survey on Performance Management Practices in States February 2002 23

37. Performance Management Efforts Result in Improved Performance for Three-Quarters of SHAs Most improvement reported in service delivery, administration/management, and policy Figure 19. Percentage of SHAs that report their performance management efforts resulted in improved performance (N=41) No Reported Outcomes Resulting From SHA Performance Management Efforts: 24% (10) Themes MAJOR THEMES Improved delivery of servicesprogram services, clinical preventive services, essential services Improved administration/management contracting, tracking/reporting, coordination Legislation or policy changes MINOR THEMES Fundingnew or sustained allocations Staff development Yes Improved health outcomes 76% (31) Turning Point Survey on Performance Management Practices in States February 2002 24

38. IV. State Infrastructure Characteristics Turning Point Survey on Performance Management Practices in States February 2002 25

39. Organization of Local Public Health Services in States Figure 20. Organization of local public health services in states (N=47) Centralized, 21% (10)Local public health services are provided through units and/or staff of the SHA Decentralized, 45% (21)Local public health services are provided through agencies that are organized and operated by units of local government Shared authority, 11% (5)Local public health services are subject to the shared authority of both the state agency and the local government Mixed authority, 23% (11)Local public health services are provided through agencies organized and operated by units of local governments in some jurisdictions and by the state in other jurisdictions No data (4) Turning Point Survey on Performance Management Practices in States February 2002 26

40. States That Have State Boards of Health Figure 21. States that have state boards of health (N=47) Yes40% (19) No60% (28) No data (4) Turning Point Survey on Performance Management Practices in States February 2002 27

41. Estimated Proportion of Public Health Budgets for Most Local Public Health Agencies That Are Provided or Administered by State Health Agencies Figure 22. Estimated proportion of public health budgets for most local public health agencies in states that are provided or administered by state health agencies (N=45) 0-25%24% (11) 26-50%16% (7) 51-75%20% (9) 76-100%27% (12) Dont know7% (3) Not applicable7% (3) No data/not answered (6) Note: Due to rounding, percentages do not add up to 100 Turning Point Survey on Performance Management Practices in States February 2002 28

42. State Public Health Performance Management Profiles Profiles are provided for each state that SHAs that apply performance management completed the Survey on Performance SHA wide and/or to local public health agencies Management Practices in States. Responses were asked to answer several questions that are used for profiles were chosen to help SHAs represented in the profiles. identify and learn from states with similar systems. States that apply performance management to categorical programs only (e.g., MCH, State characteristics are p