21
  Report to The Vermont Legislature      Medication Management Pilot Study In Accordance with Act 63 (2011), Sec. E.311, An Act Relating to Making Appropriations for the Support of Government Submitted to: House Committee on Health Care Senate Committee on Health and Welfare Submitted by: Harry Chen, MD Commissioner Prepared by: University of Vermont Area Health Education Center (AHEC) Report Date: January 15, 2014                         108 Cherry Street PO Box 70 Burlington, VT 05402 healthvermont.gov

Medication Management Pilot Study In Accordance with Act 63

Embed Size (px)

Citation preview

Page 1: Medication Management Pilot Study In Accordance with Act 63

 

 Report to 

The Vermont Legislature  

 

   

Medication Management Pilot Study

In Accordance with Act 63 (2011), Sec. E.311, An Act Relating to Making Appropriations

for the Support of Government

Submitted to: House Committee on Health Care

Senate Committee on Health and Welfare Submitted by: Harry Chen, MD Commissioner Prepared by: University of Vermont Area Health Education Center (AHEC) Report Date: January 15, 2014

    

        

  

   

   

   

 

108 Cherry Street PO Box 70 Burlington, VT 05402 healthvermont.gov

Page 2: Medication Management Pilot Study In Accordance with Act 63

  

MEDICATION MANAGEMENT PILOT STUDY

Grant #03420-5852S

Legislative Report to the Vermont Department of Health (VDH) January 2012 – December 2013

Submitted to:

Debra Wilcox, RN, BSN, MSPH Director of Planning and Healthcare Quality

Vermont Department of Health P.O. Box 70

Burlington, VT 05402

Prepared by: Amanda G. Kennedy, PharmD, BCPS Associate Professor of Medicine UVM College of Medicine Office of Primary Care Arnold 5, UHC Campus 1 South Prospect Street Burlington, VT 05401 Program contact information: Elizabeth Cote Director, UVM College of Medicine Office of Primary Care University of Vermont UHC Campus--Arnold 5 1 South Prospect Street Burlington, VT 05401 (802) 656-2179

Page 3: Medication Management Pilot Study In Accordance with Act 63

Table of Contents Executive Summary 3Purpose of the Report 4Act 63 (2011) Sec. E.311 4Project Specific Aims 4Results of Deliverables: Planning Phase 6Results of Deliverables: Implementation and Data Collection Phase 9Results of Deliverables: Evaluation Phase 10Summary of Major Findings 18References 19

Page 4: Medication Management Pilot Study In Accordance with Act 63

Executive Summary This grant supported the University of Vermont College of Medicine’s Office of Primary Care to conduct a population-based medication management pilot study in collaboration with selected primary care practices and pharmacists over a two year period. The broad goal of this pilot was to improve care for Vermonters, specifically related to optimizing medications in primary care. To support this goal, the pilot implemented and evaluated several approaches to pharmacist collaboration with primary care. The primary measure of success was the identification of real and potential problems relating to medications, commonly referred to as “drug therapy problems.” The approaches in the project reflected the pharmacists’ backgrounds and the types of practices served. Seven demonstration sites were enrolled representing five Vermont counties. The pharmacist types included academic, community, and hospital pharmacy. The practices included private, federally-qualified health center (FQHC), practices that train medical residents, and hospital-owned. The pharmacists identified over 700 drug therapy problems related to dosing, adherence, unnecessary drug therapy, etc. Identification of drug therapy problems occurred through three strategies: 1) direct patient care, 2) population-based strategies, and 3) prescriber and patient education. Common population-based strategies included dose adjustments and discontinuing unnecessary medications. The problems identified by pharmacists involved medications common in primary care, such as cardiovascular or diabetes medications, medications for mental health indications (e.g. depression), and high-risk medications (e.g. anticoagulants). Pharmacists’ recommendations to correct drug therapy problems were accepted by prescribers 86% of the time. Prescribers, staff, and patients overwhelmingly perceived the pharmacists as having value. It is estimated that $2.00 in cost was avoided for every $1.00 spent on a pharmacist. Major Recommendations Integrating pharmacists into primary care is feasible, reveals important drug therapy problems

that impact Vermonters, and results in overall cost-avoidance. Offering pharmacy services to primary care practices is recommended, including direct patient care (especially during transitions of care), population-based management, and patient and prescriber education.

Sustainability will require the identification of alternative funding mechanisms that do not rely on a fee-for-service approach. This is essential to preserve a pharmacist’s ability to provide population-based management and educational services in addition to direct patient care.

While larger practices may benefit from a full-time pharmacist, smaller practices may be served well with a pharmacist as a shared resource. A ratio of 1 pharmacist full-time equivalent (FTE) to 5 physician FTEs, or 1 pharmacist to 8,000-10,000 patients is recommended.

Pharmacists should ideally be full-time in primary care, even if serving multiple practices. At a minimum, individual pharmacists should be 0.5 FTE.

It is recommended that pharmacists be either specialty trained in primary care (e.g. primary care residency) or employed by the same organization as the practice(s). For pharmacists without specialty training in primary care, prior job experience interacting with physicians as part of an inter-professional team (e.g. participation in hospital rounds) is recommended.

Pharmacists must have full permissions to access and document findings in the practices’ electronic health records. It is unlikely pharmacists can be successful in fully integrating into primary care practices without these permissions.

It is recommended to follow the recently published Guidelines for Pharmacists Integrating into Primary Care Teams, published by the Canadian Pharmacists Journal, November 2013.[1]

Page 5: Medication Management Pilot Study In Accordance with Act 63

Purpose of the Report The purpose of this report is to provide an overview of results for the Medication Management Pilot Study and to respond to the reporting requirements detailed in Act 63. This report details all project phases and covers a timeframe of January 2012 through December 2013. Act 63 (2011) Sec. E.311 POPULATION-BASED MEDICATION MANAGEMENT PILOT PROJECT

a) As part of the evidence-based education program established in subchapter 2 of chapter 91 of Title 18, the Department of Health, in collaboration with the Department of Vermont Health Access and the University of Vermont Office of Primary Care, shall establish a population-based medication management pilot project to include a collaborative pharmacist practice using principles consistent with the Vermont Blueprint for Health.

b) The Department of Vermont Health Access shall fund the pilot project from the fee established in 33 V.S.A. § 2004 and shall transfer funds to the Department of Health for implementation of the pilot.

Sec.15a. POPULATION-BASED MEDICATION MANAGEMENT PILOT: REPORT a) January 15, 2014, the Department of Health, University of Vermont Office of Primary Care,

and the Joint Fiscal Office shall provide a report to the House Committee on Health Care and the Senate Committee on Health and Welfare describing and evaluating the effects of the population-based medication management pilot program.

b) The report shall describe how the pilot project is implemented, including which medications were targeted. The report shall assess the pilot program in terms of improvements to patient care and increases in evidence-based prescribing through improvements to prescriber-pharmacist communication and collaboration.

Project Specific Aims This grant supported the UVM Office of Primary Care to conduct a population-based medication management pilot study in collaboration with selected primary care practices and pharmacists over a two year period. The broad goal of this pilot study was to improve care for Vermonters, specifically related to optimizing medication regimens in primary care settings. To support this goal, the pilot described and evaluated varying models for pharmacist collaboration with primary care practices. Health information technology, including population-based medication reports from primary care practices’ electronic health records, served as a tool for the pharmacist collaborations. The selected demonstration sites included primary care practices from multiple Vermont counties. The evaluation of the pilot study identified the strengths and weaknesses of various pharmacist/primary care collaborative models related to costs, clinical outcomes, patient and healthcare professional satisfaction, and model sustainability. This pilot study was collaborative between pharmacists and primary care practices and was supported by a broad foundation of stakeholders, using the expertise and opinions of the Vermont Blueprint for Health and Vermont Academic Detailing Program Advisors, including the Department of Vermont Health Access and the Vermont Department of Health. The specific aims of this project were to:

1. Design and implement several models of pharmacist collaboration in primary care 2. Evaluate the impact of the pharmacist models across multiple outcomes 3. Develop and pilot a population-based approach to managing medications in primary care

using health information technology 4. Collaborate with stakeholders to inform the process and disseminate findings

Page 6: Medication Management Pilot Study In Accordance with Act 63

Background Reports A full project protocol, including the documents listed below and background references, is available from Dr. Kennedy upon request. For a complete history of this project please request the following documents:

• Generic Drug Voucher Pilot: An evidence-based advisory report to the Vermont Department of Health, submitted April 30, 2010.

• Implementation of Option 3 of UVM’s Generic Drug Voucher Advisory Report, submitted to the Vermont Department of Health on June 29, 2011.

• Medication Management Pilot Study Phase 1 (Planning) Progress Report, submitted to the Vermont Department of Health on July 31, 2012.

• Medication Management Pilot Study Phase 2 (Implementation and Data Collection), submitted to the Vermont Department of Health on July 30, 2013.

Table 1. Attachment A: Deliverable Overview

ID Phase Deliverable 1A 1 (Planning) List of demonstration site participants 1B 1 (Planning) Summary of baseline data for each demonstration site 1C 1 (Planning) Documentation of orientation attendance by participants 1D 1 (Planning) Description of the pharmacy network progress and plans 1E 1 (Planning) Summary of practical applications from literature for primary care

practices 2A Phase 2 (Implementation

and Data Collection) Description of the models piloted including which medications were targeted

2B Phase 2 (Implementation and Data Collection) 

Overview of types of data collected

2C Phase 2 (Implementation and Data Collection) 

Description of the pharmacist integration into Community Health Teams for each demonstration site

3A Phase 3 (Evaluation) Evaluation focused on: • Costs • Improvements to patient care • Patient satisfaction • Primary care provider and pharmacist satisfaction • Increases in evidence-based prescribing through

improvements to prescriber-pharmacist communication and collaboration.

• Strengths and weaknesses of various pharmacist models of collaboration between primary care practices and pharmacists

• Sustainability of the collaborative models • Data about how health information technology may be

utilized to support population-based medication management in primary care

• Plan for continued collaboration with broadly-represented stakeholders to share best practices and foster integration of pharmacy services into primary care practices and the Vermont Blueprint for Health.

Page 7: Medication Management Pilot Study In Accordance with Act 63

Results of Deliverables: Planning Phase The results of the Medication Management Pilot are presented in order of deliverables required by Attachment A of the grant (Table 1). Please note that the results presented here may vary from previously submitted progress reports, as these results represent final project data unless otherwise indicated. 1A-1B: Demonstration Sites and Baseline Data In total, seven demonstration sites were enrolled representing five Vermont counties (Table 2). However not all sites contributed the same length of time to the study. Milton Family Practice exited the project in the fall of 2012, due to the pharmacist leaving her academic position. Porter Hospital did not begin collecting data until spring 2013. No sites are currently collecting data.

Table 2. Description of Demonstration Sites and Baseline Data County  Practice*  Type  Ownership Pharmacist Project

Dates** Approx. Annual Visits***

Addison  Porter-owned practices 

Internal & Family Med 

Hospital: Porter Medical Center

Hospital: Porter Medical Center  

1/13-8/13 --

Caledonia  Corner Medical Center 

Family Med  Hospital: Northeastern Vermont Regional Hospital (NVRH)  

Hospital: NVRH  5/12-6/13 19,100

Chittenden  Aesculapius Medical Center 

Internal Med Hospital: Fletcher Allen Health Care (FAHC)  

Hospital: FAHC  

5/12-6/13 22,500

Chittenden  Given Burlington Internal Med Resident  

Hospital: FAHC Hospital: FAHC 5/12-8/13 23,000

Chittenden  Milton Family Practice 

Family Med Resident 

Hospital: FAHC Academic: Albany College of Pharmacy & Health Sciences

5/12-10/12 30,000

Franklin  Cold Hollow Family Practice  

Family Med  Private Community: Rite Aid  10/12-5/13 --

Rutland  Brandon Medical Center 

Family Med  FQHC: Community Health Centers of the Rutland Region (CHCRR)

Community: CHCRR 5/12-8/13 12,200

*We gratefully acknowledge Fletcher Allen Pharmacy Director, Karen McBride, for her generous donation of a Fletcher Allen pharmacist to open Aesculapius Medical Center as an additional demonstration site. **Please note: Phase 1 began in January 2012, however not all sites were prepared to collect data before the Phase 2 start date of July 2012. ***Porter Medical Center and Cold Hollow Family Practice contributed less than 6 months of data to the project and have not been included in the final data analysis. 

1C-1D: Orientation and Pharmacist Network All enrolled sites and pharmacists were invited to attend an orientation meeting at the University of Vermont on January 30, 2012 (meeting minutes are available upon request). This meeting included introductions, a project overview, proposed evaluation measures, and a general discussion. This meeting was well attended and helped set a foundation for the pharmacists to begin partnering with the physicians and other prescribers in the practices. Dr. Kennedy maintained close contact with practices throughout the project. The project pharmacists met weekly throughout the project by online conference call. The purpose of meeting weekly was to develop a pharmacist network for sharing of best practices, protocols, barriers,

Page 8: Medication Management Pilot Study In Accordance with Act 63

successes, etc. Three technologies were used to support the network. First, the online conferencing service, GoToMeeting, was used for the weekly meetings. This online platform allowed the pharmacists from across the state to meet “live” without the need to travel. Second, a Google Blog was established for sharing meeting minutes and for facilitating discussions among the pharmacists. The blog was set up as a private blog, with access granted only to the project pharmacists and investigators. Although the pharmacists were aware not to disclose Protected Health Information (PHI) in the blog, they were encouraged to present de-identified challenging cases for discussion. Lastly, an online file sharing tool, Dropbox, was set up for the participating pharmacists. The Dropbox allowed the pharmacists to share protocols, relevant articles, educational materials, and other electronic files with each other. Sharing resources was encouraged as a way to maximize efforts and efficiency. Dropbox was a valuable tool and information repository for the pharmacists, as protocols and documents could be shared easily across institutions and practices. The pharmacist network was also extended to faculty at Albany College of Pharmacy and Health Sciences. Michael Biddle, PharmD, BCPS, Assistant Professor of Pharmacy Practice, regularly participated in the network. Dr. Biddle maintains a faculty practice site at Richmond Family Medicine in Richmond, Vermont. 1E: Summary of the Literature Primary care patients in the United States consume a staggering number of medications. Seventy-five percent of all primary care visits include prescribing or continuing medications.[2] Nearly a quarter of all children under 18 years use one prescription medication, 90% of all adults 65 years and older use one prescription medication, and nearly 40% of all adults 65 years and older use five or more prescription medications.[3] Among adults who use prescriptions medications, one study estimated 46% of patients also use over-the-counter medications and 52% use dietary supplements.[4] There is a critical need for primary care medication management services, given the association between the number of medications used and frequency of drug-related problems, such as adverse events, medication errors, drug interactions, drug-disease interactions, and non-adherence.[5] Pharmacists are being increasingly recognized for their services beyond dispensing. Evidence suggests pharmacists in ambulatory settings reduce hospital and emergency room admissions, decrease the use of non-scheduled health services, decrease the number and cost of drugs, and improve prescribing.[6, 7] Pharmacists have been shown to optimize patient outcomes such as blood pressure[8-15], cholesterol[13, 14, 16], diabetes[17, 18], and smoking cessation.[13, 14, 19] Additionally pharmacists have improved patient safety by reducing medication errors[20], improving laboratory monitoring for medications[21], dose adjusting for renal dysfunction[22], stopping medications[23], reducing inappropriate prescribing[24], improving adherence[25], and reducing costs.[23, 26] With primary care adopting innovative care models, including patient-centered medical homes, and Accountable Care Organizations (ACO), there are new opportunities to integrate the unique skills of pharmacists as part of the primary care team. To integrate successfully, pharmacists must practice at the full scope of their license, have support from primary care providers, patients, and other stakeholders, and have payment mechanisms that align with the highest yield activities.[27-29] Pharmacist Role in Primary Care A growing body of evidence supports pharmacists’ activities in primary care. We loosely define the role of a pharmacist in primary care in three ways: providing direct patient care, providing population-based medication management services, and prescriber education. Smith et. al. integrated pharmacists into four federally qualified health centers and one private practice in Connecticut.[30] Over the course of one year the pharmacists intensively worked with 88 Medicaid patients, with an

Page 9: Medication Management Pilot Study In Accordance with Act 63

estimated savings of $1,600 per patient. Kozminski et al. integrated pharmacists into four patient-centered medical homes in Pittsburgh, Pennsylvania.[31] The pharmacists provided direct patient care, including comprehensive medication therapy management and medication reconciliation during transitions of care. Qualitative results from prescribers and staff included positive overall benefits of the integrated pharmacists, perceived time-savings, and improved workflow. It took about six months for pharmacists to feel they had integrated into the practices. Academic detailing is an evidence-based prescriber education and support strategy for improving prescribing.[32, 33] Academic detailing draws on the strategy of person-to-person social marketing and provides prescribers with the most up-to-date, evidence-based information, to support effective prescribing.[34] Pharmacists have long been recognized as an ideal group for providing academic detailing. Combining academic detailing with direct patient care and population-based medication management is likely synergistic for optimizing patient outcomes. Population management is a relatively novel, but important task for pharmacists practicing in primary care. One study found population management in combination with academic detailing for generic substitution, therapeutic interchange, and targeting drugs based on evidence-based guidelines improved generic utilization, blood pressure control, and cost avoidance for deep vein thrombosis.[35] Qualities of Successful Pharmacists In November 2013, guidelines for pharmacists integrating into primary care were published in the Canadian Pharmacists Journal.[1] The guidelines suggest 10 recommendations. The guidelines also stress that pharmacists’ personalities influence the level of success achieved in primary care integration, with a suggestion that individual pharmacists should have personality traits that include assertiveness and confidence. Table 3. Recommendations for Pharmacist Integration into Primary Care[1] Number Recommendation

1 Determine the needs and priorities of the team and its patients 2 Develop a pharmacist job description 3 Educate the team about the pharmacist role 4 Educate yourself about the roles of the other team members 5 Ensure clinic infrastructure supports the pharmacist role 6 Be highly visible and accessible 7 Ensure your skills are strong 8 Provide proactive care and take responsibility for patient outcomes 9 Regularly seek feedback from the team

10 Develop and maintain professional relationships -- Pharmacists should possess the qualities of assertiveness and confidence

Pharmacist Workforce An opinion statement of the American College of Clinical Pharmacists suggests pharmacists in primary care should practice with a ratio of 1 pharmacist full time equivalent (FTE) to 5,000-6,000 patients.[36]

Page 10: Medication Management Pilot Study In Accordance with Act 63

Results of Deliverables: Implementation and Data Collection Phase 2A-2B: Description of the Implemented Models Targeted Medications, and Data Collected The different models enrolled in the project reflect the pharmacists’ backgrounds and the types of practices served (Table 4). The pharmacist types included academic, community, and hospital pharmacy. Within community pharmacy, we selected a community pharmacist working for a federally-qualified health center and a traditional community or retail pharmacist. Academic, community, and hospital pharmacy represent the major pharmacist workforce opportunities in Vermont. The pharmacists provided services to three different types of practice sites: individual primary care practice sites, individual practices that primarily train medical residents, and all practices owned by a hospital. Table 4. Piloted Models Academic

pharmacist FQHC

Community pharmacist

Chain Drug Store Community pharmacist

Hospital pharmacist

One practice

Brandon

Medical Center

Cold Hollow

Family Practice

Aesculapius Medical Center

Corner

Medical Center One resident practice

Milton

Family Practice

Given Burlington

All hospital-owned practices

Porter

Medical Center

The pharmacists were required to engage with the primary care practices in three ways: population-based medication management, individual or direct patient care, and prescriber education. The amount of time spent in each task was left to the pharmacists and the primary care practices. Targeted medications included those common to primary care (e.g. cardiovascular, diabetes, mental health) or medications associated with a high risk of adverse events (e.g. anticoagulants, insulin, medications associated with FDA warnings). This project collected data using numerous methods:

• Assurance™ software: This HIPAA-compliant software enabled pharmacists to document their direct patient care interventions related to optimizing medications.

• IVents: This Fletcher Allen-specific tool was integrated into PRISM, the hospital’s electronic health record, allowing the Fletcher Allen project pharmacists to efficiently document their interventions.

• Microsoft Excel: Pharmacists created spreadsheets to track population-based activities and pharmacist time.

• Patient Surveys: The pharmacists piloted a medication adherence survey and satisfaction survey.

• Online Surveys: At entry and exit to the project, practice providers and staff were encouraged to provide their perceptions of having a pharmacist in primary care.

• Interviews: At project exit, Dr. Kennedy conducted interviews with selected prescribers, staff, and project pharmacists to understand the strengths and weaknesses of pharmacists in primary care.

2C: Pharmacist Integration into Blueprint Community Health Teams (CHTs)

Page 11: Medication Management Pilot Study In Accordance with Act 63

Dr. Kennedy organized meetings between the pharmacists and their Community Health Team contacts. The Fletcher Allen site pharmacists met with Pam Farnham, RN, on June 20, 2012 and August 1, 2012. The Corner Medical pharmacists met with Laural Ruggles, MBA, on June 29, 2012. The Brandon Medical Center pharmacist met with Mary Lou Bolt on July 12, 2012 and March 28, 2013. The Cold Hollow pharmacist met with Candace Collins on October 25, 2012. Dr. Kennedy participated in each of these meetings. Although no formal relationships were established (e.g. formal processes for referring patients), both the pharmacists and CHTs were interested in understanding how formal collaborations might develop in future projects. Results of Deliverables: Evaluation Phase 3A: Evaluation Improvements to patient care and increases in evidence-based prescribing through improvements to prescriber-pharmacist communication and collaboration. Identification of drug therapy problems occurred through three routes: direct patient care, population-based strategies, and education. See Table 5. Common population-based strategies included dose adjustments and discontinuing unnecessary medications. Table 5. Pharmacist Activity Types

Activity N (%)

Direct Care 336 (47.5)

Population-based 276 (38.9)

Education 96 (13.6)

Total 708 (100)

The pharmacists identified 708 drug therapy problems related to dosing, adherence, unnecessary drug therapy, etc. See Table 6. The problems identified by pharmacists involved medications common in primary care, such as cardiovascular (e.g. blood pressure, cholesterol) or diabetes medications and medications for mental health indications (e.g. depression). See Table 7. Pharmacists’ recommendations to correct drug therapy problems were accepted by prescribers 86% of the time, where data about acceptance is known. Of the 49 recommendations not accepted, 47/49 (96%) were population-based and 2/49 (4%) were related to direct patient care.

Page 12: Medication Management Pilot Study In Accordance with Act 63

Table 6. Pharmacist-identified Drug Therapy Problems Drug Therapy Problem N (%)

Dosage Too High 150 (21.2)

Adherence 105 (14.8)

Unnecessary Drug Therapy 100 (14.1)

Specific Interventions Missing (e.g. curbside consult)

96 (13.6)

Different Drug Needed 72 (10.2)

Education 53 (7.5)

Need Additional Drug Therapy 50 (7.1)

Drug Information Question 39 (5.5)

Adverse Drug Reaction 24 (3.4)

Dosage Too Low 19 (2.7)

Total 708 (100)

Table 7. Medication Categories

Medications N (%)

Mental Health and Insomnia 179 (25.3)

CV and Diabetes 177 (25.0)

GI 105 (14.8)

Asthma/Allergy 72 (10.2)

Anticoagulants 31 (4.4)

Immunizations 21 (3.0)

Benzos and Opiates 16 (2.3)

Other 107 (15.1)

Total 708 (100)

Costs The cost evaluation to date is preliminary, as the data are being independently reviewed by another pharmacist to verify the results. Additionally the cost evaluation only included the 5 study sites that contributed at least 6 months of data (i.e. Cold Hollow and Porter Medical Center were not included). Pharmacist Salary Five sites partnered with a pharmacist one day per week (0.2 FTE) for 12 months is equivalent to one full-time pharmacist. A typical hospital pharmacist’s annual salary (1.0 FTE plus 33% fringe benefits rate) is approximately $176,690. Costs Avoided Costs avoided were calculated for the recommendations that were accepted by physicians (N=309) or where the acceptance data were missing (N=350). Recommendations not accepted by prescribers (N=49) were not included in this analysis. We used a Veterans Administration (VA) perspective specific to primary care for the cost analysis, as these data were rigorously structured and may be representative of an Accountable Care Organization. Each drug therapy problem was categorized as having an outcome using the categories provided by Lee et al. See Table 8. Data were inflated to 2013 dollars.[37] Applying the VA approach to the interventions recommended by a pharmacist in the current project resulted in a total cost avoidance of $356,294. See Table 9.

Page 13: Medication Management Pilot Study In Accordance with Act 63

Table 8. Examples of Outcomes

Outcome Example

Untreated diagnosis

Patient on simvastatin 80 mg for >1 year. Patient's LDL has not been at goal of <100 since 2008. Recommended to change to a more potent statin (rosuvastatin 10 mg daily). This change was made. Repeat lipid panel 3 months later revealed LDL = 80.

Prevent or manage adverse drug event

Enoxaparin dose unclear. The discharge instructions stated 50 mg twice daily and she was on 120 mg once daily in the hospital. Based on her weight and renal function, 40 mg once daily is recommended for DVT prophylaxis. Specialist physician adjusted the dose to 40 mg once daily.

Average of any intervention

85 year old female. Reviewed med list with patient. Patient admits to regularly taking all meds each day and understands indications. Answered questions regarding losartan and association with increasing potassium (including potassium content in foods).

Avoid drug interaction

Patient on simvastatin 40 mg + gemfibrozil 600 mg BID. Lipid panel revealed TG=265 (ranged from 210-350 since 2003). Patient also has uncontrolled diabetes that could be impacting TG control. Suggested d/c of gemfibrozil. Gemfibrozil was discontinued.

Adjust dose or frequency

Suggested re-assessment of zolpidem dose/hx of treatment. Consider decrease in dose to 5 mg. MD decreases dose to 5 mg and is considering discontinuing.

Drug not indicated

Chart review and did not reveal evidence of Zollinger-Ellison Syndrome, recurrent PUD, prevention of NSAID induced peptic ulcer, or GERD with esophagitis, ongoing symptoms, or complications such as Barrett's esophagus. Recommended tapering of their proton pump inhibitor. Physician agreed with the recommendation.

Information Only

Question from physician: How often is thrombocytopenia seen in patients on plaquenil that’s attributable to the med? All medications in the 4-aminoquinoline class including plaquenil can cause thrombocytopenia. Most information is based on long term use and FDA post marketing reports. Thrombocytopenia occurs in approximately 1% of patients who take plaquenil.

Table 9. Cost Evaluation

Recommendations Costs Avoided per Recommendation

N Total Costs Avoided

Untreated diagnosis $1,919.69 44 $84,466

Prevent or manage adverse drug event $695.71 157 $109,226

Average of any intervention $567.21 161 $91,321

Avoid drug Interaction $411.46 62 $25,511

Adjust dose or frequency $375.11 96 $36,011

Drug not indicated $94.75 103 $9,759

Information only $0.00 36 $0

Total 659 $356,294

Reference for cost data: Lee AJ, et al. Clinical and economic outcomes of pharmacist recommendations in a VA medical center. Am J Health Syst Pharm. 2002 Nov 1;59(21):2070-7. PMID: 12434719 Return on Investment (ROI) Preliminary analyses suggest $2.00 in cost was avoided for every $1.00 spent on a pharmacist ($356,294/$176,690). If the analysis assumed the missing data would be accepted at 86% (i.e. the same proportion as the known data) the estimated cost avoidance would be $323,779 with an estimated R0I of $1.83 ($323,779/$176,690). One of the 5 sites contributed only 6 months of data (N=64 recommendations). If those recommendations were doubled to estimate 1 year of data, and the missing data were considered accepted at 86%, the estimated cost avoidance would be $357,191 with an estimated R0I of $2.02 ($357,191/$176,690).

Page 14: Medication Management Pilot Study In Accordance with Act 63

Patient satisfaction Pharmacists were encouraged, but not required, to mail satisfaction surveys to the patients that participated in direct patient care activities with a pharmacist. See Table 10. The satisfaction survey was adapted from those used previously in the literature.[38-40] Five patients returned surveys. Although too few surveys were collected to permit a statistical analysis, all were very positive about their interactions with a pharmacist in primary care. Table 10. Patient Satisfaction with Pharmacists’ Care (N=5 patients) Question Score

1. I am satisfied with the care that I received at my medication consult

4.4

2. The pharmacist explained things in a way that was easy to understand

4.4

3. The medication consult increased my understanding of my medications

4.4

4. The medication consult increased my understanding of my medical conditions

4.2

5. The medication consult provided me with information that I had not learned from my other health care providers

4.2

6. I was comfortable speaking to the pharmacist and asking her questions

4.8

7. I would recommend a medication consult to friends and relatives

4.6

8. I am satisfied with the amount of time that I spent with the pharmacist

4.4

9. I would be willing to pay a copay to meet with a pharmacist for a medication consult 4.2

Score: Strongly Agree=5, Agree=4, Neutral=3, Disagree=2, Strongly Disagree=1 Patient Comments

I learned a lot from you. I am a lot better! The concern and friendly attitude towards me personally [what was best about the visit] Being honest [what was best about the visit]

They were great all the way around. They really made me understand and taught me how to use my medicine which was huge!

I feel everyone taking meds should have to do this. The doctors do not know or do not tell you how to use your medicine.

Primary care provider and pharmacist satisfaction Prescribers, staff, pharmacists, and Blueprint Community Health Teams and staff were invited to participate in baseline and follow-up surveys to evaluate satisfaction with pharmacists as part of the primary care team. Sixty-seven total surveys were returned; 45 individual baseline surveys, 14 individual follow-up surveys, and 8 surveys that included both baseline and follow-up by the same physician. See Tables 11 and 12. A sample of physicians and all of the pharmacists have been interviewed to further understand the strengths and weaknesses of a pharmacist in primary care. Analyses of the interview data are ongoing. Table 11. Survey Respondents Role N (%)

Physician 26 (38.8) Nurse or Medical Assistant 10 (14.9) Physician Resident 9 (13.4) Practice Staff 8 (11.9)

Page 15: Medication Management Pilot Study In Accordance with Act 63

Community Health Team or Blueprint Administration 5 (7.5) Nurse Practitioner 3 (4.5) Pharmacist 3 (4.5) Practice Manager 3 (4.5) Total 67 (100.0)

Table 12. Prescriber Experience Prescriber Experience N (%)

0-5 years 13 (34.2) 6-10 years 3 (7.9) 11-15 years 8 (21.1) 16+ years 14 (36.8) Total 38 (100.0)

Survey participants were encouraged to include comments about their perceptions of working with pharmacists in primary care. Table 13 includes all of the comments received in the follow-up surveys. Prescribers overwhelmingly perceive the pharmacists as having value to their practice and their patients. Table 13. Follow-up Comments

Role Comment

Physician Excellent program that promoted development of relationships with pharmacist, as well as improving patient care

Physician Having a pharmacist in our office was a fantastic experience. The entry of a pharmacist into our clinic co-existed at a time when we also received assistance for our patients from the community health team. From the patient perspective there may have been some overlap in what the pharmacist and CHT nurse could provide (med reconciliation, etc.). I often have multiple issues to discuss with a patient in an office visit and by previewing which of my patients could possibly discontinue or wean their PPIs this made me more likely to address this issue in my office visits. As I was intimately aware of the role of our pharmacist, I knew of her availability but this may not have been the case with other faculty members. I hope we have the opportunity to have a pharmacist again in the near future as patients are best cared for by a team approach. We have many refugees in our clinic and I think a pharmacist could fill a unique role in bridging the cultural divide of educating this vulnerable population on their medication management.

Physician I found the pharmacist to be extremely competent and helpful. However, I found it difficult to incorporate her talents / knowledge into my daily practice. I typically would think of medication questions on the fly, and if this did not happen to be on a Thursday afternoon, I would use UptoDate or a retail pharmacist to answer my question. I had one question having to do with how to find the least expensive drug option for a patient for a particular condition, and I was told by a colleague that this would not be an appropriate reason to consult the pharmacist. This was a surprise to me, in that I would think even relatively straightforward questions would still be a learning opportunity (e.g. perhaps she would have had a good way to look such questions up, etc.). I think a list of good examples of how to utilize her skills would have been helpful. For example consider consults for elderly patients with > 8 or 10 medications to reduce poly-pharmacy. The one consult I was able to come up with was a patient with several meds to be taken at different times of the day, some with and some without meals, and some to avoid using together. The pharmacist was able to construct a regimen. Again, I feel the concept is excellent, I just did not feel I used her presence optimally.

Physician I wish we could meet an hour every week. The educational value is immense.

Physician It seems like even small improvements in medication prescribing yield big dollars in savings, and can justify the expense of having a pharmacist on site.

Physician Medicine is a very dynamic field with changes in principles and protocols happening constantly. One of the biggest areas in medicine to change is pharmacology. Drugs that used to be deemed as "safe" now

Page 16: Medication Management Pilot Study In Accordance with Act 63

have many important drug interactions or dosing limits that providers can be challenged to keep up with. Having a pharmacist as part of our team was a very valuable asset. The ability to have the pharmacist review complicated medication lists or to do outreach to patients identified as high risk based on planned data searches is one of if not the biggest patient safety measures we can apply. I hope that there is a way to support having a pharmacist available as part of our teams going forward.

Physician The pharmacists were valuable people on the team. Feedback from patients was positive and I felt that it improved their understanding of meds and disease. Her in-services for providers were great. Good feedback for me regarding FDA changes.

Physician There is huge potential to use pharmacists to both improve the quality of care and lower costs. We need to do more of this!

Physician Resident

Excellent project and extremely helpful to have an on-site pharmacist for both patient care/safety, provider education. Very informative to be able to discuss medications and problem-shoot with pharmacist while seeing patients.

Physician Resident

It was great to have a pharmacist in clinic. She brought medication interactions and dosing issues to my attention that otherwise would have gone unrecognized. She was also an excellent resource for questions when I was concerned about prescribing a certain medication for fear of an adverse effect or interaction. She was able to quantify risk and discuss other options in a manner that an online resource cannot.

Nurse Practitioner

It is difficult to bring on another player in the EMR on site. If we had a pharmacist available as a resource, especially for the complicated medical patient, that would be perfect

Nurse Practitioner

Very helpful to have the resource for sorting out implications of new interactions and recommendations on management of chronic health issues

Pharmacist Group patient education visits were very successful especially when located at the local physician practice.

Practice Manager

More guidance from the pharmacist community on how they see their roles in a community practice would have been beneficial. There was too much, "Well, how would you use a pharmacist in your primary care practice?"

Nurse or Medical Assistant

We have really enjoyed working with (pharmacist). She has discovered numerous med problems on patients that were recently discharged, reconciling meds post discharge, and thanks to her we have avoided many medication problems. She has worked closely with our residents and patients around medications. She is very knowledgeable and happy to help in any way she can.

Four physicians answered both the baseline and follow-up surveys, allowing for a pre-post comparison. The survey results demonstrate that after working with a pharmacist, there are improvements in all perceptions of pharmacists in primary care, including usefulness, ease of working with a pharmacist, trust, patient outcomes, and pharmacist role in a practice. Table 14. Paired Analyses (Results from 4 physicians) Question Baseline Follow-up Result

I find working with a pharmacist useful in my job. 6.5 6.8 Improved

I find it easy to work with a pharmacist. 6.5 7.0 Improved

I trust the pharmacist to work with my patients or my practice’s patients. 6.8 7.0 Improved

The pharmacist improves the care of my patients or my practice’s patients. 6.5 6.8 Improved

I understand the role of the pharmacist in my practice. 6.3 6.5 Improved*Baseline questions were asked as “I will find…” Scale: Strongly Agree=7, Agree=6, Somewhat Agree=5, Neutral=4, Somewhat Disagree=3, Disagree=2, Strongly Disagree=1 Strengths and weaknesses of various pharmacist models of collaboration between primary care practices and pharmacists

Page 17: Medication Management Pilot Study In Accordance with Act 63

Strengths Diverse pharmacists were successfully integrated into diverse medical home practices. These

results suggest broad generalizability for pharmacist integration. The most successful sites included specialty-trained academic pharmacists and pharmacists who were employed by the same organization as the practice.

Pharmacists successfully identified important drug therapy problems using population-based methods.

Pharmacists demonstrated cost avoidance quickly, within 6-12 months. Prescribers and staff perceived the pharmacists as having value to the practice and the

patients they served. The pharmacist network was important for sharing of best practices.

Weaknesses

Health information technology was a barrier to population activities at times. The selected practices were diverse, but did not represent every Vermont county and were

limited in number. This project did not explore the role of the independent community pharmacist and explored

the role of a chain drug store community pharmacist in a limited scope. The pharmacists did not attempt to bill for their services and therefore the feasibility of fee-for-

service billing is unknown. The part-time (one day per week) nature of the project was difficult for pharmacists. In general,

the pharmacists perceived one day per week to be inefficient and limiting in the services provided.

Understanding the pharmacists’ roles in the practices took about 6 months, even when the prescribers and pharmacists had existing working relationships.

Working across organizations presented additional challenges to integration. Sustainability of the collaborative models Sustainability is highly dependent on a funding mechanism for the pharmacists’ salaries. Currently there is no sustainable mechanism to keep the pharmacists in primary care beyond the scope of what was paid for by the grant. Given the value of the population-based and educational efforts of the pharmacists, two activities not paid for by fee-for-service billing, it is unlikely that fee-for-service billing is the optimal route to a sustainable model for pharmacist integration. Based on the current project, it is highly recommended that any future project or funding mechanism support pharmacists providing direct patient care (including transitions of care), population-based medication management, and patient and prescriber education to primary care practices. Vermont Workforce Estimates Vermont currently has 414 physician FTEs that include Family Medicine, Internal Medicine, and Pediatrics. Currently recommended ratios for pharmacist services in primary care are in the range of 1 pharmacist FTE to 5,000-6,000 patients.[36] Based on the findings of this pilot, Vermont’s practice environment, and the inclusion of population-based approaches would allow 1 pharmacist to manage up to 10,000 patients. Assuming 1 FTE physician has a patient panel of about 2,000 patients, 1 pharmacist FTE can support about 5 physician FTEs. Assuming 414 Vermont physician FTEs, we estimate 83 pharmacist FTEs (414/5) are required to support all of Vermont’s primary care patients. See Table 15. If focusing solely on Family Medicine and Internal Medicine, we estimate 64 pharmacist FTEs (318/5) are required. Table 15. Vermont Physician Workforce

Primary Care Specialty MD/DO FTEs

Page 18: Medication Management Pilot Study In Accordance with Act 63

2012

Family Medicine 202

Internal Medicine 116

Pediatrics 96

Total 414

Vermont Area Health Education Centers (AHEC) Program. The Vermont Primary Care Workforce: 2012 Snapshot. http://www.uvm.edu/medicine/ahec/documents/AHEC_PCREPORT_1_16.pdf Data about how health information technology may be used to support population-based medication management in primary care Efficiently identifying drug therapy problems using population-based approaches requires an electronic health record capable of producing the needed reports, personnel capable of producing reports in a timely manner, and pharmacist full-permission to access and document findings in the electronic health record. Health information technology was a barrier to population-based approaches at times. Our findings suggest that pharmacists’ full permission to access and document findings in the practices’ electronic health records is the most efficient means for documenting and communicating with prescribers (rather than using an outside vendor for documentation). Plan for continued collaboration with broadly represented stakeholders to share best practices and foster integration of pharmacy services into primary care practices and the Vermont Blueprint for Health. See summary of major findings and recommendations.

Page 19: Medication Management Pilot Study In Accordance with Act 63

Summary of Major Findings Pharmacists were successfully integrated into medical home practices and identified important

drug therapy problems. The most successful sites included specialty-trained academic pharmacists and pharmacists who were employed by the same organization as the practice.

Using population-based approaches to identifying drug therapy problems is novel and proved important and feasible.

Efficiently identifying drug therapy problems using population-based approaches requires personnel and an electronic health record capable of producing the needed reports, and pharmacist full-permission to access and document findings in the electronic health record. Health information technology was a barrier to population-based approaches at times.

Our findings suggest that pharmacists’ full permission to access and document findings in the practices’ electronic health records is the most efficient means for documenting and communicating with prescribers (rather than using an outside vendor for documentation).

The potential impact of pharmacists in primary care is large, based on the number of drug therapy problems identified and costs avoided in this demonstration.

Prescribers and staff perceived the pharmacists as having value to the practice and the patients they served.

Working part-time for a practice was a barrier for pharmacists. Prescribers commented they wanted access to a pharmacist as a shared resource in primary care.

Prescribers accepted the recommendations of pharmacists 86% of the time, where data about acceptance is known.

The estimated return on investment was $2 avoided for every $1 spent on a pharmacist. Recommendations

Integrating pharmacists into primary care is feasible, reveals important drug therapy problems that impact Vermonters, and results in overall cost-avoidance. Offering pharmacy services to primary care practices is recommended, including direct patient care (especially during transitions of care), population-based management, and patient and prescriber education.

Sustainability will require the identification of alternative funding mechanisms that do not rely on a fee-for-service approach. This is essential to preserve a pharmacist’s ability to provide population-based management and educational services in addition to direct patient care.

While larger practices may benefit from a full-time pharmacist, smaller practices may be served well with a pharmacist as a shared resource. A ratio of 1 pharmacist full-time equivalent (FTE) to 5 physician FTEs, or 1 pharmacist to 8,000-10,000 patients is recommended.

Pharmacists should ideally be full-time in primary care, even if serving multiple practices. At a minimum, individual pharmacists should be 0.5 FTE.

It is recommended that pharmacists be either specialty trained in primary care (e.g. primary care residency) or employed by the same organization as the practice(s). For pharmacists without specialty training in primary care, prior job experience interacting with physicians as part of an inter-professional team (e.g. participation in hospital rounds) is recommended.

Pharmacists must have full permissions to access and document findings in the practices’ electronic health records. It is unlikely pharmacists can be successful in fully integrating into primary care practices without these permissions.

It is recommended to follow the recently published Guidelines for Pharmacists Integrating into Primary Care Teams, published by the Canadian Pharmacists Journal, November 2013.[1]

The University of Vermont College of Medicine’s Office of Primary Care continues to be in conversation with the Vermont Department of Health regarding these findings and recommendations. References

Page 20: Medication Management Pilot Study In Accordance with Act 63

1. Jorgenson, D., et al., Guidelines for pharmacists integrating into primary care teams. Can Pharm J (Ott), 2013. 146(6): p. 342-52.

2. CDC. National Center for Health Statistics. National Hospital Ambulatory Medical Care Survey: 2010 Outpatient Department Summary Tables. 2010 November 21, 2013].

3. CDC. National Center for Health Statistics. Health, United States. 2012 November 21, 2013]. 4. Qato, D.M., et al., Use of prescription and over-the-counter medications and dietary

supplements among older adults in the United States. Jama, 2008. 300(24): p. 2867-78. 5. Viktil, K.K., et al., Polypharmacy as commonly defined is an indicator of limited value in the

assessment of drug-related problems. Br J Clin Pharmacol, 2007. 63(2): p. 187-95. 6. Beney, J., L.A. Bero, and C. Bond, Expanding the roles of outpatient pharmacists: effects on

health services utilisation, costs, and patient outcomes. Cochrane Database Syst Rev, 2000(3): p. CD000336.

7. Nkansah, N., et al., Effect of outpatient pharmacists' non-dispensing roles on patient outcomes and prescribing patterns. Cochrane Database Syst Rev, 2010(7): p. CD000336.

8. Margolis, K.L., et al., Effect of home blood pressure telemonitoring and pharmacist management on blood pressure control: a cluster randomized clinical trial. Jama, 2013. 310(1): p. 46-56.

9. Weber, C.A., et al., Pharmacist-physician comanagement of hypertension and reduction in 24-hour ambulatory blood pressures. Arch Intern Med, 2010. 170(18): p. 1634-9.

10. Carter, B.L., et al., The potency of team-based care interventions for hypertension: a meta-analysis. Arch Intern Med, 2009. 169(19): p. 1748-55.

11. Carter, B.L., et al., Physician and pharmacist collaboration to improve blood pressure control. Arch Intern Med, 2009. 169(21): p. 1996-2002.

12. Hunt, J.S., et al., A randomized controlled trial of team-based care: impact of physician-pharmacist collaboration on uncontrolled hypertension. J Gen Intern Med, 2008. 23(12): p. 1966-72.

13. Santschi, V., et al., Impact of pharmacist care in the management of cardiovascular disease risk factors: a systematic review and meta-analysis of randomized trials. Arch Intern Med, 2011. 171(16): p. 1441-53.

14. Santschi, V., et al., Pharmacist interventions to improve cardiovascular disease risk factors in diabetes: a systematic review and meta-analysis of randomized controlled trials. Diabetes Care, 2012. 35(12): p. 2706-17.

15. Simpson, S.H., et al., Effect of adding pharmacists to primary care teams on blood pressure control in patients with type 2 diabetes: a randomized controlled trial. Diabetes Care, 2011. 34(1): p. 20-6.

16. Pape, G.A., et al., Team-based care approach to cholesterol management in diabetes mellitus: two-year cluster randomized controlled trial. Arch Intern Med, 2011. 171(16): p. 1480-6.

17. Choe, H.M., et al., Proactive case management of high-risk patients with type 2 diabetes mellitus by a clinical pharmacist: a randomized controlled trial. Am J Manag Care, 2005. 11(4): p. 253-60.

18. Jameson, J.P. and P.J. Baty, Pharmacist collaborative management of poorly controlled diabetes mellitus: a randomized controlled trial. Am J Manag Care, 2010. 16(4): p. 250-5.

19. Dent, L.A., K.J. Harris, and C.W. Noonan, Randomized trial assessing the effectiveness of a pharmacist-delivered program for smoking cessation. Ann Pharmacother, 2009. 43(2): p. 194-201.

20. Murray, M.D., et al., Effect of a pharmacist on adverse drug events and medication errors in outpatients with cardiovascular disease. Arch Intern Med, 2009. 169(8): p. 757-63.

21. Raebel, M.A., et al., Improving laboratory monitoring at initiation of drug therapy in ambulatory care: a randomized trial. Arch Intern Med, 2005. 165(20): p. 2395-401.

22. Bhardwaja, B., et al., Improving prescribing safety in patients with renal insufficiency in the ambulatory setting: the Drug Renal Alert Pharmacy (DRAP) program. Pharmacotherapy, 2011. 31(4): p. 346-56.

Page 21: Medication Management Pilot Study In Accordance with Act 63

23. Jameson, J., G. VanNoord, and K. Vanderwoud, The impact of a pharmacotherapy consultation on the cost and outcome of medical therapy. J Fam Pract, 1995. 41(5): p. 469-72.

24. Hanlon, J.T., et al., A randomized, controlled trial of a clinical pharmacist intervention to improve inappropriate prescribing in elderly outpatients with polypharmacy. Am J Med, 1996. 100(4): p. 428-37.

25. Lee, J.K., K.A. Grace, and A.J. Taylor, Effect of a pharmacy care program on medication adherence and persistence, blood pressure, and low-density lipoprotein cholesterol: a randomized controlled trial. JAMA, 2006. 296(21): p. 2563-71.

26. Strand, L.M., et al., The impact of pharmaceutical care practice on the practitioner and the patient in the ambulatory practice setting: twenty-five years of experience. Curr Pharm Des, 2004. 10(31): p. 3987-4001.

27. Smith, M., D.W. Bates, and T.S. Bodenheimer, Pharmacists belong in accountable care organizations and integrated care teams. Health Aff (Millwood), 2013. 32(11): p. 1963-70.

28. Collaborative, P.-C.P.C. Integrating Comprehensive Medication Management to Optimize Patient Outcomes. 2012 November 21, 2013]; Available from: http://www.pcpcc.org/sites/default/files/media/medmanagement.pdf.

29. Giberson S, Y.S., Lee MP., Improving Patient and Health System Outcomes through Advanced Pharmacy Practice. A Report to the U.S. Surgeon General.2011: U.S. Public Health Service. Office of the Chief Pharmacist.

30. Smith, M., M.R. Giuliano, and M.P. Starkowski, In Connecticut: improving patient medication management in primary care. Health Aff (Millwood), 2011. 30(4): p. 646-54.

31. Kozminski, M., et al., Pharmacist integration into the medical home: qualitative analysis. J Am Pharm Assoc (2003), 2011. 51(2): p. 173-83.

32. O'Brien, M.A., et al., Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2007(4): p. CD000409.

33. Avorn, J. and S.B. Soumerai, Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based "detailing". N Engl J Med, 1983. 308(24): p. 1457-63.

34. Soumerai, S.B. and J. Avorn, Principles of educational outreach ('academic detailing') to improve clinical decision making. Jama, 1990. 263(4): p. 549-56.

35. Devine, E.B., et al., Strategies to optimize medication use in the physician group practice: the role of the clinical pharmacist. J Am Pharm Assoc (2003), 2009. 49(2): p. 181-91.

36. Nigro, S.C., et al., Clinical Pharmacists as Key Members of the Patient-Centered Medical Home: An Opinion Statement of the Ambulatory Care Practice and Research Network of the American College of Clinical Pharmacy. Pharmacotherapy, 2013.

37. Lee, A.J., et al., Clinical and economic outcomes of pharmacist recommendations in a Veterans Affairs medical center. Am J Health Syst Pharm, 2002. 59(21): p. 2070-7.

38. Larson, L.N., J.P. Rovers, and L.D. MacKeigan, Patient satisfaction with pharmaceutical care: update of a validated instrument. J Am Pharm Assoc (Wash), 2002. 42(1): p. 44-50.

39. Moczygemba, L.R., et al., Patient satisfaction with a pharmacist-provided telephone medication therapy management program. Res Social Adm Pharm, 2010. 6(2): p. 143-54.

40. Shimp, L.A., et al., Employer-based patient-centered medication therapy management program: evidence and recommendations for future programs. J Am Pharm Assoc (2003), 2012. 52(6): p. 768-76.