Florida/Caribbean AETC Project ECHO™ Program: Co-management of HIV/AIDS by Community Health

Care Providers and Specialists

Joanne J. Orrick, PharmD, AAHIVEAssociate Director

Florida/Caribbean AETCUniversity of South Florida Center for HIV Education and Research

Jeffrey Beal, MD, AAHIVS

Clinical Director & Principal Investigator

Kimberly Molnar, MAcc

Director, F/C AETC

Saniyyah Mahmoudi, ARNP

Faculty Facilitator

Joanne Orrick, PharmD, AAHIVE

Faculty Facilitator

Todd Wills, MD

Faculty Facilitator

Hesborn Wao, PhD

Evaluator

Sean McIntosh, AS

Program Coordinator

Adis Kreso, BS

Statistical Data Analyst

Danchelle Jones

Program Support

Maximo Lora, BA

Marketing & Technical Support

Christine Makar, BA

Continuing Education Support

Nathan McMullen

Technical Support

Theresa C. Skipper, MPH

Logistical Management

F/C AETC-Project ECHO™ Team

Outline

Implementation Evaluation Preliminary results Challenges Future Directions Question/Answer

Implementation

• F/C AETC site visit to University of New Mexico Project ECHO™

• Weekly implementation meetings with F/C AETC program staff

• Technical assistance from University of New Mexico Project ECHO™

• Polycom vs. Adobe Connect platform• Provider interest gauged at Annual F/C

ATEC conference and through email contact

Session Overview

• Sessions scheduled twice per month • 15 minute didactic presentation by a F/C

AETC faculty member• Remainder of time (1.5 hours scheduled

per session) spent discussing patient cases submitted by providers throughout the region

• F/C AETC faculty member facilitates discussion among participants and other F/C AETC faculty

Case Presentation FormInteractive Fillable PDF

Evaluation of the Program

Purpose of the evaluationDetermine extent to which F/C AETC Project ECHO™ helps in developing capacity among health care professionals in rural and underserved areas of Florida, Puerto Rico, and the U.S. Virgin Islands to safely and effectively treat HIV/AIDS and to monitor outcomes of this treatmentLearn from our implementation experience so that we can better manage performance and improve program design

Program logic model

Evaluation of the Program

Type of evaluation Needs evaluation: Determine care provider needs Process evaluation: Assess program activities/processes Outcome evaluation: Assess program effects/impact

Time points Participant registration: Prior to scheduled session Immediate post-training: ≤1 week (All participants) Follow-up evaluation: 4-6 months (Case presenters)

Methodology: Mixed-methods approach Quantitative: Survey (numeric items) Qualitative: Survey (open-ended items)

Program Logic Model

Evaluation Data Collection

Immediate post-trainingDemographicsObjectives for attendingKirkpatrick's levels:

Satisfaction/reaction Intention to change Knowledge,

Skills, & Attitudes (KSA) Learning (change in KSA) KSA transfer to workplace Improved patient outcome

Suggestions for improvement

Follow-up evaluationPatient benefited?Change in KSAKSA learned used to manage other patients?KSA learned shared with others at workplace?Challenges faced while implementing suggestions

Kirkpatrick, D. L. (1994). Evaluating training programs. San Francisco: Berrett-Koehler Publishers, Inc.

Evaluation of the Program

AimsDetermine HCP’s satisfaction with the programExamine change in HCPs’ KSA pre and post-sessionDetermine extent KSA are transferable to workplaceDetermine degree program impacts patient outcomes

ProcedureRetrieve data from NEC system (Excel) and cleanEnsure data integrity: Evaluation/data management team separate from program implementation teamPresent results using graphs, charts and text

Data analysis

Preliminary Results

Participants (6 ECHO sessions: 11/2011 - 02/2012)

Preliminary Results

Preliminary Results

Increase knowledge (41% participants)“Broaden my knowledge about HIV resistance testing”“Discuss ideas for increasing medication compliance” “Increase my knowledge on perinatal HIV”“Learn from every case I hear about”

Update knowledge (20% participants)“Stay updated and learn from others' experiences”“Obtain newest information in HIV care“Educate & update on providing holistic care to HIV pts”

Examples of objectives by category

Preliminary Results

Observe (13% participants)“Observe Perinatal Project ECHO™, learn about new perinatal guidelines”“Observe the new ECHO ™ session offered by F/C AETC”“Observe the TeleECHO clinic”

Network (8% participants) “Network with other HIV providers”“Hear what others perinatal providers around the state are doing”

Examples of objectives by category (cont.)

Preliminary Results

Level 1: Participants’ satisfaction/reaction

Preliminary Results

Level 1: Participants’ satisfaction/reaction cont

Preliminary Results

Level 2a: Learning (Change in Knowledge)

Preliminary Results

Level 2b: Intention to change in practice

Following the F/C AETC-Project ECHO™ session, I intend to make the following changes:“…consider role-play for pts struggling with disclosure”“Speak more directly with pts about missed doses…”“Provide local providers with updated guidelines for administration of AZT”“Change regimen on the patient that was discussed”“Network with persons with similar duties”

Preliminary Results

Level 3: Transfer of KSA to workplace

Preliminary Results

Level 3: Transfer of KSA to workplace

Preliminary Results

What participants liked MOST:

“Ability to participate without travel” “Being able to interface with a panel of experts…” “Expert discussion, 2) Enough data on the patient to

be able to follow the discussion, 3) Collegial nature of the discussion”

“Hearing from experts, discussion with logic behind it, the different opinions”

“The format: Didactic and case presentations”

Preliminary Results

What participants liked LEAST:

“It would be nice to see all the people on the call” “Would like more case presentations each session” “Early on - the technical difficulties” “Not enough time to ask questions about our own

case” “Could not see all participants” “The difficulty in getting the webcam to operate” “Technical difficulties”

Preliminary Results

How F/C AETC-Project ECHO™ can be improved

“Have a constant banner/header with how to mute/unmute phone lines”

“Include more non-clinical/treatment educational resources and alternative ways non-clinical/ treatment health care providers can implement the material and utilize within the community & patients.”

“Have more information presented on cases” “Decrease background noise, presentation of more

than one case study”

Resolution of Challenges

• Troubleshoot technical difficulties• Check list for each program• Increase participants who participate with

webcams• Engage more participants in the discussion• Increase the number of F/C AETC faculty

who serve as facilitators• Increase the number of participating clinics

and providers

Future Directions

• Curriculum outlined through June 2012• Includes Pediatric/Adolescent specialty sessions

• Develop curriculum and brochure detailing all programs for July 2012-June 2013 AETC grant year

• Expand to include other sub-specialty sessions (e.g. Hepatitis and HIV co-infection)

• Collaborate and share resources with other AETCs using the Project ECHO™ model