Embed Size (px)
Citation preview
© 2011 Commonwealth Corporation 1
Review of Pilot 1 (2011) Assessment
Navjeet Singh, Vice President of Applied Research & [email protected]
Lawren Bercaw, Research and Evaluation [email protected]
February 28, 2012
Commonwealth Corporation Evaluation of PHCAST
© 2011 Commonwealth Corporation 2
Presentation
• Purpose & Overview• The Grant• Evaluation Design• Year 1 Pilot Review
© 2011 Commonwealth Corporation 3
Purpose and Overview
To provide an overview of the evaluation required and planned for the grantTo review results from evaluation activities for the first year pilot
© 2011 Commonwealth Corporation 4
Purpose of evaluation
• Purpose of the evaluation is to ensure that for the “Core” Curriculum and Curriculum Design is consistent between the PCA and PCHM training and lead to better outcomes:– skills and knowledge gain for PCAs, PCHMs; – improved demonstration of skills on the job by PCAs
and PCHMs, and – improved outcomes for clients and consumers.
© 2011 Commonwealth Corporation 5
What is this grant about?
MA Council - Home Care
Services curriculum
BCC/BEC PCA
curriculum
PCAs-no formal
training
Personal Care HomeMaker (PCHM)
Personal Care Attendant (PCA)
Existing Training PHCAST Grant
Direct care worker competencies and PHCAST core curriculum developed
Integrated and Transferrable PHCAST Training
MA Council PHCM curriculum
BCC PCA Curriculum
© 2011 Commonwealth Corporation 6
What is the evaluation design?
• Is the PHCAST curriculum transferrable to different worker types(PCHM or PCA)? Do PCHM and PCA demonstrate positive learning outcomes? – PCAs and PCHMs understand differences in their roles? – PCAs and PCHMs demonstrate positive learning outcomes? – Do employers report greater worker satisfaction and retention?
• Concerns about evaluation results –various other factors at play:– PHCAST instructor and delivery of training– Context and environment in which training is delivered– Varying needs of independent consumers and agency based
clients
© 2011 Commonwealth Corporation 7
Formative Evaluation
• To provide feedback on the process as well as the curriculum:– From students who attended the pilot Year1
• Student reaction survey after they completed the training
– From PHCAST instructors using the curriculum• Instructor feedback• Instructor focus group
© 2011 Commonwealth Corporation 8
Student Assessment Results
DescriptionDescription Pre-Training Pre-Training AssessmentAssessment
Post-Post-Training Training
AssessmentAssessment
Students (No.) 197 188
Written Questions 67 67
Questions Answered Correctly 52 59
Change in Score
10.2% Improvement in Score Between Pre- and
Post-Test
Skills Assessment•Students had to demonstrate selected skills (e.g., hand-washing, lifting, transferring)•Most students (80%) were described as skilled in most areas and were not classified as “needing review” in any skill
Written Assessment
© 2011 Commonwealth Corporation 9
Key Issues Identified
• Are there questions that those with certain characteristics were more likely to answer incorrectly?
Characteristics we checked for:– Home Care Council classes compared to Community
College classes– Limited English ability– Education: with less than high School compared to
those with HS diploma or more education– Previous experience in health care
© 2011 Commonwealth Corporation 10
Results of Formative Evaluation
• Students reported high levels of satisfaction with PHCAST training and strong interest in working as PCAs/PCHMs
• Instructors also indicated high satisfaction overall with all materials, especially– More information and material resources– More hands-on and interactive activities
• Potential areas for improvement suggested by instructors:– Need more time or less material for most modules– More time for practicing skills– “Infection Control” had too much material, and at higher
language level– Need more visual aids, graphics, perhaps videos
