Demonstrating!Successful Outcomes! - Infant Hearing · 2015. 3. 26. · ((( infanthearing.org StatisticalAnalysis!2015! PreTest!Knowledge!Scores!by!Who!Provided!Training Pre-Test

((( infanthearing.org

Demonstrating Successful Outcomes

Randi Winston Gerson, AuD, CCC-‐A

The National Center for Hearing Assessment and Management at Utah State University (NCHAM)

The Newborn Hearing Screening Curriculum


Learning outcomes

•  Discuss the benefits of incorporating the NHSTC as a standardized method of training

•  Demonstrate the various ways that state EHDI programs, hospitals and stakeholders, can utilize the course and it’s resources to improve the standard of care for newborn hearing screening and how participants can obtain an NCHAM Certificate of Completion and CEUs

•  Share program evaluation tools and learner outcomes


NHSTC Objective To improve the standard of care to babies and families for screening and follow-‐up, through efficient practices resulting from quality, comprehensive, standardized training


Special Thanks To Will Christiansen, NCHAM webmaster, for his dedication and continued work in developing a variety of formats to facilitate online learning, data collection. for addressing ongoing technical assistance needs and to addressing individual states participant requests


Special Thanks To Matt Dante and Cesar Galdamez Villa from USU for conducting the data analysis



Benefits to screeners

• Defines and reinforces NHS as a comprehensive program, comprised of many critical components

• Instills knowledge which is powerful for self confidence


Benefits of State EHDIs Institutionalizing the NHSTC for Screener Training

•  Provide a consistent training solution for hospital and out-‐of -‐hospital screeners –  Quality Assurance –  Competency based training tools –  Consistency in knowledge for all screeners

•  Standardizes training procedures –  Unconventional methods can lack standardization and continuity –  Information gets lost in transition between trainers –  Dilution of training quality over time

•  Provides an employee tracking and documentation mechanism for risk management


http://www.infanthearing.org/


http://www.infanthearing.org/


Overview of Webpage and Other Features

www.infanthearing.org

• Moodle course –  An elearning management system –  Registration required –  Allows for competency based learning and tracking of

outcomes • pre and post tests • pre and post self assessment survey • CEUs • NCHAM Certificate of Completion


Certificate of Completion and CEUs

– NCHAM Certificate of Completion upon obtaining a score of 80% or better on the Moodle course postest

– .2 CEU Credits • American Academy of Audiology (AAA) • National Association of Pediatric Nurse Practioners (NAPNAP)



www.infanthearing.org • HTML Course – (20,488 unique page views in 2015)

– Simple viewing – No registration – No tracking of outcomes – No certificate or CEUs



•  SCORM Package –  A zip file containing all of the course files –  Can be easily sent to a facility that has their own internal LMS –  Pre and post tests and assessment tools can be incorporated,

but must submit to NCHAM for Certificate of Completion –  Requires facility/institution to score tests and run their own data

•  DVD –  Available in web version –  Doesn’t require internet connection but does require Flash and

audio to run video clips –  Won’t be able to activate internet links to websites in the resources

section –  Unable to track and collect outcome data


Stats collected from 2012-‐2013

• HTML Course: @ 4000 unique page views • Moodle Course: 612 enrolled users

–  Average quiz scores • Pretest: 84.07% • Postest: 94%

•  SCORM: 5 sites –  2 states; Indiana, AZ –  2 hospitals; AZ, WA

• DVD: 8 sites (FL, AL, KS, CO, UT, TN, IN)


Data Analysis 2015 N=1200 Measures: 1.  Did participants show an improvement in

pre-‐test versus post-‐test knowledge? 2.  Did participants feel increased confidence

as a result of taking the NHSTC? 3.  What is the frequency of use by state? 4.  Summary of comments/feedback


Number of People that Participated in NHSTC Moodle Course from January 1st, 2013 to January 1st, 2015

0 50 100 150 200 250 300 350 400

January - June 2013

July – December 2013

January - June 2014

July – December 2014

Number of People that Participated in NHSTC from January 1st, 2013 to January 1st, 2015


Data Analysis 2015 N=1200 Covariates for Consideration: 1.  What job position do they hold? 2.  How were they trained (by whom) prior to

taking the NHSTC? 3.  Did their score on the pretest vary based

on who conduct the training?


Data Analysis 2015 Measures

• Self Assessment Pre-‐Training • Self Assessment Post-‐Training • NHSTC Pretest • NHSTC Postest


Measures 1 Self-‐ Assessment Pre-‐training Tool

Rate the extent to which you agree or disagree with the statements using the following scale: 1 = Strongly Disagree 2 = Disagree 3 = Not sure 4 = Agree 5 = Strongly Agree •  I am confident in my ability to: •  Educate parents about the importance of newborn hearing screening •  Conduct an OAE screening •  Perform OAE troubleshooting •  Conduct an A-‐ABR screening •  Conduct A-‐ABR troubleshooting •  Conduct screening on NICU Babies and babies with risk factors •  Tell parents the results when the baby passes the screening •  Tell the parents the results when the baby fails the screening •  Connect parents to outpatient testing when needed


Statistical Analysis 2015 Self-‐ Assessment Post-‐Training Tool Rate the extent to which you agree or disagree with the statements using the following scale: 1 = Strongly Disagree 2 = Disagree 3 = Not sure 4 = Agree 5 = Strongly Agree This training section provided me with useful information that increased my confidence in the following areas of newborn hearing screening:

•  The Benefits of Newborn Hearing Screening •  Getting Ready to Screen •  Screening with Otoacoustic Emissions (OAEs) •  Screening with Automated Auditory Brainstem Response (A-‐ABR) •  Communicating with Parents and the Medical Providers •  Completing the Screening Process •  Screening Babies in the NICU and with Risk Indicators •  Outpatient Screening: Rescreening

Comments: Please feel free to let us know about anything you particularly liked or did not like about the training


Data Analysis 2015 – Job Position

Position N

ADMINISTRATIVE SUPPORT 20

AUDIOLOGY/NHS TECHNICIAN 446

CERTIFIED NURSES ASSISTANT 62

NURSE 519

TECHNICIAN IN AN UNRELATED AREA (E.G., RESPIRATORY THERAPY) 54

VOLUNTEER 32 !


Pre and Post Self Assessment Confidence of Knowledge


Differences in Confidence Level by Job


Overall Pre and Post Self-Assessment Comparison by Percent

Mean N Std. Deviation

Std. Error

Mean

Pair 1 Pre Self-Assess % 75.9819 1006 20.56651 .64843

Post Self-Assess % 92.5696 1006 13.21117 .41653 !


Table 1 Question)#)

Pre-Test)Mean) Post-Test)Mean)

Total) 83.05) 93.79)Question)1) 3.41) 4.40)Question)2) 3.78) 4.87)Question)3) 3.74) 4.85)Question)4) 4.56) 4.94)Question)5) 3.86) 4.63)Question)6) 4.96) 5.00)Question)7) 4.67) 4.89)Question)8) 4.95) 4.99)Question)9) 1.25) 2.49)Question)10) 2.92) 4.52)Question)11) 4.75) 4.77)Question)12) 4.15) 4.67)Question)13) 4.09) 4.74)Question)14) 4.72) 4.95)Question)15) 4.68) 4.85)Question)16) 4.95) 4.99)Question)17) 4.64) 4.94)Question)18) 4.66) 4.88)Question)19) 4.72) 4.95)Question)20) 3.64) 4.54))

Pre and Post Test Comparison


Differences on pretest scores by between individual who had previous experience and training

Previous Experience Mean Std. Deviation N

No 79.9437 10.80327 222

Yes 84.2599 9.63594 733

Total 83.2083 10.09952 960 !

Previous Training Mean Std. Deviation N

No 79.9514 11.16166 257

Yes 84.4044 9.41751 701

Total 83.2083 10.09952 960 !


Statistical Analysis 2015 Pre-‐Test Knowledge Scores by Who Provided Training

Pre-Test Knowledge Scores by Who Provided the Training Who Provided Training Mean Std. Deviation N

Blank--unknown 79.9710 11.14015 259

Audiologist 88.1604 8.62062 53

Co-worker 83.6245 9.60356 249

Equipment manufacturer 84.2917 8.74657 60

Experienced screener/trainer 84.5288 9.19635 313

Other (specify) 82.9808 11.81142 26

Total 83.2083 10.09952 960 !


Post-Test Mean Scores by Job Position Position Mean SD N

NURSE 94.0988 5.76329 516

AUDIOLOGY/NHS TECHNICIAN 94.3525 6.24601 444

CERTIFIED NURSES ASSISTANT 90.9677 6.98879 62

TECHNICIAN IN AN UNRELATED AREA (E.G., RESPIRATORY THERAPY) 91.1792 6.51558 53

VOLUNTEER 94.2969 5.79329 32

ADMINISTRATIVE SUPPORT 89.7500 8.76971 20

Total 93.8024 6.29256 1127 !

Statistical Analysis 2015 Post Test knowledge Scores by Job Position


Pretest Conclusions •  There was a significant difference between individuals on their

pre-‐test scores •  There was not a significant difference on performance between

job positions on pre-‐test scores •  There was a statistically significant difference on pretest scores

between individuals who had previous experience with screening and those that did not have previous experience

•  There was statistically significant difference on pre-‐test scores between individuals who had previous training versus those that did not have previous training

•  Both of these findings support that there is a needed for training like the NHSTC since not all people have access to good training and obviously not everyone has had previous experience


Post Test Conclusions • On average, people scored statistically significantly

higher on the post test than on the pretest •  There was not a significant difference between

individuals based on whether they had previous experience with screening

•  There was not a significant difference between individuals based on whether they had received previous training

•  The NHSTC training has an improving effect on the post test scores, thus the effects of previous training differences disappear with the training


Statistical Analysis 2015

Summary Confidence and Knowledge • Overall the NHSTC increases individual confidence

and knowledge •  Some people know know more about performing

the screenings prior to taking the training course, and some feel more confident than others, but in general, the majority of people are fairly similar in knowledge and confidence after finishing the training.


Number of Participant’s per State



Questions?

[email protected]

Documents

Demonstrating!Successful Outcomes! - Infant Hearing · 2015. 3. 26. · ((( infanthearing.org StatisticalAnalysis!2015! PreTest!Knowledge!Scores!by!Who!Provided!Training Pre-Test